You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-29 - 16:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack5slot3s.osadl.org (updated Sun Mar 29, 2026 12:44:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
179532235164,22sleep10-21swapper/107:07:101
179172231161,58sleep00-21swapper/007:06:400
178582201164,25sleep30-21swapper/307:05:543
179342200164,24sleep20-21swapper/207:06:542
2172421790,3sleep31828099cyclictest10:55:183
95821520,5sleep31828099cyclictest08:55:153
18280991190,2cyclictest0-21swapper/312:02:313
18280991101,107cyclictest0-21swapper/311:24:353
18280991091,2cyclictest0-21swapper/309:51:563
18280999894,3cyclictest0-21swapper/307:50:203
18280999884,2cyclictest0-21swapper/308:02:373
18280999390,2cyclictest0-21swapper/312:32:513
98442590,2sleep00-21swapper/009:15:140
18280995853,4cyclictest0-21swapper/312:25:213
18280995754,2cyclictest0-21swapper/308:43:533
63852560,4sleep11826899cyclictest11:35:131
108472520,2sleep230-21rcuc/209:15:232
18280994743,3cyclictest0-21swapper/309:26:053
18280994743,3cyclictest0-21swapper/309:26:053
28282450,4sleep21827399cyclictest09:00:102
258932430,1sleep30-21swapper/312:20:023
250602420,7sleep341-21ksoftirqd/308:35:203
1826399379,13cyclictest6011-21munin-run11:35:010
189112360,2sleep00-21swapper/008:20:230
1827399331,23cyclictest0-21swapper/207:22:242
1826399331,11cyclictest6971-21diskmemload10:44:410
1826399320,24cyclictest0-21swapper/010:24:000
1826399320,10cyclictest6971-21diskmemload09:47:410
1826399311,8cyclictest14235-21ntp_states08:10:210
1826399311,8cyclictest14235-21ntp_states08:10:210
1826399310,23cyclictest0-21swapper/008:27:400
1828099301,18cyclictest3415-21kworker/3:107:38:493
1826399301,22cyclictest0-21swapper/010:25:080
1826399300,23cyclictest0-21swapper/009:35:410
1826399300,12cyclictest0-21swapper/007:37:370
1827399291,12cyclictest19531-21ntp_states07:10:212
1827399291,11cyclictest0-21swapper/207:33:042
1827399291,10cyclictest0-21swapper/208:56:082
1827399290,11cyclictest0-21swapper/210:59:452
1826399291,13cyclictest27999-21munin-run12:25:010
1826399291,13cyclictest24589-21if_err_enp1s012:15:160
1826399291,13cyclictest0-21swapper/009:11:170
1826399291,11cyclictest4227-21irqstats10:15:170
1826399291,11cyclictest0-21swapper/008:39:280
1826399291,11cyclictest0-21swapper/007:22:290
1826399290,13cyclictest0-21swapper/007:41:130
149462290,3sleep339-21rcuc/311:55:013
149462290,3sleep339-21rcuc/311:55:013
1826399281,12cyclictest0-21swapper/011:12:130
1826399281,11cyclictest0-21swapper/008:43:490
1826399281,10cyclictest0-21swapper/007:46:130
1826399281,10cyclictest0-21swapper/007:16:320
1826399280,20cyclictest0-21swapper/010:38:120
1826399280,20cyclictest0-21swapper/010:38:120
1826399280,11cyclictest0-21swapper/011:37:130
1827399271,9cyclictest0-21swapper/208:12:372
1827399271,9cyclictest0-21swapper/208:12:372
1827399271,12cyclictest0-21swapper/211:32:562
1827399271,12cyclictest0-21swapper/208:17:032
1827399271,10cyclictest0-21swapper/212:22:132
1826399271,9cyclictest0-21swapper/008:58:370
1826399271,13cyclictest0-21swapper/012:25:200
1826399271,11cyclictest0-21swapper/011:47:570
1826399271,10cyclictest0-21swapper/012:36:320
1826399270,12cyclictest0-21swapper/009:26:130
1826399270,12cyclictest0-21swapper/009:26:130
1826399270,11cyclictest0-21swapper/011:59:010
1826399270,11cyclictest0-21swapper/011:03:130
1828099262,9cyclictest3415-21kworker/3:110:03:443
1828099262,7cyclictest0-21swapper/310:22:503
1828099261,9cyclictest3415-21kworker/3:108:05:563
1827399261,9cyclictest0-21swapper/208:06:532
1827399261,12cyclictest0-21swapper/210:54:292
1827399261,12cyclictest0-21swapper/209:56:492
1827399261,12cyclictest0-21swapper/208:54:472
1827399261,11cyclictest0-21swapper/208:39:532
1827399260,17cyclictest0-21swapper/207:16:172
1827399260,12cyclictest0-21swapper/211:49:252
1826399268,3cyclictest0-21swapper/007:56:370
1826399264,2cyclictest0-21swapper/007:52:370
1826399261,9cyclictest0-21swapper/011:43:290
1826399261,18cyclictest0-21swapper/011:07:170
1826399261,10cyclictest0-21swapper/009:50:210
1826399261,10cyclictest0-21swapper/008:30:400
53402251,20sleep35355-21entropy09:05:153
1828099259,2cyclictest86550irq/126-enp1s0-10:15:003
1828099257,12cyclictest86750irq/128-enp1s0-11:32:243
1827399251,12cyclictest0-21swapper/210:23:252
1827399251,10cyclictest0-21swapper/210:47:172
1827399250,11cyclictest0-21swapper/209:37:572
1826899258,14cyclictest1580-21systemd-journal11:24:531
1826399251,16cyclictest0-21swapper/010:46:130
1826399251,11cyclictest0-21swapper/011:18:290
1826399251,11cyclictest0-21swapper/007:12:450
1826399251,10cyclictest0-21swapper/011:27:250
1826399251,10cyclictest0-21swapper/010:08:210
1826399251,10cyclictest0-21swapper/008:46:170
1826399251,10cyclictest0-21swapper/008:17:280
1826399251,10cyclictest0-21swapper/008:00:450
1826399250,17cyclictest0-21swapper/011:52:250
1826399250,17cyclictest0-21swapper/011:52:250
1826399250,12cyclictest297772sleep007:33:210
1828099248,10cyclictest86650irq/127-enp1s0-11:28:133
1828099244,15cyclictest86750irq/128-enp1s0-11:02:533
1828099242,9cyclictest3415-21kworker/3:110:53:113
1828099241,11cyclictest3415-21kworker/3:109:56:163
18280992410,2cyclictest3415-21kworker/3:109:22:133
1827399241,9cyclictest0-21swapper/207:53:082
1827399241,11cyclictest0-21swapper/210:35:092
1827399241,11cyclictest0-21swapper/210:35:092
1827399241,10cyclictest0-21swapper/212:19:452
1827399241,10cyclictest0-21swapper/210:28:292
1827399241,10cyclictest0-21swapper/207:42:342
1827399241,10cyclictest0-21swapper/207:29:162
1827399240,9cyclictest0-21swapper/209:10:442
1826899243,16cyclictest25242-40dnf07:21:371
1826899241,20cyclictest26576-21perl12:20:151
1826899241,18cyclictest0-21swapper/112:07:071
1826399247,10cyclictest0-21swapper/008:53:090
1826399241,9cyclictest0-21swapper/009:22:250
1826399241,13cyclictest0-21swapper/010:55:330
1826399240,10cyclictest0-21swapper/012:02:570
1828099239,7cyclictest3415-21kworker/3:108:27:533
1828099237,12cyclictest86750irq/128-enp1s0-10:36:173
1828099237,12cyclictest86750irq/128-enp1s0-10:36:173
1828099235,7cyclictest3415-21kworker/3:107:20:133
1828099232,8cyclictest3415-21kworker/3:110:17:123
1828099232,8cyclictest3415-21kworker/3:109:49:523
1828099232,6cyclictest3415-21kworker/3:108:24:493
1828099232,10cyclictest3415-21kworker/3:109:18:113
1828099231,7cyclictest3415-21kworker/3:110:27:133
1827399237,8cyclictest0-21swapper/212:00:232
1827399237,3cyclictest0-21swapper/208:45:112
1827399237,3cyclictest0-21swapper/208:27:092
1827399234,6cyclictest0-21swapper/211:40:352
1827399231,9cyclictest0-21swapper/212:30:332
1827399231,9cyclictest0-21swapper/210:11:012
18273992316,4cyclictest6925-21entropy07:55:132
1827399231,13cyclictest0-21swapper/207:39:162
1827399231,10cyclictest0-21swapper/212:35:532
1827399231,10cyclictest0-21swapper/210:42:352
1827399231,10cyclictest0-21swapper/210:07:412
1827399231,10cyclictest0-21swapper/209:41:332
1827399231,10cyclictest0-21swapper/209:08:282
1827399231,10cyclictest0-21swapper/208:34:412
1827399230,5cyclictest24212-21kworker/u8:109:50:252
1826899231,4cyclictest31482-21cpuspeed_turbos10:05:131
1826899231,4cyclictest17911-21df_inode08:20:141
1826899231,3cyclictest1384-21awk12:35:221
1826899231,19cyclictest4727-21diskstats07:50:131
1826899231,16cyclictest0-21swapper/109:02:091
1826899230,16cyclictest3160-21kworker/u8:110:17:141
1826399239,3cyclictest0-21swapper/009:00:490
1826399239,3cyclictest0-21swapper/007:28:090
1826399233,3cyclictest0-21swapper/009:55:450
1826399231,11cyclictest0-21swapper/010:04:160
1826399230,11cyclictest6971-21diskmemload12:06:010
1826399230,11cyclictest0-21swapper/010:34:360
1826399230,11cyclictest0-21swapper/010:13:490
1828099227,6cyclictest1421-21kworker/3:012:23:173
1828099226,11cyclictest86750irq/128-enp1s0-09:40:003
1828099225,13cyclictest86750irq/128-enp1s0-11:07:053
1828099223,14cyclictest86650irq/127-enp1s0-09:40:123
1828099222,8cyclictest3415-21kworker/3:111:10:193
1828099221,9cyclictest3415-21kworker/3:110:05:203
1828099221,9cyclictest3415-21kworker/3:107:49:433
1828099221,4cyclictest0-21swapper/311:40:223
18280992211,4cyclictest1421-21kworker/3:011:56:363
1828099221,10cyclictest19430-21/usr/sbin/munin12:05:253
1827399228,3cyclictest0-21swapper/211:59:312
1827399221,8cyclictest0-21swapper/208:42:492
1827399221,10cyclictest0-21swapper/210:33:192
1827399220,7cyclictest0-21swapper/208:24:452
1827399220,11cyclictest0-21swapper/210:02:012
1827399220,10cyclictest0-21swapper/210:18:212
1826899223,2cyclictest23-21ksoftirqd/110:45:161
1826899222,15cyclictest0-21swapper/108:50:561
18268992218,3cyclictest0-21swapper/111:40:191
18268992214,3cyclictest0-21swapper/108:38:411
18268992214,3cyclictest0-21swapper/108:31:431
1826899221,18cyclictest9383-21chrt09:15:031
1826899221,16cyclictest0-21swapper/112:34:401
1826899221,16cyclictest0-21swapper/108:08:161
1826899221,15cyclictest0-21swapper/111:10:151
1826899220,15cyclictest0-21swapper/110:20:111
1826399229,3cyclictest0-21swapper/011:22:250
1826399227,3cyclictest0-21swapper/009:40:330
1826399221,14cyclictest0-21swapper/008:09:370
1826399220,8cyclictest0-21swapper/009:05:130
1826399220,14cyclictest0-21swapper/012:31:360
1826399220,10cyclictest0-21swapper/012:13:010
1828099215,12cyclictest86750irq/128-enp1s0-11:45:243
1828099212,9cyclictest3415-21kworker/3:109:00:233
1828099212,9cyclictest3415-21kworker/3:108:54:073
1828099212,9cyclictest3415-21kworker/3:108:34:593
1828099211,9cyclictest3415-21kworker/3:110:48:513
1828099211,9cyclictest3415-21kworker/3:110:43:073
1828099211,5cyclictest0-21swapper/307:33:333
1828099211,11cyclictest30253-21sshd08:46:363
1827399217,3cyclictest0-21swapper/212:28:032
1827399217,3cyclictest0-21swapper/209:45:112
1827399216,3cyclictest0-21swapper/211:25:342
1827399211,9cyclictest0-21swapper/211:12:262
1827399211,9cyclictest0-21swapper/209:23:542
1827399211,5cyclictest0-21swapper/212:09:052
1827399211,10cyclictest0-21swapper/209:27:522
1827399211,10cyclictest0-21swapper/209:27:522
1826899212,17cyclictest0-21swapper/109:11:571
18268992115,3cyclictest4037-21/usr/sbin/munin11:30:231
18268992114,2cyclictest0-21swapper/107:26:321
1826899211,3cyclictest0-21swapper/109:55:241
1826899211,19cyclictest0-21swapper/109:24:561
1826899211,18cyclictest9346-21sed10:25:251
1826899211,18cyclictest0-21swapper/110:42:521
1826899211,17cyclictest21138-21expr10:55:131
1826899211,15cyclictest0-21swapper/108:58:351
1826899211,15cyclictest0-21swapper/108:49:241
1826899211,15cyclictest0-21swapper/108:19:261
1826899211,15cyclictest0-21swapper/108:02:021
1826899211,15cyclictest0-21swapper/107:44:081
1826899211,13cyclictest0-21swapper/111:53:341
1826899211,13cyclictest0-21swapper/111:53:341
1826899211,10cyclictest20432-21munin-run07:15:011
1826899211,10cyclictest10247-21expr10:30:121
18268992110,3cyclictest29202-21sed12:25:201
1826899210,15cyclictest0-21swapper/110:13:421
1826399211,12cyclictest0-21swapper/010:53:010
1828099207,2cyclictest3415-21kworker/3:110:33:393
1828099202,8cyclictest812-21snmpd12:39:343
1828099202,2cyclictest86850irq/129-enp1s0-08:11:283
1828099202,2cyclictest86850irq/129-enp1s0-08:11:283
1828099202,14cyclictest7619-21grep09:10:163
1828099201,9cyclictest7486-21sendmail_mailtr11:35:243
1828099201,8cyclictest3415-21kworker/3:111:19:553
1828099201,8cyclictest3415-21kworker/3:107:43:073
1828099201,10cyclictest19531-21ntp_states07:10:213
1828099201,10cyclictest15992-21/usr/sbin/munin09:30:173
1828099201,10cyclictest15368-21/usr/sbin/munin08:15:203
1828099200,10cyclictest1421-21kworker/3:012:12:283
1827399208,3cyclictest0-21swapper/211:22:542
1827399206,3cyclictest0-21swapper/211:06:582
1827399205,3cyclictest0-21swapper/208:01:292
1827399201,10cyclictest0-21swapper/209:33:192
1827399200,11cyclictest0-21swapper/212:12:232
1826899209,3cyclictest29533-21/usr/sbin/munin10:00:141
1826899202,3cyclictest0-21swapper/110:53:331
1826899202,17cyclictest0-21swapper/109:52:051
1826899202,17cyclictest0-21swapper/109:47:471
1826899202,16cyclictest0-21swapper/111:45:191
1826899201,8cyclictest0-21swapper/111:56:131
1826899201,17cyclictest0-21swapper/112:04:151
1826899201,17cyclictest0-21swapper/109:40:161
1826899201,17cyclictest0-21swapper/109:05:201
1826899201,14cyclictest0-21swapper/108:41:271
1826899201,13cyclictest0-21swapper/111:05:271
1826899201,11cyclictest28619-21grep07:30:161
1826899201,11cyclictest24785-21users11:00:251
1826899201,11cyclictest16268-21cstates09:30:131
1826899201,10cyclictest19779-21users09:35:281
1826899200,18cyclictest0-21swapper/111:15:321
1828099194,2cyclictest3415-21kworker/3:107:29:493
1828099191,10cyclictest21353-21/usr/sbin/munin07:15:213
1827399196,3cyclictest0-21swapper/211:50:032
1827399196,3cyclictest0-21swapper/211:50:032
1827399196,3cyclictest0-21swapper/211:16:282
1826899198,7cyclictest0-21swapper/110:39:201
1826899198,7cyclictest0-21swapper/110:39:201
1826899192,16cyclictest0-21swapper/109:26:361
1826899192,16cyclictest0-21swapper/109:26:361
18268991915,3cyclictest0-21swapper/107:15:261
1826899191,3cyclictest0-21swapper/112:15:141
18268991913,2cyclictest0-21swapper/108:14:311
18268991913,2cyclictest0-21swapper/108:14:311
1826899191,17cyclictest0-21swapper/112:12:241
1826899191,17cyclictest0-21swapper/107:45:381
1826399196,3cyclictest0-21swapper/009:32:570
1828099185,6cyclictest3415-21kworker/3:107:59:443
1827399181,15cyclictest0-21swapper/211:35:142
1827399181,12cyclictest0-21swapper/207:45:082
1826899182,13cyclictest2431-21gltestperf11:25:161
1826899181,3cyclictest0-21swapper/108:26:461
1826899181,2cyclictest0-21swapper/107:56:291
18268991812,3cyclictest31314-21proc_pri07:35:251
18273991712,3cyclictest24083-21cat11:00:192
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional