You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-27 - 22:29

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Wed May 27, 2026 12:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27801993714,22cyclictest4929-21wc09:25:000
2780399290,28cyclictest19803-21sh12:39:372
2780199290,28cyclictest0-21swapper/012:38:190
27801992721,5cyclictest21720-21ssh10:10:190
2780199270,26cyclictest0-21swapper/012:28:120
2780199270,26cyclictest0-21swapper/012:13:160
2780199270,1cyclictest0-21swapper/009:52:230
2780199270,1cyclictest0-21swapper/009:52:230
2780199270,0cyclictest0-21swapper/010:35:570
2780299263,22cyclictest22-21ksoftirqd/112:13:111
27801992620,5cyclictest14711-21ssh11:34:370
27801992620,5cyclictest0-21swapper/009:33:180
2780199260,25cyclictest6471-21ssh09:27:060
2780199260,25cyclictest12440-21/usr/sbin/munin10:03:120
2780199260,25cyclictest0-21swapper/009:35:590
2780199260,22cyclictest21128-21diskmemload09:59:260
27808992523,1cyclictest19429-21timerandwakeup09:38:193
2780299250,24cyclictest21128-21diskmemload09:22:451
27801992524,1cyclictest62550irq/127-eno111:00:390
27801992520,4cyclictest0-21swapper/011:27:320
27801992520,4cyclictest0-21swapper/010:48:420
27801992519,5cyclictest899-21ssh10:21:360
27801992519,5cyclictest21633-21ssh10:40:520
2780199250,24cyclictest899-21sh11:52:190
2780199250,24cyclictest21128-21diskmemload11:45:260
2780199250,24cyclictest21128-21diskmemload11:07:090
2780199250,21cyclictest0-21swapper/010:33:400
2780199250,0cyclictest0-21swapper/012:33:200
27808992422,1cyclictest21128-21diskmemload09:23:103
2780899241,20cyclictest151rcu_preempt11:02:333
2780399240,23cyclictest12084-21apt-get11:33:112
2780399240,23cyclictest10080-21rm09:59:562
2780399240,23cyclictest0-21swapper/211:56:012
2780399240,1cyclictest27772-21ssh12:17:252
27802992421,2cyclictest27896-21sh10:47:141
2780299241,22cyclictest22-21ksoftirqd/112:35:001
2780299240,24cyclictest0-21swapper/111:51:201
2780299240,23cyclictest21128-21diskmemload10:55:301
2780299240,23cyclictest0-21swapper/111:14:301
2780299240,23cyclictest0-21swapper/109:40:201
2780299240,1cyclictest0-21swapper/111:10:001
27801992421,3cyclictest0-21swapper/011:57:480
27801992420,3cyclictest21128-21diskmemload12:05:220
27801992420,3cyclictest0-21swapper/011:43:550
2780199240,23cyclictest31955-21idleruntime-cro10:20:000
2780199240,23cyclictest21128-21diskmemload09:14:460
2780199240,21cyclictest21128-21diskmemload11:17:440
2780199240,0cyclictest21128-21diskmemload10:53:230
2780199240,0cyclictest0-21swapper/011:37:380
2780199240,0cyclictest0-21swapper/009:40:560
27808992323,0cyclictest21128-21diskmemload11:49:503
27808992323,0cyclictest21128-21diskmemload11:43:193
27808992323,0cyclictest21128-21diskmemload10:13:453
27808992323,0cyclictest0-21swapper/310:09:593
27808992323,0cyclictest0-21swapper/309:53:553
27808992323,0cyclictest0-21swapper/309:53:553
27808992323,0cyclictest0-21swapper/309:15:163
27808992321,1cyclictest6740-21idleruntime-cro08:40:003
27808992321,1cyclictest0-21swapper/312:23:353
2780899230,2cyclictest32747-21rm11:51:333
2780899230,2cyclictest21651-21rm12:11:053
2780899230,23cyclictest21128-21diskmemload11:39:413
2780899230,23cyclictest0-21swapper/312:16:003
2780899230,22cyclictest8702-21ssh10:58:293
2780899230,22cyclictest6559-21sh10:27:053
2780899230,22cyclictest21128-21diskmemload11:24:343
2780899230,22cyclictest21128-21diskmemload11:17:353
2780899230,22cyclictest21128-21diskmemload10:41:073
2780899230,22cyclictest21128-21diskmemload09:57:203
2780899230,22cyclictest13950-21turbostat.cron08:55:003
2780899230,22cyclictest12371-21sh10:32:533
2780899230,22cyclictest11385-21ssh11:32:093
2780899230,1cyclictest28031-21ssh10:16:483
2780899230,1cyclictest0-21swapper/311:59:493
2780899230,0cyclictest21128-21diskmemload12:30:463
2780899230,0cyclictest21128-21diskmemload11:27:413
2780899230,0cyclictest21128-21diskmemload09:45:583
2780899230,0cyclictest0-21swapper/312:36:013
2780899230,0cyclictest0-21swapper/310:49:303
27803992323,0cyclictest0-21swapper/212:11:262
27803992323,0cyclictest0-21swapper/210:19:292
27803992323,0cyclictest0-21swapper/210:14:332
27803992323,0cyclictest0-21swapper/209:38:192
27803992323,0cyclictest0-21swapper/209:15:162
27803992321,1cyclictest6410-21ssh09:27:012
27803992321,1cyclictest26894-21rm09:45:492
27803992321,1cyclictest15517-21ssh11:05:242
27803992321,1cyclictest0-21swapper/209:51:042
27803992321,1cyclictest0-21swapper/209:51:042
27803992321,1cyclictest0-21swapper/208:40:002
27803992320,2cyclictest3220-21sh11:23:542
27803992320,0cyclictest0-21swapper/211:00:232
27803992317,2cyclictest11123-21rm09:31:002
2780399230,2cyclictest27598-21ssh11:17:292
2780399230,2cyclictest17054-21ssh10:37:032
2780399230,22cyclictest5142-21ssh11:25:502
2780399230,22cyclictest22218-21sh09:41:472
2780399230,22cyclictest21128-21diskmemload12:28:382
2780399230,22cyclictest21128-21diskmemload12:07:092
2780399230,22cyclictest19616-21sh10:08:172
2780399230,22cyclictest13918-21cron08:55:002
2780399230,22cyclictest10319-21ssh12:00:272
2780399230,22cyclictest0-21swapper/210:41:072
2780399230,0cyclictest21128-21diskmemload10:49:302
27802992323,0cyclictest21128-21diskmemload10:20:371
27802992323,0cyclictest0-21swapper/112:23:351
27802992323,0cyclictest0-21swapper/109:55:291
27802992323,0cyclictest0-21swapper/109:31:081
27802992321,1cyclictest20877-21munin-run10:40:011
27802992321,1cyclictest17604-21apt-get12:08:111
27802992321,1cyclictest0-21swapper/111:22:251
27802992321,1cyclictest0-21swapper/110:34:091
27802992321,1cyclictest0-21swapper/109:52:011
27802992321,1cyclictest0-21swapper/109:52:011
2780299230,23cyclictest0-21swapper/112:36:011
2780299230,23cyclictest0-21swapper/112:26:281
2780299230,23cyclictest0-21swapper/109:36:271
2780299230,22cyclictest21128-21diskmemload12:03:121
2780299230,22cyclictest10663-21sh11:31:091
2780299230,22cyclictest0-21swapper/111:25:021
2780299230,0cyclictest21128-21diskmemload11:55:031
2780299230,0cyclictest151rcu_preempt08:50:001
2780299230,0cyclictest0-21swapper/110:06:591
2780299230,0cyclictest0-21swapper/109:46:431
2780299230,0cyclictest0-21swapper/109:25:441
27801992323,0cyclictest0-21swapper/012:17:190
27801992322,1cyclictest27094-21ssh09:15:270
27801992321,2cyclictest0-21swapper/011:21:170
27801992319,4cyclictest0-21swapper/012:02:440
27801992319,3cyclictest0-21swapper/012:20:550
2780199231,1cyclictest62550irq/127-eno109:45:540
2780199230,22cyclictest16706-21rm10:05:550
2780199230,1cyclictest0-21swapper/010:29:220
2780199230,0cyclictest21128-21diskmemload10:55:230
27808992222,0cyclictest0-21swapper/310:35:173
27808992221,1cyclictest0-21swapper/312:26:033
27808992221,1cyclictest0-21swapper/310:22:253
27808992220,1cyclictest15202-21apt-get07:53:143
27808992220,1cyclictest0-21swapper/312:05:493
27808992219,2cyclictest21862-21sh09:10:363
2780899220,22cyclictest962-21sh10:52:323
2780899220,21cyclictest22805-21sh09:42:383
2780899220,21cyclictest21462-21ssh11:11:103
2780899220,21cyclictest10528-21ssh09:30:103
2780899220,21cyclictest0-21swapper/309:10:013
2780899220,21cyclictest0-21swapper/308:58:123
2780899220,21cyclictest0-21swapper/308:48:113
27803992222,0cyclictest0-21swapper/211:51:212
27803992222,0cyclictest0-21swapper/210:27:002
27803992220,1cyclictest30198-21munin-run07:15:002
27803992220,1cyclictest0-21swapper/211:46:522
2780399221,0cyclictest0-21swapper/211:36:072
2780399220,22cyclictest0-21swapper/210:52:432
2780399220,21cyclictest21219-21awk09:10:012
2780399220,21cyclictest14447-21apt-get08:58:122
2780399220,21cyclictest10961-21ssh10:30:422
2780399220,21cyclictest0-21swapper/212:32:162
2780399220,21cyclictest0-21swapper/210:56:092
2780399220,21cyclictest0-21swapper/207:58:112
2780399220,1cyclictest23651-21irqstats11:13:152
2780399220,1cyclictest21128-21diskmemload10:24:412
2780399220,1cyclictest14446-21ssh10:03:312
27802992222,0cyclictest7279-21apt-get10:28:101
27802992222,0cyclictest0-21swapper/111:42:301
27802992222,0cyclictest0-21swapper/111:35:201
27802992220,1cyclictest0-21swapper/110:15:071
27802992220,0cyclictest151rcu_preempt09:13:471
2780299220,21cyclictest32116-21ssh10:50:391
2780299220,21cyclictest21128-21diskmemload11:00:051
2780299220,21cyclictest0-21swapper/111:15:061
2780299220,21cyclictest0-21swapper/110:01:101
2780299220,21cyclictest0-21swapper/108:13:111
2780299220,1cyclictest0-21swapper/111:46:001
2780299220,1cyclictest0-21swapper/110:13:041
2780299220,0cyclictest0-21swapper/112:15:281
2780299220,0cyclictest0-21swapper/109:15:171
27801992220,1cyclictest32598-21dump-pmu-power07:20:000
27808992120,1cyclictest30197-21perf07:15:003
2780899210,20cyclictest0-21swapper/307:58:113
2780899210,0cyclictest0-21swapper/310:00:073
27803992120,1cyclictest0-21swapper/212:20:022
2780399210,21cyclictest0-21swapper/209:20:252
2780399210,21cyclictest0-21swapper/209:10:042
2780399210,20cyclictest0-21swapper/208:48:112
2780399210,20cyclictest0-21swapper/207:53:142
2780399210,0cyclictest32065-21apt-get08:28:122
27802992119,1cyclictest1649-21cron08:10:001
2780299210,21cyclictest0-21swapper/107:53:141
2780299210,20cyclictest0-21swapper/107:35:011
2780299210,0cyclictest0-21swapper/107:18:101
27801992121,0cyclictest0-21swapper/008:43:100
27801992121,0cyclictest0-21swapper/008:18:110
27801992120,1cyclictest62550irq/127-eno108:33:130
27801992120,1cyclictest24847-21apt-get08:13:110
27801992120,0cyclictest17591-21apt-get07:58:110
27803992019,0cyclictest0-21swapper/208:03:112
27803992017,2cyclictest4968-21sh07:30:012
2780399200,10cyclictest123171/usr/sbin/munin07:45:012
2780399200,0cyclictest30685-21apt-get07:18:102
27802992020,0cyclictest0-21swapper/108:33:131
27801992020,0cyclictest0-21swapper/008:38:110
27801992020,0cyclictest0-21swapper/008:23:120
27801992020,0cyclictest0-21swapper/007:23:120
27801992020,0cyclictest0-21swapper/007:13:120
27801992018,1cyclictest32065-21apt-get08:28:120
27808991919,0cyclictest16863-21apt-get09:03:113
27808991919,0cyclictest0-21swapper/308:03:113
27808991919,0cyclictest0-21swapper/307:48:113
27808991917,1cyclictest4971-21turbostat07:30:013
27808991917,1cyclictest0-21swapper/308:25:013
27803991918,1cyclictest0-21swapper/207:23:122
27803991918,0cyclictest0-21swapper/209:03:112
27803991917,1cyclictest9165-21perf08:45:002
27803991917,1cyclictest0-21swapper/208:25:012
2780399190,18cyclictest12803-21apt-get07:48:112
27802991919,0cyclictest0-21swapper/108:28:121
27802991917,1cyclictest12279-21cron07:45:011
27801991919,0cyclictest20008-21apt-get08:03:110
27801991919,0cyclictest0-21swapper/009:08:120
27801991917,1cyclictest24357-21ls08:10:000
2780199190,18cyclictest10409-21apt-get07:43:110
27808991818,0cyclictest0-21swapper/307:23:123
27808991816,1cyclictest7263-21apt-get08:43:103
27802991818,0cyclictest0-21swapper/109:08:121
27802991816,1cyclictest0-21swapper/108:00:001
27801991818,0cyclictest0-21swapper/007:53:140
27801991816,1cyclictest0-21swapper/008:53:130
27808991716,1cyclictest0-21swapper/308:18:113
27803991714,2cyclictest3951-21basename08:35:002
27802991717,0cyclictest0-21swapper/107:13:121
27802991716,0cyclictest7985-21apt-get07:38:111
27802991715,1cyclictest18771-21munin-run09:05:011
2780299170,0cyclictest151rcu_preempt07:23:121
2780199170,14cyclictest0-21swapper/007:32:350
2780199170,14cyclictest0-21swapper/007:28:030
27808991616,0cyclictest0-21swapper/308:13:113
27808991615,0cyclictest22454-21apt-get08:08:113
27808991614,1cyclictest3947-21perf08:35:003
27803991616,0cyclictest0-21swapper/208:18:112
27803991616,0cyclictest0-21swapper/208:13:112
27802991614,1cyclictest0-21swapper/107:28:121
27802991613,2cyclictest31115-21sed08:23:211
27801991616,0cyclictest0-21swapper/007:38:160
27801991614,1cyclictest13852-21ntp_states07:48:170
2780199160,14cyclictest0-21swapper/008:46:310
27808991515,0cyclictest0-21swapper/307:38:103
27803991515,0cyclictest0-21swapper/208:08:112
27802991515,0cyclictest0-21swapper/108:03:111
27802991513,1cyclictest622-21gdbus08:20:011
27801991515,0cyclictest0-21swapper/008:58:120
2780899140,14cyclictest0-21swapper/307:40:093
2780899140,14cyclictest0-21swapper/307:30:563
2780899140,14cyclictest0-21swapper/307:15:113
2780399140,14cyclictest0-21swapper/207:30:562
27802991414,0cyclictest0-21swapper/108:42:041
2780299140,14cyclictest0-21swapper/108:57:141
2780199140,14cyclictest0-21swapper/009:02:300
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional