You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-10 - 05:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Fri Jul 10, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30376992823,4cyclictest0-21swapper/323:50:023
3037699270,23cyclictest23983-21diskmemload23:08:103
3037699270,0cyclictest0-21swapper/323:34:513
3037699260,0cyclictest0-21swapper/300:08:333
30372992624,1cyclictest0-21swapper/222:50:012
3037699250,24cyclictest0-21swapper/323:55:233
3037699250,24cyclictest0-21swapper/321:48:103
30372992522,0cyclictest0-21swapper/221:44:442
30370992525,0cyclictest0-21swapper/122:22:291
30370992523,1cyclictest0-21swapper/100:34:421
3037099250,1cyclictest0-21swapper/123:16:511
3036999253,21cyclictest62550irq/127-eno121:18:510
30369992523,1cyclictest0-21swapper/021:59:000
3036999250,1cyclictest0-21swapper/023:31:510
30376992422,1cyclictest4631-21apt-get21:21:473
30376992421,2cyclictest2695-21ssh00:10:393
30376992421,2cyclictest23983-21diskmemload00:33:453
3037699240,23cyclictest3484-21sh22:46:263
3037699240,23cyclictest2954-21missed_timers22:16:573
3037699240,23cyclictest27536-21ssh22:10:213
30372992422,1cyclictest0-21swapper/221:31:492
30372992422,1cyclictest0-21swapper/200:16:552
30372992421,2cyclictest14850-21ssh23:24:482
30372992419,4cyclictest0-21swapper/223:45:202
3037299240,24cyclictest21791-21ssh23:58:192
3037299240,23cyclictest3171-21ssh00:39:462
3037299240,23cyclictest23983-21diskmemload21:51:512
3037299240,23cyclictest23983-21diskmemload21:36:152
3037299240,23cyclictest0-21swapper/222:37:322
3037299240,23cyclictest0-21swapper/221:12:232
30370992421,2cyclictest12729-21sh22:26:111
3037099241,22cyclictest15724-21ssh21:31:321
3037099240,24cyclictest0-21swapper/100:21:431
3037099240,23cyclictest3624-21ssh23:42:441
3037099240,1cyclictest0-21swapper/121:16:551
3037099240,0cyclictest0-21swapper/123:21:521
30369992423,1cyclictest21950-21ssh22:33:480
30369992422,1cyclictest11612-21apt-get20:45:020
30369992421,2cyclictest17862-21sh21:32:100
3036999240,23cyclictest23983-21diskmemload22:10:490
3036999240,23cyclictest20156-21phc2sys23:56:560
3036999240,23cyclictest0-21swapper/000:17:410
3036999240,1cyclictest23983-21diskmemload00:32:320
30376992323,0cyclictest0-21swapper/322:04:263
30376992323,0cyclictest0-21swapper/321:52:033
30376992321,1cyclictest0-21swapper/321:11:283
3037699231,21cyclictest0-21swapper/321:20:003
3037699230,23cyclictest0-21swapper/323:35:163
3037699230,23cyclictest0-21swapper/322:59:083
3037699230,23cyclictest0-21swapper/322:35:253
3037699230,23cyclictest0-21swapper/322:20:373
3037699230,23cyclictest0-21swapper/321:55:453
3037699230,23cyclictest0-21swapper/300:38:423
3037699230,23cyclictest0-21swapper/300:22:013
3037699230,22cyclictest29865-21sh22:40:473
3037699230,22cyclictest20555-21ssh23:01:323
3037699230,22cyclictest11626-21perf20:45:003
3037699230,22cyclictest0-21swapper/300:15:053
3037699230,1cyclictest0-21swapper/300:02:453
3037699230,0cyclictest0-21swapper/321:35:313
30372992323,0cyclictest23983-21diskmemload21:21:052
30372992323,0cyclictest0-21swapper/223:43:422
30372992322,1cyclictest23983-21diskmemload21:45:262
30372992322,1cyclictest18003-21ssh22:58:302
30372992321,1cyclictest0-21swapper/200:30:012
30372992320,2cyclictest30458-21ssh22:41:402
30372992320,2cyclictest23983-21diskmemload21:26:352
3037299230,2cyclictest2462-21sh00:10:212
3037299230,23cyclictest0-21swapper/200:08:142
3037299230,23cyclictest0-21swapper/200:01:342
3037299230,22cyclictest7583-21ssh23:17:272
3037299230,22cyclictest2486-21sh23:13:122
3037299230,22cyclictest23983-21diskmemload21:57:362
3037299230,22cyclictest21032-21expr00:26:532
3037299230,22cyclictest0-21swapper/223:35:552
3037299230,1cyclictest5987-21ssh22:47:302
3037299230,1cyclictest0-21swapper/222:02:162
3037299230,0cyclictest0-21swapper/223:25:072
3037299230,0cyclictest0-21swapper/223:04:302
3037299230,0cyclictest0-21swapper/222:24:432
3037299230,0cyclictest0-21swapper/222:07:232
30370992323,0cyclictest0-21swapper/123:05:331
30370992323,0cyclictest0-21swapper/122:55:321
30370992323,0cyclictest0-21swapper/121:55:431
30370992322,1cyclictest0-21swapper/122:01:311
30370992321,1cyclictest783-21snmpd21:21:251
30370992321,1cyclictest5562-21perf19:25:011
30370992321,1cyclictest20400-21rm23:01:211
3037099230,23cyclictest23983-21diskmemload22:09:501
3037099230,23cyclictest0-21swapper/100:06:571
3037099230,22cyclictest9162-21ssh21:25:151
3037099230,22cyclictest6285-21diskmemload23:45:341
3037099230,22cyclictest2722-21ssh22:45:191
3037099230,22cyclictest24448-21ssh00:01:281
3037099230,22cyclictest20867-21rm23:57:031
3037099230,22cyclictest20497-21ssh21:35:251
3037099230,22cyclictest0-21swapper/123:33:101
3037099230,1cyclictest21611-21latency_hist21:05:011
3037099230,0cyclictest0-21swapper/100:35:461
30369992323,0cyclictest23983-21diskmemload23:15:330
30369992323,0cyclictest23983-21diskmemload22:35:090
30369992323,0cyclictest23983-21diskmemload00:26:200
30369992323,0cyclictest0-21swapper/023:54:390
30369992323,0cyclictest0-21swapper/023:01:380
30369992323,0cyclictest0-21swapper/022:06:510
30369992323,0cyclictest0-21swapper/021:48:350
30369992321,1cyclictest0-21swapper/022:41:490
30369992321,1cyclictest0-21swapper/000:11:420
30369992320,2cyclictest31496-21sh00:36:300
30369992320,1cyclictest14678-21sh22:56:170
3036999231,1cyclictest62550irq/127-eno121:51:030
3036999230,2cyclictest6401-21rm22:48:010
3036999230,23cyclictest0-21swapper/023:38:130
3036999230,23cyclictest0-21swapper/023:20:410
3036999230,22cyclictest24410-21sh21:10:150
3036999230,22cyclictest0-21swapper/023:25:570
3036999230,22cyclictest0-21swapper/023:12:100
3036999230,22cyclictest0-21swapper/022:26:340
3036999230,22cyclictest0-21swapper/000:05:480
3036999230,1cyclictest11448-21ssh22:52:170
3036999230,0cyclictest23983-21diskmemload21:38:320
3036999230,0cyclictest0-21swapper/023:49:220
3036999230,0cyclictest0-21swapper/023:07:530
3036999230,0cyclictest0-21swapper/022:16:050
3036999230,0cyclictest0-21swapper/022:01:200
3036999230,0cyclictest0-21swapper/021:23:380
30376992222,0cyclictest0-21swapper/323:40:343
30376992222,0cyclictest0-21swapper/323:16:033
30376992222,0cyclictest0-21swapper/323:10:423
30376992222,0cyclictest0-21swapper/322:50:213
30376992222,0cyclictest0-21swapper/321:32:423
30376992222,0cyclictest0-21swapper/300:28:563
30376992221,1cyclictest0-21swapper/323:46:503
30376992220,1cyclictest0-21swapper/323:20:023
30376992220,1cyclictest0-21swapper/322:09:133
3037699220,21cyclictest23983-21diskmemload21:40:053
3037699220,21cyclictest15355-21ssh23:25:073
3037699220,21cyclictest11934-21ssh22:25:123
3037699220,21cyclictest0-21swapper/321:25:553
3037699220,1cyclictest0-21swapper/322:34:173
30372992222,0cyclictest0-21swapper/223:30:052
30372992220,1cyclictest19945-21dump-pmu-power19:55:002
3037299220,21cyclictest32739-21apt-get19:15:022
3037299220,21cyclictest23983-21diskmemload23:51:342
3037299220,1cyclictest4798-21apt-get20:31:472
3037299220,1cyclictest29537-21latency_hist20:15:012
3037299220,1cyclictest0-21swapper/223:05:462
3037299220,1cyclictest0-21swapper/222:30:192
3037299220,1cyclictest0-21swapper/222:13:432
3037299220,1cyclictest0-21swapper/221:16:442
3037299220,0cyclictest23983-21diskmemload22:25:012
30370992222,0cyclictest23983-21diskmemload21:50:111
30370992222,0cyclictest0-21swapper/123:29:321
30370992220,1cyclictest9612-21apt-get20:41:461
30370992220,1cyclictest32027-21perf20:20:011
30370992220,1cyclictest31509-21ssh23:11:011
30370992220,1cyclictest11464-21sshd00:17:421
30370992220,1cyclictest0-21swapper/123:35:161
3037099220,22cyclictest0-21swapper/100:25:251
3037099220,21cyclictest32415-21apt-get20:21:461
3037099220,21cyclictest29214-21apt-get22:40:021
3037099220,21cyclictest28825-21munin-run22:40:011
3037099220,21cyclictest27324-21ssh22:10:051
3037099220,21cyclictest20334-21apt-get19:56:451
3037099220,21cyclictest0-21swapper/122:30:301
3037099220,21cyclictest0-21swapper/100:10:351
3037099220,1cyclictest26833-21ssh21:41:221
3037099220,1cyclictest0-21swapper/123:51:321
3037099220,1cyclictest0-21swapper/122:50:311
3037099220,1cyclictest0-21swapper/122:16:101
3037099220,0cyclictest0-21swapper/121:10:501
30369992222,0cyclictest0-21swapper/023:40:350
30369992221,1cyclictest0-21swapper/021:25:020
30369992221,1cyclictest0-21swapper/000:01:090
3036999220,21cyclictest0-21swapper/020:40:000
3036999220,1cyclictest0-21swapper/019:15:010
30376992119,1cyclictest606-21dbus-daemon20:40:003
3037699210,20cyclictest17190-21apt-get20:56:463
30372992120,1cyclictest0-21swapper/220:26:472
3037299210,20cyclictest0-21swapper/220:21:462
30370992120,0cyclictest0-21swapper/120:26:471
30369992120,1cyclictest0-21swapper/021:40:050
3036999210,0cyclictest17939-21apt-get19:51:460
3036999210,0cyclictest0-21swapper/019:36:470
30376992020,0cyclictest0-21swapper/320:46:463
3037699200,0cyclictest0-21swapper/320:21:463
30372992019,1cyclictest11612-21apt-get20:45:022
30372992016,3cyclictest21581-21cron21:05:012
3037299200,19cyclictest0-21swapper/219:26:472
30370992020,0cyclictest17190-21apt-get20:56:461
30370992020,0cyclictest0-21swapper/120:11:451
30370992020,0cyclictest0-21swapper/119:46:471
30369992018,1cyclictest24014-21munin-run21:10:010
3036999200,19cyclictest22368-21munin-run20:00:010
30376991919,0cyclictest0-21swapper/320:51:473
30376991919,0cyclictest0-21swapper/320:31:473
30376991919,0cyclictest0-21swapper/319:41:463
30376991919,0cyclictest0-21swapper/319:31:463
30376991917,1cyclictest5946-21apt-get19:26:483
30376991916,2cyclictest27160-21perf20:10:013
30372991919,0cyclictest0-21swapper/220:20:012
30372991919,0cyclictest0-21swapper/219:21:452
30372991918,1cyclictest22752-21apt-get20:01:462
30372991918,1cyclictest22752-21apt-get20:01:462
30372991917,1cyclictest10358-21awk19:35:002
30370991919,0cyclictest0-21swapper/120:01:471
30370991919,0cyclictest0-21swapper/120:01:471
30370991918,1cyclictest32739-21apt-get19:15:011
30370991917,1cyclictest22-21ksoftirqd/120:40:001
30370991917,1cyclictest10736-21apt-get19:36:471
30369991919,0cyclictest14795-21apt-get20:51:470
30369991919,0cyclictest0-21swapper/019:16:460
30369991917,1cyclictest4410-21perf20:30:000
30376991818,0cyclictest0-21swapper/320:16:463
30376991818,0cyclictest0-21swapper/319:51:463
30376991816,1cyclictest15547-21apt-get19:46:473
30372991818,0cyclictest0-21swapper/220:56:462
30370991818,0cyclictest0-21swapper/120:45:021
30369991816,1cyclictest11612-21apt-get20:45:000
30369991816,1cyclictest0-21swapper/019:50:000
30376991715,1cyclictest5558-21latency_hist19:25:013
30372991717,0cyclictest0-21swapper/219:36:462
30372991715,1cyclictest0-21swapper/220:40:002
30370991717,0cyclictest0-21swapper/119:51:461
3037099170,16cyclictest0-21swapper/120:06:461
3037099170,0cyclictest0-21swapper/119:26:481
30369991717,0cyclictest0-21swapper/020:56:460
30369991717,0cyclictest0-21swapper/019:21:450
30369991715,1cyclictest32012-21latency_hist20:20:010
3036999170,16cyclictest20329-21munin-node19:42:490
30376991616,0cyclictest2397-21apt-get20:26:473
30376991615,1cyclictest0-21swapper/321:01:473
30372991616,0cyclictest0-21swapper/219:46:472
30372991616,0cyclictest0-21swapper/219:41:462
30372991614,1cyclictest0-21swapper/221:10:012
3037099160,15cyclictest606-21dbus-daemon19:20:011
3037099160,15cyclictest0-21swapper/119:35:001
30369991616,0cyclictest0-21swapper/021:01:470
30369991613,2cyclictest24756-21turbostat.cron20:05:010
30369991613,2cyclictest24756-21turbostat.cron20:05:010
30376991515,0cyclictest0-21swapper/320:11:463
30376991515,0cyclictest0-21swapper/319:56:453
3037699150,1cyclictest0-21swapper/319:16:563
30372991515,0cyclictest0-21swapper/219:56:452
30369991513,1cyclictest0-21swapper/020:11:280
3036999150,14cyclictest0-21swapper/019:31:460
3036999150,11cyclictest0-21swapper/020:21:460
3037699140,14cyclictest0-21swapper/321:05:213
3037699140,14cyclictest0-21swapper/319:10:153
3037699140,0cyclictest0-21swapper/320:00:443
3037699140,0cyclictest0-21swapper/320:00:443
3037699140,0cyclictest0-21swapper/319:36:403
30372991414,0cyclictest0-21swapper/219:10:172
3037299140,14cyclictest0-21swapper/220:05:392
3037299140,0cyclictest0-21swapper/220:51:172
3037099140,14cyclictest0-21swapper/121:05:521
3037099140,0cyclictest0-21swapper/120:34:261
3037099140,0cyclictest0-21swapper/119:40:591
3036999140,14cyclictest0-21swapper/019:25:350
3036999140,0cyclictest26141-21strings20:06:510
3036999140,0cyclictest0-21swapper/020:31:480
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional