You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-27 - 22:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sat Jun 27, 2026 12:44:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
124999271,25cyclictest0-21swapper/209:20:012
124999270,26cyclictest27421-21diskmemload09:50:002
124999270,1cyclictest622-21gdbus07:50:012
124999270,1cyclictest3420-21ntp_states11:10:522
124999270,0cyclictest27421-21diskmemload12:27:012
125399260,25cyclictest27421-21diskmemload11:29:443
1249992620,5cyclictest22692-21sendmail-msp11:00:012
124999260,1cyclictest0-21swapper/210:45:422
123799260,25cyclictest18759-21diskmemload10:27:050
1249992521,3cyclictest27421-21diskmemload11:43:562
124999250,24cyclictest9641-21rm12:14:492
124999250,24cyclictest4016-21ssh10:14:592
124999250,24cyclictest27421-21diskmemload11:59:052
124999250,24cyclictest27421-21diskmemload10:43:152
124999250,21cyclictest17940-21ssh09:57:572
124999250,1cyclictest5288-21ssh12:39:432
124999250,1cyclictest25037-21idleruntime-cro09:05:002
124999250,1cyclictest21921-21ssh10:30:362
124999250,0cyclictest0-21swapper/210:16:062
124499250,24cyclictest17333-21ssh10:55:031
124499250,24cyclictest16824-21rm10:54:341
124499250,24cyclictest0-21swapper/112:06:551
124499250,24cyclictest0-21swapper/111:58:101
124499250,1cyclictest31782-21sh12:34:191
123799253,21cyclictest62550irq/127-eno110:31:090
123799250,24cyclictest0-21swapper/011:36:160
123799250,24cyclictest0-21swapper/009:39:350
125399240,23cyclictest27421-21diskmemload09:12:093
125399240,23cyclictest0-21swapper/310:46:573
1249992422,1cyclictest27421-21diskmemload09:50:012
1249992420,3cyclictest0-21swapper/211:47:212
1249992419,4cyclictest27421-21diskmemload10:50:212
1249992419,4cyclictest18254-21sh12:21:012
1249992419,4cyclictest0-21swapper/208:00:012
124999240,23cyclictest0-21swapper/210:28:412
124999240,21cyclictest0-21swapper/211:01:092
124999240,21cyclictest0-21swapper/209:30:412
124999240,20cyclictest23342-21ssh11:28:572
124999240,0cyclictest0-21swapper/211:39:382
1244992424,0cyclictest0-21swapper/111:10:121
124499240,23cyclictest19168-21ssh11:25:391
124499240,23cyclictest0-21swapper/109:17:081
124499240,1cyclictest23539-21apt-get11:00:461
124499240,0cyclictest0-21swapper/109:24:571
1237992422,1cyclictest0-21swapper/010:05:450
1237992420,3cyclictest27421-21diskmemload10:00:580
123799240,1cyclictest27421-21diskmemload11:05:440
123799240,1cyclictest0-21swapper/010:41:450
1253992323,0cyclictest27421-21diskmemload09:16:173
1253992323,0cyclictest0-21swapper/312:25:473
1253992323,0cyclictest0-21swapper/311:14:413
1253992323,0cyclictest0-21swapper/311:03:493
1253992323,0cyclictest0-21swapper/309:42:443
1253992323,0cyclictest0-21swapper/309:20:383
1253992322,1cyclictest14733-21ssh10:51:463
1253992322,0cyclictest27421-21diskmemload09:52:373
1253992321,1cyclictest6075-21perf07:20:003
1253992321,1cyclictest27421-21diskmemload10:58:143
1253992321,1cyclictest26161-21ssh09:36:453
1253992320,2cyclictest28985-21sh11:05:453
1253992320,2cyclictest15456-21idleruntime-cro08:45:013
125399230,2cyclictest3076-21ssh11:40:013
125399230,2cyclictest15756-21apt-get08:45:463
125399230,23cyclictest27421-21diskmemload11:38:013
125399230,23cyclictest0-21swapper/312:01:003
125399230,23cyclictest0-21swapper/311:31:523
125399230,22cyclictest8041-21apt-get11:15:453
125399230,22cyclictest27452-21perf09:10:003
125399230,22cyclictest27421-21diskmemload12:37:403
125399230,22cyclictest27039-21ssh12:30:123
125399230,22cyclictest15693-21ssh11:21:103
125399230,22cyclictest0-21swapper/310:40:043
125399230,22cyclictest0-21swapper/310:20:023
125399230,1cyclictest27421-21diskmemload09:29:023
125399230,1cyclictest1937-21ssh10:12:203
1249992323,0cyclictest18153-21apt-get08:50:472
1249992321,1cyclictest12444-21/usr/sbin/munin12:15:562
1249992320,2cyclictest27421-21diskmemload10:22:282
124999230,23cyclictest0-21swapper/212:01:172
124999230,23cyclictest0-21swapper/211:24:002
124999230,23cyclictest0-21swapper/210:00:342
124999230,23cyclictest0-21swapper/209:24:582
124999230,22cyclictest312992sleep209:41:032
124999230,22cyclictest12288-21ssh09:25:282
124999230,1cyclictest0-21swapper/208:00:542
124999230,0cyclictest27421-21diskmemload10:07:542
124999230,0cyclictest0-21swapper/212:30:552
124999230,0cyclictest0-21swapper/212:09:592
124999230,0cyclictest0-21swapper/211:30:152
124999230,0cyclictest0-21swapper/209:10:162
1244992323,0cyclictest0-21swapper/111:07:081
1244992323,0cyclictest0-21swapper/110:33:461
1244992323,0cyclictest0-21swapper/110:27:051
1244992323,0cyclictest0-21swapper/110:10:021
1244992323,0cyclictest0-21swapper/109:28:021
1244992321,1cyclictest4716-21ssh12:38:511
1244992321,1cyclictest30393-21apt-get11:35:461
1244992321,1cyclictest0-21swapper/112:15:171
1244992321,1cyclictest0-21swapper/109:45:561
1244992321,1cyclictest0-21swapper/109:32:581
124499230,2cyclictest24939-21ssh09:35:521
124499230,23cyclictest27421-21diskmemload11:15:531
124499230,23cyclictest27421-21diskmemload10:00:191
124499230,23cyclictest0-21swapper/111:47:111
124499230,23cyclictest0-21swapper/109:50:041
124499230,22cyclictest8866-21sh10:18:591
124499230,22cyclictest6116-21ssh12:10:511
124499230,22cyclictest2906-21ssh10:40:551
124499230,22cyclictest17651-21ssh09:57:291
124499230,22cyclictest1110-21ssh09:44:101
124499230,22cyclictest10409-21/usr/sbin/munin10:20:531
124499230,0cyclictest0-21swapper/112:25:231
124499230,0cyclictest0-21swapper/112:00:521
124499230,0cyclictest0-21swapper/111:54:101
124499230,0cyclictest0-21swapper/111:34:331
1237992323,0cyclictest27421-21diskmemload11:43:590
1237992323,0cyclictest27421-21diskmemload11:21:090
1237992323,0cyclictest0-21swapper/012:30:280
1237992323,0cyclictest0-21swapper/012:16:410
1237992323,0cyclictest0-21swapper/012:05:510
1237992323,0cyclictest0-21swapper/010:54:030
1237992323,0cyclictest0-21swapper/010:23:000
1237992323,0cyclictest0-21swapper/010:17:270
1237992323,0cyclictest0-21swapper/009:25:420
1237992323,0cyclictest0-21swapper/007:20:450
1237992321,1cyclictest0-21swapper/012:21:260
1237992321,1cyclictest0-21swapper/011:56:270
1237992321,1cyclictest0-21swapper/009:41:580
1237992321,1cyclictest0-21swapper/009:20:410
1237992320,2cyclictest30253-21sh10:36:330
123799230,2cyclictest1195-21diskmemload10:11:160
123799230,23cyclictest0-21swapper/011:50:440
123799230,23cyclictest0-21swapper/009:50:480
123799230,23cyclictest0-21swapper/009:47:290
123799230,22cyclictest3591-21rm12:37:200
123799230,22cyclictest27421-21diskmemload11:19:560
123799230,22cyclictest16209-21awk07:40:000
123799230,22cyclictest0-21swapper/012:01:290
123799230,22cyclictest0-21swapper/011:10:460
123799230,22cyclictest0-21swapper/010:55:470
123799230,1cyclictest0-21swapper/009:15:570
123799230,0cyclictest0-21swapper/009:11:490
1253992222,0cyclictest0-21swapper/312:10:313
1253992222,0cyclictest0-21swapper/310:35:573
1253992221,1cyclictest24090-21apt-get07:55:463
1253992221,1cyclictest20071-21ssh10:00:303
1253992220,1cyclictest0-21swapper/310:09:023
125399220,22cyclictest0-21swapper/311:46:463
125399220,21cyclictest15774-21diskmemload12:20:073
125399220,21cyclictest13052-21sendmail-msp08:40:013
125399220,21cyclictest0-21swapper/311:55:033
125399220,21cyclictest0-21swapper/310:30:283
125399220,21cyclictest0-21swapper/309:45:183
125399220,21cyclictest0-21swapper/309:32:343
125399220,21cyclictest0-21swapper/308:15:003
125399220,21cyclictest0-21swapper/307:10:473
125399220,1cyclictest0-21swapper/310:28:273
125399220,1cyclictest0-21swapper/309:55:493
125399220,1cyclictest0-21swapper/307:45:023
125399220,0cyclictest27421-21diskmemload11:53:473
1249992222,0cyclictest0-21swapper/211:53:362
1249992222,0cyclictest0-21swapper/211:19:132
1249992218,3cyclictest27421-21diskmemload11:07:372
124999220,21cyclictest17853-21munin-run08:50:012
124999220,21cyclictest13679-21apt07:35:002
1244992222,0cyclictest0-21swapper/110:45:021
1244992220,1cyclictest956-21latency_hist08:15:001
1244992220,1cyclictest0-21swapper/109:15:001
124499220,22cyclictest18060-21ssh11:24:251
124499220,22cyclictest0-21swapper/111:40:211
124499220,21cyclictest0-21swapper/108:45:011
124499220,21cyclictest0-21swapper/108:05:461
124499220,1cyclictest3758-21apt-get08:20:451
124499220,1cyclictest19778-21sh12:23:121
124499220,1cyclictest1531-21apt-get07:10:471
124499220,1cyclictest0-21swapper/110:06:591
124499220,1cyclictest0-21swapper/107:55:011
1237992220,1cyclictest3406-21awk08:20:000
1237992220,1cyclictest22662-21munin-run09:00:000
1237992220,1cyclictest0-21swapper/011:25:500
123799220,21cyclictest8248-21switchtime10:45:550
123799220,21cyclictest30979-21awk08:10:000
123799220,21cyclictest27421-21diskmemload09:55:160
123799220,21cyclictest0-21swapper/012:25:280
123799220,21cyclictest0-21swapper/011:30:130
123799220,21cyclictest0-21swapper/009:31:580
123799220,21cyclictest0-21swapper/008:50:470
123799220,1cyclictest27421-21diskmemload12:11:160
1253992121,0cyclictest0-21swapper/312:15:353
125399210,20cyclictest0-21swapper/310:15:143
125399210,20cyclictest0-21swapper/307:30:453
125399210,0cyclictest0-21swapper/312:05:493
1244992121,0cyclictest0-21swapper/108:35:471
1244992120,1cyclictest9194-21apt-get07:25:451
124499210,20cyclictest0-21swapper/108:00:451
1237992121,0cyclictest0-21swapper/007:45:450
1237992120,1cyclictest25360-21apt-get09:05:460
1237992120,0cyclictest0-21swapper/008:30:470
123799210,21cyclictest0-21swapper/011:45:070
123799210,1cyclictest15451-21perf08:45:010
1253992020,0cyclictest0-21swapper/308:55:453
1253992020,0cyclictest0-21swapper/307:50:453
1253992019,0cyclictest28902-21apt-get08:05:453
125399200,0cyclictest0-21swapper/308:50:473
1249992020,0cyclictest0-21swapper/208:40:472
1249992020,0cyclictest0-21swapper/208:30:472
1249992020,0cyclictest0-21swapper/208:15:462
1249992020,0cyclictest0-21swapper/207:35:462
1249992020,0cyclictest0-21swapper/207:10:472
124999200,1cyclictest8553-21munin-run07:25:002
124999200,19cyclictest0-21swapper/207:15:512
1244992020,0cyclictest1275-21apt-get08:15:451
1244992020,0cyclictest0-21swapper/107:30:451
124499200,0cyclictest0-21swapper/107:55:461
1237992020,0cyclictest3984-21apt-get07:15:450
1237992020,0cyclictest0-21swapper/007:10:480
1237992018,1cyclictest26507-21apt-get08:00:450
1253991919,0cyclictest0-21swapper/308:30:473
1253991917,1cyclictest8491-21turbostat.cron07:25:003
1253991917,1cyclictest18607-21perf07:45:013
1249991919,0cyclictest21694-21apt-get07:50:452
124999190,1cyclictest20633-21cpuspeed08:55:452
124999190,1cyclictest17531-21ntpq07:40:512
124999190,1cyclictest10105-21cat07:25:502
124999190,0cyclictest0-21swapper/208:25:462
1244991919,0cyclictest0-21swapper/109:05:461
1244991916,2cyclictest22659-21munin-plugin-st09:00:001
1237991919,0cyclictest0-21swapper/007:25:450
1237991917,1cyclictest18586-21cron07:45:010
1253991818,0cyclictest6153-21apt-get08:25:463
1249991816,1cyclictest954-21apt08:15:002
124499180,17cyclictest0-21swapper/107:45:451
1253991716,1cyclictest0-21swapper/307:25:023
1253991715,1cyclictest25059-21turbostat09:05:003
1249991717,0cyclictest0-21swapper/208:35:472
1244991717,0cyclictest0-21swapper/108:45:461
1244991717,0cyclictest0-21swapper/107:35:461
1244991716,0cyclictest8549-21apt-get08:30:471
1237991717,0cyclictest0-21swapper/009:00:450
1237991717,0cyclictest0-21swapper/008:45:460
1237991715,1cyclictest31296-21apt-get08:10:460
1253991616,0cyclictest0-21swapper/308:20:453
1253991616,0cyclictest0-21swapper/308:15:453
124999160,2cyclictest0-21swapper/208:05:452
124999160,0cyclictest3958-21cstates08:20:462
1237991616,0cyclictest0-21swapper/008:25:460
1237991616,0cyclictest0-21swapper/007:50:450
1253991513,1cyclictest16202-21munin-run07:40:003
124999150,15cyclictest0-21swapper/209:08:152
1244991515,0cyclictest0-21swapper/107:15:451
125399140,14cyclictest0-21swapper/308:00:403
1244991414,0cyclictest0-21swapper/109:00:171
1244991414,0cyclictest0-21swapper/108:50:171
124499140,14cyclictest0-21swapper/108:25:011
124499140,0cyclictest0-21swapper/107:20:121
1237991414,0cyclictest0-21swapper/007:55:040
123799140,1cyclictest0-21swapper/008:35:510
123799140,14cyclictest0-21swapper/008:20:350
123799140,14cyclictest0-21swapper/007:30:030
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional