You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-13 - 21:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Sat Dec 13, 2025 12:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6216992420,4cyclictest0-21swapper/208:14:172
621699240,1cyclictest19141-21apt-get12:04:162
6210992422,1cyclictest28272-21apt-get10:09:170
621099240,1cyclictest606-21dbus-daemon07:45:010
622099230,2cyclictest4867-21apt-get09:20:013
622099230,23cyclictest0-21swapper/312:04:263
622099230,22cyclictest5857-21apt-get09:24:313
622099230,22cyclictest16153-21perf11:55:013
621699230,22cyclictest16174-21apt-get08:39:162
621699230,22cyclictest15838-21perf07:30:012
621699230,22cyclictest0-21swapper/208:15:022
621699230,1cyclictest13234-21munin-run09:35:002
6214992323,0cyclictest0-21swapper/111:29:161
621499230,2cyclictest28181-21sh12:20:001
621499230,22cyclictest32476-21latency_hist10:15:011
621499230,22cyclictest15657-21basename09:40:001
621099230,2cyclictest20740-21idleruntime-cro07:40:010
6220992220,1cyclictest5414-21sendmail-msp12:40:013
6220992220,1cyclictest26525-21sh11:10:013
6220992220,1cyclictest20408-21idleruntime-cro09:50:003
6220992220,1cyclictest0-21swapper/308:14:173
622099220,21cyclictest11010-21sendmail-msp07:20:003
622099220,21cyclictest10810-21munin-run08:25:013
622099220,21cyclictest10810-21munin-run08:25:013
622099220,1cyclictest0-21swapper/310:35:003
6216992220,1cyclictest16424-21apt-get07:34:162
6216992220,1cyclictest0-21swapper/208:10:002
6216992219,2cyclictest1-21systemd11:25:002
621699221,1cyclictest25878-21apt-get10:04:172
621699221,1cyclictest25878-21apt-get10:04:172
621699220,21cyclictest11329-21idleruntime-cro11:45:012
621699220,21cyclictest0-21swapper/211:14:172
621699220,21cyclictest0-21swapper/209:44:162
621699220,21cyclictest0-21swapper/209:09:172
621699220,21cyclictest0-21swapper/207:59:172
621699220,1cyclictest31387-21apt-get08:04:142
621699220,1cyclictest28272-21apt-get10:09:182
621699220,1cyclictest271rcuc/210:54:162
6214992220,1cyclictest0-21swapper/112:35:011
6214992220,1cyclictest0-21swapper/111:55:011
6214992219,2cyclictest13218-21latency_hist09:35:001
6214992218,3cyclictest2492-21basename10:20:011
621499220,21cyclictest31189-21apt-get12:29:171
6210992220,1cyclictest7653-21apt-get09:25:010
6210992220,1cyclictest4835-21tr09:20:010
6210992220,1cyclictest16901-21perf10:50:010
621099221,1cyclictest0-21swapper/008:40:010
621099220,21cyclictest8403-21munin-run08:20:010
621099220,21cyclictest30636-21apt-get09:09:160
621099220,21cyclictest0-21swapper/012:39:160
621099220,21cyclictest0-21swapper/011:04:160
621099220,21cyclictest0-21swapper/009:04:160
621099220,21cyclictest0-21swapper/007:24:160
621099220,1cyclictest26380-21apt-get12:19:170
621099220,1cyclictest16424-21apt-get07:34:160
6220992120,0cyclictest23449-21apt-get08:54:273
6220992120,0cyclictest0-21swapper/311:24:163
6220992119,1cyclictest27979-21ls07:55:003
6220992119,1cyclictest16222-21apt-get09:44:163
622099210,21cyclictest32451-21perf09:10:013
622099210,21cyclictest0-21swapper/310:24:173
622099210,21cyclictest0-21swapper/309:29:173
622099210,20cyclictest6774-21apt-get07:14:163
622099210,20cyclictest0-21swapper/310:44:153
622099210,20cyclictest0-21swapper/308:49:143
622099210,1cyclictest16902-21munin-run10:50:013
622099210,1cyclictest0-21swapper/312:24:193
622099210,0cyclictest0-21swapper/312:29:173
6216992121,0cyclictest13781-21apt-get08:34:172
6216992121,0cyclictest0-21swapper/212:14:162
6216992119,1cyclictest28-21ksoftirqd/210:50:012
6216992119,1cyclictest28-21ksoftirqd/207:55:002
621699210,20cyclictest0-21swapper/212:29:172
621699210,20cyclictest0-21swapper/209:49:172
621699210,1cyclictest8929-21perf11:40:012
621699210,1cyclictest313-21systemd-journal09:25:012
621699210,1cyclictest28797-21apt-get12:24:182
6214992121,0cyclictest9170-21apt-get07:19:171
6214992121,0cyclictest0-21swapper/111:04:161
6214992120,0cyclictest21395-21apt-get07:44:151
6214992120,0cyclictest19868-21apt-get10:59:161
6214992120,0cyclictest18922-21apt-get07:39:131
6214992120,0cyclictest0-21swapper/108:09:171
6214992119,1cyclictest26543-21latency_hist11:10:011
621499210,21cyclictest0-21swapper/109:25:011
621499210,20cyclictest23461-21apt-get09:59:171
621499210,20cyclictest0-21swapper/110:49:161
621499210,20cyclictest0-21swapper/110:09:181
621499210,20cyclictest0-21swapper/107:29:151
621499210,0cyclictest0-21swapper/110:44:161
621499210,0cyclictest0-21swapper/110:34:171
621499210,0cyclictest0-21swapper/108:15:021
6210992121,0cyclictest9005-21apt-get08:24:270
6210992121,0cyclictest9005-21apt-get08:24:270
6210992121,0cyclictest25878-21apt-get10:04:160
6210992121,0cyclictest25878-21apt-get10:04:160
6210992121,0cyclictest1923-21apt-get11:29:160
621099211,20cyclictest4081-21kworker/0:211:50:070
621099210,20cyclictest0-21swapper/012:24:190
621099210,0cyclictest31936-21apt-get11:24:160
621099210,0cyclictest0-21swapper/011:39:170
621099210,0cyclictest0-21swapper/009:39:190
6220992020,0cyclictest21068-21apt-get09:54:163
6220992020,0cyclictest16724-21apt-get11:59:163
6220992020,0cyclictest0-21swapper/312:19:173
6220992020,0cyclictest0-21swapper/311:35:023
6220992020,0cyclictest0-21swapper/310:14:173
6220992020,0cyclictest0-21swapper/310:04:173
6220992020,0cyclictest0-21swapper/310:04:173
6220992019,0cyclictest0-21swapper/310:59:163
6220992019,0cyclictest0-21swapper/307:44:153
6220992018,1cyclictest7296-21munin-run10:30:003
6220992018,1cyclictest0-21swapper/310:55:013
6220992017,2cyclictest18225-21cron07:35:013
622099200,20cyclictest0-21swapper/308:15:023
622099200,19cyclictest0-21swapper/307:49:153
6216992020,0cyclictest573-21apt-get09:14:152
6216992020,0cyclictest3070-21apt-get10:24:172
6216992020,0cyclictest0-21swapper/211:59:162
6216992020,0cyclictest0-21swapper/211:54:162
6216992020,0cyclictest0-21swapper/209:54:162
6216992020,0cyclictest0-21swapper/209:19:172
6216992020,0cyclictest0-21swapper/209:03:542
6216992020,0cyclictest0-21swapper/208:59:162
6216992019,1cyclictest0-21swapper/211:19:172
6216992018,1cyclictest2495-21munin-run10:20:012
621699200,20cyclictest7653-21apt-get09:25:022
621699200,20cyclictest0-21swapper/211:29:162
6214992020,0cyclictest18589-21apt-get08:44:151
6214992020,0cyclictest11934-21apt-get11:49:171
6214992020,0cyclictest11633-21apt-get07:24:161
6214992020,0cyclictest0-21swapper/111:44:171
6214992020,0cyclictest0-21swapper/111:34:171
6214992020,0cyclictest0-21swapper/109:29:161
6214992020,0cyclictest0-21swapper/108:24:271
6214992020,0cyclictest0-21swapper/108:24:271
6214992020,0cyclictest0-21swapper/108:04:151
6214992019,1cyclictest13189-21perf08:30:011
6214992018,1cyclictest606-21dbus-daemon08:55:001
621499200,0cyclictest3772-21apt-get08:14:161
6210992020,0cyclictest23982-21apt-get12:14:160
6210992020,0cyclictest0-21swapper/012:29:170
6210992020,0cyclictest0-21swapper/011:14:170
6210992020,0cyclictest0-21swapper/010:34:170
6210992020,0cyclictest0-21swapper/010:29:160
6210992020,0cyclictest0-21swapper/009:14:150
6210992020,0cyclictest0-21swapper/008:49:150
6210992019,1cyclictest62550irq/127-eno110:59:160
6210992019,0cyclictest21589-21apt-get12:09:180
6210992019,0cyclictest0-21swapper/007:14:160
6210992018,1cyclictest30368-21sendmail08:00:000
621099200,1cyclictest0-21swapper/011:44:190
621099200,0cyclictest615-21apt-get10:19:180
621099200,0cyclictest0-21swapper/008:09:160
621099200,0cyclictest0-21swapper/007:19:170
6220991919,0cyclictest0-21swapper/311:44:163
6220991919,0cyclictest0-21swapper/311:19:173
6220991919,0cyclictest0-21swapper/309:59:173
6220991919,0cyclictest0-21swapper/308:34:163
6220991918,1cyclictest11934-21apt-get11:49:173
6216991918,1cyclictest0-21swapper/211:04:162
6214991919,0cyclictest3022-21apt-get09:19:171
6214991919,0cyclictest29523-21apt-get11:19:171
6214991919,0cyclictest0-21swapper/112:39:161
6214991919,0cyclictest0-21swapper/110:05:011
6214991919,0cyclictest0-21swapper/110:05:011
6214991918,1cyclictest606-21dbus-daemon11:40:011
6214991917,1cyclictest606-21dbus-daemon09:45:001
6210991919,0cyclictest0-21swapper/009:54:150
6210991919,0cyclictest0-21swapper/009:44:160
6210991919,0cyclictest0-21swapper/009:29:170
6210991918,1cyclictest10432-21apt-get09:30:020
6210991917,1cyclictest25300-21cut10:00:010
6210991917,1cyclictest0-21swapper/008:34:160
6220991818,0cyclictest10263-21apt-get10:39:153
6220991818,0cyclictest0-21swapper/311:34:163
6220991817,1cyclictest0-21swapper/311:29:163
6220991817,1cyclictest0-21swapper/307:29:163
622099180,17cyclictest607-21in:imuxsock09:00:003
6216991818,0cyclictest0-21swapper/210:44:152
6216991818,0cyclictest0-21swapper/210:39:152
6216991818,0cyclictest0-21swapper/208:54:262
6216991818,0cyclictest0-21swapper/207:39:132
6214991818,0cyclictest0-21swapper/108:34:171
621499180,17cyclictest0-21swapper/112:05:001
6210991818,0cyclictest0-21swapper/010:24:170
6210991818,0cyclictest0-21swapper/010:14:180
6210991818,0cyclictest0-21swapper/009:49:170
6210991818,0cyclictest0-21swapper/008:54:260
6210991817,0cyclictest17474-21apt-get10:54:160
6210991816,1cyclictest113882sleep008:29:180
6210991816,1cyclictest0-21swapper/008:59:160
6220991717,0cyclictest0-21swapper/312:14:163
6220991717,0cyclictest0-21swapper/308:09:163
6220991717,0cyclictest0-21swapper/307:59:163
6220991716,0cyclictest0-21swapper/311:04:163
6220991715,1cyclictest0-21swapper/307:40:013
6216991717,0cyclictest0-21swapper/211:49:172
6216991717,0cyclictest0-21swapper/207:24:162
6216991716,0cyclictest0-21swapper/208:44:162
6214991717,0cyclictest0-21swapper/109:14:151
6214991717,0cyclictest0-21swapper/108:49:151
6214991717,0cyclictest0-21swapper/107:14:161
6214991715,1cyclictest32421-21cron09:10:011
6214991715,1cyclictest23788-21apt-get07:49:151
6214991715,1cyclictest1328-21turbostat.cron11:25:001
6210991717,0cyclictest24733-21apt-get11:09:170
6210991717,0cyclictest0-21swapper/008:04:140
6210991717,0cyclictest0-21swapper/007:49:150
6210991717,0cyclictest0-21swapper/007:29:160
621099170,1cyclictest12683-21apt-get10:44:150
6220991616,0cyclictest0-21swapper/312:34:203
6220991616,0cyclictest0-21swapper/308:44:153
6220991616,0cyclictest0-21swapper/308:04:143
6220991612,3cyclictest774-21runrttasks08:39:163
621699167,8cyclictest28-21ksoftirqd/207:14:192
6216991616,0cyclictest0-21swapper/208:49:152
6216991616,0cyclictest0-21swapper/207:49:152
6216991615,1cyclictest5391-21latency_hist12:40:012
6216991613,2cyclictest7292-21perf10:30:002
6214991616,0cyclictest0-21swapper/107:59:161
6214991614,1cyclictest0-21swapper/109:00:001
621499160,15cyclictest17846-21grep11:59:221
621099162,0cyclictest62550irq/127-eno108:43:380
6210991616,0cyclictest26184-21apt-get07:54:150
6210991616,0cyclictest0-21swapper/012:34:200
6210991616,0cyclictest0-21swapper/011:59:160
6210991616,0cyclictest0-21swapper/011:19:170
6210991616,0cyclictest0-21swapper/010:39:150
6210991614,1cyclictest0-21swapper/008:15:010
622099150,5cyclictest0-21swapper/309:34:163
622099150,14cyclictest0-21swapper/312:09:233
622099150,14cyclictest0-21swapper/308:29:193
6216991515,0cyclictest0-21swapper/210:15:002
6216991515,0cyclictest0-21swapper/209:39:182
6216991514,1cyclictest11384-21apt-get08:29:172
6214991515,0cyclictest0-21swapper/109:54:161
621499150,1cyclictest0-21swapper/112:25:001
621499150,1cyclictest0-21swapper/112:10:531
6210991515,0cyclictest0-21swapper/011:49:170
6220991414,0cyclictest0-21swapper/309:00:363
622099140,14cyclictest0-21swapper/311:11:173
622099140,14cyclictest0-21swapper/310:15:073
622099140,14cyclictest0-21swapper/310:05:553
622099140,14cyclictest0-21swapper/309:37:223
622099140,14cyclictest0-21swapper/307:20:353
622099140,0cyclictest0-21swapper/309:11:303
6216991414,0cyclictest0-21swapper/211:05:192
6216991414,0cyclictest0-21swapper/209:55:212
621699140,14cyclictest0-21swapper/212:30:132
621699140,14cyclictest0-21swapper/212:15:262
621699140,14cyclictest0-21swapper/211:30:132
621699140,14cyclictest0-21swapper/210:56:392
621699140,14cyclictest0-21swapper/210:30:482
621699140,14cyclictest0-21swapper/208:20:152
621699140,14cyclictest0-21swapper/208:20:152
621699140,14cyclictest0-21swapper/207:15:522
621699140,0cyclictest0-21swapper/212:05:232
6214991414,0cyclictest0-21swapper/110:35:071
6214991414,0cyclictest0-21swapper/109:45:081
6214991414,0cyclictest0-21swapper/107:30:481
621499140,14cyclictest0-21swapper/110:26:471
621499140,0cyclictest0-21swapper/112:05:171
621499140,0cyclictest0-21swapper/108:35:121
621099140,14cyclictest0-21swapper/012:00:320
621099140,14cyclictest0-21swapper/011:30:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional