You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-24 - 09:22

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Wed Dec 24, 2025 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2823999240,22cyclictest313-21systemd-journal23:30:013
28233992423,1cyclictest0-21swapper/200:25:102
28239992322,0cyclictest0-21swapper/319:50:093
2823399230,22cyclictest8065-21apt-get19:35:112
2823399230,21cyclictest313-21systemd-journal23:20:012
28230992321,1cyclictest21915-21munin-run21:10:011
2823099230,1cyclictest3941-21sh22:45:011
28239992221,1cyclictest7331-21apt-get20:40:073
28239992220,1cyclictest0-21swapper/300:10:113
2823999220,21cyclictest355-21turbostat23:45:003
2823999220,21cyclictest29137-21munin-run21:25:003
2823999220,21cyclictest28011-21grep23:35:013
2823999220,21cyclictest19776-21apt-get21:05:093
2823999220,21cyclictest0-21swapper/319:35:103
28233992221,1cyclictest0-21swapper/220:20:102
28233992220,1cyclictest0-21swapper/223:50:002
2823399220,21cyclictest16218-21apt-get23:10:082
2823399220,21cyclictest11518-21munin-run21:55:012
2823399220,21cyclictest0-21swapper/222:35:092
2823399220,21cyclictest0-21swapper/222:25:102
2823399220,21cyclictest0-21swapper/221:25:002
2823399220,21cyclictest0-21swapper/220:15:102
28230992222,0cyclictest31785-21apt-get21:30:091
28230992222,0cyclictest0-21swapper/120:10:101
28230992220,1cyclictest1562-21sendmail-msp22:40:011
28230992220,1cyclictest0-21swapper/120:05:011
2823099220,21cyclictest22528-21apt-get20:05:111
2823099220,21cyclictest0-21swapper/119:20:011
2823099220,21cyclictest0-21swapper/100:35:081
2823099220,1cyclictest606-21dbus-daemon22:35:011
28228992221,1cyclictest7980-21apt-get00:00:090
28228992221,1cyclictest0-21swapper/020:50:020
28228992221,0cyclictest11773-21apt-get21:55:120
28228992220,2cyclictest13-21ksoftirqd/020:40:010
28228992220,1cyclictest17986-21apt-get00:20:080
2822899220,2cyclictest650-21apt23:45:100
2822899220,21cyclictest25585-21cron23:30:010
2822899220,21cyclictest0-21swapper/000:15:090
2822899220,1cyclictest5185-21cron23:55:000
2822899220,1cyclictest0-21swapper/020:05:010
28239992121,0cyclictest11843-21apt-get20:50:023
28239992121,0cyclictest0-21swapper/321:55:123
28239992121,0cyclictest0-21swapper/300:20:083
28239992120,0cyclictest0-21swapper/323:55:103
28239992119,1cyclictest24642-21sh20:10:013
28239992119,1cyclictest1830-21apt-get22:40:093
28239992119,1cyclictest0-21swapper/323:15:013
2823999210,21cyclictest0-21swapper/300:00:093
2823999210,20cyclictest0-21swapper/322:25:103
2823999210,20cyclictest0-21swapper/320:45:143
2823999210,1cyclictest0-21swapper/323:55:003
2823999210,0cyclictest15557-21apt-get00:15:093
28233992121,0cyclictest0-21swapper/220:00:012
28233992120,1cyclictest21897-21dump-pmu-power21:10:012
28233992120,0cyclictest5475-21apt-get23:55:112
28233992120,0cyclictest24628-21apt-get22:20:112
28233992120,0cyclictest0-21swapper/222:40:092
28233992120,0cyclictest0-21swapper/200:35:082
28233992119,1cyclictest8741-21turbostat.cron22:55:002
28233992119,1cyclictest24272-21cron21:15:002
28233992119,1cyclictest21538-21awk22:15:002
2823399210,20cyclictest0-21swapper/220:05:112
2823399210,20cyclictest0-21swapper/200:20:092
28230992121,0cyclictest13823-21apt-get23:05:091
28230992121,0cyclictest0-21swapper/123:45:101
28230992120,1cyclictest20348-21/usr/sbin/munin00:25:111
28230992120,1cyclictest0-21swapper/120:30:091
28230992120,0cyclictest17071-21munin-run21:00:011
28230992120,0cyclictest15557-21apt-get00:15:101
28230992120,0cyclictest0-21swapper/121:40:101
28230992119,1cyclictest606-21dbus-daemon22:50:011
2823099210,21cyclictest0-21swapper/123:40:121
2823099210,21cyclictest0-21swapper/123:30:121
2823099210,0cyclictest0-21swapper/119:25:081
28228992121,0cyclictest0-21swapper/019:50:090
28228992120,0cyclictest5475-21apt-get23:55:100
28228992120,0cyclictest27020-21apt-get22:25:100
28228992120,0cyclictest1487-21apt-get21:35:020
28228992119,1cyclictest21552-21apt-get22:15:020
28228992119,1cyclictest0-21swapper/023:15:010
2822899210,21cyclictest28278-21apt-get23:35:100
2822899210,21cyclictest0-21swapper/020:55:100
2822899210,20cyclictest22773-21apt-get00:30:090
2822899210,20cyclictest0-21swapper/023:15:100
2822899210,1cyclictest62550irq/127-eno119:30:140
2822899210,0cyclictest0-21swapper/023:20:110
2822899210,0cyclictest0-21swapper/021:15:100
2822899210,0cyclictest0-21swapper/021:05:090
28239992020,0cyclictest0-21swapper/322:15:023
28239992020,0cyclictest0-21swapper/322:00:103
28239992020,0cyclictest0-21swapper/321:35:023
28239992020,0cyclictest0-21swapper/321:15:113
28239992020,0cyclictest0-21swapper/320:35:103
28239992020,0cyclictest0-21swapper/319:30:153
28239992019,1cyclictest13573-21munin-run23:05:003
28239992019,1cyclictest0-21swapper/322:45:133
28239992019,0cyclictest0-21swapper/319:45:103
28239992018,1cyclictest13823-21apt-get23:05:103
2823999200,0cyclictest6619-21apt-get22:50:093
2823999200,0cyclictest0-21swapper/323:35:103
2823999200,0cyclictest0-21swapper/323:20:113
2823999200,0cyclictest0-21swapper/320:55:103
2823999200,0cyclictest0-21swapper/300:30:093
28233992020,0cyclictest4587-21apt-get21:40:102
28233992020,0cyclictest3081-21apt-get23:50:122
28233992020,0cyclictest14914-21apt-get20:55:102
28233992020,0cyclictest13165-21apt-get00:10:112
28233992020,0cyclictest0-21swapper/222:30:092
28233992020,0cyclictest0-21swapper/221:15:112
28233992019,1cyclictest24924-21apt-get20:10:102
28233992019,0cyclictest0-21swapper/223:35:102
28233992018,1cyclictest30610-21turbostat19:15:012
2823399200,0cyclictest10374-21apt-get00:05:102
2823399200,0cyclictest0-21swapper/223:40:122
2823399200,0cyclictest0-21swapper/223:20:112
28230992020,0cyclictest17329-21apt-get21:00:091
28230992020,0cyclictest0-21swapper/123:55:111
28230992020,0cyclictest0-21swapper/123:50:121
28230992020,0cyclictest0-21swapper/121:20:101
28230992020,0cyclictest0-21swapper/121:15:111
28230992020,0cyclictest0-21swapper/119:10:111
28230992018,1cyclictest0-21swapper/122:10:001
28230992017,2cyclictest13554-21awk23:05:001
2823099200,19cyclictest19424-21apt-get22:10:101
2823099200,0cyclictest0-21swapper/121:50:121
2823099200,0cyclictest0-21swapper/119:30:141
28228992020,0cyclictest9732-21apt-get20:45:130
28228992020,0cyclictest0-21swapper/022:00:100
28228992020,0cyclictest0-21swapper/019:40:090
28228992020,0cyclictest0-21swapper/019:10:110
28228992020,0cyclictest0-21swapper/000:10:110
2822899200,20cyclictest0-21swapper/022:40:090
2822899200,20cyclictest0-21swapper/022:10:100
2822899200,20cyclictest0-21swapper/021:25:000
2822899200,0cyclictest0-21swapper/022:50:090
2822899200,0cyclictest0-21swapper/021:30:090
2822899200,0cyclictest0-21swapper/019:55:120
2822899200,0cyclictest0-21swapper/000:35:070
28239991919,0cyclictest0-21swapper/319:55:123
28239991919,0cyclictest0-21swapper/319:40:093
28239991919,0cyclictest0-21swapper/319:10:113
28239991917,1cyclictest0-21swapper/321:05:013
2823999190,0cyclictest0-21swapper/322:10:103
28233991919,0cyclictest23480-21apt-get23:25:112
28233991919,0cyclictest0-21swapper/222:45:132
28233991919,0cyclictest0-21swapper/220:50:022
28233991919,0cyclictest0-21swapper/220:30:092
28233991919,0cyclictest0-21swapper/219:20:082
28233991918,1cyclictest0-21swapper/221:25:082
28233991918,1cyclictest0-21swapper/220:40:022
28233991918,0cyclictest0-21swapper/200:30:092
28233991917,1cyclictest24368-21munin-run22:20:002
28230991919,0cyclictest0-21swapper/122:00:101
28230991919,0cyclictest0-21swapper/121:25:091
28230991919,0cyclictest0-21swapper/120:25:121
28230991919,0cyclictest0-21swapper/120:15:101
28230991919,0cyclictest0-21swapper/119:50:091
28230991919,0cyclictest0-21swapper/119:35:111
28230991919,0cyclictest0-21swapper/100:30:091
28230991919,0cyclictest0-21swapper/100:10:111
28230991918,1cyclictest6981-21apt-get21:45:091
28230991918,1cyclictest17668-21apt-get19:55:131
28230991918,1cyclictest0-21swapper/121:55:121
28230991917,1cyclictest313-21systemd-journal23:30:011
2823099190,19cyclictest0-21swapper/120:40:021
28228991919,0cyclictest0-21swapper/020:05:110
28228991918,0cyclictest12875-21apt-get19:45:100
28228991917,1cyclictest27318-21apt-get20:15:100
2822899190,0cyclictest0-21swapper/023:05:100
28239991818,0cyclictest0-21swapper/321:40:103
28239991818,0cyclictest0-21swapper/321:25:093
28239991818,0cyclictest0-21swapper/320:15:103
28239991817,0cyclictest32140-21apt-get20:25:123
28239991816,1cyclictest12511-21turbostat00:10:013
2823999181,16cyclictest0-21swapper/319:20:013
28233991818,0cyclictest11773-21apt-get21:55:122
28233991818,0cyclictest0-21swapper/223:05:092
28233991818,0cyclictest0-21swapper/221:30:092
28233991818,0cyclictest0-21swapper/220:25:112
28233991816,1cyclictest0-21swapper/223:05:002
28233991816,1cyclictest0-21swapper/200:05:012
28230991818,0cyclictest863-21apt-get19:20:081
28228991818,0cyclictest4587-21apt-get21:40:100
28228991818,0cyclictest0-21swapper/023:05:000
28228991818,0cyclictest0-21swapper/022:45:130
28228991818,0cyclictest0-21swapper/021:25:090
28228991817,1cyclictest29745-21apt-get20:20:100
28239991717,0cyclictest0-21swapper/320:20:103
28239991716,1cyclictest651-21apt-get23:45:103
28239991716,0cyclictest0-21swapper/321:50:123
28239991715,1cyclictest5374-21cron19:30:013
28233991717,0cyclictest0-21swapper/223:30:122
28230991717,0cyclictest0-21swapper/123:35:101
28230991717,0cyclictest0-21swapper/123:15:101
28230991717,0cyclictest0-21swapper/122:15:101
28230991717,0cyclictest0-21swapper/121:10:091
28230991717,0cyclictest0-21swapper/120:20:101
28230991717,0cyclictest0-21swapper/100:20:081
28228991716,0cyclictest0-21swapper/020:25:120
28228991715,1cyclictest0-21swapper/019:30:010
2822899170,16cyclictest515-21cron19:20:010
28239991616,0cyclictest0-21swapper/323:15:103
28239991614,1cyclictest0-21swapper/320:05:013
2823999160,15cyclictest26757-21perf22:25:013
28233991616,0cyclictest0-21swapper/219:45:112
28233991616,0cyclictest0-21swapper/219:25:082
28233991614,1cyclictest7809-21munin-run19:35:002
2823399160,15cyclictest4522-21apt-get20:35:102
28230991616,0cyclictest10481-21apt-get19:40:091
28230991616,0cyclictest0-21swapper/122:20:111
28230991615,1cyclictest2168-21apt-get21:35:101
28230991614,1cyclictest0-21swapper/122:30:011
2823099160,15cyclictest4736-21cstates20:35:111
28228991616,0cyclictest9377-21apt-get21:50:120
28228991616,0cyclictest0-21swapper/021:05:010
28228991615,1cyclictest12515-21grep00:10:010
28228991614,1cyclictest10202-21dump-pmu-power19:40:000
2822899160,15cyclictest26769-21munin-plugin-st22:25:010
28239991515,0cyclictest0-21swapper/322:35:093
28239991515,0cyclictest0-21swapper/322:30:103
28239991514,1cyclictest2125-21apt-get20:30:093
28239991514,1cyclictest0-21swapper/319:20:083
2823999150,5cyclictest0-21swapper/322:55:113
2823999150,14cyclictest0-21swapper/321:10:173
2823999150,14cyclictest0-21swapper/300:25:183
28233991515,0cyclictest14187-21apt-get22:00:102
28233991513,1cyclictest17710-21latency_hist00:20:012
2823399150,15cyclictest19514-21perf21:05:012
2823399150,14cyclictest0-21swapper/222:55:562
28230991515,0cyclictest12517-21apt-get20:50:091
2823099150,1cyclictest11604-21chrt20:47:121
2823099150,15cyclictest0-21swapper/122:55:111
28228991514,0cyclictest31820-21apt-get22:35:090
28228991514,0cyclictest0-21swapper/022:55:110
2823999140,14cyclictest0-21swapper/322:06:053
2823999140,14cyclictest0-21swapper/321:31:143
2823999140,14cyclictest0-21swapper/320:10:343
2823999140,14cyclictest0-21swapper/300:35:593
2823999140,0cyclictest0-21swapper/321:45:243
28233991414,0cyclictest0-21swapper/220:45:142
2823399140,14cyclictest0-21swapper/222:05:072
2823399140,14cyclictest0-21swapper/221:35:182
2823399140,14cyclictest0-21swapper/219:50:532
2823399140,14cyclictest0-21swapper/219:41:172
2823399140,0cyclictest0-21swapper/221:45:302
2823399140,0cyclictest0-21swapper/220:00:032
2823399140,0cyclictest0-21swapper/219:15:062
28230991414,0cyclictest0-21swapper/123:10:051
28230991414,0cyclictest0-21swapper/100:05:101
2823099140,14cyclictest0-21swapper/122:52:321
28228991414,0cyclictest29425-21apt-get22:30:100
28228991414,0cyclictest0-21swapper/023:30:110
28228991414,0cyclictest0-21swapper/022:06:050
28228991414,0cyclictest0-21swapper/020:10:340
28228991414,0cyclictest0-21swapper/019:22:120
28228991413,0cyclictest0-21swapper/020:30:090
2822899140,6cyclictest0-21swapper/021:10:090
2822899140,14cyclictest0-21swapper/023:40:030
2822899140,14cyclictest0-21swapper/021:45:420
2822899140,14cyclictest0-21swapper/000:25:180
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional