You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-03 - 07:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Sat Jan 03, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3068599240,23cyclictest0-21swapper/319:56:023
30680992422,1cyclictest23947-21dump-pmu-power21:10:002
30676992421,1cyclictest0-21swapper/123:21:021
3067699240,2cyclictest313-21systemd-journal23:20:011
3067699240,23cyclictest20224-21apt-get19:56:021
30685992322,0cyclictest0-21swapper/321:41:003
30685992320,2cyclictest19157-21basename21:00:013
30685992316,6cyclictest20513-21idleruntime-cro00:20:003
3068599230,23cyclictest0-21swapper/319:26:003
30680992322,0cyclictest7851-21munin-run19:30:002
3067699230,22cyclictest23271-21apt-get00:26:011
3067699230,22cyclictest16389-21apt-get23:06:021
3067699230,0cyclictest0-21swapper/122:51:011
30671992323,0cyclictest0-21swapper/021:40:020
30671992321,1cyclictest0-21swapper/022:25:020
3067199230,22cyclictest3980-21sh22:40:010
30685992222,0cyclictest3385-21apt-get19:21:013
30685992220,1cyclictest18782-21apt23:11:023
30685992220,1cyclictest13347-21perf00:05:013
3068599220,21cyclictest9130-21apt-get22:51:013
3068599220,21cyclictest29888-21apt-get21:21:013
3068599220,21cyclictest26753-21latency_hist21:15:003
3068599220,21cyclictest0-21swapper/322:11:003
3068599220,1cyclictest29172-21apt-get23:31:013
30680992220,1cyclictest31612-21munin-run22:30:012
30680992220,1cyclictest26743-21apt-get22:21:012
30680992220,1cyclictest10855-21grep00:00:012
3068099221,0cyclictest18783-21apt-get23:11:012
3068099220,21cyclictest26748-21perf21:15:002
3068099220,21cyclictest25332-21turbostat00:30:002
3068099220,21cyclictest13592-21turbostat23:00:002
3068099220,21cyclictest0-21swapper/223:51:012
3068099220,21cyclictest0-21swapper/223:50:002
3068099220,21cyclictest0-21swapper/223:16:012
3068099220,21cyclictest0-21swapper/200:06:022
3068099220,1cyclictest2232-21apt-get20:25:592
30676992221,0cyclictest28813-21apt-get22:25:021
30676992220,1cyclictest23929-21sh21:10:001
3067699220,21cyclictest9093-21apt-get20:40:021
3067699220,21cyclictest30980-21apt-get19:10:591
3067699220,21cyclictest0-21swapper/121:05:001
3067699220,21cyclictest0-21swapper/119:36:021
3067699220,1cyclictest6375-21apt-get23:51:011
3067699220,1cyclictest28840-21grep23:30:001
3067699220,0cyclictest0-21swapper/121:25:591
30671992221,0cyclictest11525-21apt-get22:56:000
30671992221,0cyclictest0-21swapper/023:56:030
3067199220,1cyclictest25035-21apt-get20:06:010
3067199220,1cyclictest1953-21perf21:30:010
3067199220,1cyclictest0-21swapper/020:20:000
30685992119,1cyclictest16815-21awk22:00:003
3068599210,20cyclictest20880-21apt-get00:21:013
3068599210,20cyclictest0-21swapper/323:06:033
3068599210,1cyclictest0-21swapper/300:40:013
30680992121,0cyclictest0-21swapper/221:36:002
30680992120,0cyclictest19534-21apt-get22:06:022
30680992118,2cyclictest18102-21cron00:15:012
3068099210,20cyclictest0-21swapper/221:41:002
3068099210,20cyclictest0-21swapper/221:31:012
3068099210,0cyclictest0-21swapper/222:31:002
3068099210,0cyclictest0-21swapper/219:10:592
3068099210,0cyclictest0-21swapper/200:01:022
30676992121,0cyclictest8773-21apt-get23:56:031
30676992121,0cyclictest0-21swapper/121:40:021
30676992121,0cyclictest0-21swapper/120:25:591
30676992120,1cyclictest22644-21apt-get20:01:031
30676992119,1cyclictest0-21swapper/122:05:001
3067699210,21cyclictest15379-21apt-get19:46:011
3067699210,20cyclictest17828-21apt-get19:51:011
3067699210,1cyclictest1598-21munin-run22:35:011
3067699210,0cyclictest28058-21apt-get00:36:011
3067699210,0cyclictest0-21swapper/122:56:001
3067699210,0cyclictest0-21swapper/122:46:001
3067699210,0cyclictest0-21swapper/120:21:031
30671992121,0cyclictest0-21swapper/020:01:030
30671992121,0cyclictest0-21swapper/000:21:010
30671992121,0cyclictest0-21swapper/000:11:000
30671992120,0cyclictest0-21swapper/023:16:010
3067199210,2cyclictest17042-21/usr/sbin/munin20:56:070
3067199210,21cyclictest0-21swapper/022:51:010
3067199210,20cyclictest31933-21apt-get22:31:000
3067199210,20cyclictest0-21swapper/022:06:020
3067199210,20cyclictest0-21swapper/019:31:030
3067199210,0cyclictest0-21swapper/023:01:020
30685992020,0cyclictest6734-21apt-get22:46:013
30685992020,0cyclictest27073-21apt-get21:16:003
30685992020,0cyclictest2279-21apt-get21:31:013
30685992020,0cyclictest0-21swapper/322:06:023
30685992020,0cyclictest0-21swapper/320:41:013
30685992020,0cyclictest0-21swapper/320:06:013
30685992020,0cyclictest0-21swapper/319:31:033
30685992019,1cyclictest16068-21apt-get00:11:003
30685992019,0cyclictest12985-21apt-get19:41:003
30685992019,0cyclictest0-21swapper/323:41:003
30685992019,0cyclictest0-21swapper/322:56:003
30685992018,1cyclictest23990-21apt-get23:21:023
30685992017,2cyclictest4008-21apt22:40:013
3068599200,3cyclictest0-21swapper/319:10:593
3068599200,1cyclictest31574-21apt-get23:36:013
3068599200,0cyclictest4630-21apt-get20:31:013
3068599200,0cyclictest0-21swapper/322:31:003
3068599200,0cyclictest0-21swapper/320:46:023
3068599200,0cyclictest0-21swapper/319:51:013
3068599200,0cyclictest0-21swapper/300:31:003
30680992020,0cyclictest12234-21apt-get20:46:022
30680992020,0cyclictest0-21swapper/221:16:002
30680992020,0cyclictest0-21swapper/220:41:012
30680992019,1cyclictest17138-21apt-get22:01:002
30680992019,0cyclictest952-21apt-get19:16:012
30680992019,0cyclictest0-21swapper/222:36:002
30680992018,1cyclictest19880-21awk19:55:012
3068099200,20cyclictest0-21swapper/221:55:582
3068099200,20cyclictest0-21swapper/219:46:012
3068099200,0cyclictest0-21swapper/223:31:022
3068099200,0cyclictest0-21swapper/220:56:002
3068099200,0cyclictest0-21swapper/219:56:022
30676992020,0cyclictest4674-21apt-get21:36:011
30676992020,0cyclictest0-21swapper/121:21:011
30676992020,0cyclictest0-21swapper/121:16:001
30676992020,0cyclictest0-21swapper/120:36:011
30676992020,0cyclictest0-21swapper/120:11:021
30676992020,0cyclictest0-21swapper/119:45:011
3067699200,19cyclictest0-21swapper/123:36:011
3067699200,0cyclictest0-21swapper/122:16:001
3067699200,0cyclictest0-21swapper/120:31:011
30671992020,0cyclictest5777-21apt-get19:26:000
30671992020,0cyclictest0-21swapper/023:36:010
30671992020,0cyclictest0-21swapper/022:21:010
30671992020,0cyclictest0-21swapper/019:21:000
30671992019,0cyclictest0-21swapper/023:41:000
30671992019,0cyclictest0-21swapper/019:46:010
30671992018,1cyclictest31244-21perf23:35:000
30671992018,1cyclictest26773-21apt-get23:26:000
3067199200,0cyclictest0-21swapper/022:11:010
3067199200,0cyclictest0-21swapper/019:51:010
30685991919,0cyclictest0-21swapper/321:51:023
30685991919,0cyclictest0-21swapper/321:46:003
30685991919,0cyclictest0-21swapper/320:11:023
30685991919,0cyclictest0-21swapper/320:01:043
30685991919,0cyclictest0-21swapper/319:46:013
30680991919,0cyclictest14629-21apt-get20:51:032
30680991919,0cyclictest0-21swapper/222:51:012
30680991919,0cyclictest0-21swapper/219:21:012
30680991918,1cyclictest7028-21apt-get20:36:022
30680991918,1cyclictest32252-21apt-get20:21:032
30680991918,1cyclictest29466-21cron20:15:002
30680991918,1cyclictest0-21swapper/219:41:002
30680991917,1cyclictest0-21swapper/200:35:002
3068099190,0cyclictest0-21swapper/220:16:022
30676991919,0cyclictest1922-21apt-get22:36:001
30676991919,0cyclictest0-21swapper/123:11:021
30676991919,0cyclictest0-21swapper/121:51:021
30671991919,0cyclictest0-21swapper/021:21:010
30671991919,0cyclictest0-21swapper/019:41:000
30671991918,0cyclictest0-21swapper/019:10:590
30671991917,1cyclictest16799-21sendmail-msp22:00:000
30685991818,0cyclictest19493-21apt-get21:01:003
30685991818,0cyclictest0-21swapper/322:25:023
30685991818,0cyclictest0-21swapper/320:25:593
30685991818,0cyclictest0-21swapper/320:21:023
3068599180,1cyclictest0-21swapper/321:40:013
30680991818,0cyclictest0-21swapper/223:20:022
30680991818,0cyclictest0-21swapper/221:21:012
30680991818,0cyclictest0-21swapper/219:36:032
30680991818,0cyclictest0-21swapper/200:36:012
30680991817,1cyclictest0-21swapper/200:21:012
30680991817,0cyclictest8177-21apt-get19:31:032
30680991816,1cyclictest24319-21apt-get22:16:002
30680991816,1cyclictest20554-21grep00:20:002
30676991818,0cyclictest16068-21apt-get00:10:591
30676991818,0cyclictest0-21swapper/121:31:011
30676991818,0cyclictest0-21swapper/119:31:031
30676991818,0cyclictest0-21swapper/119:26:001
30676991817,1cyclictest14665-21apt-get21:55:581
30676991817,1cyclictest0-21swapper/121:10:021
30671991818,0cyclictest0-21swapper/023:51:010
30671991818,0cyclictest0-21swapper/021:51:020
30671991818,0cyclictest0-21swapper/020:46:020
30671991818,0cyclictest0-21swapper/020:41:010
30671991818,0cyclictest0-21swapper/020:31:010
30671991818,0cyclictest0-21swapper/019:36:030
30685991717,0cyclictest10572-21apt-get19:36:023
30685991717,0cyclictest0-21swapper/322:41:023
30685991717,0cyclictest0-21swapper/321:25:593
30685991717,0cyclictest0-21swapper/320:16:023
30685991716,1cyclictest0-21swapper/300:06:023
3068599170,16cyclictest21884-21apt-get21:06:003
30680991717,0cyclictest0-21swapper/223:26:002
30680991717,0cyclictest0-21swapper/223:01:022
30680991717,0cyclictest0-21swapper/220:06:012
30680991716,0cyclictest0-21swapper/223:36:012
30676991717,0cyclictest0-21swapper/123:31:011
30676991717,0cyclictest0-21swapper/123:01:021
30676991717,0cyclictest0-21swapper/120:56:001
30676991717,0cyclictest0-21swapper/120:46:011
30676991715,1cyclictest5444-21turbostat.cron19:25:001
30676991715,1cyclictest13351-21grep00:05:011
30676991714,2cyclictest2487-21memory23:41:041
30671991717,0cyclictest27431-21apt-get20:11:020
30671991717,0cyclictest0-21swapper/023:46:010
30671991717,0cyclictest0-21swapper/023:21:020
30671991715,1cyclictest313-21systemd-journal00:10:010
30685991616,0cyclictest0-21swapper/323:46:013
30685991616,0cyclictest0-21swapper/322:21:013
30685991616,0cyclictest0-21swapper/322:01:003
30685991616,0cyclictest0-21swapper/319:16:013
30685991614,1cyclictest25321-21sh00:30:003
30680991616,0cyclictest0-21swapper/222:11:012
30676991616,0cyclictest0-21swapper/122:21:021
30676991615,1cyclictest27735-21latency_hist00:35:001
30676991614,1cyclictest6397-21latency_hist22:45:001
30676991614,1cyclictest606-21dbus-daemon23:50:001
30676991614,1cyclictest14097-21df_inode00:06:031
3067699160,15cyclictest0-21swapper/120:50:471
3067699160,14cyclictest313-21systemd-journal21:50:011
30671991616,0cyclictest1915-21perf20:25:000
30671991616,0cyclictest0-21swapper/021:31:010
30671991616,0cyclictest0-21swapper/020:51:030
30671991616,0cyclictest0-21swapper/020:36:010
30671991615,1cyclictest25665-21apt-get00:31:000
30671991615,0cyclictest0-21swapper/022:41:020
30671991614,1cyclictest2232-21apt-get20:25:580
30671991614,1cyclictest0-21swapper/022:00:010
30671991613,2cyclictest22311-21sendmail-msp20:00:010
3067199160,15cyclictest6755-21perf21:40:010
3068599159,5cyclictest25269-21cat22:16:043
3068599155,7cyclictest10523-21chrt23:56:273
30685991515,0cyclictest0-21swapper/323:27:073
3068599150,15cyclictest0-21swapper/323:50:293
3068599150,14cyclictest15075-21uniq20:51:053
3068599150,0cyclictest0-21swapper/320:36:023
30680991515,0cyclictest0-21swapper/221:46:002
3068099150,15cyclictest0-21swapper/223:43:002
3068099150,14cyclictest20396-21memory21:01:052
30676991514,0cyclictest0-21swapper/100:16:051
3067699150,8cyclictest0-21swapper/120:06:011
30671991515,0cyclictest0-21swapper/023:11:020
30671991515,0cyclictest0-21swapper/023:06:020
30671991515,0cyclictest0-21swapper/022:46:000
30671991515,0cyclictest0-21swapper/000:36:020
30671991515,0cyclictest0-21swapper/000:16:060
30671991512,2cyclictest24677-21apt21:11:010
30685991413,1cyclictest0-21swapper/323:15:033
3068599140,14cyclictest0-21swapper/323:00:363
30680991414,0cyclictest0-21swapper/222:42:242
30680991414,0cyclictest0-21swapper/220:33:062
3068099140,14cyclictest0-21swapper/223:06:232
3068099140,14cyclictest0-21swapper/221:25:022
3068099140,0cyclictest0-21swapper/222:47:182
3068099140,0cyclictest0-21swapper/221:50:052
3068099140,0cyclictest0-21swapper/220:00:192
30676991414,0cyclictest0-21swapper/122:05:511
3067699140,14cyclictest0-21swapper/120:15:021
30671991414,0cyclictest0-21swapper/022:16:140
30671991414,0cyclictest0-21swapper/000:26:010
3067199140,0cyclictest0-21swapper/021:05:230
3067199140,0cyclictest0-21swapper/021:00:330
3067199140,0cyclictest0-21swapper/019:15:160
3067199140,0cyclictest0-21swapper/000:00:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional