You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-02 - 07:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Tue Dec 02, 2025 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1062499250,24cyclictest0-21swapper/319:13:043
10620992523,1cyclictest593-21dbus-daemon21:35:001
1062099250,24cyclictest0-21swapper/121:48:001
1062499240,0cyclictest17610-21apt-get21:38:013
1062499240,0cyclictest0-21swapper/300:33:013
10616992421,2cyclictest7500-21dump-pmu-power21:15:000
1061699241,22cyclictest13-21ksoftirqd/019:50:000
1061699240,23cyclictest9870-21cron21:20:000
1061699240,23cyclictest778-21dump-pmu-power23:15:000
1061699240,23cyclictest22508-21perf00:00:000
10624992320,2cyclictest313-21systemd-journal22:10:003
1062499230,22cyclictest23603-21idleruntime-cro22:55:003
1062499230,22cyclictest22424-21apt-get21:48:003
1062499230,22cyclictest15774-21latency_hist19:20:003
10622992320,2cyclictest23046-21turbostat.cron20:40:002
1062299231,0cyclictest0-21swapper/220:23:012
1062299230,22cyclictest23607-21ls22:55:002
10620992321,1cyclictest0-21swapper/100:20:001
1062099230,22cyclictest9897-21apt21:20:001
10616992323,0cyclictest0-21swapper/023:08:030
10616992321,1cyclictest31940-21dump-pmu-power22:05:000
10616992320,2cyclictest26732-21wc21:55:000
1061699230,22cyclictest593-21dbus-daemon20:05:000
10624992222,0cyclictest0-21swapper/300:38:003
10624992221,1cyclictest3525-21apt-get20:03:013
10624992220,2cyclictest0-21swapper/320:28:013
10624992220,1cyclictest23059-21perf20:40:003
10624992220,1cyclictest0-21swapper/323:19:593
1062499220,21cyclictest27844-21grep00:09:593
1062499220,21cyclictest12815-21apt-get21:28:043
1062499220,21cyclictest10407-21apt-get21:23:013
1062499220,21cyclictest0-21swapper/300:00:003
1062499220,1cyclictest0-21swapper/323:40:003
10622992221,1cyclictest25888-21apt-get00:08:032
10622992220,1cyclictest18186-21munin-run19:25:012
10622992219,2cyclictest5041-21turbostat.cron00:30:002
1062299220,21cyclictest2474-21apt-get22:13:032
1062299220,21cyclictest12335-21munin-run21:25:002
1062299220,21cyclictest10237-21apt-get20:15:022
1062299220,1cyclictest30818-21apt23:09:592
10620992222,0cyclictest32429-21apt-get22:08:001
10620992220,1cyclictest26737-21munin-run21:55:001
10620992220,1cyclictest20097-21turbostat23:54:591
10620992219,1cyclictest313-21systemd-journal23:50:001
1062099220,21cyclictest31311-21apt-get23:13:021
1062099220,21cyclictest27863-21grep20:50:001
1062099220,21cyclictest0-21swapper/123:38:041
1062099220,1cyclictest313-21systemd-journal19:53:001
1062099220,1cyclictest23055-21sendmail-msp20:40:001
1062099220,1cyclictest19534-21grep21:40:001
1062099220,1cyclictest0-21swapper/123:35:001
1062099220,1cyclictest0-21swapper/122:54:591
10616992220,1cyclictest6171-21apt-get23:28:020
10616992220,1cyclictest0-21swapper/021:00:010
1061699220,21cyclictest29130-21ls22:00:000
1061699220,21cyclictest0-21swapper/020:43:000
1061699220,1cyclictest714-21apt-get00:23:000
1061699220,1cyclictest0-21swapper/022:48:010
10624992121,0cyclictest0-21swapper/322:03:003
10624992120,1cyclictest13536-21apt-get20:23:003
10624992119,1cyclictest0-21swapper/320:20:003
10624992118,3cyclictest0-21swapper/322:18:013
10624992117,3cyclictest5919-21apt-get20:08:013
1062499210,20cyclictest0-21swapper/319:43:003
1062499210,1cyclictest27224-21apt-get21:58:033
10622992121,0cyclictest26272-21apt-get19:43:002
10622992120,1cyclictest31063-21apt-get19:53:002
10622992120,1cyclictest19308-21apt-get22:48:002
10622992119,1cyclictest0-21swapper/220:00:002
1062299210,20cyclictest5532-21apt-get00:33:012
1062299210,20cyclictest3194-21apt-get21:08:002
1062299210,20cyclictest30730-21apt-get00:18:002
1062299210,20cyclictest27847-21sh20:50:002
1062299210,0cyclictest0-21swapper/222:00:012
10620992121,0cyclictest0-21swapper/122:13:031
10620992120,0cyclictest0-21swapper/120:15:021
10620992119,1cyclictest5055-21basename00:30:001
10620992117,3cyclictest5075-21cron21:10:001
1062099210,20cyclictest0-21swapper/119:28:021
1062099210,1cyclictest14728-21munin-run21:30:001
1062099210,0cyclictest0-21swapper/100:33:011
1062099210,0cyclictest0-21swapper/100:03:031
10616992121,0cyclictest0-21swapper/021:38:000
10616992119,1cyclictest593-21dbus-daemon00:34:590
10616992119,1cyclictest0-21swapper/021:45:000
1061699210,21cyclictest0-21swapper/021:48:000
1061699210,21cyclictest0-21swapper/021:03:020
1061699210,20cyclictest8569-21apt-get23:33:030
1061699210,0cyclictest0-21swapper/023:03:020
1061699210,0cyclictest0-21swapper/020:53:020
1061699210,0cyclictest0-21swapper/000:38:010
1061699210,0cyclictest0-21swapper/000:13:020
10624992020,0cyclictest7991-21apt-get21:18:013
10624992020,0cyclictest3140-21apt-get00:28:033
10624992020,0cyclictest18733-21apt-get20:33:033
10624992020,0cyclictest0-21swapper/323:13:023
10624992020,0cyclictest0-21swapper/323:08:033
10624992020,0cyclictest0-21swapper/322:48:013
10624992020,0cyclictest0-21swapper/322:34:023
10624992020,0cyclictest0-21swapper/319:28:003
10624992018,1cyclictest28668-21apt-get19:48:003
10624992018,1cyclictest0-21swapper/323:45:003
1062499200,20cyclictest25015-21perf00:05:003
1062499200,20cyclictest0-21swapper/323:28:013
1062499200,0cyclictest719-21apt-get21:03:023
1062499200,0cyclictest28432-21perf23:05:003
1062499200,0cyclictest0-21swapper/323:48:023
1062499200,0cyclictest0-21swapper/321:53:003
1062499200,0cyclictest0-21swapper/321:33:023
10622992020,0cyclictest28340-21apt-get20:53:022
10622992020,0cyclictest28334-21apt-get00:13:022
10622992020,0cyclictest20589-21apt-get23:58:022
10622992020,0cyclictest20030-21apt-get21:43:012
10622992020,0cyclictest16330-21apt-get20:28:022
10622992020,0cyclictest0-21swapper/222:18:012
10622992020,0cyclictest0-21swapper/221:58:032
10622992020,0cyclictest0-21swapper/221:48:002
10622992020,0cyclictest0-21swapper/221:18:022
10622992020,0cyclictest0-21swapper/221:13:012
10622992020,0cyclictest0-21swapper/220:58:022
10622992020,0cyclictest0-21swapper/219:32:592
10622992020,0cyclictest0-21swapper/219:28:012
1062299200,20cyclictest0-21swapper/222:38:002
1062299200,20cyclictest0-21swapper/219:48:012
1062299200,0cyclictest9687-21apt-get22:28:032
1062299200,0cyclictest26520-21apt-get23:03:012
1062299200,0cyclictest11448-21apt-get19:13:052
1062299200,0cyclictest0-21swapper/220:08:012
1062299200,0cyclictest0-21swapper/200:03:022
10620992020,0cyclictest0-21swapper/123:43:001
10620992020,0cyclictest0-21swapper/122:38:001
10620992020,0cyclictest0-21swapper/119:43:001
10620992020,0cyclictest0-21swapper/100:13:021
10620992019,0cyclictest0-21swapper/120:33:031
10620992019,0cyclictest0-21swapper/119:38:001
10620992017,2cyclictest22466-21mandb00:00:001
1062099200,20cyclictest0-21swapper/121:43:001
1062099200,20cyclictest0-21swapper/120:53:011
1062099200,20cyclictest0-21swapper/119:23:001
1062099200,0cyclictest30032-21apt-get22:03:001
10616992020,0cyclictest7288-21apt-get22:23:010
10616992020,0cyclictest7288-21apt-get22:23:010
10616992020,0cyclictest0-21swapper/022:53:020
10616992020,0cyclictest0-21swapper/022:08:010
10616992020,0cyclictest0-21swapper/021:33:020
10616992020,0cyclictest0-21swapper/021:08:000
10616992020,0cyclictest0-21swapper/020:48:020
10616992020,0cyclictest0-21swapper/019:42:590
10616992020,0cyclictest0-21swapper/019:18:010
10616992020,0cyclictest0-21swapper/000:05:020
10616992018,1cyclictest20650-21munin-plugin-st20:35:000
10616992018,1cyclictest0-21swapper/022:35:010
1061699200,20cyclictest0-21swapper/023:18:020
1061699200,0cyclictest0-21swapper/023:23:010
10624991919,0cyclictest14490-21apt-get22:38:003
10624991919,0cyclictest0-21swapper/323:53:013
10624991919,0cyclictest0-21swapper/322:23:013
10624991919,0cyclictest0-21swapper/322:23:003
10624991919,0cyclictest0-21swapper/321:13:013
10624991919,0cyclictest0-21swapper/319:23:013
10624991919,0cyclictest0-21swapper/300:13:023
10622991919,0cyclictest0-21swapper/223:43:012
10622991919,0cyclictest0-21swapper/221:38:012
10622991919,0cyclictest0-21swapper/220:13:002
10622991918,1cyclictest12094-21apt-get22:33:002
10622991917,1cyclictest25789-21basename19:40:002
10622991917,1cyclictest12881-21perf23:40:002
1062299190,1cyclictest26024-21perf23:00:002
10620991919,0cyclictest0-21swapper/123:28:011
10620991919,0cyclictest0-21swapper/123:08:021
10620991919,0cyclictest0-21swapper/123:03:021
10620991919,0cyclictest0-21swapper/122:43:001
10620991919,0cyclictest0-21swapper/122:28:031
10620991919,0cyclictest0-21swapper/120:08:011
10620991919,0cyclictest0-21swapper/119:58:011
10620991919,0cyclictest0-21swapper/119:32:591
10620991918,0cyclictest0-21swapper/120:03:011
10620991917,1cyclictest18226-21latency_hist20:30:001
10616991919,0cyclictest10237-21apt-get20:15:020
10616991919,0cyclictest0-21swapper/019:13:050
10616991918,1cyclictest0-21swapper/019:35:010
10616991918,0cyclictest0-21swapper/020:38:050
10624991818,0cyclictest0-21swapper/322:13:033
10624991818,0cyclictest0-21swapper/320:13:013
10624991818,0cyclictest0-21swapper/300:23:003
10624991816,1cyclictest11602-21perf22:30:003
1062499180,9cyclictest593-21dbus-daemon20:00:003
1062499180,17cyclictest32658-21sh21:00:013
10622991818,0cyclictest0-21swapper/223:53:012
10622991818,0cyclictest0-21swapper/223:33:032
10622991818,0cyclictest0-21swapper/223:13:022
10622991818,0cyclictest0-21swapper/222:07:592
10622991818,0cyclictest0-21swapper/219:18:002
10622991816,1cyclictest17124-21cat21:35:002
10620991818,0cyclictest0-21swapper/122:33:001
10620991818,0cyclictest0-21swapper/121:03:031
10620991818,0cyclictest0-21swapper/120:13:011
10620991816,1cyclictest593-21dbus-daemon23:00:001
10620991815,2cyclictest29131-21grep22:00:001
10616991818,0cyclictest0-21swapper/023:48:020
10616991818,0cyclictest0-21swapper/023:43:000
10616991818,0cyclictest0-21swapper/022:37:590
10616991818,0cyclictest0-21swapper/022:13:030
10616991818,0cyclictest0-21swapper/021:28:030
10616991818,0cyclictest0-21swapper/020:25:010
10616991818,0cyclictest0-21swapper/020:08:010
10616991818,0cyclictest0-21swapper/019:58:020
10616991816,1cyclictest0-21swapper/021:25:000
10616991815,2cyclictest20564-21perf19:30:000
10624991717,0cyclictest3774-21apt-get23:22:593
10624991717,0cyclictest0-21swapper/300:18:003
10622991717,0cyclictest24821-21apt-get21:53:002
10620991717,0cyclictest0-21swapper/119:48:001
10620991716,0cyclictest0-21swapper/120:58:011
10616991717,0cyclictest0-21swapper/023:53:010
10616991717,0cyclictest0-21swapper/022:43:000
10616991717,0cyclictest0-21swapper/020:23:010
10616991717,0cyclictest0-21swapper/020:13:020
10616991717,0cyclictest0-21swapper/000:18:000
10616991717,0cyclictest0-21swapper/000:03:030
10616991716,1cyclictest0-21swapper/019:53:010
10624991616,0cyclictest0-21swapper/323:33:033
10624991616,0cyclictest0-21swapper/319:35:013
10624991615,1cyclictest0-21swapper/321:43:013
10624991614,1cyclictest4574-21sensors_fan21:08:093
1062499160,12cyclictest612-21gdbus23:00:003
10622991616,0cyclictest15794-21apt-get23:48:022
10622991616,0cyclictest0-21swapper/200:23:002
10620991616,0cyclictest0-21swapper/123:23:001
10620991616,0cyclictest0-21swapper/100:08:021
10620991614,1cyclictest5590-21apt-get21:13:011
10620991614,1cyclictest0-21swapper/119:18:011
1062099160,15cyclictest13752-21cstates20:23:021
1062099160,0cyclictest0-21swapper/120:43:011
10616991616,0cyclictest0-21swapper/023:38:040
10616991615,1cyclictest13-21ksoftirqd/023:00:000
10616991614,1cyclictest22964-21dump-pmu-power19:35:000
10622991515,0cyclictest23547-21apt-get20:43:022
10622991515,0cyclictest0-21swapper/221:28:032
10622991514,1cyclictest20035-21gzip20:33:102
10622991514,1cyclictest0-21swapper/221:00:412
10622991513,1cyclictest7928-21apt-get00:38:012
10622991512,2cyclictest18804-21idleruntime-cro22:45:002
10620991513,1cyclictest4872-21apt-get22:18:021
10620991512,2cyclictest9196-21dump-pmu-power22:25:001
10620991512,2cyclictest9196-21dump-pmu-power22:25:001
10620991512,2cyclictest12071-21cut19:13:061
1062099150,14cyclictest612-21gdbus22:50:001
1061699150,0cyclictest0-21swapper/022:18:020
1062499140,14cyclictest0-21swapper/322:42:343
1062499140,14cyclictest0-21swapper/320:51:093
1062499140,14cyclictest0-21swapper/320:40:323
1062499140,0cyclictest0-21swapper/320:46:063
1062499140,0cyclictest0-21swapper/319:32:593
10622991412,1cyclictest0-21swapper/223:26:172
1062299140,14cyclictest0-21swapper/223:23:392
1062299140,14cyclictest0-21swapper/223:18:092
1062299140,14cyclictest0-21swapper/220:00:492
1062299140,0cyclictest0-21swapper/222:20:032
1062299140,0cyclictest0-21swapper/222:20:032
10620991414,0cyclictest0-21swapper/123:18:021
10620991414,0cyclictest0-21swapper/100:20:191
1062099140,14cyclictest0-21swapper/121:23:531
1062099140,14cyclictest0-21swapper/100:37:321
1061699140,14cyclictest0-21swapper/022:26:100
1061699140,0cyclictest0-21swapper/019:23:160
1061699140,0cyclictest0-21swapper/000:26:150
10624991313,0cyclictest0-21swapper/319:51:083
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional