You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-17 - 09:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Fri Oct 17, 2025 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2487699240,23cyclictest32610-21apt21:44:043
2487699240,1cyclictest3034-21apt-get19:33:593
2487699240,0cyclictest28744-21apt-get23:59:003
2487099240,23cyclictest17190-21dump-pmu-power21:09:592
2486999240,23cyclictest11290-21apt-get19:54:011
2486599240,23cyclictest6161-21cat23:09:590
24876992317,5cyclictest0-21swapper/321:09:043
24876992316,6cyclictest609-21polkitd22:19:593
2487699230,22cyclictest28175-21dump-pmu-power23:55:003
2487699230,22cyclictest19407-21apt-get22:34:033
2487699230,0cyclictest15647-21apt-get00:39:053
2486999230,22cyclictest7912-21awk19:45:001
2486999230,22cyclictest17504-21sendmail-msp00:39:591
2486999230,0cyclictest0-21swapper/123:24:001
2486599230,2cyclictest16058-21munin-run22:25:000
2486599230,0cyclictest0-21swapper/020:58:560
24876992219,2cyclictest10596-21munin-plugin-st22:09:593
2487699220,21cyclictest5750-21sh21:59:593
2487699220,21cyclictest0-21swapper/300:29:003
2487699220,1cyclictest7272-21perf19:39:593
2487699220,1cyclictest18819-21sh22:30:013
2487099220,21cyclictest18570-21munin-run23:34:592
2487099220,21cyclictest0-21swapper/223:14:022
2487099220,21cyclictest0-21swapper/222:53:592
24869992220,1cyclictest0-21swapper/120:00:011
24869992219,2cyclictest7832-21cron00:20:001
2486999220,21cyclictest8451-21apt-get00:24:011
2486999220,21cyclictest2795-21dump-pmu-power20:40:001
2486999220,21cyclictest17945-21munin-run20:05:001
2486999220,21cyclictest16661-21apt-get22:25:021
2486999220,21cyclictest0-21swapper/120:09:011
2486999220,1cyclictest5789-21munin-plugin-st22:00:001
2486999220,1cyclictest32753-21munin-run20:35:001
2486999220,1cyclictest0-21swapper/121:25:001
2486999220,0cyclictest0-21swapper/122:09:011
24865992221,1cyclictest10297-21munin-plugin-st00:24:590
24865992220,1cyclictest0-21swapper/023:29:020
24865992220,1cyclictest0-21swapper/000:00:010
2486599220,21cyclictest31250-21apt-get00:04:020
2486599220,21cyclictest0-21swapper/021:20:020
2486599220,21cyclictest0-21swapper/020:48:580
2486599220,21cyclictest0-21swapper/000:14:010
2486599220,1cyclictest19601-21dump-pmu-power21:15:000
24876992121,0cyclictest12985-21apt-get21:04:053
24876992121,0cyclictest0-21swapper/322:03:563
24876992120,0cyclictest27778-21apt-get19:18:593
2487699210,20cyclictest0-21swapper/320:43:553
2487699210,20cyclictest0-21swapper/320:33:553
2487699210,1cyclictest26674-21apt-get22:45:013
2487699210,0cyclictest0-21swapper/323:14:013
24870992121,0cyclictest0-21swapper/220:58:572
24870992119,2cyclictest22642-21apt-get21:20:012
24870992119,1cyclictest3634-21apt00:14:022
24870992119,1cyclictest32045-21sendmail-msp21:40:012
24870992119,1cyclictest32045-21sendmail-msp21:40:012
24870992119,1cyclictest0-21swapper/223:04:012
2487099210,21cyclictest0-21swapper/220:18:572
2487099210,20cyclictest3289-21apt-get21:53:552
2487099210,20cyclictest27796-21apt-get21:34:042
2487099210,20cyclictest0-21swapper/222:34:032
2487099210,20cyclictest0-21swapper/200:39:052
24869992119,1cyclictest8549-21apt-get19:45:021
24869992118,2cyclictest23387-21perf23:45:001
2486999210,21cyclictest0-21swapper/123:09:041
2486999210,21cyclictest0-21swapper/119:13:581
2486999210,20cyclictest0-21swapper/121:38:591
2486999210,20cyclictest0-21swapper/121:38:581
2486999210,20cyclictest0-21swapper/121:14:001
2486999210,20cyclictest0-21swapper/120:14:011
2486999210,1cyclictest3635-21apt-get00:14:021
2486999210,1cyclictest0-21swapper/119:34:001
2486999210,0cyclictest0-21swapper/121:34:031
24865992121,0cyclictest0-21swapper/022:03:570
24865992120,0cyclictest0-21swapper/000:19:010
24865992119,1cyclictest18845-21munin-run22:30:000
24865992119,1cyclictest14823-21munin-run21:05:010
24865992119,1cyclictest11221-21perf22:15:000
2486599210,21cyclictest0-21swapper/023:24:000
2486599210,21cyclictest0-21swapper/019:24:000
2486599210,20cyclictest25719-21apt-get20:24:000
2486599210,0cyclictest27020-21apt-get22:49:020
2486599210,0cyclictest23943-21apt-get23:49:010
2486599210,0cyclictest0-21swapper/023:19:010
2486599210,0cyclictest0-21swapper/022:44:010
2486599210,0cyclictest0-21swapper/021:53:560
2486599210,0cyclictest0-21swapper/019:48:560
2486599210,0cyclictest0-21swapper/019:29:000
2486599210,0cyclictest0-21swapper/019:18:590
24876992020,0cyclictest0-21swapper/323:49:013
24876992020,0cyclictest0-21swapper/323:34:033
24876992020,0cyclictest0-21swapper/321:34:043
24876992020,0cyclictest0-21swapper/300:09:003
24876992019,0cyclictest11933-21apt-get23:24:013
2487699200,20cyclictest0-21swapper/323:29:023
2487699200,1cyclictest11368-21perf23:20:013
2487699200,0cyclictest23290-21apt-get20:18:583
2487699200,0cyclictest0-21swapper/321:24:003
24870992020,0cyclictest529-21apt-get19:28:592
24870992020,0cyclictest28167-21dpkg20:25:012
24870992020,0cyclictest20178-21apt-get21:19:032
24870992020,0cyclictest0-21swapper/222:09:012
24870992020,0cyclictest0-21swapper/221:29:002
24870992020,0cyclictest0-21swapper/221:13:592
24870992020,0cyclictest0-21swapper/220:54:012
24870992020,0cyclictest0-21swapper/200:34:032
2487099200,20cyclictest0-21swapper/220:23:592
2487099200,0cyclictest26351-21apt-get23:54:022
24869992020,0cyclictest27548-21apt-get20:25:011
24869992020,0cyclictest19126-21apt-get23:38:591
24869992020,0cyclictest0-21swapper/123:34:591
24869992020,0cyclictest0-21swapper/123:19:021
24869992020,0cyclictest0-21swapper/122:49:011
24869992020,0cyclictest0-21swapper/122:39:001
24869992020,0cyclictest0-21swapper/119:39:001
24869992019,0cyclictest0-21swapper/121:09:031
2486999200,0cyclictest0-21swapper/120:43:551
24865992020,0cyclictest0-21swapper/022:59:050
24865992020,0cyclictest0-21swapper/021:34:030
24865992020,0cyclictest0-21swapper/020:39:010
24865992020,0cyclictest0-21swapper/000:39:060
24865992020,0cyclictest0-21swapper/000:34:030
24865992019,1cyclictest0-21swapper/022:34:020
24865992019,1cyclictest0-21swapper/019:59:020
24865992019,0cyclictest0-21swapper/023:34:020
24865992018,1cyclictest5776-21grep22:00:000
2486599200,19cyclictest0-21swapper/022:55:000
24876991919,0cyclictest18500-21apt-get20:09:013
24876991919,0cyclictest10559-21apt-get20:58:573
24876991919,0cyclictest0-21swapper/322:59:033
24876991919,0cyclictest0-21swapper/320:24:003
24876991918,0cyclictest28508-21apt-get20:28:573
24876991917,1cyclictest10725-21munin-plugin-st19:49:593
24870991919,0cyclictest5769-21apt-get20:48:592
24870991919,0cyclictest24220-21apt-get22:44:012
24870991919,0cyclictest17007-21apt-get22:29:022
24870991919,0cyclictest0-21swapper/223:49:002
24870991919,0cyclictest0-21swapper/223:19:022
24870991919,0cyclictest0-21swapper/222:49:012
24870991919,0cyclictest0-21swapper/219:48:562
24870991919,0cyclictest0-21swapper/200:24:022
24870991919,0cyclictest0-21swapper/200:04:012
2487099190,19cyclictest0-21swapper/222:03:562
24869991919,0cyclictest0-21swapper/123:49:011
24869991919,0cyclictest0-21swapper/122:34:021
24869991919,0cyclictest0-21swapper/120:49:001
24869991919,0cyclictest0-21swapper/119:18:581
24869991919,0cyclictest0-21swapper/100:09:011
24869991917,1cyclictest16156-21dump-pmu-power23:30:001
24869991917,1cyclictest14782-21cron21:05:001
24865991919,0cyclictest0-21swapper/020:54:000
24865991919,0cyclictest0-21swapper/019:54:000
24865991918,1cyclictest25381-21apt-get19:13:580
24865991918,1cyclictest0-21swapper/022:18:550
24865991918,1cyclictest0-21swapper/021:44:040
24865991918,0cyclictest0-21swapper/021:38:580
24865991918,0cyclictest0-21swapper/021:38:580
2486599190,19cyclictest0-21swapper/020:33:560
24876991818,0cyclictest5130-21perf21:55:003
24876991818,0cyclictest20896-21apt-get20:14:023
24876991818,0cyclictest0-21swapper/323:04:003
24876991818,0cyclictest0-21swapper/320:49:003
24876991818,0cyclictest0-21swapper/319:29:003
24876991818,0cyclictest0-21swapper/319:23:593
24876991818,0cyclictest0-21swapper/300:19:013
24876991817,1cyclictest13246-21apt-get00:34:043
24876991816,1cyclictest0-21swapper/321:50:013
24876991814,4cyclictest0-21swapper/322:44:013
24876991811,1cyclictest0-21swapper/320:04:093
24870991818,0cyclictest0-21swapper/223:24:012
24870991818,0cyclictest0-21swapper/219:23:592
24870991818,0cyclictest0-21swapper/219:18:582
24870991817,1cyclictest886-21apt-get20:39:012
24870991816,1cyclictest30682-21ls00:00:002
24870991816,1cyclictest11223-21idleruntime-cro22:15:002
24870991816,1cyclictest0-21swapper/221:05:012
24869991818,0cyclictest7116-21apt-get23:14:021
24869991818,0cyclictest0-21swapper/121:44:031
24869991817,0cyclictest1879-21apt-get23:04:011
24869991815,2cyclictest5138-21munin-plugin-st21:55:011
24865991818,0cyclictest0-21swapper/020:28:590
24865991818,0cyclictest0-21swapper/020:03:590
24865991818,0cyclictest0-21swapper/019:34:000
24865991817,1cyclictest0-21swapper/021:29:000
24876991717,0cyclictest17773-21apt-get21:14:003
24876991717,0cyclictest0-21swapper/300:24:023
2487699170,14cyclictest0-21swapper/323:43:443
24870991717,0cyclictest14332-21apt-get23:29:012
24870991717,0cyclictest0-21swapper/222:24:002
24870991717,0cyclictest0-21swapper/220:33:562
24870991716,0cyclictest6034-21apt-get00:19:022
24869991717,0cyclictest0-21swapper/122:24:001
24869991717,0cyclictest0-21swapper/122:18:551
24869991717,0cyclictest0-21swapper/121:28:591
24869991717,0cyclictest0-21swapper/120:54:011
24869991717,0cyclictest0-21swapper/120:24:001
24869991717,0cyclictest0-21swapper/100:34:021
24869991715,1cyclictest10847-21apt-get00:29:001
24865991717,0cyclictest0-21swapper/023:54:020
24865991717,0cyclictest0-21swapper/000:09:010
24865991715,1cyclictest0-21swapper/023:04:250
2486599170,0cyclictest0-21swapper/020:14:020
24876991616,0cyclictest14211-21apt-get22:23:593
24876991616,0cyclictest0-21swapper/323:09:043
24876991616,0cyclictest0-21swapper/319:54:013
24876991616,0cyclictest0-21swapper/319:13:593
24876991614,1cyclictest32043-21perf21:40:013
24876991614,1cyclictest32043-21perf21:40:013
2487699160,15cyclictest2775-21cron20:39:593
2487699160,15cyclictest0-21swapper/320:55:013
2487699160,0cyclictest25392-21apt-get21:28:593
24870991616,0cyclictest0-21swapper/223:09:042
24870991616,0cyclictest0-21swapper/222:59:052
24870991616,0cyclictest0-21swapper/219:54:012
24870991615,0cyclictest13690-21apt-get19:59:032
24870991614,2cyclictest5775-21dump-pmu-power21:59:592
24869991616,0cyclictest30196-21apt-get19:23:591
24869991616,0cyclictest0-21swapper/123:59:011
24869991616,0cyclictest0-21swapper/122:54:001
24869991615,1cyclictest0-21swapper/120:18:571
24865991616,0cyclictest0-21swapper/023:39:000
24865991616,0cyclictest0-21swapper/021:19:030
24865991616,0cyclictest0-21swapper/020:18:580
24865991616,0cyclictest0-21swapper/020:09:020
24865991616,0cyclictest0-21swapper/000:29:010
2486599160,15cyclictest0-21swapper/023:14:000
2486599160,15cyclictest0-21swapper/021:09:130
24876991515,0cyclictest0-21swapper/319:59:043
24876991515,0cyclictest0-21swapper/300:04:013
24870991515,0cyclictest1226-21apt-get00:09:012
24870991515,0cyclictest11788-21apt-get22:18:552
24870991515,0cyclictest0-21swapper/220:14:012
24870991515,0cyclictest0-21swapper/219:13:592
24870991514,1cyclictest7299-21idleruntime-cro19:40:002
2486999152,11cyclictest0-21swapper/122:44:001
24869991515,0cyclictest0-21swapper/100:04:011
24869991513,1cyclictest20596-21df21:19:051
24865991515,0cyclictest0-21swapper/019:39:000
24876991414,0cyclictest0-21swapper/322:38:373
24876991414,0cyclictest0-21swapper/319:43:153
2487699140,14cyclictest0-21swapper/322:50:223
2487699140,0cyclictest0-21swapper/323:35:263
2487699140,0cyclictest0-21swapper/300:10:043
24870991414,0cyclictest0-21swapper/223:38:322
24870991414,0cyclictest0-21swapper/220:04:002
24870991414,0cyclictest0-21swapper/200:28:042
2487099140,14cyclictest0-21swapper/223:41:492
2487099140,14cyclictest0-21swapper/222:37:272
2487099140,14cyclictest0-21swapper/221:49:502
2487099140,14cyclictest0-21swapper/221:43:222
2487099140,14cyclictest0-21swapper/220:08:062
2487099140,14cyclictest0-21swapper/219:30:332
2487099140,0cyclictest0-21swapper/219:40:152
24869991414,0cyclictest0-21swapper/123:53:131
2486999140,14cyclictest0-21swapper/122:55:571
2486999140,14cyclictest0-21swapper/121:48:581
2486999140,14cyclictest0-21swapper/120:58:491
2486999140,0cyclictest0-21swapper/122:10:011
2486999140,0cyclictest0-21swapper/122:00:301
2486599140,14cyclictest0-21swapper/022:06:060
2486599140,14cyclictest0-21swapper/021:49:500
2486599140,14cyclictest0-21swapper/019:40:480
2486599140,0cyclictest0-21swapper/023:40:310
2486599140,0cyclictest0-21swapper/020:41:080
24876991313,0cyclictest0-21swapper/321:18:283
24870991313,0cyclictest0-21swapper/220:41:112
24876991212,0cyclictest0-21swapper/322:14:473
24865991212,0cyclictest0-21swapper/022:36:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional