You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-30 - 12:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sun Nov 30, 2025 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2964099250,1cyclictest5552-21apt-get22:47:532
2964199240,23cyclictest15240-21apt-get00:12:533
2963999240,23cyclictest15755-21cron20:54:591
29641992321,2cyclictest583-21avahi-daemon21:45:003
29641992321,2cyclictest583-21avahi-daemon21:45:003
2964199230,22cyclictest5088-21basename22:45:013
2964199230,22cyclictest29586-21munin-run00:40:003
29640992321,1cyclictest0-21swapper/220:05:002
29639992321,1cyclictest17170-21munin-plugin-st00:15:011
29639992321,1cyclictest0-21swapper/119:59:591
2963999231,1cyclictest12290-21cron23:00:001
2963999230,2cyclictest17135-21perf23:10:001
2963999230,22cyclictest900-21perf20:25:001
29638992322,1cyclictest17608-21apt-get23:12:510
2963899230,0cyclictest151rcu_preempt23:25:000
29641992221,0cyclictest0-21swapper/300:27:513
29641992220,1cyclictest593-21dbus-daemon21:19:593
29641992220,1cyclictest12257-21apt-get21:55:013
29641992219,2cyclictest25352-21cron21:15:003
2964199220,21cyclictest16243-21apt-get20:57:513
2964199220,21cyclictest0-21swapper/319:57:503
29640992221,1cyclictest0-21swapper/220:22:502
29640992221,0cyclictest0-21swapper/222:12:522
29640992220,1cyclictest32015-21dump-pmu-power19:14:592
29640992220,1cyclictest12839-21apt-get00:07:492
29640992220,1cyclictest0-21swapper/221:00:002
29640992220,1cyclictest0-21swapper/219:27:502
29640992219,2cyclictest9844-21cron22:55:002
29640992219,2cyclictest17167-21latency_hist00:15:012
2964099220,21cyclictest9844-40mandb00:00:002
2964099220,21cyclictest2506-21apt-get19:22:502
2964099220,21cyclictest0-21swapper/220:37:522
2964099220,1cyclictest0-21swapper/223:47:512
29639992222,0cyclictest22448-21apt-get00:27:511
29639992220,1cyclictest0-21swapper/123:59:591
29639992219,2cyclictest16478-21cron19:50:001
2963999220,21cyclictest7299-21apt-get19:32:501
2963999220,21cyclictest21056-21apt-get21:07:511
2963999220,21cyclictest19865-21apt-get22:10:001
2963999220,21cyclictest0-21swapper/123:15:001
2963999220,1cyclictest25068-21perf22:20:001
29638992220,1cyclictest27785-21latency_hist21:20:000
2963899220,21cyclictest0-21swapper/019:57:500
29641992119,1cyclictest0-21swapper/322:25:003
29641992119,1cyclictest0-21swapper/300:00:003
2964199210,21cyclictest0-21swapper/323:02:503
2964199210,21cyclictest0-21swapper/321:57:503
2964199210,20cyclictest5134-21apt-get23:52:503
2964199210,20cyclictest20730-21apt-get22:12:523
2964199210,1cyclictest28977-21apt-get20:17:513
2964199210,0cyclictest17636-21apt-get00:17:513
2964199210,0cyclictest0-21swapper/321:27:523
2964199210,0cyclictest0-21swapper/319:52:503
29640992121,0cyclictest0-21swapper/223:07:502
29640992120,1cyclictest27797-21wc21:20:002
29640992120,0cyclictest0-21swapper/222:07:502
29640992119,1cyclictest28-21ksoftirqd/220:07:512
29640992119,1cyclictest12321-21latency_hist22:59:592
2964099210,21cyclictest0-21swapper/220:40:022
2964099210,20cyclictest28275-21apt-get21:22:512
2964099210,20cyclictest25548-21apt-get22:22:512
2964099210,20cyclictest0-21swapper/223:32:522
2964099210,20cyclictest0-21swapper/222:37:522
2964099210,20cyclictest0-21swapper/221:32:522
2964099210,20cyclictest0-21swapper/220:27:512
2964099210,20cyclictest0-21swapper/200:17:522
2964099210,20cyclictest0-21swapper/200:02:492
2964099210,1cyclictest0-21swapper/221:55:062
2964099210,0cyclictest3068-21apt-get21:37:532
2964099210,0cyclictest0-21swapper/219:37:512
29639992121,0cyclictest0-21swapper/121:52:511
29639992119,1cyclictest0-21swapper/100:35:021
2963999210,21cyclictest0-21swapper/122:22:511
2963999210,21cyclictest0-21swapper/122:10:011
2963999210,20cyclictest20010-21apt-get23:17:521
2963999210,0cyclictest0-21swapper/123:22:521
2963999210,0cyclictest0-21swapper/122:47:531
2963999210,0cyclictest0-21swapper/119:22:501
29638992121,0cyclictest11396-21apt-get20:47:510
29638992121,0cyclictest0-21swapper/022:47:530
29638992121,0cyclictest0-21swapper/020:27:510
29638992120,1cyclictest18190-21munin-run21:00:000
29638992120,1cyclictest10342-21apt-get22:57:490
2963899210,21cyclictest15213-21apt-get23:07:500
2963899210,20cyclictest7932-21apt-get23:57:530
2963899210,0cyclictest0-21swapper/020:40:000
29641992020,0cyclictest0-21swapper/323:47:513
29641992020,0cyclictest0-21swapper/323:42:503
29641992020,0cyclictest0-21swapper/322:57:503
29641992020,0cyclictest0-21swapper/322:37:523
29641992020,0cyclictest0-21swapper/319:47:523
29641992020,0cyclictest0-21swapper/319:42:503
29641992020,0cyclictest0-21swapper/300:02:493
29641992019,1cyclictest7879-21apt-get21:47:513
29641992018,1cyclictest4997-21perf21:40:003
29641992018,1cyclictest0-21swapper/319:40:003
2964199200,20cyclictest0-21swapper/323:37:503
2964199200,19cyclictest21056-21apt-get21:07:503
2964199200,0cyclictest0-21swapper/323:20:413
2964199200,0cyclictest0-21swapper/322:47:533
2964199200,0cyclictest0-21swapper/320:37:523
2964199200,0cyclictest0-21swapper/320:32:533
2964199200,0cyclictest0-21swapper/320:22:503
29640992020,0cyclictest0-21swapper/221:42:532
29640992020,0cyclictest0-21swapper/221:42:532
29640992020,0cyclictest0-21swapper/221:07:502
29640992020,0cyclictest0-21swapper/219:17:532
29640992020,0cyclictest0-21swapper/200:27:512
29640992019,1cyclictest0-21swapper/219:47:522
29640992019,0cyclictest30330-21apt-get23:37:502
29640992018,1cyclictest0-21swapper/221:55:012
2964099200,20cyclictest0-21swapper/222:42:502
2964099200,0cyclictest0-21swapper/219:52:502
29639992020,0cyclictest0-21swapper/123:27:521
29639992020,0cyclictest0-21swapper/122:52:511
29639992020,0cyclictest0-21swapper/122:42:491
29639992020,0cyclictest0-21swapper/122:37:511
29639992020,0cyclictest0-21swapper/121:57:501
29639992020,0cyclictest0-21swapper/121:27:511
29639992020,0cyclictest0-21swapper/121:12:531
29639992020,0cyclictest0-21swapper/120:47:511
29639992020,0cyclictest0-21swapper/120:02:511
29639992019,1cyclictest25857-21apt-get21:17:521
29639992019,0cyclictest0-21swapper/122:32:531
29639992018,1cyclictest10442-21apt-get00:02:491
2963999200,20cyclictest0-21swapper/123:52:501
2963999200,0cyclictest0-21swapper/120:37:521
29638992020,0cyclictest27926-21apt-get23:32:520
29638992020,0cyclictest24368-21perf00:30:000
29638992020,0cyclictest24178-21apt-get20:07:510
29638992020,0cyclictest0-21swapper/023:27:510
29638992020,0cyclictest0-21swapper/022:35:020
29638992020,0cyclictest0-21swapper/021:42:530
29638992020,0cyclictest0-21swapper/021:42:530
29638992020,0cyclictest0-21swapper/021:32:510
29638992020,0cyclictest0-21swapper/019:47:510
29638992020,0cyclictest0-21swapper/019:17:530
29638992020,0cyclictest0-21swapper/000:37:500
29638992020,0cyclictest0-21swapper/000:22:510
29638992019,1cyclictest27944-21apt-get22:27:490
29638992019,1cyclictest0-21swapper/023:50:000
29638992018,2cyclictest12879-21apt-get21:55:060
2963899200,0cyclictest30679-21apt-get21:27:520
29641991919,0cyclictest8111-21apt-get20:40:023
29641991919,0cyclictest0-21swapper/321:32:523
29640991919,0cyclictest15542-21apt-get22:02:502
29640991919,0cyclictest12815-21apt-get23:02:502
29640991919,0cyclictest0-21swapper/223:12:512
29640991919,0cyclictest0-21swapper/222:27:512
29640991919,0cyclictest0-21swapper/221:47:502
29640991919,0cyclictest0-21swapper/219:57:502
29640991918,1cyclictest0-21swapper/219:32:492
2964099191,3cyclictest151rcu_preempt23:55:002
29639991919,0cyclictest1375-21apt-get20:27:511
29639991919,0cyclictest0-21swapper/123:02:501
29639991919,0cyclictest0-21swapper/120:32:531
29639991918,1cyclictest0-21swapper/120:42:521
29639991918,0cyclictest4902-21apt-get19:27:501
29639991917,1cyclictest0-21swapper/120:10:001
2963999190,1cyclictest18161-21latency_hist21:00:001
29638991919,0cyclictest30351-21apt-get22:32:520
29638991919,0cyclictest0-21swapper/020:02:520
29638991918,1cyclictest17938-21apt-get22:07:500
29641991818,0cyclictest21781-21apt-get20:02:523
29641991818,0cyclictest0-21swapper/323:32:513
29641991818,0cyclictest0-21swapper/323:07:493
29641991818,0cyclictest0-21swapper/320:47:513
29641991818,0cyclictest0-21swapper/319:27:493
29641991816,1cyclictest2022-21sendmail-msp19:20:003
29640991818,0cyclictest0-21swapper/223:17:522
29640991818,0cyclictest0-21swapper/220:32:532
29640991816,1cyclictest593-21dbus-daemon00:40:002
2964099180,0cyclictest151rcu_preempt22:34:002
29639991818,0cyclictest0-21swapper/123:47:511
29639991818,0cyclictest0-21swapper/119:42:501
29638991818,0cyclictest6184-21apt-get20:37:510
29638991818,0cyclictest24847-21apt-get00:32:530
29638991818,0cyclictest0-21swapper/021:37:530
29638991818,0cyclictest0-21swapper/021:07:510
29638991818,0cyclictest0-21swapper/019:37:510
29638991818,0cyclictest0-21swapper/019:22:500
29641991717,0cyclictest0-21swapper/323:17:523
29641991717,0cyclictest0-21swapper/322:27:513
29641991717,0cyclictest0-21swapper/320:27:513
29641991717,0cyclictest0-21swapper/320:07:523
29641991715,1cyclictest0-21swapper/322:32:533
29640991717,0cyclictest32749-21apt-get23:42:512
29640991717,0cyclictest0-21swapper/221:27:512
29640991716,1cyclictest28-21ksoftirqd/220:15:002
29640991715,0cyclictest0-21swapper/220:17:522
29639991717,0cyclictest32501-21apt-get19:17:531
29639991717,0cyclictest0-21swapper/121:32:511
29639991715,1cyclictest19561-21dump-pmu-power00:20:011
29639991715,1cyclictest17465-21latency_hist22:05:001
29639991715,1cyclictest128062sleep100:07:501
2963999170,2cyclictest11634-21perf19:40:001
29638991717,0cyclictest5720-21awk20:35:000
29638991717,0cyclictest0-21swapper/019:52:500
29638991717,0cyclictest0-21swapper/000:17:520
29638991717,0cyclictest0-21swapper/000:02:490
29638991716,1cyclictest31400-21apt-get20:22:500
29638991716,1cyclictest0-21swapper/022:12:520
29638991715,1cyclictest0-21swapper/019:15:000
29641991616,0cyclictest23131-21apt-get22:17:523
29641991616,0cyclictest0-21swapper/319:32:503
29641991616,0cyclictest0-21swapper/300:22:513
2964199160,15cyclictest18462-21perl22:07:513
29640991616,0cyclictest12119-21apt-get19:42:502
29640991616,0cyclictest0-21swapper/220:47:512
29639991616,0cyclictest20053-21apt-get00:22:511
29639991616,0cyclictest0-21swapper/121:42:531
29639991616,0cyclictest0-21swapper/121:42:521
29639991616,0cyclictest0-21swapper/121:37:531
29639991616,0cyclictest0-21swapper/121:22:511
29639991616,0cyclictest0-21swapper/120:17:521
2963999160,15cyclictest0-21swapper/123:32:541
29638991616,0cyclictest7945-21apt-get22:52:510
29638991616,0cyclictest0-21swapper/022:42:500
29638991616,0cyclictest0-21swapper/022:22:500
29638991616,0cyclictest0-21swapper/021:22:510
29638991615,1cyclictest768-21runrttasks19:27:500
29638991615,1cyclictest0-21swapper/000:07:500
29638991614,1cyclictest17165-21idleruntime-cro00:15:010
29638991614,1cyclictest13239-21diskstats23:02:520
29638991614,1cyclictest0-21swapper/021:47:510
2963899160,16cyclictest0-21swapper/023:42:500
2963899160,16cyclictest0-21swapper/020:52:510
29641991515,0cyclictest0-21swapper/322:02:493
29641991515,0cyclictest0-21swapper/321:22:513
29641991515,0cyclictest0-21swapper/320:12:513
29640991514,1cyclictest0-21swapper/223:27:522
2964099150,14cyclictest0-21swapper/221:13:012
2964099150,13cyclictest0-21swapper/223:22:522
29639991515,0cyclictest16963-21apt-get19:52:501
29639991515,0cyclictest0-21swapper/122:27:511
29639991515,0cyclictest0-21swapper/121:02:521
2963999150,15cyclictest0-21swapper/119:14:231
29638991515,0cyclictest0-21swapper/023:37:510
29638991513,1cyclictest0-21swapper/019:32:490
2964199140,14cyclictest30077-21acpi19:12:493
2964199140,14cyclictest0-21swapper/323:26:143
2964199140,14cyclictest0-21swapper/323:11:323
2964199140,14cyclictest0-21swapper/322:50:433
2964199140,14cyclictest0-21swapper/319:20:523
2964199140,14cyclictest0-21swapper/300:07:493
2964199140,0cyclictest0-21swapper/321:00:523
2964199140,0cyclictest0-21swapper/320:52:303
2964199140,0cyclictest0-21swapper/300:30:333
29640991414,0cyclictest0-21swapper/200:21:172
2964099140,14cyclictest0-21swapper/220:50:042
2964099140,0cyclictest0-21swapper/222:15:442
2964099140,0cyclictest0-21swapper/221:01:022
2963999140,0cyclictest0-21swapper/120:10:241
2963999140,0cyclictest0-21swapper/100:33:141
29638991414,0cyclictest0-21swapper/023:16:100
2963899140,14cyclictest0-21swapper/023:53:000
2963899140,14cyclictest0-21swapper/022:15:140
2963899140,14cyclictest0-21swapper/022:00:020
2963899140,14cyclictest0-21swapper/021:11:240
2963899140,14cyclictest0-21swapper/021:01:040
2963899140,0cyclictest0-21swapper/021:52:160
29638991313,0cyclictest0-21swapper/020:14:550
29638991313,0cyclictest0-21swapper/019:41:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional