You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-30 - 00:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot5s.osadl.org (updated Sat Nov 29, 2025 12:44:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1046099270,1cyclictest593-21dbus-daemon12:10:001
1046099250,24cyclictest0-21swapper/109:37:501
1046099250,21cyclictest21315-21apt-get10:52:501
10454992422,1cyclictest313-21systemd-journal10:15:000
10454992421,2cyclictest4673-21apt09:09:590
10462992321,2cyclictest23245-21idleruntime-cro10:55:002
10462992321,1cyclictest598-21rs:main2
1046299230,22cyclictest23956-21cron09:49:592
10460992321,1cyclictest14362-21ls09:29:591
10460992320,3cyclictest0-21swapper/108:17:471
1046099230,22cyclictest19489-21munin-plugin-st08:35:011
1046099230,22cyclictest16766-21munin-run09:34:591
10454992320,2cyclictest2608-21perf08:00:000
1045499230,22cyclictest28283-21apt-get07:47:470
1046599220,22cyclictest30681-21apt-get07:52:493
1046599220,21cyclictest9970-21apt-get09:22:483
1046599220,21cyclictest4035-21apt-get10:17:483
1046599220,21cyclictest30191-21cron07:50:003
1046599220,21cyclictest1641-21cron10:15:003
1046599220,21cyclictest0-21swapper/311:12:483
10462992220,1cyclictest32108-21cron12:20:002
10462992220,1cyclictest28-21ksoftirqd/208:10:002
10462992220,1cyclictest0-21swapper/210:00:002
1046299220,21cyclictest31896-21apt-get09:00:022
1046299220,21cyclictest25407-21dump-pmu-power07:40:002
1046299220,21cyclictest24294-21apt-get08:45:012
1046299220,21cyclictest18923-21apt-get10:47:492
1046299220,21cyclictest17719-21munin-run07:25:002
1046299220,21cyclictest15801-21apt-get11:47:492
1046299220,21cyclictest15155-21apt-get08:27:482
1046299220,21cyclictest14351-21perf09:30:002
1046299220,21cyclictest0-21swapper/212:22:482
10460992222,0cyclictest0-21swapper/111:57:511
10460992221,0cyclictest10928-21apt-get11:37:471
10460992220,1cyclictest2627-21munin-run08:00:001
10460992220,1cyclictest0-21swapper/111:10:011
1046099220,21cyclictest8524-21apt-get11:32:491
1046099220,21cyclictest7406-21apt-get12:37:491
1046099220,21cyclictest21870-21systemctl08:40:001
1046099220,21cyclictest1298-21apt-get11:17:501
1046099220,21cyclictest12764-21apt-get08:22:471
1046099220,21cyclictest0-21swapper/111:07:491
1046099220,21cyclictest0-21swapper/110:27:501
1046099220,21cyclictest0-21swapper/110:15:001
1046099220,21cyclictest0-21swapper/108:07:511
1046099220,21cyclictest0-21swapper/108:02:471
1046099220,21cyclictest0-21swapper/107:12:481
1046099220,0cyclictest7551-21apt-get09:17:481
10454992220,1cyclictest1123-21cron10:10:010
10454992219,2cyclictest313-21systemd-journal11:30:000
10454992219,2cyclictest313-21systemd-journal11:29:590
10454992219,2cyclictest21314-21apt10:52:500
1045499220,21cyclictest29953-21apt-get08:57:490
1045499220,21cyclictest26376-21perf09:55:000
1045499220,21cyclictest13404-21apt-get11:42:490
1045499220,1cyclictest5152-21apt-get09:12:470
1045499220,1cyclictest16061-21munin-run10:40:000
10465992121,0cyclictest20598-21apt-get11:57:513
10465992121,0cyclictest0-21swapper/312:32:493
10465992121,0cyclictest0-21swapper/308:27:523
10465992120,1cyclictest28529-21apt-get11:07:483
10465992119,1cyclictest331rcuc/309:57:473
10465992119,1cyclictest26378-21munin-run09:55:003
10465992119,1cyclictest25658-21perf11:00:013
10465992118,2cyclictest24280-21awk08:45:003
1046599210,21cyclictest6453-21apt-get10:22:483
1046599210,20cyclictest29251-21apt-get10:02:473
1046599210,1cyclictest331rcuc/308:40:003
10462992120,1cyclictest10280-21apt-get08:17:472
10462992120,0cyclictest31319-21apt-get11:12:492
10462992120,0cyclictest19643-21apt-get09:42:462
10462992120,0cyclictest1628-21apt-get10:12:472
10462992120,0cyclictest0-21swapper/207:32:492
1046299210,21cyclictest0-21swapper/211:57:502
1046299210,20cyclictest10912-21apt-get07:12:482
1046299210,20cyclictest0-21swapper/210:22:492
1046299210,20cyclictest0-21swapper/209:07:482
1046299210,20cyclictest0-21swapper/207:57:482
1046299210,0cyclictest0-21swapper/211:02:482
1046299210,0cyclictest0-21swapper/210:32:482
10460992121,0cyclictest0-21swapper/107:32:491
10460992119,2cyclictest31648-21apt-get10:07:491
10460992119,1cyclictest28041-21cron11:05:001
1046099210,20cyclictest0-21swapper/111:27:491
1046099210,20cyclictest0-21swapper/111:27:491
1046099210,20cyclictest0-21swapper/110:22:491
1046099210,20cyclictest0-21swapper/109:12:481
1046099210,1cyclictest0-21swapper/111:50:001
1046099210,0cyclictest2609-21apt-get12:27:491
1046099210,0cyclictest0-21swapper/110:47:491
1046099210,0cyclictest0-21swapper/107:17:481
10454992121,0cyclictest0-21swapper/012:02:480
10454992121,0cyclictest0-21swapper/011:32:490
10454992121,0cyclictest0-21swapper/008:07:510
10454992120,1cyclictest25157-21apt-get08:47:480
10454992120,1cyclictest20971-21apt-get07:32:480
10454992119,2cyclictest22037-21apt-get09:47:480
1045499210,21cyclictest0-21swapper/010:32:480
1045499210,21cyclictest0-21swapper/009:17:470
1045499210,20cyclictest16524-21apt-get10:42:490
1045499210,20cyclictest0-21swapper/011:47:490
1045499210,20cyclictest0-21swapper/011:07:490
1045499210,20cyclictest0-21swapper/010:02:480
1045499210,1cyclictest31319-21apt-get11:12:490
1045499210,1cyclictest25674-21idleruntime-cro11:00:010
1045499210,1cyclictest0-21swapper/010:50:000
1045499210,0cyclictest7882-21apt-get08:12:470
1045499210,0cyclictest0-21swapper/012:12:510
10465992020,0cyclictest5490-21apt-get08:07:513
10465992020,0cyclictest0-21swapper/311:22:493
10465992020,0cyclictest0-21swapper/309:32:493
10465992020,0cyclictest0-21swapper/308:57:483
10465992019,0cyclictest0-21swapper/312:17:503
1046599200,19cyclictest0-21swapper/311:32:493
1046599200,0cyclictest23012-21apt-get12:02:473
1046599200,0cyclictest0-21swapper/311:17:503
1046599200,0cyclictest0-21swapper/310:52:503
1046599200,0cyclictest0-21swapper/309:02:473
10462992020,0cyclictest0-21swapper/211:32:492
10462992020,0cyclictest0-21swapper/210:57:502
10462992020,0cyclictest0-21swapper/210:02:482
10462992020,0cyclictest0-21swapper/209:52:472
10462992020,0cyclictest0-21swapper/207:42:472
10462992020,0cyclictest0-21swapper/207:27:472
10462992020,0cyclictest0-21swapper/207:27:472
10462992018,1cyclictest593-21dbus-daemon10:40:002
10460992020,0cyclictest0-21swapper/110:32:471
10460992020,0cyclictest0-21swapper/109:00:021
10460992020,0cyclictest0-21swapper/107:52:491
10460992020,0cyclictest0-21swapper/107:47:471
10460992020,0cyclictest0-21swapper/107:22:461
10460992019,1cyclictest26360-21awk09:55:001
10460992019,0cyclictest0-21swapper/109:42:471
1046099200,20cyclictest0-21swapper/108:12:481
1046099200,19cyclictest24296-21perf08:45:001
1046099200,0cyclictest0-21swapper/108:52:471
1046099200,0cyclictest0-21swapper/107:30:001
1046099200,0cyclictest0-21swapper/107:30:001
10454992020,0cyclictest0-21swapper/011:22:490
10454992020,0cyclictest0-21swapper/009:42:480
10454992020,0cyclictest0-21swapper/008:22:490
10454992019,1cyclictest27811-21turbostat07:45:010
10454992018,1cyclictest12286-21turbostat08:20:000
10454992017,2cyclictest593-21dbus-daemon10:20:000
1045499200,20cyclictest0-21swapper/008:52:470
1045499200,19cyclictest2282-21perf09:05:000
1045499200,19cyclictest0-21swapper/009:30:000
1045499200,0cyclictest0-21swapper/008:42:500
1045499200,0cyclictest0-21swapper/007:52:490
10465991919,0cyclictest8852-21apt-get10:27:503
10465991919,0cyclictest27553-21apt-get08:52:463
10465991919,0cyclictest26130-21apt-get11:02:473
10465991919,0cyclictest11260-21apt-get10:32:483
10465991919,0cyclictest0-21swapper/312:27:493
10465991919,0cyclictest0-21swapper/311:47:493
10465991919,0cyclictest0-21swapper/310:35:033
10465991919,0cyclictest0-21swapper/308:12:483
10465991919,0cyclictest0-21swapper/307:42:473
10465991918,1cyclictest2755-21apt-get09:07:483
10465991917,1cyclictest21567-21sleep09:45:003
1046599190,19cyclictest0-21swapper/309:47:473
1046599190,0cyclictest0-21swapper/308:45:013
10462991919,0cyclictest0-21swapper/212:37:502
10462991919,0cyclictest0-21swapper/210:07:492
10462991919,0cyclictest0-21swapper/207:52:492
10462991918,1cyclictest0-21swapper/208:12:482
10462991917,1cyclictest17227-21apt-get09:37:502
10462991917,1cyclictest12939-21sendmail-msp11:40:012
10460991919,0cyclictest29251-21apt-get10:02:481
10454991919,0cyclictest19950-21apt-get08:37:480
10454991919,0cyclictest0-21swapper/011:37:480
10454991919,0cyclictest0-21swapper/009:57:470
10454991919,0cyclictest0-21swapper/008:32:480
10454991919,0cyclictest0-21swapper/007:27:470
10454991919,0cyclictest0-21swapper/007:27:460
10454991919,0cyclictest0-21swapper/007:17:480
1045499190,0cyclictest0-21swapper/012:22:470
10465991818,0cyclictest0-21swapper/311:27:483
10465991818,0cyclictest0-21swapper/311:27:483
10465991818,0cyclictest0-21swapper/307:32:483
1046599180,0cyclictest0-21swapper/308:22:483
10462991818,0cyclictest0-21swapper/212:32:492
10462991818,0cyclictest0-21swapper/209:12:482
10462991818,0cyclictest0-21swapper/208:57:492
10462991818,0cyclictest0-21swapper/208:02:472
10462991818,0cyclictest0-21swapper/207:47:462
10462991818,0cyclictest0-21swapper/207:17:482
10462991817,0cyclictest0-21swapper/209:32:492
10462991816,1cyclictest19950-21apt-get08:37:492
10460991818,0cyclictest26832-21apt-get09:57:471
10460991818,0cyclictest0-21swapper/112:12:511
10460991818,0cyclictest0-21swapper/112:02:481
10460991818,0cyclictest0-21swapper/111:52:481
10460991817,0cyclictest0-21swapper/108:47:481
10454991818,0cyclictest0-21swapper/009:37:500
10465991717,0cyclictest0-21swapper/309:12:483
10462991717,0cyclictest0-21swapper/212:12:512
10462991715,1cyclictest11893-21latency_hist09:25:002
10460991717,0cyclictest0-21swapper/109:07:481
10460991717,0cyclictest0-21swapper/107:37:471
10460991715,1cyclictest0-21swapper/111:45:011
10454991717,0cyclictest0-21swapper/010:22:480
10454991716,1cyclictest5006-21apt-get12:32:490
10454991716,1cyclictest13-21ksoftirqd/012:37:500
1045499170,16cyclictest5026-21munin-run08:05:000
1045499170,16cyclictest0-21swapper/012:17:510
10465991616,0cyclictest9513-21awk09:20:013
10465991616,0cyclictest0-21swapper/310:07:493
10465991615,0cyclictest12429-21apt-get09:27:473
10465991614,1cyclictest18180-21apt-get07:27:473
10465991614,1cyclictest18180-21apt-get07:27:473
10465991614,1cyclictest0-21swapper/311:55:003
1046599160,15cyclictest18123-21grep10:42:583
10462991616,0cyclictest0-21swapper/212:07:492
10462991616,0cyclictest0-21swapper/208:42:502
10460991616,0cyclictest0-21swapper/112:32:491
10460991616,0cyclictest0-21swapper/110:57:491
10460991616,0cyclictest0-21swapper/108:57:491
1046099160,0cyclictest0-21swapper/111:23:361
10454991616,0cyclictest0-21swapper/007:12:480
10454991614,1cyclictest3236-21apt11:20:000
10465991515,0cyclictest13368-21apt-get07:17:473
10465991515,0cyclictest0-21swapper/309:37:503
10465991515,0cyclictest0-21swapper/307:22:463
10465991514,1cyclictest34-21ksoftirqd/312:22:573
10465991513,1cyclictest25410-21apt-get12:07:473
10462991515,0cyclictest0-21swapper/212:02:482
10462991515,0cyclictest0-21swapper/211:07:492
10462991515,0cyclictest0-21swapper/209:17:482
10462991514,1cyclictest29470-21sh08:55:002
1046299150,15cyclictest583-21avahi-daemon11:41:512
10460991515,0cyclictest0-21swapper/109:22:481
10460991514,1cyclictest25888-21apt-get07:42:461
10460991514,1cyclictest0-21swapper/110:41:291
10460991513,1cyclictest0-21swapper/112:22:531
1046099150,14cyclictest0-21swapper/108:26:211
10454991515,0cyclictest0-21swapper/012:07:480
10465991414,0cyclictest0-21swapper/311:36:473
10465991414,0cyclictest0-21swapper/308:30:263
10465991414,0cyclictest0-21swapper/307:12:473
1046599140,14cyclictest0-21swapper/312:36:193
1046599140,14cyclictest0-21swapper/310:46:103
1046599140,14cyclictest0-21swapper/308:00:213
1046599140,0cyclictest0-21swapper/312:10:013
1046599140,0cyclictest0-21swapper/308:15:453
1046599140,0cyclictest0-21swapper/307:35:093
1046299140,14cyclictest0-21swapper/211:26:412
1046299140,14cyclictest0-21swapper/211:26:412
1046299140,14cyclictest0-21swapper/211:15:272
1046299140,14cyclictest0-21swapper/210:40:172
1046299140,14cyclictest0-21swapper/210:15:062
1046299140,0cyclictest0-21swapper/212:27:472
1046299140,0cyclictest0-21swapper/211:20:052
1046299140,0cyclictest0-21swapper/210:26:412
1046299140,0cyclictest0-21swapper/208:21:282
1046099140,14cyclictest0-21swapper/109:47:021
10454991414,0cyclictest0-21swapper/011:55:170
10454991414,0cyclictest0-21swapper/011:00:040
10454991414,0cyclictest0-21swapper/007:22:530
1045499140,14cyclictest0-21swapper/012:27:330
1045499140,14cyclictest0-21swapper/011:50:090
1045499140,14cyclictest0-21swapper/009:31:080
1045499140,0cyclictest0-21swapper/009:20:100
1045499140,0cyclictest0-21swapper/008:25:550
10462991313,0cyclictest0-21swapper/211:50:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional