You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-01 - 04:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot5s.osadl.org (updated Thu Jan 01, 2026 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2946499250,24cyclictest3122-21cron21:35:011
29471992320,2cyclictest7947-21awk21:45:003
29471992320,2cyclictest7137-21sleep00:00:013
29469992322,1cyclictest0-21swapper/200:05:512
2946999230,2cyclictest30760-21perf21:25:012
2946999230,22cyclictest605-21avahi-daemon00:00:012
29464992320,2cyclictest23862-21cron20:05:011
2946499230,22cyclictest21969-21apt-get23:20:491
2946199230,22cyclictest622-21gdbus20:30:010
29471992222,0cyclictest0-21swapper/323:25:513
29471992220,1cyclictest17003-21apt19:50:503
29471992220,1cyclictest15212-21munin-plugin-st22:00:003
29471992220,1cyclictest0-21swapper/320:25:003
29471992219,2cyclictest11430-21apt-get19:40:013
2947199220,1cyclictest29758-21apt-get19:10:513
29469992221,1cyclictest17159-21apt-get23:10:502
29469992220,1cyclictest4415-21munin-run22:45:002
29469992220,1cyclictest25916-21sh21:15:002
2946999220,21cyclictest6202-21apt-get20:35:512
2946999220,21cyclictest24821-21awk22:20:012
2946999220,21cyclictest1811-21dump-pmu-power19:20:002
2946999220,21cyclictest0-21swapper/223:30:002
2946999220,21cyclictest0-21swapper/220:15:502
29464992222,0cyclictest14764-21apt-get23:05:511
29464992221,1cyclictest0-21swapper/122:35:011
29464992220,1cyclictest9016-21apt-get20:40:501
2946499220,21cyclictest11447-21grep19:40:011
2946499220,21cyclictest0-21swapper/122:45:511
2946499220,21cyclictest0-21swapper/122:35:501
2946499220,21cyclictest0-21swapper/122:10:491
2946499220,1cyclictest2015-21apt-get23:45:521
2946499220,1cyclictest0-21swapper/122:50:021
2946499220,0cyclictest0-21swapper/100:10:511
29461992222,0cyclictest0-21swapper/022:56:060
29461992220,1cyclictest313-21systemd-journal21:25:010
29461992220,1cyclictest0-21swapper/023:15:010
29461992220,1cyclictest0-21swapper/019:15:010
2946199220,21cyclictest27200-21sh22:25:010
2946199220,21cyclictest21140-21grep21:05:000
2946199220,21cyclictest16677-21basename19:50:010
2946199220,21cyclictest1292-21munin-run23:45:010
2946199220,21cyclictest0-21swapper/022:40:000
2946199220,21cyclictest0-21swapper/021:35:480
2946199220,21cyclictest0-21swapper/020:45:510
2946199220,21cyclictest0-21swapper/020:40:510
29471992121,0cyclictest17918-21apt-get22:05:513
29471992121,0cyclictest0-21swapper/319:50:013
29471992119,1cyclictest31309-21dump-pmu-power23:40:003
29471992119,1cyclictest28690-21munin-run20:15:003
2947199210,21cyclictest0-21swapper/322:30:523
2947199210,20cyclictest28667-21apt-get21:20:513
2947199210,20cyclictest2326-21apt-get22:40:503
2947199210,20cyclictest22741-21apt-get00:30:503
2947199210,20cyclictest0-21swapper/321:00:533
2947199210,20cyclictest0-21swapper/300:25:513
2947199210,0cyclictest21816-21apt-get20:00:503
2947199210,0cyclictest0-21swapper/300:05:513
29469992121,0cyclictest26604-21apt-get20:10:512
29469992120,1cyclictest3808-21apt-get20:30:502
29469992120,0cyclictest0-21swapper/223:00:532
29469992119,1cyclictest9579-21dump-pmu-power22:55:012
2946999210,20cyclictest0-21swapper/222:20:502
2946999210,20cyclictest0-21swapper/221:05:522
2946999210,20cyclictest0-21swapper/219:55:532
2946999210,20cyclictest0-21swapper/219:35:002
2946999210,0cyclictest0-21swapper/222:05:512
29464992121,0cyclictest6952-21apt-get19:30:501
29464992121,0cyclictest0-21swapper/119:55:531
29464992120,1cyclictest31429-21apt-get20:20:521
29464992120,1cyclictest14612-21apt-get19:45:521
29464992120,1cyclictest0-21swapper/120:50:511
29464992120,1cyclictest0-21swapper/120:05:501
29464992120,0cyclictest25134-21apt-get00:35:511
29464992119,1cyclictest7158-21idleruntime-cro00:00:011
29464992119,1cyclictest0-21swapper/122:05:001
29464992119,1cyclictest0-21swapper/120:50:011
29464992118,2cyclictest29611-21perf22:30:011
2946499210,21cyclictest0-21swapper/123:40:491
2946499210,20cyclictest5551-21perf21:40:011
2946499210,20cyclictest0-21swapper/122:20:501
2946499210,1cyclictest29159-21apt-get23:35:521
2946499210,0cyclictest0-21swapper/122:05:511
2946499210,0cyclictest0-21swapper/121:50:501
29461992121,0cyclictest0-21swapper/023:00:530
29461992120,0cyclictest24364-21apt-get23:25:510
29461992120,0cyclictest22400-21munin-run22:15:010
29461992119,1cyclictest5049-21apt-get23:55:480
2946199210,20cyclictest26762-21apt-get23:30:500
2946199210,20cyclictest0-21swapper/023:35:520
2946199210,20cyclictest0-21swapper/022:45:510
2946199210,20cyclictest0-21swapper/022:05:510
2946199210,20cyclictest0-21swapper/019:20:530
2946199210,1cyclictest11430-21apt-get19:40:020
2946199210,0cyclictest0-21swapper/020:05:510
29471992020,0cyclictest0-21swapper/323:30:503
29471992020,0cyclictest0-21swapper/322:56:063
29471992020,0cyclictest0-21swapper/321:05:523
29471992020,0cyclictest0-21swapper/300:35:513
29471992020,0cyclictest0-21swapper/300:15:513
29471992019,1cyclictest0-21swapper/321:25:513
29471992018,1cyclictest29604-21grep22:30:013
29471992018,1cyclictest28350-21munin-run21:20:013
29471992018,1cyclictest1303-21perf23:45:013
2947199200,20cyclictest0-21swapper/323:15:523
2947199200,0cyclictest16644-21apt-get20:55:503
2947199200,0cyclictest0-21swapper/322:00:503
2947199200,0cyclictest0-21swapper/321:50:503
29469992020,0cyclictest5867-21apt-get21:40:512
29469992020,0cyclictest0-21swapper/223:15:522
29469992020,0cyclictest0-21swapper/222:56:062
29469992020,0cyclictest0-21swapper/222:25:512
29469992020,0cyclictest0-21swapper/220:25:502
29469992020,0cyclictest0-21swapper/220:00:502
29469992019,0cyclictest0-21swapper/221:15:512
29469992018,1cyclictest606-21dbus-daemon23:45:012
29469992018,1cyclictest0-21swapper/222:40:002
29469992018,1cyclictest0-21swapper/200:35:012
29469992017,2cyclictest18746-21munin-run21:00:002
2946999200,1cyclictest14290-21latency_hist19:45:012
29464992020,0cyclictest0-21swapper/122:56:061
29464992020,0cyclictest0-21swapper/121:15:511
29464992020,0cyclictest0-21swapper/120:35:511
29464992020,0cyclictest0-21swapper/119:26:011
29464992020,0cyclictest0-21swapper/119:15:501
29464992020,0cyclictest0-21swapper/119:10:511
29464992019,1cyclictest21455-21apt-get21:05:521
2946499200,0cyclictest22741-21apt-get00:30:501
2946499200,0cyclictest0-21swapper/123:25:501
2946499200,0cyclictest0-21swapper/121:25:501
2946499200,0cyclictest0-21swapper/120:25:501
2946499200,0cyclictest0-21swapper/119:40:511
2946499200,0cyclictest0-21swapper/100:20:511
29461992020,0cyclictest8261-21apt-get21:45:510
29461992020,0cyclictest0-21swapper/022:25:510
29461992020,0cyclictest0-21swapper/021:25:510
29461992020,0cyclictest0-21swapper/021:15:510
29461992020,0cyclictest0-21swapper/021:05:520
29461992020,0cyclictest0-21swapper/019:15:500
29461992019,1cyclictest4738-21perf23:55:010
2946199200,0cyclictest0-21swapper/000:35:510
2946199200,0cyclictest0-21swapper/000:00:510
29471991919,0cyclictest4570-21apt-get19:26:023
29471991919,0cyclictest1397-21apt-get20:25:503
29471991919,0cyclictest0-21swapper/322:15:513
29471991919,0cyclictest0-21swapper/321:10:473
29471991919,0cyclictest0-21swapper/319:30:493
29471991919,0cyclictest0-21swapper/300:20:513
29471991918,1cyclictest9591-21dump-pmu-power22:55:013
29469991919,0cyclictest15525-21apt-get22:00:502
29469991919,0cyclictest0-21swapper/223:30:502
29469991919,0cyclictest0-21swapper/223:20:502
29469991919,0cyclictest0-21swapper/223:05:512
29469991919,0cyclictest0-21swapper/221:35:482
29469991919,0cyclictest0-21swapper/220:45:022
29469991919,0cyclictest0-21swapper/219:45:522
29469991917,1cyclictest27208-21perf00:40:012
2946999190,0cyclictest8282-21apt-get20:40:022
29464991919,0cyclictest19555-21apt-get23:15:521
29464991919,0cyclictest0-21swapper/122:45:001
29464991919,0cyclictest0-21swapper/121:40:511
29464991919,0cyclictest0-21swapper/120:15:501
29461991919,0cyclictest0-21swapper/023:45:520
29461991919,0cyclictest0-21swapper/022:40:500
29461991919,0cyclictest0-21swapper/021:40:510
29461991919,0cyclictest0-21swapper/020:55:500
29461991919,0cyclictest0-21swapper/019:50:500
29471991818,0cyclictest24208-21apt-get20:05:513
29471991818,0cyclictest14249-21apt-get20:50:513
29471991818,0cyclictest0-21swapper/323:10:503
29471991818,0cyclictest0-21swapper/322:45:513
29471991818,0cyclictest0-21swapper/321:45:513
29471991818,0cyclictest0-21swapper/321:30:503
29471991818,0cyclictest0-21swapper/320:45:023
29471991816,1cyclictest8306-21munin-run20:40:013
29471991816,1cyclictest1917-21apt-get23:45:023
29469991818,0cyclictest8261-21apt-get21:45:502
29469991818,0cyclictest0-21swapper/200:15:512
29469991818,0cyclictest0-21swapper/200:10:512
2946999180,17cyclictest0-21swapper/219:40:012
29464991818,0cyclictest0-21swapper/121:45:501
29464991818,0cyclictest0-21swapper/100:00:511
29464991817,1cyclictest9950-21apt-get00:05:511
29461991818,0cyclictest17956-21apt-get00:20:510
29461991818,0cyclictest0-21swapper/021:50:500
29461991818,0cyclictest0-21swapper/021:10:480
29461991818,0cyclictest0-21swapper/020:20:530
2946199180,0cyclictest0-21swapper/020:30:500
29471991717,0cyclictest32150-21apt-get19:15:503
29471991717,0cyclictest2164-21apt-get19:20:523
29471991717,0cyclictest12344-21apt-get00:10:513
29471991717,0cyclictest0-21swapper/323:20:503
29471991717,0cyclictest0-21swapper/323:05:513
29469991717,0cyclictest4721-21apt-get22:45:512
29469991717,0cyclictest0-21swapper/223:45:522
29469991715,1cyclictest0-21swapper/220:20:522
2946999170,16cyclictest31328-21switchtime22:31:002
2946999170,16cyclictest27801-21munin-node23:51:052
29464991717,0cyclictest23847-21apt-get21:10:471
29464991717,0cyclictest0-21swapper/120:30:501
29461991717,0cyclictest26604-21apt-get20:10:510
29461991717,0cyclictest0-21swapper/020:50:510
29461991717,0cyclictest0-21swapper/020:00:510
29461991717,0cyclictest0-21swapper/019:35:480
29461991717,0cyclictest0-21swapper/019:30:490
29461991717,0cyclictest0-21swapper/019:26:010
29461991717,0cyclictest0-21swapper/000:05:510
29461991716,1cyclictest29924-21apt-get22:30:520
29461991712,4cyclictest0-21swapper/020:15:500
29471991616,0cyclictest0-21swapper/322:10:493
29471991616,0cyclictest0-21swapper/320:30:503
29469991616,0cyclictest7556-21apt-get00:00:522
29469991616,0cyclictest17956-21apt-get00:20:512
29469991615,0cyclictest31068-21apt-get21:25:512
29469991614,1cyclictest0-21swapper/221:00:582
29469991613,2cyclictest18985-21sshd19:54:112
29464991616,0cyclictest0-21swapper/121:55:511
29464991616,0cyclictest0-21swapper/121:00:531
29464991616,0cyclictest0-21swapper/120:10:511
29464991616,0cyclictest0-21swapper/119:20:531
2946499160,15cyclictest30336-21grep21:21:001
29461991616,0cyclictest0-21swapper/023:20:500
29461991616,0cyclictest0-21swapper/023:15:520
29461991616,0cyclictest0-21swapper/022:50:020
29461991616,0cyclictest0-21swapper/022:15:510
29461991614,1cyclictest8321-21http20:40:010
29471991515,0cyclictest0-21swapper/322:25:013
29469991515,0cyclictest0-21swapper/222:12:332
29469991515,0cyclictest0-21swapper/220:50:512
29469991513,1cyclictest4251-21munin-run19:25:002
29469991513,1cyclictest31827-21awk19:15:012
29464991515,0cyclictest16644-21apt-get20:55:501
29464991514,0cyclictest0-21swapper/122:15:511
2946499150,1cyclictest0-21swapper/100:15:011
29461991515,0cyclictest0-21swapper/022:00:500
29461991515,0cyclictest0-21swapper/000:30:500
29461991513,1cyclictest0-21swapper/000:25:510
2946199150,1cyclictest1010-21/usr/sbin/munin21:30:500
29471991414,0cyclictest19402-21apt-get19:55:533
29471991414,0cyclictest0-21swapper/321:37:143
29471991414,0cyclictest0-21swapper/320:15:503
2947199140,14cyclictest0-21swapper/323:50:423
2947199140,14cyclictest0-21swapper/323:00:103
2947199140,14cyclictest0-21swapper/322:36:073
2947199140,14cyclictest0-21swapper/320:40:163
2947199140,14cyclictest0-21swapper/319:42:233
2947199140,0cyclictest0-21swapper/300:00:423
29469991413,0cyclictest10654-21apt-get21:50:502
29469991412,1cyclictest0-21swapper/223:35:582
2946999140,14cyclictest0-21swapper/221:55:122
2946999140,14cyclictest0-21swapper/221:31:102
2946999140,14cyclictest0-21swapper/220:05:192
2946999140,14cyclictest0-21swapper/219:25:302
29464991414,0cyclictest0-21swapper/123:30:501
29464991414,0cyclictest0-21swapper/123:00:141
2946499140,14cyclictest0-21swapper/123:50:191
2946499140,14cyclictest0-21swapper/119:50:241
2946499140,14cyclictest0-21swapper/100:26:391
29461991414,0cyclictest0-21swapper/021:55:120
29461991414,0cyclictest0-21swapper/000:15:020
2946199140,14cyclictest0-21swapper/023:05:230
2946199140,14cyclictest0-21swapper/000:10:100
2946199140,0cyclictest0-21swapper/019:55:470
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional