You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-11 - 12:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackeslot5s.osadl.org (updated Sun Jan 11, 2026 00:44:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28154992523,1cyclictest0-21swapper/019:25:000
2815499250,24cyclictest0-21swapper/023:46:420
28166992421,2cyclictest3272-21cron19:25:003
2815899240,23cyclictest886-21grep19:20:012
2815899240,23cyclictest0-21swapper/222:46:412
2816699230,2cyclictest4226-21munin-plugin-st21:40:013
2816699230,22cyclictest13109-21dump-pmu-power23:05:013
2816699230,22cyclictest0-21swapper/322:31:413
2816699230,1cyclictest9388-21apt-get21:51:413
2815899230,22cyclictest1812-21munin-run21:35:012
2815899230,22cyclictest13123-21munin-plugin-st23:05:012
28156992321,1cyclictest30399-21lockfile-remove23:40:001
28156992320,2cyclictest5542-21apt-get23:56:401
2815699230,2cyclictest31818-21munin-plugin-st21:30:011
2815699230,2cyclictest16219-21grep22:05:001
2815699230,22cyclictest13824-21turbostat22:00:001
28154992321,1cyclictest23334-21apt-get23:25:030
2815499230,22cyclictest28131-21turbostat20:15:000
2815499230,22cyclictest10472-21plymouth19:40:000
2816699220,21cyclictest727-21apt-get23:46:413
2816699220,21cyclictest21773-21apt-get22:16:413
2816699220,21cyclictest20132-21apt-get21:06:423
2816699220,21cyclictest15481-21cron23:10:013
2816699220,21cyclictest15306-21munin-run19:50:013
2816699220,21cyclictest0-21swapper/322:56:413
2816699220,1cyclictest0-21swapper/321:05:003
28158992222,0cyclictest0-21swapper/222:51:432
28158992220,1cyclictest606-21dbus-daemon00:15:002
2815899220,21cyclictest7562-40mandb00:00:012
2815899220,21cyclictest27018-21wc21:20:002
2815899220,21cyclictest20084-21awk00:25:002
2815899220,21cyclictest11784-21apt-get21:56:412
2815899220,21cyclictest0-21swapper/222:01:422
2815899220,21cyclictest0-21swapper/221:46:422
2815899220,1cyclictest622-21gdbus00:05:012
2815899220,1cyclictest3474-21perf22:45:002
28156992220,1cyclictest15891-21apt-get23:11:411
2815699221,20cyclictest22-21ksoftirqd/122:21:481
2815699220,21cyclictest2778-21perf23:50:011
2815699220,21cyclictest0-21swapper/119:41:421
2815699220,21cyclictest0-21swapper/100:26:421
2815699220,1cyclictest0-21swapper/122:15:021
28154992220,1cyclictest609-21rs:main0
28154992220,1cyclictest0-21swapper/022:51:430
2815499220,21cyclictest883-21latency_hist19:20:010
2815499220,21cyclictest3476-21munin-run22:45:000
2815499220,21cyclictest30899-21idleruntime-cro19:15:010
2815499220,21cyclictest22854-21apt-get00:31:430
2815499220,21cyclictest0-21swapper/019:56:430
2815499220,21cyclictest0-21swapper/000:01:410
2815499220,1cyclictest20724-21apt-get20:00:020
2815499220,1cyclictest15509-21awk23:10:010
2815499220,0cyclictest0-21swapper/021:11:420
28166992120,1cyclictest6624-21awk21:45:013
28166992120,1cyclictest0-21swapper/319:11:413
28166992120,0cyclictest0-21swapper/300:26:423
28166992119,1cyclictest28114-21sh20:15:003
28166992119,1cyclictest24923-21apt-get21:16:423
28166992118,2cyclictest30387-21idleruntime-cro23:40:003
28166992117,3cyclictest5173-21awk23:55:013
2816699210,20cyclictest28662-21awk22:30:013
2816699210,1cyclictest896-21idleruntime-cro19:20:013
2816699210,1cyclictest13826-21grep22:00:003
2816699210,1cyclictest0-21swapper/319:26:413
2816699210,0cyclictest20083-21apt-get20:00:023
28158992121,0cyclictest3149-21apt-get23:51:432
28158992120,0cyclictest0-21swapper/221:26:412
28158992119,1cyclictest0-21swapper/219:50:012
2815899210,21cyclictest0-21swapper/220:41:402
2815899210,20cyclictest27289-21sendmail-msp00:40:012
2815899210,20cyclictest0-21swapper/221:41:412
2815899210,20cyclictest0-21swapper/220:56:412
2815899210,20cyclictest0-21swapper/220:21:412
2815899210,20cyclictest0-21swapper/219:26:412
2815899210,0cyclictest0-21swapper/219:41:422
2815899210,0cyclictest0-21swapper/219:21:402
28156992121,0cyclictest0-21swapper/119:56:431
28156992120,1cyclictest30942-21apt-get20:21:411
28156992120,1cyclictest26591-21apt-get22:26:431
28156992120,0cyclictest0-21swapper/123:31:421
28156992119,1cyclictest8302-21perf22:55:001
28156992119,1cyclictest1021-21sendmail22:40:001
28156992119,1cyclictest0-21swapper/123:25:021
28156992118,2cyclictest19747-21perf21:05:001
2815699210,20cyclictest6074-21apt-get19:31:411
2815699210,20cyclictest0-21swapper/121:16:411
2815699210,0cyclictest15676-21apt-get19:51:411
2815699210,0cyclictest15676-21apt-get19:51:411
2815699210,0cyclictest1267-21apt-get19:21:401
2815699210,0cyclictest0-21swapper/123:51:431
2815699210,0cyclictest0-21swapper/100:31:431
28154992121,0cyclictest3676-21apt-get19:26:410
28154992121,0cyclictest0-21swapper/023:16:420
28154992121,0cyclictest0-21swapper/020:51:420
28154992120,1cyclictest17738-21apt-get21:01:400
2815499210,20cyclictest0-21swapper/023:01:420
2815499210,1cyclictest0-21swapper/021:46:410
2815499210,0cyclictest0-21swapper/022:06:390
2815499210,0cyclictest0-21swapper/021:56:410
2815499210,0cyclictest0-21swapper/019:51:410
2815499210,0cyclictest0-21swapper/019:51:410
28166992020,0cyclictest0-21swapper/323:21:403
28166992020,0cyclictest0-21swapper/322:51:433
28166992020,0cyclictest0-21swapper/322:06:393
28166992020,0cyclictest0-21swapper/300:20:023
28166992020,0cyclictest0-21swapper/300:00:013
28166992019,1cyclictest10891-21apt-get19:41:423
2816699200,20cyclictest0-21swapper/322:46:413
2816699200,0cyclictest0-21swapper/320:36:413
28158992020,0cyclictest0-21swapper/222:16:412
28158992020,0cyclictest0-21swapper/221:06:422
28158992020,0cyclictest0-21swapper/220:36:412
28158992020,0cyclictest0-21swapper/200:26:422
28158992019,0cyclictest0-21swapper/223:26:422
28158992018,1cyclictest10506-21awk19:40:002
2815899200,2cyclictest4215-21latency_hist21:40:012
2815899200,20cyclictest0-21swapper/200:16:412
2815899200,19cyclictest0-21swapper/220:01:412
2815899200,0cyclictest27396-21apt-get21:21:422
2815899200,0cyclictest0-21swapper/223:31:412
28156992020,0cyclictest0-21swapper/121:06:421
28156992020,0cyclictest0-21swapper/120:16:411
28156992020,0cyclictest0-21swapper/120:10:021
28156992018,1cyclictest5897-21cron22:50:001
2815699200,19cyclictest0-21swapper/120:46:421
2815699200,0cyclictest28877-21apt-get19:11:401
2815699200,0cyclictest0-21swapper/121:21:421
2815699200,0cyclictest0-21swapper/120:01:421
28154992020,0cyclictest5721-21apt-get20:36:410
28154992020,0cyclictest10453-21apt-get00:06:420
28154992020,0cyclictest0-21swapper/023:11:410
28154992020,0cyclictest0-21swapper/021:36:410
28154992020,0cyclictest0-21swapper/020:06:410
28154992020,0cyclictest0-21swapper/000:36:380
28154992020,0cyclictest0-21swapper/000:11:420
2815499200,20cyclictest0-21swapper/020:56:410
2815499200,19cyclictest28522-21apt-get20:16:400
2815499200,0cyclictest0-21swapper/022:01:420
2815499200,0cyclictest0-21swapper/021:31:410
2815499200,0cyclictest0-21swapper/021:21:410
2815499200,0cyclictest0-21swapper/020:21:410
2815499200,0cyclictest0-21swapper/000:21:430
28166991919,0cyclictest14200-21apt-get22:01:423
28166991919,0cyclictest0-21swapper/323:16:423
28166991919,0cyclictest0-21swapper/321:46:423
28166991919,0cyclictest0-21swapper/320:41:403
28166991918,1cyclictest0-21swapper/322:36:413
28166991917,1cyclictest12848-21apt-get00:11:423
28166991917,1cyclictest0-21swapper/319:36:403
28158991919,0cyclictest10533-21apt-get20:46:432
28158991919,0cyclictest0-21swapper/222:11:412
28158991919,0cyclictest0-21swapper/219:51:412
28158991919,0cyclictest0-21swapper/219:51:412
28158991919,0cyclictest0-21swapper/200:31:432
28158991916,2cyclictest313-21systemd-journal20:35:012
28156991919,0cyclictest0-21swapper/123:01:421
28156991919,0cyclictest0-21swapper/122:41:411
28156991918,1cyclictest29048-21apt-get22:31:421
28156991918,0cyclictest20702-21apt-get23:21:401
28154991917,1cyclictest8103-21munin-run19:35:000
28154991917,1cyclictest27011-21sendmail-msp21:20:000
28154991917,1cyclictest0-21swapper/023:36:410
28166991818,0cyclictest0-21swapper/321:31:413
28166991818,0cyclictest0-21swapper/321:11:423
28166991818,0cyclictest0-21swapper/320:46:423
28166991818,0cyclictest0-21swapper/300:01:413
28166991817,0cyclictest0-21swapper/300:16:423
28166991816,1cyclictest31803-21turbostat21:30:013
2816699180,18cyclictest0-21swapper/320:06:413
28158991818,0cyclictest0-21swapper/220:26:412
28158991818,0cyclictest0-21swapper/220:11:422
28158991815,2cyclictest20118-21basename20:00:012
28156991818,0cyclictest8468-21apt-get19:36:401
28156991818,0cyclictest0-21swapper/122:06:391
28156991818,0cyclictest0-21swapper/121:51:411
28156991818,0cyclictest0-21swapper/121:31:411
28156991817,0cyclictest0-21swapper/119:26:411
28156991816,1cyclictest12468-21apt00:10:011
28154991818,0cyclictest13284-21apt-get19:46:420
28154991818,0cyclictest0-21swapper/023:31:420
28154991818,0cyclictest0-21swapper/022:26:430
28154991818,0cyclictest0-21swapper/022:16:410
28154991818,0cyclictest0-21swapper/021:41:410
28154991818,0cyclictest0-21swapper/020:41:400
28154991817,1cyclictest0-21swapper/023:41:410
28154991817,1cyclictest0-21swapper/022:40:000
28166991717,0cyclictest0-21swapper/323:41:413
28166991717,0cyclictest0-21swapper/323:25:033
28166991717,0cyclictest0-21swapper/323:11:413
28166991717,0cyclictest0-21swapper/321:21:423
28166991716,1cyclictest26212-21perf22:25:013
28166991716,1cyclictest0-21swapper/319:31:413
28166991714,2cyclictest24859-21cron00:35:013
2816699170,16cyclictest5346-21perf20:35:013
28158991717,0cyclictest0-21swapper/223:46:422
28158991717,0cyclictest0-21swapper/223:11:412
28158991717,0cyclictest0-21swapper/222:21:482
28158991717,0cyclictest0-21swapper/219:10:022
28158991716,0cyclictest0-21swapper/221:51:412
28158991714,2cyclictest1069-21awk22:40:002
2815899170,0cyclictest0-21swapper/221:01:402
28156991717,0cyclictest0-21swapper/123:41:411
28156991717,0cyclictest0-21swapper/123:06:421
28156991717,0cyclictest0-21swapper/120:36:411
28156991717,0cyclictest0-21swapper/119:46:421
28156991715,1cyclictest8980-21cron21:50:001
28156991715,1cyclictest25302-21munin-plugin-st20:10:011
2815699170,16cyclictest15322-21apt-get20:56:411
28154991717,0cyclictest0-21swapper/021:06:420
28154991717,0cyclictest0-21swapper/000:26:420
28154991715,1cyclictest3846-21apt-get22:46:410
28154991714,2cyclictest313-21systemd-journal22:35:010
28166991616,0cyclictest0-21swapper/320:56:413
28166991616,0cyclictest0-21swapper/320:51:423
28166991616,0cyclictest0-21swapper/300:36:383
28166991614,1cyclictest0-21swapper/320:30:003
28158991614,1cyclictest18607-21apt22:10:002
28156991616,0cyclictest8689-21apt-get22:56:411
28156991616,0cyclictest8136-21apt-get20:41:401
28156991616,0cyclictest0-21swapper/120:26:411
28156991616,0cyclictest0-21swapper/119:16:411
28156991615,0cyclictest0-21swapper/121:36:411
2815699160,1cyclictest0-21swapper/100:40:011
28154991616,0cyclictest0-21swapper/022:56:410
28154991616,0cyclictest0-21swapper/020:46:430
28154991614,1cyclictest0-21swapper/023:51:420
28154991614,1cyclictest0-21swapper/000:16:460
2815499160,15cyclictest0-21swapper/022:25:010
2815499160,11cyclictest0-21swapper/020:31:410
28166991515,0cyclictest0-21swapper/322:41:413
28166991515,0cyclictest0-21swapper/300:06:423
28156991515,0cyclictest0-21swapper/121:11:421
28156991515,0cyclictest0-21swapper/100:16:411
28156991514,1cyclictest783-21snmpd00:02:041
2815499150,1cyclictest315772sleep021:27:260
2815499150,1cyclictest0-21swapper/021:51:250
28166991414,0cyclictest0-21swapper/323:30:303
2816699140,14cyclictest0-21swapper/322:11:073
2816699140,14cyclictest0-21swapper/320:22:103
2816699140,14cyclictest0-21swapper/320:15:343
2816699140,14cyclictest0-21swapper/319:55:203
2816699140,14cyclictest0-21swapper/319:50:043
2816699140,14cyclictest0-21swapper/319:50:043
28158991414,0cyclictest0-21swapper/223:15:072
28158991414,0cyclictest0-21swapper/220:51:002
2815899140,14cyclictest0-21swapper/223:40:152
2815899140,14cyclictest0-21swapper/223:20:052
2815899140,14cyclictest0-21swapper/223:05:302
2815899140,14cyclictest0-21swapper/222:30:102
2815899140,14cyclictest0-21swapper/222:25:482
2815899140,14cyclictest0-21swapper/221:10:252
2815899140,14cyclictest0-21swapper/220:15:272
2815899140,14cyclictest0-21swapper/220:05:192
2815899140,14cyclictest0-21swapper/200:07:472
2815899140,0cyclictest0-21swapper/219:31:102
28156991414,0cyclictest0-21swapper/122:10:031
2815699140,14cyclictest0-21swapper/123:15:041
2815699140,14cyclictest0-21swapper/120:30:101
2815699140,14cyclictest0-21swapper/100:10:401
2815699140,0cyclictest0-21swapper/121:40:411
2815699140,0cyclictest0-21swapper/120:50:111
2815499140,14cyclictest0-21swapper/019:40:390
2815499140,0cyclictest0-21swapper/023:20:140
2815499140,0cyclictest0-21swapper/022:10:340
2815499140,0cyclictest0-21swapper/020:25:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional