You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-20 - 03:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot5s.osadl.org (updated Sat Dec 20, 2025 00:44:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1394099300,24cyclictest25730-21latency_hist00:00:013
13940992621,4cyclictest4142-21apt-get19:59:503
1394099250,0cyclictest0-21swapper/321:24:483
1392699253,21cyclictest13-21ksoftirqd/020:25:010
13940992424,0cyclictest0-21swapper/322:59:523
13940992420,4cyclictest0-21swapper/322:49:493
1394099240,23cyclictest0-21swapper/322:54:503
1394099240,22cyclictest31002-21awk00:10:003
1394099240,0cyclictest20321-21apt-get22:44:503
1394099240,0cyclictest0-21swapper/320:04:503
1393699240,22cyclictest313-21systemd-journal21:45:002
13926992422,1cyclictest12467-21apt22:25:000
1392699240,24cyclictest0-21swapper/023:59:500
13940992318,4cyclictest0-21swapper/321:04:483
1394099230,2cyclictest12437-21idleruntime-cro21:20:013
1394099230,22cyclictest23798-21apt-get20:39:513
1394099230,22cyclictest19175-21apt-get23:49:503
13936992321,1cyclictest8949-21apt-get20:09:482
1393699230,2cyclictest20375-21cron20:30:012
1393699230,22cyclictest21491-21basename19:25:012
13930992321,1cyclictest1080-21cron19:50:001
1393099230,2cyclictest4102-21grep23:15:001
1393099230,22cyclictest313-21systemd-journal23:10:011
1393099230,22cyclictest27980-21perf20:45:001
1393099230,22cyclictest19632-21awk21:35:001
1393099230,22cyclictest0-21swapper/122:40:001
13926992321,1cyclictest0-21swapper/022:10:010
13926992321,1cyclictest0-21swapper/020:04:490
13940992220,1cyclictest10695-21cron20:10:013
1394099220,21cyclictest23067-21apt-get21:44:483
1394099220,21cyclictest0-21swapper/322:04:473
1394099220,21cyclictest0-21swapper/321:34:503
1394099220,1cyclictest7167-21apt-get23:24:503
1393699220,21cyclictest6865-21apt-get00:29:522
1393699220,21cyclictest0-21swapper/221:24:472
1393699220,1cyclictest5225-21awk21:05:012
1393699220,1cyclictest31757-21idleruntime-cro23:05:012
1393099229,13cyclictest22-21ksoftirqd/123:24:501
13930992222,0cyclictest0-21swapper/119:34:481
13930992222,0cyclictest0-21swapper/119:10:021
13930992220,1cyclictest17294-21apt-get19:19:511
13930992220,1cyclictest0-21swapper/121:05:011
1393099220,21cyclictest23862-21latency_hist19:30:011
1393099220,21cyclictest13433-21perf00:40:011
1393099220,21cyclictest0-21swapper/123:54:501
1393099220,21cyclictest0-21swapper/122:29:491
1393099220,21cyclictest0-21swapper/121:19:501
1393099220,21cyclictest0-21swapper/120:39:511
1393099220,1cyclictest10726-21basename20:10:011
13926992220,1cyclictest7657-21perf22:15:010
13926992220,1cyclictest21570-21apt-get23:54:500
13926992220,1cyclictest13-21ksoftirqd/022:20:010
13926992220,1cyclictest0-21swapper/000:24:490
1392699220,21cyclictest28609-21apt-get20:49:490
1392699220,21cyclictest13431-21grep00:40:010
1392699220,21cyclictest0-21swapper/022:49:490
1392699220,21cyclictest0-21swapper/019:54:490
1392699220,1cyclictest32369-21apt-get23:09:500
1392699220,1cyclictest26871-21apt-get00:04:510
13940992121,0cyclictest24498-21apt-get19:34:483
13940992120,0cyclictest0-21swapper/321:45:023
13940992120,0cyclictest0-21swapper/300:29:523
13940992120,0cyclictest0-21swapper/300:04:513
1394099210,21cyclictest0-21swapper/322:24:503
1394099210,20cyclictest0-21swapper/319:29:493
1394099210,0cyclictest14892-21apt-get19:14:483
13936992121,0cyclictest0-21swapper/223:44:502
13936992121,0cyclictest0-21swapper/221:45:022
13936992121,0cyclictest0-21swapper/219:59:502
13936992120,0cyclictest0-21swapper/222:14:502
13936992119,2cyclictest11346-21apt-get20:14:502
13936992119,1cyclictest31100-21apt-get21:59:492
13936992119,1cyclictest1649-21cron23:30:002
13936992119,1cyclictest0-21swapper/221:14:502
1393699210,20cyclictest13456-21sendmail-msp00:40:012
1393699210,20cyclictest0-21swapper/200:04:512
1393699210,0cyclictest0-21swapper/220:04:502
1393699210,0cyclictest0-21swapper/219:14:482
13930992121,0cyclictest9561-21apt-get23:29:501
13930992121,0cyclictest28259-21apt-get21:54:501
13930992121,0cyclictest0-21swapper/100:24:491
13930992120,0cyclictest0-21swapper/122:14:501
13930992119,1cyclictest30439-21latency_hist20:50:001
13930992119,1cyclictest22780-21grep20:35:011
13930992118,2cyclictest31731-21awk23:05:011
1393099210,20cyclictest0-21swapper/123:44:501
1393099210,20cyclictest0-21swapper/120:19:491
13926992121,0cyclictest26894-21apt-get19:39:500
13926992120,1cyclictest13806-21apt-get19:10:020
13926992120,1cyclictest0-21swapper/023:45:010
1392699210,20cyclictest606-21dbus-daemon22:00:010
1392699210,20cyclictest0-21swapper/020:34:470
1392699210,20cyclictest0-21swapper/019:49:500
1392699210,1cyclictest13061-21apt-get21:24:480
1392699210,1cyclictest0-21swapper/023:15:000
1392699210,0cyclictest0-21swapper/023:29:500
1392699210,0cyclictest0-21swapper/019:59:510
1392699210,0cyclictest0-21swapper/019:24:480
13940992020,0cyclictest0-21swapper/323:19:493
13940992020,0cyclictest0-21swapper/320:14:503
13940992020,0cyclictest0-21swapper/319:24:483
13940992020,0cyclictest0-21swapper/300:19:503
13940992018,1cyclictest0-21swapper/320:54:493
13940992017,3cyclictest0-21swapper/319:54:483
1394099200,20cyclictest0-21swapper/323:15:003
1394099200,20cyclictest0-21swapper/321:29:493
13936992020,0cyclictest31759-21apt-get19:49:502
13936992020,0cyclictest0-21swapper/221:09:492
13936992020,0cyclictest0-21swapper/220:19:492
13936992019,1cyclictest0-21swapper/222:20:012
13936992019,0cyclictest0-21swapper/223:24:512
13936992018,1cyclictest1722-21perf23:10:012
1393699200,1cyclictest1091-21apt-get22:04:472
1393699200,0cyclictest3503-21apt-get22:09:502
1393699200,0cyclictest0-21swapper/221:30:012
1393699200,0cyclictest0-21swapper/220:54:492
1393699200,0cyclictest0-21swapper/219:34:482
13930992020,0cyclictest0-21swapper/122:59:521
13930992020,0cyclictest0-21swapper/121:14:491
13930992020,0cyclictest0-21swapper/120:14:491
13930992020,0cyclictest0-21swapper/119:59:501
13930992020,0cyclictest0-21swapper/100:09:501
13930992019,1cyclictest8293-21apt-get22:19:501
13930992019,0cyclictest15507-21apt-get22:34:481
13930992019,0cyclictest0-21swapper/122:49:491
1393099200,0cyclictest0-21swapper/120:29:491
13926992020,0cyclictest9253-21apt-get00:34:490
13926992020,0cyclictest0-21swapper/023:34:490
13926992020,0cyclictest0-21swapper/023:04:490
13926992020,0cyclictest0-21swapper/020:44:490
13926992020,0cyclictest0-21swapper/020:29:490
13926992018,1cyclictest13086-21cron20:15:010
1392699200,20cyclictest0-21swapper/023:24:510
1392699200,20cyclictest0-21swapper/019:19:510
1392699200,0cyclictest22102-21apt-get19:29:490
1392699200,0cyclictest19621-21apt-get21:35:020
1392699200,0cyclictest0-21swapper/023:39:470
1392699200,0cyclictest0-21swapper/022:34:470
13940991919,0cyclictest31660-21apt-get00:14:523
13940991919,0cyclictest20649-21apt-get21:39:483
13940991919,0cyclictest0-21swapper/323:54:503
13940991919,0cyclictest0-21swapper/323:39:483
13940991919,0cyclictest0-21swapper/323:09:503
13940991919,0cyclictest0-21swapper/321:54:503
13940991919,0cyclictest0-21swapper/321:09:493
13940991919,0cyclictest0-21swapper/320:24:463
13940991917,1cyclictest4064-21apt-get00:20:023
13940991916,2cyclictest28815-21cat19:40:013
13936991919,0cyclictest0-21swapper/222:39:512
13936991919,0cyclictest0-21swapper/221:54:502
13936991919,0cyclictest0-21swapper/220:59:492
13936991919,0cyclictest0-21swapper/220:35:022
13936991916,2cyclictest3487-21cron19:55:002
13936991916,2cyclictest12395-21cron21:20:012
13930991919,0cyclictest0-21swapper/121:59:491
13930991919,0cyclictest0-21swapper/100:29:531
13930991919,0cyclictest0-21swapper/100:04:511
13930991917,1cyclictest2799-21idleruntime-cro21:00:011
13930991917,1cyclictest25750-40mandb00:00:011
13930991917,1cyclictest17237-21sleep21:30:011
13930991916,2cyclictest606-21dbus-daemon19:45:011
13930991916,2cyclictest3406-21sendmail00:20:001
1393099190,0cyclictest0-21swapper/122:44:501
13926991919,0cyclictest0-21swapper/023:49:500
13926991919,0cyclictest0-21swapper/022:54:490
13926991919,0cyclictest0-21swapper/021:54:500
13926991919,0cyclictest0-21swapper/021:34:490
13926991919,0cyclictest0-21swapper/021:04:480
13926991919,0cyclictest0-21swapper/019:34:480
13926991919,0cyclictest0-21swapper/000:29:530
13926991919,0cyclictest0-21swapper/000:19:490
13926991917,1cyclictest22097-21munin-run22:45:010
1392699190,0cyclictest0-21swapper/000:14:520
13940991818,0cyclictest0-21swapper/321:59:493
13940991818,0cyclictest0-21swapper/300:34:493
13940991817,1cyclictest30427-21perf20:50:003
13940991816,1cyclictest7657-21perf22:15:013
1394099180,1cyclictest19074-21munin-run19:20:013
13936991818,0cyclictest4750-21apt-get23:19:502
13936991818,0cyclictest22712-21apt-get22:49:492
13936991818,0cyclictest0-21swapper/221:39:482
13936991818,0cyclictest0-21swapper/219:44:492
13936991818,0cyclictest0-21swapper/219:39:492
13936991817,1cyclictest27506-21apt-get22:59:522
13936991817,1cyclictest0-21swapper/222:29:492
13936991815,2cyclictest11035-21munin-plugin-st00:35:012
13930991818,0cyclictest0-21swapper/122:09:501
13930991818,0cyclictest0-21swapper/121:49:481
13930991818,0cyclictest0-21swapper/121:24:481
13930991815,2cyclictest3527-21idleruntime-cro19:55:001
1393099180,18cyclictest0-21swapper/121:39:481
13926991818,0cyclictest0-21swapper/021:44:480
13926991818,0cyclictest0-21swapper/020:09:480
13926991816,1cyclictest0-21swapper/022:39:510
13940991716,1cyclictest9561-21apt-get23:29:503
13940991715,1cyclictest18414-21cron23:45:013
13936991717,0cyclictest3414-21apt-get00:20:022
13936991717,0cyclictest0-21swapper/223:54:502
13936991717,0cyclictest0-21swapper/222:34:482
13936991717,0cyclictest0-21swapper/220:49:502
13936991717,0cyclictest0-21swapper/200:14:522
13936991716,1cyclictest21004-21apt-get20:34:472
13936991715,1cyclictest0-21swapper/200:09:502
13930991717,0cyclictest0-21swapper/100:34:491
13930991716,0cyclictest5855-21apt-get21:09:491
13926991717,0cyclictest0-21swapper/023:19:500
13926991717,0cyclictest0-21swapper/020:59:500
13926991716,0cyclictest0-21swapper/020:39:500
13926991714,2cyclictest14875-21perf22:30:000
13940991616,0cyclictest8253-21apt-get21:14:493
13940991616,0cyclictest0-21swapper/320:29:483
13940991615,1cyclictest1097-21turbostat19:50:003
1394099160,14cyclictest0-21swapper/322:05:593
13936991616,0cyclictest0-21swapper/200:19:502
13936991615,1cyclictest0-21swapper/222:24:502
1393699160,15cyclictest20612-21ls23:49:582
13930991616,0cyclictest0-21swapper/123:39:481
13930991614,1cyclictest8309-21basename20:05:001
1393099160,0cyclictest0-21swapper/121:44:481
13926991614,1cyclictest31059-21apt-get20:54:490
13926991614,1cyclictest27323-21chrt22:57:530
13926991614,1cyclictest0-21swapper/022:04:520
13940991513,2cyclictest0-21swapper/320:59:463
1394099150,14cyclictest0-21swapper/322:34:573
13936991515,0cyclictest0-21swapper/223:34:482
13936991513,1cyclictest27968-21idleruntime-cro20:45:002
1393699150,1cyclictest0-21swapper/222:45:012
13930991513,1cyclictest0-21swapper/123:35:011
13926991515,0cyclictest29263-21apt-get00:09:500
13926991515,0cyclictest0-21swapper/020:19:490
1392699150,1cyclictest15406-21taskset21:29:280
13940991414,0cyclictest0-21swapper/320:30:463
13940991414,0cyclictest0-21swapper/320:15:283
1394099140,14cyclictest0-21swapper/323:31:113
1394099140,14cyclictest0-21swapper/323:00:043
1394099140,14cyclictest0-21swapper/322:26:403
1394099140,14cyclictest0-21swapper/320:40:173
1394099140,14cyclictest0-21swapper/319:40:353
1394099140,0cyclictest0-21swapper/322:35:303
1394099140,0cyclictest0-21swapper/322:15:313
1394099140,0cyclictest0-21swapper/300:35:013
13936991414,0cyclictest0-21swapper/219:15:442
1393699140,14cyclictest0-21swapper/223:55:492
1393699140,0cyclictest0-21swapper/223:36:332
1393699140,0cyclictest0-21swapper/223:10:362
1393699140,0cyclictest0-21swapper/222:50:242
1393699140,0cyclictest0-21swapper/221:30:352
1393699140,0cyclictest0-21swapper/220:20:532
1393699140,0cyclictest0-21swapper/219:25:122
13930991414,0cyclictest0-21swapper/123:45:511
13930991414,0cyclictest0-21swapper/120:50:021
13930991414,0cyclictest0-21swapper/120:20:051
13930991414,0cyclictest0-21swapper/119:20:021
1393099140,14cyclictest0-21swapper/123:15:551
1393099140,0cyclictest0-21swapper/122:50:061
1393099140,0cyclictest0-21swapper/100:11:221
13926991414,0cyclictest0-21swapper/019:41:280
1392699140,14cyclictest0-21swapper/021:15:370
1392699140,14cyclictest0-21swapper/021:10:020
1392699140,14cyclictest0-21swapper/021:05:090
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional