You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-04 - 06:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #5, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Intel
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack5slot4s.osadl.org (updated Fri Sep 04, 2026 00:44:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
4101386997363,7cyclictest275-21systemd-journal19:40:000
4101386996455,6cyclictest4187877-21latency_hist20:35:010
4101388995853,4cyclictest68128-21kworker/u8:0+flush-8:022:22:451
4101388995751,4cyclictest207675-21kworker/u8:0+events_unbound00:25:001
4101388995751,4cyclictest207675-21kworker/u8:0+events_unbound00:25:001
4101388995750,5cyclictest4113228-21kworker/u8:2+events_unbound21:52:421
4101391995648,6cyclictest237773-21cp00:40:002
4101388995547,6cyclictest4153394-21/usr/sbin/munin20:02:451
4101386995547,6cyclictest88229-21kworker/u8:3+flush-8:023:19:590
4101386995547,6cyclictest4113228-21kworker/u8:2+flush-8:020:17:300
4101388995247,4cyclictest68128-21kworker/u8:0+events_unbound22:17:321
4101388995242,8cyclictest275-21systemd-journal00:20:001
4101391995140,8cyclictest103482-21cp22:25:012
4101388995045,4cyclictest197009-21kworker/u8:1+events_unbound00:07:421
4101388995039,9cyclictest275-21systemd-journal21:35:011
4101388994942,5cyclictest4157924-21kworker/u8:3+events_unbound21:42:381
4101388994942,5cyclictest128260-21kworker/u8:2+flush-8:023:32:401
4101391994836,9cyclictest38858-21awk21:20:002
4101388994743,3cyclictest4118619-21kworker/u8:0+events_unbound20:32:371
4101388994741,4cyclictest4113228-21kworker/u8:2+events_unbound20:27:421
4101386994740,5cyclictest203196-21/usr/sbin/munin00:07:430
41088752460,1sleep14108876-21/usr/sbin/munin19:17:281
4101388994642,3cyclictest122902-21kworker/u8:1+events_unbound22:47:301
4101388994640,5cyclictest68128-21kworker/u8:0+flush-8:022:07:451
4101388994640,5cyclictest3964904-21kworker/u8:1+events_unbound19:12:411
4101388994640,5cyclictest217422-21kworker/u8:3+flush-8:000:37:471
4101388994640,4cyclictest4157924-21kworker/u8:3+events_unbound21:12:401
4101386994642,3cyclictest68128-21kworker/u8:0+flush-8:022:42:310
4101386994639,5cyclictest182646-21latency_hist23:45:000
4101386994636,8cyclictest236947-21ntpq00:37:430
4101388994540,4cyclictest88229-21kworker/u8:3+events_unbound23:37:471
4101388994538,5cyclictest197009-21kworker/u8:1+flush-8:000:02:471
4101388994536,7cyclictest142320-21ntpq23:02:381
4101386994537,6cyclictest67679-21ntpq21:47:400
4101386994536,7cyclictest108891-21diskstats22:32:300
822882440,4sleep10-21swapper/122:02:371
4101388994438,5cyclictest4113228-21kworker/u8:2+events_unbound21:27:411
4101388994438,4cyclictest88229-21kworker/u8:3+flush-8:023:12:291
4101386994438,4cyclictest4113228-21kworker/u8:2+flush-8:020:27:310
4101388994339,3cyclictest8753-21kworker/u8:1+flush-8:021:57:321
4101386994334,7cyclictest183569-21diskstats23:47:300
4101388994237,4cyclictest128260-21kworker/u8:2+flush-8:023:57:441
4101388994236,4cyclictest4113228-21kworker/u8:2+flush-8:019:37:351
4101388994236,4cyclictest207675-21kworker/u8:0+flush-8:000:32:411
4101388994236,4cyclictest207675-21kworker/u8:0+flush-8:000:32:411
4101386994233,6cyclictest63551-21latency_hist21:45:010
4101391994131,7cyclictest4162875-21awk20:10:002
4101388994137,3cyclictest168037-21kworker/u8:0+events_unbound23:47:351
4101388994136,4cyclictest2762707-21kworker/u8:4+flush-8:019:22:291
4101388994135,4cyclictest4113228-21kworker/u8:2+flush-8:021:22:401
4101388994132,7cyclictest13672-21latency_hist20:55:001
4101391994028,9cyclictest4188554-21diskstats20:37:292
4101388994036,3cyclictest207675-21kworker/u8:0+events_unbound00:30:001
4101388994035,4cyclictest153005-21kworker/u8:1+flush-8:023:17:391
4101388994035,3cyclictest4118619-21kworker/u8:0+flush-8:019:52:351
4101391993930,7cyclictest23563-21apt-get21:05:012
4101388993932,5cyclictest4123119-21grep19:27:471
4101386993933,5cyclictest128260-21kworker/u8:2+flush-8:023:15:000
4101386993933,4cyclictest207675-21kworker/u8:0+flush-8:000:35:000
4101386993933,4cyclictest207675-21kworker/u8:0+flush-8:000:35:000
4101388993833,4cyclictest68128-21kworker/u8:0+flush-8:022:27:411
4101388993833,3cyclictest4157924-21kworker/u8:3+flush-8:021:47:431
4101388993833,3cyclictest4118619-21kworker/u8:0+events_unbound20:07:331
4101388993833,3cyclictest197009-21kworker/u8:1+flush-8:000:12:351
4101388993831,5cyclictest4157924-21kworker/u8:3+flush-8:021:17:481
4101386993831,5cyclictest138158-21latency_hist23:00:010
4101388993732,3cyclictest128260-21kworker/u8:2+flush-8:022:57:421
4101388993730,5cyclictest88229-21kworker/u8:3+flush-8:022:42:411
4101388993729,6cyclictest132990-21apt-get22:55:001
4101388993729,4cyclictest108568-21kworker/u8:2+flush-8:022:37:431
4101388993727,8cyclictest275-21systemd-journal23:55:001
4101386993727,8cyclictest275-21systemd-journal20:40:000
4101386993727,7cyclictest128030-21latency_hist22:50:010
4101388993632,3cyclictest4157924-21kworker/u8:3+events_unbound20:17:431
4101388993631,4cyclictest4118619-21kworker/u8:0+flush-8:019:57:401
4101388993631,3cyclictest4113228-21kworker/u8:2+events_unbound20:37:321
4101388993629,5cyclictest275-21systemd-journal23:06:551
4101388993628,6cyclictest24203-21/usr/sbin/munin21:07:451
4101388993529,4cyclictest4113228-21kworker/u8:2+flush-8:020:57:351
4101388993528,6cyclictest108568-21kworker/u8:2+flush-8:022:32:341
4101388993528,5cyclictest275-21systemd-journal20:21:551
4101388993526,7cyclictest167083-21ntpq23:27:411
4101386993531,2cyclictest33756-21base6421:15:010
4101386993527,6cyclictest4105521-21ntpq19:12:390
4101386993527,6cyclictest275-21systemd-journal19:16:560
410138699350,33cyclictest1-21systemd21:59:550
4101391993426,6cyclictest133252-21cat22:55:012
4101391993419,9cyclictest88649-21apt-get22:12:282
4101391993416,11cyclictest4193266-21taskset20:42:282
4101391993416,11cyclictest4193266-21taskset20:42:282
4101388993430,3cyclictest4118619-21kworker/u8:0+events_unbound19:32:311
4101388993428,5cyclictest4113228-21kworker/u8:2+events_unbound21:37:451
4101388993428,3cyclictest4113228-21kworker/u8:2+events_unbound22:15:001
4101386993427,5cyclictest3964904-21kworker/u8:1+flush-8:019:42:290
410139199339,12cyclictest192110-21ntpq23:52:412
4101391993317,10cyclictest4148443-21apt-get19:57:282
4101388993329,3cyclictest4157924-21kworker/u8:3+events_unbound20:47:361
4101388993329,3cyclictest4157924-21kworker/u8:3+events_unbound20:47:361
4101388993328,4cyclictest4157924-21kworker/u8:3+flush-8:020:42:421
4101388993328,4cyclictest4157924-21kworker/u8:3+flush-8:020:42:421
4101388993328,3cyclictest8753-21kworker/u8:1+events_unbound21:02:341
4101388993327,3cyclictest3964904-21kworker/u8:1+events_unbound19:42:351
4101388993326,5cyclictest4157924-21kworker/u8:3+flush-8:020:12:411
410139199328,9cyclictest112407-21ntpq22:32:392
4101388993228,3cyclictest4113228-21kworker/u8:2+flush-8:019:47:361
4101386993126,4cyclictest207675-21kworker/u8:0+events_unbound00:22:430
4101386993126,4cyclictest207675-21kworker/u8:0+events_unbound00:22:430
4101386993126,4cyclictest177529-21kworker/u8:1+events_unbound23:40:000
4101386993122,6cyclictest92939-21grep22:12:480
41940672300,3sleep0201rcuc/020:42:340
41940672300,3sleep0201rcuc/020:42:340
4101388993023,5cyclictest168037-21kworker/u8:0+flush-8:023:42:411
4101386993025,3cyclictest8753-21kworker/u8:1+flush-8:021:07:470
40967122304,9sleep20-21swapper/219:06:502
410139199298,5cyclictest122573-21/usr/sbin/munin22:42:452
410139199295,8cyclictest1-21systemd19:35:002
4101391992924,3cyclictest182779-21latency_hist23:45:002
4101391992919,4cyclictest168238-21apt-get23:32:262
4101391992915,9cyclictest4108310-21fschecks_time19:15:052
4101386992925,3cyclictest128260-21kworker/u8:2+events_unbound22:52:310
4101386992925,2cyclictest4114085-21cpuspeed_turbos19:22:270
4101386992924,3cyclictest207675-21kworker/u8:0+flush-8:000:20:000
4101386992924,3cyclictest197009-21kworker/u8:1+events_unbound00:02:450
410139199286,10cyclictest157177-21ntpq23:17:422
4101391992824,2cyclictest68235-21unixbench_singl21:47:482
4101386992823,4cyclictest3964904-21kworker/u8:1+flush-8:019:48:260
4101386992822,4cyclictest88005-21grep22:07:460
4101386992820,6cyclictest103069-21basename22:25:000
4101386992816,9cyclictest275-21systemd-journal22:25:030
410139199276,11cyclictest4122652-21ntpq19:27:422
4101391992723,2cyclictest73933-21aten2_r5power_e21:57:312
4101391992723,2cyclictest4426-21cpuspeed_turbos20:47:282
4101391992723,2cyclictest4426-21cpuspeed_turbos20:47:282
4101391992723,2cyclictest228279-21fschecks_time00:32:302
4101391992723,2cyclictest228279-21fschecks_time00:32:302
4101391992719,3cyclictest202076-21ntpq00:02:422
4101391992718,7cyclictest28245-21/usr/sbin/munin21:07:462
4101391992718,7cyclictest133899-21cut22:57:352
4101391992716,8cyclictest4142343-21ntpq19:47:402
4101391992712,9cyclictest83734-21apt-get22:07:272
4101388992721,3cyclictest0-21swapper/123:25:011
4101386992723,3cyclictest4157924-21kworker/u8:3+flush-8:021:27:500
4101386992722,4cyclictest4113228-21kworker/u8:2+flush-8:020:07:450
4101386992722,4cyclictest128260-21kworker/u8:2+flush-8:023:27:570
4101386992721,5cyclictest68128-21kworker/u8:0+flush-8:022:04:260
4101386992721,5cyclictest122902-21kworker/u8:1+events_unbound23:07:440
4101386992720,5cyclictest275-21systemd-journal20:54:560
410139199265,11cyclictest0-21swapper/220:52:442
4101391992622,2cyclictest217615-21sendmail00:20:002
4101391992622,2cyclictest117886-21unixbench_singl22:37:502
4101391992619,3cyclictest4105613-21ntpq19:12:412
4101386992622,3cyclictest851-21mta-sts-daemon20:12:300
4101386992621,3cyclictest0-21swapper/021:00:010
4101386992618,6cyclictest275-21systemd-journal23:25:000
410139199257,3cyclictest62882-21ntp_states21:42:422
4101391992521,2cyclictest197795-21sh00:00:002
4101391992520,3cyclictest103785-21aten2_r5power_e22:27:282
4101391992514,9cyclictest275-21systemd-journal22:15:542
4101391992514,7cyclictest124433-21ps22:47:362
4101386992521,2cyclictest72751-21ntpq21:52:410
4101386992519,4cyclictest97265-21needreboot22:17:400
4101386992517,6cyclictest275-21systemd-journal20:24:510
41534522240,2sleep0141ktimers/020:02:290
410139199249,12cyclictest68734-21chrt21:50:422
410139199249,12cyclictest4114236-21smartctl19:22:302
4101391992421,2cyclictest167800-21turbostat.cron23:30:002
4101391992420,2cyclictest4163661-21fschecks_time20:12:322
4101391992415,5cyclictest4147338-21ntpq19:52:432
4101391992414,5cyclictest4183937-21ps20:32:332
4101391992413,8cyclictest4133705-21/usr/sbin/munin19:42:262
4101391992412,8cyclictest158849-21/usr/sbin/munin23:22:332
410138699242,21cyclictest0-21swapper/019:57:460
4101386992420,3cyclictest212443-21unixbench_singl00:12:460
4101386992420,3cyclictest193764-21sort23:57:360
4101386992419,3cyclictest4157924-21kworker/u8:3+flush-8:021:17:470
41594352230,3sleep30-21swapper/320:07:343
410139199238,12cyclictest203166-21chrt00:07:022
410139199235,8cyclictest0-21swapper/220:55:192
4101391992319,2cyclictest78861-21aten2_r5power_e22:02:282
4101391992319,2cyclictest208195-21aten2_r5power_e00:12:282
4101391992314,7cyclictest43134-21/usr/sbin/munin21:22:432
4101391992313,8cyclictest220995-21tr00:22:372
4101391992313,8cyclictest220995-21tr00:22:372
4101391992313,7cyclictest29393-21/usr/sbin/munin21:12:282
4101391992313,6cyclictest47934-21awk21:27:402
4101391992313,6cyclictest139127-21/usr/sbin/munin23:02:322
4101391992313,4cyclictest173179-21apt-get23:37:322
4101391992312,9cyclictest4178764-21/usr/sbin/munin20:27:312
4101391992312,9cyclictest152786-21/usr/sbin/munin23:12:452
4101386992319,3cyclictest207675-21kworker/u8:0+flush-8:000:28:140
4101386992318,4cyclictest4118619-21kworker/u8:0+flush-8:019:32:450
4101386992318,3cyclictest49751-21sort21:32:350
4101386992316,5cyclictest4123108-21grep19:27:470
4101386992314,7cyclictest275-21systemd-journal23:05:000
410139199228,12cyclictest183189-21chrt23:46:032
410139199225,10cyclictest0-21swapper/220:02:452
410139199224,4cyclictest0-21swapper/219:37:312
4101391992218,2cyclictest143789-21fschecks_count23:07:312
4101391992212,8cyclictest54820-21/usr/sbin/munin21:37:392
4101386992218,3cyclictest88229-21kworker/u8:3+flush-8:023:52:390
4101386992218,3cyclictest117343-21ntp_kernel_err22:37:420
4101386992217,3cyclictest54689-21sort21:37:370
40967232224,13sleep10-21swapper/119:06:571
410139199218,10cyclictest52244-21tr21:32:382
410139199215,10cyclictest0-21swapper/220:20:272
4101391992114,5cyclictest227028-21/usr/sbin/munin00:27:442
4101386992117,2cyclictest0-21swapper/020:47:340
4101386992117,2cyclictest0-21swapper/020:47:330
4101386992116,4cyclictest4118619-21kworker/u8:0+events_unbound19:52:370
410138699211,15cyclictest40003-21sed21:22:330
410139199205,8cyclictest0-21swapper/220:17:192
4101386992016,3cyclictest8753-21kworker/u8:1+flush-8:021:02:340
4101386992015,3cyclictest0-21swapper/023:32:450
41010612202,12sleep00-21swapper/019:08:050
40965762193,8sleep30-21swapper/319:05:023
41923142143,7sleep30-21swapper/320:37:453
1987282140,2sleep3471rcuc/300:02:343
981262132,6sleep30-21swapper/322:20:023
637262132,6sleep30-21swapper/321:45:023
1972572130,9sleep30-21swapper/323:57:493
934602122,6sleep30-21swapper/322:15:563
891212122,7sleep30-21swapper/322:12:353
839782122,6sleep30-21swapper/322:07:303
801612122,6sleep30-21swapper/322:02:363
734522122,5sleep30-21swapper/321:55:023
68442123,6sleep30-21swapper/320:47:363
68442123,6sleep30-21swapper/320:47:363
491662122,6sleep30-21swapper/321:32:303
431122122,6sleep30-21swapper/321:22:433
41842332123,5sleep30-21swapper/320:32:353
41735952122,6sleep30-21swapper/320:22:323
41642662122,6sleep30-21swapper/320:12:363
41431222122,5sleep30-21swapper/319:50:013
41428952122,6sleep30-21swapper/319:47:483
41355612122,6sleep30-21swapper/319:42:353
41143632122,6sleep30-21swapper/319:22:323
41086732122,6sleep30-21swapper/319:17:273
32832122,6sleep30-21swapper/320:42:463
32832122,6sleep30-21swapper/320:42:463
295642122,6sleep30-21swapper/321:12:323
2370062122,6sleep30-21swapper/300:37:463
235462122,5sleep30-21swapper/321:05:013
2279882122,6sleep30-21swapper/300:32:283
2279882122,6sleep30-21swapper/300:32:283
2077272122,5sleep30-21swapper/300:10:013
2070692122,6sleep30-21swapper/300:07:443
1826682122,5sleep30-21swapper/323:45:013
1781362123,6sleep30-21swapper/323:42:283
1767942122,5sleep30-21swapper/323:37:433
1687102122,6sleep30-21swapper/323:32:333
1579242122,5sleep30-21swapper/323:20:013
1533472122,6sleep30-21swapper/323:16:563
1481332122,5sleep30-21swapper/323:10:023
1452462122,6sleep30-21swapper/323:07:373
1389862122,6sleep30-21swapper/323:02:313
1288752122,6sleep30-21swapper/322:52:323
1230992122,6sleep30-21swapper/322:45:023
1134092122,6sleep30-21swapper/322:35:023
1035832122,6sleep30-21swapper/322:25:553
687772112,6sleep30-21swapper/321:51:163
630632112,5sleep30-21swapper/321:42:463
539502112,5sleep30-21swapper/321:35:553
440282112,6sleep30-21swapper/321:25:323
41823352112,6sleep30-21swapper/320:27:453
41678582112,5sleep30-21swapper/320:15:013
41537932112,5sleep30-21swapper/320:02:333
41484172112,5sleep30-21swapper/319:57:283
41285282112,6sleep30-21swapper/319:35:243
41237082112,5sleep30-21swapper/319:31:553
41183982112,5sleep30-21swapper/319:25:013
41043942112,5sleep30-21swapper/319:12:373
347272112,5sleep30-21swapper/321:17:333
241852112,6sleep30-21swapper/321:07:143
2228572112,6sleep30-21swapper/300:25:303
2180582112,5sleep30-21swapper/300:22:293
2180582112,5sleep30-21swapper/300:22:293
2131232112,5sleep30-21swapper/300:17:273
1886432112,5sleep30-21swapper/323:52:313
139572113,5sleep30-21swapper/320:55:543
1373982111,6sleep30-21swapper/322:57:463
1185802112,5sleep30-21swapper/322:42:293
1124652112,5sleep30-21swapper/322:32:423
96502102,5sleep30-21swapper/320:52:333
1632362102,5sleep30-21swapper/323:26:553
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional