You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-06 - 15:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Wed May 06, 2026 12:43:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3843037995648,5cyclictest3967712-21ssh11:00:002
3843036995243,6cyclictest4018984-21systemd-run12:05:011
3843037995030,13cyclictest0-21swapper/210:05:492
384303599481,41cyclictest3923916-21kworker/u8:5+rpciod10:10:480
3843036994737,7cyclictest4013013-21unixbench_singl11:56:001
3843035994733,6cyclictest12350irq/32-eth009:30:470
3843035994722,14cyclictest151ktimers/011:50:490
3843035994722,14cyclictest151ktimers/011:50:490
3843037994539,3cyclictest0-21swapper/210:50:472
3843037994539,3cyclictest0-21swapper/210:50:472
3843037994535,7cyclictest421-21dbus-daemon12:15:452
3843037994436,5cyclictest3903390-21ls09:40:002
3843037994434,6cyclictest420-21avahi-daemon10:05:022
3843037994433,6cyclictest0-21swapper/212:05:512
3843037994337,3cyclictest0-21swapper/208:50:012
3843036994333,5cyclictest0-21swapper/112:25:461
3843036994332,7cyclictest0-21swapper/111:50:011
3843035994333,7cyclictest753-21ntpd11:30:530
3843035994333,7cyclictest753-21ntpd11:30:530
3843035994317,18cyclictest12350irq/32-eth010:45:490
3843035994317,18cyclictest12350irq/32-eth010:45:490
3843037994233,6cyclictest3943701-21ssh10:30:012
384303699424,33cyclictest3904759-21ssh09:40:491
3843036994235,3cyclictest0-21swapper/111:15:011
3843035994224,13cyclictest12350irq/32-eth011:10:500
3843037994133,4cyclictest3862510-21munin-plugin-st08:15:012
3843037994132,5cyclictest3955758-21ssh10:45:012
384303699415,30cyclictest3992721-21ssh11:30:471
384303699415,30cyclictest3992721-21ssh11:30:471
3843036994132,6cyclictest1-21systemd09:20:021
3843036994132,6cyclictest1-21systemd09:20:021
3843036994111,9cyclictest3878478-21ntpq09:05:511
3843035994133,5cyclictest1054-21mta-sts-daemon12:30:510
3843035994123,13cyclictest12350irq/32-eth010:55:560
3843037994033,4cyclictest3967762-21kworker/u8:9+xprtiod11:20:012
3843037994032,5cyclictest3932144-21/usr/sbin/munin10:15:502
384303699402,29cyclictest3956968-21ssh10:45:461
384303699402,29cyclictest3956968-21ssh10:45:461
3843035994034,4cyclictest3923629-21systemd-run10:05:010
3843035994030,7cyclictest3027766-21cups-browsed12:07:490
3843035994030,6cyclictest3902439-21kworker/u8:3-xprtiod09:45:500
3843035994015,17cyclictest151ktimers/010:40:550
3843037993932,4cyclictest4024700-21head12:10:592
3843037993931,4cyclictest0-21swapper/212:30:012
3843037993930,6cyclictest3027766-21cups-browsed09:40:412
384303699395,29cyclictest421-21dbus-daemon11:25:571
384303699395,29cyclictest421-21dbus-daemon11:25:571
3843036993933,3cyclictest0-21swapper/110:10:031
3843036993927,6cyclictest0-21swapper/109:10:471
384303599394,26cyclictest4043696-21ssh12:35:480
384303599393,29cyclictest4004558-21ssh11:45:480
3843037993833,2cyclictest0-21swapper/211:40:002
3843037993828,6cyclictest420-21avahi-daemon09:31:192
3843037993827,6cyclictest1017-21sendmail-mta08:25:472
384303699384,28cyclictest3964997-21ssh10:55:501
3843036993831,3cyclictest0-21swapper/109:30:011
3843036993831,3cyclictest0-21swapper/109:30:011
3843036993830,5cyclictest421-21dbus-daemon11:50:561
3843036993830,5cyclictest421-21dbus-daemon11:50:561
3843036993829,6cyclictest4046367-21sendmail12:40:011
3843036993829,4cyclictest0-21swapper/111:05:491
3843036993827,7cyclictest420-21avahi-daemon10:14:541
3843036993826,7cyclictest4040011-21sendmail_mailtr12:30:551
384303699382,33cyclictest3940866-21ssh10:25:431
3843036993818,4cyclictest3879008-21diskmemload10:44:261
3843036993814,16cyclictest421-21dbus-daemon10:35:501
3843035993831,5cyclictest3951754-21sh10:40:020
3843035993829,6cyclictest3999383-21kworker/u8:7-xprtiod12:00:000
3843035993829,6cyclictest3861084-21latency_hist08:10:020
3843035993826,8cyclictest3961133-21ssh10:50:520
3843035993826,8cyclictest3961133-21ssh10:50:520
3843037993731,3cyclictest0-21swapper/208:15:552
3843037993727,5cyclictest0-21swapper/212:35:502
384303699373,26cyclictest3916877-21rm09:55:531
3843036993730,4cyclictest1056-21mta-sts-daemon09:50:451
3843036993729,5cyclictest3969623-21unixbench_multi11:01:011
3843036993728,5cyclictest1051-21mta-sts-daemon10:18:571
3843036993726,7cyclictest3027766-21cups-browsed11:19:421
3843036993711,8cyclictest3996729-21ntpq11:35:511
3843035993730,5cyclictest3916206-21aten_r2power_fr09:55:390
384303599372,27cyclictest4028051-21ssh12:15:500
3843035993719,6cyclictest420-21avahi-daemon11:30:000
3843035993719,6cyclictest420-21avahi-daemon11:30:000
384303599371,31cyclictest151ktimers/007:25:480
384303599371,31cyclictest151ktimers/007:25:480
384303799364,26cyclictest3929060-21ssh10:10:552
384303799363,23cyclictest3957023-21ssh10:45:482
384303799363,23cyclictest3957023-21ssh10:45:482
3843037993627,5cyclictest1056-21mta-sts-daemon11:49:432
3843036993627,5cyclictest0-21swapper/109:25:001
3843036993627,5cyclictest0-21swapper/109:25:001
3843036993624,6cyclictest0-21swapper/110:50:481
3843036993624,6cyclictest0-21swapper/110:50:481
3843036993610,7cyclictest3985203-21ps11:20:591
3843035993629,4cyclictest3985618-21unixbench_multi11:21:070
3843035993628,6cyclictest4032543-21unixbench_multi12:21:070
3843035993628,5cyclictest421-21dbus-daemon10:30:010
3843035993627,6cyclictest3932274-21aten_r2power_po10:15:380
384303599361,28cyclictest3969008-21ssh11:00:490
3843035993612,13cyclictest151ktimers/007:35:490
384303599360,27cyclictest4035906-21ssh12:25:510
3843037993529,4cyclictest1056-21mta-sts-daemon09:05:462
3843037993529,3cyclictest0-21swapper/209:50:012
3843037993527,5cyclictest695-21cron07:25:002
3843037993527,5cyclictest3851048-21fschecks_time07:35:442
3843037993524,7cyclictest421-21dbus-daemon10:20:462
384303699359,7cyclictest3845486-21ntp_states07:15:541
3843036993528,4cyclictest1051-21mta-sts-daemon10:22:341
3843036993527,5cyclictest3867539-21grep08:30:411
3843036993527,5cyclictest3867539-21grep08:30:411
3843036993527,5cyclictest3846564-21fschecks_time07:20:411
3843036993527,5cyclictest1051-21mta-sts-daemon10:01:531
3843036993527,4cyclictest0-21swapper/112:15:481
3843036993521,8cyclictest3860501-21ntpq08:05:521
3843035993526,5cyclictest1054-21mta-sts-daemon09:15:540
3843035993526,5cyclictest1054-21mta-sts-daemon09:15:540
3843037993430,2cyclictest0-21swapper/212:21:002
3843037993427,5cyclictest3843034-21cyclictest09:15:012
3843037993426,3cyclictest0-21swapper/211:50:502
3843037993426,3cyclictest0-21swapper/211:50:502
3843037993425,5cyclictest3984933-21/usr/sbin/munin11:20:482
384303799342,26cyclictest4000811-21ssh11:40:542
384303699343,23cyclictest3908919-21ssh09:45:541
3843036993428,3cyclictest0-21swapper/108:00:011
3843036993427,4cyclictest3843892-21ntp_kernel_pll_07:10:481
3843036993426,5cyclictest3895903-21aten_r2power_fr09:30:371
3843036993426,5cyclictest3864546-21fschecks_count08:20:421
3843036993425,6cyclictest3867030-21systemd-run08:30:011
384303599348,19cyclictest3892603-21ssh09:25:520
384303599348,19cyclictest3892603-21ssh09:25:520
384303599342,9cyclictest3027766-21cups-browsed09:05:490
3843035993427,4cyclictest3871537-21systemd-run08:45:010
3843035993421,10cyclictest1056-21mta-sts-daemon07:45:580
3843035993416,12cyclictest12350irq/32-eth010:30:500
3843037993328,2cyclictest0-21swapper/209:25:482
3843037993328,2cyclictest0-21swapper/209:25:482
3843037993325,5cyclictest3969656-21unixbench_singl11:01:012
3843037993325,3cyclictest0-21swapper/209:21:002
3843037993325,3cyclictest0-21swapper/209:21:002
3843037993322,9cyclictest3867824-21needreboot08:30:452
3843037993322,9cyclictest3867824-21needreboot08:30:452
384303699338,7cyclictest4023272-21ssh12:10:301
384303699335,21cyclictest428-21mta-sts-daemon09:00:471
3843036993324,6cyclictest3874517-21systemd-run08:55:011
3843036993320,8cyclictest3872431-21ntpq08:45:481
3843036993319,8cyclictest3863384-21ntpq08:15:511
3843036993319,8cyclictest3858946-21ntpq08:00:491
3843035993329,2cyclictest0-21swapper/009:40:460
3843035993325,5cyclictest3925051-21proc_pri10:05:570
3843035993324,6cyclictest4024673-21unixbench_multi12:10:590
3843035993324,6cyclictest0-21swapper/009:40:010
3843037993227,2cyclictest0-21swapper/210:00:012
3843037993225,4cyclictest3869917-21unixbench_singl08:35:592
3843037993225,4cyclictest3869917-21unixbench_singl08:35:592
3843037993224,5cyclictest3877513-21idleruntime-cro09:05:012
3843037993224,5cyclictest3027766-21cups-browsed12:31:542
3843037993224,3cyclictest0-21swapper/211:30:512
3843037993224,3cyclictest0-21swapper/211:30:512
3843037993224,3cyclictest0-21swapper/208:45:002
3843037993223,5cyclictest3949023-21ntpq10:35:502
3843037993223,5cyclictest3947727-21munin-plugin-st10:35:012
3843037993222,5cyclictest0-21swapper/211:55:502
384303699329,7cyclictest3868992-21df08:35:391
384303699329,7cyclictest3868992-21df08:35:391
384303699327,15cyclictest3900747-21ssh09:35:541
384303699325,22cyclictest4031902-21ssh12:20:521
384303699324,11cyclictest3874994-21df_inode08:55:391
3843036993223,5cyclictest0-21swapper/110:30:471
3843036993222,6cyclictest4002177-21ssh11:42:461
3843036993218,8cyclictest3848450-21ntpq07:25:501
3843036993218,8cyclictest3848450-21ntpq07:25:501
3843035993225,4cyclictest3996971-21systemctl11:35:570
3843035993225,3cyclictest151ktimers/007:40:500
3843035993219,9cyclictest12350irq/32-eth012:00:510
3843035993215,14cyclictest421-21dbus-daemon09:50:550
3843035993215,13cyclictest151ktimers/008:15:510
3843037993123,4cyclictest3865537-21kworker/u8:0-xprtiod09:15:442
3843037993123,4cyclictest3865537-21kworker/u8:0-xprtiod09:15:442
3843037993122,6cyclictest420-21avahi-daemon09:55:012
3843036993118,9cyclictest3849986-21ntpq07:30:511
3843036993118,8cyclictest3855995-21ntpq07:50:551
3843036993118,8cyclictest3851378-21ntpq07:35:501
3843035993125,4cyclictest3980436-21aten_r2power_vo11:15:400
3843035993123,5cyclictest3880270-21ntp_kernel_err09:10:470
3843035993123,5cyclictest1-21systemd08:55:020
3843035993113,13cyclictest151ktimers/007:55:520
384303799304,4cyclictest171rcu_preempt11:05:502
3843037993024,4cyclictest3843034-21cyclictest08:00:012
3843037993022,5cyclictest4015909-21gltestperf12:00:422
3843037993022,5cyclictest0-21swapper/207:20:012
384303799302,19cyclictest3988919-21ssh11:25:532
384303799302,19cyclictest3988919-21ssh11:25:532
384303699309,7cyclictest3879008-21diskmemload12:06:051
3843036993018,7cyclictest3861881-21ntpq08:10:451
3843035993023,4cyclictest0-21swapper/008:25:490
3843035993021,6cyclictest3891348-21latency_hist09:25:020
3843035993021,6cyclictest3891348-21latency_hist09:25:020
3843035993011,16cyclictest753-21ntpd11:40:480
384303799294,16cyclictest1051-21mta-sts-daemon11:10:492
3843037992920,6cyclictest548-21gdbus08:00:522
3843036992918,7cyclictest3852937-21ntpq07:40:491
3843036992917,8cyclictest3854488-21ntpq07:45:511
3843035992916,10cyclictest0-21swapper/008:40:000
3843035992916,10cyclictest0-21swapper/008:40:000
384303799283,19cyclictest428-21mta-sts-daemon08:50:532
3843037992820,5cyclictest3027766-21cups-browsed07:44:212
3843036992812,12cyclictest3870975-21ntpq08:40:501
384303599282,22cyclictest3973025-21ssh11:05:480
3843035992820,6cyclictest3939670-21debian-sa110:25:010
3843037992719,5cyclictest3875515-21ps08:55:532
3843035992720,4cyclictest3846339-21tail07:20:370
3843037992619,4cyclictest3860160-21kernelversion08:05:422
3843037992519,3cyclictest3854463-21ntp_offset07:45:492
3843037992518,4cyclictest3846063-21kworker/u8:8+rpciod07:55:012
3843037992518,4cyclictest3844666-21latency_hist07:15:022
384303699255,13cyclictest3837679-21kworker/u8:12+xprtiod07:10:021
3843035992519,3cyclictest0-21swapper/008:15:010
3843037992418,3cyclictest3156421-21kworker/u8:9-nfsiod07:30:002
3843037992418,3cyclictest3156421-21kworker/u8:9-nfsiod07:30:002
3843037992416,5cyclictest3850520-21grep07:35:002
3843035992419,3cyclictest3867580-21idleruntime08:30:410
3843035992419,3cyclictest3867580-21idleruntime08:30:410
3843035992416,5cyclictest0-21swapper/007:15:010
3843035992415,6cyclictest12350irq/32-eth008:45:450
3843035992414,4cyclictest3850267-21timerandwakeup07:30:580
3843037992318,3cyclictest3863982-21kworker/u8:3-xprtiod08:20:422
3843035992318,2cyclictest0-21swapper/008:20:400
3843035992317,4cyclictest3876003-21systemctl09:00:010
3843035992316,5cyclictest3027766-21cups-browsed07:52:150
384303599231,16cyclictest3876955-21ntpq09:00:500
3843035992216,4cyclictest3837678-21kworker/u8:11+xprtiod07:16:000
3843037992114,4cyclictest1051-21mta-sts-daemon07:10:022
3843035992115,3cyclictest0-21swapper/008:05:010
384303899202,10cyclictest0-21swapper/312:30:453
3843035992011,6cyclictest3843171-21latency_hist07:10:020
384303899192,12cyclictest0-21swapper/310:00:553
384303899191,10cyclictest0-21swapper/310:50:483
384303899191,10cyclictest0-21swapper/310:50:483
384303899183,8cyclictest0-21swapper/312:05:473
384303899173,7cyclictest0-21swapper/309:40:513
384303899172,8cyclictest0-21swapper/312:10:473
384303899172,8cyclictest0-21swapper/310:55:493
384303899172,10cyclictest0-21swapper/309:55:563
384303899171,8cyclictest0-21swapper/309:35:463
384303899164,8cyclictest0-21swapper/309:30:543
384303899163,7cyclictest0-21swapper/309:25:493
384303899163,7cyclictest0-21swapper/309:25:493
384303899162,9cyclictest0-21swapper/310:10:593
384303899161,9cyclictest0-21swapper/312:20:533
384303899161,9cyclictest0-21swapper/311:55:523
384303899161,9cyclictest0-21swapper/310:30:543
384303899161,8cyclictest0-21swapper/312:35:463
384303899161,8cyclictest0-21swapper/310:35:483
384303899161,8cyclictest0-21swapper/309:20:453
384303899161,8cyclictest0-21swapper/309:20:453
384303899153,7cyclictest0-21swapper/311:30:473
384303899153,7cyclictest0-21swapper/311:30:473
384303899152,9cyclictest0-21swapper/310:40:383
384303899152,9cyclictest0-21swapper/309:45:413
384303899152,8cyclictest0-21swapper/311:15:543
384303899152,8cyclictest0-21swapper/311:05:493
384303899152,8cyclictest0-21swapper/310:20:513
384303899152,8cyclictest0-21swapper/309:10:473
384303899152,7cyclictest0-21swapper/311:20:413
384303899151,9cyclictest0-21swapper/312:26:013
384303899151,9cyclictest0-21swapper/312:15:483
384303899151,8cyclictest0-21swapper/308:25:493
384303899151,7cyclictest0-21swapper/311:10:373
384303899151,7cyclictest0-21swapper/308:55:463
384303899151,7cyclictest0-21swapper/307:55:533
384303899144,7cyclictest0-21swapper/309:15:483
384303899144,7cyclictest0-21swapper/309:15:483
384303899142,8cyclictest0-21swapper/311:50:463
384303899142,8cyclictest0-21swapper/311:50:463
384303899142,5cyclictest0-21swapper/308:50:463
384303899141,9cyclictest0-21swapper/311:00:393
384303899141,8cyclictest0-21swapper/310:25:513
384303899141,8cyclictest0-21swapper/310:15:473
384303899141,7cyclictest0-21swapper/311:45:463
384303899141,7cyclictest0-21swapper/311:25:493
384303899141,7cyclictest0-21swapper/311:25:493
384303899141,7cyclictest0-21swapper/307:30:473
384303899141,7cyclictest0-21swapper/307:10:493
384303899133,6cyclictest0-21swapper/308:15:553
384303899133,5cyclictest0-21swapper/307:45:453
384303899132,7cyclictest0-21swapper/312:00:383
384303899132,7cyclictest0-21swapper/311:40:383
384303899132,7cyclictest0-21swapper/311:35:483
384303899132,7cyclictest0-21swapper/309:50:363
384303899131,7cyclictest0-21swapper/310:45:503
384303899131,7cyclictest0-21swapper/310:45:503
384303899131,7cyclictest0-21swapper/308:30:473
384303899131,7cyclictest0-21swapper/308:30:473
384303899131,7cyclictest0-21swapper/307:40:483
384303899131,7cyclictest0-21swapper/307:35:503
384303899131,6cyclictest0-21swapper/309:05:473
384303899131,6cyclictest0-21swapper/308:40:453
384303899131,6cyclictest0-21swapper/308:20:513
384303899131,6cyclictest0-21swapper/308:10:463
384303899123,6cyclictest0-21swapper/310:05:553
384303899121,6cyclictest0-21swapper/308:35:513
384303899121,6cyclictest0-21swapper/308:35:513
384303899121,6cyclictest0-21swapper/308:00:473
384303899121,6cyclictest0-21swapper/307:15:483
384303899120,6cyclictest0-21swapper/308:05:453
384303899112,4cyclictest0-21swapper/307:20:463
384303899111,6cyclictest0-21swapper/307:50:533
384303899102,6cyclictest0-21swapper/308:45:383
384303899101,6cyclictest0-21swapper/309:00:413
38430389992,6cyclictest0-21swapper/307:10:013
38430389992,5cyclictest0-21swapper/307:25:473
38430389992,5cyclictest0-21swapper/307:25:473
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional