You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-29 - 17:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Wed Jul 29, 2026 12:43:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3511592995043,4cyclictest3602620-21basename10:40:011
3511592995042,5cyclictest3535603-21systemd-run08:30:011
3511592994740,4cyclictest3558437-21unixbench-2d09:25:591
3511592994737,6cyclictest421-21dbus-daemon10:35:011
3511592994638,5cyclictest3587408-21latency_hist10:15:011
3511592994638,5cyclictest3587408-21latency_hist10:15:011
3511592994637,6cyclictest3564286-21wc09:35:561
3511591994627,13cyclictest12650irq/32-eth009:30:470
3511592994536,6cyclictest421-21dbus-daemon08:20:011
3511591994534,5cyclictest12650irq/32-eth011:05:490
351159399443,37cyclictest3631191-21ssh11:25:482
3511593994431,9cyclictest3575881-21entropy09:55:402
351159299444,35cyclictest3609875-21ssh10:50:511
3511592994434,7cyclictest3637172-21missed_timers11:35:481
3511591994433,8cyclictest3639198-21run-parts11:40:010
3511593994334,5cyclictest3663756-21latency_hist12:20:032
3511592994334,6cyclictest3538604-21true08:40:011
3511592994334,6cyclictest2728660-21cups-browsed10:55:471
3511592994233,6cyclictest421-21dbus-daemon11:45:021
3511592994232,7cyclictest3649851-21threads11:56:011
3511592994230,9cyclictest3579951-21unixbench_multi10:01:021
3511592994230,9cyclictest3579951-21unixbench_multi10:01:021
3511591994235,4cyclictest1-21systemd11:25:010
351159199422,28cyclictest3588491-21ssh10:15:490
3511593994133,5cyclictest3654447-21ssh12:05:012
3511593994132,6cyclictest420-21avahi-daemon11:07:212
3511593994132,6cyclictest2728660-21cups-browsed09:44:022
3511593994130,5cyclictest0-21swapper/210:10:482
3511593994130,5cyclictest0-21swapper/210:10:482
3511593994125,12cyclictest3588744-21mailstats10:15:562
3511592994132,4cyclictest0-21swapper/108:55:451
3511591994133,5cyclictest3627027-21munin-run11:20:010
3511591994132,6cyclictest420-21avahi-daemon10:07:190
3511591994132,6cyclictest420-21avahi-daemon10:07:190
3511591994132,6cyclictest3655800-21sendmail_mailqu12:05:550
3511593994034,3cyclictest0-21swapper/208:00:012
3511591994033,4cyclictest3674322-21timerwakeupswit12:36:000
3511591994032,3cyclictest0-21swapper/010:00:560
3511591994032,3cyclictest0-21swapper/010:00:560
3511593993931,5cyclictest1-21systemd09:35:012
3511593993929,6cyclictest0-21swapper/210:55:532
3511592993931,5cyclictest3636144-21cron11:35:011
3511592993930,7cyclictest3520568-21/usr/sbin/munin07:40:021
3511591993933,4cyclictest3645301-21cron11:50:010
3511593993831,4cyclictest3553686-21sendmail09:20:002
3511593993829,5cyclictest0-21swapper/209:50:512
351159299383,30cyclictest3588589-21ssh10:15:521
3511592993831,3cyclictest0-21swapper/111:15:511
3511592993830,6cyclictest3620926-21ssh11:10:011
3511592993829,6cyclictest3534090-21cut08:25:011
3511592993829,5cyclictest3550628-21systemd-run09:15:011
351159299382,28cyclictest3603769-21ssh10:40:481
3511591993818,13cyclictest151ktimers/008:55:480
3511593993730,4cyclictest694-21cron10:40:012
3511593993729,4cyclictest0-21swapper/211:35:012
3511593993729,3cyclictest0-21swapper/212:05:482
3511593993728,6cyclictest420-21avahi-daemon10:01:162
3511593993728,6cyclictest420-21avahi-daemon10:01:162
3511593993728,6cyclictest3645504-21latency_hist11:50:032
3511592993731,4cyclictest0-21swapper/111:45:531
3511592993729,5cyclictest1-21systemd10:10:011
3511592993729,5cyclictest1-21systemd10:10:011
3511592993728,6cyclictest3664985-21mailstats12:20:571
3511591993723,11cyclictest0-21swapper/011:00:010
3511591993718,12cyclictest151ktimers/007:35:570
3511593993630,3cyclictest0-21swapper/207:10:012
3511593993630,3cyclictest0-21swapper/207:10:012
3511593993629,4cyclictest3605666-21cron10:45:012
351159399362,27cyclictest3640460-21ssh11:40:552
3511592993630,3cyclictest0-21swapper/109:30:571
3511592993629,3cyclictest0-21swapper/109:20:591
3511592993628,5cyclictest421-21dbus-daemon08:00:011
3511592993626,5cyclictest3663575-21idleruntime-cro12:20:011
3511592993624,7cyclictest0-21swapper/112:35:011
3511591993627,6cyclictest3595207-21ssh10:26:030
3511591993627,6cyclictest1000-21mta-sts-daemon10:40:460
351159399354,24cyclictest3673879-21ssh12:35:492
3511593993528,3cyclictest0-21swapper/211:00:502
3511593993528,3cyclictest0-21swapper/211:00:502
3511593993527,5cyclictest3548117-21aten_r2power_po09:10:412
3511593993527,5cyclictest3516115-21systemd-run07:25:012
3511592993525,7cyclictest3578217-21latency_hist10:00:011
3511591993523,9cyclictest3532615-21systemd-run08:20:010
3511593993427,4cyclictest1003-21mta-sts-daemon10:50:042
3511593993426,5cyclictest3627027-21munin-run11:20:012
3511593993426,4cyclictest0-21swapper/209:35:512
3511593993425,6cyclictest3556943-21latency_hist09:25:022
3511593993424,7cyclictest694-21cron08:10:012
3511593993421,10cyclictest3557827-21needreboot09:25:452
3511593993413,15cyclictest421-21dbus-daemon12:10:522
351159299343,25cyclictest3606964-21ssh10:45:541
3511592993427,4cyclictest1003-21mta-sts-daemon10:25:431
3511592993426,5cyclictest3516089-21cron07:25:001
3511592993425,6cyclictest421-21dbus-daemon10:22:101
3511592993425,5cyclictest3525089-21systemd-run07:55:011
3511592993425,5cyclictest3519099-21systemd-run07:35:021
3511592993424,6cyclictest2728660-21cups-browsed07:26:031
3511592993424,5cyclictest3573776-21grep09:51:011
3511592993424,4cyclictest0-21swapper/112:06:001
3511592993418,2cyclictest171rcu_preempt08:00:491
3511593993327,3cyclictest0-21swapper/207:40:012
3511593993326,4cyclictest3539104-21fschecks_count08:40:432
3511593993325,5cyclictest3669877-21latency_hist12:30:032
351159399332,4cyclictest171rcu_preempt08:35:542
351159299339,19cyclictest3551840-21ssh09:15:521
351159299335,23cyclictest3567346-21ssh09:40:561
3511592993325,3cyclictest0-21swapper/112:30:031
3511592993324,5cyclictest3660536-21cron12:15:011
3511592993323,6cyclictest3540086-21munin-plugin-st08:45:011
3511592993322,8cyclictest3521488-21ntp_kernel_pll_07:40:511
3511591993327,4cyclictest3616469-21sed11:01:030
3511591993327,4cyclictest3616469-21sed11:01:030
3511591993326,5cyclictest3526568-21grep08:00:010
3511591993326,4cyclictest3663623-21systemd-run12:20:020
3511591993326,4cyclictest3610432-21unixbench_singl10:51:010
351159399323,23cyclictest3670907-21ssh12:30:542
3511593993226,3cyclictest0-21swapper/212:25:022
3511593993225,3cyclictest0-21swapper/207:25:502
351159299324,23cyclictest3631308-21ssh11:25:531
351159299323,7cyclictest171rcu_preempt11:00:551
351159299323,7cyclictest171rcu_preempt11:00:551
3511592993226,3cyclictest0-21swapper/111:25:011
3511592993225,4cyclictest3547571-21munin-run09:10:011
3511592993224,5cyclictest3512106-21fschecks_time07:10:421
3511592993224,5cyclictest3512106-21fschecks_time07:10:421
3511592993223,6cyclictest3675830-21systemd12:40:031
3511592993223,5cyclictest3543175-21latency_hist08:55:021
3511592993222,6cyclictest428-21mta-sts-daemon11:50:451
3511592993222,6cyclictest428-21mta-sts-daemon11:50:451
351159299322,21cyclictest3652687-21ssh12:01:001
3511592993221,8cyclictest3531060-21cron08:15:011
3511591993225,4cyclictest3636311-21latency_hist11:35:030
3511591993224,5cyclictest3647391-21sshd11:52:030
3511591993224,5cyclictest3647391-21sshd11:52:030
3511591993224,4cyclictest0-21swapper/010:45:520
3511591993222,7cyclictest3672804-21sort12:35:020
3511591993220,9cyclictest3534014-21unixbench_singl08:20:590
3511593993125,3cyclictest0-21swapper/209:10:012
3511593993124,3cyclictest0-21swapper/211:50:502
3511593993124,3cyclictest0-21swapper/211:50:502
3511593993124,3cyclictest0-21swapper/211:20:472
3511593993123,5cyclictest421-21dbus-daemon11:15:022
3511593993123,5cyclictest3593982-21aten_r2power_en10:25:382
3511593993123,5cyclictest3518962-21unixbench_multi07:31:012
351159299313,21cyclictest3514580-21munin-run07:20:011
351159299313,21cyclictest3514580-21munin-run07:20:011
3511591993123,5cyclictest3657922-21/usr/sbin/munin12:10:430
3511591993122,7cyclictest3523578-21/usr/sbin/munin07:50:020
351159199312,23cyclictest3548667-21ssh09:10:520
3511591993119,9cyclictest0-21swapper/009:20:000
3511591993118,11cyclictest0-21swapper/011:30:020
351159399304,12cyclictest3547569-21diskmemload11:59:332
3511593993025,3cyclictest0-21swapper/208:20:012
3511593993024,3cyclictest0-21swapper/210:20:462
3511593993024,3cyclictest0-21swapper/209:50:012
3511593993023,3cyclictest0-21swapper/210:05:492
3511593993023,3cyclictest0-21swapper/210:05:492
351159299307,11cyclictest3570208-21ssh09:45:491
351159299303,20cyclictest3622134-21ssh11:10:551
351159199307,19cyclictest3564112-21ssh09:35:510
3511591993024,4cyclictest3514615-21systemd-run07:20:010
3511591993024,4cyclictest3514615-21systemd-run07:20:010
3511591993023,5cyclictest265-21systemd-journal09:44:080
3511591993023,4cyclictest3534419-21aten_r2power_fr08:25:390
3511591993022,6cyclictest3575200-21sh09:55:020
3511591993020,7cyclictest3667146-21aten_r2power_fr12:25:390
3511592992922,3cyclictest271ktimers/108:45:501
3511592992922,3cyclictest271ktimers/108:45:501
3511592992920,6cyclictest3537006-21unixbench_singl08:31:001
351159199296,6cyclictest14-21ksoftirqd/008:35:540
3511591992923,3cyclictest3560023-21latency_hist09:30:020
3511591992923,3cyclictest0-21swapper/008:35:010
3511591992923,3cyclictest0-21swapper/008:00:530
3511591992922,5cyclictest3531069-21idleruntime-cro08:15:010
3511591992922,4cyclictest421-21dbus-daemon11:41:410
3511591992922,4cyclictest0-21swapper/012:25:010
3511591992921,4cyclictest0-21swapper/011:10:550
3511591992915,11cyclictest3519082-21munin-plugin-st07:35:010
3511593992822,3cyclictest0-21swapper/208:35:012
3511593992821,4cyclictest3524973-21unixbench_multi07:51:022
3511593992821,3cyclictest0-21swapper/211:36:012
3511593992820,5cyclictest3543062-21idleruntime-cro08:55:002
3511593992819,6cyclictest1000-21mta-sts-daemon10:45:362
3511593992816,8cyclictest3534079-21cron08:25:012
3511591992823,3cyclictest0-21swapper/009:46:000
3511591992822,4cyclictest3542088-21fschecks_count08:50:420
3511591992821,4cyclictest3525063-21cron07:55:010
3511591992819,7cyclictest3512987-21unixbench_singl07:10:590
3511591992819,7cyclictest3512987-21unixbench_singl07:10:590
351159199281,22cyclictest3554862-21ntpq09:20:490
3511593992720,4cyclictest1002-21mta-sts-daemon07:40:512
351159199272,18cyclictest3649480-21ssh11:55:510
3511591992720,3cyclictest0-21swapper/012:00:460
3511591992715,3cyclictest0-21swapper/010:25:010
3511593992619,4cyclictest0-21swapper/209:01:032
3511591992619,5cyclictest420-21avahi-daemon10:30:590
3511591992619,5cyclictest0-21swapper/010:40:010
3511591992618,5cyclictest3557020-21kworker/u8:13-xprtiod09:56:000
351159199261,20cyclictest3585521-21ssh10:10:500
351159199261,20cyclictest3585521-21ssh10:10:500
351159399255,16cyclictest3530596-21ps08:10:532
351159399252,17cyclictest665-21snmpd08:55:492
351159399252,17cyclictest3527565-21ntpq08:00:542
3511593992518,4cyclictest3514611-21sendmail07:20:002
3511593992518,4cyclictest3514611-21sendmail07:20:002
3511593992516,5cyclictest2728660-21cups-browsed10:30:372
351159299251,19cyclictest271ktimers/108:05:481
3511591992520,3cyclictest151ktimers/008:40:530
3511591992519,3cyclictest1002-21mta-sts-daemon09:05:500
3511593992415,7cyclictest3541224-21switchtime08:45:572
3511593992415,7cyclictest3541224-21switchtime08:45:572
351159299242,16cyclictest3545502-21ntpq09:00:511
3511591992419,2cyclictest0-21swapper/009:05:000
3511591992418,4cyclictest3540605-21fschecks_time08:45:410
3511591992418,4cyclictest3540605-21fschecks_time08:45:410
3511591992418,4cyclictest3516456-21aten2.4-expect07:25:390
3511591992416,5cyclictest3516115-21systemd-run07:25:010
3511593992317,3cyclictest0-21swapper/208:25:482
3511593992317,3cyclictest0-21swapper/207:45:522
3511592992316,4cyclictest1002-21mta-sts-daemon07:45:371
351159199231,18cyclictest3522001-21sed07:41:010
3511593992212,7cyclictest3512955-21grep07:10:592
3511593992212,7cyclictest3512955-21grep07:10:592
351159199229,10cyclictest12750irq/33-eth008:10:010
351159499184,7cyclictest0-21swapper/311:25:533
351159499181,10cyclictest0-21swapper/311:15:593
3511591991813,2cyclictest0-21swapper/007:10:010
3511591991813,2cyclictest0-21swapper/007:10:010
351159499171,7cyclictest0-21swapper/309:05:513
351159499163,8cyclictest0-21swapper/309:20:553
351159499163,7cyclictest0-21swapper/310:40:013
351159499163,6cyclictest0-21swapper/312:00:513
351159499162,8cyclictest0-21swapper/311:50:483
351159499162,8cyclictest0-21swapper/311:50:483
351159499162,7cyclictest0-21swapper/311:05:483
351159499162,7cyclictest0-21swapper/309:35:493
351159499162,5cyclictest0-21swapper/309:00:533
3511592991611,2cyclictest0-21swapper/107:10:021
3511592991611,2cyclictest0-21swapper/107:10:021
351159499153,7cyclictest0-21swapper/309:40:563
351159499152,7cyclictest0-21swapper/310:50:593
351159499152,6cyclictest0-21swapper/310:20:023
351159499152,5cyclictest0-21swapper/307:55:503
351159499151,8cyclictest0-21swapper/311:55:573
351159499151,8cyclictest0-21swapper/309:30:503
351159499151,6cyclictest0-21swapper/307:10:473
351159499151,6cyclictest0-21swapper/307:10:473
351159499143,8cyclictest0-21swapper/312:35:463
351159499143,6cyclictest0-21swapper/312:16:043
351159499143,6cyclictest0-21swapper/311:45:543
351159499142,8cyclictest0-21swapper/309:15:543
351159499142,7cyclictest0-21swapper/311:30:453
351159499142,7cyclictest0-21swapper/309:55:373
351159499142,7cyclictest0-21swapper/307:40:013
351159499142,6cyclictest0-21swapper/310:25:373
351159499142,5cyclictest0-21swapper/308:45:493
351159499142,5cyclictest0-21swapper/308:45:493
351159499141,8cyclictest0-21swapper/309:50:563
351159499141,7cyclictest0-21swapper/310:45:483
351159499141,7cyclictest0-21swapper/310:20:453
351159499141,6cyclictest0-21swapper/308:50:483
351159499141,6cyclictest0-21swapper/308:30:503
351159499141,6cyclictest0-21swapper/308:25:463
351159499141,6cyclictest0-21swapper/307:45:473
351159499141,6cyclictest0-21swapper/307:30:473
351159499134,6cyclictest0-21swapper/310:31:003
351159499133,7cyclictest0-21swapper/310:05:423
351159499133,7cyclictest0-21swapper/310:05:423
351159499132,7cyclictest0-21swapper/311:35:413
351159499132,7cyclictest0-21swapper/309:10:403
351159499132,6cyclictest0-21swapper/312:05:503
351159499132,6cyclictest0-21swapper/308:55:463
351159499132,6cyclictest0-21swapper/308:20:013
351159499131,8cyclictest0-21swapper/312:29:473
351159499131,8cyclictest0-21swapper/309:45:433
351159499131,7cyclictest0-21swapper/312:20:393
351159499131,7cyclictest0-21swapper/311:20:523
351159499131,7cyclictest0-21swapper/311:10:463
351159499131,7cyclictest0-21swapper/310:55:443
351159499131,7cyclictest0-21swapper/310:40:473
351159499131,6cyclictest0-21swapper/307:16:003
351159499131,6cyclictest0-21swapper/307:16:003
351159499122,7cyclictest0-21swapper/311:00:443
351159499122,7cyclictest0-21swapper/311:00:443
351159499122,6cyclictest0-21swapper/312:10:393
351159499122,6cyclictest0-21swapper/310:00:423
351159499122,6cyclictest0-21swapper/310:00:423
351159499122,6cyclictest0-21swapper/309:25:383
351159499122,6cyclictest0-21swapper/308:15:013
351159499121,7cyclictest0-21swapper/311:40:383
351159499121,7cyclictest0-21swapper/310:10:473
351159499121,7cyclictest0-21swapper/310:10:473
351159499112,6cyclictest0-21swapper/312:30:403
351159499112,5cyclictest0-21swapper/307:25:003
351159499111,6cyclictest0-21swapper/308:05:513
351159499111,6cyclictest0-21swapper/307:40:553
351159499111,6cyclictest0-21swapper/307:30:013
351159499111,5cyclictest0-21swapper/308:20:383
351159499102,5cyclictest0-21swapper/308:40:443
351159499101,6cyclictest0-21swapper/307:50:453
351159499101,5cyclictest0-21swapper/308:35:383
35115949993,4cyclictest0-21swapper/308:00:473
35115949990,5cyclictest0-21swapper/307:10:023
35115949990,5cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional