You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-30 - 08:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Thu Apr 30, 2026 00:43:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3687770996934,19cyclictest1-21systemd19:45:011
3687770996934,19cyclictest1-21systemd19:45:011
3687770996050,5cyclictest0-21swapper/121:25:491
3687769995223,21cyclictest0-21swapper/021:25:560
3687771995144,4cyclictest1056-21mta-sts-daemon19:45:492
3687770994916,19cyclictest0-21swapper/121:33:391
3687771994736,8cyclictest3702741-21sendmail20:00:012
368776999475,11cyclictest3813617-21ssh23:20:480
3687770994636,6cyclictest3723830-21latency_hist21:10:021
3687770994625,18cyclictest0-21swapper/100:15:011
3687770994625,18cyclictest0-21swapper/100:15:011
3687769994636,6cyclictest151ktimers/020:50:460
3687769994621,3cyclictest0-21swapper/000:05:010
3687771994538,4cyclictest1056-21mta-sts-daemon22:41:472
3687771994538,4cyclictest1056-21mta-sts-daemon22:41:472
3687771994535,7cyclictest261-21systemd-journal22:50:272
3687770994537,5cyclictest3809590-21systemd-run23:15:011
3687769994537,5cyclictest3845589-21systemd-run00:10:010
3687769994528,15cyclictest3713236-21cron20:35:000
3687769994528,15cyclictest3713236-21cron20:35:000
3687771994434,6cyclictest3834034-21unixbench_singl23:50:562
3687771994424,9cyclictest1017-21sendmail-mta20:05:452
368777099444,34cyclictest3813628-21ssh23:20:491
3687769994428,14cyclictest3792554-21munin-plugin-st22:50:000
3687769994428,14cyclictest3792554-21munin-plugin-st22:50:000
3687771994336,4cyclictest0-21swapper/223:15:012
3687770994334,6cyclictest3773186-21latency_hist22:20:021
368777099433,32cyclictest3702230-21ntpq19:55:541
3687770994324,15cyclictest3710257-21latency_hist20:25:011
368777099431,4cyclictest3699182-21ntpq19:45:451
3687769994336,4cyclictest3856188-21sh00:25:000
3687769994336,4cyclictest151ktimers/021:05:440
368777199425,28cyclictest3733107-21ssh21:20:492
3687771994232,6cyclictest3745285-21sed21:35:572
3687771994232,6cyclictest3745285-21sed21:35:572
368777099429,30cyclictest3791712-21ssh22:48:181
368777099429,30cyclictest3791712-21ssh22:48:181
368777099423,33cyclictest3752223-21ntpq21:45:501
368777099423,33cyclictest3752223-21ntpq21:45:501
3687770994220,19cyclictest421-21dbus-daemon21:15:011
3687770994217,22cyclictest3704262-21idleruntime-cro20:05:011
368777099421,10cyclictest3838393-21ssh23:59:081
368777099421,10cyclictest3830352-21ssh23:46:251
3687769994221,3cyclictest0-21swapper/020:45:010
3687769994221,2cyclictest0-21swapper/023:40:010
3687769994219,3cyclictest3748502-21ssh21:40:490
3687769994216,8cyclictest12350irq/32-eth021:10:450
3687771994134,4cyclictest695-21cron23:25:012
3687770994134,4cyclictest3713261-21kworker/u8:0+xprtiod20:35:011
3687770994134,4cyclictest3713261-21kworker/u8:0+xprtiod20:35:011
3687770994133,5cyclictest0-21swapper/121:25:011
368777099412,4cyclictest3691736-21ntpq19:20:461
368777099412,34cyclictest3826855-21ntpq23:40:511
368777099412,34cyclictest3694723-21ntpq19:30:521
368777099412,34cyclictest3694723-21ntpq19:30:521
368777099412,33cyclictest3706665-21ntpq20:10:471
368777099411,10cyclictest3760330-21ssh21:59:471
3687769994134,5cyclictest1-21systemd22:10:020
3687769994129,10cyclictest3711768-21systemd-run20:30:010
3687771994033,4cyclictest1055-21mta-sts-daemon22:27:032
3687771994031,6cyclictest1895805-21cups-browsed23:41:092
368777099409,29cyclictest3731646-21ssh21:19:531
368777099409,27cyclictest3747647-21cat21:40:331
368777099403,31cyclictest3761611-21ntpq22:00:441
3687770994031,6cyclictest3770477-21ssh22:15:011
368777099402,5cyclictest3705125-21ntpq20:05:441
368777099402,34cyclictest3780763-21ntpq22:30:471
368777099402,34cyclictest3717129-21ntpq20:45:441
368777099402,34cyclictest3690116-21ntpq19:15:431
368777099402,31cyclictest3820189-21ssh23:30:481
368777099402,29cyclictest428-21mta-sts-daemon19:25:491
3687770994018,20cyclictest0-21swapper/123:00:011
3687770994018,18cyclictest0-21swapper/122:40:431
3687770994018,18cyclictest0-21swapper/122:40:431
368777099401,7cyclictest3757514-21sshd21:53:271
368777099401,7cyclictest3757514-21sshd21:53:271
368777099401,5cyclictest3715726-21ntpq20:40:501
368777099401,4cyclictest3700718-21ntpq19:50:501
368777099401,33cyclictest3816717-21sh23:25:351
368777099401,33cyclictest3718769-21ps20:50:501
368777099401,13cyclictest3744608-21ntpq21:35:441
368777099401,13cyclictest3744608-21ntpq21:35:441
368777099401,11cyclictest3714778-21systemd-run20:40:011
368777199393,26cyclictest3846895-21ssh00:10:472
368777199393,26cyclictest3846895-21ssh00:10:472
3687771993931,5cyclictest3761238-21fschecks_count22:00:382
3687771993931,5cyclictest3754592-21latency_hist21:50:012
3687771993931,5cyclictest3754592-21latency_hist21:50:012
3687771993931,4cyclictest1054-21mta-sts-daemon22:22:182
3687771993928,5cyclictest0-21swapper/223:30:462
368777099398,28cyclictest3777432-21ssh22:25:421
3687770993931,5cyclictest3776889-21munin-plugin-st22:25:011
3687770993930,6cyclictest3840038-21needreboot00:00:431
368777099392,4cyclictest3711136-21ntpq20:25:441
368777099392,33cyclictest3721637-21ntpq21:00:451
368777099392,33cyclictest3696177-21ntpq19:35:471
368777099392,33cyclictest3696177-21ntpq19:35:471
368777099391,9cyclictest3843140-21ssh00:05:101
368777099391,8cyclictest3800497-21ntpq23:00:561
368777099391,5cyclictest3708232-21ntpq20:15:471
3687770993915,18cyclictest0-21swapper/119:10:401
368777099391,4cyclictest3853529-21ntpq00:20:471
368777099391,11cyclictest3785537-21ssh22:39:071
368777099391,11cyclictest3785537-21ssh22:39:071
368777099391,10cyclictest3823280-21ssh23:35:411
3687769993932,4cyclictest3760429-21cron22:00:000
3687769993932,4cyclictest3693989-21kworker/u8:17-rpciod20:50:010
368776999392,18cyclictest3740748-21ssh21:30:430
3687771993829,6cyclictest261-21systemd-journal23:10:012
3687771993829,5cyclictest3852718-21acpi00:20:332
3687771993825,10cyclictest3755498-21kernelversion21:50:402
3687771993825,10cyclictest3755498-21kernelversion21:50:402
368777099387,29cyclictest3850704-21ssh00:16:551
368777099385,29cyclictest3856917-21ssh00:26:041
368777099381,9cyclictest3859390-21/usr/sbin/munin00:30:581
368777099381,9cyclictest3804942-21ssh23:08:331
368777099381,11cyclictest3863693-21ssh00:36:121
368777099381,11cyclictest3793749-21ntpq22:50:431
368777099381,11cyclictest3764451-21ssh22:05:131
368777099381,10cyclictest3720124-21ntpq20:55:471
368777199374,27cyclictest3800506-21ssh23:00:562
3687771993729,5cyclictest3832166-21cron23:50:012
3687771993729,5cyclictest3786003-21ssh22:40:012
3687771993729,5cyclictest3786003-21ssh22:40:012
3687771993728,5cyclictest1055-21mta-sts-daemon22:19:242
368777099371,9cyclictest3832438-21ssh23:50:071
368777099371,8cyclictest3810649-21ssh23:16:471
3687769993730,4cyclictest3770448-21cron22:15:010
3687769993730,3cyclictest0-21swapper/022:50:460
3687769993729,5cyclictest1-21systemd23:55:010
368777199363,28cyclictest3767826-21ssh22:10:432
3687771993629,4cyclictest0-21swapper/200:40:012
3687771993627,5cyclictest3701247-21idleruntime-cro19:55:012
3687771993626,6cyclictest421-21dbus-daemon21:14:552
3687769993631,3cyclictest3799090-21ls23:00:000
3687769993629,5cyclictest0-21swapper/021:00:010
368777199353,26cyclictest659-21snmpd21:30:472
3687771993528,3cyclictest0-21swapper/200:33:092
3687771993527,5cyclictest3852209-21idleruntime-cro00:20:002
368777099351,4cyclictest3687724-21munin-run19:10:001
368777099351,4cyclictest3687724-21munin-run19:10:001
3687769993529,4cyclictest1895805-21cups-browsed21:15:390
3687769993528,4cyclictest3722443-21latency_hist21:05:020
3687769993528,3cyclictest0-21swapper/022:20:440
3687771993428,3cyclictest0-21swapper/221:40:422
3687771993427,4cyclictest3760438-21cut22:00:002
3687771993425,6cyclictest261-21systemd-journal21:10:012
3687771993422,9cyclictest3691679-21ntp_kernel_pll_19:20:442
368777199341,25cyclictest3736972-21sh21:25:482
368776999343,26cyclictest3820276-21ssh23:30:510
3687769993428,4cyclictest3789861-21cron22:45:000
3687769993428,4cyclictest3789861-21cron22:45:000
3687769993427,5cyclictest421-21dbus-daemon21:40:000
3687769993427,5cyclictest421-21dbus-daemon21:40:000
3687769993424,4cyclictest0-21swapper/020:00:460
3687771993326,4cyclictest3818917-21munin-run23:30:012
3687771993325,5cyclictest3695487-21/usr/sbin/munin19:35:382
3687771993325,5cyclictest3695487-21/usr/sbin/munin19:35:382
3687771993325,4cyclictest1051-21mta-sts-daemon23:16:012
368776999336,23cyclictest0-21swapper/019:10:000
368776999336,23cyclictest0-21swapper/019:10:000
3687769993328,3cyclictest3735688-21kworker/u8:15-xprtiod21:55:000
3687769993328,3cyclictest3735688-21kworker/u8:15-xprtiod21:55:000
3687769993327,3cyclictest3743443-21kworker/u8:4-xprtiod22:40:010
3687769993327,3cyclictest3743443-21kworker/u8:4-xprtiod22:40:010
3687769993326,5cyclictest421-21dbus-daemon00:30:010
3687769993325,5cyclictest421-21dbus-daemon19:35:020
3687769993325,5cyclictest421-21dbus-daemon19:35:020
3687769993322,7cyclictest420-21avahi-daemon22:29:370
368776999332,23cyclictest3807044-21ssh23:10:510
3687769993320,3cyclictest1054-21mta-sts-daemon00:36:380
3687771993226,3cyclictest0-21swapper/200:00:012
3687771993224,3cyclictest0-21swapper/200:00:482
3687769993226,4cyclictest3799943-21sh23:00:410
3687769993224,6cyclictest0-21swapper/000:30:300
3687769993224,5cyclictest1895805-21cups-browsed23:25:120
3687769993223,7cyclictest3696746-21sh19:40:000
3687769993223,7cyclictest3696746-21sh19:40:000
3687769993222,8cyclictest3846575-21latency00:10:390
3687769993222,8cyclictest3846575-21latency00:10:390
3687771993123,4cyclictest1054-21mta-sts-daemon20:15:422
3687771993121,7cyclictest439-21systemd-logind20:40:012
3687769993125,3cyclictest0-21swapper/021:20:560
3687769993124,4cyclictest3702455-21threads19:55:590
3687769993122,6cyclictest1-21systemd23:20:010
3687769993121,8cyclictest0-21swapper/021:45:530
3687769993121,8cyclictest0-21swapper/021:45:530
3687769993116,10cyclictest12350irq/32-eth023:40:440
3687771993023,3cyclictest0-21swapper/200:30:012
3687771993020,7cyclictest0-21swapper/219:25:372
3687769993024,4cyclictest3781380-21/usr/sbin/munin22:30:580
3687769993020,6cyclictest151ktimers/019:10:450
368777199298,5cyclictest62450lldpd22:07:302
3687771992922,4cyclictest3721279-21fschecks_time21:00:382
3687771992922,3cyclictest0-21swapper/222:30:352
3687771992921,5cyclictest3690754-21munin-plugin-st19:20:002
3687769992920,7cyclictest1895805-21cups-browsed23:05:210
368776999291,21cyclictest3761796-21ssh22:00:510
368777199285,15cyclictest428-21mta-sts-daemon20:25:442
3687771992822,3cyclictest0-21swapper/223:00:002
3687771992821,3cyclictest0-21swapper/222:47:132
3687771992821,3cyclictest0-21swapper/222:47:132
3687771992821,3cyclictest0-21swapper/221:15:472
3687771992820,5cyclictest3709159-21aten_r2power_vo20:20:352
3687771992820,5cyclictest3687781-21systemd-run19:10:012
3687771992820,5cyclictest3687781-21systemd-run19:10:012
3687771992820,5cyclictest0-21swapper/200:10:032
3687771992817,8cyclictest3718617-21ntp_kernel_pll_20:50:442
3687771992816,9cyclictest3712596-21ntp_kernel_err20:30:412
3687771992816,9cyclictest3712596-21ntp_kernel_err20:30:412
3687771992811,12cyclictest421-21dbus-daemon20:10:512
3687771992722,2cyclictest0-21swapper/220:50:022
3687771992719,5cyclictest3716252-21idleruntime-cro20:45:012
3687771992718,5cyclictest3698216-21unixbench_singl19:40:562
3687771992718,5cyclictest3698216-21unixbench_singl19:40:562
3687771992718,4cyclictest0-21swapper/223:38:462
3687771992716,8cyclictest3719913-21latency20:55:422
3687769992722,3cyclictest0-21swapper/000:20:000
3687769992721,3cyclictest3707395-21latency_hist20:15:020
3687769992720,4cyclictest3689788-21fschecks_time19:15:380
3687769992718,7cyclictest3699106-21ntp_kernel_err19:45:430
3687769992620,4cyclictest3705675-21head20:05:570
3687769992620,4cyclictest1051-21mta-sts-daemon19:41:420
3687769992620,4cyclictest1051-21mta-sts-daemon19:41:420
3687769992619,4cyclictest3747430-21kworker/u8:7-xprtiod22:20:010
3687769992618,5cyclictest0-21swapper/020:40:000
3687769992615,8cyclictest0-21swapper/000:00:010
3687771992520,2cyclictest0-21swapper/220:05:012
3687771992518,3cyclictest0-21swapper/219:10:502
3687769992519,3cyclictest1055-21mta-sts-daemon19:25:450
3687769992518,5cyclictest1895805-21cups-browsed19:50:080
3687771992417,4cyclictest3694096-21aten_r2power_fr19:30:362
3687771992417,4cyclictest3694096-21aten_r2power_fr19:30:362
368776999244,16cyclictest151ktimers/019:20:470
3687769992418,3cyclictest0-21swapper/023:49:020
3687769992310,11cyclictest151ktimers/020:15:440
3687769992213,6cyclictest3709075-21telnet20:20:340
368777299202,9cyclictest0-21swapper/320:15:433
368777299182,7cyclictest0-21swapper/319:20:473
368777299172,8cyclictest0-21swapper/323:00:563
368777299172,8cyclictest0-21swapper/300:10:463
368777299172,8cyclictest0-21swapper/300:10:463
368777299171,9cyclictest0-21swapper/322:30:543
368777299171,10cyclictest0-21swapper/322:20:533
368777299162,9cyclictest0-21swapper/321:40:453
368777299162,7cyclictest0-21swapper/320:05:453
368777299161,7cyclictest0-21swapper/319:40:433
368777299161,7cyclictest0-21swapper/319:40:433
368777299161,6cyclictest0-21swapper/320:25:443
368777299154,6cyclictest0-21swapper/321:15:473
368777299153,7cyclictest0-21swapper/321:25:453
368777299152,9cyclictest0-21swapper/321:30:363
368777299152,8cyclictest0-21swapper/321:45:473
368777299152,8cyclictest0-21swapper/321:45:473
368777299152,7cyclictest0-21swapper/321:20:463
368777299152,7cyclictest0-21swapper/300:30:343
368777299151,8cyclictest0-21swapper/323:30:353
368777299151,8cyclictest0-21swapper/322:40:423
368777299151,8cyclictest0-21swapper/322:40:423
368777299151,8cyclictest0-21swapper/321:50:503
368777299151,8cyclictest0-21swapper/321:50:503
368777299151,7cyclictest0-21swapper/320:20:483
368777299151,6cyclictest0-21swapper/320:10:453
368777299144,7cyclictest0-21swapper/321:10:433
368777299143,8cyclictest0-21swapper/321:35:453
368777299143,8cyclictest0-21swapper/321:35:453
368777299143,7cyclictest0-21swapper/322:10:483
368777299142,8cyclictest0-21swapper/323:40:473
368777299142,8cyclictest0-21swapper/322:50:333
368777299142,8cyclictest0-21swapper/321:00:443
368777299142,7cyclictest0-21swapper/323:10:373
368777299142,7cyclictest0-21swapper/322:00:483
368777299141,8cyclictest0-21swapper/323:05:553
368777299141,8cyclictest0-21swapper/300:20:453
368777299141,7cyclictest0-21swapper/320:35:453
368777299141,7cyclictest0-21swapper/319:25:443
368777299141,6cyclictest0-21swapper/320:40:513
368777299133,7cyclictest0-21swapper/320:50:553
368777299133,6cyclictest0-21swapper/323:40:013
368777299132,7cyclictest0-21swapper/323:19:283
368777299132,6cyclictest0-21swapper/300:00:333
368777299131,8cyclictest0-21swapper/323:20:433
368777299131,7cyclictest0-21swapper/320:45:433
368777299131,7cyclictest0-21swapper/320:00:483
368777299131,7cyclictest0-21swapper/319:55:493
368777299131,6cyclictest0-21swapper/322:40:033
368777299131,6cyclictest0-21swapper/322:40:033
368777299131,6cyclictest0-21swapper/319:45:423
368777299131,6cyclictest0-21swapper/319:30:523
368777299131,6cyclictest0-21swapper/319:30:523
368777299123,5cyclictest0-21swapper/319:40:003
368777299123,5cyclictest0-21swapper/319:40:003
368777299122,7cyclictest0-21swapper/323:50:463
368777299122,7cyclictest0-21swapper/300:39:383
368777299122,6cyclictest0-21swapper/322:48:283
368777299122,6cyclictest0-21swapper/322:48:283
368777299122,6cyclictest0-21swapper/322:00:003
368777299122,6cyclictest0-21swapper/321:00:013
368777299122,6cyclictest0-21swapper/300:29:183
368777299122,6cyclictest0-21swapper/300:00:013
368777299122,5cyclictest0-21swapper/323:30:033
368777299122,5cyclictest0-21swapper/319:50:393
368777299121,6cyclictest0-21swapper/319:10:433
368777299121,5cyclictest0-21swapper/323:50:023
368777299120,7cyclictest0-21swapper/322:10:023
368777299112,5cyclictest0-21swapper/321:05:453
368777299111,7cyclictest0-21swapper/300:16:053
368777299111,5cyclictest0-21swapper/322:20:033
368777299111,5cyclictest0-21swapper/320:30:323
368777299111,5cyclictest0-21swapper/320:30:323
368777299111,4cyclictest0-21swapper/322:30:033
368777299102,6cyclictest0-21swapper/322:58:243
368777299101,6cyclictest0-21swapper/300:10:033
36877729992,5cyclictest0-21swapper/319:15:433
36877729992,5cyclictest0-21swapper/319:10:013
36877729992,5cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional