You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-23 - 18:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Thu Apr 23, 2026 12:43:17)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3441326996338,21cyclictest439-21systemd-logind11:30:022
3441326996026,8cyclictest3524016-21basename10:10:012
3441325995641,5cyclictest0-21swapper/112:25:431
3441324995441,7cyclictest0-21swapper/007:10:400
3441324994936,6cyclictest0-21swapper/009:54:430
3441324994837,9cyclictest695-21cron12:20:010
3441326994739,5cyclictest3531564-21munin-plugin-st10:20:012
3441326994737,6cyclictest3626212-21unin-run12:25:012
3441326994631,12cyclictest3517868-21mailstats10:00:482
3441325994636,6cyclictest3631988-21unixbench_multi12:31:021
3441325994636,6cyclictest3631988-21unixbench_multi12:31:021
3441324994628,15cyclictest3448920-21latency_hist07:35:020
3441325994535,4cyclictest0-21swapper/110:40:391
3441325994535,4cyclictest0-21swapper/110:40:391
344132499451,37cyclictest3521392-21ssh10:05:400
3441326994436,5cyclictest439-21systemd-logind12:00:012
3441325994434,6cyclictest3459332-21systemd-run08:10:011
3441324994438,4cyclictest3501400-21grep09:40:010
3441324994438,4cyclictest3501400-21grep09:40:010
3441324994428,13cyclictest0-21swapper/010:20:010
3441326994333,6cyclictest3603404-21cron11:55:012
3441326994333,6cyclictest3603404-21cron11:55:012
3441325994335,5cyclictest3441335-21systemd-run07:10:011
3441325994334,6cyclictest3588274-21unin-run11:35:011
3441324994323,12cyclictest151ktimers/011:45:450
3441325994234,4cyclictest0-21swapper/110:10:461
344132499424,28cyclictest3600851-21ssh11:50:420
344132499424,28cyclictest3600851-21ssh11:50:420
3441326994132,6cyclictest3481402-21systemd-run09:15:022
3441326994132,6cyclictest3481402-21systemd-run09:15:022
3441326994129,8cyclictest420-21avahi-daemon12:30:522
3441326994129,8cyclictest420-21avahi-daemon12:30:522
3441325994133,5cyclictest3494053-21aten_r2power_vo09:30:321
3441325994132,6cyclictest3530030-21kworker/u8:3+xprtiod10:25:011
3441325994126,6cyclictest3511693-21sh09:53:021
3441324994134,4cyclictest3477837-21aten_r2power_en09:10:280
3441324994134,4cyclictest3477837-21aten_r2power_en09:10:280
3441324994134,3cyclictest0-21swapper/011:05:000
3441324994127,8cyclictest0-21swapper/009:25:440
3441326994034,3cyclictest0-21swapper/209:35:482
3441326994034,3cyclictest0-21swapper/209:35:482
3441326994032,5cyclictest3565556-21munin-plugin-st11:05:002
3441326994032,5cyclictest3561806-21systemd-run11:00:012
3441325994033,4cyclictest0-21swapper/112:00:001
3441325994033,3cyclictest0-21swapper/107:35:011
344132499403,30cyclictest3555412-21ssh10:50:410
344132499403,30cyclictest3555412-21ssh10:50:410
3441324994027,6cyclictest12350irq/32-eth008:40:440
3441324994026,9cyclictest12350irq/32-eth010:45:440
3441324994026,9cyclictest12350irq/32-eth010:45:440
3441326993929,7cyclictest421-21dbus-daemon10:20:492
344132699392,31cyclictest3597089-21ssh11:45:432
3441324993931,4cyclictest0-21swapper/011:55:390
3441324993926,10cyclictest0-21swapper/010:55:480
344132499392,30cyclictest3589456-21ssh11:35:420
344132699386,22cyclictest428-21mta-sts-daemon10:35:442
344132699386,22cyclictest428-21mta-sts-daemon10:35:442
344132699384,30cyclictest3513861-21sshd09:55:432
3441326993830,5cyclictest3628050-21unixbench_singl12:25:542
3441326993830,5cyclictest0-21swapper/207:40:012
3441326993829,6cyclictest421-21dbus-daemon10:30:012
3441326993829,6cyclictest3592098-21systemd-run11:40:022
3441326993829,5cyclictest3614792-21systemd-run12:10:012
3441326993823,8cyclictest3477308-21diskmemload10:50:442
3441326993823,8cyclictest3477308-21diskmemload10:50:442
3441325993832,3cyclictest0-21swapper/112:15:011
3441325993831,4cyclictest0-21swapper/112:15:431
3441325993830,5cyclictest3637962-21cron12:40:011
3441325993830,5cyclictest3554241-21systemd-run10:50:011
3441325993830,5cyclictest3554241-21systemd-run10:50:011
3441325993829,6cyclictest3542922-21idleruntime-cro10:35:001
3441325993829,6cyclictest3542922-21idleruntime-cro10:35:001
3441325993827,7cyclictest420-21avahi-daemon11:35:161
3441325993826,9cyclictest3477308-21diskmemload11:05:011
3441325993825,8cyclictest0-21swapper/111:55:011
3441325993825,8cyclictest0-21swapper/111:55:011
344132499383,28cyclictest3612315-21ssh12:05:460
3441324993829,7cyclictest3474310-21sendmail09:00:000
344132499380,17cyclictest3495860-21ssh09:32:000
3441326993729,5cyclictest3608977-21unixbench_multi12:00:522
3441326993728,6cyclictest421-21dbus-daemon09:30:052
344132699372,26cyclictest3482699-21ssh09:15:482
3441325993728,6cyclictest1056-21mta-sts-daemon10:55:461
3441325993728,5cyclictest0-21swapper/109:25:421
344132599372,15cyclictest3578041-21ssh11:20:401
3441324993725,7cyclictest0-21swapper/010:00:460
344132499371,27cyclictest3570563-21ssh11:10:420
3441326993628,5cyclictest3473316-21fschecks_count08:55:332
3441326993628,5cyclictest0-21swapper/207:20:012
3441326993628,4cyclictest0-21swapper/210:45:502
3441326993628,4cyclictest0-21swapper/210:45:502
3441326993627,6cyclictest420-21avahi-daemon10:10:412
3441326993623,6cyclictest0-21swapper/211:30:392
3441325993629,4cyclictest3573161-21latency_hist11:15:021
3441325993628,5cyclictest3520207-21cron10:05:011
3441325993627,4cyclictest0-21swapper/107:25:431
3441325993625,8cyclictest3450339-21systemd-run07:40:011
3441325993624,9cyclictest3608130-21kernelversion12:00:351
3441325993623,9cyclictest3536248-21/usr/sbin/munin10:25:361
3441325993622,10cyclictest3543687-21sh10:35:331
3441325993622,10cyclictest3543687-21sh10:35:331
3441325993617,9cyclictest3477308-21diskmemload11:45:441
3441326993529,3cyclictest1051-21mta-sts-daemon09:50:412
3441326993527,5cyclictest3574361-21grep11:15:432
3441326993525,7cyclictest3616544-21tail12:10:522
344132599353,28cyclictest3502665-21ssh09:40:481
3441325993524,7cyclictest3498880-21sendmail_mailqu09:35:481
3441325993524,7cyclictest3498880-21sendmail_mailqu09:35:481
3441325993516,7cyclictest3593354-21ntpq11:40:451
3441324993522,9cyclictest2619226-21cups-browsed12:30:520
3441324993522,9cyclictest2619226-21cups-browsed12:30:520
3441324993520,4cyclictest3462192-21tail08:15:530
344132599343,24cyclictest3566834-21ssh11:05:441
3441325993427,4cyclictest3443324-21fschecks_count07:15:351
3441325993427,3cyclictest0-21swapper/112:05:541
3441325993427,3cyclictest0-21swapper/109:10:011
3441325993427,3cyclictest0-21swapper/108:20:431
3441325993426,5cyclictest3466795-21cron08:35:001
3441325993423,8cyclictest3581789-21ntp_kernel_err11:25:391
344132599342,29cyclictest3513814-21ssh09:55:411
3441324993428,4cyclictest3441323-21cyclictest12:10:510
3441324993427,5cyclictest261-21systemd-journal11:20:010
3441324993422,10cyclictest3578142-21ntp_kernel_pll_11:20:460
344132699334,24cyclictest3502629-21ssh09:40:472
3441326993325,5cyclictest3469835-21systemd-run08:45:012
3441326993323,6cyclictest1-21systemd07:50:012
3441325993326,3cyclictest0-21swapper/112:20:521
3441325993321,7cyclictest2619226-21cups-browsed09:47:451
3441325993316,4cyclictest171rcu_preempt09:20:421
3441325993311,3cyclictest171rcu_preempt08:10:411
344132499339,15cyclictest12350irq/32-eth010:25:420
344132499338,19cyclictest3593264-21ssh11:40:420
3441324993327,4cyclictest3566043-21aten_r2power_cu11:05:310
3441324993327,4cyclictest3485349-21idleruntime-cro09:20:000
3441324993327,3cyclictest0-21swapper/011:30:010
3441324993327,3cyclictest0-21swapper/008:05:420
3441324993325,6cyclictest3455722-21sshd07:55:430
3441324993323,7cyclictest3452811-21sshd07:45:440
344132499332,24cyclictest3623486-21diskmemload12:20:390
344132499330,4cyclictest171rcu_preempt09:20:420
3441326993226,3cyclictest0-21swapper/208:25:522
3441326993225,3cyclictest0-21swapper/209:30:012
3441326993224,5cyclictest3569334-21/usr/sbin/munin11:10:012
344132699322,24cyclictest3593312-21ssh11:40:432
344132599327,4cyclictest171rcu_preempt07:40:481
344132599325,17cyclictest3574326-21ssh11:15:421
3441325993224,5cyclictest3456342-21systemd-run08:00:021
3441325993224,5cyclictest2619226-21cups-browsed10:08:071
3441325993213,3cyclictest171rcu_preempt10:54:251
3441325993213,3cyclictest171rcu_preempt10:54:251
344132599321,23cyclictest3482728-21sh09:15:501
344132599320,9cyclictest28-21ksoftirqd/109:00:441
344132499327,20cyclictest624-21lldpd08:20:410
344132499325,20cyclictest12350irq/32-eth010:20:410
3441324993226,4cyclictest3611000-21latency_hist12:05:000
3441324993226,4cyclictest3447315-21idleruntime-cro07:30:010
3441324993224,5cyclictest548-21gdbus12:35:570
3441324993224,5cyclictest420-21avahi-daemon11:32:180
3441324993220,9cyclictest439-21systemd-logind08:15:010
3441326993126,3cyclictest3441323-21cyclictest09:20:372
3441326993125,3cyclictest0-21swapper/212:20:012
3441326993125,3cyclictest0-21swapper/209:50:012
3441326993123,5cyclictest0-21swapper/212:40:012
3441326993123,4cyclictest0-21swapper/208:35:012
3441326993121,7cyclictest3468333-21sh08:40:012
3441325993123,5cyclictest3474311-21sendmail09:00:001
3441325993121,5cyclictest0-21swapper/110:15:181
3441324993125,3cyclictest0-21swapper/008:25:430
3441324993124,4cyclictest0-21swapper/010:10:360
3441324993123,5cyclictest3540916-21grep10:30:520
3441324993123,5cyclictest3540916-21grep10:30:520
344132499310,18cyclictest0-21swapper/009:45:140
3441326993024,3cyclictest0-21swapper/208:15:012
3441326993023,3cyclictest0-21swapper/210:40:522
3441326993023,3cyclictest0-21swapper/210:40:522
3441326993022,5cyclictest3543130-21latency_hist10:35:022
3441326993022,5cyclictest3543130-21latency_hist10:35:022
3441326993020,6cyclictest3447399-21kworker/u8:9-xprtiod07:50:392
3441325993023,5cyclictest271ktimers/107:10:451
3441325993021,4cyclictest0-21swapper/109:12:291
3441325993021,4cyclictest0-21swapper/109:12:291
3441325993020,7cyclictest3470683-21ntpq08:45:431
3441325993020,7cyclictest3470683-21ntpq08:45:431
3441325993018,3cyclictest171rcu_preempt08:50:441
3441325993018,3cyclictest171rcu_preempt07:20:441
3441324993023,5cyclictest3445500-21switchtime07:20:500
3441324993023,3cyclictest0-21swapper/012:25:320
3441324993020,8cyclictest0-21swapper/010:35:430
3441324993020,8cyclictest0-21swapper/010:35:430
344132499301,3cyclictest171rcu_preempt09:55:410
3441326992923,3cyclictest0-21swapper/208:20:012
3441326992922,4cyclictest0-21swapper/207:25:012
344132699292,20cyclictest3578036-21ssh11:20:402
344132699291,20cyclictest420-21avahi-daemon07:10:402
3441325992924,4cyclictest3441323-21cyclictest07:45:341
3441325992921,6cyclictest420-21avahi-daemon08:40:331
3441325992916,9cyclictest271ktimers/108:35:401
3441325992914,2cyclictest171rcu_preempt08:15:461
3441324992912,14cyclictest421-21dbus-daemon08:50:050
3441326992823,2cyclictest0-21swapper/208:00:012
3441326992821,4cyclictest3571133-21sed11:10:522
3441326992821,4cyclictest3471946-21kernelversion08:50:372
3441326992820,5cyclictest439-21systemd-logind07:30:012
3441326992818,7cyclictest3475830-21sh09:05:012
3441325992821,4cyclictest3454389-21mailstats07:50:471
3441325992819,6cyclictest420-21avahi-daemon08:00:111
344132499288,3cyclictest171rcu_preempt08:30:440
3441324992822,4cyclictest3454809-21munin-plugin-st07:55:000
3441324992821,5cyclictest3475805-21cron09:05:000
3441324992821,5cyclictest3443330-21fschecks_time07:15:350
3441324992821,4cyclictest0-21swapper/010:41:250
3441324992821,4cyclictest0-21swapper/010:41:250
344132499281,2cyclictest0-21swapper/007:35:380
3441326992721,3cyclictest0-21swapper/207:45:012
3441326992718,5cyclictest3456716-21aten_r2power_vo08:00:312
3441325992714,3cyclictest171rcu_preempt08:25:391
344132499278,10cyclictest171rcu_preempt08:35:320
344132499273,13cyclictest1017-21sendmail-mta08:00:420
3441324992721,3cyclictest0-21swapper/009:45:020
344132499269,14cyclictest3470322-21fschecks_count08:45:360
344132499269,14cyclictest3470322-21fschecks_count08:45:360
3441326992417,4cyclictest3477340-21ssh09:10:022
3441326992410,9cyclictest3447293-21kworker/2:0-mm_percpu_wq07:30:442
3441326992317,3cyclictest0-21swapper/208:45:342
3441326992317,3cyclictest0-21swapper/208:45:342
3441326992316,5cyclictest3458146-21aten_r2power_fr08:05:302
344132499232,15cyclictest3451307-21ntpq07:40:480
3441324992310,10cyclictest3477312-21cut09:10:010
344132699211,15cyclictest3463165-21ntpq08:20:392
344132699191,14cyclictest421-21dbus-daemon07:10:012
3441324991814,2cyclictest0-21swapper/007:10:020
344132799173,8cyclictest0-21swapper/311:15:423
344132799172,9cyclictest0-21swapper/311:05:413
344132799172,8cyclictest0-21swapper/310:20:393
344132799172,8cyclictest0-21swapper/309:25:423
344132799163,8cyclictest0-21swapper/308:05:423
344132799163,7cyclictest0-21swapper/311:55:393
344132799163,7cyclictest0-21swapper/310:30:423
344132799163,7cyclictest0-21swapper/310:30:423
344132799163,7cyclictest0-21swapper/309:35:393
344132799163,7cyclictest0-21swapper/309:35:393
344132799162,8cyclictest0-21swapper/312:35:423
344132799162,8cyclictest0-21swapper/311:50:393
344132799162,8cyclictest0-21swapper/311:50:393
344132799162,8cyclictest0-21swapper/309:30:453
344132799162,8cyclictest0-21swapper/309:15:443
344132799162,7cyclictest0-21swapper/308:15:413
344132799161,8cyclictest0-21swapper/310:15:413
344132799161,6cyclictest0-21swapper/307:30:423
344132799154,6cyclictest0-21swapper/309:45:513
344132799154,6cyclictest0-21swapper/307:20:023
344132799153,8cyclictest0-21swapper/310:40:433
344132799153,8cyclictest0-21swapper/310:40:433
344132799153,8cyclictest0-21swapper/310:10:413
344132799153,6cyclictest0-21swapper/312:25:413
344132799152,8cyclictest0-21swapper/311:30:403
344132799152,8cyclictest0-21swapper/309:55:413
344132799152,8cyclictest0-21swapper/307:50:393
344132799152,7cyclictest0-21swapper/308:45:463
344132799152,7cyclictest0-21swapper/308:45:463
344132799152,6cyclictest0-21swapper/308:40:403
344132799151,8cyclictest0-21swapper/311:20:453
344132799151,8cyclictest0-21swapper/310:50:453
344132799151,8cyclictest0-21swapper/310:50:453
344132799151,7cyclictest0-21swapper/308:25:463
344132799151,7cyclictest0-21swapper/308:10:463
344132799151,7cyclictest0-21swapper/307:25:423
344132799151,6cyclictest0-21swapper/307:55:433
344132799144,6cyclictest0-21swapper/310:45:433
344132799144,6cyclictest0-21swapper/310:45:433
344132799143,7cyclictest0-21swapper/312:20:433
344132799143,7cyclictest0-21swapper/308:00:313
344132799143,5cyclictest0-21swapper/310:25:313
344132799142,8cyclictest0-21swapper/312:00:343
344132799142,8cyclictest0-21swapper/309:50:413
344132799142,7cyclictest0-21swapper/311:25:393
344132799142,7cyclictest0-21swapper/309:20:413
344132799142,6cyclictest0-21swapper/312:10:383
344132799141,9cyclictest0-21swapper/310:35:343
344132799141,9cyclictest0-21swapper/310:35:343
344132799141,8cyclictest0-21swapper/312:30:333
344132799141,8cyclictest0-21swapper/312:30:333
344132799141,8cyclictest0-21swapper/312:05:473
344132799141,7cyclictest0-21swapper/312:15:373
344132799141,7cyclictest0-21swapper/311:45:353
344132799141,7cyclictest0-21swapper/311:00:423
344132799141,7cyclictest0-21swapper/307:40:413
344132799141,6cyclictest0-21swapper/309:05:383
344132799134,6cyclictest0-21swapper/310:00:403
344132799132,7cyclictest0-21swapper/307:10:403
344132799132,6cyclictest0-21swapper/308:40:003
344132799132,6cyclictest0-21swapper/308:25:013
344132799131,8cyclictest0-21swapper/311:35:363
344132799131,8cyclictest0-21swapper/310:55:413
344132799131,7cyclictest0-21swapper/311:40:333
344132799131,7cyclictest0-21swapper/310:05:443
344132799131,7cyclictest0-21swapper/309:00:463
344132799130,6cyclictest0-21swapper/308:55:383
344132799122,7cyclictest0-21swapper/309:40:473
344132799122,6cyclictest0-21swapper/309:10:283
344132799122,6cyclictest0-21swapper/309:10:283
344132799121,7cyclictest0-21swapper/311:15:023
344132799121,6cyclictest0-21swapper/307:20:403
344132799121,5cyclictest0-21swapper/307:35:383
344132799113,4cyclictest0-21swapper/308:30:393
344132799103,5cyclictest0-21swapper/307:45:373
344132799101,6cyclictest0-21swapper/307:10:033
344132799101,5cyclictest0-21swapper/308:50:403
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional