You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-25 - 14:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Sat Apr 25, 2026 12:43:16)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3188554995850,5cyclictest3385312-21systemd-run12:30:011
3188554995850,5cyclictest3385312-21systemd-run12:30:011
3188553995145,3cyclictest0-21swapper/009:55:340
3188554994940,6cyclictest3233069-21aten_r2power_en09:20:311
3188553994840,6cyclictest2389957-21cups-browsed11:25:420
3188554994638,5cyclictest3325092-21grep11:15:001
3188554994638,5cyclictest3325092-21grep11:15:001
3188554994637,6cyclictest421-21dbus-daemon10:50:011
3188553994638,5cyclictest3250389-21unixbench_multi09:40:540
3188555994536,6cyclictest3361277-21systemd-run12:00:012
3188555994438,3cyclictest0-21swapper/211:40:012
3188555994436,5cyclictest3393336-21systemd-run12:40:012
3188555994435,7cyclictest3209526-21munin-run08:20:002
3188555994435,6cyclictest1-21systemd10:35:022
3188555994435,6cyclictest1-21systemd10:35:022
318855399445,33cyclictest3390522-21ssh12:35:430
3188553994437,5cyclictest0-21swapper/010:20:020
3188555994334,6cyclictest421-21dbus-daemon08:40:012
3188555994232,6cyclictest3256800-21latency_hist09:50:022
3188554994230,8cyclictest3237742-21sh09:25:431
3188553994233,7cyclictest12350irq/32-eth011:39:050
318855599416,30cyclictest3237833-21ssh09:25:462
318855599412,32cyclictest3245752-21ssh09:35:422
3188554994133,5cyclictest1055-21mta-sts-daemon09:50:471
3188554994132,6cyclictest3220031-21systemd-run08:55:011
3188553994136,3cyclictest151ktimers/008:45:470
3188553994133,5cyclictest1055-21mta-sts-daemon11:20:460
3188553994126,12cyclictest3256652-21systemd-run09:50:010
3188555994032,5cyclictest3272900-21latency_hist10:10:022
3188555994032,5cyclictest3224527-21diskmemload12:00:372
3188554994031,5cyclictest3224530-21munin-plugin-st09:10:011
3188553994034,4cyclictest3362683-21proc_pri12:00:500
3188553994034,3cyclictest3244528-21grep09:35:000
3188553994034,3cyclictest3244528-21grep09:35:000
3188553994012,24cyclictest3229848-21ssh09:15:480
3188555993929,6cyclictest421-21dbus-daemon09:52:372
3188553993931,5cyclictest3361263-21idleruntime-cro12:00:000
3188553993931,5cyclictest3286556-21unixbench-2d10:25:550
3188553993930,6cyclictest2389953-21cupsd09:28:360
3188553993925,11cyclictest3385334-21latency_hist12:30:010
3188553993925,11cyclictest3385334-21latency_hist12:30:010
318855399392,34cyclictest420-21avahi-daemon12:11:040
3188553993920,17cyclictest191rcuc/009:35:520
3188555993830,5cyclictest261-21systemd-journal08:30:032
3188555993830,4cyclictest0-21swapper/211:15:462
3188555993827,8cyclictest1-21systemd11:35:022
318855599382,30cyclictest3241853-21ssh09:30:472
318855599382,30cyclictest3241853-21ssh09:30:472
3188554993832,3cyclictest0-21swapper/110:20:011
318855399388,24cyclictest3322421-21ssh11:10:470
318855399388,24cyclictest3322421-21ssh11:10:470
3188553993830,5cyclictest3339035-21vmstat11:30:540
3188553993827,8cyclictest3294946-21sshd10:36:200
318855399382,31cyclictest3265879-21ssh10:00:410
318855399382,31cyclictest3265879-21ssh10:00:410
3188555993729,5cyclictest3337303-21latency_hist11:30:022
3188555993729,5cyclictest0-21swapper/212:10:022
3188555993728,6cyclictest3301537-21aten_r2power_po10:45:322
3188555993725,9cyclictest3230374-21unixbench_singl09:15:562
318855499374,6cyclictest171rcu_preempt10:10:421
318855499373,26cyclictest3282009-21ssh10:20:471
3188554993728,7cyclictest3268743-21sh10:05:011
3188554993728,7cyclictest3268743-21sh10:05:011
3188553993729,6cyclictest3306417-21mailstats10:50:490
3188553993727,3cyclictest3260645-21munin-plugin-st09:55:010
3188553993721,10cyclictest1055-21mta-sts-daemon11:45:420
3188553993721,10cyclictest1055-21mta-sts-daemon11:45:420
3188555993631,2cyclictest0-21swapper/211:15:002
3188555993631,2cyclictest0-21swapper/211:15:002
3188555993626,6cyclictest3282282-21mailstats10:20:562
318855599361,5cyclictest171rcu_preempt10:35:452
3188554993628,5cyclictest3295598-21ssh10:37:361
3188554993613,3cyclictest171rcu_preempt09:55:421
3188553993630,4cyclictest3215553-21systemd-run08:40:010
3188553993630,4cyclictest3203507-21cron08:00:010
3188553993620,9cyclictest12350irq/32-eth010:20:520
3188553993620,12cyclictest12350irq/32-eth009:20:460
318855599353,26cyclictest3233814-21ssh09:20:452
3188555993529,3cyclictest0-21swapper/211:10:012
3188555993529,3cyclictest0-21swapper/211:10:012
3188555993527,5cyclictest3354401-21needreboot11:50:412
3188555993524,8cyclictest3306004-21missed_timers10:50:382
318855599352,26cyclictest3386558-21ssh12:30:432
3188554993530,2cyclictest0-21swapper/107:30:021
3188554993528,4cyclictest0-21swapper/107:40:401
3188554993528,3cyclictest0-21swapper/112:05:011
3188554993527,5cyclictest3216980-21unixbench_singl08:40:531
3188554993526,7cyclictest3244527-21/usr/sbin/munin09:35:011
3188554993526,7cyclictest3244527-21/usr/sbin/munin09:35:011
3188554993525,7cyclictest695-21cron11:55:001
3188553993518,5cyclictest171rcu_preempt08:10:400
3188553993518,5cyclictest171rcu_preempt08:10:400
3188553993517,14cyclictest2389957-21cups-browsed12:15:500
3188553993516,4cyclictest171rcu_preempt09:05:480
3188553993514,16cyclictest659-21snmpd11:09:390
3188553993514,16cyclictest659-21snmpd11:09:390
3188555993428,3cyclictest0-21swapper/208:05:012
3188555993426,5cyclictest3298743-21unixbench_singl10:40:552
3188555993422,9cyclictest3190041-21idleruntime-cro07:15:012
3188555993420,9cyclictest420-21avahi-daemon07:55:452
3188554993428,3cyclictest0-21swapper/111:50:011
3188554993428,3cyclictest0-21swapper/111:50:011
3188554993427,4cyclictest3387075-21unixbench-2d12:30:531
3188554993426,5cyclictest3325658-21aten_r2power_en11:15:341
3188554993425,6cyclictest420-21avahi-daemon11:08:361
3188554993425,6cyclictest420-21avahi-daemon11:08:361
3188553993427,3cyclictest0-21swapper/010:55:380
3188553993426,6cyclictest2389957-21cups-browsed11:04:560
318855399341,16cyclictest3289940-21cut10:30:410
318855399341,16cyclictest3289940-21cut10:30:410
3188553993411,16cyclictest3225885-21ssh09:10:520
3188555993328,3cyclictest0-21swapper/211:40:412
3188555993328,3cyclictest0-21swapper/211:40:412
3188555993326,3cyclictest0-21swapper/211:00:462
3188554993326,5cyclictest0-21swapper/109:00:011
3188554993326,4cyclictest3191587-21latency_hist07:20:011
3188554993326,4cyclictest1056-21mta-sts-daemon09:44:091
3188554993326,4cyclictest1051-21mta-sts-daemon09:10:531
3188554993325,5cyclictest3342627-21sendmail_mailqu11:35:501
3188554993325,5cyclictest3341188-21ssh11:35:011
3188554993325,5cyclictest3313086-21latency_hist11:00:021
3188554993325,5cyclictest3306737-21unixbench_multi10:50:541
3188554993325,5cyclictest3256636-21munin-plugin-st09:50:011
3188554993324,6cyclictest3288796-21sh10:30:011
3188554993323,4cyclictest1055-21mta-sts-daemon08:25:481
318855399332,26cyclictest3326371-21ssh11:15:450
3188553993321,9cyclictest3379031-21timerwakeupswit12:20:530
3188553993317,12cyclictest151ktimers/008:50:470
3188555993227,2cyclictest0-21swapper/207:10:022
3188555993225,4cyclictest3373322-21ssh12:15:022
3188555993224,5cyclictest3377285-21cron12:20:012
3188555993224,5cyclictest3353253-21latency_hist11:50:002
3188555993224,5cyclictest3353253-21latency_hist11:50:002
3188555993224,4cyclictest3312477-21kworker/u8:6-xprtiod11:25:012
3188555993223,6cyclictest1051-21mta-sts-daemon10:00:112
3188555993223,6cyclictest1051-21mta-sts-daemon10:00:112
3188555993221,8cyclictest3224527-21diskmemload09:40:372
3188555993219,10cyclictest3274401-21timerandwakeup10:10:532
318855599321,6cyclictest171rcu_preempt08:20:432
318855599321,6cyclictest171rcu_preempt08:20:432
3188554993226,3cyclictest0-21swapper/108:40:011
3188554993225,3cyclictest0-21swapper/112:15:011
3188554993223,6cyclictest420-21avahi-daemon11:25:501
318855399323,23cyclictest3302090-21ssh10:45:410
3188553993225,3cyclictest0-21swapper/011:50:540
3188553993223,4cyclictest0-21swapper/012:05:450
3188553993221,8cyclictest0-21swapper/009:00:010
3188553993220,8cyclictest151ktimers/007:50:430
318855399320,13cyclictest14-21ksoftirqd/007:45:420
3188555993124,4cyclictest3225620-21missed_timers09:10:442
3188554993125,3cyclictest0-21swapper/109:20:011
3188554993125,2cyclictest0-21swapper/112:40:011
3188554993124,4cyclictest3359155-21unixbench_singl11:56:001
3188554993124,3cyclictest0-21swapper/108:00:011
3188554993123,5cyclictest3246354-21unixbench_multi09:35:551
3188554993121,8cyclictest3365957-21sh12:05:331
3188554993121,6cyclictest2389957-21cups-browsed08:30:031
318855399317,3cyclictest171rcu_preempt09:00:440
3188553993123,5cyclictest2389953-21cupsd12:31:250
3188553993111,12cyclictest3269922-21sh10:05:400
318855599304,12cyclictest3313185-21sed11:00:032
3188555993023,4cyclictest3278593-21unixbench_multi10:15:562
3188555993023,3cyclictest0-21swapper/212:30:012
3188555993023,3cyclictest0-21swapper/212:30:012
3188555993021,7cyclictest3217540-21fschecks_count08:45:362
3188555993021,6cyclictest2389957-21cups-browsed12:20:472
318855599301,4cyclictest171rcu_preempt08:40:532
318855499309,3cyclictest171rcu_preempt10:05:431
3188554993024,3cyclictest0-21swapper/111:45:021
3188554993024,3cyclictest0-21swapper/111:45:021
3188554993023,4cyclictest3314863-21unixbench_singl11:00:551
3188554993023,4cyclictest3298323-21processes10:40:501
3188554993022,5cyclictest420-21avahi-daemon09:01:361
3188554993021,7cyclictest3205845-21needreboot08:05:401
3188554993021,6cyclictest1056-21mta-sts-daemon11:20:501
318855399306,3cyclictest171rcu_preempt07:10:570
318855399305,14cyclictest3274071-21ssh10:10:460
3188553993024,4cyclictest3208036-21kworker/u8:9-nfsiod08:15:470
318855399302,20cyclictest3224527-21diskmemload11:40:410
318855399302,20cyclictest3224527-21diskmemload11:40:410
318855599294,6cyclictest171rcu_preempt08:05:442
3188555992923,3cyclictest1051-21mta-sts-daemon08:50:452
3188555992922,4cyclictest3264655-21cron10:00:012
3188555992922,4cyclictest3193273-21/usr/sbin/munin07:25:422
3188555992922,4cyclictest1056-21mta-sts-daemon10:29:502
3188555992921,5cyclictest261-21systemd-journal07:45:012
3188555992910,9cyclictest1055-21mta-sts-daemon07:35:392
318855499293,21cyclictest3208997-21sshd08:15:461
3188554992921,5cyclictest3378569-21ntp_kernel_pll_12:20:431
318855399293,11cyclictest3298159-21ssh10:40:440
3188553992913,5cyclictest171rcu_preempt07:35:480
3188553992911,13cyclictest171rcu_preempt07:25:510
3188555992822,3cyclictest0-21swapper/209:05:582
3188555992821,4cyclictest1056-21mta-sts-daemon07:45:392
3188555992821,4cyclictest0-21swapper/209:05:012
3188555992820,5cyclictest3195938-21unixbench_multi07:30:532
3188555992818,7cyclictest3212216-21switchtime08:25:542
3188554992823,2cyclictest0-21swapper/110:30:271
3188554992823,2cyclictest0-21swapper/110:30:271
3188554992822,4cyclictest3209878-21aten_r2power_fr08:20:341
3188554992822,4cyclictest3209878-21aten_r2power_fr08:20:341
3188554992821,3cyclictest0-21swapper/112:20:011
318855399285,16cyclictest428-21mta-sts-daemon08:30:440
318855399284,15cyclictest171rcu_preempt07:15:370
318855399282,8cyclictest14-21ksoftirqd/008:40:440
3188553992820,3cyclictest171rcu_preempt07:30:570
3188555992719,5cyclictest3192928-21unixbench_multi07:20:542
318855499279,3cyclictest171rcu_preempt07:10:541
3188554992719,5cyclictest1054-21mta-sts-daemon07:51:531
318855399274,15cyclictest428-21mta-sts-daemon08:00:480
3188553992721,3cyclictest0-21swapper/007:40:400
3188553992716,9cyclictest0-21swapper/008:05:450
3188553992711,2cyclictest171rcu_preempt07:10:010
3188555992620,3cyclictest0-21swapper/207:55:012
3188555992610,5cyclictest62450lldpd08:12:532
3188555992610,5cyclictest62450lldpd08:12:532
318855399268,2cyclictest171rcu_preempt08:20:440
318855399268,2cyclictest171rcu_preempt08:20:440
3188553992611,13cyclictest2500773-21kworker/u8:5-events_unbound07:21:420
3188555992518,4cyclictest3220532-21fschecks_count08:55:372
3188554992518,4cyclictest3207051-21fschecks_time08:10:351
3188554992518,4cyclictest3207051-21fschecks_time08:10:351
3188554992517,5cyclictest3218408-21unixbench_multi08:45:541
318855699241,5cyclictest0-21swapper/312:00:463
318855699241,4cyclictest0-21swapper/312:25:423
318855699241,4cyclictest0-21swapper/312:25:423
318855699241,4cyclictest0-21swapper/312:05:413
318855699241,4cyclictest0-21swapper/310:00:483
318855699241,4cyclictest0-21swapper/310:00:483
3188555992417,5cyclictest0-21swapper/207:20:002
3188554992416,5cyclictest421-21dbus-daemon07:10:021
3188554992416,4cyclictest0-21swapper/107:20:371
318855699232,8cyclictest0-21swapper/310:25:493
318855699232,7cyclictest0-21swapper/309:45:313
318855699232,16cyclictest0-21swapper/311:45:423
318855699232,16cyclictest0-21swapper/311:45:423
318855699232,16cyclictest0-21swapper/309:25:453
318855699232,15cyclictest0-21swapper/310:45:423
318855699231,6cyclictest0-21swapper/310:20:453
318855699231,5cyclictest0-21swapper/311:25:413
318855699231,4cyclictest0-21swapper/310:05:403
318855699231,4cyclictest0-21swapper/309:20:413
318855699231,16cyclictest0-21swapper/311:00:473
318855699231,15cyclictest0-21swapper/311:05:493
318855699231,15cyclictest0-21swapper/311:05:493
3188554992316,4cyclictest1054-21mta-sts-daemon08:00:471
3188554992313,7cyclictest3200253-21timerandwakeup07:45:541
318855699223,15cyclictest0-21swapper/310:10:493
318855699223,13cyclictest0-21swapper/309:35:433
318855699222,6cyclictest0-21swapper/309:55:483
318855699222,5cyclictest0-21swapper/311:55:493
318855699222,4cyclictest0-21swapper/312:10:453
318855699222,4cyclictest0-21swapper/309:15:423
318855699222,3cyclictest0-21swapper/309:40:423
318855699222,14cyclictest0-21swapper/312:30:393
318855699222,14cyclictest0-21swapper/312:15:423
318855699221,6cyclictest0-21swapper/311:35:453
318855699221,5cyclictest0-21swapper/311:50:353
318855699221,5cyclictest0-21swapper/311:40:423
318855699221,5cyclictest0-21swapper/311:40:423
318855699221,3cyclictest0-21swapper/312:35:423
318855699221,16cyclictest0-21swapper/311:20:463
318855699221,15cyclictest0-21swapper/309:50:443
318855699221,15cyclictest0-21swapper/309:10:543
3188554992216,3cyclictest0-21swapper/107:35:371
3188554992215,4cyclictest3195281-21missed_timers07:30:381
3188553992210,2cyclictest171rcu_preempt08:25:470
318855699212,5cyclictest0-21swapper/312:20:353
318855699212,5cyclictest0-21swapper/310:55:403
318855699212,5cyclictest0-21swapper/310:40:423
318855699212,5cyclictest0-21swapper/310:30:433
318855699212,5cyclictest0-21swapper/310:30:433
318855699212,4cyclictest0-21swapper/307:35:423
318855699212,13cyclictest0-21swapper/310:50:433
318855699211,6cyclictest0-21swapper/310:35:413
318855699211,6cyclictest0-21swapper/308:10:443
318855699211,6cyclictest0-21swapper/308:10:443
318855699211,5cyclictest0-21swapper/310:15:453
318855699211,5cyclictest0-21swapper/308:35:443
318855699211,4cyclictest0-21swapper/311:10:263
318855699211,4cyclictest0-21swapper/311:10:263
318855699211,15cyclictest0-21swapper/311:15:443
318855699211,15cyclictest0-21swapper/309:30:443
318855699211,15cyclictest0-21swapper/309:30:443
318855699202,6cyclictest0-21swapper/309:10:023
318855699202,5cyclictest0-21swapper/308:20:433
318855699202,5cyclictest0-21swapper/308:20:433
318855699202,4cyclictest0-21swapper/308:00:023
318855699202,4cyclictest0-21swapper/307:30:433
318855699201,6cyclictest0-21swapper/308:55:413
318855699201,6cyclictest0-21swapper/307:25:333
318855699201,5cyclictest0-21swapper/311:30:313
318855699201,5cyclictest0-21swapper/307:40:423
318855699201,4cyclictest0-21swapper/308:30:463
318855699201,4cyclictest0-21swapper/307:50:473
318855699191,5cyclictest0-21swapper/309:00:403
318855699191,5cyclictest0-21swapper/308:40:403
318855699191,5cyclictest0-21swapper/308:15:463
318855699191,5cyclictest0-21swapper/307:25:013
318855699191,5cyclictest0-21swapper/307:20:013
318855699191,4cyclictest0-21swapper/308:05:333
318855699191,4cyclictest0-21swapper/308:00:423
318855699182,3cyclictest0-21swapper/307:11:083
318855699181,3cyclictest0-21swapper/308:50:423
318855699181,3cyclictest0-21swapper/308:25:453
318855699162,7cyclictest0-21swapper/308:45:443
318855699161,5cyclictest0-21swapper/307:45:373
31885569991,4cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional