You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-07 - 14:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Fri Aug 07, 2026 12:43:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3241441994920,5cyclictest171rcu_preempt11:45:522
3241441994920,5cyclictest171rcu_preempt11:45:522
3241439994838,8cyclictest3298991-21kworker/u8:7-rpciod11:25:010
3241441994739,5cyclictest3365876-21ssh11:35:012
3241439994740,4cyclictest0-21swapper/011:20:010
3241441994635,5cyclictest0-21swapper/209:45:592
3241440994640,3cyclictest0-21swapper/109:10:011
3241440994638,5cyclictest3248946-21systemd-run07:35:011
3241439994639,3cyclictest151ktimers/008:25:510
3241439994637,5cyclictest151ktimers/009:30:540
3241440994535,7cyclictest265-21systemd-journal10:55:021
3241441994432,7cyclictest420-21avahi-daemon12:38:592
324144199441,32cyclictest3330518-21ssh10:35:552
3241440994436,5cyclictest3262464-21systemd-run08:20:011
3241440994435,6cyclictest2454516-21cups-browsed10:40:471
324144099443,34cyclictest3360998-21ssh11:25:551
3241439994432,7cyclictest151ktimers/010:35:510
3241441994336,4cyclictest3295836-21ssh09:40:012
3241441994336,4cyclictest3292806-21systemd09:35:022
3241441994335,5cyclictest421-21dbus-daemon10:50:012
3241441994333,6cyclictest3341537-21systemd-run10:55:012
324143999435,32cyclictest3379213-21ssh11:55:530
3241441994233,6cyclictest1-21systemd09:20:012
3241441994233,5cyclictest3305929-21ssh09:55:492
3241440994231,8cyclictest3350657-21/usr/sbin/munin11:10:021
3241439994236,4cyclictest3350692-21latency_hist11:10:020
3241439994215,21cyclictest3284754-21ssh09:20:530
3241441994131,7cyclictest3312665-21timerandwakeup10:06:052
3241440994133,5cyclictest3303361-21mailstats09:51:041
3241440994132,4cyclictest0-21swapper/110:05:531
324144099412,31cyclictest3391381-21rm12:16:001
3241441994032,5cyclictest3387087-21ssh12:10:012
3241441994032,5cyclictest3361551-21unixbench_singl11:26:062
3241441994031,5cyclictest3327982-21unixbench_multi10:31:042
3241441993932,3cyclictest0-21swapper/212:20:012
3241441993931,5cyclictest3280488-21cron09:15:012
3241440993930,6cyclictest3354200-21aten_r2power_en11:15:421
324144099392,32cyclictest3388238-21ssh12:10:521
3241439993932,4cyclictest3361354-21timerandwakeup11:26:050
3241441993829,6cyclictest3375919-21kernelversion11:50:482
3241441993829,6cyclictest3375919-21kernelversion11:50:482
3241440993829,6cyclictest3260934-21systemd-run08:15:011
3241440993829,5cyclictest1-21systemd09:35:011
3241439993830,5cyclictest1-21systemd11:45:010
3241439993830,5cyclictest1-21systemd11:45:010
3241441993730,3cyclictest0-21swapper/208:20:012
3241441993729,5cyclictest3381042-21cron12:00:002
3241441993728,5cyclictest3356745-21munin-plugin-st11:20:012
3241440993730,4cyclictest3259398-21kworker/u8:12-xprtiod08:45:001
3241440993730,4cyclictest3259398-21kworker/u8:12-xprtiod08:45:001
3241440993729,5cyclictest3311059-21ssh10:05:011
3241440993721,6cyclictest3300737-21sshd09:46:171
324143999373,27cyclictest3287785-21ssh09:25:510
3241439993728,6cyclictest2553372-21munin-node10:25:410
3241439993726,5cyclictest12650irq/32-eth010:10:530
3241439993716,5cyclictest14-21ksoftirqd/009:35:570
3241441993625,5cyclictest0-21swapper/211:20:532
324144099363,27cyclictest3394475-21rm12:21:021
3241440993630,4cyclictest3241438-21cyclictest11:50:581
3241440993630,4cyclictest3241438-21cyclictest11:50:581
3241440993629,4cyclictest3382767-21unixbench_multi12:01:041
3241440993627,6cyclictest439-21systemd-logind08:40:011
3241440993627,4cyclictest0-21swapper/110:16:001
3241440993627,4cyclictest0-21swapper/110:16:001
3241440993625,8cyclictest3379856-21unixbench_singl11:56:071
3241439993630,3cyclictest0-21swapper/009:05:010
3241439993629,5cyclictest1-21systemd11:00:010
3241439993627,6cyclictest420-21avahi-daemon12:29:490
3241439993611,4cyclictest14-21ksoftirqd/007:50:540
3241441993528,4cyclictest0-21swapper/209:00:012
3241441993527,5cyclictest3382777-21unixbench_multi12:01:042
3241441993527,5cyclictest3286726-21latency_hist09:25:022
3241441993527,3cyclictest0-21swapper/208:55:012
324144099357,13cyclictest271ktimers/109:55:521
3241440993528,5cyclictest3289702-21latency_hist09:30:021
3241440993525,6cyclictest2454516-21cups-browsed09:23:291
324143999352,27cyclictest3345827-21ssh11:00:570
3241439993511,11cyclictest14-21ksoftirqd/012:00:570
324144199344,22cyclictest3397391-21ssh12:26:002
3241441993428,2cyclictest0-21swapper/211:35:422
3241441993426,5cyclictest3268401-21cron08:40:012
3241441993425,6cyclictest2454516-21cups-browsed12:10:432
324144199342,25cyclictest3297189-21ps09:41:022
3241440993428,3cyclictest0-21swapper/107:40:561
3241440993428,3cyclictest0-21swapper/107:40:561
3241440993426,5cyclictest3274414-21sendmail09:00:011
3241440993424,7cyclictest3367439-21timerandwakeup11:36:031
3241440993424,6cyclictest1003-21mta-sts-daemon09:15:491
3241440993422,3cyclictest171rcu_preempt12:25:591
3241440993422,3cyclictest171rcu_preempt11:32:271
3241439993428,4cyclictest3321393-21ntp_kernel_pll_10:20:560
3241439993428,4cyclictest3321393-21ntp_kernel_pll_10:20:560
3241439993428,4cyclictest3274153-21timerandwakeup08:56:010
3241439993427,5cyclictest3329331-21ls10:35:010
3241439993427,4cyclictest3390104-21cron12:15:010
3241439993410,4cyclictest14-21ksoftirqd/012:05:550
3241441993327,3cyclictest0-21swapper/211:06:062
3241441993327,3cyclictest0-21swapper/207:20:012
3241441993326,4cyclictest0-21swapper/210:25:582
3241441993326,3cyclictest0-21swapper/210:00:522
3241441993325,5cyclictest421-21dbus-daemon09:30:012
3241441993325,5cyclictest3369698-21fschecks_count11:40:452
3241441993325,5cyclictest3369698-21fschecks_count11:40:452
3241441993325,5cyclictest0-21swapper/211:00:012
3241441993325,4cyclictest3323401-21latency_hist10:25:022
3241441993325,4cyclictest3323401-21latency_hist10:25:022
3241441993324,6cyclictest3246458-21fschecks_count07:25:462
3241441993317,11cyclictest2454516-21cups-browsed10:10:532
324144099338,20cyclictest271ktimers/107:55:531
3241440993327,4cyclictest0-21swapper/110:40:011
3241440993325,5cyclictest1-21systemd07:20:011
3241440993324,5cyclictest3315859-21unixbench_singl10:11:051
324144099331,5cyclictest0-21swapper/108:05:591
3241440993312,3cyclictest171rcu_preempt08:20:571
3241439993326,5cyclictest3272910-21cron08:55:010
3241439993323,3cyclictest171rcu_preempt12:15:460
3241439993319,5cyclictest3249041-21sed07:35:020
3241439993319,12cyclictest3317649-21sh10:15:440
3241439993319,12cyclictest3317649-21sh10:15:440
3241439993315,3cyclictest171rcu_preempt10:49:340
3241441993225,3cyclictest0-21swapper/211:05:022
3241441993224,5cyclictest3334073-21sed10:41:082
3241441993223,6cyclictest1-21systemd09:10:012
3241441993221,8cyclictest3251278-21ntp_kernel_err07:40:502
3241441993221,8cyclictest3251278-21ntp_kernel_err07:40:502
324144099324,8cyclictest28-21ksoftirqd/111:20:591
3241440993226,4cyclictest0-21swapper/108:25:461
324144099322,24cyclictest3385327-21ssh12:05:571
3241440993220,9cyclictest3342482-21latency10:55:481
3241440993219,9cyclictest3246747-21needreboot07:25:501
3241440993219,2cyclictest171rcu_preempt11:10:531
3241440993211,3cyclictest171rcu_preempt07:50:571
324143999329,4cyclictest14-21ksoftirqd/009:40:510
3241439993226,4cyclictest3405165-21munin-run12:40:010
3241439993225,4cyclictest1000-21mta-sts-daemon08:30:540
3241439993210,5cyclictest14-21ksoftirqd/011:10:500
3241439993210,5cyclictest14-21ksoftirqd/007:40:590
3241439993210,5cyclictest14-21ksoftirqd/007:40:590
3241441993123,5cyclictest3353701-21munin-run11:15:012
3241441993123,4cyclictest1002-21mta-sts-daemon09:53:212
324144199312,22cyclictest3400397-21ssh12:30:582
3241441993118,10cyclictest3318770-21timerwakeupswit10:16:062
3241441993118,10cyclictest3318770-21timerwakeupswit10:16:062
324144099317,12cyclictest271ktimers/111:40:531
324144099317,12cyclictest271ktimers/111:40:531
324144099315,9cyclictest28-21ksoftirqd/109:40:501
3241440993123,5cyclictest0-21swapper/112:35:011
3241440993122,6cyclictest420-21avahi-daemon11:01:531
324144099312,21cyclictest3327424-21ssh10:30:521
3241440993121,7cyclictest420-21avahi-daemon10:26:181
3241440993117,6cyclictest421-21dbus-daemon08:45:301
3241440993117,5cyclictest171rcu_preempt08:30:431
3241440993117,3cyclictest171rcu_preempt09:00:581
324143999318,6cyclictest14-21ksoftirqd/007:25:520
324143999318,4cyclictest14-21ksoftirqd/009:05:500
324143999318,4cyclictest14-21ksoftirqd/008:10:510
3241439993124,4cyclictest3365892-21munin-plugin-st11:35:010
3241439993124,4cyclictest3242947-21systemd-run07:15:010
3241439993118,3cyclictest171rcu_preempt11:45:040
3241439993118,3cyclictest171rcu_preempt11:45:040
324143999311,24cyclictest3333549-21ssh10:40:560
3241439993112,4cyclictest14-21ksoftirqd/008:45:510
3241439993111,5cyclictest14-21ksoftirqd/007:35:580
324144199304,17cyclictest428-21mta-sts-daemon08:10:522
324144099304,13cyclictest271ktimers/109:35:581
3241440993023,4cyclictest261rcuc/108:55:021
3241440993018,2cyclictest171rcu_preempt10:45:561
3241440993017,3cyclictest171rcu_preempt09:10:461
3241440993014,3cyclictest171rcu_preempt11:45:441
3241440993014,3cyclictest171rcu_preempt11:45:441
3241440993010,5cyclictest271ktimers/107:37:241
324143999309,5cyclictest14-21ksoftirqd/008:40:520
324143999309,5cyclictest14-21ksoftirqd/008:40:520
324143999309,5cyclictest14-21ksoftirqd/007:45:560
324143999305,10cyclictest0-21swapper/010:00:520
324143999304,20cyclictest428-21mta-sts-daemon09:55:580
3241439993025,3cyclictest3399901-21fschecks_time12:30:470
3241439993024,4cyclictest3339340-21ssh10:50:460
3241439993022,4cyclictest151ktimers/011:35:430
3241439993020,7cyclictest12650irq/32-eth012:20:540
3241439993016,10cyclictest12650irq/32-eth009:10:510
3241439993010,5cyclictest14-21ksoftirqd/008:15:540
324144099295,9cyclictest28-21ksoftirqd/107:10:501
324144099292,8cyclictest28-21ksoftirqd/110:20:591
324144099292,8cyclictest28-21ksoftirqd/110:20:591
3241440992921,5cyclictest3405166-21grep12:40:011
3241440992912,3cyclictest171rcu_preempt07:20:461
324143999299,4cyclictest14-21ksoftirqd/008:20:500
324143999297,6cyclictest14-21ksoftirqd/008:35:520
324143999293,12cyclictest3280933-21cat09:15:390
3241439992921,5cyclictest420-21avahi-daemon09:47:340
3241441992822,3cyclictest0-21swapper/212:20:482
3241441992821,4cyclictest0-21swapper/208:00:012
3241441992812,14cyclictest351ktimers/208:30:532
3241440992818,2cyclictest171rcu_preempt07:45:541
3241440992812,2cyclictest0-21swapper/108:00:501
324143999289,5cyclictest14-21ksoftirqd/009:52:360
324143999286,5cyclictest14-21ksoftirqd/008:00:520
324143999283,20cyclictest3376297-21ssh11:50:580
324143999283,20cyclictest3376297-21ssh11:50:580
3241439992823,3cyclictest0-21swapper/010:05:520
3241439992822,4cyclictest3256286-21grep07:56:040
3241441992718,6cyclictest3253943-21fschecks_count07:50:452
3241440992719,5cyclictest421-21dbus-daemon07:10:011
324143999275,4cyclictest14-21ksoftirqd/007:20:540
3241439992716,8cyclictest3259449-21systemd08:10:020
324144199262,19cyclictest3275390-21cron09:01:022
3241441992621,2cyclictest0-21swapper/208:25:012
3241441992620,3cyclictest0-21swapper/207:50:002
3241441992620,3cyclictest0-21swapper/207:11:032
3241441992618,3cyclictest0-21swapper/208:45:532
3241439992617,3cyclictest151ktimers/007:16:020
3241441992515,7cyclictest3268964-21fschecks_time08:40:442
3241441992515,7cyclictest3268964-21fschecks_time08:40:442
3241441992418,3cyclictest0-21swapper/208:30:012
3241441992418,3cyclictest0-21swapper/207:31:042
3241441992417,3cyclictest0-21swapper/207:21:042
3241441992416,5cyclictest3256953-21fschecks_time08:00:452
3241441992317,4cyclictest0-21swapper/207:10:022
3241441992317,3cyclictest0-21swapper/207:40:012
3241439992114,4cyclictest191rcuc/007:10:010
324144299193,8cyclictest0-21swapper/309:25:493
324144299193,10cyclictest0-21swapper/311:25:523
3241441991913,3cyclictest0-21swapper/208:09:442
324144299184,8cyclictest0-21swapper/309:46:023
324144299182,10cyclictest0-21swapper/310:20:553
324144299182,10cyclictest0-21swapper/310:20:553
324144299181,9cyclictest0-21swapper/311:20:523
324144299173,9cyclictest0-21swapper/312:15:593
324144299173,9cyclictest0-21swapper/311:05:553
324144299173,7cyclictest0-21swapper/312:00:453
324144299172,9cyclictest0-21swapper/312:05:543
324144299172,9cyclictest0-21swapper/311:00:013
324144299172,8cyclictest0-21swapper/310:00:553
324144299172,11cyclictest0-21swapper/311:00:423
324144299171,8cyclictest0-21swapper/307:25:583
324144299171,10cyclictest0-21swapper/310:25:593
324144299164,6cyclictest0-21swapper/311:40:563
324144299164,6cyclictest0-21swapper/311:40:563
324144299162,9cyclictest0-21swapper/310:40:483
324144299162,8cyclictest0-21swapper/309:50:553
324144299162,7cyclictest0-21swapper/312:10:503
324144299161,9cyclictest0-21swapper/310:16:013
324144299161,9cyclictest0-21swapper/310:16:013
324144299161,8cyclictest0-21swapper/309:35:543
324144299161,8cyclictest0-21swapper/307:20:523
324144299154,8cyclictest0-21swapper/310:35:563
324144299154,6cyclictest0-21swapper/312:35:583
324144299153,8cyclictest0-21swapper/309:20:463
324144299153,7cyclictest0-21swapper/311:15:513
324144299152,8cyclictest0-21swapper/311:45:533
324144299152,8cyclictest0-21swapper/311:45:533
324144299152,8cyclictest0-21swapper/311:35:533
324144299152,8cyclictest0-21swapper/309:40:503
324144299152,8cyclictest0-21swapper/309:10:543
324144299152,8cyclictest0-21swapper/308:15:583
324144299152,7cyclictest0-21swapper/312:25:553
324144299152,7cyclictest0-21swapper/310:10:463
324144299152,6cyclictest0-21swapper/308:25:513
324144299151,9cyclictest0-21swapper/310:45:513
324144299151,8cyclictest0-21swapper/311:50:593
324144299151,8cyclictest0-21swapper/311:50:593
324144299151,8cyclictest0-21swapper/309:15:503
324144299151,7cyclictest0-21swapper/307:50:543
324144299151,7cyclictest0-21swapper/307:10:523
324144299151,6cyclictest0-21swapper/308:35:513
324144299151,6cyclictest0-21swapper/308:20:503
324144299143,6cyclictest0-21swapper/312:30:483
324144299143,6cyclictest0-21swapper/309:55:543
324144299142,8cyclictest0-21swapper/307:35:013
324144299142,6cyclictest0-21swapper/310:50:423
324144299142,6cyclictest0-21swapper/308:45:523
324144299141,9cyclictest0-21swapper/311:55:533
324144299141,8cyclictest0-21swapper/312:20:473
324144299141,8cyclictest0-21swapper/310:05:593
324144299141,8cyclictest0-21swapper/309:30:443
324144299141,6cyclictest0-21swapper/309:00:583
324144299141,6cyclictest0-21swapper/308:55:503
324144299141,6cyclictest0-21swapper/307:40:513
324144299141,6cyclictest0-21swapper/307:40:513
324144299141,6cyclictest0-21swapper/307:35:563
324144299133,6cyclictest0-21swapper/311:30:513
324144299132,5cyclictest0-21swapper/310:30:173
324144299132,5cyclictest0-21swapper/308:05:563
324144299131,8cyclictest0-21swapper/311:10:503
324144299131,6cyclictest0-21swapper/308:50:543
324144299131,6cyclictest0-21swapper/307:55:543
324144299131,6cyclictest0-21swapper/307:15:503
324144299123,6cyclictest0-21swapper/307:45:543
324144299123,5cyclictest0-21swapper/308:30:523
324144299122,6cyclictest0-21swapper/309:10:013
324144299111,6cyclictest0-21swapper/308:40:553
324144299111,6cyclictest0-21swapper/308:40:553
324144299111,6cyclictest0-21swapper/308:00:453
324144299101,5cyclictest0-21swapper/308:10:413
32414429981,4cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional