You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-09 - 01:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Thu Apr 09, 2026 00:43:24)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3372880996327,29cyclictest3417910-21ssh21:20:360
3372882995645,6cyclictest0-21swapper/221:30:342
3372882995645,6cyclictest0-21swapper/221:30:342
337288299504,9cyclictest171rcu_preempt22:40:342
3372880994939,4cyclictest0-21swapper/019:35:380
337288299482,37cyclictest3503142-21ssh23:05:352
337288299482,37cyclictest3503142-21ssh23:05:352
3372882994735,8cyclictest420-21avahi-daemon21:10:022
337288099472,35cyclictest3429835-21ssh21:35:290
3372882994638,5cyclictest421-21dbus-daemon21:30:022
3372882994638,5cyclictest421-21dbus-daemon21:30:022
3372882994637,6cyclictest421-21dbus-daemon19:10:012
337288299463,34cyclictest3511233-21ssh23:15:332
3372880994619,21cyclictest3442195-21ssh21:50:380
3372882994530,10cyclictest421-21dbus-daemon21:35:312
3372881994536,6cyclictest3536063-21threads23:45:461
3372881994518,14cyclictest3408864-21diskmemload00:01:171
3372881994436,5cyclictest3514125-21systemctl23:20:001
3372881994432,9cyclictest3466944-21timerwakeupswit22:20:431
3372882994332,7cyclictest3416724-21systemd21:20:022
3372882994231,7cyclictest421-21dbus-daemon22:17:532
3372882994231,7cyclictest3546645-21cups-browsed00:08:502
3372882994229,9cyclictest3440948-21ssh21:50:012
3372880994230,9cyclictest0-21swapper/023:40:010
337288299415,26cyclictest3531640-21ssh23:40:342
337288299413,30cyclictest3450299-21ssh22:00:362
3372882994131,5cyclictest0-21swapper/200:00:262
3372881994133,5cyclictest3518199-21cron23:25:001
3372881994133,4cyclictest0-21swapper/122:05:311
337288199413,27cyclictest3551731-21ssh00:05:331
3372880994125,13cyclictest3465350-21latency_hist22:20:020
3372882994032,5cyclictest3542611-21systemd23:55:022
3372882994032,5cyclictest3497869-21systemd-run23:00:012
337288299401,30cyclictest3442107-21ssh21:50:342
337288099401,33cyclictest3543863-21ssh23:55:380
3372882993933,3cyclictest0-21swapper/220:40:002
3372882993933,3cyclictest0-21swapper/220:40:002
3372882993930,6cyclictest261-21systemd-journal19:15:012
3372882993928,7cyclictest3408864-21diskmemload23:00:242
3372881993932,3cyclictest0-21swapper/123:35:351
3372881993928,8cyclictest3495638-21unixbench_singl22:55:531
337288199392,30cyclictest3462350-21sshd22:15:311
337288099393,27cyclictest3478605-21ssh22:35:320
3372880993931,6cyclictest3477487-21cron22:35:000
3372880993930,7cyclictest439-21systemd-logind23:50:020
337288299383,27cyclictest3559710-21ssh00:15:302
3372882993828,5cyclictest0-21swapper/200:30:372
3372881993832,3cyclictest0-21swapper/122:50:331
3372881993830,3cyclictest0-21swapper/100:20:031
3372880993830,5cyclictest3572433-21unixbench_singl00:30:460
3372882993723,7cyclictest0-21swapper/223:55:372
337288199378,23cyclictest3523458-21ssh23:30:341
3372881993731,3cyclictest0-21swapper/100:00:011
3372881993729,4cyclictest3502643-21fschecks_count23:05:251
3372881993729,4cyclictest3502643-21fschecks_count23:05:251
3372881993729,4cyclictest0-21swapper/122:00:011
3372881993728,6cyclictest420-21avahi-daemon21:15:431
3372881993728,4cyclictest0-21swapper/121:25:321
3372881993728,4cyclictest0-21swapper/121:25:321
3372881993726,7cyclictest3458087-21latency22:10:291
3372881993724,10cyclictest3470302-21latency22:25:301
3372881993724,10cyclictest3470302-21latency22:25:301
3372880993729,6cyclictest3380371-21/usr/sbin/munin19:35:010
337288299363,27cyclictest3434069-21ssh21:40:362
3372882993628,5cyclictest3566684-21latency_hist00:25:022
3372882993626,6cyclictest3404373-21systemd-run20:55:012
3372882993624,7cyclictest1431064-21cups-browsed22:54:242
337288299362,28cyclictest3446290-21ssh21:55:412
337288299362,27cyclictest3466421-21ssh22:20:312
337288199367,11cyclictest3409931-21ntpq21:10:301
3372881993630,3cyclictest0-21swapper/121:10:011
3372881993628,5cyclictest3434628-21unixbench_singl21:40:441
3372881993628,4cyclictest0-21swapper/122:35:341
3372881993628,3cyclictest0-21swapper/121:20:371
3372881993617,14cyclictest548-21gdbus00:25:351
3372881993610,11cyclictest1431064-21cups-browsed22:31:171
337288099363,22cyclictest3486835-21ssh22:45:380
3372880993631,2cyclictest0-21swapper/023:25:270
3372880993628,5cyclictest421-21dbus-daemon00:03:180
3372882993529,3cyclictest0-21swapper/220:10:012
3372882993529,3cyclictest0-21swapper/200:40:002
3372882993526,6cyclictest421-21dbus-daemon22:10:012
3372882993525,5cyclictest0-21swapper/222:30:412
3372881993528,4cyclictest0-21swapper/122:00:451
3372881993527,5cyclictest421-21dbus-daemon21:40:011
3372881993527,5cyclictest3440936-21ssh21:50:011
3372881993527,5cyclictest3398360-21idleruntime-cro20:35:001
337288199352,26cyclictest3563802-21ssh00:20:331
3372881993521,8cyclictest3390298-21ntpq20:05:351
3372880993528,4cyclictest3399900-21kworker/u8:0-rpciod21:26:320
3372880993528,4cyclictest3399900-21kworker/u8:0-rpciod21:26:320
3372880993526,6cyclictest3546645-21cups-browsed00:23:330
3372880993524,5cyclictest3578653-21idleruntime-cro00:40:000
337288099351,30cyclictest3511311-21ssh23:15:360
3372882993426,5cyclictest3556375-21unixbench_singl00:10:442
3372882993426,5cyclictest3471203-21unixbench_singl22:25:472
3372882993426,5cyclictest3471203-21unixbench_singl22:25:472
3372882993426,4cyclictest3514150-21kworker/u8:1-xprtiod23:25:452
3372882993426,4cyclictest1056-21mta-sts-daemon23:10:392
3372882993426,4cyclictest1056-21mta-sts-daemon23:10:392
3372882993425,6cyclictest3515108-21ssh23:20:302
3372882993423,6cyclictest0-21swapper/223:30:252
3372881993427,4cyclictest1056-21mta-sts-daemon22:49:031
3372881993426,6cyclictest3514150-21kworker/u8:1+xprtiod23:25:271
3372881993425,7cyclictest3375872-21sendmail19:20:011
3372881993425,6cyclictest3558613-21cron00:15:011
3372881993425,6cyclictest1431064-21cups-browsed23:10:211
3372881993425,6cyclictest1431064-21cups-browsed23:10:211
3372880993428,3cyclictest3460720-21kworker/u8:3-rpciod23:05:020
3372880993427,5cyclictest3469362-21cron22:25:010
3372880993412,19cyclictest151ktimers/021:45:320
3372882993328,2cyclictest0-21swapper/200:30:012
3372882993325,6cyclictest0-21swapper/222:50:022
3372882993324,6cyclictest3381897-21systemd-run19:40:022
3372882993323,6cyclictest3396886-21true20:30:012
3372882993323,4cyclictest0-21swapper/221:20:282
337288299331,24cyclictest3530611-21date23:40:032
3372881993327,3cyclictest0-21swapper/121:35:011
3372881993327,3cyclictest0-21swapper/121:35:011
3372881993327,3cyclictest0-21swapper/119:45:011
3372881993325,5cyclictest1431064-21cups-browsed23:41:281
3372881993324,4cyclictest3408864-21diskmemload00:35:401
3372881993320,8cyclictest3405210-21ntpq20:55:311
3372880993327,4cyclictest3515487-21proc_pri23:20:400
3372882993223,6cyclictest1-21systemd23:50:022
337288299322,24cyclictest3409950-21ssh21:10:312
3372881993224,5cyclictest3498909-21needreboot23:00:321
3372881993224,5cyclictest1431064-21cups-browsed20:19:041
3372881993222,7cyclictest3571636-21missed_timers00:30:301
3372881993219,8cyclictest3442214-21ntpq21:50:391
337288099323,14cyclictest3412083-21ssh21:13:490
3372880993228,2cyclictest0-21swapper/000:20:010
3372880993226,4cyclictest1-21systemd22:05:020
3372880993225,5cyclictest3457576-21/usr/sbin/munin22:10:280
3372880993224,5cyclictest3546645-21cups-browsed00:14:200
3372880993223,6cyclictest3494568-21cut22:55:340
3372882993124,3cyclictest0-21swapper/222:35:352
3372882993122,6cyclictest439-21systemd-logind19:55:022
3372881993124,4cyclictest3395259-21unixbench_multi20:20:431
3372881993123,5cyclictest3386363-21idleruntime-cro19:55:011
3372881993123,5cyclictest3376408-21fschecks_time19:20:251
3372881993123,5cyclictest0-21swapper/120:00:011
3372880993125,3cyclictest3542620-21systemd23:55:020
3372880993125,3cyclictest0-21swapper/023:35:010
3372880993124,5cyclictest3425851-21ntp_kernel_pll_21:30:320
3372880993124,5cyclictest3425851-21ntp_kernel_pll_21:30:320
3372880993123,6cyclictest3375255-21ntp_kernel_pll_19:15:330
3372880993123,5cyclictest3506382-21/usr/sbin/munin23:10:300
3372880993123,5cyclictest3506382-21/usr/sbin/munin23:10:300
3372880993123,5cyclictest1431064-21cups-browsed22:41:550
3372880993121,7cyclictest3433846-21missed_timers21:40:290
337288299302,21cyclictest3458330-21ssh22:10:342
3372882993020,4cyclictest0-21swapper/220:55:332
337288199306,16cyclictest3539847-21ssh23:50:391
3372881993025,2cyclictest0-21swapper/120:45:011
3372881993021,5cyclictest1054-21mta-sts-daemon21:00:331
3372881993018,8cyclictest3378219-21ntpq19:25:311
3372881993017,9cyclictest0-21swapper/122:40:331
3372880993025,3cyclictest3404375-21kworker/u8:10-rpciod22:00:010
3372880993025,3cyclictest0-21swapper/020:55:010
3372880993024,3cyclictest0-21swapper/022:25:460
3372880993024,3cyclictest0-21swapper/022:25:460
3372880993023,5cyclictest3454953-21sh22:05:460
3372880993022,5cyclictest0-21swapper/023:40:310
3372880993021,4cyclictest191rcuc/020:20:370
337288299292,21cyclictest1054-21mta-sts-daemon20:45:342
3372882992921,5cyclictest261-21systemd-journal20:25:012
3372882992920,6cyclictest420-21avahi-daemon20:45:012
3372881992920,7cyclictest3381889-21sendmail-msp19:40:011
3372881992919,8cyclictest3374359-21idleruntime-cro19:15:011
3372881992919,6cyclictest3402234-21ntpq20:45:301
3372881992914,12cyclictest3384840-21sshd19:46:571
337288099298,17cyclictest420-21avahi-daemon00:25:350
337288099296,17cyclictest659-21snmpd21:15:360
3372880992916,7cyclictest12350irq/32-eth000:05:330
3372880992912,7cyclictest3503384-21ssh23:05:420
3372880992912,7cyclictest3503384-21ssh23:05:420
337288099291,25cyclictest0-21swapper/022:50:330
337288299282,5cyclictest171rcu_preempt19:20:352
3372881992822,3cyclictest0-21swapper/120:15:021
3372881992816,9cyclictest3396193-21netstat20:25:311
3372881992816,8cyclictest3399260-21ntpq20:35:331
3372881992816,8cyclictest3399260-21ntpq20:35:331
3372881992816,7cyclictest3403725-21ntpq20:50:311
3372880992824,3cyclictest3372879-21cyclictest20:10:350
3372880992824,2cyclictest0-21swapper/020:40:000
3372880992824,2cyclictest0-21swapper/020:40:000
3372880992822,4cyclictest3388402-21fschecks_time20:00:250
3372880992817,7cyclictest3390288-21ntpq20:05:340
3372882992720,4cyclictest1051-21mta-sts-daemon20:30:332
3372881992716,7cyclictest439-21systemd-logind19:10:011
3372881992716,7cyclictest3379832-21ntpq19:30:351
3372881992711,11cyclictest3388204-21tail20:00:221
337288099275,18cyclictest0-21swapper/019:25:330
337288099274,19cyclictest421-21dbus-daemon19:20:340
3372880992716,7cyclictest3384337-21ntpq19:45:370
3372882992618,6cyclictest421-21dbus-daemon21:05:012
3372882992614,9cyclictest3389370-21munin-plugin-st20:05:022
337288099266,15cyclictest3392327-21kworker/u8:2+rpciod20:40:310
3372880992621,3cyclictest518-21rs:main0
3372880992620,4cyclictest3407397-21systemd21:05:020
3372880992619,5cyclictest1431064-21cups-browsed19:55:020
3372882992519,3cyclictest0-21swapper/220:20:012
3372882992517,5cyclictest3391393-21fschecks_time20:10:262
3372882992516,6cyclictest420-21avahi-daemon19:15:432
337288099255,15cyclictest0-21swapper/020:25:350
3372880992520,3cyclictest3367581-21kworker/u8:14-xprtiod19:15:010
3372880992519,4cyclictest3393874-21idleruntime-cro20:20:010
3372880992516,4cyclictest3382738-21ntpq19:40:310
3372882992418,3cyclictest3358618-21kworker/u8:6-xprtiod19:30:242
3372882992418,3cyclictest0-21swapper/220:00:012
3372880992419,3cyclictest3395603-21kworker/u8:17-xprtiod21:05:270
3372880992417,3cyclictest3402290-21ntpq20:45:330
3372880992412,5cyclictest3405210-21ntpq20:55:320
337288099239,11cyclictest3386846-21df19:55:240
3372882992216,3cyclictest3383390-21kworker/u8:7-xprtiod19:45:222
3372882992216,3cyclictest0-21swapper/219:45:012
3372882992213,6cyclictest3377902-21fschecks_time19:25:262
337288099206,10cyclictest3373017-21grep19:10:020
337288399193,11cyclictest0-21swapper/323:50:343
337288399193,11cyclictest0-21swapper/300:05:333
337288399193,10cyclictest0-21swapper/323:25:343
337288399193,10cyclictest0-21swapper/321:15:403
337288399191,12cyclictest0-21swapper/321:45:343
337288399183,9cyclictest0-21swapper/300:20:313
337288399183,10cyclictest0-21swapper/321:25:323
337288399183,10cyclictest0-21swapper/321:25:323
337288399183,10cyclictest0-21swapper/300:25:353
337288399181,13cyclictest0-21swapper/323:55:373
337288399181,13cyclictest0-21swapper/321:40:313
337288399181,12cyclictest0-21swapper/323:40:323
337288399181,12cyclictest0-21swapper/323:35:333
337288399181,12cyclictest0-21swapper/322:20:313
337288399181,12cyclictest0-21swapper/319:30:363
337288399179,5cyclictest0-21swapper/322:05:323
337288399179,5cyclictest0-21swapper/321:20:333
337288399172,11cyclictest0-21swapper/321:35:213
337288399171,12cyclictest0-21swapper/323:45:363
337288399171,12cyclictest0-21swapper/323:20:343
337288399171,12cyclictest0-21swapper/322:55:433
337288399171,12cyclictest0-21swapper/322:50:353
337288399171,12cyclictest0-21swapper/322:40:353
337288399171,12cyclictest0-21swapper/322:30:333
337288399171,12cyclictest0-21swapper/321:05:333
337288399171,12cyclictest0-21swapper/320:55:333
337288399171,12cyclictest0-21swapper/320:35:313
337288399171,12cyclictest0-21swapper/320:35:313
337288399171,12cyclictest0-21swapper/300:10:313
337288399171,11cyclictest0-21swapper/323:10:323
337288399171,11cyclictest0-21swapper/323:10:323
337288399171,11cyclictest0-21swapper/320:25:383
337288399171,11cyclictest0-21swapper/319:40:313
337288399171,11cyclictest0-21swapper/319:25:323
337288399170,14cyclictest0-21swapper/319:10:013
337288399162,10cyclictest0-21swapper/320:20:313
337288399161,11cyclictest0-21swapper/323:15:213
337288399161,11cyclictest0-21swapper/323:05:313
337288399161,11cyclictest0-21swapper/323:05:313
337288399161,11cyclictest0-21swapper/323:00:353
337288399161,11cyclictest0-21swapper/322:35:273
337288399161,11cyclictest0-21swapper/322:25:253
337288399161,11cyclictest0-21swapper/322:25:253
337288399161,11cyclictest0-21swapper/322:15:333
337288399161,11cyclictest0-21swapper/322:10:323
337288399161,11cyclictest0-21swapper/321:50:333
337288399161,11cyclictest0-21swapper/321:30:233
337288399161,11cyclictest0-21swapper/321:30:233
337288399161,11cyclictest0-21swapper/321:10:323
337288399161,11cyclictest0-21swapper/320:45:333
337288399161,11cyclictest0-21swapper/320:40:313
337288399161,11cyclictest0-21swapper/320:15:323
337288399161,11cyclictest0-21swapper/320:05:323
337288399161,11cyclictest0-21swapper/320:00:313
337288399161,11cyclictest0-21swapper/319:55:333
337288399161,11cyclictest0-21swapper/319:50:323
337288399161,11cyclictest0-21swapper/319:45:333
337288399161,11cyclictest0-21swapper/319:35:333
337288399161,11cyclictest0-21swapper/319:20:313
337288399161,11cyclictest0-21swapper/319:15:323
337288399161,11cyclictest0-21swapper/319:10:313
337288399161,11cyclictest0-21swapper/300:30:333
337288399161,11cyclictest0-21swapper/300:15:313
337288399161,11cyclictest0-21swapper/300:00:263
3372883991610,4cyclictest0-21swapper/321:55:343
337288399160,12cyclictest0-21swapper/323:30:373
337288399160,12cyclictest0-21swapper/320:30:343
337288399160,11cyclictest0-21swapper/322:45:363
337288399160,11cyclictest0-21swapper/321:00:343
337288399151,11cyclictest0-21swapper/320:10:343
337288399151,11cyclictest0-21swapper/300:35:223
337288399150,11cyclictest0-21swapper/322:00:083
337288399150,11cyclictest0-21swapper/320:50:323
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional