You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-05 - 10:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Sun Jul 05, 2026 00:43:21)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3554947996046,10cyclictest265-21systemd-journal00:26:412
3554947996046,10cyclictest265-21systemd-journal00:26:412
3554947995041,6cyclictest421-21dbus-daemon22:45:542
355494799494,34cyclictest3668524-21ssh23:15:372
3554946994941,4cyclictest0-21swapper/123:45:391
3554945994824,19cyclictest12650irq/32-eth023:50:370
3554945994737,7cyclictest3708296-21sshd00:20:400
3554946994636,6cyclictest3625384-21fschecks_count22:05:281
3554947994536,6cyclictest3587970-21grep21:00:012
3554946994538,4cyclictest3717916-21grep00:35:481
3554946994538,4cyclictest3717916-21grep00:35:481
3554945994530,12cyclictest3578988-21munin-plugin-st20:30:010
3554946994434,8cyclictest3646133-21ssh22:40:021
3554945994423,15cyclictest151ktimers/022:10:370
3554946994334,6cyclictest3668888-21awk23:15:481
3554945994337,4cyclictest3622434-21if_wlan022:00:300
3554947994234,5cyclictest3694922-21sendmail00:00:012
3554947994223,3cyclictest171rcu_preempt20:20:352
3554945994232,7cyclictest3695122-21ssh00:00:040
355494599422,31cyclictest3641204-21ssh22:30:450
3554947994132,5cyclictest3655230-21grep22:55:002
3554947994132,5cyclictest3655230-21grep22:55:002
3554947994130,7cyclictest265-21systemd-journal23:08:092
3554946994136,2cyclictest0-21swapper/122:00:011
3554946994133,4cyclictest3597177-21latency_hist21:20:021
3554946994133,4cyclictest1002-21mta-sts-daemon22:02:541
3554945994131,7cyclictest3650494-21mailstats22:45:570
3554947994031,5cyclictest3710323-21latency_hist00:25:032
355494799403,10cyclictest36-21ksoftirqd/223:00:362
3554947994029,7cyclictest3609413-21cron21:40:012
3554947994029,6cyclictest0-21swapper/221:50:002
3554947994029,6cyclictest0-21swapper/221:50:002
355494799402,32cyclictest3638023-21ssh22:25:362
3554946994032,5cyclictest421-21dbus-daemon22:50:011
3554946994032,5cyclictest3612511-21munin-plugin-st21:45:011
3554946994032,5cyclictest3603274-21cron21:30:011
3554946994032,5cyclictest3591102-21latency_hist21:10:021
3554945994033,4cyclictest3615550-21idleruntime-cro21:50:000
3554945994033,4cyclictest3615550-21idleruntime-cro21:50:000
3554945994032,5cyclictest3687382-21unixbench_multi23:45:480
3554945994031,6cyclictest3637833-21missed_timers22:25:310
3554945994031,5cyclictest421-21dbus-daemon00:09:340
355494599401,34cyclictest3677684-21ssh23:30:390
3554947993930,6cyclictest1813734-21cupsd22:12:282
3554947993923,11cyclictest0-21swapper/200:15:402
3554946993932,4cyclictest3683149-21sshd23:40:251
3554946993931,4cyclictest1002-21mta-sts-daemon21:54:121
3554947993831,4cyclictest0-21swapper/222:05:482
3554947993830,5cyclictest3559961-21fschecks_count19:25:282
3554947993829,6cyclictest3704091-21cron00:15:012
3554946993832,3cyclictest0-21swapper/100:25:411
3554946993832,3cyclictest0-21swapper/100:25:411
3554946993830,5cyclictest3594074-21ssh21:15:011
3554947993730,4cyclictest3554944-21cyclictest22:30:392
3554947993730,4cyclictest1002-21mta-sts-daemon00:30:052
3554946993730,5cyclictest0-21swapper/123:25:021
3554946993730,4cyclictest0-21swapper/121:45:421
3554946993730,4cyclictest0-21swapper/121:45:421
3554946993728,6cyclictest420-21avahi-daemon19:20:471
3554946993725,8cyclictest3628340-21cpu22:10:261
3554945993730,4cyclictest3646113-21sendmail22:40:010
3554945993730,4cyclictest3591686-21gltestperf21:10:300
3554945993729,5cyclictest1813736-21cups-browsed21:55:020
3554945993722,5cyclictest0-21swapper/020:00:010
3554945993719,14cyclictest12650irq/32-eth023:00:360
3554947993629,4cyclictest3682717-21ssh23:40:012
3554947993629,4cyclictest1001-21mta-sts-daemon22:59:332
3554947993629,3cyclictest0-21swapper/223:40:342
3554947993628,5cyclictest3583999-21fschecks_count20:45:282
3554947993627,6cyclictest439-21systemd-logind20:45:012
3554947993625,8cyclictest3676602-21cut23:30:022
3554947993625,7cyclictest421-21dbus-daemon22:04:022
3554947993625,5cyclictest0-21swapper/222:20:342
355494799361,31cyclictest3601345-21ssh21:25:372
3554946993629,3cyclictest0-21swapper/119:45:301
3554946993628,5cyclictest0-21swapper/119:40:011
3554945993628,4cyclictest3574504-21kworker/u8:4-xprtiod21:25:250
355494799355,27cyclictest3590986-21diskmemload22:35:382
355494799354,24cyclictest3619838-21grep21:55:432
3554947993528,4cyclictest0-21swapper/220:20:002
3554947993527,5cyclictest3688792-21cron23:50:012
3554947993527,4cyclictest0-21swapper/223:50:252
3554947993527,4cyclictest0-21swapper/221:15:402
3554947993526,4cyclictest0-21swapper/219:50:012
3554946993527,5cyclictest3587993-21sendmail21:00:011
3554946993527,4cyclictest0-21swapper/100:05:311
3554946993526,5cyclictest3634094-21latency_hist22:20:031
3554945993529,4cyclictest3619276-21fschecks_time21:55:290
355494599352,7cyclictest0-21swapper/020:40:340
3554945993526,5cyclictest0-21swapper/022:20:290
3554945993524,4cyclictest3647747-21head22:40:490
355494599352,25cyclictest3607579-21grep21:35:430
355494799343,24cyclictest3717375-21ssh00:35:372
355494799343,24cyclictest3717375-21ssh00:35:372
3554947993428,3cyclictest0-21swapper/223:25:012
3554947993427,3cyclictest0-21swapper/222:20:012
3554947993426,5cyclictest3590015-21fschecks_time21:05:282
355494699343,20cyclictest3604375-21ssh21:30:381
3554946993428,3cyclictest0-21swapper/100:00:041
3554946993426,5cyclictest0-21swapper/123:10:011
3554945993427,5cyclictest3711787-21unixbench-2d00:25:470
3554945993427,5cyclictest3711787-21unixbench-2d00:25:470
3554945993426,5cyclictest420-21avahi-daemon22:54:420
3554945993426,5cyclictest420-21avahi-daemon22:54:420
3554945993424,8cyclictest3702020-21needreboot00:10:320
3554945993423,7cyclictest3604306-21ntpq21:30:340
355494599342,27cyclictest3668649-21rm23:15:430
3554945993411,19cyclictest3598207-21ssh21:20:360
3554947993323,6cyclictest567-21gdbus20:10:432
355494799331,15cyclictest36-21ksoftirqd/219:20:362
355494699334,14cyclictest3637952-21ssh22:25:331
3554946993326,4cyclictest3643032-21cron22:35:011
3554946993325,5cyclictest3571366-21head20:00:471
3554946993324,6cyclictest1000-21mta-sts-daemon23:50:551
3554946993323,7cyclictest3649165-21systemd-run22:45:021
3554946993321,9cyclictest3585785-21needreboot20:50:331
3554946993316,5cyclictest3667440-21cron23:15:011
3554945993325,4cyclictest0-21swapper/022:20:010
355494599332,23cyclictest3625754-21rm22:05:350
355494599331,28cyclictest151ktimers/020:15:440
3554947993226,2cyclictest0-21swapper/219:40:012
3554947993224,5cyclictest57-21kcompactd000:00:302
3554947993224,5cyclictest3597838-21ssh21:20:292
3554947993222,6cyclictest1001-21mta-sts-daemon21:00:372
355494699325,21cyclictest3656404-21ssh22:55:421
3554946993226,3cyclictest0-21swapper/123:40:011
3554946993226,3cyclictest0-21swapper/100:30:421
3554946993225,4cyclictest1000-21mta-sts-daemon21:24:401
3554946993225,4cyclictest0-21swapper/122:50:371
3554946993225,4cyclictest0-21swapper/122:50:371
3554946993225,3cyclictest0-21swapper/100:03:171
3554946993224,4cyclictest0-21swapper/123:00:431
3554945993226,4cyclictest3671990-21timerandwakeup23:20:490
3554945993224,5cyclictest3557930-21munin-run19:20:010
3554945993224,4cyclictest0-21swapper/000:30:390
3554945993222,6cyclictest3656282-21ssh22:55:370
3554947993124,3cyclictest0-21swapper/222:40:402
3554947993123,4cyclictest3565389-21unixbench_singl19:40:562
3554947993123,3cyclictest0-21swapper/221:30:412
3554947993121,5cyclictest3591971-21needreboot21:10:342
3554947993119,3cyclictest171rcu_preempt19:10:332
3554946993125,3cyclictest0-21swapper/100:10:471
3554946993124,4cyclictest1001-21mta-sts-daemon22:21:011
3554946993122,6cyclictest3607305-21needreboot21:35:321
3554946993122,5cyclictest3575848-21unixbench_multi20:15:501
3554945993125,4cyclictest3584848-21tail20:45:460
3554945993125,4cyclictest0-21swapper/019:40:010
3554945993124,4cyclictest3680149-21sh23:35:280
3554945993124,4cyclictest3562459-21systemd-run19:35:020
3554945993124,4cyclictest0-21swapper/000:00:290
3554945993122,7cyclictest3585490-21fschecks_count20:50:290
355494599311,24cyclictest3665413-21ssh23:10:320
3554947993022,4cyclictest0-21swapper/200:05:432
355494799301,21cyclictest665-21snmpd20:35:352
3554947993010,4cyclictest171rcu_preempt20:00:372
3554946993024,3cyclictest0-21swapper/100:20:011
3554946993023,4cyclictest3554967-21latency_hist19:10:021
3554946993022,5cyclictest3674228-21fschecks_time23:25:301
3554946993022,5cyclictest3589526-21latency_hist21:05:021
3554946993022,5cyclictest3569976-21latency_hist20:00:021
3554946993021,5cyclictest3576286-21aten_r2power_en20:20:261
3554945993024,4cyclictest3596140-21cron21:17:010
3554945993024,4cyclictest3590021-21fschecks_time21:05:280
3554945993024,4cyclictest3568422-21cron19:55:010
3554947992922,3cyclictest0-21swapper/221:50:492
3554947992920,5cyclictest3577760-21aten_r2power_cu20:25:242
3554947992920,5cyclictest0-21swapper/221:41:502
3554947992913,4cyclictest171rcu_preempt19:15:402
355494699293,21cyclictest665-21snmpd20:25:351
3554946992924,2cyclictest0-21swapper/120:10:011
3554945992924,2cyclictest0-21swapper/000:36:020
3554945992924,2cyclictest0-21swapper/000:36:020
3554945992923,4cyclictest3564460-21fschecks_count19:40:320
3554945992922,4cyclictest0-21swapper/021:40:290
3554945992922,3cyclictest1002-21mta-sts-daemon23:28:030
3554945992912,13cyclictest3683780-21ntp_kernel_pll_23:40:370
355494799282,18cyclictest3677741-21ntpq23:30:412
355494799282,16cyclictest1813736-21cups-browsed19:55:372
3554946992821,3cyclictest0-21swapper/119:55:011
3554946992819,6cyclictest3584833-21unixbench-2d20:45:461
355494599281,22cyclictest3662405-21ntpq23:05:370
3554947992718,3cyclictest171rcu_preempt23:10:322
3554947992717,6cyclictest341rcuc/220:50:402
3554947992715,3cyclictest171rcu_preempt20:07:452
3554946992721,3cyclictest0-21swapper/123:35:011
3554945992721,3cyclictest0-21swapper/020:40:010
3554945992720,5cyclictest3570479-21gltestperf20:00:290
3554945992720,4cyclictest3588496-21grep21:00:280
3554946992619,4cyclictest1002-21mta-sts-daemon20:34:541
3554946992619,4cyclictest0-21swapper/120:40:011
3554946992619,3cyclictest0-21swapper/119:40:561
3554946992618,5cyclictest1813736-21cups-browsed19:20:031
3554946992618,5cyclictest1-21systemd19:35:021
355494599269,10cyclictest151ktimers/019:20:390
355494599264,16cyclictest627-21lldpd20:10:390
3554945992621,2cyclictest0-21swapper/019:10:270
355494599261,4cyclictest171rcu_preempt20:55:410
3554945992614,7cyclictest3704962-21latency00:15:350
355494799257,3cyclictest171rcu_preempt19:50:392
355494799252,18cyclictest1813736-21cups-browsed20:30:412
3554947992515,3cyclictest171rcu_preempt19:30:422
355494699257,10cyclictest428-21mta-sts-daemon19:25:371
355494699252,17cyclictest3708388-21ps00:20:431
355494599252,17cyclictest3579952-21ntp_states20:30:460
3554945992521,3cyclictest3554944-21cyclictest20:05:350
3554945992517,5cyclictest3559767-21aten_r2power_fr19:25:250
3554947992418,3cyclictest1003-21mta-sts-daemon19:10:022
3554946992416,4cyclictest3555719-21missed_timers19:10:311
3554945992416,6cyclictest3576736-21missed_timers20:20:320
3554945992416,6cyclictest3566846-21unixbench_singl19:45:470
3554946992214,2cyclictest0-21swapper/120:10:371
355494699212,14cyclictest3582488-21diskstats20:40:271
3554945992014,3cyclictest0-21swapper/019:10:020
355494899182,8cyclictest0-21swapper/322:45:533
355494899182,8cyclictest0-21swapper/322:10:373
355494899162,8cyclictest0-21swapper/323:10:353
355494899162,7cyclictest0-21swapper/300:35:353
355494899162,7cyclictest0-21swapper/300:35:353
355494899154,7cyclictest0-21swapper/321:55:393
355494899152,6cyclictest0-21swapper/321:10:363
355494899151,7cyclictest0-21swapper/300:25:353
355494899151,7cyclictest0-21swapper/300:25:353
355494899143,7cyclictest0-21swapper/321:50:343
355494899143,6cyclictest0-21swapper/320:00:013
355494899142,7cyclictest0-21swapper/323:50:413
355494899142,6cyclictest0-21swapper/323:55:343
355494899142,6cyclictest0-21swapper/323:30:403
355494899142,6cyclictest0-21swapper/320:15:423
355494899142,5cyclictest0-21swapper/320:45:353
355494899142,5cyclictest0-21swapper/319:25:363
355494899141,7cyclictest0-21swapper/323:40:423
355494899141,7cyclictest0-21swapper/300:30:363
355494899141,6cyclictest0-21swapper/320:10:383
355494899141,6cyclictest0-21swapper/320:05:383
355494899133,5cyclictest0-21swapper/321:35:333
355494899132,7cyclictest0-21swapper/322:40:303
355494899132,6cyclictest0-21swapper/323:45:273
355494899132,6cyclictest0-21swapper/322:15:493
355494899132,6cyclictest0-21swapper/320:35:363
355494899132,5cyclictest0-21swapper/319:20:363
355494899132,5cyclictest0-21swapper/319:10:333
355494899132,4cyclictest0-21swapper/319:40:393
355494899131,8cyclictest0-21swapper/322:20:413
355494899131,7cyclictest0-21swapper/323:15:443
355494899131,7cyclictest0-21swapper/322:55:363
355494899131,7cyclictest0-21swapper/322:30:293
355494899131,7cyclictest0-21swapper/322:05:373
355494899131,7cyclictest0-21swapper/300:15:453
355494899131,7cyclictest0-21swapper/300:05:383
355494899131,6cyclictest0-21swapper/320:30:413
355494899131,5cyclictest0-21swapper/321:05:333
355494899130,5cyclictest0-21swapper/320:55:363
355494899124,6cyclictest0-21swapper/300:00:433
355494899123,4cyclictest0-21swapper/321:00:453
355494899122,6cyclictest0-21swapper/323:25:263
355494899122,6cyclictest0-21swapper/322:50:343
355494899122,6cyclictest0-21swapper/322:50:343
355494899122,6cyclictest0-21swapper/321:40:373
355494899122,6cyclictest0-21swapper/300:10:253
355494899122,5cyclictest0-21swapper/323:00:313
355494899122,4cyclictest0-21swapper/319:35:343
355494899121,7cyclictest0-21swapper/323:35:273
355494899121,7cyclictest0-21swapper/323:05:283
355494899121,7cyclictest0-21swapper/322:35:403
355494899121,7cyclictest0-21swapper/321:30:393
355494899121,6cyclictest0-21swapper/321:20:353
355494899121,6cyclictest0-21swapper/321:15:343
355494899121,6cyclictest0-21swapper/320:40:373
355494899121,6cyclictest0-21swapper/320:20:353
355494899113,6cyclictest0-21swapper/300:20:353
355494899112,6cyclictest0-21swapper/322:00:303
355494899112,6cyclictest0-21swapper/321:45:293
355494899112,6cyclictest0-21swapper/321:45:293
355494899112,5cyclictest0-21swapper/319:15:393
355494899111,6cyclictest0-21swapper/322:25:293
355494899111,6cyclictest0-21swapper/321:25:243
355494899111,5cyclictest0-21swapper/320:00:393
355494899104,4cyclictest0-21swapper/319:50:403
355494899103,5cyclictest0-21swapper/319:45:343
355494899102,5cyclictest0-21swapper/323:20:243
355494899101,5cyclictest0-21swapper/320:50:293
355494899101,5cyclictest0-21swapper/320:25:293
355494899101,5cyclictest0-21swapper/319:30:363
35549489981,4cyclictest0-21swapper/319:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional