You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-03 - 05:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Mon Aug 03, 2026 00:43:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3436301995851,5cyclictest3587827-21sh00:20:010
3436302995645,6cyclictest3490805-21munin-run21:40:011
3436303995547,3cyclictest0-21swapper/200:40:012
3436302995548,4cyclictest0-21swapper/122:15:421
3436303995145,3cyclictest0-21swapper/221:40:012
343630299516,33cyclictest3570933-21ssh23:50:581
3436302995039,7cyclictest3567680-21awk23:45:501
3436302995039,7cyclictest3480215-21unixbench_singl21:21:031
3436303994734,5cyclictest0-21swapper/222:16:012
343630199472,38cyclictest1727787-21cups-browsed22:05:560
3436303994637,5cyclictest1003-21mta-sts-daemon00:00:522
343630399462,32cyclictest3567662-21ssh23:45:492
3436302994637,5cyclictest3472280-21cron21:10:011
3436303994536,6cyclictest421-21dbus-daemon23:25:012
3436303994536,6cyclictest421-21dbus-daemon23:25:012
3436303994531,11cyclictest3574160-21timerandwakeup23:56:012
3436303994531,11cyclictest3574160-21timerandwakeup23:56:012
3436302994534,7cyclictest1-21systemd22:05:021
3436301994534,8cyclictest151ktimers/000:05:580
3436302994433,7cyclictest421-21dbus-daemon19:20:481
3436301994437,4cyclictest3445318-21systemd-run19:40:010
343630199442,33cyclictest3588895-21ssh00:20:470
3436303994334,6cyclictest3547102-21grep23:11:042
3436303994231,8cyclictest3500845-21if_wlan021:55:482
3436301994236,3cyclictest3560983-21aten_r2power_cu23:35:400
3436301994234,6cyclictest3575727-21cups-browsed00:15:010
3436301994215,18cyclictest3540602-21ssh23:00:550
3436303994118,15cyclictest341rcuc/222:55:582
3436303994032,5cyclictest3449806-21systemd-run19:55:012
3436303994021,6cyclictest171rcu_preempt23:35:532
3436302994034,3cyclictest0-21swapper/123:20:491
3436302994034,3cyclictest0-21swapper/123:20:491
3436302994030,5cyclictest3589585-21vmstat00:21:021
3436302994029,8cyclictest3497377-21acpi21:50:391
3436301994026,3cyclictest0-21swapper/021:00:010
3436301994021,16cyclictest3490805-21munin-run21:40:010
343630399393,26cyclictest3588967-21ssh00:20:512
3436302993930,5cyclictest439-21systemd-logind19:30:011
3436302993930,5cyclictest3516287-21ntp_kernel_err22:20:501
3436302993928,7cyclictest420-21avahi-daemon23:19:571
3436301993932,5cyclictest3457276-21cron20:20:000
3436301993930,6cyclictest3559177-21tail23:31:020
3436301993930,6cyclictest3486368-21unixbench_multi21:31:030
3436301993923,9cyclictest12650irq/32-eth022:10:590
3436303993831,4cyclictest3507172-21needreboot22:05:522
3436303993830,5cyclictest3533311-21cron22:50:012
3436302993829,6cyclictest3510498-21mailstats22:11:021
3436302993825,10cyclictest3452840-21latency_hist20:05:021
3436301993830,5cyclictest12750irq/33-eth023:45:500
3436301993829,5cyclictest3501709-21unixbench_multi21:56:060
3436301993817,14cyclictest12650irq/32-eth023:50:540
3436303993729,5cyclictest3587805-21idleruntime-cro00:20:012
3436303993729,5cyclictest3466310-21systemd-run20:50:012
3436303993728,5cyclictest1000-21mta-sts-daemon21:47:022
3436303993727,7cyclictest421-21dbus-daemon21:20:012
3436303993727,6cyclictest3579789-21needreboot00:05:502
3436303993726,7cyclictest3592203-21mailstats00:26:022
3436303993723,10cyclictest3492285-21switchtime21:41:012
3436302993729,5cyclictest3584761-21cron00:15:001
3436302993729,5cyclictest3461781-21cron20:35:011
3436302993729,5cyclictest0-21swapper/121:50:011
3436302993728,7cyclictest3503077-21sh22:00:021
3436302993728,6cyclictest265-21systemd-journal00:40:011
3436302993726,6cyclictest0-21swapper/122:35:431
343630299372,30cyclictest3476512-21ssh21:15:501
343630199379,9cyclictest3577498-21ssh00:01:060
343630199378,9cyclictest3528482-21ntp_states22:40:540
343630199378,8cyclictest3476522-21ntpq21:15:500
3436301993731,4cyclictest3575690-21sendmail-msp00:00:010
3436301993731,4cyclictest3575690-21sendmail-msp00:00:010
3436301993724,8cyclictest3472289-21diskmemload23:40:500
343630199372,32cyclictest3498037-21ssh21:50:500
3436303993628,5cyclictest3540979-21timerwakeupswit23:01:022
3436303993622,7cyclictest0-21swapper/222:50:502
343630399362,26cyclictest3473457-21ssh21:10:512
3436302993628,5cyclictest3566592-21systemd-run23:45:011
3436302993628,5cyclictest3545430-21systemd-run23:10:011
3436302993627,5cyclictest3457340-21latency_hist20:20:011
343630199368,8cyclictest3484020-21ssh21:28:070
343630199368,7cyclictest3437196-21ntpq19:10:520
343630199368,10cyclictest3598017-21ntpq00:35:530
343630199367,12cyclictest3546638-21ntpq23:10:560
343630199366,10cyclictest3594953-21ntpq00:30:500
3436301993627,6cyclictest3493868-21cron21:45:010
3436301993625,8cyclictest3463284-21cat20:40:010
3436301993621,12cyclictest3470788-21cron21:05:010
343630199361,27cyclictest3516503-21ssh22:20:590
3436303993528,4cyclictest3518365-21latency_hist22:25:022
3436303993528,4cyclictest1003-21mta-sts-daemon22:12:092
3436303993527,5cyclictest3499982-21sh21:55:012
3436303993525,6cyclictest3485778-21ssh21:30:522
3436302993525,6cyclictest420-21avahi-daemon21:43:331
3436302993524,5cyclictest0-21swapper/121:10:501
3436302993512,15cyclictest421-21dbus-daemon00:00:511
343630199358,8cyclictest3551129-21ssh23:19:030
3436301993528,4cyclictest3442193-21unixbench_multi19:26:020
3436301993526,6cyclictest1727784-21cupsd22:51:510
343630199351,30cyclictest3555787-21sh23:25:570
343630199351,30cyclictest3555787-21sh23:25:570
3436303993427,4cyclictest3445295-21basename19:40:012
3436303993426,6cyclictest0-21swapper/223:31:002
3436303993425,6cyclictest3457290-21idleruntime-cro20:20:002
343630399341,29cyclictest3504231-21ssh22:00:532
3436302993428,3cyclictest0-21swapper/120:40:011
3436302993427,4cyclictest3455777-21cron20:15:001
3436302993426,5cyclictest427-21polkitd00:25:071
3436301993423,8cyclictest3531761-21switchtime22:46:000
343630399334,4cyclictest171rcu_preempt23:15:562
3436303993325,5cyclictest3467778-21sh20:55:012
3436303993325,5cyclictest3439192-21unixbench_singl19:16:032
3436303993324,6cyclictest1827262-21munin-node22:32:562
3436303993320,10cyclictest3458116-21needreboot20:20:462
343630299333,23cyclictest3595118-21ssh00:30:581
343630299333,23cyclictest3558620-21ssh23:30:501
3436302993326,3cyclictest0-21swapper/123:35:441
3436302993325,5cyclictest3554946-21aten_r2power_cu23:25:401
3436302993325,5cyclictest3554946-21aten_r2power_cu23:25:401
3436302993325,4cyclictest0-21swapper/123:00:021
3436302993324,5cyclictest0-21swapper/121:30:531
3436301993326,5cyclictest12750irq/33-eth022:30:520
343630199330,28cyclictest3537474-21ssh22:55:530
3436303993226,3cyclictest3575694-21kworker/u8:11-xprtiod00:35:012
3436303993225,3cyclictest0-21swapper/222:30:012
343630399322,24cyclictest3479587-21ssh21:20:492
343630299327,16cyclictest3531446-21ssh22:45:501
343630299323,21cyclictest3534556-21ssh22:50:551
3436302993225,4cyclictest0-21swapper/121:00:011
3436302993223,4cyclictest0-21swapper/100:05:551
343630199326,10cyclictest3455694-21unixbench_singl20:11:010
3436301993223,6cyclictest421-21dbus-daemon23:06:220
3436301993214,14cyclictest0-21swapper/019:45:530
3436303993126,2cyclictest0-21swapper/220:40:012
3436303993125,3cyclictest0-21swapper/221:00:012
3436303993124,4cyclictest0-21swapper/220:00:012
3436303993123,5cyclictest3557497-21latency_hist23:30:022
3436303993123,5cyclictest3557497-21latency_hist23:30:022
3436303993122,5cyclictest0-21swapper/221:25:552
343630399311,27cyclictest3582853-21ssh00:10:482
343630299313,23cyclictest3573734-21ssh23:55:501
343630299313,23cyclictest3573734-21ssh23:55:501
3436302993124,4cyclictest3522931-21head22:31:041
3436302993123,5cyclictest3439210-21head19:16:031
3436302993121,6cyclictest1-21systemd21:05:021
343630299311,25cyclictest3546544-21ssh23:10:521
343630199319,5cyclictest14-21ksoftirqd/000:25:210
343630199319,4cyclictest14-21ksoftirqd/020:35:020
343630199318,4cyclictest14-21ksoftirqd/021:20:490
343630199317,7cyclictest3553593-21sh23:22:120
343630199317,7cyclictest3553593-21sh23:22:120
343630199316,20cyclictest3473386-21ssh21:10:490
343630199314,9cyclictest3446733-21awk19:40:590
3436301993123,5cyclictest3472404-21latency_hist21:10:020
3436301993117,11cyclictest3439180-21tail19:16:030
3436301993114,10cyclictest3519352-21ssh22:25:560
3436301993111,5cyclictest14-21ksoftirqd/022:35:530
3436303993025,2cyclictest0-21swapper/220:05:012
3436303993022,5cyclictest3455687-21unixbench_multi20:11:012
343630299302,22cyclictest3586036-21ssh00:15:571
3436302993020,7cyclictest1000-21mta-sts-daemon22:43:261
343630199305,8cyclictest3466573-21aten_r2power_en20:50:400
3436301993024,3cyclictest0-21swapper/021:46:010
3436301993023,4cyclictest3458792-21cron20:25:000
3436301993021,7cyclictest485-21in:imuxsock19:35:010
3436301993011,5cyclictest14-21ksoftirqd/022:00:460
3436303992921,5cyclictest3470788-21cron21:05:012
3436303992921,5cyclictest1727787-21cups-browsed23:42:322
3436303992921,4cyclictest1002-21mta-sts-daemon22:39:302
343630199299,5cyclictest14-21ksoftirqd/020:00:560
343630199295,9cyclictest3454053-21sed20:06:010
343630199295,9cyclictest3454053-21sed20:06:010
3436301992922,3cyclictest0-21swapper/020:45:010
3436301992920,6cyclictest3459336-21fschecks_time20:25:440
343630399282,20cyclictest3543583-21ssh23:05:562
3436303992821,5cyclictest0-21swapper/223:55:012
3436303992817,5cyclictest3530292-21latency_hist22:45:022
3436302992823,2cyclictest0-21swapper/119:55:011
3436302992820,3cyclictest0-21swapper/121:25:591
3436302992819,3cyclictest1002-21mta-sts-daemon22:05:451
343630199289,4cyclictest14-21ksoftirqd/020:46:010
343630199286,7cyclictest3450642-21ntpq19:55:510
3436303992719,6cyclictest627-21lldpd19:30:522
3436302992720,4cyclictest0-21swapper/123:00:441
3436302992720,4cyclictest0-21swapper/120:55:011
3436302992719,5cyclictest3451486-21latency_hist20:00:021
3436302992719,3cyclictest0-21swapper/122:25:501
343630199271,17cyclictest3513362-21ssh22:15:570
3436301992711,3cyclictest14-21ksoftirqd/019:50:580
343630499262,19cyclictest0-21swapper/322:15:593
343630399262,17cyclictest351ktimers/220:40:552
3436303992617,7cyclictest3460752-21cpuspeed20:30:432
3436302992618,5cyclictest3445274-21cron19:40:011
3436302992618,5cyclictest1727787-21cups-browsed20:27:251
3436302992618,5cyclictest1000-21mta-sts-daemon19:13:391
343630499253,18cyclictest0-21swapper/322:06:013
343630499252,8cyclictest0-21swapper/321:20:553
343630499251,4cyclictest0-21swapper/322:55:573
343630499251,18cyclictest0-21swapper/321:40:513
343630399258,3cyclictest171rcu_preempt21:05:512
3436303992519,3cyclictest0-21swapper/219:20:492
3436303992517,5cyclictest1727787-21cups-browsed20:28:202
3436302992518,4cyclictest3453466-21latency20:05:461
3436302992518,4cyclictest3453466-21latency20:05:461
3436302992517,5cyclictest0-21swapper/119:50:011
3436302992516,7cyclictest3458320-21proc_pri20:20:551
3436301992519,3cyclictest0-21swapper/019:25:020
343630499243,3cyclictest0-21swapper/321:00:553
343630499242,6cyclictest0-21swapper/322:25:593
343630499242,5cyclictest0-21swapper/321:30:523
343630499242,4cyclictest0-21swapper/320:55:573
343630499242,3cyclictest0-21swapper/321:15:563
343630499242,15cyclictest0-21swapper/323:35:563
343630499241,5cyclictest0-21swapper/320:20:513
343630499241,4cyclictest0-21swapper/323:30:493
343630499241,4cyclictest0-21swapper/321:05:563
343630499240,5cyclictest0-21swapper/321:10:493
343630399245,3cyclictest171rcu_preempt19:10:562
343630399243,4cyclictest171rcu_preempt20:05:572
343630399243,4cyclictest171rcu_preempt20:05:572
3436303992418,3cyclictest0-21swapper/219:45:012
3436303992416,6cyclictest3447095-21aten_r2power_en19:45:412
3436303992416,5cyclictest3442294-21idleruntime-cro19:30:012
3436302992417,4cyclictest428-21mta-sts-daemon19:30:461
3436302992417,4cyclictest3463569-21aten_r2power_cu20:40:401
3436302992415,7cyclictest3466290-21/usr/sbin/munin20:50:021
343630499233,7cyclictest0-21swapper/323:25:523
343630499233,7cyclictest0-21swapper/323:25:523
343630499233,6cyclictest0-21swapper/322:40:453
343630499232,4cyclictest0-21swapper/322:30:513
343630499232,4cyclictest0-21swapper/322:11:003
343630499232,4cyclictest0-21swapper/321:50:533
343630499232,4cyclictest0-21swapper/300:30:503
343630499232,4cyclictest0-21swapper/300:15:573
343630499232,3cyclictest0-21swapper/323:20:563
343630499232,3cyclictest0-21swapper/323:20:563
343630499232,14cyclictest0-21swapper/322:20:503
343630499231,6cyclictest0-21swapper/323:40:523
343630499231,6cyclictest0-21swapper/322:50:503
343630499231,5cyclictest0-21swapper/323:55:513
343630499231,5cyclictest0-21swapper/323:55:513
343630499231,5cyclictest0-21swapper/323:15:503
343630499231,5cyclictest0-21swapper/319:20:023
343630499231,4cyclictest0-21swapper/322:35:533
343630499231,4cyclictest0-21swapper/321:35:523
343630499231,4cyclictest0-21swapper/300:05:543
343630499231,3cyclictest0-21swapper/320:15:503
343630499222,7cyclictest0-21swapper/323:05:473
343630499222,5cyclictest0-21swapper/323:10:513
343630499222,5cyclictest0-21swapper/322:45:503
343630499222,4cyclictest0-21swapper/323:00:533
343630499222,4cyclictest0-21swapper/321:55:533
343630499222,4cyclictest0-21swapper/300:00:473
343630499222,14cyclictest0-21swapper/321:45:553
343630499221,6cyclictest0-21swapper/323:45:513
343630499221,5cyclictest0-21swapper/322:00:503
343630499221,5cyclictest0-21swapper/320:45:493
343630499221,5cyclictest0-21swapper/320:30:513
343630499221,5cyclictest0-21swapper/320:11:183
343630499221,5cyclictest0-21swapper/319:35:473
343630499221,5cyclictest0-21swapper/300:35:523
343630499221,5cyclictest0-21swapper/300:15:003
343630499221,4cyclictest0-21swapper/321:25:573
343630499221,4cyclictest0-21swapper/320:40:513
343630499221,4cyclictest0-21swapper/320:25:513
343630499221,4cyclictest0-21swapper/319:55:553
343630499221,4cyclictest0-21swapper/319:40:483
343630499221,4cyclictest0-21swapper/319:31:023
343630499221,4cyclictest0-21swapper/319:10:513
3436302992214,5cyclictest265-21systemd-journal19:45:011
343630499212,4cyclictest0-21swapper/323:50:573
343630499212,4cyclictest0-21swapper/300:25:543
343630499211,5cyclictest0-21swapper/319:20:513
343630499211,4cyclictest0-21swapper/320:35:533
343630499211,4cyclictest0-21swapper/319:50:513
343630499211,4cyclictest0-21swapper/319:45:543
343630499211,4cyclictest0-21swapper/319:25:513
343630499211,4cyclictest0-21swapper/300:20:493
343630399210,5cyclictest3436306-21systemd-run19:10:022
343630499201,4cyclictest0-21swapper/320:50:503
343630499201,4cyclictest0-21swapper/320:05:443
343630499201,4cyclictest0-21swapper/320:05:443
343630499201,4cyclictest0-21swapper/320:00:513
3436302991913,3cyclictest0-21swapper/119:10:021
3436301991911,6cyclictest1000-21mta-sts-daemon19:10:010
343630499182,3cyclictest0-21swapper/319:10:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional