You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-29 - 08:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Wed Jul 29, 2026 00:43:09)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2588658995334,9cyclictest2726644-21sh23:55:480
2588658995237,11cyclictest0-21swapper/023:35:490
258866099518,8cyclictest2624647-21diskmemload21:50:532
2588658995131,16cyclictest2693403-21switchtime23:01:010
2588659995038,7cyclictest843869-21cups-browsed22:52:391
2588659995022,21cyclictest843869-21cups-browsed21:45:521
2588659994941,5cyclictest2621632-21cron21:00:011
258866099484,41cyclictest2634998-21ssh21:25:472
2588658994810,34cyclictest2641167-21ssh21:35:490
2588660994736,7cyclictest1-21systemd00:00:012
2588658994724,16cyclictest12650irq/32-eth000:15:580
2588660994635,7cyclictest843869-21cups-browsed23:25:222
2588660994635,7cyclictest843869-21cups-browsed23:25:222
2588659994630,11cyclictest420-21avahi-daemon00:39:201
2588659994537,5cyclictest2695696-21fschecks_time23:05:441
2588659994537,4cyclictest0-21swapper/122:30:541
2588659994537,4cyclictest0-21swapper/122:30:541
2588659994536,5cyclictest1003-21mta-sts-daemon22:03:361
2588660994436,5cyclictest2615642-21grep20:40:002
2588660994436,5cyclictest2615642-21grep20:40:002
2588660994433,7cyclictest420-21avahi-daemon21:48:482
258866099442,33cyclictest2741750-21ssh00:20:482
2588659994435,5cyclictest2623228-21latency_hist21:05:021
2588660994332,8cyclictest1-21systemd00:30:022
258865999435,32cyclictest2662580-21ssh22:10:461
2588659994335,5cyclictest0-21swapper/100:00:011
2588658994336,4cyclictest2730189-21timerandwakeup00:01:090
2588660994234,5cyclictest2655348-21cron22:00:012
2588659994233,6cyclictest0-21swapper/121:20:001
2588658994234,5cyclictest2732044-21aten_r2power_cu00:05:410
2588660994032,5cyclictest2677222-21aten2.4-expect22:35:372
2588660994029,8cyclictest2667505-21latency_hist22:20:022
2588660994029,7cyclictest420-21avahi-daemon00:19:502
2588660994029,7cyclictest2695516-21aten_r2power_po23:05:412
2588659994032,5cyclictest2617185-21latency_hist20:45:011
2588658994032,5cyclictest1-21systemd20:30:010
2588658994032,5cyclictest1-21systemd20:30:010
2588658994031,6cyclictest2691864-21ssh23:00:010
2588658994031,5cyclictest151ktimers/019:30:500
2588658994025,4cyclictest0-21swapper/019:50:010
2588658994025,4cyclictest0-21swapper/019:50:010
2588660993931,5cyclictest0-21swapper/222:00:502
2588659993930,5cyclictest2711894-21unixbench_multi23:31:011
2588659993930,5cyclictest2711894-21unixbench_multi23:31:011
2588659993929,6cyclictest487-21rs:main1
2588658993934,3cyclictest2599156-21grep19:45:000
2588658993932,4cyclictest1002-21mta-sts-daemon21:25:580
2588658993928,7cyclictest2741661-21missed_timers00:20:460
2588658993926,4cyclictest2620789-21kernelversion20:55:450
2588658993926,10cyclictest0-21swapper/022:25:010
2588658993924,3cyclictest171rcu_preempt22:30:550
2588658993924,3cyclictest171rcu_preempt22:30:550
2588658993924,12cyclictest0-21swapper/022:00:010
2588660993830,5cyclictest2736214-21timerwakeupswit00:11:002
2588660993830,5cyclictest2736214-21timerwakeupswit00:11:002
2588660993829,6cyclictest421-21dbus-daemon23:20:012
2588659993829,6cyclictest421-21dbus-daemon00:10:011
2588659993829,6cyclictest420-21avahi-daemon21:24:081
258865999382,27cyclictest2741968-21ssh00:20:581
2588658993831,4cyclictest2698162-21latency_hist23:10:030
2588658993829,6cyclictest2745010-21sendmail_mailst00:25:570
2588658993829,6cyclictest2591735-21latency_hist19:20:020
258865899381,33cyclictest2686987-21sh22:50:540
2588660993730,4cyclictest2685859-21latency_hist22:50:022
2588660993728,5cyclictest2752691-21ssh00:40:002
2588660993727,6cyclictest843865-21cupsd22:53:132
2588659993731,3cyclictest0-21swapper/120:20:011
2588659993730,4cyclictest0-21swapper/123:15:011
2588659993729,5cyclictest2610000-21aten_r2power_fr20:20:371
2588659993728,6cyclictest2744523-21kernelversion00:25:441
2588659993728,4cyclictest0-21swapper/121:55:551
258865899374,24cyclictest665-21snmpd22:35:500
2588658993728,6cyclictest2617143-21cron20:45:000
2588658993725,9cyclictest2624651-21grep21:10:010
2588658993723,3cyclictest0-21swapper/023:46:030
258865899372,29cyclictest2735735-21ssh00:10:480
258865899372,29cyclictest2735735-21ssh00:10:480
2588658993721,3cyclictest0-21swapper/021:40:480
2588660993627,6cyclictest265-21systemd-journal19:55:012
2588660993627,6cyclictest2590070-21head19:11:052
2588659993628,6cyclictest2643140-21kworker/u8:5-xprtiod21:45:011
2588659993628,5cyclictest2666282-21unixbench_singl22:16:021
2588659993628,5cyclictest2624674-21systemd-run21:10:021
2588659993626,6cyclictest2747864-21ssh00:30:511
258865899366,8cyclictest14-21ksoftirqd/021:22:290
2588658993629,4cyclictest2609637-21cron20:20:010
2588658993628,5cyclictest843869-21cups-browsed21:53:230
2588658993624,9cyclictest2608025-21tail20:10:590
2588660993527,5cyclictest2617674-21fschecks_count20:45:422
2588660993526,6cyclictest2620440-21aten_r2power_en20:55:402
2588660993525,8cyclictest2613460-21needreboot20:30:452
2588660993525,8cyclictest2613460-21needreboot20:30:452
2588660993524,7cyclictest2707373-21latency_hist23:25:032
2588659993530,2cyclictest0-21swapper/100:20:011
2588659993528,3cyclictest0-21swapper/100:00:441
2588658993528,4cyclictest1002-21mta-sts-daemon22:14:240
2588658993524,6cyclictest14-21ksoftirqd/023:50:520
258866099345,23cyclictest2668756-21ssh22:20:522
258866099344,21cyclictest428-21mta-sts-daemon23:30:542
258866099344,21cyclictest428-21mta-sts-daemon23:30:542
2588660993426,5cyclictest2661692-21latency_hist22:10:022
2588660993426,5cyclictest2605524-21aten_r2power_po20:05:382
2588660993425,5cyclictest1003-21mta-sts-daemon21:22:322
2588660993419,7cyclictest0-21swapper/222:10:482
2588659993429,3cyclictest0-21swapper/120:40:001
2588659993429,3cyclictest0-21swapper/120:40:001
2588659993428,3cyclictest0-21swapper/122:40:001
2588659993426,5cyclictest2627709-21sh21:15:011
258865899345,14cyclictest2671688-21ssh22:25:460
258865899344,14cyclictest2665754-21ntpq22:15:530
2588658993427,4cyclictest1000-21mta-sts-daemon23:27:400
2588658993427,4cyclictest1000-21mta-sts-daemon23:27:400
2588658993425,7cyclictest2638689-21unixbench_singl21:31:000
2588658993411,6cyclictest14-21ksoftirqd/022:05:490
258866099333,23cyclictest2680968-21ssh22:41:022
2588660993326,4cyclictest1000-21mta-sts-daemon00:07:532
2588660993325,5cyclictest2643104-21munin-run21:40:012
258865999334,16cyclictest428-21mta-sts-daemon23:40:511
2588659993326,4cyclictest1002-21mta-sts-daemon21:39:031
2588659993326,4cyclictest1000-21mta-sts-daemon22:26:311
2588659993325,5cyclictest2690512-21unixbench_multi22:56:021
2588659993325,5cyclictest2667984-21aten_r2power_cu22:20:371
2588659993324,7cyclictest2597641-21sendmail19:40:011
2588659993323,7cyclictest2593155-21latency_hist19:25:011
258865999332,25cyclictest2680860-21ssh22:40:541
258865899338,5cyclictest14-21ksoftirqd/023:30:460
258865899338,5cyclictest14-21ksoftirqd/023:30:460
2588658993325,5cyclictest2682735-21cron22:45:000
2588658993324,7cyclictest2750715-21grep00:35:450
2588658993324,5cyclictest1001-21mta-sts-daemon21:15:460
258866099325,17cyclictest2714410-21ssh23:35:502
2588660993224,5cyclictest1002-21mta-sts-daemon00:30:472
2588660993223,7cyclictest2720454-21needreboot23:45:482
2588660993223,6cyclictest421-21dbus-daemon19:20:012
2588660993222,7cyclictest2600659-21sh19:50:012
2588660993222,7cyclictest2600659-21sh19:50:012
258866099322,26cyclictest2644339-21ssh21:40:522
2588660993216,11cyclictest2693439-21timerandwakeup23:01:012
2588659993225,4cyclictest2735315-21fschecks_count00:10:401
2588659993225,4cyclictest2735315-21fschecks_count00:10:401
2588659993225,4cyclictest1001-21mta-sts-daemon23:20:261
2588659993224,5cyclictest2661572-21latency_hist22:10:021
2588659993222,6cyclictest2702094-21needreboot23:15:471
2588658993222,8cyclictest151ktimers/022:45:560
2588658993218,11cyclictest2622187-21fschecks_time21:00:400
2588658993216,4cyclictest14-21ksoftirqd/023:18:040
258866099316,18cyclictest2729907-21ssh00:01:032
2588660993125,4cyclictest2699700-21unixbench_multi23:11:002
2588660993124,4cyclictest1003-21mta-sts-daemon19:35:482
2588660993123,5cyclictest2691862-21cron23:00:012
2588659993124,4cyclictest2638663-21tail21:31:001
2588659993124,4cyclictest2593690-21fschecks_time19:25:421
2588659993123,5cyclictest439-21systemd-logind21:55:011
2588659993123,5cyclictest2601173-21fschecks_time19:50:421
2588659993122,5cyclictest2634919-21needreboot21:25:441
2588659993117,10cyclictest421-21dbus-daemon23:48:071
2588658993124,4cyclictest1003-21mta-sts-daemon00:33:350
2588658993124,4cyclictest0-21swapper/020:30:510
2588658993124,4cyclictest0-21swapper/020:30:510
2588658993111,5cyclictest14-21ksoftirqd/020:20:490
2588660993023,4cyclictest2595734-21mailstats19:30:542
2588660993023,3cyclictest0-21swapper/223:45:022
2588660993020,6cyclictest2722910-21aten_r2power_cu23:50:392
2588660993020,6cyclictest2592177-21fschecks_count19:20:432
2588659993023,4cyclictest1000-21mta-sts-daemon22:46:341
2588659993022,5cyclictest2693382-21switchtime23:01:011
2588659993022,5cyclictest265-21systemd-journal20:35:011
2588659993022,5cyclictest265-21systemd-journal20:35:011
2588659993022,5cyclictest2600254-21systemctl19:45:571
2588659993022,5cyclictest2600254-21systemctl19:45:571
2588659993021,5cyclictest2611528-21aten_r2power_po20:25:391
2588659993021,5cyclictest2611528-21aten_r2power_po20:25:391
258865899308,3cyclictest14-21ksoftirqd/019:25:480
2588658993025,2cyclictest0-21swapper/020:00:000
2588658993022,6cyclictest2704853-21fschecks_count23:20:430
2588658993022,5cyclictest265-21systemd-journal20:10:010
2588658993021,6cyclictest2590164-21systemd-run19:15:010
2588658993010,17cyclictest191rcuc/020:00:520
2588660992923,3cyclictest0-21swapper/222:30:012
258866099292,22cyclictest2624647-21diskmemload21:15:532
2588660992920,6cyclictest2611156-21cron20:25:012
2588659992923,3cyclictest0-21swapper/119:35:011
258865999292,23cyclictest2714378-21ssh23:35:481
2588659992920,6cyclictest2588665-21systemd-run19:10:011
2588659992919,7cyclictest2620140-21munin-plugin-st20:55:021
2588659992919,7cyclictest0-21swapper/123:25:441
2588659992919,7cyclictest0-21swapper/123:25:441
258865899293,7cyclictest14-21ksoftirqd/021:12:120
2588658992923,3cyclictest0-21swapper/023:40:400
2588658992922,3cyclictest0-21swapper/023:10:550
2588658992920,3cyclictest171rcu_preempt19:21:030
258866099282,16cyclictest428-21mta-sts-daemon21:00:472
2588660992821,4cyclictest2625115-21aten2.4-expect21:10:382
2588660992821,3cyclictest0-21swapper/221:30:552
2588660992818,7cyclictest2674508-21kernelversion22:30:462
2588660992818,7cyclictest2674508-21kernelversion22:30:462
2588659992823,2cyclictest0-21swapper/120:00:001
2588659992821,4cyclictest0-21swapper/120:15:011
2588659992821,3cyclictest0-21swapper/119:10:541
258865899285,4cyclictest14-21ksoftirqd/021:46:110
2588658992820,3cyclictest171rcu_preempt19:35:490
2588658992813,12cyclictest2601173-21fschecks_time19:50:420
258865899281,22cyclictest2656532-21ntpq22:00:500
2588660992719,5cyclictest0-21swapper/219:45:002
2588659992719,5cyclictest843869-21cups-browsed20:49:041
2588659992717,7cyclictest843865-21cupsd19:45:011
258865899277,7cyclictest14-21ksoftirqd/020:50:540
2588658992721,4cyclictest191rcuc/020:45:350
258866099262,19cyclictest843869-21cups-browsed20:25:472
258866099262,19cyclictest843869-21cups-browsed20:25:472
2588660992618,6cyclictest0-21swapper/221:10:022
2588660992617,6cyclictest2602509-21aten_r2power_po19:55:392
2588659992620,3cyclictest0-21swapper/120:10:011
2588658992620,3cyclictest0-21swapper/020:40:000
2588658992620,3cyclictest0-21swapper/020:40:000
2588659992516,6cyclictest843869-21cups-browsed20:02:391
2588658992512,3cyclictest171rcu_preempt19:10:020
2588660992417,5cyclictest2605070-21unixbench_singl20:01:022
2588660992417,3cyclictest351ktimers/220:10:502
2588660992416,6cyclictest2616723-21sendmail_mailst20:40:562
2588660992416,5cyclictest2593927-21missed_timers19:25:452
2588659992418,3cyclictest0-21swapper/119:20:011
2588660992314,7cyclictest2608837-21latency20:15:462
258866099220,20cyclictest351ktimers/220:50:542
258866099202,12cyclictest421-21dbus-daemon19:10:012
258866199172,8cyclictest0-21swapper/322:55:503
258866199172,8cyclictest0-21swapper/300:10:483
258866199172,8cyclictest0-21swapper/300:10:483
258866199163,9cyclictest0-21swapper/323:50:523
258866199162,7cyclictest0-21swapper/321:45:483
258866199161,7cyclictest0-21swapper/300:20:483
258866199152,8cyclictest0-21swapper/320:50:023
258866199152,7cyclictest0-21swapper/322:10:513
258866199151,8cyclictest0-21swapper/323:45:503
258866199151,7cyclictest0-21swapper/322:30:513
258866199151,7cyclictest0-21swapper/322:30:513
258866199151,7cyclictest0-21swapper/321:15:503
258866199151,7cyclictest0-21swapper/321:10:503
258866199151,7cyclictest0-21swapper/319:40:523
258866199151,6cyclictest0-21swapper/320:15:563
258866199151,6cyclictest0-21swapper/319:25:503
258866199143,4cyclictest0-21swapper/320:55:503
258866199142,7cyclictest0-21swapper/322:05:553
258866199142,7cyclictest0-21swapper/321:55:523
258866199142,6cyclictest0-21swapper/300:15:553
258866199142,5cyclictest0-21swapper/320:05:463
258866199141,8cyclictest0-21swapper/322:45:023
258866199141,8cyclictest0-21swapper/321:40:373
258866199141,7cyclictest0-21swapper/323:15:543
258866199141,7cyclictest0-21swapper/323:05:523
258866199141,7cyclictest0-21swapper/323:00:533
258866199141,7cyclictest0-21swapper/321:50:533
258866199141,7cyclictest0-21swapper/321:35:573
258866199141,7cyclictest0-21swapper/320:40:493
258866199141,6cyclictest0-21swapper/320:30:483
258866199141,6cyclictest0-21swapper/320:30:483
258866199141,6cyclictest0-21swapper/320:00:483
258866199141,6cyclictest0-21swapper/319:30:523
258866199141,6cyclictest0-21swapper/319:10:553
258866199141,6cyclictest0-21swapper/300:35:463
258866199133,7cyclictest0-21swapper/323:55:523
258866199133,6cyclictest0-21swapper/319:50:463
258866199132,7cyclictest0-21swapper/323:35:503
258866199132,6cyclictest0-21swapper/323:25:553
258866199132,6cyclictest0-21swapper/323:25:553
258866199132,6cyclictest0-21swapper/322:00:493
258866199131,8cyclictest0-21swapper/300:05:493
258866199131,7cyclictest0-21swapper/322:15:493
258866199131,7cyclictest0-21swapper/300:30:403
258866199131,6cyclictest0-21swapper/320:10:493
258866199131,6cyclictest0-21swapper/319:45:493
258866199131,6cyclictest0-21swapper/319:45:493
258866199130,6cyclictest0-21swapper/321:05:463
258866199124,5cyclictest0-21swapper/323:40:463
258866199123,6cyclictest0-21swapper/323:30:393
258866199123,6cyclictest0-21swapper/323:30:393
258866199122,6cyclictest0-21swapper/321:30:453
258866199122,6cyclictest0-21swapper/300:01:043
258866199122,5cyclictest0-21swapper/320:40:013
258866199122,5cyclictest0-21swapper/320:40:013
258866199122,5cyclictest0-21swapper/319:15:533
258866199121,7cyclictest0-21swapper/323:20:523
258866199121,7cyclictest0-21swapper/322:35:413
258866199121,7cyclictest0-21swapper/322:20:453
258866199121,7cyclictest0-21swapper/321:20:483
258866199121,6cyclictest0-21swapper/323:10:383
258866199121,6cyclictest0-21swapper/322:25:413
258866199121,6cyclictest0-21swapper/321:00:463
258866199121,6cyclictest0-21swapper/320:50:583
258866199121,6cyclictest0-21swapper/300:25:433
258866199121,5cyclictest0-21swapper/320:20:503
258866199120,5cyclictest0-21swapper/320:25:473
258866199120,5cyclictest0-21swapper/320:25:473
258866199112,6cyclictest0-21swapper/321:25:473
258866199112,5cyclictest0-21swapper/319:25:013
258866199112,4cyclictest0-21swapper/319:35:473
258866199111,6cyclictest0-21swapper/319:55:483
258866199104,4cyclictest0-21swapper/322:45:503
25886619991,5cyclictest0-21swapper/322:50:363
25886619981,4cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional