You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-20 - 15:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack2slot7s.osadl.org (updated Wed May 20, 2026 12:43:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2584101995141,7cyclictest421-21dbus-daemon09:55:011
258410299508,9cyclictest171rcu_preempt08:45:542
2584101994938,8cyclictest1744621-21cups-browsed09:14:441
2584100994741,4cyclictest151ktimers/011:40:560
2584101994537,5cyclictest2745844-21rm11:51:061
2584100994538,4cyclictest1055-21mta-sts-daemon10:30:590
2584100994537,5cyclictest261-21systemd-journal09:55:010
2584100994532,10cyclictest2614094-21systemd-run08:50:010
2584101994436,5cyclictest2781349-21ssh12:36:061
2584100994438,3cyclictest2621003-21fschecks_count09:10:480
2584102994335,5cyclictest2639863-21grep09:35:002
2584101994335,5cyclictest2658507-21awk10:00:011
2584101994335,5cyclictest2588622-21systemd-run07:25:011
2584100994338,3cyclictest0-21swapper/010:10:580
2584100994338,3cyclictest0-21swapper/010:10:580
2584100994329,11cyclictest2753524-21systemctl12:01:040
2584102994230,6cyclictest2712947-21aten_r2power_vo11:10:462
2584102994133,5cyclictest2620081-21diskmemload11:40:512
2584102994132,5cyclictest2688415-21latency_hist10:40:022
2584101994133,5cyclictest2647637-21cat09:45:021
2584101994133,5cyclictest2636537-21sh09:30:451
2584101994133,5cyclictest2633413-21sh09:25:541
2584101994133,5cyclictest2633413-21sh09:25:541
2584100994129,9cyclictest0-21swapper/012:06:080
2584100994127,11cyclictest1051-21mta-sts-daemon11:45:470
2584101994035,2cyclictest0-21swapper/111:50:001
2584101994033,4cyclictest2617102-21sh09:00:011
2584101994032,5cyclictest421-21dbus-daemon11:30:011
2584101994032,5cyclictest421-21dbus-daemon11:30:011
2584101994030,7cyclictest2717269-21latency11:15:541
2584100994025,12cyclictest2610952-21unixbench_multi08:36:040
2584100994025,12cyclictest0-21swapper/010:05:560
2584100994025,12cyclictest0-21swapper/010:05:560
2584102993932,4cyclictest2632111-21idleruntime-cro09:25:012
2584102993932,4cyclictest2628121-21sendmail09:20:012
2584101993931,5cyclictest2773733-21unixbench_multi12:26:091
2584101993931,5cyclictest2670477-21ntp_kernel_pll_10:15:591
2584100993933,4cyclictest2698246-21unixbench_singl10:51:070
2584100993932,4cyclictest0-21swapper/009:45:560
2584100993931,5cyclictest420-21avahi-daemon11:11:070
2584100993931,5cyclictest12350irq/32-eth010:41:000
258410099391,15cyclictest14-21ksoftirqd/007:30:590
2584100993830,5cyclictest421-21dbus-daemon09:59:310
2584100993827,8cyclictest2647621-21sh09:45:010
2584100993823,8cyclictest151ktimers/011:20:570
2584102993730,5cyclictest341rcuc/210:00:552
2584102993730,4cyclictest2618580-21cron09:05:012
2584102993730,3cyclictest0-21swapper/212:36:002
2584102993729,5cyclictest2656047-21needreboot09:55:522
2584102993728,5cyclictest2599340-21aten_r2power_cu08:00:442
2584101993732,2cyclictest0-21swapper/112:00:011
2584101993729,5cyclictest2756034-21latency_hist12:05:021
2584101993729,5cyclictest2662891-21grep10:05:471
2584101993729,5cyclictest2662891-21grep10:05:471
2584101993729,5cyclictest2620201-21latency_hist09:10:021
2584101993729,5cyclictest2593084-21munin-run07:40:011
2584100993730,5cyclictest261-21systemd-journal11:10:010
2584100993729,5cyclictest420-21avahi-daemon11:28:100
2584100993729,5cyclictest420-21avahi-daemon11:28:100
2584100993714,13cyclictest171rcu_preempt09:15:560
2584102993630,3cyclictest0-21swapper/210:10:552
2584102993630,3cyclictest0-21swapper/210:10:552
2584102993629,3cyclictest0-21swapper/211:21:032
2584102993629,3cyclictest0-21swapper/207:40:012
2584102993628,5cyclictest2615645-21latency_hist08:55:022
2584102993628,4cyclictest0-21swapper/211:01:052
2584102993628,3cyclictest0-21swapper/211:35:592
2584102993627,4cyclictest0-21swapper/212:25:012
2584102993624,9cyclictest2677675-21missed_timers10:25:512
2584102993622,9cyclictest600-21gmain08:05:592
2584101993630,3cyclictest2600614-21kworker/u8:8-xprtiod09:40:001
2584101993630,3cyclictest0-21swapper/107:20:001
2584101993630,3cyclictest0-21swapper/107:20:001
2584101993628,5cyclictest2729894-21unixbench-2d11:31:071
2584101993628,5cyclictest2729894-21unixbench-2d11:31:071
2584101993628,5cyclictest2659859-21ssh10:00:581
2584101993627,6cyclictest261-21systemd-journal11:15:011
2584101993627,5cyclictest695-21cron11:01:001
2584100993631,3cyclictest2603595-21kworker/u8:2-xprtiod09:00:010
2584100993630,4cyclictest2696264-21idleruntime-cro10:50:010
2584100993630,4cyclictest1054-21mta-sts-daemon10:00:480
2584100993629,5cyclictest2636003-21rm09:30:010
2584100993629,5cyclictest2636003-21rm09:30:010
2584100993629,4cyclictest2671455-21kworker/u8:10-xprtiod10:20:440
2584100993629,4cyclictest2639863-21grep09:35:000
2584100993628,5cyclictest1744621-21cups-browsed11:52:230
2584100993625,6cyclictest151ktimers/009:35:560
2584102993529,3cyclictest0-21swapper/208:00:012
2584102993527,5cyclictest2692322-21munin-plugin-st10:45:012
2584102993527,5cyclictest2597586-21munin-plugin-st07:55:012
2584102993526,6cyclictest2612590-21grep08:45:012
2584102993520,12cyclictest2641888-21sh09:36:112
2584101993529,3cyclictest0-21swapper/111:40:581
2584101993527,5cyclictest2680418-21ssh10:30:011
2584101993527,5cyclictest2629029-21fschecks_count09:20:471
2584101993516,11cyclictest600-21gmain09:16:591
2584100993529,4cyclictest2733806-21timerwakeupswit11:36:070
2584100993528,4cyclictest2783647-21sendmail12:40:000
2584100993525,7cyclictest0-21swapper/011:20:020
2584100993522,5cyclictest2600614-21kworker/u8:8+xprtiod08:20:440
2584100993518,11cyclictest2701755-21ssh10:56:020
2584100993518,11cyclictest2701755-21ssh10:56:020
2584102993427,4cyclictest2585599-21kthreadd07:15:012
2584102993427,4cyclictest2585599-21kthreadd07:15:012
2584102993427,4cyclictest0-21swapper/209:50:542
2584102993426,5cyclictest2645708-21unixbench_multi09:41:092
2584102993425,6cyclictest2725311-21modprobe11:25:542
2584102993425,6cyclictest2725311-21modprobe11:25:542
2584102993422,9cyclictest2729710-21switchtime11:31:062
2584102993422,9cyclictest2729710-21switchtime11:31:062
2584101993428,3cyclictest0-21swapper/110:40:011
2584101993426,5cyclictest2606594-21sh08:25:001
2584100993426,6cyclictest2729116-21latency11:30:510
2584100993426,6cyclictest2729116-21latency11:30:510
2584100993424,7cyclictest2705489-21needreboot11:00:550
2584100993419,11cyclictest2749250-21ssh11:55:550
2584102993324,6cyclictest2673422-21head10:20:472
2584102993324,5cyclictest2670048-21fschecks_count10:15:512
2584102993324,5cyclictest2590603-21fschecks_count07:30:492
2584102993323,8cyclictest2717132-21if_err_eth011:15:522
2584102993323,6cyclictest2741487-21proc_pri11:46:012
2584102993318,11cyclictest292-21systemd-udevd07:20:592
2584101993327,3cyclictest0-21swapper/112:35:011
2584101993327,3cyclictest0-21swapper/112:35:011
2584101993326,3cyclictest0-21swapper/111:06:011
2584101993325,5cyclictest2684375-21ls10:35:011
2584101993325,5cyclictest261-21systemd-journal11:25:011
2584101993324,5cyclictest2589996-21unixbench_multi07:26:101
2584101993323,7cyclictest2620081-21diskmemload12:05:581
2584101993323,6cyclictest420-21avahi-daemon08:30:011
258410099337,20cyclictest2773227-21ssh12:26:000
2584100993325,5cyclictest1744621-21cups-browsed12:15:390
2584100993323,5cyclictest151ktimers/008:25:590
2584100993322,5cyclictest2751945-21kworker/u8:15+xprtiod12:20:070
2584100993320,3cyclictest0-21swapper/008:30:540
258410099331,28cyclictest2777153-21ssh12:30:580
258410099331,28cyclictest2777153-21ssh12:30:580
2584102993225,4cyclictest1056-21mta-sts-daemon11:51:442
2584102993224,5cyclictest2757148-21needreboot12:05:532
2584102993224,5cyclictest2698176-21unixbench_multi10:51:072
2584101993227,3cyclictest2648290-21aten2.4-expect09:45:461
2584101993225,4cyclictest1051-21mta-sts-daemon10:54:591
2584101993225,4cyclictest0-21swapper/111:36:031
2584101993224,6cyclictest271ktimers/110:10:591
2584101993224,6cyclictest271ktimers/110:10:591
2584101993224,6cyclictest2680434-21kworker/u8:19+xprtiod10:45:011
2584101993223,2cyclictest171rcu_preempt07:40:581
2584100993225,5cyclictest2592115-21fschecks_time07:35:490
2584100993224,6cyclictest2603586-21cut08:15:010
2584100993223,7cyclictest2587073-21cron07:20:000
2584100993223,7cyclictest2587073-21cron07:20:000
2584100993223,4cyclictest0-21swapper/009:20:540
258410099321,4cyclictest171rcu_preempt07:40:580
258410099321,26cyclictest2685726-21sh10:35:550
2584102993125,4cyclictest2584099-21cyclictest09:10:292
2584102993124,4cyclictest2751906-21sh12:00:012
2584101993125,3cyclictest0-21swapper/110:21:001
2584101993123,5cyclictest2761557-21swap12:11:061
2584101993123,5cyclictest2608895-21needreboot08:30:521
2584101993123,5cyclictest2596092-21munin-plugin-st07:50:011
2584100993124,5cyclictest2620082-21/usr/sbin/munin09:10:010
2584100993124,5cyclictest2588614-21systemd-run07:25:010
2584100993124,4cyclictest421-21dbus-daemon10:16:020
2584100993123,3cyclictest0-21swapper/007:51:010
2584102993024,3cyclictest0-21swapper/209:45:452
2584102993024,3cyclictest0-21swapper/207:25:532
2584102993023,3cyclictest0-21swapper/211:00:002
2584102993023,3cyclictest0-21swapper/211:00:002
2584102993022,4cyclictest0-21swapper/212:16:022
2584102993022,4cyclictest0-21swapper/211:06:012
2584102993017,10cyclictest2775929-21latency_hist12:30:032
2584102993015,12cyclictest2694143-21timerwakeupswit10:46:102
2584101993025,2cyclictest0-21swapper/108:40:011
2584101993023,4cyclictest0-21swapper/112:20:021
2584101993018,9cyclictest2704215-21systemctl11:00:001
2584101993018,9cyclictest2704215-21systemctl11:00:001
258410099306,3cyclictest171rcu_preempt07:55:590
2584100993025,3cyclictest1055-21mta-sts-daemon12:11:040
2584100993019,4cyclictest171rcu_preempt08:50:540
2584100993013,6cyclictest171rcu_preempt10:25:530
2584102992921,5cyclictest421-21dbus-daemon10:07:452
2584102992921,5cyclictest421-21dbus-daemon10:07:452
2584102992913,3cyclictest171rcu_preempt08:30:582
2584101992923,3cyclictest0-21swapper/110:50:011
2584101992922,4cyclictest2603102-21proc_pri08:11:031
2584101992922,3cyclictest0-21swapper/108:16:021
2584101992921,5cyclictest2597598-21systemd-run07:55:011
2584101992914,4cyclictest2770338-21ssh12:22:091
2584100992916,2cyclictest171rcu_preempt09:00:590
2584102992822,3cyclictest1055-21mta-sts-daemon12:33:082
2584102992822,3cyclictest1055-21mta-sts-daemon12:33:082
2584102992822,3cyclictest0-21swapper/208:20:012
258410299282,21cyclictest2753321-21ssh12:00:582
2584102992821,4cyclictest1054-21mta-sts-daemon12:12:012
2584102992821,4cyclictest0-21swapper/210:30:452
2584102992814,5cyclictest1744621-21cups-browsed07:15:082
2584102992814,5cyclictest1744621-21cups-browsed07:15:082
2584101992823,2cyclictest0-21swapper/108:41:081
2584101992820,5cyclictest2612854-21head08:45:441
2584100992819,5cyclictest151ktimers/008:15:550
2584100992813,3cyclictest171rcu_preempt08:00:550
2584100992810,3cyclictest171rcu_preempt07:25:590
2584102992720,4cyclictest2594596-21cron07:45:012
2584102992719,5cyclictest2603604-21systemd-run08:15:012
2584101992717,3cyclictest171rcu_preempt08:51:021
258410099276,4cyclictest0-21swapper/008:05:550
258410099273,7cyclictest14-21ksoftirqd/008:40:580
258410299269,4cyclictest171rcu_preempt09:25:552
258410299269,4cyclictest171rcu_preempt09:25:552
2584102992618,5cyclictest2608103-21systemd-run08:30:012
2584102992618,5cyclictest0-21swapper/208:40:012
2584101992619,5cyclictest271ktimers/107:56:011
2584100992619,5cyclictest191rcuc/007:11:000
2584100992619,5cyclictest191rcuc/007:11:000
2584100992616,2cyclictest171rcu_preempt07:45:540
2584102992514,8cyclictest2616816-21wc08:56:062
2584101992519,3cyclictest0-21swapper/107:35:011
2584101992517,5cyclictest2618396-21timerwakeupswit09:01:081
2584101992517,5cyclictest2600592-21cron08:05:011
258410299248,3cyclictest171rcu_preempt09:05:572
2584101992414,3cyclictest2602259-21latency_hist08:10:021
258410099245,17cyclictest151ktimers/007:10:000
258410099245,17cyclictest151ktimers/007:10:000
2584102992316,5cyclictest0-21swapper/208:20:582
2584101992317,3cyclictest1056-21mta-sts-daemon07:10:051
2584101992317,3cyclictest1056-21mta-sts-daemon07:10:051
258410399212,10cyclictest0-21swapper/311:55:543
258410399211,11cyclictest0-21swapper/312:05:543
258410299212,14cyclictest2595559-21ntpq07:46:002
2584101992114,5cyclictest2584201-21latency_hist07:10:021
2584101992114,5cyclictest2584201-21latency_hist07:10:021
258410399207,9cyclictest0-21swapper/310:45:573
258410399203,9cyclictest0-21swapper/310:56:003
258410399203,9cyclictest0-21swapper/310:56:003
258410399203,8cyclictest0-21swapper/309:35:583
258410399203,10cyclictest0-21swapper/309:45:563
258410399193,9cyclictest0-21swapper/307:37:483
258410399193,8cyclictest0-21swapper/310:15:593
258410399193,10cyclictest0-21swapper/310:50:573
258410399192,8cyclictest0-21swapper/309:25:533
258410399192,8cyclictest0-21swapper/309:25:533
258410399192,12cyclictest0-21swapper/310:01:013
258410399192,10cyclictest0-21swapper/312:00:593
258410399192,10cyclictest0-21swapper/311:20:573
258410399191,12cyclictest0-21swapper/310:10:573
258410399191,12cyclictest0-21swapper/310:10:573
258410399191,11cyclictest0-21swapper/309:31:043
258410399184,8cyclictest0-21swapper/311:31:013
258410399184,8cyclictest0-21swapper/311:31:013
258410399182,9cyclictest0-21swapper/309:50:533
258410399182,10cyclictest0-21swapper/310:35:533
258410399182,10cyclictest0-21swapper/310:35:023
258410399182,10cyclictest0-21swapper/310:20:443
258410399182,10cyclictest0-21swapper/309:55:533
2584102991813,2cyclictest0-21swapper/207:10:022
2584102991813,2cyclictest0-21swapper/207:10:022
258410399173,8cyclictest0-21swapper/308:00:013
258410399172,9cyclictest0-21swapper/307:45:573
258410399172,8cyclictest0-21swapper/310:40:573
258410399172,8cyclictest0-21swapper/309:20:533
258410399172,11cyclictest0-21swapper/311:40:533
258410399171,9cyclictest0-21swapper/312:30:573
258410399171,9cyclictest0-21swapper/312:30:573
258410399171,10cyclictest0-21swapper/312:35:543
258410399171,10cyclictest0-21swapper/312:25:583
258410399162,8cyclictest0-21swapper/312:15:463
258410399162,7cyclictest0-21swapper/311:50:593
258410399162,7cyclictest0-21swapper/309:10:533
258410399162,10cyclictest0-21swapper/311:00:573
258410399162,10cyclictest0-21swapper/310:05:523
258410399162,10cyclictest0-21swapper/310:05:523
258410399161,10cyclictest0-21swapper/308:35:523
258410399154,7cyclictest0-21swapper/311:45:553
258410399152,9cyclictest0-21swapper/311:35:533
258410399152,8cyclictest0-21swapper/311:25:543
258410399152,8cyclictest0-21swapper/311:25:543
258410399152,7cyclictest0-21swapper/311:05:453
258410399151,9cyclictest0-21swapper/311:11:023
258410399151,8cyclictest0-21swapper/308:40:583
258410399151,8cyclictest0-21swapper/307:50:563
258410399151,7cyclictest0-21swapper/308:00:553
258410399151,10cyclictest0-21swapper/312:20:563
258410399151,10cyclictest0-21swapper/311:15:573
258410399142,8cyclictest0-21swapper/310:25:533
258410399142,7cyclictest0-21swapper/309:10:023
258410399142,7cyclictest0-21swapper/307:15:563
258410399142,7cyclictest0-21swapper/307:15:563
258410399142,5cyclictest0-21swapper/307:10:573
258410399142,5cyclictest0-21swapper/307:10:573
258410399141,9cyclictest0-21swapper/312:10:543
258410399141,9cyclictest0-21swapper/309:40:553
258410399141,9cyclictest0-21swapper/309:15:533
258410399141,9cyclictest0-21swapper/308:51:013
258410399141,8cyclictest0-21swapper/308:20:013
258410399141,6cyclictest0-21swapper/307:40:533
258410399141,6cyclictest0-21swapper/307:30:583
258410399141,6cyclictest0-21swapper/307:25:573
258410399131,9cyclictest0-21swapper/308:45:543
258410399131,9cyclictest0-21swapper/308:05:533
258410399131,8cyclictest0-21swapper/308:55:553
258410399131,7cyclictest0-21swapper/309:00:593
258410399131,7cyclictest0-21swapper/308:20:523
258410399131,7cyclictest0-21swapper/308:10:563
258410399131,6cyclictest0-21swapper/308:30:533
258410399131,6cyclictest0-21swapper/308:25:553
258410399111,7cyclictest0-21swapper/307:20:543
25841039982,4cyclictest0-21swapper/307:10:013
25841039982,4cyclictest0-21swapper/307:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional