You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-14 - 22:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Sun Jun 14, 2026 12:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2666359995931,19cyclictest627-21lldpd11:25:240
2666361994937,8cyclictest1863115-21cups-browsed09:48:422
2666360994938,7cyclictest421-21dbus-daemon11:15:281
2666359994837,7cyclictest151ktimers/009:40:240
266636199462,36cyclictest2778572-21ssh11:10:232
2666359994638,5cyclictest421-21dbus-daemon12:30:010
2666359994635,7cyclictest2813008-21ssh12:05:250
2666359994627,14cyclictest12650irq/32-eth009:35:290
2666359994627,14cyclictest12650irq/32-eth009:35:290
2666361994536,6cyclictest421-21dbus-daemon08:00:012
2666361994536,6cyclictest421-21dbus-daemon08:00:012
2666361994520,5cyclictest171rcu_preempt11:35:282
2666361994520,5cyclictest171rcu_preempt11:35:282
266636099447,31cyclictest2791084-21ssh11:30:221
2666360994437,4cyclictest0-21swapper/111:20:311
2666360994432,9cyclictest2771641-21sh11:00:141
2666361994334,4cyclictest0-21swapper/207:45:012
266636099434,34cyclictest2741150-21ssh10:10:291
2666360994335,5cyclictest2697831-21munin-plugin-st08:55:011
2666361994132,5cyclictest1-21systemd10:30:012
2666361994132,5cyclictest0-21swapper/210:35:232
266636199411,31cyclictest2686751-21sshd08:15:272
2666360994133,5cyclictest2808954-21latency_hist12:00:021
2666360994133,5cyclictest2808954-21latency_hist12:00:021
2666360994133,4cyclictest0-21swapper/111:40:251
2666360994133,4cyclictest0-21swapper/111:40:251
2666359994133,5cyclictest2752604-21sh10:30:010
2666359994124,12cyclictest151ktimers/007:25:240
2666359994122,9cyclictest12650irq/32-eth010:15:250
2666361994032,5cyclictest2694843-21debian-sa108:45:012
2666361994032,4cyclictest0-21swapper/211:00:252
266636199401,34cyclictest2763068-21ssh10:45:302
2666360994031,6cyclictest421-21dbus-daemon08:05:351
2666360994031,5cyclictest2831608-21ssh12:35:271
2666360994031,5cyclictest2831608-21ssh12:35:271
2666360994030,6cyclictest420-21avahi-daemon08:35:011
2666359994024,4cyclictest171rcu_preempt11:15:140
266636199393,25cyclictest2775637-21ssh11:05:322
2666361993929,5cyclictest2760155-21mailstats10:40:432
2666361993923,10cyclictest420-21avahi-daemon12:34:472
2666360993931,5cyclictest2768815-21fschecks_time10:55:171
2666360993931,5cyclictest2673954-21latency_hist07:35:021
2666360993931,4cyclictest0-21swapper/111:25:261
2666360993928,8cyclictest2827513-21/usr/sbin/munin12:30:021
2666359993926,6cyclictest1002-21mta-sts-daemon11:35:270
2666359993926,6cyclictest1002-21mta-sts-daemon11:35:270
2666359993926,10cyclictest12650irq/32-eth010:30:210
2666361993830,5cyclictest2671378-21fschecks_count07:25:192
2666361993828,7cyclictest2815145-21systemd-run12:10:012
2666361993827,7cyclictest420-21avahi-daemon09:52:232
2666360993832,3cyclictest0-21swapper/111:05:261
2666360993831,4cyclictest2752581-21idleruntime-cro10:30:011
2666360993831,3cyclictest0-21swapper/111:10:271
2666360993831,3cyclictest0-21swapper/109:35:281
2666360993831,3cyclictest0-21swapper/109:35:281
2666360993830,5cyclictest2718122-21munin-run09:35:001
2666360993830,5cyclictest2718122-21munin-run09:35:001
2666360993830,5cyclictest2687256-21unixbench_singl08:15:361
2666360993830,3cyclictest0-21swapper/112:20:291
2666360993829,7cyclictest2816043-21needreboot12:10:221
2666360993829,6cyclictest420-21avahi-daemon12:07:261
2666360993829,6cyclictest2805731-21cron11:55:001
2666360993829,6cyclictest2805731-21cron11:55:001
2666360993829,6cyclictest1-21systemd07:50:011
2666360993829,4cyclictest0-21swapper/109:00:241
2666360993829,4cyclictest0-21swapper/109:00:241
266636099382,31cyclictest2828491-21ssh12:30:241
2666359993831,4cyclictest0-21swapper/011:30:260
2666359993830,5cyclictest2802790-21latency_hist11:50:020
2666359993830,5cyclictest2802790-21latency_hist11:50:020
2666359993830,5cyclictest2700780-21unixbench_singl09:00:360
2666359993830,5cyclictest2700780-21unixbench_singl09:00:360
2666359993812,18cyclictest12650irq/32-eth012:35:270
2666359993812,18cyclictest12650irq/32-eth012:35:270
266636199371,29cyclictest2731672-21ssh09:55:262
2666360993730,4cyclictest2672420-21latency_hist07:30:021
2666360993728,6cyclictest420-21avahi-daemon10:51:041
2666360993727,4cyclictest0-21swapper/110:30:241
2666360993719,16cyclictest2761100-21kworker/u8:6+xprtiod11:45:281
2666360993719,16cyclictest2761100-21kworker/u8:6+xprtiod11:45:281
266635999376,24cyclictest2806806-21ssh11:55:270
266635999376,24cyclictest2806806-21ssh11:55:270
266635999373,26cyclictest2797334-21ssh11:40:240
266635999373,26cyclictest2797334-21ssh11:40:240
2666359993732,3cyclictest2816188-21sshd12:10:270
2666359993730,5cyclictest2756946-21wc10:35:310
2666359993727,4cyclictest2708632-21cron09:20:000
266636199363,26cyclictest1863115-21cups-browsed08:20:262
2666361993628,5cyclictest421-21dbus-daemon10:05:012
2666361993627,5cyclictest2711816-21basename09:25:002
2666361993627,5cyclictest2711816-21basename09:25:002
2666361993626,8cyclictest1000-21mta-sts-daemon11:54:392
2666361993626,8cyclictest1000-21mta-sts-daemon11:54:392
2666361993626,6cyclictest2818998-21kernelversion12:15:202
2666361993626,6cyclictest2800690-21ps11:45:332
2666361993626,6cyclictest2800690-21ps11:45:332
266636099363,24cyclictest2706584-21ssh09:15:291
2666359993630,4cyclictest2737309-21aten_r2power_cu10:05:190
2666359993628,6cyclictest2681873-21fschecks_time08:00:220
266635999362,30cyclictest2712881-21ssh09:25:270
266635999362,30cyclictest2712881-21ssh09:25:270
2666361993526,6cyclictest2793278-21latency_hist11:35:022
2666361993526,6cyclictest2718123-21grep09:35:002
2666361993526,6cyclictest2718123-21grep09:35:002
2666361993522,5cyclictest2761100-21kworker/u8:6+xprtiod11:00:012
2666361993511,17cyclictest421-21dbus-daemon11:15:282
2666360993528,4cyclictest2710209-21unixbench-2d09:20:371
2666360993528,4cyclictest2710209-21unixbench-2d09:20:371
2666360993528,3cyclictest0-21swapper/110:25:021
2666360993525,7cyclictest2703364-21ntp_kernel_pll_09:10:271
2666360993525,7cyclictest2703364-21ntp_kernel_pll_09:10:271
266636099352,27cyclictest2819267-21ssh12:15:251
2666359993530,3cyclictest1002-21mta-sts-daemon07:45:250
2666359993529,4cyclictest2780731-21sh11:15:010
266635999352,27cyclictest2728536-21sshd09:50:260
266635999352,25cyclictest2759936-21ssh10:40:350
266636199343,6cyclictest36-21ksoftirqd/209:15:242
266636199343,25cyclictest2797486-21ssh11:40:292
266636199343,25cyclictest2797486-21ssh11:40:292
2666361993427,4cyclictest2788558-21grep11:25:372
2666360993429,2cyclictest0-21swapper/112:00:291
2666360993426,4cyclictest1000-21mta-sts-daemon10:15:241
2666360993425,6cyclictest623-21sshd09:47:271
2666359993428,4cyclictest2663823-21kworker/u8:15-xprtiod07:35:010
2666359993427,4cyclictest2667914-21latency_hist07:15:010
266635999342,25cyclictest2828491-21ssh12:30:240
2666359993419,10cyclictest12650irq/32-eth010:55:240
266636199336,18cyclictest2702337-21diskmemload10:20:242
2666361993327,3cyclictest2745136-21kworker/u8:7-xprtiod11:25:022
2666361993326,4cyclictest2673779-21unixbench_singl07:30:382
2666361993325,5cyclictest2816670-21ssh12:10:362
2666361993325,4cyclictest0-21swapper/209:10:002
2666361993325,4cyclictest0-21swapper/209:10:002
2666361993324,6cyclictest2702973-21fschecks_time09:10:202
2666361993324,6cyclictest2702973-21fschecks_time09:10:202
266636199332,24cyclictest2738115-21ssh10:05:352
2666361993321,3cyclictest171rcu_preempt09:35:262
2666361993321,3cyclictest171rcu_preempt09:35:262
2666360993325,5cyclictest2699317-21latency_hist09:00:001
2666360993325,4cyclictest2702358-21kthreadd09:10:011
2666360993325,4cyclictest2702358-21kthreadd09:10:011
2666360993324,4cyclictest0-21swapper/109:50:281
2666360993322,8cyclictest623-21sshd07:35:271
266636099332,26cyclictest2756727-21ssh10:35:241
266636099332,24cyclictest2759891-21ssh10:40:321
266635999333,21cyclictest12650irq/32-eth009:45:260
2666359993328,3cyclictest0-21swapper/009:20:270
2666359993328,3cyclictest0-21swapper/009:20:270
2666359993327,4cyclictest2666358-21cyclictest10:55:010
2666359993327,3cyclictest0-21swapper/011:05:270
2666359993325,6cyclictest2687869-21fschecks_count08:20:180
2666359993317,13cyclictest265-21systemd-journal08:52:270
2666361993225,3cyclictest0-21swapper/207:20:012
2666361993217,3cyclictest171rcu_preempt12:25:272
2666361993217,3cyclictest171rcu_preempt10:30:272
2666361993216,4cyclictest171rcu_preempt09:00:252
2666361993216,4cyclictest171rcu_preempt09:00:252
2666360993226,4cyclictest2695760-21sshd08:45:261
2666360993223,6cyclictest2670891-21latency_hist07:25:021
2666359993222,7cyclictest2803967-21switchtime11:50:350
2666359993222,7cyclictest2803967-21switchtime11:50:350
266635999322,24cyclictest428-21mta-sts-daemon07:55:260
266635999322,24cyclictest428-21mta-sts-daemon07:55:260
2666359993221,8cyclictest2819435-21grep12:15:310
266635999321,27cyclictest12650irq/32-eth010:00:320
2666361993125,3cyclictest0-21swapper/207:20:152
2666361993123,4cyclictest0-21swapper/210:10:262
266636199312,23cyclictest2766082-21ssh10:50:252
2666361993112,4cyclictest171rcu_preempt07:50:282
2666361993112,4cyclictest171rcu_preempt07:50:282
266636099314,8cyclictest171rcu_preempt08:20:301
266636099314,22cyclictest2763080-21ssh10:45:301
2666360993125,3cyclictest0-21swapper/110:00:381
2666360993124,4cyclictest2667190-21ntp_kernel_err07:10:221
2666360993123,4cyclictest0-21swapper/109:40:381
2666359993127,2cyclictest0-21swapper/010:15:010
2666359993125,3cyclictest0-21swapper/009:55:280
266635999312,26cyclictest0-21swapper/011:00:290
2666359993120,7cyclictest2703472-21sh09:10:310
2666359993120,7cyclictest2703472-21sh09:10:310
266636199304,17cyclictest428-21mta-sts-daemon09:40:232
2666361993024,3cyclictest0-21swapper/210:15:372
2666361993023,5cyclictest1001-21mta-sts-daemon11:56:392
2666361993023,5cyclictest1001-21mta-sts-daemon11:56:392
2666361993022,4cyclictest0-21swapper/209:25:272
2666361993022,4cyclictest0-21swapper/209:25:272
2666361993020,7cyclictest2810317-21timerandwakeup12:00:402
2666361993019,3cyclictest171rcu_preempt08:05:292
2666361993016,3cyclictest171rcu_preempt12:20:422
266636099305,19cyclictest2680724-21sshd07:55:271
266636099305,19cyclictest2680724-21sshd07:55:271
2666360993024,3cyclictest0-21swapper/111:35:281
2666360993024,3cyclictest0-21swapper/111:35:281
2666360993023,4cyclictest2693320-21cron08:40:011
2666360993023,4cyclictest1001-21mta-sts-daemon07:15:271
266635999307,19cyclictest2763089-21ssh10:45:310
2666359993025,3cyclictest2718117-21sh09:35:000
2666359993025,3cyclictest2718117-21sh09:35:000
2666359993023,5cyclictest265-21systemd-journal10:25:010
2666359993016,12cyclictest2809334-21ssh12:00:170
266636199294,8cyclictest36-21ksoftirqd/208:00:302
2666361992922,5cyclictest2695910-21sendmail_mailst08:45:322
2666361992922,4cyclictest2692375-21gltestperf08:35:222
266636199292,20cyclictest2685257-21sshd08:10:272
2666361992921,4cyclictest351ktimers/208:30:282
2666361992914,4cyclictest171rcu_preempt07:35:282
2666361992914,3cyclictest171rcu_preempt12:35:272
2666361992914,3cyclictest171rcu_preempt12:35:272
2666361992913,4cyclictest171rcu_preempt07:45:302
266636199290,8cyclictest36-21ksoftirqd/208:55:262
2666360992921,4cyclictest0-21swapper/109:25:281
2666360992921,4cyclictest0-21swapper/109:25:281
2666360992921,3cyclictest0-21swapper/108:40:271
2666359992924,2cyclictest0-21swapper/009:00:010
2666359992922,5cyclictest421-21dbus-daemon07:25:010
2666359992922,5cyclictest2690853-21fschecks_count08:30:190
2666361992813,3cyclictest171rcu_preempt08:50:242
2666361992812,3cyclictest171rcu_preempt07:10:352
2666361992810,3cyclictest171rcu_preempt08:25:252
266636099283,19cyclictest2676263-21sshd07:40:261
2666360992821,4cyclictest2727341-21kworker/u8:0-xprtiod09:55:311
2666360992820,4cyclictest2685834-21munin-run08:15:011
2666360992820,4cyclictest0-21swapper/108:00:321
266635999287,18cyclictest0-21swapper/008:25:300
2666359992816,9cyclictest12650irq/32-eth007:40:250
2666359992816,9cyclictest0-21swapper/008:20:010
266635999281,22cyclictest2784939-21sshd11:20:280
2666360992720,3cyclictest0-21swapper/110:05:221
266635999272,10cyclictest420-21avahi-daemon08:10:320
2666359992718,7cyclictest2668740-21ntpq07:15:270
2666359992621,3cyclictest1000-21mta-sts-daemon07:53:510
2666359992621,3cyclictest1000-21mta-sts-daemon07:53:510
2666359992620,3cyclictest0-21swapper/008:45:010
2666359992616,7cyclictest0-21swapper/008:10:010
266636199251,5cyclictest421-21dbus-daemon07:10:022
2666360992520,2cyclictest0-21swapper/108:25:091
266635999255,15cyclictest2822563-21ssh12:20:460
2666359992520,2cyclictest0-21swapper/008:50:020
2666359992519,3cyclictest0-21swapper/007:35:170
2666359992518,5cyclictest0-21swapper/009:10:020
2666359992518,5cyclictest0-21swapper/009:10:020
2666359992319,2cyclictest0-21swapper/008:40:010
2666360992216,4cyclictest265-21systemd-journal07:10:021
2666360992216,3cyclictest0-21swapper/107:50:201
2666360992216,3cyclictest0-21swapper/107:50:201
266636299183,7cyclictest0-21swapper/311:55:273
266636299183,7cyclictest0-21swapper/311:55:273
266635999185,8cyclictest2666315-21munin-run07:10:010
266636299173,8cyclictest0-21swapper/312:35:263
266636299173,8cyclictest0-21swapper/312:35:263
266636299163,7cyclictest0-21swapper/311:20:273
266636299163,7cyclictest0-21swapper/311:15:243
266636299162,8cyclictest0-21swapper/310:55:243
266636299162,7cyclictest0-21swapper/311:45:263
266636299162,7cyclictest0-21swapper/311:45:263
266636299162,7cyclictest0-21swapper/310:50:243
266636299161,8cyclictest0-21swapper/312:05:253
266636299161,8cyclictest0-21swapper/311:00:243
266636299156,6cyclictest0-21swapper/310:45:283
266636299153,9cyclictest0-21swapper/311:50:273
266636299153,9cyclictest0-21swapper/311:50:273
266636299153,8cyclictest0-21swapper/309:15:263
266636299153,6cyclictest0-21swapper/307:30:273
266636299152,7cyclictest0-21swapper/310:15:273
266636299151,8cyclictest0-21swapper/311:25:253
266636299151,7cyclictest0-21swapper/312:25:273
266636299151,6cyclictest0-21swapper/309:45:243
266636299144,4cyclictest0-21swapper/307:50:263
266636299144,4cyclictest0-21swapper/307:50:263
266636299143,7cyclictest0-21swapper/310:40:303
266636299143,5cyclictest0-21swapper/308:55:303
266636299142,7cyclictest0-21swapper/309:40:253
266636299142,6cyclictest0-21swapper/311:05:203
266636299142,6cyclictest0-21swapper/308:45:263
266636299141,8cyclictest0-21swapper/310:20:293
266636299141,7cyclictest0-21swapper/312:00:273
266636299141,7cyclictest0-21swapper/311:40:253
266636299141,7cyclictest0-21swapper/311:40:253
266636299141,6cyclictest0-21swapper/309:25:223
266636299141,6cyclictest0-21swapper/309:25:223
266636299141,6cyclictest0-21swapper/308:50:273
266636299133,6cyclictest0-21swapper/309:20:243
266636299133,6cyclictest0-21swapper/309:20:243
266636299132,7cyclictest0-21swapper/311:30:273
266636299132,7cyclictest0-21swapper/309:50:293
266636299132,6cyclictest0-21swapper/310:30:213
266636299132,6cyclictest0-21swapper/310:10:183
266636299132,6cyclictest0-21swapper/310:05:183
266636299132,6cyclictest0-21swapper/309:35:243
266636299132,6cyclictest0-21swapper/309:35:243
266636299131,8cyclictest0-21swapper/312:30:283
266636299131,8cyclictest0-21swapper/310:25:273
266636299131,8cyclictest0-21swapper/309:55:293
266636299131,7cyclictest0-21swapper/312:20:223
266636299131,7cyclictest0-21swapper/311:35:283
266636299131,7cyclictest0-21swapper/311:35:283
266636299131,7cyclictest0-21swapper/309:10:273
266636299131,7cyclictest0-21swapper/309:10:273
266636299131,6cyclictest0-21swapper/311:10:223
266636299131,6cyclictest0-21swapper/308:40:263
266636299131,6cyclictest0-21swapper/308:30:293
266636299131,6cyclictest0-21swapper/308:20:283
266636299131,6cyclictest0-21swapper/308:15:303
266636299131,6cyclictest0-21swapper/308:10:243
266636299131,6cyclictest0-21swapper/307:40:243
266636299131,6cyclictest0-21swapper/307:25:283
266636299131,6cyclictest0-21swapper/307:15:243
266636299131,5cyclictest0-21swapper/307:55:263
266636299131,5cyclictest0-21swapper/307:55:263
266636299123,4cyclictest0-21swapper/308:00:323
266636299123,4cyclictest0-21swapper/307:35:253
266636299122,6cyclictest0-21swapper/312:15:203
266636299122,6cyclictest0-21swapper/308:40:013
266636299122,6cyclictest0-21swapper/307:20:253
266636299121,7cyclictest0-21swapper/310:35:243
266636299121,7cyclictest0-21swapper/309:30:253
266636299121,7cyclictest0-21swapper/309:30:253
266636299121,6cyclictest0-21swapper/312:10:203
266636299121,6cyclictest0-21swapper/307:15:013
266636299121,5cyclictest0-21swapper/307:45:253
266636299112,6cyclictest0-21swapper/310:00:223
266636299112,5cyclictest0-21swapper/308:30:013
266636299111,5cyclictest0-21swapper/309:00:183
266636299111,5cyclictest0-21swapper/309:00:183
266636299104,4cyclictest0-21swapper/309:10:013
266636299104,4cyclictest0-21swapper/309:10:013
26663629991,5cyclictest0-21swapper/307:10:033
26663629981,6cyclictest0-21swapper/308:05:283
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional