You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-23 - 16:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack2slot7s.osadl.org (updated Tue Jun 23, 2026 12:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2860987995645,7cyclictest3014494-21sendmail12:20:011
2860988995443,7cyclictest439-21systemd-logind09:45:012
2860987995345,5cyclictest0-21swapper/111:00:421
286098799534,37cyclictest2922965-21ssh09:50:351
2860988995139,8cyclictest420-21avahi-daemon10:13:472
286098899509,20cyclictest171rcu_preempt10:20:362
2860987995033,13cyclictest2980881-21/usr/sbin/munin11:25:451
286098799493,37cyclictest2898030-21ssh09:10:311
286098799493,37cyclictest2898030-21ssh09:10:311
286098699482,37cyclictest2963120-21ssh10:55:340
2860988994737,5cyclictest0-21swapper/210:35:372
2860986994717,23cyclictest2981685-21ssh11:25:350
2860986994536,6cyclictest421-21dbus-daemon12:25:010
2860986994525,16cyclictest420-21avahi-daemon10:49:560
286098899443,33cyclictest3003160-21ssh12:00:292
2860987994437,4cyclictest0-21swapper/112:35:011
2860987994436,5cyclictest2903187-21munin-run09:20:011
286098799443,32cyclictest3009477-21ssh12:10:371
286098899434,30cyclictest2913687-21ssh09:35:352
2860988994333,6cyclictest2971325-21/usr/sbin/munin11:10:022
2860988994333,4cyclictest0-21swapper/210:35:022
2860987994333,6cyclictest2060917-21cups-browsed10:11:131
2860987994332,7cyclictest2060917-21cups-browsed12:29:251
2860986994334,6cyclictest2060917-21cups-browsed09:35:250
2860986994323,13cyclictest12650irq/32-eth011:15:310
2860988994231,6cyclictest421-21dbus-daemon12:08:132
286098799425,32cyclictest2978511-21sh11:20:291
2860987994234,5cyclictest2874465-21munin-plugin-st07:55:001
2860986994231,8cyclictest2974410-21systemd-run11:15:020
2860988994133,5cyclictest2992886-21cron11:45:002
2860988994133,5cyclictest2992886-21cron11:45:002
286098899413,26cyclictest2060914-21cupsd10:00:352
2860988994132,5cyclictest2989820-21idleruntime-cro11:40:012
2860986994133,5cyclictest2906504-21latency_hist09:25:020
2860988994033,3cyclictest0-21swapper/208:00:012
2860988994031,6cyclictest2925553-21fschecks_count09:55:222
2860987994030,7cyclictest2962084-21latency_hist10:55:021
2860986994029,6cyclictest12650irq/32-eth009:10:290
2860986994029,6cyclictest12650irq/32-eth009:10:290
2860988993933,3cyclictest0-21swapper/212:00:002
2860988993929,6cyclictest3014475-21run-parts12:20:012
2860988993928,7cyclictest2954420-21vmstat10:40:462
2860988993928,7cyclictest2060917-21cups-browsed09:33:252
2860987993931,5cyclictest2904798-21unixbench_multi09:20:421
2860987993931,5cyclictest2893994-21systemd-run09:00:011
2860987993929,6cyclictest2938077-21kernelversion10:15:271
2860986993933,4cyclictest2896958-21cron09:10:000
2860986993933,4cyclictest2896958-21cron09:10:000
286098699393,31cyclictest2922999-21rm09:50:370
2860986993930,5cyclictest151ktimers/009:00:320
2860986993930,5cyclictest151ktimers/009:00:320
2860986993926,7cyclictest12650irq/32-eth007:15:330
2860987993831,4cyclictest2890885-21unixbench_singl08:45:431
2860987993831,3cyclictest0-21swapper/111:55:181
2860986993830,4cyclictest151ktimers/007:55:310
2860986993829,6cyclictest12650irq/32-eth011:35:300
2860986993814,20cyclictest3024857-21ssh12:35:340
2860988993727,4cyclictest0-21swapper/208:05:322
2860988993717,16cyclictest351ktimers/208:15:312
2860987993730,4cyclictest2977461-21grep11:20:011
2860987993730,4cyclictest1000-21mta-sts-daemon10:41:411
2860987993729,5cyclictest2912533-21munin-run09:35:011
286098699373,29cyclictest2919962-21diskmemload09:45:370
286098699373,28cyclictest2944476-21ssh10:25:290
2860986993730,4cyclictest2925382-21aten_r2power_po09:55:200
2860986993727,5cyclictest1000-21mta-sts-daemon10:35:300
2860986993725,8cyclictest2968195-21systemd-run11:05:020
286098899363,24cyclictest2898052-21ssh09:10:322
286098899363,24cyclictest2898052-21ssh09:10:322
2860988993628,5cyclictest2859599-21kworker/u8:10+xprtiod07:20:012
2860988993627,5cyclictest3023734-21munin-plugin-st12:35:012
2860988993627,5cyclictest2962632-21fschecks_time10:55:232
2860988993622,10cyclictest2923343-21timerwakeupswit09:50:422
2860987993629,4cyclictest2861047-21latency_hist07:10:021
2860987993628,5cyclictest2926122-21proc_pri09:55:361
2860987993628,5cyclictest2887958-21grep08:40:011
2860987993627,6cyclictest1-21systemd08:00:021
2860987993627,4cyclictest1003-21mta-sts-daemon09:29:331
2860987993627,4cyclictest0-21swapper/112:05:281
2860987993625,8cyclictest2947449-21missed_timers10:30:261
286098799361,29cyclictest2924952-21kworker/u8:0+rpciod11:05:281
2860986993624,9cyclictest2865474-21cron07:25:010
2860986993623,10cyclictest0-21swapper/010:05:020
286098699362,25cyclictest3018649-21ssh12:25:320
2860988993526,5cyclictest2994057-21ssh11:45:372
2860988993526,5cyclictest2994057-21ssh11:45:372
2860988993525,5cyclictest0-21swapper/209:15:362
286098899352,18cyclictest2984706-21ssh11:30:322
2860986993528,4cyclictest1002-21mta-sts-daemon10:15:250
2860986993527,3cyclictest0-21swapper/011:40:280
2860986993527,3cyclictest0-21swapper/011:40:280
2860986993522,10cyclictest0-21swapper/010:45:010
2860988993427,3cyclictest0-21swapper/211:00:282
2860988993427,3cyclictest0-21swapper/209:45:312
2860988993425,6cyclictest2934246-21latency_hist10:10:022
2860988993425,5cyclictest0-21swapper/211:13:242
286098899342,24cyclictest2907451-21ssh09:25:352
286098899342,23cyclictest2060917-21cups-browsed07:10:302
286098799343,25cyclictest2997049-21ssh11:50:311
286098799343,25cyclictest2997049-21ssh11:50:311
2860987993427,5cyclictest1000-21mta-sts-daemon12:20:281
2860987993426,5cyclictest2896966-21diskmemload11:40:411
2860987993426,5cyclictest2896966-21diskmemload11:40:411
2860987993426,5cyclictest2872971-21idleruntime-cro07:50:001
2860987993426,5cyclictest0-21swapper/112:05:011
2860987993426,4cyclictest0-21swapper/110:00:221
2860987993410,9cyclictest28-21ksoftirqd/111:30:301
2860986993425,7cyclictest3005248-21munin-run12:05:010
2860986993425,7cyclictest2961992-21/usr/sbin/munin10:55:010
2860986993422,10cyclictest265-21systemd-journal08:35:010
2860988993327,3cyclictest0-21swapper/211:30:012
2860988993327,3cyclictest0-21swapper/211:15:282
2860988993327,3cyclictest0-21swapper/208:05:012
2860988993325,4cyclictest0-21swapper/209:10:002
2860988993325,4cyclictest0-21swapper/209:10:002
2860988993324,5cyclictest3008956-21fschecks_time12:10:242
2860988993324,5cyclictest2880484-21systemd-run08:15:022
2860988993323,7cyclictest3025385-21sed12:35:422
2860987993328,2cyclictest0-21swapper/108:05:021
2860987993326,3cyclictest0-21swapper/111:00:011
2860987993325,5cyclictest2974603-21latency_hist11:15:031
2860987993323,7cyclictest271ktimers/110:20:341
2860987993322,8cyclictest2918769-21sh09:45:011
2860986993327,4cyclictest2869955-21cron07:40:000
2860986993327,4cyclictest1001-21mta-sts-daemon08:15:340
2860986993327,3cyclictest0-21swapper/007:50:010
2860986993326,4cyclictest2969042-21missed_timers11:05:240
286098899323,22cyclictest3015615-21ssh12:20:352
2860988993225,3cyclictest0-21swapper/211:50:302
2860988993225,3cyclictest0-21swapper/211:50:302
2860988993223,5cyclictest0-21swapper/210:50:022
2860988993212,5cyclictest62750lldpd07:30:272
2860987993224,5cyclictest2913879-21switchtime09:35:401
2860987993224,5cyclictest2895924-21cpuspeed09:05:181
2860987993224,5cyclictest2895924-21cpuspeed09:05:181
2860987993224,5cyclictest2878983-21systemd-run08:10:011
2860987993224,5cyclictest1003-21mta-sts-daemon10:06:461
2860987993223,4cyclictest0-21swapper/111:50:021
2860987993223,4cyclictest0-21swapper/111:50:021
2860987993218,7cyclictest171rcu_preempt09:00:261
2860987993218,7cyclictest171rcu_preempt09:00:261
2860986993226,4cyclictest2949666-21ssh10:35:010
2860986993225,4cyclictest2880352-21unixbench_multi08:10:450
2860986993224,3cyclictest12750irq/33-eth007:50:320
286098699322,23cyclictest2984779-21sh11:30:360
2860986993221,8cyclictest2883473-21sh08:25:010
2860986993218,10cyclictest420-21avahi-daemon10:10:360
2860986993217,12cyclictest0-21swapper/012:15:010
2860988993125,2cyclictest0-21swapper/209:25:012
2860988993123,4cyclictest0-21swapper/210:15:342
2860987993123,5cyclictest3024366-21sort12:35:231
2860987993123,5cyclictest0-21swapper/108:20:011
2860987993123,3cyclictest0-21swapper/108:40:361
2860987993113,3cyclictest171rcu_preempt07:40:311
2860987993113,3cyclictest171rcu_preempt07:40:311
2860986993125,3cyclictest0-21swapper/010:05:360
2860986993124,4cyclictest2978097-21rm11:20:220
2860988993024,3cyclictest0-21swapper/209:00:012
2860988993023,4cyclictest2962004-21systemd-run10:55:012
286098799303,8cyclictest28-21ksoftirqd/110:35:301
2860987993023,4cyclictest2868388-21unixbench_singl07:30:411
2860987993022,4cyclictest0-21swapper/110:25:311
2860987993017,2cyclictest171rcu_preempt07:10:361
2860987993013,3cyclictest171rcu_preempt09:45:301
2860986993024,3cyclictest2958922-21kworker/u8:5-nfsiod12:10:010
2860986993021,7cyclictest3014466-21sendmail12:20:010
2860986993019,8cyclictest2892473-21sh08:55:010
2860986993010,15cyclictest12650irq/32-eth009:25:310
286098899293,8cyclictest171rcu_preempt08:30:332
2860988992922,4cyclictest2867006-21grep07:30:022
2860988992919,5cyclictest2895464-21cron09:05:002
2860988992919,5cyclictest2895464-21cron09:05:002
286098899291,22cyclictest3018711-21ssh12:25:352
286098799299,14cyclictest1002-21mta-sts-daemon07:20:321
2860987992921,5cyclictest271ktimers/107:35:281
2860987992920,5cyclictest271ktimers/108:50:271
2860987992915,3cyclictest171rcu_preempt11:35:331
2860986992922,4cyclictest2877998-21fschecks_time08:05:230
2860986992921,5cyclictest2910053-21fschecks_time09:30:230
2860986992921,5cyclictest1002-21mta-sts-daemon11:49:540
2860986992921,5cyclictest1002-21mta-sts-daemon11:49:540
2860988992821,4cyclictest2884689-21timerandwakeup08:25:422
2860988992821,3cyclictest0-21swapper/210:25:292
2860988992821,3cyclictest0-21swapper/207:50:302
2860988992820,5cyclictest421-21dbus-daemon08:45:022
2860988992817,8cyclictest2890336-21ntp_kernel_pll_08:45:312
2860987992820,5cyclictest2862976-21fschecks_count07:15:231
2860987992817,3cyclictest171rcu_preempt08:30:321
2860987992813,4cyclictest171rcu_preempt10:46:521
286098699282,23cyclictest2941487-21ssh10:20:330
2860986992821,4cyclictest2999175-21latency_hist11:55:020
2860986992821,4cyclictest2999175-21latency_hist11:55:020
2860986992819,7cyclictest1003-21mta-sts-daemon09:17:480
286098699281,23cyclictest3021662-21ssh12:30:300
2860988992720,5cyclictest2872785-21timerwakeupswit07:45:412
2860988992720,3cyclictest0-21swapper/208:50:422
2860988992713,10cyclictest421-21dbus-daemon07:20:352
286098799278,5cyclictest28-21ksoftirqd/108:25:331
286098799275,9cyclictest271ktimers/108:25:011
2860987992718,3cyclictest171rcu_preempt08:10:281
2860987992713,10cyclictest1001-21mta-sts-daemon07:25:311
2860986992720,4cyclictest3000295-21sendmail_mailqu11:55:350
2860986992714,8cyclictest12650irq/32-eth009:40:310
286098899262,19cyclictest2978492-21ssh11:20:282
2860986992620,4cyclictest2886990-21fschecks_time08:35:260
2860986992620,4cyclictest2865994-21fschecks_count07:25:230
2860988992518,5cyclictest0-21swapper/208:40:012
2860986992521,2cyclictest0-21swapper/007:10:420
2860986992520,3cyclictest2877491-21kworker/u8:13-xprtiod08:45:230
2860986992517,5cyclictest567-21gdbus07:34:530
2860986992514,9cyclictest151ktimers/007:40:310
2860986992514,9cyclictest151ktimers/007:40:310
2860986992514,8cyclictest2060917-21cups-browsed08:55:280
286098899248,11cyclictest2060914-21cupsd08:20:362
2860988992417,4cyclictest0-21swapper/207:40:002
286098699245,12cyclictest428-21mta-sts-daemon08:25:320
286098699231,5cyclictest421-21dbus-daemon08:00:350
2860988992216,3cyclictest0-21swapper/207:40:252
2860988992216,3cyclictest0-21swapper/207:40:252
2860988992214,5cyclictest439-21systemd-logind07:10:012
286098699211,17cyclictest2888851-21ntpq08:40:300
286098699202,14cyclictest2860942-21munin-run07:10:010
286098999194,8cyclictest0-21swapper/312:15:323
286098999183,7cyclictest0-21swapper/311:30:293
286098999182,9cyclictest0-21swapper/310:25:333
286098999174,8cyclictest0-21swapper/309:30:293
286098999173,8cyclictest0-21swapper/311:10:323
286098999173,8cyclictest0-21swapper/309:50:373
286098999172,9cyclictest0-21swapper/311:50:303
286098999172,9cyclictest0-21swapper/311:50:303
286098999172,9cyclictest0-21swapper/310:35:373
286098999171,9cyclictest0-21swapper/310:40:313
286098999171,8cyclictest0-21swapper/308:10:383
286098999170,8cyclictest0-21swapper/309:55:333
286098999164,6cyclictest0-21swapper/310:10:373
286098999163,7cyclictest0-21swapper/310:15:313
286098999162,8cyclictest0-21swapper/311:05:283
286098999162,7cyclictest0-21swapper/311:00:293
286098999162,7cyclictest0-21swapper/309:10:313
286098999162,7cyclictest0-21swapper/309:10:313
286098999161,9cyclictest0-21swapper/309:25:323
286098999155,6cyclictest0-21swapper/308:15:293
286098999153,7cyclictest0-21swapper/311:55:323
286098999153,7cyclictest0-21swapper/311:20:303
286098999152,8cyclictest0-21swapper/311:25:343
286098999152,8cyclictest0-21swapper/310:50:453
286098999152,8cyclictest0-21swapper/309:20:283
286098999152,6cyclictest0-21swapper/307:15:303
286098999151,8cyclictest0-21swapper/312:20:343
286098999151,8cyclictest0-21swapper/311:15:323
286098999151,8cyclictest0-21swapper/309:40:363
286098999151,7cyclictest0-21swapper/312:35:303
286098999151,7cyclictest0-21swapper/309:00:323
286098999151,7cyclictest0-21swapper/309:00:323
286098999151,7cyclictest0-21swapper/308:35:373
286098999151,6cyclictest0-21swapper/307:30:303
286098999143,7cyclictest0-21swapper/311:40:283
286098999143,7cyclictest0-21swapper/311:40:283
286098999143,7cyclictest0-21swapper/310:20:373
286098999143,5cyclictest0-21swapper/309:05:283
286098999143,5cyclictest0-21swapper/309:05:283
286098999142,7cyclictest0-21swapper/312:30:303
286098999142,7cyclictest0-21swapper/311:35:283
286098999142,7cyclictest0-21swapper/310:05:303
286098999141,8cyclictest0-21swapper/310:58:463
286098999141,7cyclictest0-21swapper/308:25:333
286098999141,6cyclictest0-21swapper/308:45:293
286098999141,6cyclictest0-21swapper/308:30:273
286098999141,6cyclictest0-21swapper/307:40:283
286098999141,6cyclictest0-21swapper/307:40:283
286098999141,6cyclictest0-21swapper/307:20:323
286098999134,6cyclictest0-21swapper/310:00:363
286098999134,6cyclictest0-21swapper/309:45:373
286098999134,5cyclictest0-21swapper/307:55:363
286098999133,7cyclictest0-21swapper/312:25:243
286098999132,8cyclictest0-21swapper/312:10:303
286098999132,7cyclictest0-21swapper/308:00:353
286098999132,7cyclictest0-21swapper/307:10:303
286098999132,6cyclictest0-21swapper/312:05:223
286098999132,5cyclictest0-21swapper/307:50:323
286098999131,8cyclictest0-21swapper/309:35:193
286098999122,7cyclictest0-21swapper/309:15:303
286098999122,7cyclictest0-21swapper/308:10:023
286098999122,7cyclictest0-21swapper/307:25:313
286098999122,6cyclictest0-21swapper/311:45:233
286098999122,6cyclictest0-21swapper/311:45:233
286098999122,6cyclictest0-21swapper/310:30:253
286098999122,6cyclictest0-21swapper/308:40:303
286098999122,5cyclictest0-21swapper/308:55:303
286098999121,7cyclictest0-21swapper/308:50:293
286098999121,6cyclictest0-21swapper/312:00:213
286098999113,6cyclictest0-21swapper/307:10:023
286098999112,7cyclictest0-21swapper/310:45:203
286098999112,6cyclictest0-21swapper/308:20:223
286098999111,7cyclictest0-21swapper/307:45:233
286098999101,6cyclictest0-21swapper/307:35:293
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional