You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-03 - 00:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #2, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack2slot7s.osadl.org (updated Thu Apr 02, 2026 12:43:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2265193996338,22cyclictest2379631-21cron10:55:010
226519399533,42cyclictest2323859-21ssh09:45:360
226519399533,42cyclictest2323859-21ssh09:45:360
2265193995125,5cyclictest2266905-21head07:15:360
2265194995041,6cyclictest2319451-21ntp_kernel_err09:40:271
2265193994836,9cyclictest2453344-21aten_r2power_cu12:25:170
2265195994738,6cyclictest420-21avahi-daemon12:25:042
2265194994739,5cyclictest2367388-21cut10:40:001
2265194994635,7cyclictest439-21systemd-logind11:15:011
2265194994537,5cyclictest2346987-21munin-plugin-st10:15:011
2265194994532,9cyclictest2389224-21mailstats11:05:381
2265194994532,9cyclictest2389224-21mailstats11:05:381
2265195994437,4cyclictest2363301-21cron10:35:002
2265195994437,4cyclictest2363301-21cron10:35:002
2265194994433,8cyclictest421-21dbus-daemon07:10:011
2265194994336,4cyclictest2318388-21sendmail09:40:011
2265195994234,5cyclictest421-21dbus-daemon07:45:022
2265194994235,5cyclictest2265192-21cyclictest10:45:271
2265194994233,6cyclictest421-21dbus-daemon12:40:011
2265194994232,6cyclictest420-21avahi-daemon12:30:371
226519399422,31cyclictest2425622-21ssh11:50:310
2265195994134,3cyclictest0-21swapper/212:35:012
2265195994132,6cyclictest2393399-21switchtime11:10:392
2265195994130,8cyclictest2413768-21switchtime11:35:402
226519499414,27cyclictest2421639-21ssh11:45:351
2265194994133,5cyclictest2274321-21idleruntime-cro08:05:011
2265194994130,8cyclictest421-21dbus-daemon11:00:011
2265193994132,6cyclictest2444906-21latency_hist12:15:020
2265193994132,6cyclictest2444906-21latency_hist12:15:020
2265193994128,10cyclictest261-21systemd-journal10:30:010
2265194994033,4cyclictest2328370-21unixbench_multi09:50:431
2265194994030,6cyclictest2428710-21latency_hist11:55:021
226519499402,6cyclictest171rcu_preempt12:05:341
2265193994031,7cyclictest421-21dbus-daemon11:15:360
226519399402,31cyclictest2450033-21ssh12:20:310
2265195993932,4cyclictest0-21swapper/212:20:002
2265195993930,6cyclictest2266980-21munin-plugin-st07:20:002
2265195993929,7cyclictest1421253-21cups-browsed11:06:092
2265195993929,7cyclictest1421253-21cups-browsed11:06:092
2265195993929,6cyclictest420-21avahi-daemon10:14:332
2265194993928,8cyclictest2348775-21unixbench_singl10:15:421
2265194993920,14cyclictest421-21dbus-daemon11:55:331
2265194993919,13cyclictest2293798-21diskmemload11:00:341
2265193993932,4cyclictest2420427-21systemd-run11:45:020
2265193993928,7cyclictest2297892-21basename09:15:010
2265193993928,7cyclictest2297892-21basename09:15:010
226519399391,28cyclictest2372601-21ssh10:45:290
2265195993831,4cyclictest0-21swapper/211:50:342
2265195993830,5cyclictest439-21systemd-logind07:14:482
2265195993830,5cyclictest439-21systemd-logind07:14:482
2265195993829,6cyclictest431-21rsyslogd09:50:382
2265195993829,6cyclictest2302471-21basename09:20:182
2265195993829,6cyclictest2278844-21systemd-run08:20:012
2265195993828,7cyclictest2274025-21cron08:00:012
2265195993828,6cyclictest261-21systemd-journal10:45:002
2265194993831,3cyclictest0-21swapper/110:50:341
2265194993829,6cyclictest2448913-21latency_hist12:20:021
2265194993829,6cyclictest1421253-21cups-browsed10:44:421
2265194993828,7cyclictest2420389-21cron11:45:011
2265193993819,16cyclictest2384882-21ssh11:00:310
2265195993728,6cyclictest2383812-21latency_hist11:00:022
2265195993728,5cyclictest2275844-21latency_hist08:10:022
2265195993727,7cyclictest2284818-21sendmail08:40:002
2265195993726,8cyclictest2314649-21/usr/sbin/munin09:35:402
2265194993729,5cyclictest2274087-21latency_hist08:00:021
2265194993727,6cyclictest1421253-21cups-browsed09:25:241
2265194993721,13cyclictest2450129-21sh12:20:351
2265193993730,5cyclictest421-21dbus-daemon08:40:400
2265193993730,5cyclictest421-21dbus-daemon08:40:400
226519599363,23cyclictest2397196-21ssh11:15:332
2265195993629,4cyclictest2274324-21cut08:05:022
2265195993629,3cyclictest0-21swapper/210:05:332
2265195993627,6cyclictest2418098-21unixbench_multi11:40:442
226519599362,27cyclictest2307265-21ssh09:25:322
2265194993630,3cyclictest0-21swapper/107:55:011
2265194993628,5cyclictest2294519-21fschecks_time09:10:241
2265194993628,5cyclictest2294519-21fschecks_time09:10:241
2265194993627,6cyclictest261-21systemd-journal11:20:021
2265194993625,8cyclictest2360370-21ntp_kernel_pll_10:30:291
2265194993625,8cyclictest2360370-21ntp_kernel_pll_10:30:291
2265194993625,8cyclictest2351932-21kernelversion10:20:241
2265193993630,4cyclictest2351061-21munin-run10:20:010
2265193993630,2cyclictest0-21swapper/010:00:370
2265193993628,6cyclictest2289294-21/usr/sbin/munin08:55:010
2265193993626,7cyclictest2364475-21ssh10:35:300
226519599355,21cyclictest428-21mta-sts-daemon12:10:322
226519599355,21cyclictest428-21mta-sts-daemon12:10:322
2265195993528,4cyclictest2377322-21unixbench_singl10:50:422
2265195993528,3cyclictest0-21swapper/211:30:352
2265195993527,5cyclictest2320020-21timerwakeupswit09:40:392
2265195993526,5cyclictest2269520-21fschecks_count07:35:182
2265195993522,10cyclictest2311139-21missed_timers09:30:252
2265194993526,6cyclictest2444844-21true12:15:021
2265194993526,6cyclictest2444844-21true12:15:021
226519499351,29cyclictest2311442-21ssh09:30:341
226519399354,25cyclictest1421253-21cups-browsed08:15:330
2265193993529,3cyclictest0-21swapper/011:45:250
2265193993528,4cyclictest2408364-21latency_hist11:30:020
2265193993528,4cyclictest2302024-21kworker/u8:4-xprtiod09:55:210
2265193993527,5cyclictest2363299-21cron10:35:000
2265193993527,5cyclictest2363299-21cron10:35:000
2265193993524,8cyclictest2307516-21mailstats09:25:380
226519399352,28cyclictest2311276-21ssh09:30:280
226519399352,25cyclictest2462330-21ssh12:35:360
2265195993428,3cyclictest0-21swapper/209:10:212
2265195993428,3cyclictest0-21swapper/209:10:212
2265195993426,5cyclictest2338862-21systemd-run10:05:012
2265195993424,6cyclictest2462597-21threads12:35:412
2265195993423,8cyclictest2438067-21mailstats12:05:392
2265195993423,8cyclictest2363959-21cpuspeed10:35:202
2265194993428,3cyclictest2270549-21kworker/u8:3-nfsiod07:49:531
2265194993427,4cyclictest2412989-21gltestperf11:35:231
2265194993426,5cyclictest2454316-21mailstats12:25:361
2265194993425,6cyclictest439-21systemd-logind07:25:011
2265194993425,6cyclictest439-21systemd-logind07:25:011
2265194993425,6cyclictest421-21dbus-daemon10:10:011
2265194993425,5cyclictest2289366-21latency_hist08:55:011
2265193993427,4cyclictest2319957-21timerandwakeup09:40:390
2265193993425,5cyclictest2380019-21/usr/sbin/munin10:55:240
2265193993423,8cyclictest2272298-21systemd-run07:50:010
226519399342,26cyclictest2401181-21ssh11:20:290
2265193993413,14cyclictest12350irq/32-eth011:10:290
2265195993326,4cyclictest2421769-21sendmail_mailst11:45:382
2265195993325,5cyclictest2355156-21cron10:25:012
2265195993325,5cyclictest2332245-21timerandwakeup09:55:392
2265195993324,5cyclictest2290879-21latency_hist09:00:022
2265195993322,7cyclictest2293798-21diskmemload11:55:322
2265194993327,3cyclictest0-21swapper/107:40:011
2265194993325,5cyclictest2356184-21needreboot10:25:261
2265194993325,3cyclictest0-21swapper/109:45:321
2265194993325,3cyclictest0-21swapper/109:45:321
2265193993323,5cyclictest2286828-21fschecks_count08:45:220
2265193993320,10cyclictest0-21swapper/012:05:020
2265193993319,12cyclictest2299645-21unixbench_singl09:15:400
2265193993317,4cyclictest0-21swapper/011:55:370
226519399331,28cyclictest2344041-21ssh10:10:300
226519399331,25cyclictest2388970-21ssh11:05:310
226519399331,25cyclictest2388970-21ssh11:05:310
2265195993226,3cyclictest0-21swapper/211:25:012
2265195993225,4cyclictest1055-21mta-sts-daemon10:49:152
2265195993224,5cyclictest421-21dbus-daemon11:30:022
2265195993224,5cyclictest2450185-21proc_pri12:20:362
2265195993224,4cyclictest2287817-21kworker/u8:14-xprtiod09:10:012
2265195993223,6cyclictest420-21avahi-daemon09:47:132
2265195993223,6cyclictest420-21avahi-daemon09:47:132
226519599322,24cyclictest2433879-21ssh12:00:352
2265194993223,4cyclictest0-21swapper/107:25:291
2265193993225,5cyclictest1421253-21cups-browsed11:33:200
2265193993222,3cyclictest2448840-21cron12:20:000
226519599313,23cyclictest2348161-21ssh10:15:322
2265195993125,3cyclictest1722505-21kworker/u8:5-xprtiod09:15:312
2265195993123,5cyclictest1-21systemd07:25:012
2265195993123,5cyclictest1-21systemd07:25:012
2265195993123,4cyclictest2283489-21latency_hist08:35:022
2265194993123,5cyclictest2302016-21systemd-run09:20:011
2265194993123,5cyclictest2285042-21/usr/sbin/munin08:40:281
2265194993123,5cyclictest2285042-21/usr/sbin/munin08:40:281
2265194993121,7cyclictest2265483-21cut07:15:011
2265194993121,7cyclictest2265483-21cut07:15:011
2265194993120,8cyclictest2281212-21ntp_kernel_pll_08:25:301
2265193993124,4cyclictest0-21swapper/012:05:220
2265193993123,5cyclictest2339935-21sh10:05:290
2265193993122,6cyclictest1421253-21cups-browsed09:51:150
2265193993120,8cyclictest2413631-21mailstats11:35:370
2265194993023,5cyclictest2279907-21mailstats08:20:351
2265194993022,5cyclictest2292329-21systemd-run09:05:021
2265194993021,6cyclictest2434370-21unixbench_singl12:00:411
2265194993021,4cyclictest0-21swapper/108:55:301
2265194993020,7cyclictest2409821-21timerwakeupswit11:30:411
2265193993024,4cyclictest2302766-21ssh09:20:220
2265193993024,3cyclictest0-21swapper/010:20:410
2265193993022,6cyclictest2269526-21fschecks_count07:35:180
2265193993020,4cyclictest2274923-21if_wlan008:05:190
2265193993010,17cyclictest2458158-21ssh12:30:320
2265195992919,7cyclictest2384707-21missed_timers11:00:262
226519499293,19cyclictest2401128-21ssh11:20:271
2265194992923,3cyclictest0-21swapper/107:20:001
2265194992922,3cyclictest0-21swapper/109:55:331
2265194992919,7cyclictest2408212-21systemd-run11:30:011
2265193992922,5cyclictest2283323-21systemd-run08:35:010
2265193992922,4cyclictest12450irq/33-eth009:35:310
2265193992921,6cyclictest2280985-21latency08:25:240
2265193992921,3cyclictest0-21swapper/010:40:290
2265193992920,7cyclictest2270806-21systemd-run07:45:020
2265195992823,2cyclictest0-21swapper/208:45:282
2265195992820,5cyclictest2277186-21unixbench_multi08:10:412
2265195992820,5cyclictest2265267-21latency_hist07:10:022
2265195992817,8cyclictest2291675-21ntp_kernel_pll_09:00:292
2265194992823,2cyclictest0-21swapper/110:05:021
2265193992821,5cyclictest2277316-21systemd-run08:15:010
2265193992820,6cyclictest0-21swapper/008:05:010
2265195992719,5cyclictest2288132-21aten_r2power_fr08:50:192
2265195992712,11cyclictest1421253-21cups-browsed10:28:112
2265194992721,3cyclictest2277314-21kworker/u8:7-xprtiod08:17:011
2265194992721,3cyclictest0-21swapper/108:15:011
2265194992718,7cyclictest2304907-21sh09:22:471
2265193992721,4cyclictest1055-21mta-sts-daemon09:03:200
2265195992618,5cyclictest439-21systemd-logind08:45:012
2265195992618,5cyclictest439-21systemd-logind08:45:012
2265194992616,7cyclictest2293791-21cron09:10:011
226519399262,18cyclictest659-21snmpd08:55:290
2265195992519,3cyclictest2272300-21kworker/u8:8-xprtiod07:55:012
2265195992519,3cyclictest2270549-21kworker/u8:3-nfsiod07:49:522
2265195992519,3cyclictest0-21swapper/207:35:012
2265195992517,5cyclictest548-21gdbus08:23:042
2265194992517,5cyclictest2269033-21systemd-run07:35:021
226519499251,10cyclictest62450lldpd08:10:021
2265193992519,4cyclictest2274025-21cron08:00:010
2265193992519,4cyclictest1521528-21munin-node08:35:200
2265193992518,5cyclictest1421253-21cups-browsed08:21:250
2265193992517,6cyclictest2269012-21/usr/sbin/munin07:35:020
2265193992412,9cyclictest0-21swapper/007:30:000
2265195992316,4cyclictest2268690-21unixbench_singl07:25:362
226519499232,15cyclictest2284198-21ntpq08:35:301
2265194992317,4cyclictest0-21swapper/107:45:011
2265193992317,4cyclictest2293819-21ssh09:10:020
2265195992214,5cyclictest420-21avahi-daemon08:25:272
226519499222,15cyclictest2282748-21ntpq08:30:321
226519499212,14cyclictest2287224-21ntpq08:45:311
2265193992115,3cyclictest0-21swapper/007:14:360
2265193992115,3cyclictest0-21swapper/007:14:360
2265193991914,3cyclictest1722505-21kworker/u8:5-xprtiod07:25:000
2265193991914,3cyclictest1722505-21kworker/u8:5-xprtiod07:25:000
2265193991914,3cyclictest1054-21mta-sts-daemon07:54:280
2265193991811,2cyclictest0-21swapper/007:10:020
226519699172,9cyclictest0-21swapper/311:55:333
226519699172,8cyclictest0-21swapper/310:35:323
226519699163,7cyclictest0-21swapper/310:20:203
226519699162,9cyclictest0-21swapper/312:15:263
226519699162,8cyclictest0-21swapper/309:55:283
226519699162,7cyclictest0-21swapper/309:30:303
226519699161,7cyclictest0-21swapper/307:45:303
226519699154,5cyclictest0-21swapper/311:30:023
226519699154,5cyclictest0-21swapper/308:40:293
226519699154,5cyclictest0-21swapper/308:40:293
226519699153,8cyclictest0-21swapper/309:15:303
226519699153,6cyclictest0-21swapper/309:10:013
226519699153,4cyclictest0-21swapper/308:55:273
226519699152,8cyclictest0-21swapper/312:35:333
226519699152,8cyclictest0-21swapper/311:50:293
226519699152,7cyclictest0-21swapper/310:00:313
226519699152,6cyclictest0-21swapper/308:20:273
226519699151,6cyclictest0-21swapper/310:40:173
226519699151,10cyclictest0-21swapper/311:00:233
226519699144,6cyclictest0-21swapper/312:05:213
226519699143,6cyclictest0-21swapper/309:40:193
226519699142,7cyclictest0-21swapper/311:40:313
226519699142,7cyclictest0-21swapper/311:20:243
226519699142,7cyclictest0-21swapper/309:10:343
226519699142,7cyclictest0-21swapper/309:10:343
226519699142,6cyclictest0-21swapper/310:10:223
226519699141,7cyclictest0-21swapper/312:30:313
226519699141,7cyclictest0-21swapper/310:15:303
226519699141,7cyclictest0-21swapper/309:20:183
226519699141,6cyclictest0-21swapper/308:35:273
226519699141,6cyclictest0-21swapper/308:25:333
226519699141,6cyclictest0-21swapper/307:35:243
226519699141,6cyclictest0-21swapper/307:25:303
226519699131,8cyclictest0-21swapper/312:20:183
226519699131,8cyclictest0-21swapper/310:55:253
226519699131,7cyclictest0-21swapper/311:45:293
226519699131,7cyclictest0-21swapper/310:30:323
226519699131,7cyclictest0-21swapper/310:30:323
226519699131,7cyclictest0-21swapper/310:05:203
226519699131,7cyclictest0-21swapper/309:50:083
226519699131,7cyclictest0-21swapper/309:45:283
226519699131,7cyclictest0-21swapper/309:45:283
226519699130,7cyclictest0-21swapper/309:25:303
226519699124,6cyclictest0-21swapper/310:25:253
226519699124,5cyclictest431ktimers/307:10:003
226519699122,6cyclictest0-21swapper/312:10:183
226519699122,6cyclictest0-21swapper/312:10:183
226519699122,6cyclictest0-21swapper/312:00:193
226519699122,6cyclictest0-21swapper/311:05:203
226519699122,6cyclictest0-21swapper/311:05:203
226519699122,5cyclictest0-21swapper/309:00:343
226519699122,5cyclictest0-21swapper/307:15:243
226519699121,7cyclictest0-21swapper/312:25:313
226519699121,7cyclictest0-21swapper/311:35:283
226519699121,7cyclictest0-21swapper/311:30:243
226519699121,7cyclictest0-21swapper/311:15:023
226519699121,7cyclictest0-21swapper/310:50:213
226519699121,7cyclictest0-21swapper/309:35:183
226519699121,6cyclictest0-21swapper/310:50:023
226519699121,6cyclictest0-21swapper/308:45:423
226519699121,5cyclictest0-21swapper/307:55:253
226519699121,5cyclictest0-21swapper/307:55:013
226519699112,6cyclictest0-21swapper/311:20:023
226519699112,5cyclictest0-21swapper/307:35:013
226519699111,6cyclictest0-21swapper/308:05:233
226519699111,5cyclictest0-21swapper/307:25:003
226519699111,5cyclictest0-21swapper/307:25:003
226519699101,6cyclictest0-21swapper/308:05:013
226519699101,5cyclictest0-21swapper/308:30:223
226519699101,5cyclictest0-21swapper/308:10:223
22651969993,5cyclictest0-21swapper/308:50:293
22651969991,5cyclictest0-21swapper/307:45:013
22651969991,5cyclictest0-21swapper/307:14:363
22651969991,5cyclictest0-21swapper/307:14:363
22651969984,3cyclictest0-21swapper/308:15:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional