You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-16 - 01:07
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8.osadl.org (updated Wed Oct 15, 2025 12:43:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25927282920,1sleep20-21swapper/210:50:252
25639972300,2sleep02564003-21irqrtprio10:35:170
2273834992522,2cyclictest0-21swapper/211:40:002
2273834992521,3cyclictest761-21cups-browsed11:55:002
2273831992421,2cyclictest0-21swapper/109:20:001
2273829992421,2cyclictest547-21gdbus12:20:000
2273835992320,1cyclictest0-21swapper/308:00:003
2273835992319,1cyclictest0-21swapper/308:30:013
2273829992320,2cyclictest2506191-21idleruntime-cro10:00:000
2273835992218,1cyclictest0-21swapper/308:10:003
2273829992219,2cyclictest2288855-21gltestperf07:20:170
2273831992118,2cyclictest2563541-21irqcore10:35:161
2273831992118,2cyclictest2473682-21latency_hist09:40:001
2273831992118,2cyclictest0-21swapper/108:20:011
2273829992118,2cyclictest1-21systemd09:35:000
2273835992017,1cyclictest0-21swapper/310:55:183
2273835992017,1cyclictest0-21swapper/308:20:013
2273834992017,2cyclictest2387949-21latency_hist08:40:012
2273834992017,2cyclictest2338173-21latency_hist08:00:012
2273831992017,2cyclictest2747849-21kworker/u8:2-ext4-rsv-conversion12:35:001
2273831992017,2cyclictest0-21swapper/111:20:001
227383599198,6cyclictest23249202sleep307:45:223
2273835991915,1cyclictest0-21swapper/310:40:223
2273834991916,2cyclictest2764978-21latency_hist12:40:012
2273834991915,3cyclictest2566039-21kthreadcore10:35:182
2273834991915,2cyclictest0-21swapper/209:15:002
2273831991917,1cyclictest2412711-21idleruntime-cro09:00:011
2273831991917,1cyclictest0-21swapper/112:00:001
2273831991916,2cyclictest2748908-21latency_hist12:30:011
2273831991916,2cyclictest2600652-21threads10:55:251
2273831991916,2cyclictest2288203-21sendmail07:20:001
2273829991910,8cyclictest0-21swapper/010:45:180
2273835991815,1cyclictest0-21swapper/309:10:003
2273835991814,1cyclictest0-21swapper/309:45:163
2273834991815,2cyclictest2301522-21irqcore07:30:192
2273834991814,3cyclictest522-21in:imuxsock07:40:002
2273834991814,3cyclictest2399697-21unixbench-2d08:45:262
2273834991813,4cyclictest2657434-21unixbench_singl11:30:252
2273834991813,4cyclictest2657434-21unixbench_singl11:30:252
2273834991810,1cyclictest191rcu_preempt08:10:002
2273831991816,1cyclictest0-21swapper/107:55:001
2273831991815,2cyclictest2465300-21cron09:35:001
2273831991815,2cyclictest2418961-21latency_hist09:05:001
227382999186,11cyclictest2676312-21apt-get11:45:120
2273829991816,1cyclictest2643806-21tr11:25:130
2273829991815,2cyclictest2473682-21latency_hist09:40:010
2273835991714,1cyclictest0-21swapper/310:50:003
2273835991713,1cyclictest0-21swapper/310:36:273
2273835991713,1cyclictest0-21swapper/310:17:123
2273835991713,1cyclictest0-21swapper/308:35:013
2273834991714,2cyclictest2619968-21irqcore11:10:172
2273834991714,2cyclictest2530615-21latency_hist10:15:012
2273834991714,2cyclictest2439010-21unixbench_multi09:15:252
2273834991714,2cyclictest2368699-21unixbench-2d08:20:252
2273834991714,2cyclictest2295241-21gltestperf07:25:162
2273834991713,3cyclictest2499584-21irqrtprio09:55:162
2273834991713,3cyclictest2495931-21unixbench-2d09:50:252
2273829991714,2cyclictest2761616-21kthreadcore12:35:200
2273829991714,2cyclictest2388541-21gltestperf08:40:150
2273829991713,3cyclictest2324646-21ntp_kernel_err07:45:180
26157302161,2sleep02616311-21wc11:05:190
2273835991614,1cyclictest0-21swapper/312:10:173
2273835991613,1cyclictest0-21swapper/312:00:203
2273835991613,1cyclictest0-21swapper/311:50:163
2273835991613,1cyclictest0-21swapper/311:15:003
2273835991613,1cyclictest0-21swapper/310:20:173
2273835991613,1cyclictest0-21swapper/310:10:173
2273835991613,1cyclictest0-21swapper/309:50:233
2273835991613,1cyclictest0-21swapper/309:40:013
2273835991613,1cyclictest0-21swapper/309:30:193
2273835991613,1cyclictest0-21swapper/307:20:003
2273835991611,2cyclictest2619129-21turbostat11:10:003
2273834991613,2cyclictest2726021-21irqrtprio12:15:172
2273834991613,2cyclictest2584177-21needreboot10:45:192
2273834991613,2cyclictest2547217-21fschecks_time10:25:162
2273834991613,2cyclictest2511631-21ntp_kernel_err10:00:212
2273834991613,2cyclictest2482668-21gltestperf09:45:132
2273834991613,2cyclictest2413095-21cpuspeed_turbos09:00:172
2273834991613,2cyclictest2412214-21unixbench-2d08:55:252
2273834991613,2cyclictest2319808-21cpuspeed_turbos07:45:112
2273834991613,2cyclictest2273822-21cyclictest12:25:182
2273834991613,2cyclictest0-21swapper/211:05:182
2273834991612,3cyclictest2600525-21mailstats10:55:242
2273834991612,2cyclictest0-21swapper/208:45:012
2273834991611,4cyclictest2645129-21kthreadcore11:25:172
2273831991613,2cyclictest761-21cups-browsed10:10:011
2273831991613,2cyclictest2552183-21systemctl10:25:241
2273831991613,2cyclictest2535816-21head10:15:201
2273831991613,2cyclictest2349618-21processes08:05:201
2273831991613,2cyclictest0-21swapper/109:55:011
2273831991613,2cyclictest0-21swapper/109:45:181
2273831991613,2cyclictest0-21swapper/108:00:171
2273831991612,3cyclictest0-21swapper/110:55:001
2273831991610,5cyclictest2722503-21unixbench_multi12:10:241
2273831991610,4cyclictest3818356-21systemd-journal07:40:001
2273829991614,1cyclictest2741334-21cpuspeed_turbos12:25:180
2273829991614,1cyclictest2661271-21kthreadcore11:35:190
2273829991614,1cyclictest2474360-21tr09:40:130
2273829991614,1cyclictest0-21swapper/008:55:000
2273829991613,2cyclictest2653227-21kthreadcore11:30:190
2273829991613,2cyclictest2653227-21kthreadcore11:30:180
2273829991613,2cyclictest2558424-21kthreadcore10:30:190
2273829991613,2cyclictest2383390-21kthreadcore08:35:160
2273829991613,2cyclictest2301193-21tr07:30:170
2273829991612,3cyclictest171ktimers/010:55:150
2273829991611,4cyclictest2381729-21cron08:35:010
26247132150,13sleep02624811-21grep11:10:240
2273835991513,1cyclictest0-21swapper/307:15:003
2273835991512,1cyclictest0-21swapper/312:30:183
2273835991512,1cyclictest0-21swapper/312:20:183
2273835991512,1cyclictest0-21swapper/312:05:243
2273835991512,1cyclictest0-21swapper/311:41:033
2273835991512,1cyclictest0-21swapper/310:32:403
2273835991512,1cyclictest0-21swapper/310:25:253
2273835991512,1cyclictest0-21swapper/310:08:523
2273835991512,1cyclictest0-21swapper/310:00:223
2273835991512,1cyclictest0-21swapper/309:40:313
2273835991512,1cyclictest0-21swapper/309:25:173
2273835991512,1cyclictest0-21swapper/309:20:003
2273835991512,1cyclictest0-21swapper/308:50:193
2273835991512,1cyclictest0-21swapper/307:30:283
2273835991512,1cyclictest0-21swapper/307:22:183
2273835991510,2cyclictest0-21swapper/308:25:003
227383499158,5cyclictest0-21swapper/210:05:162
2273834991512,2cyclictest514-21systemd-logind09:30:002
2273834991512,2cyclictest2754095-21missed_timers12:30:192
2273834991512,2cyclictest2734524-21kthreadcore12:20:172
2273834991512,2cyclictest2700883-21gltestperf12:00:172
2273834991512,2cyclictest2681822-21unixbench-2d11:45:242
2273834991512,2cyclictest2668089-21cpuspeed_turbos11:40:132
2273834991512,2cyclictest2640712-21needreboot11:20:212
2273834991512,2cyclictest2474556-21gltestperf09:40:152
2273834991512,2cyclictest2441779-21cut09:20:142
2273834991512,2cyclictest2376188-21gltestperf08:30:142
2273834991512,2cyclictest2375607-21/usr/sbin/munin08:30:012
2273834991512,2cyclictest2362743-21cron08:20:002
2273834991512,2cyclictest2355758-21proc_pri08:10:242
2273834991512,2cyclictest2288478-21acpi07:20:132
2273834991512,2cyclictest2274033-21acpi07:10:122
2273834991512,2cyclictest0-21swapper/210:20:282
2273834991512,2cyclictest0-21swapper/209:40:012
2273834991511,3cyclictest0-21swapper/210:35:012
227383199153,11cyclictest0-21swapper/108:20:191
2273831991513,1cyclictest2642777-21kworker/u8:3-rpciod11:33:341
2273831991513,1cyclictest2642777-21kworker/u8:3-rpciod11:33:341
2273831991513,1cyclictest0-21swapper/112:00:151
2273831991512,2cyclictest508-21polkitd10:10:231
2273831991512,2cyclictest2556384-21kthreadcore10:30:171
2273831991512,2cyclictest2503513-21tail09:55:201
2273831991512,2cyclictest2394410-21cpuspeed_turbos08:45:141
2273831991512,2cyclictest2294917-21apt-get07:25:131
2273831991512,2cyclictest0-21swapper/111:40:001
2273831991512,2cyclictest0-21swapper/111:05:001
2273831991512,2cyclictest0-21swapper/108:55:001
2273831991511,3cyclictest2579438-21cut10:45:141
227383199150,3cyclictest2381173-21timerwakeupswit08:30:231
227382999157,7cyclictest0-21swapper/012:30:180
2273829991512,2cyclictest2700731-21tr12:00:150
2273829991512,2cyclictest2693852-21kthreadcore11:55:170
2273829991512,2cyclictest2692124-21latency_hist11:55:000
2273829991512,2cyclictest2611073-21sh11:05:000
2273829991512,2cyclictest2588249-21irqrtprio10:50:170
2273829991512,2cyclictest2494026-21kthreadcore09:50:190
2273829991512,2cyclictest2399774-21unixbench_singl08:45:270
2273829991512,2cyclictest2372667-21kthreadcore08:25:220
2273829991510,4cyclictest2523176-21irqcore10:10:170
2273829991510,3cyclictest2574050-21kthreadcore10:40:160
2273835991412,1cyclictest0-21swapper/312:25:193
2273835991412,1cyclictest0-21swapper/312:15:153
2273835991412,1cyclictest0-21swapper/309:10:143
2273835991412,1cyclictest0-21swapper/308:00:153
2273835991411,1cyclictest0-21swapper/312:35:203
2273835991411,1cyclictest0-21swapper/311:55:453
2273835991411,1cyclictest0-21swapper/311:45:133
2273835991411,1cyclictest0-21swapper/311:35:473
2273835991411,1cyclictest0-21swapper/311:30:183
2273835991411,1cyclictest0-21swapper/311:30:183
2273835991411,1cyclictest0-21swapper/310:52:413
2273835991411,1cyclictest0-21swapper/309:55:073
2273835991411,1cyclictest0-21swapper/307:40:223
2273835991410,1cyclictest0-21swapper/307:50:203
227383499149,4cyclictest2406031-21sshd08:51:432
227383499148,4cyclictest2571045-21apt10:40:102
227383499147,5cyclictest2282095-21diskstats07:15:022
227383499145,2cyclictest0-21swapper/211:00:392
2273834991411,2cyclictest2714169-21unixbench_multi12:05:242
2273834991411,2cyclictest2457942-21gltestperf09:30:162
2273834991411,2cyclictest2338421-21cpu08:00:142
2273834991411,2cyclictest2329171-21kthreadcore07:50:222
2273834991410,2cyclictest0-21swapper/211:55:162
227383199149,4cyclictest3818356-21systemd-journal07:44:551
227383199144,5cyclictest2412472-21kworker/1:2-events10:20:191
227383199143,2cyclictest191rcu_preempt11:20:131
2273831991412,1cyclictest2425028-21diskmemload12:18:151
2273831991412,1cyclictest2425028-21diskmemload09:11:341
2273831991411,2cyclictest331ktimers/111:30:001
2273831991411,2cyclictest2710040-21kthreadcore12:05:171
2273831991411,2cyclictest2689839-21timerwakeupswit11:50:251
2273831991411,2cyclictest2684030-21cron11:50:001
2273831991411,2cyclictest2623209-21kthreadcore11:10:191
2273831991411,2cyclictest2475233-21irqrtprio09:40:171
2273831991411,2cyclictest1-21systemd09:22:081
2273831991411,2cyclictest0-21swapper/112:35:161
2273831991411,2cyclictest0-21swapper/111:40:151
2273831991411,2cyclictest0-21swapper/110:45:001
2273831991411,2cyclictest0-21swapper/108:40:151
2273831991410,3cyclictest0-21swapper/112:25:001
227382999149,3cyclictest2426811-21kthreadcore09:10:170
227382999148,4cyclictest2737316-21cut12:20:190
2273829991412,1cyclictest2434204-21irqcore09:15:170
2273829991412,1cyclictest2287885-21kworker/0:1+kacpid10:21:450
2273829991411,2cyclictest2550618-21kthreadcore10:25:190
2273829991411,2cyclictest2535349-21kthreadcore10:15:190
2273829991411,2cyclictest2441616-21cpu09:20:130
2273829991411,2cyclictest2355983-21timerandwakeup08:10:250
2273829991411,2cyclictest2282642-21gltestperf07:15:150
2273829991411,2cyclictest171ktimers/007:55:000
2273829991411,2cyclictest0-21swapper/010:05:160
2273829991411,2cyclictest0-21swapper/009:30:000
2273829991411,2cyclictest0-21swapper/008:20:010
2273835991311,1cyclictest0-21swapper/311:20:243
2273835991311,1cyclictest0-21swapper/311:16:143
2273835991311,1cyclictest0-21swapper/311:00:083
2273835991311,1cyclictest0-21swapper/309:00:173
2273835991311,1cyclictest0-21swapper/308:10:183
2273835991311,1cyclictest0-21swapper/307:35:183
2273835991311,1cyclictest0-21swapper/307:25:333
2273835991310,1cyclictest0-21swapper/311:25:503
2273835991310,1cyclictest0-21swapper/309:20:103
2273835991310,1cyclictest0-21swapper/308:45:153
2273835991310,1cyclictest0-21swapper/308:40:223
2273835991310,1cyclictest0-21swapper/308:35:213
227383499138,4cyclictest0-21swapper/212:15:012
2273834991311,1cyclictest0-21swapper/209:05:192
2273834991310,2cyclictest2313908-21gltestperf07:40:142
2273834991310,2cyclictest0-21swapper/211:15:242
2273834991310,2cyclictest0-21swapper/210:15:212
227383199139,3cyclictest2386977-21ntp_kernel_err08:35:191
227383199138,4cyclictest3818356-21systemd-journal08:14:331
227383199138,4cyclictest0-21swapper/108:25:261
227383199131,2cyclictest471rcuog/208:00:001
2273831991310,2cyclictest2619145-21sh11:10:001
2273831991310,2cyclictest2507057-21grep10:00:161
2273831991310,2cyclictest2301985-21irqrtprio07:30:201
2273831991310,2cyclictest2288813-21gltestperf07:20:171
2273831991310,2cyclictest0-21swapper/107:10:131
227383199130,4cyclictest2457305-21sh09:30:001
227382999139,3cyclictest2414395-21sed09:00:210
227382999138,4cyclictest501-21dbus-daemon07:25:020
2273829991311,1cyclictest2287885-21kworker/0:1+kacpid07:40:260
2273829991310,2cyclictest2636155-21irqcore11:20:170
2273829991310,2cyclictest2507979-21kthreadcore10:00:170
2273829991310,2cyclictest2486052-21kthreadcore09:45:160
2273829991310,2cyclictest2350422-21cron08:10:000
2273829991310,2cyclictest2332345-21tr07:55:140
2273829991310,2cyclictest2279976-21turbostat07:15:000
2273829991310,1cyclictest0-21swapper/012:05:130
227382999130,3cyclictest2724671-21munin-run12:15:000
2273835991210,1cyclictest0-21swapper/308:55:153
227383199126,4cyclictest2421668-21cut09:05:141
227383199126,4cyclictest2320472-21sed07:45:141
227382999129,2cyclictest2310159-21kthreadcore07:35:180
227382999128,3cyclictest2406948-21cpuspeed_turbos08:55:140
227382999128,3cyclictest2364690-21sed08:20:180
2273829991210,1cyclictest2668109-21tr11:40:130
2273829991210,1cyclictest2611113-21kworker/0:2+kacpid11:15:420
227382999118,2cyclictest2419329-21fschecks_count09:05:110
227382999115,4cyclictest2338634-21grep08:00:150
227382999107,1cyclictest0-21swapper/007:10:000
22738359986,1cyclictest0-21swapper/307:10:003
22738319980,6cyclictest2273734-21latency_hist07:10:001
2269762281,4sleep22271807-21cut07:05:182
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional