You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 10:25
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2.osadl.org (updated Wed Nov 19, 2025 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
474724432,0sleep20-21swapper/218:19:552
444723929,0sleep30-21swapper/318:16:273
3018223827,0sleep00-21swapper/018:15:160
439823727,0sleep10-21swapper/118:15:491
329980,0watchdog/30-21swapper/322:50:213
48329976,0cyclictest0-21swapper/220:33:232
329970,0watchdog/3299250irq/32-eth0-rx-21:30:253
329970,0watchdog/30-21swapper/322:35:173
329970,0watchdog/30-21swapper/320:38:373
329970,0watchdog/30-21swapper/320:00:253
29925070,0irq/32-eth0-rx-0-21swapper/323:42:443
29925070,0irq/32-eth0-rx-0-21swapper/322:34:373
29925070,0irq/32-eth0-rx-0-21swapper/320:45:233
8315060,0irq/26-0000:00:5759-21python22:20:412
48329960,0cyclictest0-21swapper/221:07:552
48319966,0cyclictest16173-21diskmemload22:16:431
48319966,0cyclictest0-21swapper/122:43:021
48319960,0cyclictest0-21swapper/119:05:191
48309960,0cyclictest0-21swapper/022:20:280
339960,0migration/329718-21munin-node22:00:193
339960,0migration/311185-21fschecks_time22:45:223
329960,0watchdog/3432-21memory20:05:253
329960,0watchdog/3299250irq/32-eth0-rx-21:53:373
329960,0watchdog/323088-21sh23:46:093
329960,0watchdog/30-21swapper/323:35:373
329960,0watchdog/30-21swapper/323:31:453
329960,0watchdog/30-21swapper/323:20:053
329960,0watchdog/30-21swapper/323:17:373
329960,0watchdog/30-21swapper/323:07:573
329960,0watchdog/30-21swapper/322:26:173
329960,0watchdog/30-21swapper/321:57:013
329960,0watchdog/30-21swapper/321:40:053
329960,0watchdog/30-21swapper/321:35:053
329960,0watchdog/30-21swapper/321:20:253
329960,0watchdog/30-21swapper/321:15:053
329960,0watchdog/30-21swapper/321:12:373
329960,0watchdog/30-21swapper/321:00:213
329960,0watchdog/30-21swapper/320:59:093
329960,0watchdog/30-21swapper/320:54:213
329960,0watchdog/30-21swapper/320:23:053
329960,0watchdog/30-21swapper/319:42:413
329960,0watchdog/30-21swapper/319:20:213
329960,0watchdog/30-21swapper/318:55:333
329960,0watchdog/30-21swapper/318:30:253
29925060,0irq/32-eth0-rx-5608-21cut18:40:273
29925060,0irq/32-eth0-rx-29092-21ssh23:13:413
29925060,0irq/32-eth0-rx-0-21swapper/323:27:383
29925060,0irq/32-eth0-rx-0-21swapper/323:02:283
29925060,0irq/32-eth0-rx-0-21swapper/322:56:373
29925060,0irq/32-eth0-rx-0-21swapper/322:41:503
29925060,0irq/32-eth0-rx-0-21swapper/322:19:223
29925060,0irq/32-eth0-rx-0-21swapper/322:06:093
29925060,0irq/32-eth0-rx-0-21swapper/321:05:053
29925060,0irq/32-eth0-rx-0-21swapper/320:40:193
29925060,0irq/32-eth0-rx-0-21swapper/320:25:193
8315050,0irq/26-0000:00:6120-21tune2fs19:50:232
8315050,0irq/26-0000:00:0-21swapper/218:25:052
48329955,0cyclictest0-21swapper/223:40:032
48329955,0cyclictest0-21swapper/223:06:382
48329955,0cyclictest0-21swapper/222:34:092
48329955,0cyclictest0-21swapper/221:00:302
48329954,0cyclictest28081-21grep19:20:252
48329950,0cyclictest0-21swapper/223:34:222
48329950,0cyclictest0-21swapper/221:33:552
48319955,0cyclictest0-21swapper/123:41:221
48319950,0cyclictest0-21swapper/123:03:511
48319950,0cyclictest0-21swapper/120:39:551
48309955,0cyclictest0-21swapper/023:26:220
48309955,0cyclictest0-21swapper/022:36:420
48309955,0cyclictest0-21swapper/021:57:360
48309955,0cyclictest0-21swapper/021:22:460
48309955,0cyclictest0-21swapper/020:56:260
48309954,0cyclictest23871-21snmpd18:25:150
48309954,0cyclictest0-21swapper/019:25:280
48309950,0cyclictest0-21swapper/023:30:560
48309950,0cyclictest0-21swapper/022:49:380
48309950,0cyclictest0-21swapper/021:41:500
48309950,0cyclictest0-21swapper/020:36:430
339950,0migration/38881-21latency_hist19:30:053
339950,0migration/33511-21kthreadcore21:25:243
339950,0migration/327574-21fschecks_time18:35:213
339950,0migration/32016-21kthreadcore19:00:263
339950,0migration/319154-21taskset19:11:553
339950,0migration/316708-21fschecks_time18:50:193
339950,0migration/31256-21kthreadcore19:45:253
329950,0watchdog/31569-21kthreadcore22:20:333
329950,0watchdog/314612-21cat18:25:253
329950,0watchdog/30-21swapper/321:45:373
329950,0watchdog/30-21swapper/320:15:333
329950,0watchdog/30-21swapper/320:10:213
329950,0watchdog/30-21swapper/319:56:013
329950,0watchdog/30-21swapper/319:51:093
329950,0watchdog/30-21swapper/319:35:213
329950,0watchdog/30-21swapper/319:26:093
329950,0watchdog/30-21swapper/319:15:493
329950,0watchdog/30-21swapper/319:05:173
329950,0watchdog/30-21swapper/318:45:053
329950,0watchdog/30-21swapper/318:20:493
29935050,0irq/33-eth0-tx-9003-21ssh22:42:092
29935050,0irq/33-eth0-tx-7651-21ssh21:25:332
29935050,0irq/33-eth0-tx-5506-21tr21:45:242
29935050,0irq/33-eth0-tx-32095-21ssh20:26:012
29935050,0irq/33-eth0-tx-29901-21munin-node23:15:232
29935050,0irq/33-eth0-tx-27764-21perl21:40:212
29935050,0irq/33-eth0-tx-26669-21cut21:20:242
29935050,0irq/33-eth0-tx-21242-21munin-node23:10:312
29935050,0irq/33-eth0-tx-20818-21ssh20:56:402
29935050,0irq/33-eth0-tx-10345-21munin-node22:45:252
29935050,0irq/33-eth0-tx-0-21swapper/222:37:252
29935050,0irq/33-eth0-tx-0-21swapper/222:05:252
29935050,0irq/33-eth0-tx-0-21swapper/221:37:542
29935050,0irq/33-eth0-tx-0-21swapper/019:30:210
29935050,0irq/33-eth0-tx-0-21swapper/019:15:290
29925050,0irq/32-eth0-rx-1694-21munin-node20:30:233
29925050,0irq/32-eth0-rx-0-21swapper/322:10:203
27150,0ktimersoftd/20-21swapper/221:11:512
259950,0migration/28460-21fschecks_time23:40:212
259950,0migration/27919-21latency_hist22:25:052
259950,0migration/229768-21sh22:15:452
259950,0migration/227297-21fschecks_count18:35:202
259950,0migration/221224-21kthreadcore23:45:262
259950,0migration/220968-21munin-node22:10:332
259950,0migration/218081-21kthreadcore19:10:282
259950,0migration/212201-21irqrtprio20:35:242
249950,0watchdog/26567-21df23:20:212
249950,0watchdog/223373-21cut19:40:172
19150,0ktimersoftd/10-21swapper/122:38:261
19150,0ktimersoftd/10-21swapper/120:10:351
179950,0migration/1633-21kthreadcore19:00:251
179950,0migration/119983-21cpuspeed_turbos20:40:211
179950,0migration/110742-21kthreadcore19:50:281
169950,0watchdog/15582-21swap23:35:331
169950,0watchdog/112760-21fschecks_count18:25:211
149950,0migration/06819-21irqcore19:50:260
149950,0migration/030788-21cpuspeed_turbos19:45:190
149950,0migration/030472-21sh21:04:310
149950,0migration/025387-21kthreadcore23:10:240
149950,0migration/018127-21irqrtprio22:30:230
149950,0migration/01486-21kthreadcore20:45:250
149950,0migration/013946-21sh23:23:420
8315040,0irq/26-0000:00:23165-21kthreadcore19:15:242
8315040,0irq/26-0000:00:0-21swapper/220:40:052
8315040,0irq/26-0000:00:0-21swapper/220:05:242
48329944,0cyclictest0-21swapper/221:15:192
48329943,0cyclictest9807-21awk18:45:222
48329943,0cyclictest28675-21awk22:55:212
48329943,0cyclictest23117-21sadc19:40:052
48329943,0cyclictest21851-21awk21:55:242
48329943,0cyclictest12219-21sadc19:10:052
48319944,0cyclictest0-21swapper/121:00:051
48319944,0cyclictest0-21swapper/120:25:151
48319944,0cyclictest0-21swapper/120:21:091
48319943,0cyclictest3599-21lldpd22:55:331
48319943,0cyclictest3599-21lldpd21:56:321
48319943,0cyclictest3599-21lldpd20:31:301
48319943,0cyclictest3262-21irqbalance23:30:191
48319943,0cyclictest3262-21irqbalance23:25:191
48319943,0cyclictest3262-21irqbalance22:47:091
48319943,0cyclictest3262-21irqbalance22:01:091
48319943,0cyclictest3262-21irqbalance20:50:191
48319943,0cyclictest3262-21irqbalance18:30:191
48319943,0cyclictest32032-21awk21:05:211
48319943,0cyclictest29592-21awk20:45:221
48319943,0cyclictest27729-21awk18:35:221
48319943,0cyclictest20986-21awk20:00:231
48319943,0cyclictest19602-21ps21:35:231
48319943,0cyclictest17133-21awk21:15:231
48319943,0cyclictest17039-21awk18:50:211
48309944,0cyclictest0-21swapper/020:26:100
48309944,0cyclictest0-21swapper/018:55:210
48309943,0cyclictest9838-21awk18:45:220
48309943,0cyclictest5004-21awk23:00:200
48309943,0cyclictest27910-21awk19:20:240
48309943,0cyclictest2714-21awk22:40:220
48309943,0cyclictest23871-21snmpd21:52:000
48309943,0cyclictest20732-21awk20:40:230
48309943,0cyclictest19426-21awk21:35:210
48309943,0cyclictest17403-21awk23:45:240
48309943,0cyclictest17213-21awk19:35:210
48309940,0cyclictest0-21swapper/019:03:350
29935040,0irq/33-eth0-tx-5504-21kthreadcore23:00:232
29935040,0irq/33-eth0-tx-0-21swapper/223:27:242
29935040,0irq/33-eth0-tx-0-21swapper/222:50:112
29935040,0irq/33-eth0-tx-0-21swapper/220:50:222
29935040,0irq/33-eth0-tx-0-21swapper/220:46:492
29935040,0irq/33-eth0-tx-0-21swapper/220:20:322
29935040,0irq/33-eth0-tx-0-21swapper/018:50:180
29935040,0irq/33-eth0-tx-0-21swapper/018:20:440
259940,0migration/27637-21missed_timers19:25:292
259940,0migration/231358-21munin-node19:00:222
259940,0migration/230601-21iostat22:00:222
259940,0migration/22186-21fschecks_time18:40:232
259940,0migration/220962-21if_eth120:00:232
259940,0migration/214661-21kthreadcore20:15:262
259940,0migration/212992-21fschecks_time21:50:212
249940,0watchdog/21870-21telnet20:10:172
179940,0migration/19546-21irqrtprio22:25:251
179940,0migration/19173-21kthreadcore22:05:241
179940,0migration/18635-21taskset19:29:151
179940,0migration/17939-21fschecks_count21:10:221
179940,0migration/17219-21kthreadcore18:20:221
179940,0migration/16318-21kthreadcore21:45:251
179940,0migration/130602-21aten_r1power_en19:45:181
179940,0migration/130330-21munin-node23:15:191
179940,0migration/126658-21idleruntime-cro19:20:051
179940,0migration/126138-21kthreadcore19:40:241
179940,0migration/119643-21munin-node19:35:231
179940,0migration/119510-21latency_hist19:15:051
179940,0migration/119046-21kthreadcore22:30:241
179940,0migration/115888-21runrttasks20:17:001
179940,0migration/114260-21latency_hist22:10:051
179940,0migration/113361-21taskset23:21:551
179940,0migration/111740-21kthreadcore21:30:241
169940,0watchdog/17780-21users21:25:331
169940,0watchdog/127948-21cut21:40:211
169940,0watchdog/123183-21munin-node18:55:211
169940,0watchdog/114142-21kernelversion23:05:251
169940,0watchdog/10-21swapper/122:50:211
169940,0watchdog/10-21swapper/121:50:211
169940,0watchdog/10-21swapper/121:22:291
169940,0watchdog/10-21swapper/120:55:251
169940,0watchdog/10-21swapper/118:40:211
149940,0migration/06436-21kthreadcore21:45:250
149940,0migration/02989-21missed_timers22:00:250
149940,0migration/028772-21kthreadcore19:40:250
149940,0migration/027571-21taskset22:55:080
149940,0migration/01839-21fschecks_time21:25:210
149940,0migration/016899-21fschecks_count20:20:190
149940,0migration/015082-21runrttasks23:42:370
149940,0migration/015008-21sh22:26:240
149940,0migration/011800-21kthreadcore21:30:240
48329933,0cyclictest3599-21lldpd18:58:472
48329932,0cyclictest9987-21awk19:30:222
48329932,0cyclictest20553-21awk18:30:232
48329932,0cyclictest17024-21awk18:50:212
48329932,0cyclictest13758-21awk19:55:232
48319932,0cyclictest529-21sed22:20:321
48319932,0cyclictest3599-21lldpd19:55:171
48319932,0cyclictest22224-21cat23:45:271
48319930,0cyclictest9372-21diskstats19:30:191
48309933,0cyclictest0-21swapper/021:10:090
48309932,0cyclictest8197-21cut20:50:240
48309932,0cyclictest5434-21ssh21:05:410
48309932,0cyclictest4425-21kthreadcore23:35:250
48309932,0cyclictest3599-21lldpd20:32:300
48309932,0cyclictest28122-21awk20:05:220
48309932,0cyclictest23871-21snmpd20:19:150
48309932,0cyclictest20259-21sh23:08:110
48309932,0cyclictest19476-21kthreadcore22:10:260
48309932,0cyclictest19223-21tr22:50:200
48309932,0cyclictest1460-21munin-node18:40:260
48309932,0cyclictest12951-21sh22:06:580
48309930,0cyclictest5940-21awk23:20:040
48309930,0cyclictest5245-21munin-node19:05:190
48309930,0cyclictest24026-21sh21:19:020
48309930,0cyclictest2268-21df_inode20:10:190
48309930,0cyclictest0-21swapper/020:00:280
48309930,0cyclictest0-21swapper/018:34:040
29935030,0irq/33-eth0-tx-0-21swapper/019:55:180
29935030,0irq/33-eth0-tx-0-21swapper/019:10:260
29935030,0irq/33-eth0-tx-0-21swapper/018:35:320
249930,0watchdog/20-21swapper/219:45:252
249930,0watchdog/20-21swapper/218:25:212
179930,0migration/128367-21diskmemload23:11:411
169930,0watchdog/10-21swapper/118:45:251
48309920,0cyclictest0-21swapper/022:15:050
169920,0watchdog/10-21swapper/120:05:051
169920,0watchdog/10-21swapper/119:20:051
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional