You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-15 - 19:53
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Wed Oct 15, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2768025243,0sleep30-21swapper/306:20:273
2770424031,0sleep10-21swapper/106:20:451
2763124028,0sleep20-21swapper/206:19:542
2685223622,0sleep00-21swapper/006:17:320
329970,0watchdog/319539-21kthreadcore08:27:303
329970,0watchdog/31616-21ssh11:10:423
329970,0watchdog/314345-21fschecks_count10:02:263
329970,0watchdog/30-21swapper/308:57:223
29925070,0irq/32-eth0-rx-0-21swapper/309:46:293
339960,0migration/37464-21sh10:36:153
339960,0migration/34088-21sh09:36:413
339960,0migration/3333-21unixbench_multi08:52:373
339960,0migration/326804-21sh11:23:363
339960,0migration/324123-21sh10:45:533
339960,0migration/324059-21munin-node07:22:383
339960,0migration/318123-21sh09:05:063
329960,0watchdog/3299250irq/32-eth0-rx-11:12:583
329960,0watchdog/3299250irq/32-eth0-rx-09:19:383
329960,0watchdog/30-21swapper/311:48:143
329960,0watchdog/30-21swapper/311:37:103
329960,0watchdog/30-21swapper/311:33:423
329960,0watchdog/30-21swapper/311:31:143
329960,0watchdog/30-21swapper/311:18:303
329960,0watchdog/30-21swapper/310:56:063
329960,0watchdog/30-21swapper/310:49:103
329960,0watchdog/30-21swapper/310:28:103
329960,0watchdog/30-21swapper/310:19:343
329960,0watchdog/30-21swapper/310:15:343
329960,0watchdog/30-21swapper/310:07:303
329960,0watchdog/30-21swapper/309:37:343
329960,0watchdog/30-21swapper/309:08:383
329960,0watchdog/30-21swapper/308:25:103
329960,0watchdog/30-21swapper/308:09:343
29925060,0irq/32-eth0-rx-3311-21strings09:52:273
29925060,0irq/32-eth0-rx-28343-21munin-plugin-st09:32:093
29925060,0irq/32-eth0-rx-16061-21hwlatdetect10:22:273
29925060,0irq/32-eth0-rx-0-21swapper/311:43:383
29925060,0irq/32-eth0-rx-0-21swapper/310:39:113
29925060,0irq/32-eth0-rx-0-21swapper/310:00:563
29925060,0irq/32-eth0-rx-0-21swapper/308:42:323
29925060,0irq/32-eth0-rx-0-21swapper/308:39:133
278949960,0cyclictest0-21swapper/011:27:410
8315050,0irq/26-0000:00:24815-21sed07:27:092
339950,0migration/39570-21timerandwakeup07:12:373
339950,0migration/33806-21cpuspeed_turbos07:57:243
339950,0migration/332030-21kthreadcore07:07:293
339950,0migration/326576-21kthreadcore07:02:323
339950,0migration/323793-21kthreadcore07:47:273
339950,0migration/313538-21kthreadcore06:32:303
339950,0migration/312679-21munin-node06:52:303
329950,0watchdog/324399-21ssh08:32:223
329950,0watchdog/32231-21netstat07:52:303
329950,0watchdog/30-21swapper/311:02:143
329950,0watchdog/30-21swapper/310:57:263
329950,0watchdog/30-21swapper/309:47:503
329950,0watchdog/30-21swapper/309:22:103
329950,0watchdog/30-21swapper/308:48:183
329950,0watchdog/30-21swapper/308:18:303
329950,0watchdog/30-21swapper/308:12:183
329950,0watchdog/30-21swapper/308:04:303
329950,0watchdog/30-21swapper/307:42:263
329950,0watchdog/30-21swapper/307:39:143
329950,0watchdog/30-21swapper/307:32:303
329950,0watchdog/30-21swapper/307:27:143
329950,0watchdog/30-21swapper/307:18:553
329950,0watchdog/30-21swapper/306:57:393
329950,0watchdog/30-21swapper/306:47:273
329950,0watchdog/30-21swapper/306:47:273
329950,0watchdog/30-21swapper/306:43:113
329950,0watchdog/30-21swapper/306:37:153
329950,0watchdog/30-21swapper/306:27:233
329950,0watchdog/30-21swapper/306:22:513
29935050,0irq/33-eth0-tx-8315-21netstat08:57:282
29935050,0irq/33-eth0-tx-4532-21munin-node09:37:242
29935050,0irq/33-eth0-tx-3660-21munin-node09:17:272
29935050,0irq/33-eth0-tx-31811-21munin-node06:47:310
29935050,0irq/33-eth0-tx-31811-21munin-node06:47:310
29935050,0irq/33-eth0-tx-27153-21perl11:07:232
29935050,0irq/33-eth0-tx-24793-21munin-node10:47:242
29935050,0irq/33-eth0-tx-17997-21munin-node08:07:390
29935050,0irq/33-eth0-tx-14252-21munin-node07:42:330
29935050,0irq/33-eth0-tx-13569-21irqstats09:22:282
29935050,0irq/33-eth0-tx-10807-21chrt11:16:592
29935050,0irq/33-eth0-tx-10479-21munin-node07:17:250
29935050,0irq/33-eth0-tx-0-21swapper/211:51:442
29935050,0irq/33-eth0-tx-0-21swapper/211:43:322
29935050,0irq/33-eth0-tx-0-21swapper/211:37:322
29935050,0irq/33-eth0-tx-0-21swapper/210:43:582
29935050,0irq/33-eth0-tx-0-21swapper/209:07:262
29935050,0irq/33-eth0-tx-0-21swapper/008:22:220
29935050,0irq/33-eth0-tx-0-21swapper/007:52:230
29925050,0irq/32-eth0-rx-0-21swapper/309:12:173
278969955,0cyclictest83150irq/26-0000:00:10:32:352
278969955,0cyclictest0-21swapper/211:27:322
278949955,0cyclictest0-21swapper/010:34:240
278949955,0cyclictest0-21swapper/009:44:440
278949955,0cyclictest0-21swapper/009:16:120
278949950,0cyclictest0-21swapper/011:48:390
278949950,0cyclictest0-21swapper/010:38:010
278949950,0cyclictest0-21swapper/006:58:150
27150,0ktimersoftd/20-21swapper/209:04:132
259950,0migration/27079-21df_abs10:17:232
259950,0migration/23715-21kthreadcore08:37:312
259950,0migration/227325-21sh07:02:362
259950,0migration/224188-21kthreadcore10:07:302
259950,0migration/21626-21missed_timers06:22:312
249950,0watchdog/227205-21kthreadcore10:27:302
249950,0watchdog/223332-21needreboot06:37:312
249950,0watchdog/222086-21rt-features08:27:342
179950,0migration/17451-21kthreadcore09:57:281
179950,0migration/16841-21latency10:52:301
179950,0migration/14881-21kthreadcore08:17:261
179950,0migration/131949-21latency_hist07:32:091
179950,0migration/127468-21ntp_offset07:47:301
179950,0migration/126421-21munin-node08:12:311
179950,0migration/121992-21latency_hist10:07:091
179950,0migration/120615-21taskset06:59:171
179950,0migration/113676-21munin-node08:22:331
149950,0migration/09576-21kthreadcore10:17:290
149950,0migration/09215-21aten_r1power_fr08:42:260
149950,0migration/08877-21kthreadcore07:57:300
149950,0migration/07083-21unixbench-2d08:37:390
149950,0migration/06007-21cpuspeed_turbos09:57:230
149950,0migration/05901-21kthreadcore11:12:310
149950,0migration/05557-21kthreadcore08:57:260
149950,0migration/05272-21fschecks_time09:37:250
149950,0migration/03857-21taskset09:35:290
149950,0migration/028247-21taskset07:49:590
149950,0migration/027760-21kthreadcore07:27:280
149950,0migration/026565-21kthreadcore08:32:280
149950,0migration/025031-21kernelversion10:27:290
149950,0migration/021964-21sh11:42:170
149950,0migration/021876-21kthreadcore08:47:300
149950,0migration/019216-21latency11:37:340
149950,0migration/017179-21irqcore08:27:290
149950,0migration/017129-21timerandwakeup08:02:500
149950,0migration/017020-21kthreadcore10:22:300
149950,0migration/016133-21kthreadcore10:02:300
149950,0migration/013875-21kthreadcore06:32:300
149950,0migration/013188-21kthreadcore09:02:340
149950,0migration/011802-21sh11:34:040
149950,0migration/011796-21fschecks_count11:17:230
8315040,0irq/26-0000:00:17397-21latency_hist08:47:092
29935040,0irq/33-eth0-tx-4957-21fschecks_count11:32:242
29935040,0irq/33-eth0-tx-4838-21kthreadcore10:52:292
29935040,0irq/33-eth0-tx-25500-21ssh08:32:272
29935040,0irq/33-eth0-tx-21130-21ssh09:47:212
29935040,0irq/33-eth0-tx-17681-21munin-node07:22:320
29935040,0irq/33-eth0-tx-15898-21kthreadcore11:17:282
29935040,0irq/33-eth0-tx-12829-21ssh09:42:192
29935040,0irq/33-eth0-tx-0-21swapper/211:22:292
29935040,0irq/33-eth0-tx-0-21swapper/211:02:102
29935040,0irq/33-eth0-tx-0-21swapper/210:57:242
29935040,0irq/33-eth0-tx-0-21swapper/210:22:292
29935040,0irq/33-eth0-tx-0-21swapper/210:12:122
29935040,0irq/33-eth0-tx-0-21swapper/210:02:312
29935040,0irq/33-eth0-tx-0-21swapper/209:32:192
29935040,0irq/33-eth0-tx-0-21swapper/007:39:020
29935040,0irq/33-eth0-tx-0-21swapper/006:37:270
278969943,0cyclictest7766-21awk06:52:252
278969943,0cyclictest6905-21awk09:57:272
278969943,0cyclictest3599-21lldpd08:52:272
278969943,0cyclictest3599-21lldpd08:12:352
278969943,0cyclictest3599-21lldpd08:01:212
278969943,0cyclictest28659-21awk09:12:262
278969943,0cyclictest23871-21snmpd08:42:252
278969943,0cyclictest11543-21awk07:17:252
278959944,0cyclictest0-21swapper/109:27:261
278959943,0cyclictest5633-21awk09:37:271
278959943,0cyclictest3599-21lldpd11:32:021
278959943,0cyclictest3599-21lldpd09:52:501
278959943,0cyclictest3599-21lldpd09:42:501
278959943,0cyclictest3262-21irqbalance10:12:341
278959943,0cyclictest3262-21irqbalance10:10:041
278959943,0cyclictest31862-21awk11:47:261
278959943,0cyclictest23871-21snmpd08:58:101
278959943,0cyclictest16905-21awk08:27:281
278959943,0cyclictest10214-21awk08:42:311
278949943,0cyclictest4278-21awk07:12:270
278949943,0cyclictest3599-21lldpd10:16:200
278949943,0cyclictest3262-21irqbalance11:07:240
278949943,0cyclictest3262-21irqbalance10:07:240
278949943,0cyclictest3262-21irqbalance09:52:140
278949943,0cyclictest3262-21irqbalance09:48:340
278949943,0cyclictest3262-21irqbalance09:28:140
278949943,0cyclictest3262-21irqbalance09:18:140
278949943,0cyclictest3262-21irqbalance09:10:540
278949943,0cyclictest3262-21irqbalance08:17:240
278949943,0cyclictest3262-21irqbalance08:12:540
278949943,0cyclictest29194-21awk06:22:280
278949943,0cyclictest23871-21snmpd11:22:110
278949943,0cyclictest23871-21snmpd11:02:260
278949943,0cyclictest23871-21snmpd10:44:260
278949943,0cyclictest22175-21awk07:02:290
278949943,0cyclictest13079-21awk09:22:270
278949940,0cyclictest32154-21munin-node07:32:280
259940,0migration/2968-21kthreadcore09:52:262
259940,0migration/28714-21kthreadcore07:37:322
259940,0migration/27049-21runrttasks08:21:272
259940,0migration/25429-21kthreadcore07:32:302
259940,0migration/24409-21kthreadcore06:47:282
259940,0migration/24409-21kthreadcore06:47:282
259940,0migration/23614-21fschecks_count06:27:252
259940,0migration/214453-21switchtime10:37:362
259940,0migration/213978-21aten_r1power_fr06:57:202
259940,0migration/211177-21df_abs08:02:382
259940,0migration/210798-21kthreadcore08:22:282
249940,0watchdog/226614-21python09:27:342
249940,0watchdog/216444-21python06:32:352
249940,0watchdog/20-21swapper/207:22:312
249940,0watchdog/20-21swapper/206:42:272
179940,0migration/1872-21kthreadcore06:22:301
179940,0migration/17847-21kthreadcore07:12:301
179940,0migration/17569-21fschecks_time10:17:241
179940,0migration/1710-21irqcore06:47:271
179940,0migration/1710-21irqcore06:47:271
179940,0migration/16929-21kthreadcore09:17:281
179940,0migration/16804-21munin-node08:37:351
179940,0migration/15290-21kthreadcore11:12:311
179940,0migration/14418-21ksmtuned11:32:141
179940,0migration/14271-21irqcore06:27:281
179940,0migration/1330-21kthreadcore09:32:321
179940,0migration/130194-21kthreadcore10:47:301
179940,0migration/128418-21aten_r1power_po07:07:231
179940,0migration/12649-21timerandwakeup07:52:361
179940,0migration/125224-21fschecks_count06:42:271
179940,0migration/125155-21fschecks_time08:32:241
179940,0migration/121500-21kthreadcore11:22:271
179940,0migration/120937-21kthreadcore11:02:281
179940,0migration/120745-21kthreadcore07:22:301
179940,0migration/120190-21kthreadcore10:42:291
179940,0migration/118580-21fschecks_count08:07:261
179940,0migration/118428-21fschecks_time08:47:251
179940,0migration/117659-21taskset08:06:051
179940,0migration/116951-21kthreadcore07:42:281
179940,0migration/113415-21kthreadcore06:32:301
179940,0migration/111794-21kthreadcore07:37:331
179940,0migration/111662-21fschecks_count11:17:231
179940,0migration/111482-21kthreadcore10:37:281
179940,0migration/111379-21fschecks_count09:02:301
179940,0migration/110566-21aten_r1power_fr07:17:211
179940,0migration/110185-21fschecks_time10:57:241
169940,0watchdog/17119-21diskmemload09:49:341
169940,0watchdog/15632-21munin-node07:32:301
169940,0watchdog/132474-21ssh11:07:301
169940,0watchdog/13157-21ssh09:16:341
169940,0watchdog/123331-21needreboot06:37:311
169940,0watchdog/120928-21ssh11:40:101
169940,0watchdog/118096-21tr10:22:301
169940,0watchdog/10-21swapper/109:09:581
149940,0migration/06271-21kthreadcore06:27:290
149940,0migration/028461-21kthreadcore06:42:320
149940,0migration/011223-21kthreadcore10:57:280
278969932,0cyclictest3599-21lldpd07:12:252
278969932,0cyclictest23871-21snmpd07:47:102
278969930,0cyclictest0-21swapper/207:42:092
278959932,0cyclictest3599-21lldpd10:27:221
278959932,0cyclictest32534-21sensors08:52:331
278959932,0cyclictest10666-21cut06:52:281
278949932,0cyclictest8112-21sh10:54:100
278949932,0cyclictest26410-21cpuspeed_turbos08:52:230
278949930,0cyclictest6701-21munin-node06:52:280
259930,0migration/229014-21df_abs07:52:232
249930,0watchdog/220673-21kthreadcore08:07:302
249930,0watchdog/20-21swapper/207:07:112
179930,0migration/18207-21kthreadcore07:57:291
169930,0watchdog/10-21swapper/111:42:381
169930,0watchdog/10-21swapper/110:32:461
169930,0watchdog/10-21swapper/109:26:141
169930,0watchdog/10-21swapper/107:02:311
149930,0migration/025326-21fschecks_count10:47:250
278949920,0cyclictest28528-21cpu07:07:230
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional