You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-01 - 01:16
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot2.osadl.org (updated Fri Oct 31, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2316025041,0sleep20-21swapper/206:18:472
2322623928,0sleep30-21swapper/306:19:363
1644523726,0sleep00-21swapper/006:16:160
2326323525,0sleep10-21swapper/106:19:581
29925080,0irq/32-eth0-rx-0-21swapper/310:46:313
339960,0migration/327216-21sh11:27:153
339960,0migration/322949-21sh09:26:393
339960,0migration/316694-21dump-pmu-power07:46:073
329960,0watchdog/320543-21ssh08:48:443
329960,0watchdog/30-21swapper/311:41:283
329960,0watchdog/30-21swapper/311:37:563
329960,0watchdog/30-21swapper/311:07:243
329960,0watchdog/30-21swapper/311:03:323
329960,0watchdog/30-21swapper/310:54:403
329960,0watchdog/30-21swapper/310:44:083
329960,0watchdog/30-21swapper/310:31:563
329960,0watchdog/30-21swapper/310:26:283
329960,0watchdog/30-21swapper/310:16:123
329960,0watchdog/30-21swapper/310:02:123
329960,0watchdog/30-21swapper/309:46:323
329960,0watchdog/30-21swapper/309:41:283
329960,0watchdog/30-21swapper/309:36:363
329960,0watchdog/30-21swapper/309:22:363
329960,0watchdog/30-21swapper/308:58:363
329960,0watchdog/30-21swapper/308:53:043
329960,0watchdog/30-21swapper/308:42:523
329960,0watchdog/30-21swapper/308:37:323
329960,0watchdog/30-21swapper/308:29:483
329960,0watchdog/30-21swapper/308:25:203
329960,0watchdog/30-21swapper/307:52:483
329960,0watchdog/30-21swapper/307:21:323
29925060,0irq/32-eth0-rx-3483-21iostat_ios09:56:323
29925060,0irq/32-eth0-rx-0-21swapper/311:46:263
29925060,0irq/32-eth0-rx-0-21swapper/311:33:263
29925060,0irq/32-eth0-rx-0-21swapper/311:21:293
29925060,0irq/32-eth0-rx-0-21swapper/311:11:253
29925060,0irq/32-eth0-rx-0-21swapper/310:22:493
29925060,0irq/32-eth0-rx-0-21swapper/309:51:243
29925060,0irq/32-eth0-rx-0-21swapper/309:07:083
29925060,0irq/32-eth0-rx-0-21swapper/309:03:013
29925060,0irq/32-eth0-rx-0-21swapper/308:33:333
259960,0migration/228844-21latency_hist11:31:112
234649965,0cyclictest20504-21crond08:11:122
234589960,0cyclictest0-21swapper/111:46:341
149960,0migration/03948-21if_eth1.311:16:260
149960,0migration/030798-21sh11:49:400
149960,0migration/016993-21kthreadcore11:41:300
8315050,0irq/26-0000:00:17642-21sed11:06:112
8315050,0irq/26-0000:00:13137-21sed07:21:122
339950,0migration/38315-21kthreadcore07:16:323
339950,0migration/35574-21munin-node10:36:363
339950,0migration/332725-21kthreadcore07:11:293
339950,0migration/32881-21munin-node07:36:253
339950,0migration/327819-21aten_r1power_vo07:31:253
329950,0watchdog/315094-21cut06:56:283
329950,0watchdog/30-21swapper/311:16:243
329950,0watchdog/30-21swapper/310:56:123
329950,0watchdog/30-21swapper/310:11:123
329950,0watchdog/30-21swapper/308:16:243
329950,0watchdog/30-21swapper/308:11:363
329950,0watchdog/30-21swapper/308:07:363
329950,0watchdog/30-21swapper/308:03:083
329950,0watchdog/30-21swapper/307:49:283
329950,0watchdog/30-21swapper/307:30:123
329950,0watchdog/30-21swapper/307:07:243
329950,0watchdog/30-21swapper/307:01:203
329950,0watchdog/30-21swapper/306:53:443
329950,0watchdog/30-21swapper/306:46:483
329950,0watchdog/30-21swapper/306:41:203
329950,0watchdog/30-21swapper/306:37:403
329950,0watchdog/30-21swapper/306:27:443
329950,0watchdog/30-21swapper/306:22:083
29935050,0irq/33-eth0-tx-9493-21munin-node06:56:300
29935050,0irq/33-eth0-tx-6140-21munin-node09:01:352
29935050,0irq/33-eth0-tx-6015-21munin-node07:16:270
29935050,0irq/33-eth0-tx-5389-21munin-node11:36:312
29935050,0irq/33-eth0-tx-32512-21ssh10:56:002
29935050,0irq/33-eth0-tx-31124-21munin-node07:11:240
29935050,0irq/33-eth0-tx-25459-21grep10:11:322
29935050,0irq/33-eth0-tx-224022sleep211:45:352
29935050,0irq/33-eth0-tx-20262-21munin-node11:26:262
29935050,0irq/33-eth0-tx-13430-21kthreadcore11:01:302
29935050,0irq/33-eth0-tx-0-21swapper/209:06:302
29935050,0irq/33-eth0-tx-0-21swapper/008:21:280
29935050,0irq/33-eth0-tx-0-21swapper/006:41:350
29925050,0irq/32-eth0-rx-0-21swapper/310:07:283
29925050,0irq/32-eth0-rx-0-21swapper/309:31:373
29925050,0irq/32-eth0-rx-0-21swapper/309:16:123
29925050,0irq/32-eth0-rx-0-21swapper/309:11:323
29925050,0irq/32-eth0-rx-0-21swapper/307:56:263
29925050,0irq/32-eth0-rx-0-21swapper/306:31:253
259950,0migration/28706-21irqrtprio10:41:282
259950,0migration/2773-21munin-node10:56:262
259950,0migration/27138-21munin-node09:36:332
259950,0migration/229215-21sh11:46:352
259950,0migration/226464-21munin-node09:51:242
259950,0migration/214448-21kthreadcore08:26:302
259950,0migration/211717-21sh08:42:522
259950,0migration/210364-21munin-node09:41:272
234649955,0cyclictest0-21swapper/206:21:282
234649954,0cyclictest0-21swapper/207:46:372
234649950,0cyclictest0-21swapper/208:11:122
234589955,0cyclictest0-21swapper/111:35:411
234589955,0cyclictest0-21swapper/111:30:481
234589955,0cyclictest0-21swapper/110:55:381
234589955,0cyclictest0-21swapper/110:03:081
234589955,0cyclictest0-21swapper/108:41:381
234589950,0cyclictest0-21swapper/111:12:401
234589950,0cyclictest0-21swapper/109:16:001
234589950,0cyclictest0-21swapper/108:54:051
234589950,0cyclictest0-21swapper/108:31:231
234529955,0cyclictest17155-21kworker/0:010:59:160
234529955,0cyclictest0-21swapper/011:36:260
234529955,0cyclictest0-21swapper/009:06:580
234529954,0cyclictest0-21swapper/006:41:120
234529950,0cyclictest0-21swapper/009:48:260
234529950,0cyclictest0-21swapper/009:28:150
179950,0migration/119146-21kthreadcore09:26:281
169950,0watchdog/18900-21kthreadcore11:36:321
169950,0watchdog/10-21swapper/107:06:321
149950,0migration/032210-21idleruntime10:36:260
149950,0migration/032058-21fschecks_time07:56:270
149950,0migration/031415-21kthreadcore11:31:280
149950,0migration/030570-21kthreadcore09:31:300
149950,0migration/029930-21sh10:33:150
149950,0migration/029325-21kthreadcore11:11:300
149950,0migration/028978-21kthreadcore10:51:280
149950,0migration/028425-21fschecks_time07:31:270
149950,0migration/027828-21unixbench_singl10:11:400
149950,0migration/026214-21kthreadcore06:21:300
149950,0migration/025701-21kthreadcore09:11:290
149950,0migration/023157-21kthreadcore07:26:310
149950,0migration/02170-21kthreadcore06:26:310
149950,0migration/017880-21sh11:21:350
149950,0migration/017543-21sh10:01:440
149950,0migration/017339-21kthreadcore10:26:280
149950,0migration/016696-21kthreadcore08:26:300
149950,0migration/013917-21kthreadcore09:41:300
149950,0migration/013230-21latency10:41:300
149950,0migration/01283-21kthreadcore08:36:300
149950,0migration/011041-21kthreadcore10:21:300
149950,0migration/010318-21kthreadcore08:41:310
8315040,0irq/26-0000:00:0-21swapper/209:26:232
8315040,0irq/26-0000:00:0-21swapper/207:31:122
29935040,0irq/33-eth0-tx-19490-21ssh10:07:012
29935040,0irq/33-eth0-tx-0-21swapper/211:21:542
29935040,0irq/33-eth0-tx-0-21swapper/210:46:402
29935040,0irq/33-eth0-tx-0-21swapper/210:26:312
29935040,0irq/33-eth0-tx-0-21swapper/210:21:162
29935040,0irq/33-eth0-tx-0-21swapper/209:56:282
29935040,0irq/33-eth0-tx-0-21swapper/209:46:222
29935040,0irq/33-eth0-tx-0-21swapper/209:11:162
29935040,0irq/33-eth0-tx-0-21swapper/208:46:262
29935040,0irq/33-eth0-tx-0-21swapper/208:37:232
29935040,0irq/33-eth0-tx-0-21swapper/007:51:390
259940,0migration/25403-21kthreadcore08:21:342
259940,0migration/231764-21fschecks_count10:36:242
259940,0migration/23046-21fschecks_count07:36:262
259940,0migration/227115-21kthreadcore08:51:312
259940,0migration/226397-21latency_hist11:11:112
259940,0migration/22371-21kthreadcore07:56:312
259940,0migration/22171-21kthreadcore08:56:312
259940,0migration/212981-21aten_r1power_cu06:36:232
249940,0watchdog/23564-21lldpd09:21:122
249940,0watchdog/224710-21ssh09:31:122
249940,0watchdog/22370-21cpu06:51:242
249940,0watchdog/223524-21ssh10:31:242
249940,0watchdog/20-21swapper/211:16:282
249940,0watchdog/20-21swapper/206:46:242
234649943,0cyclictest3599-21lldpd10:01:552
234649943,0cyclictest3599-21lldpd08:17:132
234649943,0cyclictest3599-21lldpd07:51:522
234649943,0cyclictest3599-21lldpd07:43:222
234649943,0cyclictest3599-21lldpd07:06:212
234649943,0cyclictest32756-21awk09:16:302
234649943,0cyclictest30688-21awk10:16:282
234649943,0cyclictest12984-21sadc07:21:122
234649940,0cyclictest2539-21diskmemload08:32:482
234589944,0cyclictest0-21swapper/106:31:151
234589943,0cyclictest3599-21lldpd08:20:141
234589943,0cyclictest3599-21lldpd06:28:061
234589943,0cyclictest3262-21irqbalance09:56:261
234589943,0cyclictest3262-21irqbalance09:37:061
234589943,0cyclictest3262-21irqbalance09:21:461
234589943,0cyclictest3262-21irqbalance09:16:161
234589943,0cyclictest3262-21irqbalance07:11:261
234589943,0cyclictest3262-21irqbalance07:01:361
234589943,0cyclictest23917-21awk10:31:261
234589943,0cyclictest23871-21snmpd10:56:121
234589943,0cyclictest23871-21snmpd10:16:271
234589943,0cyclictest23871-21snmpd10:11:271
234589943,0cyclictest23871-21snmpd10:08:121
234589943,0cyclictest23871-21snmpd07:56:411
234589943,0cyclictest23871-21snmpd07:52:261
234589943,0cyclictest23871-21snmpd07:46:261
234589943,0cyclictest23871-21snmpd07:21:261
234589943,0cyclictest21209-21awk06:41:301
234589943,0cyclictest19058-21awk09:46:301
234589943,0cyclictest14051-21awk06:36:281
234589943,0cyclictest12578-21awk11:21:261
234529944,0cyclictest0-21swapper/011:06:190
234529943,0cyclictest7223-21awk09:01:280
234529943,0cyclictest5722-21munin-node06:31:320
234529943,0cyclictest3599-21lldpd08:56:280
234529943,0cyclictest3599-21lldpd07:46:220
234529943,0cyclictest3262-21irqbalance11:27:460
234529943,0cyclictest3262-21irqbalance10:17:360
234529943,0cyclictest3262-21irqbalance08:11:560
234529943,0cyclictest3262-21irqbalance08:06:160
234529943,0cyclictest3262-21irqbalance08:01:160
234529943,0cyclictest3262-21irqbalance06:51:360
234529943,0cyclictest3262-21irqbalance06:46:460
234529943,0cyclictest17768-21awk07:01:290
234529943,0cyclictest10801-21awk07:41:280
179940,0migration/18513-21irqcore10:41:281
179940,0migration/17149-21kthreadcore07:36:301
179940,0migration/17039-21fschecks_time08:01:301
179940,0migration/15982-21latency_hist07:16:121
179940,0migration/15443-21kthreadcore08:21:341
179940,0migration/14744-21kthreadcore06:51:291
179940,0migration/130628-21kthreadcore07:31:311
179940,0migration/128720-21kthreadcore09:51:291
179940,0migration/125797-21idleruntime09:31:261
179940,0migration/12050-21kthreadcore08:56:311
179940,0migration/120422-21taskset07:26:121
179940,0migration/119588-21kthreadcore09:06:301
179940,0migration/118554-21kthreadcore11:41:311
179940,0migration/116221-21sh11:02:391
179940,0migration/116200-21sh09:41:581
179940,0migration/115008-21kthreadcore08:26:301
179940,0migration/11409-21kthreadcore08:36:301
179940,0migration/111037-21taskset11:20:421
169940,0watchdog/17286-21if_eth009:01:281
169940,0watchdog/120431-21port_sshpriv10:26:321
169940,0watchdog/117735-21ssh11:06:161
169940,0watchdog/10-21swapper/108:06:281
149940,0migration/029965-21kthreadcore08:16:300
149940,0migration/02983-21fschecks_count07:36:260
149940,0migration/029015-21kthreadcore09:51:290
149940,0migration/028921-21sh08:53:120
149940,0migration/020369-21fschecks_count08:31:260
149940,0migration/019007-21latency07:21:320
259930,0migration/215091-21kthreadcore06:56:282
249930,0watchdog/216774-21aten2.4-expect07:01:242
249930,0watchdog/20-21swapper/206:31:362
234649932,0cyclictest31932-21awk06:26:282
234649932,0cyclictest1626-21cut07:11:302
234649930,0cyclictest27268-21latency_hist06:46:122
234649930,0cyclictest0-21swapper/208:03:582
234649930,0cyclictest0-21swapper/207:26:362
234589932,0cyclictest3599-21lldpd10:36:131
234589932,0cyclictest2539-21diskmemload10:21:241
234589932,0cyclictest24829-21awk06:21:281
234589932,0cyclictest23871-21snmpd08:12:111
234589930,0cyclictest16765-21ssh10:46:281
234529933,0cyclictest8889-21awk09:21:310
234529933,0cyclictest8285-21kworker/0:209:57:100
234529933,0cyclictest8285-21kworker/0:209:16:220
234529933,0cyclictest8285-21kworker/0:208:46:120
234529933,0cyclictest17155-21kworker/0:010:46:220
234529933,0cyclictest17155-21kworker/0:010:06:280
234529932,0cyclictest9323-21aten2.4-expect11:01:240
234529930,0cyclictest1262-21diskstats09:36:250
234529930,0cyclictest0-21swapper/007:06:260
169930,0watchdog/19897-21aten_r1power_po07:41:241
169930,0watchdog/118647-21kthreadcore08:46:321
169930,0watchdog/10-21swapper/106:56:281
169930,0watchdog/10-21swapper/106:46:281
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional