You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-30 - 16:12
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Thu Oct 30, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
449524334,0sleep20-21swapper/206:18:052
3021623929,0sleep30-21swapper/306:16:263
440323526,0sleep10-21swapper/106:17:001
475723430,0sleep00-21swapper/006:21:070
329970,0watchdog/35629-21cpuspeed_turbos09:16:323
29925070,0irq/32-eth0-rx-19072-21fschecks_time08:26:313
29925070,0irq/32-eth0-rx-16509-21aten2.4-expect10:01:263
48409965,0cyclictest1887-21port_sshpriv08:51:382
48409960,0cyclictest0-21swapper/209:46:302
48399960,0cyclictest0-21swapper/111:17:421
339960,0migration/31083-21munin-node10:11:273
329960,0watchdog/30-21swapper/311:42:113
329960,0watchdog/30-21swapper/311:38:113
329960,0watchdog/30-21swapper/311:26:273
329960,0watchdog/30-21swapper/311:16:393
329960,0watchdog/30-21swapper/311:06:513
329960,0watchdog/30-21swapper/311:01:393
329960,0watchdog/30-21swapper/310:53:273
329960,0watchdog/30-21swapper/310:34:353
329960,0watchdog/30-21swapper/310:30:153
329960,0watchdog/30-21swapper/310:30:153
329960,0watchdog/30-21swapper/310:21:313
329960,0watchdog/30-21swapper/310:17:553
329960,0watchdog/30-21swapper/309:51:163
329960,0watchdog/30-21swapper/309:47:123
329960,0watchdog/30-21swapper/309:26:163
329960,0watchdog/30-21swapper/309:22:003
329960,0watchdog/30-21swapper/308:45:163
329960,0watchdog/30-21swapper/308:39:083
329960,0watchdog/30-21swapper/308:34:403
329960,0watchdog/30-21swapper/308:25:363
329960,0watchdog/30-21swapper/308:11:523
329960,0watchdog/30-21swapper/308:11:083
329960,0watchdog/30-21swapper/307:26:283
329960,0watchdog/30-21swapper/307:00:483
29935060,0irq/33-eth0-tx-0-21swapper/007:24:070
29935060,0irq/33-eth0-tx-0-21swapper/007:16:350
29925060,0irq/32-eth0-rx-6558-21kthreadcore11:46:353
29925060,0irq/32-eth0-rx-13846-21irqstats06:26:343
29925060,0irq/32-eth0-rx-0-21swapper/311:32:193
29925060,0irq/32-eth0-rx-0-21swapper/311:25:013
29925060,0irq/32-eth0-rx-0-21swapper/310:56:213
29925060,0irq/32-eth0-rx-0-21swapper/310:46:163
29925060,0irq/32-eth0-rx-0-21swapper/310:41:193
29925060,0irq/32-eth0-rx-0-21swapper/310:39:263
29925060,0irq/32-eth0-rx-0-21swapper/310:08:253
29925060,0irq/32-eth0-rx-0-21swapper/309:41:373
29925060,0irq/32-eth0-rx-0-21swapper/309:38:393
29925060,0irq/32-eth0-rx-0-21swapper/309:08:183
29925060,0irq/32-eth0-rx-0-21swapper/309:02:093
29925060,0irq/32-eth0-rx-0-21swapper/308:56:583
29925060,0irq/32-eth0-rx-0-21swapper/308:53:493
29925060,0irq/32-eth0-rx-0-21swapper/308:48:513
149960,0migration/018607-21sh10:39:560
8315050,0irq/26-0000:00:5208-21cat09:16:152
48409955,0cyclictest0-21swapper/211:48:512
48409950,0cyclictest0-21swapper/211:11:382
48409950,0cyclictest0-21swapper/210:53:302
48399955,0cyclictest0-21swapper/111:00:401
48399955,0cyclictest0-21swapper/108:47:181
48399954,0cyclictest1435-21sed07:01:351
48399950,0cyclictest0-21swapper/111:25:441
48399950,0cyclictest0-21swapper/110:47:261
48399950,0cyclictest0-21swapper/108:40:121
48389955,0cyclictest0-21swapper/011:26:260
48389955,0cyclictest0-21swapper/010:56:290
48389955,0cyclictest0-21swapper/010:46:330
48389955,0cyclictest0-21swapper/010:21:570
48389954,0cyclictest0-21swapper/011:13:310
48389950,0cyclictest0-21swapper/011:05:000
48389950,0cyclictest0-21swapper/009:39:050
48389950,0cyclictest0-21swapper/008:48:280
4150,0ktimersoftd/00-21swapper/010:16:170
339950,0migration/332194-21munin-node07:46:333
339950,0migration/331542-21kthreadcore07:21:333
339950,0migration/331036-21kthreadcore06:36:353
329950,0watchdog/30-21swapper/309:32:003
329950,0watchdog/30-21swapper/308:20:443
329950,0watchdog/30-21swapper/308:03:403
329950,0watchdog/30-21swapper/307:52:003
329950,0watchdog/30-21swapper/307:42:083
329950,0watchdog/30-21swapper/307:40:043
329950,0watchdog/30-21swapper/307:32:083
329950,0watchdog/30-21swapper/307:16:523
329950,0watchdog/30-21swapper/307:13:203
329950,0watchdog/30-21swapper/307:08:123
329950,0watchdog/30-21swapper/307:01:483
329950,0watchdog/30-21swapper/306:51:283
329950,0watchdog/30-21swapper/306:46:283
329950,0watchdog/30-21swapper/306:41:243
329950,0watchdog/30-21swapper/306:31:283
329950,0watchdog/30-21swapper/306:22:203
29935050,0irq/33-eth0-tx-23871-21snmpd07:54:120
29935050,0irq/33-eth0-tx-18260-21munin-node08:26:352
29935050,0irq/33-eth0-tx-17874-21ssh08:25:562
29935050,0irq/33-eth0-tx-0-21swapper/007:46:300
29935050,0irq/33-eth0-tx-0-21swapper/007:31:300
29925050,0irq/32-eth0-rx-0-21swapper/311:12:053
29925050,0irq/32-eth0-rx-0-21swapper/309:56:273
29925050,0irq/32-eth0-rx-0-21swapper/309:11:543
29925050,0irq/32-eth0-rx-0-21swapper/307:56:313
27150,0ktimersoftd/20-21swapper/210:29:182
27150,0ktimersoftd/20-21swapper/210:29:182
259950,0migration/23734-21hddtemp_smartct08:16:312
259950,0migration/222182-21ntp_offset11:36:362
249950,0watchdog/220993-21ssh09:23:122
249950,0watchdog/213706-21gltestperf07:56:322
179950,0migration/17821-21munin-node07:26:371
179950,0migration/17404-21kthreadcore09:16:371
179950,0migration/119817-21cpuspeed_turbos06:31:291
179950,0migration/116389-21sh10:17:051
179950,0migration/113899-21latency09:56:351
169950,0watchdog/125207-21ssh08:27:241
169950,0watchdog/112310-21grep07:31:361
149950,0migration/09556-21fschecks_time06:46:310
149950,0migration/07984-21sh09:56:260
149950,0migration/07704-21df_abs11:31:280
149950,0migration/06862-21kthreadcore06:41:350
149950,0migration/06668-21kthreadcore08:56:330
149950,0migration/05618-21kthreadcore10:51:320
149950,0migration/04330-21kthreadcore09:51:350
149950,0migration/04057-21if_eth008:36:320
149950,0migration/032063-21sh11:43:100
149950,0migration/031688-21sh10:06:470
149950,0migration/031614-21gltestperf09:31:300
149950,0migration/030624-21cpuspeed_turbos07:01:290
149950,0migration/029410-21kernelversion08:11:330
149950,0migration/026491-21kthreadcore09:46:370
149950,0migration/024877-21kthreadcore06:56:330
149950,0migration/02435-21fschecks_time07:26:290
149950,0migration/023510-21sh10:02:460
149950,0migration/02181-21idleruntime11:46:320
149950,0migration/021075-21kthreadcore11:36:340
149950,0migration/015801-21kthreadcore07:56:360
8315040,0irq/26-0000:00:0-21swapper/208:06:152
48409944,0cyclictest0-21swapper/211:41:172
48409943,0cyclictest9948-21awk06:46:332
48409943,0cyclictest8447-21awk11:31:312
48409943,0cyclictest6120-21awk06:21:322
48409943,0cyclictest3599-21lldpd10:31:272
48409943,0cyclictest3599-21lldpd07:49:032
48409943,0cyclictest3599-21lldpd07:41:522
48409943,0cyclictest3599-21lldpd07:31:512
48409943,0cyclictest3599-21lldpd07:30:212
48409943,0cyclictest3599-21lldpd06:53:432
48409943,0cyclictest3599-21lldpd06:42:122
48409943,0cyclictest28917-21awk08:11:312
48409943,0cyclictest23871-21snmpd09:56:182
48409943,0cyclictest23871-21snmpd07:06:482
48409943,0cyclictest23871-21snmpd06:36:472
48409943,0cyclictest21993-21awk11:01:342
48409943,0cyclictest17351-21awk07:36:322
48409940,0cyclictest0-21swapper/210:38:032
48399944,0cyclictest9758-21diskmemload09:21:591
48399944,0cyclictest0-21swapper/107:16:201
48399943,0cyclictest6246-21awk06:21:321
48399943,0cyclictest592-21awk09:51:321
48399943,0cyclictest3823-21awk08:16:321
48399943,0cyclictest3599-21lldpd10:04:511
48399943,0cyclictest3599-21lldpd07:54:031
48399943,0cyclictest3599-21lldpd07:15:451
48399943,0cyclictest3599-21lldpd07:08:451
48399943,0cyclictest3599-21lldpd06:28:401
48399943,0cyclictest27779-21awk06:36:331
48399943,0cyclictest2602-21awk06:41:321
48399943,0cyclictest24456-21awk07:41:311
48399943,0cyclictest24273-21awk06:56:311
48399943,0cyclictest23871-21snmpd10:36:491
48399943,0cyclictest23871-21snmpd09:39:481
48399943,0cyclictest23871-21snmpd09:34:331
48399943,0cyclictest23871-21snmpd08:09:331
48399943,0cyclictest23871-21snmpd07:23:331
48399943,0cyclictest20553-21awk10:41:361
48399943,0cyclictest19101-21awk10:21:321
48389943,0cyclictest6589-21awk09:16:350
48389943,0cyclictest3599-21lldpd06:27:410
48389943,0cyclictest3262-21irqbalance10:42:490
48389943,0cyclictest3262-21irqbalance09:41:190
48389943,0cyclictest3262-21irqbalance09:29:090
48389943,0cyclictest3262-21irqbalance08:26:200
48389943,0cyclictest3262-21irqbalance08:23:000
48389943,0cyclictest3262-21irqbalance08:06:400
48389943,0cyclictest3262-21irqbalance08:01:200
48389943,0cyclictest3262-21irqbalance07:44:300
48389943,0cyclictest3262-21irqbalance07:36:300
48389943,0cyclictest3262-21irqbalance07:11:200
48389943,0cyclictest3262-21irqbalance06:51:300
48389943,0cyclictest3262-21irqbalance06:36:500
48389943,0cyclictest3262-21irqbalance06:31:500
48389943,0cyclictest23871-21snmpd09:12:330
48389943,0cyclictest23871-21snmpd08:33:180
48389943,0cyclictest12569-21awk08:41:310
29935040,0irq/33-eth0-tx-9758-21diskmemload09:33:082
29935040,0irq/33-eth0-tx-4753-21pmu-power11:26:362
29935040,0irq/33-eth0-tx-17972-21munin-node10:21:282
29935040,0irq/33-eth0-tx-14208-21ssh09:38:562
29935040,0irq/33-eth0-tx-0-21swapper/211:21:422
29935040,0irq/33-eth0-tx-0-21swapper/211:06:282
29935040,0irq/33-eth0-tx-0-21swapper/210:42:222
29935040,0irq/33-eth0-tx-0-21swapper/210:12:402
29935040,0irq/33-eth0-tx-0-21swapper/210:06:192
29935040,0irq/33-eth0-tx-0-21swapper/209:41:382
29935040,0irq/33-eth0-tx-0-21swapper/209:17:242
29935040,0irq/33-eth0-tx-0-21swapper/208:56:302
29935040,0irq/33-eth0-tx-0-21swapper/208:47:162
29935040,0irq/33-eth0-tx-0-21swapper/208:41:272
29935040,0irq/33-eth0-tx-0-21swapper/208:36:362
29935040,0irq/33-eth0-tx-0-21swapper/208:32:052
259940,0migration/27559-21kthreadcore07:51:332
259940,0migration/2364-21fschecks_time09:51:302
259940,0migration/227855-21aten_r1power_fr10:46:282
259940,0migration/227031-21aten_r1power_vo07:21:272
259940,0migration/221821-21kthreadcore07:16:352
259940,0migration/221634-21fschecks_time09:06:302
259940,0migration/217389-21kthreadcore07:11:342
259940,0migration/212376-21sh10:56:262
249940,0watchdog/23901-21missed_timers07:01:362
249940,0watchdog/229956-21ssh09:29:562
249940,0watchdog/219212-21kthreadcore11:16:352
249940,0watchdog/218780-21pmu-power09:01:402
249940,0watchdog/217714-21ps10:01:312
249940,0watchdog/215912-21smartctl10:16:392
179940,0migration/17857-21fschecks_count11:31:291
179940,0migration/16257-21kthreadcore11:46:351
179940,0migration/14936-21kthreadcore10:31:371
179940,0migration/14560-21df_abs08:56:281
179940,0migration/1414-21kthreadcore11:26:331
179940,0migration/128696-21kthreadcore10:26:331
179940,0migration/128696-21kthreadcore10:26:331
179940,0migration/128524-21kthreadcore08:31:321
179940,0migration/123226-21kthreadcore11:01:361
179940,0migration/122025-21if_err_eth1.309:06:321
179940,0migration/11812-21fschecks_time10:11:291
179940,0migration/117475-21irqcore11:36:331
179940,0migration/115622-21munin-node09:41:301
179940,0migration/111612-21kthreadcore08:21:371
169940,0watchdog/126718-21kthreadcore10:06:311
169940,0watchdog/125035-21munin-node11:41:271
169940,0watchdog/124685-21if_eth1.309:46:361
169940,0watchdog/123871-21snmpd07:48:481
169940,0watchdog/122133-21ssh09:26:201
169940,0watchdog/11568-21kthreadcore09:11:361
169940,0watchdog/113707-21gltestperf07:56:321
169940,0watchdog/113361-21perl09:01:321
169940,0watchdog/10-21swapper/111:14:111
149940,0migration/05200-21aten_r1power_vo06:21:280
149940,0migration/04265-21irqrtprio08:16:340
149940,0migration/030993-21kthreadcore08:51:340
149940,0migration/026270-21sh10:26:220
149940,0migration/026270-21sh10:26:220
149940,0migration/026044-21kthreadcore09:06:350
149940,0migration/010684-21kthreadcore07:06:320
8315030,0irq/26-0000:00:0-21swapper/208:01:272
48409933,0cyclictest3599-21lldpd06:57:132
48399932,0cyclictest9058-21expr06:46:291
48399932,0cyclictest30373-21awk11:06:311
48399932,0cyclictest23871-21snmpd08:11:331
48399932,0cyclictest22417-21sensors07:36:381
48399932,0cyclictest12697-21awk08:41:321
48389933,0cyclictest0-21swapper/011:19:590
48389932,0cyclictest3002-21grep10:31:320
48389932,0cyclictest26396-21kthreadcore11:21:350
48389932,0cyclictest2079-21awk10:11:310
48389932,0cyclictest14987-21awk09:21:310
48389932,0cyclictest13636-21awk09:01:340
48389931,0cyclictest0-21swapper/011:06:330
259930,0migration/222420-21kthreadcore06:31:342
249930,0watchdog/20-21swapper/206:26:282
19130,0ktimersoftd/10-21swapper/108:03:001
179930,0migration/116473-21fschecks_count06:51:291
169930,0watchdog/19758-21diskmemload10:52:551
169930,0watchdog/10-21swapper/108:53:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional