You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-24 - 19:09
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2.osadl.org (updated Tue Jun 24, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
478628373,0sleep10-21swapper/106:25:101
416424845,0sleep20-21swapper/206:24:272
5195994444,0cyclictest3974-21md0_resync08:52:410
5197993737,0cyclictest3973-21md1_resync08:52:412
502622824,0sleep00-21swapper/006:27:570
473522723,0sleep30-21swapper/306:24:403
339980,0migration/314778-21kernelversion10:39:243
8315070,0irq/26-0000:00:781-21cut10:29:313
8315070,0irq/26-0000:00:6210-21rm08:56:283
8315070,0irq/26-0000:00:31023-21unixbench-2d07:04:313
8315070,0irq/26-0000:00:0-21swapper/310:09:233
329970,0watchdog/30-21swapper/310:14:203
329970,0watchdog/30-21swapper/309:54:243
29925070,0irq/32-eth0-rx-6264-21irqcore10:34:223
29925070,0irq/32-eth0-rx-18922-21ethtool09:24:213
8315060,0irq/26-0000:00:826-21cat09:14:163
8315060,0irq/26-0000:00:22980-21diskstats08:49:173
8315060,0irq/26-0000:00:18906-21kthreadcore08:04:233
8315060,0irq/26-0000:00:15600-21irqstats08:44:233
8315060,0irq/26-0000:00:122762sleep306:30:523
8315060,0irq/26-0000:00:10194-21grep08:59:223
8315060,0irq/26-0000:00:0-21swapper/311:54:213
8315060,0irq/26-0000:00:0-21swapper/311:49:243
8315060,0irq/26-0000:00:0-21swapper/311:39:223
8315060,0irq/26-0000:00:0-21swapper/310:54:193
8315060,0irq/26-0000:00:0-21swapper/310:08:463
8315060,0irq/26-0000:00:0-21swapper/309:44:303
8315060,0irq/26-0000:00:0-21swapper/309:34:213
8315060,0irq/26-0000:00:0-21swapper/309:19:293
8315060,0irq/26-0000:00:0-21swapper/309:04:553
8315060,0irq/26-0000:00:0-21swapper/308:39:193
8315060,0irq/26-0000:00:0-21swapper/307:39:243
8315060,0irq/26-0000:00:0-21swapper/307:00:093
8315060,0irq/26-0000:00:0-21swapper/307:00:093
51979966,0cyclictest0-21swapper/208:39:302
51979966,0cyclictest0-21swapper/208:29:282
51979960,0cyclictest0-21swapper/206:48:022
51969966,0cyclictest0-21swapper/106:34:491
51969960,0cyclictest0-21swapper/108:33:421
4160,0ktimersoftd/03974-21md0_resync09:55:030
339960,0migration/331238-21kthreadcore11:44:203
339960,0migration/327530-21latency07:24:233
339960,0migration/323670-21unixbench-2d11:19:313
339960,0migration/318770-21sh10:02:103
329960,0watchdog/30-21swapper/311:35:033
329960,0watchdog/30-21swapper/311:31:113
329960,0watchdog/30-21swapper/311:24:523
329960,0watchdog/30-21swapper/311:05:203
329960,0watchdog/30-21swapper/310:49:163
329960,0watchdog/30-21swapper/310:44:043
329960,0watchdog/30-21swapper/310:24:283
329960,0watchdog/30-21swapper/309:49:043
329960,0watchdog/30-21swapper/309:39:043
329960,0watchdog/30-21swapper/309:30:163
329960,0watchdog/30-21swapper/308:29:243
329960,0watchdog/30-21swapper/308:26:083
329960,0watchdog/30-21swapper/307:59:043
329960,0watchdog/30-21swapper/307:53:163
329960,0watchdog/30-21swapper/307:29:403
329960,0watchdog/30-21swapper/307:22:363
329960,0watchdog/30-21swapper/307:14:243
329960,0watchdog/30-21swapper/306:50:323
329960,0watchdog/30-21swapper/306:45:123
329960,0watchdog/30-21swapper/306:43:323
329960,0watchdog/30-21swapper/306:35:283
29925060,0irq/32-eth0-rx-0-21swapper/311:09:063
29925060,0irq/32-eth0-rx-0-21swapper/309:12:383
29925060,0irq/32-eth0-rx-0-21swapper/308:34:103
29925060,0irq/32-eth0-rx-0-21swapper/306:54:063
19160,0ktimersoftd/10-21swapper/108:26:081
149960,0migration/013284-21switchtime07:14:300
149960,0migration/012214-21latency_hist11:54:030
8315050,0irq/26-0000:00:5877-21fschecks_count08:19:193
8315050,0irq/26-0000:00:0-21swapper/307:54:173
8315050,0irq/26-0000:00:0-21swapper/307:44:213
8315050,0irq/26-0000:00:0-21swapper/307:34:043
8315050,0irq/26-0000:00:0-21swapper/307:09:043
51979955,0cyclictest0-21swapper/211:34:112
51979955,0cyclictest0-21swapper/211:31:112
51979955,0cyclictest0-21swapper/208:34:272
51979955,0cyclictest0-21swapper/208:07:302
51979955,0cyclictest0-21swapper/207:54:532
51969955,0cyclictest0-21swapper/111:34:221
51969955,0cyclictest0-21swapper/108:36:131
51969955,0cyclictest0-21swapper/106:50:161
51969955,0cyclictest0-21swapper/106:48:421
51969950,0cyclictest0-21swapper/110:55:541
51969950,0cyclictest0-21swapper/110:53:511
51969950,0cyclictest0-21swapper/107:29:321
51959950,0cyclictest0-21swapper/011:24:550
4150,0ktimersoftd/00-21swapper/010:25:290
329950,0watchdog/30-21swapper/311:16:003
329950,0watchdog/30-21swapper/310:59:043
329950,0watchdog/30-21swapper/310:19:203
329950,0watchdog/30-21swapper/308:14:203
329950,0watchdog/30-21swapper/308:10:403
29935050,0irq/33-eth0-tx-3057-21perl09:34:202
29935050,0irq/33-eth0-tx-27248-21munin-node09:49:182
29935050,0irq/33-eth0-tx-23069-21missed_timers11:19:242
29935050,0irq/33-eth0-tx-18915-21munin-node11:39:312
29935050,0irq/33-eth0-tx-17780-21munin-node09:24:182
29935050,0irq/33-eth0-tx-15079-21munin-node10:59:172
29935050,0irq/33-eth0-tx-14045-21munin-node08:44:252
29935050,0irq/33-eth0-tx-0-21swapper/210:52:342
29935050,0irq/33-eth0-tx-0-21swapper/210:29:092
29935050,0irq/33-eth0-tx-0-21swapper/209:13:482
29935050,0irq/33-eth0-tx-0-21swapper/208:57:262
29935050,0irq/33-eth0-tx-0-21swapper/008:29:060
27150,0ktimersoftd/20-21swapper/209:19:042
27150,0ktimersoftd/20-21swapper/207:49:412
27150,0ktimersoftd/20-21swapper/206:35:062
259950,0migration/26610-21runrttasks07:13:532
259950,0migration/229538-21timerandwakeup08:09:292
259950,0migration/228704-21sh07:29:032
259950,0migration/228278-21sh10:28:052
259950,0migration/214882-21kthreadcore09:59:222
259950,0migration/213818-21sh10:56:002
259950,0migration/210533-21kthreadcore08:59:222
19150,0ktimersoftd/10-21swapper/110:25:501
19150,0ktimersoftd/10-21swapper/108:51:211
179950,0migration/19606-21kthreadcore11:49:211
179950,0migration/131990-21gltestperf08:54:191
179950,0migration/131755-21kthreadcore10:29:301
179950,0migration/126603-21timerwakeupswit06:39:331
179950,0migration/122766-21taskset08:08:221
179950,0migration/121060-21sh08:44:581
179950,0migration/110776-21sh10:15:071
179950,0migration/110500-21irqcore09:19:271
169950,0watchdog/114117-21cut10:19:241
149950,0migration/07950-21kthreadcore08:19:230
149950,0migration/07424-21kthreadcore10:14:240
149950,0migration/06536-21kthreadcore09:34:240
149950,0migration/0542-21sh09:29:470
149950,0migration/032327-21aten_r1power_fr11:09:140
149950,0migration/032289-21sh09:12:430
149950,0migration/031005-3199-raid-check08:54:070
149950,0migration/030649-21kthreadcore07:49:250
149950,0migration/03037-21kthreadcore07:09:230
149950,0migration/020230-21kthreadcore06:59:220
149950,0migration/020230-21kthreadcore06:59:220
149950,0migration/019171-21switchtime06:34:340
149950,0migration/018364-21taskset09:43:200
149950,0migration/018089-21ksmtuned07:40:020
149950,0migration/017080-21kthreadcore07:19:280
149950,0migration/016638-21fschecks_time09:04:210
149950,0migration/016577-21kthreadcore08:44:230
149950,0migration/015565-21latency_hist08:04:030
149950,0migration/014286-21taskset08:59:430
149950,0migration/013879-21sh10:56:100
149950,0migration/01386-21munin-node06:44:330
149950,0migration/012582-21idleruntime-cro06:34:040
149950,0migration/011690-21munin-node10:34:320
149950,0migration/011100-21turbostat07:39:030
51979944,0cyclictest0-21swapper/206:56:232
51979943,0cyclictest6593-21awk06:29:272
51979943,0cyclictest29993-21awk07:29:192
51979943,0cyclictest25632-21awk07:04:212
51979943,0cyclictest23871-21snmpd11:45:232
51979943,0cyclictest23871-21snmpd10:44:072
51979943,0cyclictest23871-21snmpd10:39:072
51979943,0cyclictest23871-21snmpd09:55:372
51979943,0cyclictest23871-21snmpd09:44:072
51979943,0cyclictest23871-21snmpd07:46:362
51979943,0cyclictest23871-21snmpd07:17:062
51979940,0cyclictest0-21swapper/209:04:212
51969944,0cyclictest3599-21lldpd09:09:241
51969944,0cyclictest0-21swapper/111:00:001
51969944,0cyclictest0-21swapper/107:49:521
51969943,0cyclictest9477-21awk07:59:201
51969943,0cyclictest6530-21awk08:39:211
51969943,0cyclictest6496-21awk08:19:211
51969943,0cyclictest3599-21lldpd10:40:361
51969943,0cyclictest3599-21lldpd09:05:221
51969943,0cyclictest3599-21lldpd07:56:271
51969943,0cyclictest3599-21lldpd07:45:261
51969943,0cyclictest24139-21awk08:09:191
51969943,0cyclictest18845-21awk09:24:211
51969943,0cyclictest18062-21awk11:19:201
51969943,0cyclictest12336-21awk09:59:191
51969943,0cyclictest11710-21awk09:39:231
51969940,0cyclictest0-21swapper/108:15:431
51959944,0cyclictest9441-21kworker/0:111:54:120
51959944,0cyclictest9441-21kworker/0:111:44:200
51959944,0cyclictest9441-21kworker/0:111:39:360
51959944,0cyclictest9441-21kworker/0:111:34:080
51959944,0cyclictest9441-21kworker/0:111:29:080
51959944,0cyclictest9441-21kworker/0:111:19:240
51959944,0cyclictest8215-21kworker/0:108:34:260
51959944,0cyclictest8215-21kworker/0:108:09:060
51959944,0cyclictest8215-21kworker/0:107:54:260
51959944,0cyclictest8215-21kworker/0:107:44:040
51959944,0cyclictest8215-21kworker/0:107:04:060
51959944,0cyclictest8215-21kworker/0:106:56:140
51959944,0cyclictest8215-21kworker/0:106:49:100
51959944,0cyclictest13728-21kworker/0:211:04:040
51959944,0cyclictest13728-21kworker/0:210:59:040
51959944,0cyclictest13728-21kworker/0:210:49:340
51959944,0cyclictest13728-21kworker/0:210:44:040
51959944,0cyclictest13728-21kworker/0:210:39:100
51959944,0cyclictest13728-21kworker/0:210:19:140
51959944,0cyclictest13728-21kworker/0:210:09:120
51959944,0cyclictest13728-21kworker/0:210:04:060
51959944,0cyclictest13728-21kworker/0:209:24:040
51959944,0cyclictest13728-21kworker/0:209:19:540
51959944,0cyclictest13728-21kworker/0:209:14:140
51959943,0cyclictest3262-21irqbalance11:14:070
51959943,0cyclictest3262-21irqbalance09:44:080
51959943,0cyclictest3262-21irqbalance08:39:380
51959943,0cyclictest3262-21irqbalance08:24:080
51959943,0cyclictest3262-21irqbalance08:04:080
51959943,0cyclictest30073-21awk07:29:190
51959943,0cyclictest28203-21cut09:49:190
51959942,0cyclictest31479-21awk08:14:220
29935040,0irq/33-eth0-tx-21686-21telnet07:24:160
29935040,0irq/33-eth0-tx-0-21swapper/211:54:192
29935040,0irq/33-eth0-tx-0-21swapper/211:49:092
29935040,0irq/33-eth0-tx-0-21swapper/211:24:112
29935040,0irq/33-eth0-tx-0-21swapper/211:04:252
29935040,0irq/33-eth0-tx-0-21swapper/210:19:502
29935040,0irq/33-eth0-tx-0-21swapper/210:14:042
29935040,0irq/33-eth0-tx-0-21swapper/210:09:362
29935040,0irq/33-eth0-tx-0-21swapper/209:41:312
29935040,0irq/33-eth0-tx-0-21swapper/209:29:192
29935040,0irq/33-eth0-tx-0-21swapper/209:14:312
259940,0migration/25803-21kthreadcore07:34:222
259940,0migration/25398-21fschecks_count10:34:192
259940,0migration/224826-21kthreadcore06:39:252
259940,0migration/22275-21latency_hist06:49:042
259940,0migration/219463-21latency_hist10:04:032
259940,0migration/218940-21ntp_kernel_pll_08:24:252
259940,0migration/213847-21kthreadcore07:59:222
249940,0watchdog/28373-21awk11:14:042
249940,0watchdog/26386-21runrttasks07:20:042
249940,0watchdog/20-21swapper/211:09:242
249940,0watchdog/20-21swapper/207:39:162
179940,0migration/16885-21ksmtuned11:10:031
179940,0migration/130728-21kthreadcore09:49:231
179940,0migration/127819-21kthreadcore11:04:261
179940,0migration/121774-21aten_r1power_vo07:24:161
179940,0migration/12124-21taskset09:34:121
179940,0migration/120430-21fschecks_time10:04:191
179940,0migration/114864-21kthreadcore11:54:211
179940,0migration/112561-21kthreadcore11:14:261
179940,0migration/111040-21taskset07:38:351
179940,0migration/110394-21fschecks_count06:54:201
169940,0watchdog/18605-21ssh11:29:431
169940,0watchdog/14868-21aten_r1power_cu10:34:161
169940,0watchdog/130371-21kthreadcore11:24:241
169940,0watchdog/127939-21sed11:44:151
169940,0watchdog/124917-21munin-node09:44:241
169940,0watchdog/122965-21kthreadcore11:39:271
169940,0watchdog/12065-21md0_raid110:44:241
169940,0watchdog/118796-21kthreadcore07:19:281
149940,0migration/028904-21taskset10:29:220
149940,0migration/020804-21fschecks_time06:39:210
149940,0migration/012045-21fschecks_time09:59:180
51979932,0cyclictest18211-21awk06:59:202
51979932,0cyclictest18211-21awk06:59:202
51979930,0cyclictest5828-21df_abs08:19:182
51969933,0cyclictest3599-21lldpd07:42:261
51969932,0cyclictest6986-21cut07:14:171
51969932,0cyclictest582-21kthreadcore10:09:251
51969932,0cyclictest28123-21cut07:04:231
51969930,0cyclictest1026-21df_inode09:14:171
51969930,0cyclictest0-21swapper/106:59:551
51969930,0cyclictest0-21swapper/106:59:551
249930,0watchdog/21081-21grep08:14:242
179930,0migration/17267-21latency_hist08:59:031
169930,0watchdog/10-21swapper/109:54:241
169930,0watchdog/10-21swapper/109:29:201
169930,0watchdog/10-21swapper/107:10:001
169930,0watchdog/10-21swapper/106:29:281
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional