You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-09 - 06:01
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2.osadl.org (updated Fri May 09, 2025 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
860623829,0sleep20-21swapper/218:31:312
852123728,0sleep30-21swapper/318:30:313
842323626,0sleep10-21swapper/118:29:211
830923525,0sleep00-21swapper/018:28:040
29925080,0irq/32-eth0-rx-0-21swapper/321:20:113
87079977,0cyclictest6805-21awk19:37:080
329970,0watchdog/316169-21cut23:17:133
329970,0watchdog/30-21swapper/321:15:493
29925070,0irq/32-eth0-rx-27318-21awk22:07:143
29925070,0irq/32-eth0-rx-0-21swapper/323:31:423
87099960,0cyclictest0-21swapper/221:27:102
339960,0migration/36591-21sh22:35:073
339960,0migration/327060-21sh20:32:243
339960,0migration/312187-21gltestperf21:42:053
329960,0watchdog/311347-21cut20:02:093
329960,0watchdog/30-21swapper/323:55:453
329960,0watchdog/30-21swapper/323:42:053
329960,0watchdog/30-21swapper/323:15:093
329960,0watchdog/30-21swapper/322:53:213
329960,0watchdog/30-21swapper/322:47:013
329960,0watchdog/30-21swapper/322:28:253
329960,0watchdog/30-21swapper/322:26:253
329960,0watchdog/30-21swapper/322:17:573
329960,0watchdog/30-21swapper/322:12:373
329960,0watchdog/30-21swapper/321:49:413
329960,0watchdog/30-21swapper/321:38:253
329960,0watchdog/30-21swapper/321:34:573
329960,0watchdog/30-21swapper/321:22:413
329960,0watchdog/30-21swapper/321:07:093
329960,0watchdog/30-21swapper/320:42:173
329960,0watchdog/30-21swapper/319:22:093
329960,0watchdog/30-21swapper/319:02:093
29935060,0irq/33-eth0-tx-30809-21munin-node19:32:070
29925060,0irq/32-eth0-rx-26646-21sh23:24:573
29925060,0irq/32-eth0-rx-196302sleep319:44:233
29925060,0irq/32-eth0-rx-0-21swapper/322:38:153
29925060,0irq/32-eth0-rx-0-21swapper/321:27:473
29925060,0irq/32-eth0-rx-0-21swapper/321:03:133
29925060,0irq/32-eth0-rx-0-21swapper/320:57:183
29925060,0irq/32-eth0-rx-0-21swapper/320:47:043
29925060,0irq/32-eth0-rx-0-21swapper/320:39:173
87099954,0cyclictest23871-21snmpd19:47:332
87099954,0cyclictest10207-21grep20:02:062
87099950,0cyclictest0-21swapper/222:46:562
87089955,0cyclictest0-21swapper/123:16:521
87089955,0cyclictest0-21swapper/122:21:411
87089955,0cyclictest0-21swapper/120:45:411
87089950,0cyclictest0-21swapper/120:58:361
87079955,0cyclictest30026-21sensors20:12:130
8315050,0irq/26-0000:00:18597-21sh23:37:262
8315050,0irq/26-0000:00:0-21swapper/318:31:533
339950,0migration/36599-21kthreadcore19:57:093
339950,0migration/33497-21taskset23:32:003
339950,0migration/317918-21kthreadcore20:27:123
339950,0migration/31028-21kthreadcore20:17:123
329950,0watchdog/38706-21cyclictest18:52:253
329950,0watchdog/3846-21nfs_client19:52:133
329950,0watchdog/331787-21hwlatdetect19:32:093
329950,0watchdog/317574-21sh20:07:093
329950,0watchdog/30-21swapper/323:06:533
329950,0watchdog/30-21swapper/323:01:533
329950,0watchdog/30-21swapper/322:57:093
329950,0watchdog/30-21swapper/322:01:573
329950,0watchdog/30-21swapper/321:51:533
329950,0watchdog/30-21swapper/320:51:533
329950,0watchdog/30-21swapper/320:13:013
329950,0watchdog/30-21swapper/319:48:093
329950,0watchdog/30-21swapper/319:37:093
329950,0watchdog/30-21swapper/319:28:533
329950,0watchdog/30-21swapper/319:17:453
329950,0watchdog/30-21swapper/319:14:333
329950,0watchdog/30-21swapper/319:07:053
329950,0watchdog/30-21swapper/318:57:253
329950,0watchdog/30-21swapper/318:42:013
329950,0watchdog/30-21swapper/318:36:573
29935050,0irq/33-eth0-tx-21162-21ssh20:47:512
29935050,0irq/33-eth0-tx-20556-21grep23:22:132
29935050,0irq/33-eth0-tx-17572-21munin-node23:02:122
29935050,0irq/33-eth0-tx-16609-21munin-node20:07:210
29935050,0irq/33-eth0-tx-0-21swapper/223:36:462
29935050,0irq/33-eth0-tx-0-21swapper/223:14:052
29935050,0irq/33-eth0-tx-0-21swapper/222:39:172
29935050,0irq/33-eth0-tx-0-21swapper/222:10:332
29935050,0irq/33-eth0-tx-0-21swapper/222:00:052
29935050,0irq/33-eth0-tx-0-21swapper/221:35:272
29935050,0irq/33-eth0-tx-0-21swapper/221:23:052
29925050,0irq/32-eth0-rx-5821-21irqrtprio21:57:113
29925050,0irq/32-eth0-rx-22279-21ssh22:42:203
29925050,0irq/32-eth0-rx-0-21swapper/323:57:303
29925050,0irq/32-eth0-rx-0-21swapper/323:48:073
29925050,0irq/32-eth0-rx-0-21swapper/323:37:083
29925050,0irq/32-eth0-rx-0-21swapper/320:22:063
29925050,0irq/32-eth0-rx-0-21swapper/318:47:583
259950,0migration/25856-21sh22:32:522
259950,0migration/226148-21kthreadcore23:42:112
259950,0migration/220001-21runrttasks22:04:012
259950,0migration/216315-21latency_hist21:06:522
259950,0migration/213769-21gltestperf23:57:082
19150,0ktimersoftd/10-21swapper/123:51:431
19150,0ktimersoftd/10-21swapper/122:32:251
19150,0ktimersoftd/10-21swapper/122:22:481
179950,0migration/18989-21idleruntime21:02:081
179950,0migration/14891-21kthreadcore21:17:151
179950,0migration/13578-21aten_r1power_en23:32:061
179950,0migration/131585-21kthreadcore23:07:121
179950,0migration/131121-21kthreadcore20:37:111
179950,0migration/12310-21fschecks_count19:12:051
179950,0migration/122286-21munin-node22:42:211
179950,0migration/119813-21latency_hist19:01:531
179950,0migration/116477-21kthreadcore20:27:121
179950,0migration/115373-21munin-node19:17:151
179950,0migration/112745-21taskset22:02:041
169950,0watchdog/18202-21sh22:54:571
169950,0watchdog/113450-21kthreadcore21:22:131
149950,0migration/08451-21ksmtuned21:19:080
149950,0migration/08327-21kthreadcore21:57:120
149950,0migration/08048-21aten_r1power_fr21:02:050
149950,0migration/07269-21kthreadcore18:52:110
149950,0migration/05779-21kthreadcore19:57:090
149950,0migration/05399-21kthreadcore23:12:090
149950,0migration/04715-21kthreadcore22:52:140
149950,0migration/0331-21kthreadcore19:07:120
149950,0migration/031050-21ntp_kernel_err21:12:160
149950,0migration/030412-21fschecks_time22:12:070
149950,0migration/02953-21munin-node19:12:070
149950,0migration/029423-21latency19:27:120
149950,0migration/027525-21runrttasks20:33:370
149950,0migration/02743-21diskmemload20:37:130
149950,0migration/026363-21munin-node23:07:070
149950,0migration/025845-21taskset21:30:310
149950,0migration/025706-21iostat22:47:130
149950,0migration/025226-21kthreadcore22:27:120
149950,0migration/025082-21kthreadcore18:42:070
149950,0migration/024034-21kthreadcore22:07:120
149950,0migration/023193-21kthreadcore21:47:120
149950,0migration/022727-21kthreadcore19:47:120
149950,0migration/022452-21kthreadcore23:42:100
149950,0migration/019332-21munin-node22:02:200
149950,0migration/019173-21kthreadcore22:42:160
149950,0migration/019113-21kthreadcore21:07:110
149950,0migration/016415-21taskset22:59:130
149950,0migration/016269-21kthreadcore20:27:120
149950,0migration/014990-21kthreadcore23:17:120
149950,0migration/014295-21irqcore19:42:070
149950,0migration/012844-21kthreadcore20:02:100
149950,0migration/012366-21kthreadcore21:22:130
149950,0migration/010917-21processes23:52:160
149950,0migration/010186-21kthreadcore22:37:090
87099943,0cyclictest6531-21awk18:52:102
87099943,0cyclictest3599-21lldpd22:12:032
87099943,0cyclictest3599-21lldpd21:41:532
87099943,0cyclictest3599-21lldpd20:36:542
87099943,0cyclictest3599-21lldpd19:56:542
87099943,0cyclictest3262-21irqbalance23:17:062
87099943,0cyclictest3262-21irqbalance23:08:372
87099943,0cyclictest3262-21irqbalance22:42:172
87099943,0cyclictest3262-21irqbalance21:02:572
87099943,0cyclictest3262-21irqbalance20:27:572
87099943,0cyclictest3262-21irqbalance19:31:572
87099943,0cyclictest3262-21irqbalance19:17:072
87099943,0cyclictest3262-21irqbalance19:11:172
87099943,0cyclictest3262-21irqbalance18:46:572
87099943,0cyclictest3262-21irqbalance18:37:072
87099943,0cyclictest24488-21awk18:42:062
87099943,0cyclictest22899-21munin-node20:52:102
87099943,0cyclictest10006-21awk18:32:082
87089944,0cyclictest0-21swapper/122:57:101
87089944,0cyclictest0-21swapper/121:26:591
87089944,0cyclictest0-21swapper/119:48:081
87089943,0cyclictest9974-21awk18:32:081
87089943,0cyclictest3599-21lldpd20:56:281
87089943,0cyclictest3262-21irqbalance19:07:471
87089943,0cyclictest29258-21awk21:52:081
87089943,0cyclictest28467-21awk19:52:101
87089943,0cyclictest27048-21grep21:32:081
87089943,0cyclictest23871-21snmpd23:05:341
87089943,0cyclictest23871-21snmpd22:15:481
87089943,0cyclictest23871-21snmpd22:00:331
87089943,0cyclictest23871-21snmpd21:09:031
87089943,0cyclictest23871-21snmpd19:31:181
87079943,0cyclictest6982-21awk22:17:100
87079943,0cyclictest543-21kworker/0:221:35:230
87079943,0cyclictest3773-21awk21:37:090
87079943,0cyclictest32426-21awk22:32:090
87079943,0cyclictest23871-21snmpd23:31:190
87079943,0cyclictest23871-21snmpd18:40:470
87079943,0cyclictest12945-21awk23:37:110
87079940,0cyclictest31735-21perl20:17:090
8315040,0irq/26-0000:00:2065-21md0_raid120:31:532
8315040,0irq/26-0000:00:16276-21cat19:21:532
8315040,0irq/26-0000:00:0-21swapper/220:41:532
8315040,0irq/26-0000:00:0-21swapper/219:41:532
8315040,0irq/26-0000:00:0-21swapper/219:11:532
29935040,0irq/33-eth0-tx-7182-21cut22:17:102
29935040,0irq/33-eth0-tx-26540-21ssh21:47:492
29935040,0irq/33-eth0-tx-2613-21ssh21:36:572
29935040,0irq/33-eth0-tx-2481-21ssh21:52:292
29935040,0irq/33-eth0-tx-15601-21irqcore22:22:102
29935040,0irq/33-eth0-tx-0-21swapper/223:47:042
29935040,0irq/33-eth0-tx-0-21swapper/222:56:542
29935040,0irq/33-eth0-tx-0-21swapper/222:52:052
29935040,0irq/33-eth0-tx-0-21swapper/221:17:242
29935040,0irq/33-eth0-tx-0-21swapper/221:12:192
29935040,0irq/33-eth0-tx-0-21swapper/019:52:100
29935040,0irq/33-eth0-tx-0-21swapper/019:22:030
29935040,0irq/33-eth0-tx-0-21swapper/019:17:060
29935040,0irq/33-eth0-tx-0-21swapper/019:02:080
259940,0migration/27006-21irqcore19:37:092
259940,0migration/26849-21gltestperf20:22:082
259940,0migration/2320-21kthreadcore20:17:122
259940,0migration/229612-21kthreadcore19:52:112
259940,0migration/229116-21irqrtprio23:27:092
259940,0migration/229052-21kthreadcore19:27:122
259940,0migration/225807-21kthreadcore22:27:122
259940,0migration/224324-21fschecks_count20:12:042
259940,0migration/21642-21kthreadcore20:57:112
249940,0watchdog/25565-21sh23:52:092
249940,0watchdog/220327-21cut19:02:052
249940,0watchdog/20-21swapper/220:07:092
179940,0migration/19530-21kthreadcore19:37:101
179940,0migration/18855-21irqcore22:37:081
179940,0migration/15396-21aten_r1power_cu18:52:051
179940,0migration/126256-21kthreadcore22:27:121
179940,0migration/123764-21munin-node18:42:031
179940,0migration/122655-21timerandwakeup19:22:171
179940,0migration/120777-21kthreadcore20:07:131
179940,0migration/120568-21rt-features20:47:201
179940,0migration/119337-21munin-node23:57:221
179940,0migration/115388-21kthreadcore18:57:121
179940,0migration/112780-21irqrtprio21:42:081
179940,0migration/111844-21taskset23:37:011
169940,0watchdog/131069-21unixbench_multi22:47:251
169940,0watchdog/129181-21kthreadcore20:12:091
169940,0watchdog/127960-40yum-updatesd-he22:07:371
169940,0watchdog/126460-21ssh21:47:331
169940,0watchdog/126248-21cut21:12:131
169940,0watchdog/10-21swapper/123:52:131
169940,0watchdog/10-21swapper/119:57:131
169940,0watchdog/10-21swapper/119:32:091
149940,0migration/024815-21kthreadcore20:52:110
149940,0migration/017212-21kthreadcore23:57:120
149940,0migration/013918-21kthreadcore18:32:100
87099932,0cyclictest3599-21lldpd18:58:222
87089932,0cyclictest4381-21sensors18:47:161
87089932,0cyclictest3847-21awk21:37:101
87089932,0cyclictest3212-21awk23:12:071
87089932,0cyclictest11239-21grep20:22:131
87089930,0cyclictest28326-21perl23:27:061
87079933,0cyclictest543-21kworker/0:222:22:230
87079933,0cyclictest543-21kworker/0:221:52:050
87079933,0cyclictest543-21kworker/0:221:41:530
87079933,0cyclictest543-21kworker/0:220:56:570
87079933,0cyclictest543-21kworker/0:220:46:570
87079933,0cyclictest543-21kworker/0:220:41:570
87079933,0cyclictest543-21kworker/0:220:21:550
87079933,0cyclictest543-21kworker/0:218:56:560
87079933,0cyclictest543-21kworker/0:218:46:540
87079933,0cyclictest31215-21kworker/0:123:46:590
87079933,0cyclictest31215-21kworker/0:123:21:590
87079933,0cyclictest31215-21kworker/0:123:01:570
87079933,0cyclictest0-21swapper/023:32:100
179930,0migration/118531-21kthreadcore18:37:111
169930,0watchdog/120380-21ssh20:32:051
169930,0watchdog/10-21swapper/123:42:051
169930,0watchdog/10-21swapper/123:21:531
169930,0watchdog/10-21swapper/120:17:131
169930,0watchdog/10-21swapper/120:02:131
87089920,0cyclictest12898-21taskset19:41:571
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional