You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-12-10 - 13:01
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Tue Dec 10, 2024 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
372424738,0sleep20-21swapper/218:40:222
375423930,0sleep10-21swapper/118:40:411
336923728,0sleep30-21swapper/318:36:313
342322925,0sleep00-21swapper/018:37:040
339970,0migration/316365-21gltestperf20:41:043
329970,0watchdog/315950-21sh23:52:213
329970,0watchdog/30-21swapper/320:31:103
29925070,0irq/32-eth0-rx-0-21swapper/321:16:083
38429966,0cyclictest15148-21diskmemload22:26:532
38429966,0cyclictest15148-21diskmemload21:37:322
38429960,0cyclictest0-21swapper/222:14:262
38419965,0cyclictest2338-21sed22:11:091
339960,0migration/332162-21df_abs21:51:053
339960,0migration/315605-21fschecks_count19:56:043
339960,0migration/311838-21fschecks_count18:46:073
329960,0watchdog/321122-21sensors19:11:143
329960,0watchdog/30-21swapper/323:56:413
329960,0watchdog/30-21swapper/323:49:453
329960,0watchdog/30-21swapper/323:36:053
329960,0watchdog/30-21swapper/323:32:053
329960,0watchdog/30-21swapper/323:27:573
329960,0watchdog/30-21swapper/323:18:373
329960,0watchdog/30-21swapper/323:03:453
329960,0watchdog/30-21swapper/322:41:173
329960,0watchdog/30-21swapper/322:26:133
329960,0watchdog/30-21swapper/322:21:253
329960,0watchdog/30-21swapper/322:13:253
329960,0watchdog/30-21swapper/322:05:213
329960,0watchdog/30-21swapper/321:47:533
329960,0watchdog/30-21swapper/321:40:023
329960,0watchdog/30-21swapper/321:25:583
329960,0watchdog/30-21swapper/321:10:503
329960,0watchdog/30-21swapper/320:58:063
329960,0watchdog/30-21swapper/320:53:303
329960,0watchdog/30-21swapper/320:53:303
329960,0watchdog/30-21swapper/320:47:143
329960,0watchdog/30-21swapper/320:01:183
329960,0watchdog/30-21swapper/318:55:343
29935060,0irq/33-eth0-tx-8188-21munin-node20:36:130
29925060,0irq/32-eth0-rx-25200-21munin-node19:36:193
29925060,0irq/32-eth0-rx-219572sleep321:42:333
29925060,0irq/32-eth0-rx-11856-21tune2fs22:36:033
29925060,0irq/32-eth0-rx-0-21swapper/323:22:153
29925060,0irq/32-eth0-rx-0-21swapper/323:12:293
29925060,0irq/32-eth0-rx-0-21swapper/322:55:213
29925060,0irq/32-eth0-rx-0-21swapper/322:17:533
29925060,0irq/32-eth0-rx-0-21swapper/322:10:293
29925060,0irq/32-eth0-rx-0-21swapper/321:55:573
29925060,0irq/32-eth0-rx-0-21swapper/321:01:063
29925060,0irq/32-eth0-rx-0-21swapper/320:16:003
29925060,0irq/32-eth0-rx-0-21swapper/319:06:073
29925060,0irq/32-eth0-rx-0-21swapper/300:01:503
149960,0migration/05571-21latency_hist23:10:480
149960,0migration/030346-21sh21:47:230
149960,0migration/023537-21sh23:38:490
149960,0migration/023155-21kthreadcore23:01:070
4150,0ktimersoftd/00-21swapper/022:08:130
38429955,0cyclictest0-21swapper/223:04:532
38429955,0cyclictest0-21swapper/222:40:362
38429955,0cyclictest0-21swapper/221:22:342
38429950,0cyclictest0-21swapper/223:48:242
38429950,0cyclictest0-21swapper/223:27:162
38429950,0cyclictest0-21swapper/220:01:122
38419950,0cyclictest0-21swapper/120:52:521
38419950,0cyclictest0-21swapper/120:52:521
38419950,0cyclictest0-21swapper/100:01:251
38409955,0cyclictest15148-21diskmemload22:21:000
38409954,0cyclictest15474-21awk23:16:070
38409953,0cyclictest27361-21kworker/0:021:37:220
38409950,0cyclictest0-21swapper/020:31:060
339950,0migration/34431-21cpuspeed_turbos19:26:023
339950,0migration/328677-21kthreadcore21:06:113
339950,0migration/323750-21kthreadcore19:16:093
339950,0migration/320264-21munin-node20:21:053
339950,0migration/313936-21kthreadcore20:36:083
339950,0migration/310337-21switchtime18:41:143
329950,0watchdog/328551-21cut19:41:063
329950,0watchdog/328499-21df_inode22:46:013
329950,0watchdog/30-21swapper/323:40:493
329950,0watchdog/30-21swapper/323:05:493
329950,0watchdog/30-21swapper/322:30:573
329950,0watchdog/30-21swapper/321:30:503
329950,0watchdog/30-21swapper/321:21:023
329950,0watchdog/30-21swapper/320:26:023
329950,0watchdog/30-21swapper/320:13:503
329950,0watchdog/30-21swapper/320:06:103
329950,0watchdog/30-21swapper/319:50:543
329950,0watchdog/30-21swapper/319:31:103
329950,0watchdog/30-21swapper/318:55:543
29935050,0irq/33-eth0-tx-8602-21kthreadcore21:11:082
29935050,0irq/33-eth0-tx-31480-21ssh23:42:022
29935050,0irq/33-eth0-tx-17694-21munin-node21:01:102
29935050,0irq/33-eth0-tx-14944-21munin-node19:11:160
29935050,0irq/33-eth0-tx-0-21swapper/221:30:082
29935050,0irq/33-eth0-tx-0-21swapper/221:07:472
29935050,0irq/33-eth0-tx-0-21swapper/020:41:250
29935050,0irq/33-eth0-tx-0-21swapper/020:11:010
29935050,0irq/33-eth0-tx-0-21swapper/019:51:190
29925050,0irq/32-eth0-rx-6766-21sensors19:01:133
29925050,0irq/32-eth0-rx-0-21swapper/322:56:053
29925050,0irq/32-eth0-rx-0-21swapper/300:05:573
259950,0migration/25036-21kthreadcore21:51:112
259950,0migration/232026-21munin-node00:01:132
259950,0migration/227007-21munin-node19:41:042
259950,0migration/226542-21fschecks_count20:26:052
259950,0migration/225869-21dump-pmu-power20:25:492
259950,0migration/213669-21needreboot19:06:092
249950,0watchdog/224053-21munin-node20:46:022
19150,0ktimersoftd/10-21swapper/123:10:581
179950,0migration/14506-21df_abs19:26:021
149950,0migration/09500-21kthreadcore19:06:070
149950,0migration/0833-21irqrtprio21:51:090
149950,0migration/07488-21kthreadcore22:31:110
149950,0migration/0681-21latency_hist20:30:490
149950,0migration/06493-21kthreadcore22:51:080
149950,0migration/05153-21kthreadcore00:06:080
149950,0migration/04700-21runrttasks18:41:030
149950,0migration/04048-21kthreadcore23:46:080
149950,0migration/03918-21needreboot21:31:090
149950,0migration/032026-21kthreadcore22:46:070
149950,0migration/031822-21fschecks_count23:26:020
149950,0migration/028164-21kthreadcore23:41:090
149950,0migration/027374-21sh22:44:340
149950,0migration/026478-21fschecks_time19:41:020
149950,0migration/019226-21kthreadcore22:01:080
149950,0migration/018221-21sh21:16:570
149950,0migration/018094-21df_abs21:00:590
149950,0migration/015511-21kthreadcore20:16:080
149950,0migration/013072-21taskset23:12:560
149950,0migration/01192-21taskset00:05:150
38429943,0cyclictest5545-21awk20:11:062
38429943,0cyclictest3744-21awk22:31:092
38429943,0cyclictest23871-21snmpd22:51:072
38429943,0cyclictest23871-21snmpd20:40:512
38429943,0cyclictest23871-21snmpd20:36:512
38429943,0cyclictest23871-21snmpd20:09:362
38429943,0cyclictest23871-21snmpd19:45:512
38429943,0cyclictest23871-21snmpd19:35:512
38429943,0cyclictest23871-21snmpd19:28:212
38429943,0cyclictest23871-21snmpd18:58:212
38429943,0cyclictest23871-21snmpd18:45:512
38429943,0cyclictest19617-21awk18:51:052
38429943,0cyclictest12719-21awk20:16:052
38429943,0cyclictest10024-21awk23:51:052
38419944,0cyclictest0-21swapper/122:50:471
38419943,0cyclictest8574-21awk23:31:051
38419943,0cyclictest3599-21lldpd23:42:131
38419943,0cyclictest3599-21lldpd21:49:321
38419943,0cyclictest3599-21lldpd19:04:471
38419943,0cyclictest3262-21irqbalance23:27:331
38419943,0cyclictest30360-21awk19:21:071
38419943,0cyclictest2813-21awk00:06:061
38419943,0cyclictest27511-21awk22:26:051
38419943,0cyclictest27258-21ps18:56:071
38419943,0cyclictest23871-21snmpd23:45:521
38419943,0cyclictest23871-21snmpd23:39:371
38419943,0cyclictest23871-21snmpd22:17:521
38419943,0cyclictest23871-21snmpd20:15:211
38419943,0cyclictest23871-21snmpd19:36:361
38419943,0cyclictest23871-21snmpd19:08:361
38419943,0cyclictest23871-21snmpd18:53:061
38419943,0cyclictest1887-21awk20:31:051
38409943,0cyclictest4042-21kworker/0:018:51:000
38409943,0cyclictest3925-21awk21:11:050
38409943,0cyclictest3599-21lldpd22:38:520
38409943,0cyclictest23871-21snmpd22:26:220
38409943,0cyclictest2028-21awk22:11:080
38409943,0cyclictest19714-21awk19:36:070
38409943,0cyclictest17426-21sadc21:00:490
38409940,0cyclictest15288-21telnet19:56:020
38409940,0cyclictest0-21swapper/021:09:250
329940,0watchdog/30-21swapper/319:46:143
29935040,0irq/33-eth0-tx-16342-21ssh22:00:492
29935040,0irq/33-eth0-tx-15148-21diskmemload23:10:572
29935040,0irq/33-eth0-tx-0-21swapper/223:56:092
29935040,0irq/33-eth0-tx-0-21swapper/223:36:012
29935040,0irq/33-eth0-tx-0-21swapper/223:21:112
29935040,0irq/33-eth0-tx-0-21swapper/223:16:072
29935040,0irq/33-eth0-tx-0-21swapper/223:05:562
29935040,0irq/33-eth0-tx-0-21swapper/222:56:092
29935040,0irq/33-eth0-tx-0-21swapper/222:45:582
29935040,0irq/33-eth0-tx-0-21swapper/222:41:172
29935040,0irq/33-eth0-tx-0-21swapper/222:15:492
29935040,0irq/33-eth0-tx-0-21swapper/222:06:022
29935040,0irq/33-eth0-tx-0-21swapper/221:42:142
29935040,0irq/33-eth0-tx-0-21swapper/221:30:532
29935040,0irq/33-eth0-tx-0-21swapper/221:16:172
29935040,0irq/33-eth0-tx-0-21swapper/220:56:192
29935040,0irq/33-eth0-tx-0-21swapper/220:50:522
29935040,0irq/33-eth0-tx-0-21swapper/220:50:522
29935040,0irq/33-eth0-tx-0-21swapper/020:21:050
29935040,0irq/33-eth0-tx-0-21swapper/020:06:040
29935040,0irq/33-eth0-tx-0-21swapper/019:26:120
29935040,0irq/33-eth0-tx-0-21swapper/018:46:170
29925040,0irq/32-eth0-rx-0-21swapper/319:21:033
259940,0migration/29815-21kthreadcore23:31:082
259940,0migration/28572-21fschecks_count19:51:022
259940,0migration/28236-21kthreadcore18:41:082
259940,0migration/26930-21diskmemload00:06:092
259940,0migration/2351-21latency_hist19:00:492
259940,0migration/228438-21timerandwakeup19:16:172
259940,0migration/228390-21kthreadcore21:46:082
259940,0migration/218679-21kthreadcore19:11:092
259940,0migration/217818-21kthreadcore19:56:082
259940,0migration/21272-21munin-node20:31:032
249940,0watchdog/28286-21df_abs21:56:012
249940,0watchdog/22443-21kthreadcore19:21:102
249940,0watchdog/214922-21ps19:31:102
179940,0migration/17435-21idleruntime21:36:051
179940,0migration/16389-21kthreadcore22:31:111
179940,0migration/16200-21runrttasks21:51:311
179940,0migration/13872-21taskset21:11:051
179940,0migration/131804-21taskset20:49:151
179940,0migration/129510-21latency_hist23:05:491
179940,0migration/126891-21fschecks_count21:06:071
179940,0migration/123109-21fschecks_time20:01:051
179940,0migration/118200-21gltestperf23:56:041
179940,0migration/116610-21kthreadcore21:41:071
179940,0migration/115751-21fschecks_count19:56:041
179940,0migration/115362-21df_inode19:11:041
179940,0migration/111670-21fschecks_count22:36:031
179940,0migration/111233-21kthreadcore20:36:071
169940,0watchdog/19260-21irqcore19:51:061
169940,0watchdog/130521-21df21:31:021
169940,0watchdog/119323-21fschecks_count20:21:021
169940,0watchdog/116499-21if_err_eth1.320:41:061
169940,0watchdog/111629-21df_abs18:46:061
169940,0watchdog/10-21swapper/123:21:051
169940,0watchdog/10-21swapper/123:16:091
169940,0watchdog/10-21swapper/123:04:171
169940,0watchdog/10-21swapper/122:41:091
169940,0watchdog/10-21swapper/121:56:011
169940,0watchdog/10-21swapper/120:26:101
169940,0watchdog/10-21swapper/119:31:101
149940,0migration/09653-21aten_r1power_vo22:16:020
149940,0migration/030653-21kthreadcore18:56:080
149940,0migration/029084-21sh21:28:150
149940,0migration/02764-21kthreadcore19:01:080
149940,0migration/025675-21kthreadcore20:01:090
149940,0migration/019041-21kthreadcore23:56:080
149940,0migration/018021-21kthreadcore21:41:070
38419933,0cyclictest3599-21lldpd18:43:471
38419932,0cyclictest26776-21grep19:41:021
38419932,0cyclictest23871-21snmpd23:50:521
38419932,0cyclictest23871-21snmpd22:20:521
38419932,0cyclictest2290-21cut20:06:091
38419932,0cyclictest22417-21cut21:25:591
38419932,0cyclictest21057-21grep21:01:031
38419932,0cyclictest1673-21awk19:46:061
38419930,0cyclictest9718-21perl20:56:051
38409933,0cyclictest4042-21kworker/0:019:15:500
38409933,0cyclictest28314-21kworker/0:120:50:540
38409933,0cyclictest28314-21kworker/0:120:50:540
38409933,0cyclictest28314-21kworker/0:120:45:540
38409933,0cyclictest28314-21kworker/0:119:45:500
38409933,0cyclictest28314-21kworker/0:119:30:580
38409933,0cyclictest28314-21kworker/0:119:20:520
38409933,0cyclictest27361-21kworker/0:022:55:540
38409933,0cyclictest27361-21kworker/0:021:55:500
38409933,0cyclictest27361-21kworker/0:021:20:540
38409933,0cyclictest25478-21kworker/0:123:50:490
38409933,0cyclictest22129-21kworker/0:123:30:490
38409933,0cyclictest22129-21kworker/0:123:20:490
29935030,0irq/33-eth0-tx-0-21swapper/222:21:012
179930,0migration/122114-21runrttasks19:16:021
179930,0migration/112690-21munin-node20:16:051
169930,0watchdog/119953-21ssh21:21:021
169930,0watchdog/10-21swapper/122:55:571
169930,0watchdog/10-21swapper/122:51:011
169930,0watchdog/10-21swapper/122:06:091
169930,0watchdog/10-21swapper/122:01:411
169920,0watchdog/10-21swapper/121:15:501
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional