You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-28 - 23:14
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2.osadl.org (updated Sat Jun 28, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
838423826,0sleep20-21swapper/206:28:232
136623828,0sleep30-21swapper/306:23:553
821723525,0sleep10-21swapper/106:26:251
807923026,0sleep00-21swapper/006:24:480
329970,0watchdog/331384-21fschecks_time07:29:093
329970,0watchdog/321876-21sh11:01:173
329970,0watchdog/315170-21ssh09:19:253
329970,0watchdog/30-21swapper/311:09:133
329970,0watchdog/30-21swapper/310:19:093
329970,0watchdog/30-21swapper/309:59:053
29925070,0irq/32-eth0-rx-27730-21munin-node09:49:073
29925070,0irq/32-eth0-rx-23946-21open_inodes09:44:153
29925070,0irq/32-eth0-rx-0-21swapper/311:23:173
29925070,0irq/32-eth0-rx-0-21swapper/310:12:083
29925070,0irq/32-eth0-rx-0-21swapper/309:14:083
84919966,0cyclictest28226-21ntpq08:49:160
84919965,0cyclictest11741-21ssh10:35:320
8315060,0irq/26-0000:00:10027-21if_eth1.307:14:062
35160,0ktimersoftd/30-21swapper/308:40:473
339960,0migration/37980-21sh09:34:363
339960,0migration/323829-21sh09:25:213
339960,0migration/319892-21sh08:28:493
339960,0migration/314507-21sh11:14:453
329960,0watchdog/326948-21kthreadcore08:09:093
329960,0watchdog/30-21swapper/311:40:293
329960,0watchdog/30-21swapper/311:35:213
329960,0watchdog/30-21swapper/311:32:533
329960,0watchdog/30-21swapper/311:04:133
329960,0watchdog/30-21swapper/310:35:053
329960,0watchdog/30-21swapper/310:27:093
329960,0watchdog/30-21swapper/310:04:133
329960,0watchdog/30-21swapper/309:39:053
329960,0watchdog/30-21swapper/308:51:253
329960,0watchdog/30-21swapper/308:45:533
329960,0watchdog/30-21swapper/308:08:013
329960,0watchdog/30-21swapper/307:40:133
329960,0watchdog/30-21swapper/307:14:053
29925060,0irq/32-eth0-rx-30464-21diskstats10:49:043
29925060,0irq/32-eth0-rx-0-21swapper/311:56:143
29925060,0irq/32-eth0-rx-0-21swapper/311:52:043
29925060,0irq/32-eth0-rx-0-21swapper/311:44:303
29925060,0irq/32-eth0-rx-0-21swapper/311:26:113
29925060,0irq/32-eth0-rx-0-21swapper/310:53:523
29925060,0irq/32-eth0-rx-0-21swapper/310:32:393
29925060,0irq/32-eth0-rx-0-21swapper/310:15:523
29925060,0irq/32-eth0-rx-0-21swapper/309:31:153
29925060,0irq/32-eth0-rx-0-21swapper/309:09:133
29925060,0irq/32-eth0-rx-0-21swapper/309:09:133
29925060,0irq/32-eth0-rx-0-21swapper/309:05:043
29925060,0irq/32-eth0-rx-0-21swapper/308:59:473
29925060,0irq/32-eth0-rx-0-21swapper/308:57:053
29925060,0irq/32-eth0-rx-0-21swapper/308:29:113
27160,0ktimersoftd/20-21swapper/208:40:472
149960,0migration/07050-21kthreadcore07:34:110
149960,0migration/029883-21kthreadcore10:09:120
149960,0migration/02942-21sh10:29:350
149960,0migration/018472-21sh10:03:280
84939950,0cyclictest0-21swapper/210:19:102
84929955,0cyclictest0-21swapper/111:20:221
84929955,0cyclictest0-21swapper/109:58:331
84929950,0cyclictest0-21swapper/109:46:251
84929950,0cyclictest0-21swapper/109:38:221
84919955,0cyclictest0-21swapper/011:49:570
84919955,0cyclictest0-21swapper/009:24:070
84919955,0cyclictest0-21swapper/008:04:110
84919955,0cyclictest0-21swapper/007:09:190
84919950,0cyclictest0-21swapper/011:56:150
84919950,0cyclictest0-21swapper/011:22:210
84919950,0cyclictest0-21swapper/008:38:240
8315050,0irq/26-0000:00:2717-21perl07:54:082
8315050,0irq/26-0000:00:22661-21munin-run11:03:482
8315050,0irq/26-0000:00:0-21swapper/306:49:443
4150,0ktimersoftd/00-21swapper/010:56:580
339950,0migration/3914-21munin-node07:04:203
329950,0watchdog/3299250irq/32-eth0-rx-08:34:053
329950,0watchdog/320549-21hddtemp_smartct06:59:053
329950,0watchdog/30-21swapper/310:44:253
329950,0watchdog/30-21swapper/308:19:093
329950,0watchdog/30-21swapper/308:14:013
329950,0watchdog/30-21swapper/307:59:173
329950,0watchdog/30-21swapper/307:54:133
329950,0watchdog/30-21swapper/307:48:533
329950,0watchdog/30-21swapper/307:45:533
329950,0watchdog/30-21swapper/307:34:053
329950,0watchdog/30-21swapper/307:23:493
329950,0watchdog/30-21swapper/307:18:533
329950,0watchdog/30-21swapper/307:09:093
329950,0watchdog/30-21swapper/306:55:093
329950,0watchdog/30-21swapper/306:44:053
329950,0watchdog/30-21swapper/306:42:093
329950,0watchdog/30-21swapper/306:29:133
29935050,0irq/33-eth0-tx-8768-21if_eth1.311:14:062
29935050,0irq/33-eth0-tx-32386-21cut10:49:082
29935050,0irq/33-eth0-tx-2612-21ssh11:45:222
29935050,0irq/33-eth0-tx-23311-21munin-node07:24:180
29935050,0irq/33-eth0-tx-20648-21kthreadcore11:19:102
29935050,0irq/33-eth0-tx-14476-21cut11:54:082
29935050,0irq/33-eth0-tx-14321-21kthreadcore09:39:102
29935050,0irq/33-eth0-tx-12933-21munin-node08:24:040
29935050,0irq/33-eth0-tx-1021-21ssh09:33:522
29935050,0irq/33-eth0-tx-0-21swapper/211:37:502
29935050,0irq/33-eth0-tx-0-21swapper/211:31:592
29935050,0irq/33-eth0-tx-0-21swapper/211:27:222
29935050,0irq/33-eth0-tx-0-21swapper/210:04:072
29935050,0irq/33-eth0-tx-0-21swapper/210:01:482
29935050,0irq/33-eth0-tx-0-21swapper/209:16:112
29935050,0irq/33-eth0-tx-0-21swapper/209:03:342
29935050,0irq/33-eth0-tx-0-21swapper/007:41:530
29935050,0irq/33-eth0-tx-0-21swapper/007:04:090
29935050,0irq/33-eth0-tx-0-21swapper/006:31:400
29925050,0irq/32-eth0-rx-4288-21grep09:54:143
29925050,0irq/32-eth0-rx-0-21swapper/310:39:053
29925050,0irq/32-eth0-rx-0-21swapper/306:34:083
27150,0ktimersoftd/20-21swapper/211:52:212
259950,0migration/27364-21if_eth1.310:54:092
259950,0migration/232301-21irqrtprio08:14:082
259950,0migration/230787-21kthreadcore10:29:122
259950,0migration/226095-21kthreadcore08:49:142
259950,0migration/225950-21kthreadcore08:09:092
259950,0migration/223991-21idleruntime11:04:082
259950,0migration/223068-21kthreadcore10:24:132
259950,0migration/220915-21fschecks_time08:29:062
259950,0migration/22045-21kthreadcore07:29:132
249950,0watchdog/283150irq/26-0000:00:10:14:092
249950,0watchdog/22928-21ssh10:13:332
249950,0watchdog/224521-21ps07:24:092
249950,0watchdog/217635-21perl09:24:052
19150,0ktimersoftd/10-21swapper/108:40:471
179950,0migration/128378-21kthreadcore09:49:071
179950,0migration/124386-21irqrtprio11:04:101
179950,0migration/122490-21kthreadcore08:29:091
179950,0migration/119989-21aten_r1power_fr07:44:121
179950,0migration/119368-21idleruntime-cro06:58:491
179950,0migration/116485-21idleruntime-cro09:23:491
179950,0migration/115663-21cpu09:04:041
179950,0migration/11519-21taskset07:51:121
179950,0migration/110170-21latency_hist11:33:481
169950,0watchdog/114777-21aten_r1power_vo10:59:011
149950,0migration/07165-21sh09:15:450
149950,0migration/05699-21taskset08:19:010
149950,0migration/03449-21kthreadcore10:49:090
149950,0migration/03277-21taskset10:13:570
149950,0migration/030776-21sh09:09:500
149950,0migration/030776-21sh09:09:500
149950,0migration/027706-21latency_hist11:43:480
149950,0migration/02706-21kthreadcore06:44:080
149950,0migration/026688-21fschecks_count09:49:030
149950,0migration/026026-21fschecks_time09:29:040
149950,0migration/025773-21kthreadcore10:24:140
149950,0migration/02070-21kthreadcore11:09:120
149950,0migration/019371-21kthreadcore10:59:070
149950,0migration/015983-21aten_r1power_po06:34:030
149950,0migration/015794-21sh11:18:070
149950,0migration/015499-21sh09:04:030
149950,0migration/015405-21proc_pri07:59:130
149950,0migration/013453-21kthreadcore10:19:130
149950,0migration/012617-21kthreadcore09:39:090
149950,0migration/01206-21sh08:54:120
84939944,0cyclictest0-21swapper/211:39:062
84939944,0cyclictest0-21swapper/209:04:072
84939943,0cyclictest25109-21awk09:09:082
84939943,0cyclictest25109-21awk09:09:082
84939943,0cyclictest23871-21snmpd07:48:372
84939943,0cyclictest23871-21snmpd07:11:512
84939943,0cyclictest23871-21snmpd06:32:362
84939943,0cyclictest22753-21awk10:44:092
84939943,0cyclictest20741-21awk06:59:072
84939943,0cyclictest13428-21awk06:54:082
84929944,0cyclictest0-21swapper/107:54:101
84929943,0cyclictest8191-21awk08:59:091
84929943,0cyclictest6576-21awk07:34:101
84929943,0cyclictest3599-21lldpd10:41:141
84929943,0cyclictest3599-21lldpd10:27:371
84929943,0cyclictest3599-21lldpd10:18:061
84929943,0cyclictest32444-21awk11:09:101
84929943,0cyclictest29646-21awk10:29:091
84929943,0cyclictest29127-21awk11:44:091
84929943,0cyclictest28488-21awk10:09:101
84929943,0cyclictest27803-21awk07:04:081
84929943,0cyclictest2748-21awk07:09:141
84929943,0cyclictest23871-21snmpd11:14:081
84929943,0cyclictest23871-21snmpd10:44:231
84929943,0cyclictest23871-21snmpd07:28:511
84929943,0cyclictest19953-21awk10:04:071
84929943,0cyclictest17943-21awk09:24:071
84929943,0cyclictest14888-21grep07:14:101
84929943,0cyclictest14285-21awk11:54:071
84929943,0cyclictest13983-21awk08:24:071
84929943,0cyclictest13687-21awk07:39:121
84919944,0cyclictest0-21swapper/010:03:510
84919944,0cyclictest0-21swapper/009:56:170
84919943,0cyclictest9498-21awk09:19:110
84919943,0cyclictest5880-21kworker/0:010:44:110
84919943,0cyclictest28996-21awk11:44:090
84919943,0cyclictest28338-21awk07:49:070
84919943,0cyclictest24789-21awk08:09:070
84919943,0cyclictest17148-21awk07:19:080
29935040,0irq/33-eth0-tx-5154-21munin-node06:49:130
29935040,0irq/33-eth0-tx-4533-21ssh10:33:532
29935040,0irq/33-eth0-tx-28580-21ssh08:33:512
29935040,0irq/33-eth0-tx-17765-21ssh09:43:582
29935040,0irq/33-eth0-tx-0-21swapper/211:09:302
29935040,0irq/33-eth0-tx-0-21swapper/210:39:122
29935040,0irq/33-eth0-tx-0-21swapper/209:54:092
29935040,0irq/33-eth0-tx-0-21swapper/209:48:592
29935040,0irq/33-eth0-tx-0-21swapper/209:19:592
29935040,0irq/33-eth0-tx-0-21swapper/208:54:292
29935040,0irq/33-eth0-tx-0-21swapper/008:28:510
29935040,0irq/33-eth0-tx-0-21swapper/008:14:030
29935040,0irq/33-eth0-tx-0-21swapper/007:29:170
29935040,0irq/33-eth0-tx-0-21swapper/006:54:100
259940,0migration/28728-21kthreadcore08:19:102
259940,0migration/28174-21kthreadcore07:34:112
259940,0migration/26738-21kernelversion06:49:082
259940,0migration/231502-21kthreadcore07:49:102
259940,0migration/228877-21kthreadcore09:29:082
259940,0migration/218007-21kthreadcore08:24:112
259940,0migration/217060-21kthreadcore07:39:152
259940,0migration/216418-21fschecks_count06:34:052
259940,0migration/214141-21fschecks_time08:44:062
249940,0watchdog/230342-21aten_r1power_en06:44:012
249940,0watchdog/227189-21diskstats07:04:052
249940,0watchdog/224420-21irqcore06:39:092
249940,0watchdog/218392-21kthreadcore08:04:092
249940,0watchdog/20-21swapper/207:59:172
179940,0migration/18385-21kthreadcore06:49:091
179940,0migration/129394-21kthreadcore09:09:111
179940,0migration/129394-21kthreadcore09:09:111
179940,0migration/128036-21kthreadcore06:39:111
179940,0migration/122716-21kthreadcore06:59:091
179940,0migration/118600-21kthreadcore08:04:091
179940,0migration/114501-21kthreadcore10:19:131
179940,0migration/114118-21fschecks_count08:44:061
179940,0migration/113001-21taskset10:54:491
179940,0migration/110964-21sendmail_mailqu10:34:221
179940,0migration/110535-21aten_r1power_en11:34:051
169940,0watchdog/14331-21ssh11:49:011
169940,0watchdog/13460-21grep08:54:131
169940,0watchdog/132750-21kthreadcore08:34:091
169940,0watchdog/124526-21grep07:24:091
169940,0watchdog/124388-21perl08:09:051
169940,0watchdog/120849-21kthreadcore06:34:091
169940,0watchdog/114945-21tr07:59:091
169940,0watchdog/111060-21df_abs09:59:051
169940,0watchdog/110114-21fschecks_time09:39:051
169940,0watchdog/10-21swapper/111:27:091
169940,0watchdog/10-21swapper/109:30:491
169940,0watchdog/10-21swapper/108:49:091
169940,0watchdog/10-21swapper/106:44:011
149940,0migration/024533-21sh09:44:260
149940,0migration/02165-21if_err_eth109:34:080
149940,0migration/021573-21kthreadcore07:44:170
149940,0migration/011285-21kthreadcore08:59:110
149940,0migration/010243-21irqcore07:14:070
84939932,0cyclictest3599-21lldpd07:19:182
84929932,0cyclictest9819-21awk06:29:101
84929932,0cyclictest5750-21basename08:19:031
84929932,0cyclictest32011-21awk08:14:071
84929932,0cyclictest1017-21awk09:14:061
84919933,0cyclictest7840-21kworker/0:106:39:060
84919933,0cyclictest5880-21kworker/0:011:04:090
84919933,0cyclictest5880-21kworker/0:010:39:130
84919933,0cyclictest24290-21kworker/0:208:38:550
84919933,0cyclictest2303-21kworker/0:111:33:590
84919933,0cyclictest0-21swapper/011:28:490
84919932,0cyclictest19587-21munin-node06:59:060
84919932,0cyclictest14571-21awk08:44:080
4130,0ktimersoftd/025149-21ssh11:23:590
29935030,0irq/33-eth0-tx-0-21swapper/007:54:070
169930,0watchdog/10-21swapper/111:39:051
169930,0watchdog/10-21swapper/110:49:091
169930,0watchdog/10-21swapper/107:18:491
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional