You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-09-16 - 14:45
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2.osadl.org (updated Tue Sep 16, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2261627271,0sleep00-21swapper/006:20:490
22961995555,0cyclictest29390-21md1_resync10:58:361
2268123525,0sleep20-21swapper/206:21:312
2256823425,0sleep10-21swapper/106:20:141
22960993333,0cyclictest29391-21md0_resync10:58:360
2280223026,0sleep30-21swapper/306:22:563
22962991919,0cyclictest25288-21turbostat09:13:552
8315080,0irq/26-0000:00:8102-21vmstat07:59:233
29925080,0irq/32-eth0-rx-22831-21kthreadcore11:24:143
229629988,0cyclictest2065-21md0_raid110:58:362
8315070,0irq/26-0000:00:849-21taskset07:57:093
8315070,0irq/26-0000:00:173992sleep309:25:213
8315070,0irq/26-0000:00:13815-21grep10:59:143
8315070,0irq/26-0000:00:0-21swapper/311:45:343
8315070,0irq/26-0000:00:0-21swapper/309:01:443
329970,0watchdog/32631-21gltestperf09:19:163
29925070,0irq/32-eth0-rx-30247-21kthreadcore09:14:163
29925070,0irq/32-eth0-rx-0-21swapper/311:35:003
8315060,0irq/26-0000:00:6386-21runrttasks08:39:283
8315060,0irq/26-0000:00:5514-21cut06:34:083
8315060,0irq/26-0000:00:29987-21perl07:34:133
8315060,0irq/26-0000:00:2909-21ntp_states11:29:183
8315060,0irq/26-0000:00:22639-21grep09:29:153
8315060,0irq/26-0000:00:22029-21expr08:09:163
8315060,0irq/26-0000:00:0-21swapper/311:11:233
8315060,0irq/26-0000:00:0-21swapper/310:36:083
8315060,0irq/26-0000:00:0-21swapper/310:29:153
8315060,0irq/26-0000:00:0-21swapper/310:27:073
8315060,0irq/26-0000:00:0-21swapper/309:34:153
8315060,0irq/26-0000:00:0-21swapper/308:48:573
8315060,0irq/26-0000:00:0-21swapper/308:29:163
8315060,0irq/26-0000:00:0-21swapper/308:26:353
8315060,0irq/26-0000:00:0-21swapper/307:49:033
8315060,0irq/26-0000:00:0-21swapper/307:24:183
8315060,0irq/26-0000:00:0-21swapper/307:24:183
8315060,0irq/26-0000:00:0-21swapper/307:00:483
8315060,0irq/26-0000:00:0-21swapper/306:51:433
8315060,0irq/26-0000:00:0-21swapper/306:24:143
339960,0migration/32875-21missed_timers09:54:183
329960,0watchdog/329287-21cut08:14:163
329960,0watchdog/30-21swapper/311:43:473
329960,0watchdog/30-21swapper/311:20:113
329960,0watchdog/30-21swapper/311:15:073
329960,0watchdog/30-21swapper/310:49:483
329960,0watchdog/30-21swapper/310:20:483
329960,0watchdog/30-21swapper/310:05:243
329960,0watchdog/30-21swapper/309:52:123
329960,0watchdog/30-21swapper/309:45:523
329960,0watchdog/30-21swapper/309:39:163
329960,0watchdog/30-21swapper/308:55:203
329960,0watchdog/30-21swapper/308:43:563
329960,0watchdog/30-21swapper/308:04:163
329960,0watchdog/30-21swapper/307:43:563
329960,0watchdog/30-21swapper/307:30:523
329960,0watchdog/30-21swapper/307:17:283
329960,0watchdog/30-21swapper/307:09:403
329960,0watchdog/30-21swapper/307:07:003
329960,0watchdog/30-21swapper/306:57:163
329960,0watchdog/30-21swapper/306:45:363
29935060,0irq/33-eth0-tx-30819-21munin-node08:19:110
29925060,0irq/32-eth0-rx-3857-21cut10:14:153
29925060,0irq/32-eth0-rx-20875-21kthreadcore10:44:213
29925060,0irq/32-eth0-rx-0-21swapper/311:51:013
29925060,0irq/32-eth0-rx-0-21swapper/310:56:343
29925060,0irq/32-eth0-rx-0-21swapper/310:09:163
29925060,0irq/32-eth0-rx-0-21swapper/309:08:163
29925060,0irq/32-eth0-rx-0-21swapper/308:37:093
229619966,0cyclictest0-21swapper/111:10:411
229619965,0cyclictest1172-21grep09:34:181
229609966,0cyclictest0-21swapper/010:46:170
229609960,0cyclictest0-21swapper/006:37:200
149960,0migration/09675-21gltestperf09:04:090
149960,0migration/06159-21fschecks_time09:59:110
149960,0migration/018487-21sh11:03:180
149960,0migration/018307-21ntp_kernel_err09:44:150
149960,0migration/016318-21sh10:41:320
8315050,0irq/26-0000:00:31470-21munin-node06:29:193
8315050,0irq/26-0000:00:18964-21ssh11:04:013
8315050,0irq/26-0000:00:0-21swapper/310:39:023
8315050,0irq/26-0000:00:0-21swapper/309:58:593
8315050,0irq/26-0000:00:0-21swapper/308:20:163
8315050,0irq/26-0000:00:0-21swapper/307:19:083
8315050,0irq/26-0000:00:0-21swapper/306:39:013
329950,0watchdog/30-21swapper/309:09:083
329950,0watchdog/30-21swapper/307:39:083
29935050,0irq/33-eth0-tx-9678-21sshd06:57:240
29935050,0irq/33-eth0-tx-6315-21iceccd09:08:192
29935050,0irq/33-eth0-tx-5345-21munin-node11:34:112
29935050,0irq/33-eth0-tx-32580-21munin-node10:34:122
29935050,0irq/33-eth0-tx-24414-21munin-node08:54:132
29935050,0irq/33-eth0-tx-18989-21munin-node11:04:122
29935050,0irq/33-eth0-tx-14469-21ssh10:21:322
29935050,0irq/33-eth0-tx-0-21swapper/211:01:052
29935050,0irq/33-eth0-tx-0-21swapper/210:48:312
29935050,0irq/33-eth0-tx-0-21swapper/209:00:322
29935050,0irq/33-eth0-tx-0-21swapper/208:38:102
29935050,0irq/33-eth0-tx-0-21swapper/007:09:180
29935050,0irq/33-eth0-tx-0-21swapper/006:59:230
259950,0migration/26599-21sh11:52:142
259950,0migration/230954-21munin-node06:29:182
259950,0migration/229100-21aten_r1power_cu11:29:082
259950,0migration/221868-21sh11:42:522
259950,0migration/217624-21kthreadcore06:39:132
259950,0migration/217182-21turbostat07:03:562
259950,0migration/212074-21sh11:19:022
259950,0migration/211734-21sh09:43:412
259950,0migration/211033-21sh11:16:182
249950,0watchdog/224155-21ssh10:29:082
249950,0watchdog/221633-21basename09:49:122
249950,0watchdog/214308-21cut08:04:162
229629955,0cyclictest0-21swapper/208:42:222
229629954,0cyclictest23871-21snmpd06:49:312
229629954,0cyclictest17548-21ssh09:25:522
229629950,0cyclictest0-21swapper/208:17:082
229629950,0cyclictest0-21swapper/207:09:022
229619955,0cyclictest0-21swapper/111:32:081
229619955,0cyclictest0-21swapper/109:58:191
229619955,0cyclictest0-21swapper/108:53:221
229619955,0cyclictest0-21swapper/108:44:231
229619955,0cyclictest0-21swapper/108:31:081
229619955,0cyclictest0-21swapper/108:02:361
229619955,0cyclictest0-21swapper/107:00:481
229619955,0cyclictest0-21swapper/106:30:511
229619950,0cyclictest0-21swapper/110:53:301
229619950,0cyclictest0-21swapper/109:25:131
229619950,0cyclictest0-21swapper/107:43:031
229609955,0cyclictest0-21swapper/011:52:480
229609955,0cyclictest0-21swapper/011:17:580
229609955,0cyclictest0-21swapper/011:06:080
229609955,0cyclictest0-21swapper/010:09:140
229609955,0cyclictest0-21swapper/009:35:580
229609955,0cyclictest0-21swapper/007:54:160
229609955,0cyclictest0-21swapper/007:30:180
229609950,0cyclictest0-21swapper/010:48:570
229609950,0cyclictest0-21swapper/008:36:500
229609950,0cyclictest0-21swapper/008:12:520
229609950,0cyclictest0-21swapper/007:26:030
229609950,0cyclictest0-21swapper/007:26:030
229609950,0cyclictest0-21swapper/007:15:580
179950,0migration/15877-21aten_r1power_cu08:24:081
179950,0migration/125902-21sh09:30:071
179950,0migration/123715-21munin-node08:34:121
179950,0migration/121569-21idleruntime11:24:121
179950,0migration/120253-21irqcore07:49:121
179950,0migration/117363-21taskset10:59:291
179950,0migration/112689-21kthreadcore07:19:151
179950,0migration/112568-21taskset10:00:441
149950,0migration/08922-21kthreadcore08:44:150
149950,0migration/08596-21kthreadcore11:34:160
149950,0migration/08562-21latency_hist10:38:550
149950,0migration/06768-21latency_hist10:18:550
149950,0migration/04882-21kthreadcore09:19:200
149950,0migration/03508-21sh09:39:070
149950,0migration/03151-21kthreadcore08:39:150
149950,0migration/03094-21munin-node07:34:210
149950,0migration/030740-21kthreadcore09:14:160
149950,0migration/030633-21munin-node08:54:200
149950,0migration/030331-21kthreadcore06:49:110
149950,0migration/028595-21kthreadcore06:24:150
149950,0migration/024850-21fschecks_time10:29:100
149950,0migration/023482-21missed_timers07:04:220
149950,0migration/023049-21kthreadcore09:49:140
149950,0migration/020587-21kthreadcore09:29:140
149950,0migration/020315-21kthreadcore08:49:140
149950,0migration/020212-21kthreadcore09:09:130
149950,0migration/019054-21aten_r1power_fr07:49:070
149950,0migration/016669-21kthreadcore11:39:120
149950,0migration/011832-21kthreadcore08:04:150
149950,0migration/011405-21kthreadcore10:19:140
29935040,0irq/33-eth0-tx-30534-21telnet06:29:120
29935040,0irq/33-eth0-tx-28170-21ssh11:09:092
29935040,0irq/33-eth0-tx-0-21swapper/211:44:442
29935040,0irq/33-eth0-tx-0-21swapper/210:49:002
29935040,0irq/33-eth0-tx-0-21swapper/210:14:302
29935040,0irq/33-eth0-tx-0-21swapper/210:05:492
29935040,0irq/33-eth0-tx-0-21swapper/209:44:562
29935040,0irq/33-eth0-tx-0-21swapper/209:34:122
29935040,0irq/33-eth0-tx-0-21swapper/209:29:122
29935040,0irq/33-eth0-tx-0-21swapper/208:49:052
29935040,0irq/33-eth0-tx-0-21swapper/007:44:170
259940,0migration/28956-21kthreadcore08:44:152
259940,0migration/28929-21aten_r1power_po10:39:082
259940,0migration/231214-21kthreadcore07:54:122
259940,0migration/229231-21aten_r1power_po09:54:092
259940,0migration/227163-21kthreadcore10:09:162
259940,0migration/224254-21kthreadcore07:29:152
259940,0migration/221307-21fschecks_time11:24:102
259940,0migration/217773-21kthreadcore10:24:162
259940,0migration/21671-21kthreadcore08:19:152
259940,0migration/216683-21kthreadcore07:24:182
259940,0migration/216683-21kthreadcore07:24:182
259940,0migration/214905-21fschecks_time08:29:102
249940,0watchdog/22630-21gltestperf09:19:162
249940,0watchdog/219494-21cut07:04:202
249940,0watchdog/20-21swapper/207:49:122
249940,0watchdog/20-21swapper/206:54:162
229629944,0cyclictest0-21swapper/207:14:232
229629943,0cyclictest6924-21awk08:24:122
229629943,0cyclictest3599-21lldpd06:37:132
229629943,0cyclictest26665-21awk09:14:132
229629943,0cyclictest23871-21snmpd08:01:312
229629943,0cyclictest12779-21awk07:44:142
229629943,0cyclictest0-21swapper/209:59:092
229619944,0cyclictest0-21swapper/109:15:361
229619944,0cyclictest0-21swapper/106:34:381
229619943,0cyclictest9927-21awk08:04:131
229619943,0cyclictest3599-21lldpd09:42:141
229619943,0cyclictest31899-21awk08:19:131
229619943,0cyclictest28718-21awk06:49:091
229619943,0cyclictest27375-21awk07:54:091
229619943,0cyclictest23871-21snmpd07:15:161
229619943,0cyclictest20040-21awk11:04:111
229619943,0cyclictest18607-21awk07:04:191
229619943,0cyclictest17147-21awk08:09:121
229619943,0cyclictest1613-21irqrtprio08:59:181
229619943,0cyclictest15714-21awk07:24:171
229619943,0cyclictest15714-21awk07:24:171
229619943,0cyclictest13192-21awk09:44:111
229609944,0cyclictest2092-21md1_raid111:20:390
229609943,0cyclictest30236-21awk11:29:130
229609943,0cyclictest2436-21awk07:59:130
229609943,0cyclictest23871-21snmpd08:27:160
179940,0migration/19053-21sensors_fan06:54:271
179940,0migration/14260-21fschecks_count11:14:101
179940,0migration/130925-21taskset10:29:361
179940,0migration/129746-21taskset10:11:351
179940,0migration/129027-21kthreadcore07:09:151
179940,0migration/127054-21kthreadcore08:54:171
179940,0migration/122699-21fschecks_time07:29:111
179940,0migration/122593-21kthreadcore06:44:151
179940,0migration/120743-21sh11:39:361
179940,0migration/119102-21kthreadcore10:04:151
179940,0migration/118843-21kthreadcore09:09:131
179940,0migration/116484-21kthreadcore06:39:131
179940,0migration/116432-21taskset09:07:231
169940,0watchdog/17069-21kthreadcore11:34:151
169940,0watchdog/15582-21diskmemload10:41:161
169940,0watchdog/12632-21gltestperf09:19:161
169940,0watchdog/117933-21grep10:24:161
169940,0watchdog/11679-21grep11:49:151
169940,0watchdog/10-21swapper/110:34:201
169940,0watchdog/10-21swapper/110:20:561
169940,0watchdog/10-21swapper/109:49:121
169940,0watchdog/10-21swapper/108:14:161
169940,0watchdog/10-21swapper/106:24:081
149940,0migration/032656-21aten2.4-expect08:59:110
149940,0migration/021208-21kthreadcore10:24:170
149940,0migration/015829-21kthreadcore06:39:130
4130,0ktimersoftd/08195-1kworker/0:2H08:14:170
249930,0watchdog/26386-21runrttasks06:44:122
249930,0watchdog/23010-21munin-node07:34:202
249930,0watchdog/20-21swapper/207:19:162
249930,0watchdog/20-21swapper/206:24:082
229629933,0cyclictest0-21swapper/208:09:062
229629930,0cyclictest5129-21perl07:39:102
229619932,0cyclictest5364-21grep08:39:221
229619932,0cyclictest31722-21grep10:14:111
229619932,0cyclictest30406-21awk07:34:161
229619932,0cyclictest13171-21awk11:19:131
229619932,0cyclictest12023-21sed07:44:101
229609933,0cyclictest8162-21kworker/0:207:19:320
229609933,0cyclictest16936-21kworker/0:209:54:240
229609933,0cyclictest16936-21kworker/0:209:24:120
229609933,0cyclictest13794-21kworker/0:011:24:480
229609933,0cyclictest13794-21kworker/0:011:09:100
229609932,0cyclictest22749-21tr11:44:110
229609932,0cyclictest20855-21grep06:44:110
229609932,0cyclictest20283-21sensors10:04:190
229609932,0cyclictest15228-21awk08:29:120
229609930,0cyclictest4695-21diskstats07:39:090
169930,0watchdog/10-21swapper/111:43:551
169930,0watchdog/10-21swapper/110:44:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional