You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-19 - 14:37
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2.osadl.org (updated Sun Oct 19, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3084625140,0sleep00-21swapper/006:19:430
3074523728,0sleep30-21swapper/306:18:343
3098223626,0sleep20-21swapper/206:21:172
2408123526,0sleep10-21swapper/106:17:071
329970,0watchdog/329702-21kthreadcore10:27:203
329970,0watchdog/30-21swapper/309:15:043
29925070,0irq/32-eth0-rx-0-21swapper/311:50:343
8315060,0irq/26-0000:00:9653-21crond06:51:553
8315060,0irq/26-0000:00:5897-21grep07:52:203
8315060,0irq/26-0000:00:27751-21crond07:26:553
339960,0migration/318224-21munin-node06:57:123
329960,0watchdog/3299250irq/32-eth0-rx-09:22:003
329960,0watchdog/30-21swapper/311:38:233
329960,0watchdog/30-21swapper/311:28:193
329960,0watchdog/30-21swapper/311:06:553
329960,0watchdog/30-21swapper/310:59:353
329960,0watchdog/30-21swapper/310:33:123
329960,0watchdog/30-21swapper/310:22:283
329960,0watchdog/30-21swapper/310:18:083
329960,0watchdog/30-21swapper/310:07:203
329960,0watchdog/30-21swapper/310:03:283
329960,0watchdog/30-21swapper/309:59:523
329960,0watchdog/30-21swapper/309:47:163
329960,0watchdog/30-21swapper/309:42:283
329960,0watchdog/30-21swapper/309:37:163
329960,0watchdog/30-21swapper/309:32:123
329960,0watchdog/30-21swapper/309:03:123
329960,0watchdog/30-21swapper/308:58:363
329960,0watchdog/30-21swapper/308:39:363
329960,0watchdog/30-21swapper/308:22:363
329960,0watchdog/30-21swapper/307:02:163
329960,0watchdog/30-21swapper/306:40:203
329960,0watchdog/30-21swapper/306:22:163
311069965,0cyclictest0-21swapper/211:42:112
29935060,0irq/33-eth0-tx-20657-40yum-updatesd-he08:04:110
29935060,0irq/33-eth0-tx-13593-21munin-node07:17:230
29925060,0irq/32-eth0-rx-400-21kthreadcore10:52:153
29925060,0irq/32-eth0-rx-3576-21cut09:52:143
29925060,0irq/32-eth0-rx-13652-21irqstats08:42:213
29925060,0irq/32-eth0-rx-0-21swapper/311:23:253
29925060,0irq/32-eth0-rx-0-21swapper/311:06:543
29925060,0irq/32-eth0-rx-0-21swapper/310:49:073
29925060,0irq/32-eth0-rx-0-21swapper/310:39:573
29925060,0irq/32-eth0-rx-0-21swapper/309:21:263
29925060,0irq/32-eth0-rx-0-21swapper/309:07:153
29925060,0irq/32-eth0-rx-0-21swapper/308:32:363
29925060,0irq/32-eth0-rx-0-21swapper/308:27:103
149960,0migration/027109-21unixbench-2d08:47:240
149960,0migration/023843-21kthreadcore07:22:150
149960,0migration/023223-21timerwakeupswit10:02:240
8315050,0irq/26-0000:00:10557-21kthreadcore10:57:162
339950,0migration/37351-21fschecks_count07:57:093
339950,0migration/36517-21aten_r1power_vo07:12:083
339950,0migration/33910-21idleruntime07:32:133
339950,0migration/323823-21kthreadcore11:42:163
339950,0migration/319090-21kthreadcore08:02:123
329950,0watchdog/332747-21kthreadcore07:27:163
329950,0watchdog/319801-21munin-node07:17:203
329950,0watchdog/30-21swapper/311:17:033
329950,0watchdog/30-21swapper/311:11:593
329950,0watchdog/30-21swapper/310:43:113
329950,0watchdog/30-21swapper/310:12:123
329950,0watchdog/30-21swapper/308:52:003
329950,0watchdog/30-21swapper/308:47:043
329950,0watchdog/30-21swapper/308:07:203
329950,0watchdog/30-21swapper/307:47:043
329950,0watchdog/30-21swapper/307:42:243
329950,0watchdog/30-21swapper/307:37:123
329950,0watchdog/30-21swapper/307:07:243
329950,0watchdog/30-21swapper/306:55:323
329950,0watchdog/30-21swapper/306:32:203
329950,0watchdog/30-21swapper/306:28:083
311069955,0cyclictest0-21swapper/210:41:202
311069955,0cyclictest0-21swapper/208:25:072
311069950,0cyclictest0-21swapper/211:28:192
311059955,0cyclictest10181-21diskmemload11:28:141
311059955,0cyclictest0-21swapper/110:58:161
311059955,0cyclictest0-21swapper/108:07:131
311059950,0cyclictest0-21swapper/109:17:411
311059950,0cyclictest0-21swapper/108:58:181
311059950,0cyclictest0-21swapper/108:54:591
311049954,0cyclictest17102-21ssh11:39:160
311049954,0cyclictest10358-21grep10:17:080
29935050,0irq/33-eth0-tx-4860-21aten_r1power_en08:57:072
29935050,0irq/33-eth0-tx-21548-21munin-node09:07:112
29935050,0irq/33-eth0-tx-17912-21munin-node10:22:162
29935050,0irq/33-eth0-tx-15631-21interrupts09:22:122
29935050,0irq/33-eth0-tx-0-21swapper/211:24:072
29935050,0irq/33-eth0-tx-0-21swapper/211:03:212
29935050,0irq/33-eth0-tx-0-21swapper/210:46:352
29935050,0irq/33-eth0-tx-0-21swapper/209:58:342
29935050,0irq/33-eth0-tx-0-21swapper/208:40:032
29935050,0irq/33-eth0-tx-0-21swapper/007:58:120
29935050,0irq/33-eth0-tx-0-21swapper/007:52:170
29935050,0irq/33-eth0-tx-0-21swapper/006:37:130
29925050,0irq/32-eth0-rx-5431-21kthreadcore08:17:173
29925050,0irq/32-eth0-rx-0-21swapper/311:32:073
29925050,0irq/32-eth0-rx-0-21swapper/309:27:133
29925050,0irq/32-eth0-rx-0-21swapper/308:12:173
29925050,0irq/32-eth0-rx-0-21swapper/306:42:093
27150,0ktimersoftd/20-21swapper/211:15:322
259950,0migration/29305-21sh11:36:422
259950,0migration/24459-21sh09:12:422
259950,0migration/230405-21latency07:47:272
259950,0migration/221542-21gltestperf06:37:142
259950,0migration/217720-21sh11:40:542
259950,0migration/212690-21fschecks_count08:42:172
259950,0migration/21131-21irqcore09:52:132
249950,0watchdog/228926-21grep08:12:122
249950,0watchdog/222888-21proc_pri10:02:202
179950,0migration/17493-21kthreadcore07:32:161
179950,0migration/13758-21kthreadcore06:22:181
179950,0migration/131689-21unixbench_singl10:07:221
179950,0migration/127876-21fschecks_time11:47:091
179950,0migration/12434-21munin-node11:32:121
179950,0migration/124211-21kthreadcore08:27:171
179950,0migration/120939-21aten_r1power_vo10:47:071
179950,0migration/115226-21kthreadcore11:37:141
179950,0migration/113583-21kthreadcore09:57:171
179950,0migration/112318-21kthreadcore09:37:151
179950,0migration/112140-21kthreadcore08:22:161
179950,0migration/111148-21sh10:37:121
169950,0watchdog/127104-21perl10:27:161
169950,0watchdog/10-21swapper/110:12:241
149950,0migration/0999-21kthreadcore11:12:130
149950,0migration/08640-21sh11:35:000
149950,0migration/06499-21aten_r1power_vo07:12:080
149950,0migration/05991-21kthreadcore08:17:170
149950,0migration/05690-21kthreadcore10:32:140
149950,0migration/05415-21aten_r1power_fr10:57:090
149950,0migration/04942-21kthreadcore07:32:160
149950,0migration/03876-21kthreadcore07:07:120
149950,0migration/030589-21kthreadcore07:27:150
149950,0migration/029669-21sh09:28:340
149950,0migration/027656-21kthreadcore07:47:260
149950,0migration/026645-21kthreadcore10:47:130
149950,0migration/025884-21kthreadcore07:02:130
149950,0migration/025258-21kthreadcore11:07:150
149950,0migration/024951-21timerwakeupswit11:42:230
149950,0migration/024352-21switchtime10:22:220
149950,0migration/023812-21taskset09:47:050
149950,0migration/020884-21kthreadcore06:57:140
149950,0migration/016783-21kthreadcore11:02:130
149950,0migration/014841-21unixbench_multi09:57:240
149950,0migration/013866-21taskset11:17:500
311069944,0cyclictest0-21swapper/206:56:592
311069943,0cyclictest529-21awk07:52:112
311069943,0cyclictest3599-21lldpd09:27:222
311069943,0cyclictest28850-21grep07:27:122
311069943,0cyclictest25324-21awk07:02:112
311069943,0cyclictest23871-21snmpd08:09:492
311069943,0cyclictest23871-21snmpd08:06:492
311069943,0cyclictest23871-21snmpd07:34:192
311069943,0cyclictest23871-21snmpd07:19:192
311069943,0cyclictest23871-21snmpd06:52:332
311069940,0cyclictest19516-21grep08:27:122
311059943,0cyclictest32536-21awk07:07:091
311059943,0cyclictest31187-21awk09:12:111
311059943,0cyclictest30190-21awk10:52:121
311059943,0cyclictest29045-21awk07:27:131
311059943,0cyclictest23871-21snmpd11:12:201
311059943,0cyclictest23871-21snmpd09:08:351
311059943,0cyclictest23871-21snmpd08:42:351
311059943,0cyclictest23871-21snmpd08:37:351
311059943,0cyclictest23871-21snmpd08:32:341
311059943,0cyclictest23871-21snmpd08:17:341
311059943,0cyclictest23871-21snmpd07:54:341
311059943,0cyclictest23871-21snmpd07:48:191
311059943,0cyclictest23871-21snmpd07:03:341
311059943,0cyclictest23871-21snmpd06:31:481
311059943,0cyclictest21649-21awk06:37:151
311049943,0cyclictest840-21awk09:52:120
311049943,0cyclictest3599-21lldpd09:12:360
311049943,0cyclictest29700-21sadc09:11:550
311049943,0cyclictest28313-21awk11:47:110
311049943,0cyclictest23871-21snmpd11:26:050
311049943,0cyclictest22202-21awk08:07:110
311049943,0cyclictest19991-21awk08:27:150
311049943,0cyclictest16517-21awk09:42:120
311049943,0cyclictest14325-21awk09:02:120
311049943,0cyclictest13141-21awk08:42:190
311049943,0cyclictest11259-21awk07:37:190
311049940,0cyclictest0-21swapper/010:27:410
311049940,0cyclictest0-21swapper/009:35:300
29935040,0irq/33-eth0-tx-27618-21fschecks_count11:47:092
29935040,0irq/33-eth0-tx-13145-21ssh09:01:572
29935040,0irq/33-eth0-tx-10420-21fschecks_time10:17:082
29935040,0irq/33-eth0-tx-0-21swapper/211:17:002
29935040,0irq/33-eth0-tx-0-21swapper/211:07:172
29935040,0irq/33-eth0-tx-0-21swapper/210:52:062
29935040,0irq/33-eth0-tx-0-21swapper/210:27:192
29935040,0irq/33-eth0-tx-0-21swapper/210:07:022
29935040,0irq/33-eth0-tx-0-21swapper/209:41:592
29935040,0irq/33-eth0-tx-0-21swapper/209:37:192
29935040,0irq/33-eth0-tx-0-21swapper/209:34:072
29935040,0irq/33-eth0-tx-0-21swapper/209:19:462
29935040,0irq/33-eth0-tx-0-21swapper/208:52:242
29935040,0irq/33-eth0-tx-0-21swapper/208:32:362
29935040,0irq/33-eth0-tx-0-21swapper/008:12:100
29935040,0irq/33-eth0-tx-0-21swapper/006:52:100
27140,0ktimersoftd/22123-21jbd2/md0-807:42:002
259940,0migration/29301-21kthreadcore07:12:142
259940,0migration/26967-21kthreadcore06:47:172
259940,0migration/26324-21kthreadcore08:17:172
259940,0migration/2508-21kthreadcore06:42:152
259940,0migration/24247-21kthreadcore10:32:132
259940,0migration/232381-21fschecks_time07:07:072
259940,0migration/225536-21kthreadcore09:47:142
259940,0migration/223232-21kthreadcore10:47:122
259940,0migration/222720-21kthreadcore08:47:152
259940,0migration/220034-21runrttasks06:33:162
249940,0watchdog/27753-21irqstats06:27:122
249940,0watchdog/231655-21df06:22:122
249940,0watchdog/21856-21fschecks_time10:12:122
249940,0watchdog/20-21swapper/207:22:162
179940,0migration/19569-21kthreadcore07:12:141
179940,0migration/19301-21runrttasks06:47:521
179940,0migration/17524-21kthreadcore10:32:141
179940,0migration/129640-21kthreadcore06:42:141
179940,0migration/123953-21kthreadcore07:22:151
179940,0migration/123834-21runrttasks10:03:571
179940,0migration/123546-21unixbench_multi06:57:231
179940,0migration/12211-21kthreadcore09:32:171
179940,0migration/115134-21kthreadcore06:52:171
179940,0migration/114745-21if_eth1.207:17:131
179940,0migration/111316-21kthreadcore07:57:141
169940,0watchdog/128973-21kthreadcore09:47:161
169940,0watchdog/123772-21if_eth1.309:27:121
169940,0watchdog/121348-21perl08:47:121
169940,0watchdog/117812-21kthreadcore08:02:121
169940,0watchdog/112285-21kthreadcore10:17:121
169940,0watchdog/111668-21irqstats07:37:201
169940,0watchdog/10-21swapper/111:23:191
169940,0watchdog/10-21swapper/111:08:471
169940,0watchdog/10-21swapper/111:02:111
169940,0watchdog/10-21swapper/109:42:161
169940,0watchdog/10-21swapper/108:12:121
169940,0watchdog/10-21swapper/106:32:121
149940,0migration/07752-21irqrtprio06:27:120
149940,0migration/032135-21kthreadcore10:52:140
149940,0migration/027786-21latency_hist06:41:550
149940,0migration/020464-21sh10:46:490
149940,0migration/019597-21sensors_temp06:32:210
149940,0migration/01405-21python08:32:190
149940,0migration/011114-21fschecks_time08:22:120
311059933,0cyclictest0-21swapper/107:42:281
311059932,0cyclictest23871-21snmpd09:02:051
311059930,0cyclictest7921-21perl11:17:131
311059930,0cyclictest427-21perl09:52:091
311049933,0cyclictest8772-21kworker/0:206:47:000
311049933,0cyclictest8772-21kworker/0:206:22:000
311049933,0cyclictest7710-21kworker/0:210:07:240
311049933,0cyclictest7710-21kworker/0:209:36:580
311049933,0cyclictest7710-21kworker/0:209:17:280
311049933,0cyclictest7710-21kworker/0:208:56:580
311049933,0cyclictest7710-21kworker/0:208:52:220
311049933,0cyclictest23287-21kworker/0:007:42:040
311049933,0cyclictest1943-21kworker/0:010:37:000
311049933,0cyclictest1943-21kworker/0:010:12:320
311049933,0cyclictest10168-21kworker/0:111:27:260
311049930,0cyclictest4609-21perl08:37:130
311049930,0cyclictest15221-21perl09:22:100
249930,0watchdog/26386-21runrttasks07:37:202
249930,0watchdog/20-21swapper/207:57:122
169930,0watchdog/10-21swapper/111:42:111
169930,0watchdog/10-21swapper/110:43:031
169930,0watchdog/10-21swapper/110:22:081
169930,0watchdog/10-21swapper/109:22:001
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional