You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 23:22
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2.osadl.org (updated Wed Nov 19, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1854424938,0sleep00-21swapper/006:17:140
1870824634,0sleep20-21swapper/206:19:082
1857223826,0sleep30-21swapper/306:17:373
1845423624,0sleep10-21swapper/106:16:111
329970,0watchdog/328670-21kthreadcore08:35:233
329970,0watchdog/321744-21fschecks_count10:10:193
329970,0watchdog/321217-21kthreadcore11:25:233
329970,0watchdog/30-21swapper/309:19:033
329970,0watchdog/30-21swapper/308:53:073
4160,0ktimersoftd/00-21swapper/010:21:070
339960,0migration/332556-21sh11:13:383
339960,0migration/326318-21sh11:47:143
339960,0migration/321327-21sh10:45:273
329960,0watchdog/316132-21memory09:25:233
329960,0watchdog/30-21swapper/311:38:513
329960,0watchdog/30-21swapper/311:22:353
329960,0watchdog/30-21swapper/311:01:593
329960,0watchdog/30-21swapper/310:56:193
329960,0watchdog/30-21swapper/310:38:033
329960,0watchdog/30-21swapper/310:25:593
329960,0watchdog/30-21swapper/310:16:593
329960,0watchdog/30-21swapper/310:07:513
329960,0watchdog/30-21swapper/310:03:313
329960,0watchdog/30-21swapper/309:50:153
329960,0watchdog/30-21swapper/308:55:553
329960,0watchdog/30-21swapper/308:20:553
329960,0watchdog/30-21swapper/307:54:433
329960,0watchdog/30-21swapper/307:30:233
329960,0watchdog/30-21swapper/307:08:523
329960,0watchdog/30-21swapper/306:25:323
29935060,0irq/33-eth0-tx-26927-21munin-node09:35:282
29935060,0irq/33-eth0-tx-22344-21munin-node10:30:222
29935060,0irq/33-eth0-tx-0-21swapper/007:06:460
29925060,0irq/32-eth0-rx-9313-21diskstats08:05:183
29925060,0irq/32-eth0-rx-3564-21lldpd07:57:023
29925060,0irq/32-eth0-rx-1805-21nfs_client09:55:253
29925060,0irq/32-eth0-rx-141732sleep308:28:493
29925060,0irq/32-eth0-rx-0-21swapper/311:07:153
29925060,0irq/32-eth0-rx-0-21swapper/309:49:413
29925060,0irq/32-eth0-rx-0-21swapper/309:40:183
29925060,0irq/32-eth0-rx-0-21swapper/309:33:443
29925060,0irq/32-eth0-rx-0-21swapper/309:09:453
29925060,0irq/32-eth0-rx-0-21swapper/309:00:453
29925060,0irq/32-eth0-rx-0-21swapper/308:45:593
29925060,0irq/32-eth0-rx-0-21swapper/308:40:223
29925060,0irq/32-eth0-rx-0-21swapper/307:40:253
188609960,0cyclictest0-21swapper/210:09:062
188599966,0cyclictest23871-21snmpd09:07:091
149960,0migration/05531-21munin-node08:40:270
149960,0migration/030903-21cpuspeed_turbos10:35:170
149960,0migration/025269-21sh11:28:110
149960,0migration/024719-21latency_hist11:10:020
149960,0migration/01542-21sh11:33:590
8315050,0irq/26-0000:00:5405-21munin-run10:20:022
339950,0migration/39310-21fschecks_time06:35:303
339950,0migration/327506-21kthreadcore10:50:213
339950,0migration/327230-21munin-node11:30:223
339950,0migration/323438-21latency09:10:233
339950,0migration/322243-21kthreadcore06:20:213
339950,0migration/31469-21kthreadcore06:50:223
329950,0watchdog/321707-21pluginstate06:40:283
329950,0watchdog/318476-21kthreadcore07:25:233
329950,0watchdog/30-21swapper/311:40:033
329950,0watchdog/30-21swapper/311:15:393
329950,0watchdog/30-21swapper/310:40:433
329950,0watchdog/30-21swapper/310:30:153
329950,0watchdog/30-21swapper/310:20:313
329950,0watchdog/30-21swapper/309:20:033
329950,0watchdog/30-21swapper/308:16:353
329950,0watchdog/30-21swapper/308:10:153
329950,0watchdog/30-21swapper/308:10:153
329950,0watchdog/30-21swapper/308:00:033
329950,0watchdog/30-21swapper/307:46:033
329950,0watchdog/30-21swapper/307:36:593
329950,0watchdog/30-21swapper/307:20:163
329950,0watchdog/30-21swapper/307:16:163
329950,0watchdog/30-21swapper/307:10:163
329950,0watchdog/30-21swapper/307:00:203
329950,0watchdog/30-21swapper/306:56:483
329950,0watchdog/30-21swapper/306:45:083
29935050,0irq/33-eth0-tx-8516-40yum-updatesd-he08:03:350
29935050,0irq/33-eth0-tx-31798-21munin-node10:55:212
29935050,0irq/33-eth0-tx-31491-21munin-node08:40:232
29935050,0irq/33-eth0-tx-31005-21ssh08:39:342
29935050,0irq/33-eth0-tx-23281-21munin-node08:15:160
29935050,0irq/33-eth0-tx-23154-21ssh11:05:402
29935050,0irq/33-eth0-tx-22983-21sed08:50:322
29935050,0irq/33-eth0-tx-0-21swapper/211:43:122
29935050,0irq/33-eth0-tx-0-21swapper/008:11:590
29935050,0irq/33-eth0-tx-0-21swapper/008:11:590
29925050,0irq/32-eth0-rx-3046-21grep06:30:263
29925050,0irq/32-eth0-rx-0-21swapper/309:35:113
29925050,0irq/32-eth0-rx-0-21swapper/308:30:253
259950,0migration/25418-21sh10:36:452
259950,0migration/229264-21kthreadcore09:15:232
259950,0migration/224831-21kernelversion08:15:212
259950,0migration/223461-21sh10:50:152
259950,0migration/217485-21latency_hist09:10:022
249950,0watchdog/231736-21iostat_ios08:20:232
188609955,0cyclictest0-21swapper/210:44:062
188609955,0cyclictest0-21swapper/209:56:202
188609955,0cyclictest0-21swapper/209:54:362
188609955,0cyclictest0-21swapper/209:26:042
188609950,0cyclictest0-21swapper/208:26:002
188609950,0cyclictest0-21swapper/207:40:072
188599955,0cyclictest0-21swapper/111:16:491
188599955,0cyclictest0-21swapper/109:15:161
188599955,0cyclictest0-21swapper/108:59:111
188599950,0cyclictest0-21swapper/110:24:191
188599950,0cyclictest0-21swapper/109:44:451
188599950,0cyclictest0-21swapper/108:33:291
188589955,0cyclictest0-21swapper/010:19:050
188589955,0cyclictest0-21swapper/007:50:340
188589954,0cyclictest5167-21kthreadcore09:00:240
179950,0migration/15500-21sh08:22:441
179950,0migration/128628-21fschecks_count09:55:171
179950,0migration/127775-21idleruntime07:55:181
179950,0migration/126000-21kthreadcore10:50:211
179950,0migration/12582-21kthreadcore10:15:261
179950,0migration/123697-21needreboot11:25:241
179950,0migration/122757-21taskset07:29:411
179950,0migration/118602-21munin-node09:10:191
179950,0migration/117558-21kthreadcore07:00:221
179950,0migration/115111-21kthreadcore10:05:221
179950,0migration/114729-21pmu-power07:20:261
179950,0migration/110136-21kthreadcore06:35:331
149950,0migration/09931-21irqcore07:20:210
149950,0migration/09703-21irqcore06:35:330
149950,0migration/08087-21sh11:17:010
149950,0migration/05155-21sh08:21:440
149950,0migration/04232-21latency_hist10:00:020
149950,0migration/032100-21kthreadcore09:35:260
149950,0migration/031228-21kthreadcore07:10:250
149950,0migration/031001-21gltestperf06:50:180
149950,0migration/030665-21taskset08:38:360
149950,0migration/028047-21taskset09:55:050
149950,0migration/025898-21kthreadcore06:45:240
149950,0migration/025790-21taskset06:22:150
149950,0migration/024971-21taskset09:12:570
149950,0migration/022614-21kthreadcore09:30:230
149950,0migration/019671-21munin-node11:45:190
149950,0migration/017636-21kthreadcore07:45:220
149950,0migration/017389-21hwlatdetect11:05:200
149950,0migration/016383-21kthreadcore08:30:240
149950,0migration/01592-21cpuspeed_turbos07:15:170
149950,0migration/015002-21gltestperf10:25:210
149950,0migration/01428-21kthreadcore10:55:230
149950,0migration/012909-21kthreadcore09:25:220
149950,0migration/012759-21mdstat10:40:230
149950,0migration/011744-21munin-node09:45:210
149950,0migration/011562-21kthreadcore09:05:270
149950,0migration/011194-21kthreadcore08:05:240
149950,0migration/010163-21kthreadcore11:00:260
29935040,0irq/33-eth0-tx-836-21munin-node11:15:192
29935040,0irq/33-eth0-tx-0-21swapper/211:45:162
29935040,0irq/33-eth0-tx-0-21swapper/211:31:012
29935040,0irq/33-eth0-tx-0-21swapper/211:20:352
29935040,0irq/33-eth0-tx-0-21swapper/211:10:032
29935040,0irq/33-eth0-tx-0-21swapper/211:00:112
29935040,0irq/33-eth0-tx-0-21swapper/210:20:322
29935040,0irq/33-eth0-tx-0-21swapper/210:10:182
29935040,0irq/33-eth0-tx-0-21swapper/209:45:072
29935040,0irq/33-eth0-tx-0-21swapper/209:00:222
29935040,0irq/33-eth0-tx-0-21swapper/208:30:142
29935040,0irq/33-eth0-tx-0-21swapper/007:35:160
29935040,0irq/33-eth0-tx-0-21swapper/006:55:200
259940,0migration/29291-21aten_r1power_cu09:05:162
259940,0migration/28701-21kthreadcore09:40:242
259940,0migration/227765-21idleruntime07:55:182
259940,0migration/22542-21irqrtprio06:30:262
259940,0migration/220621-21kernelversion06:20:202
259940,0migration/215852-21fschecks_time10:45:202
259940,0migration/215527-21irqrtprio10:25:252
249940,0watchdog/27249-21kthreadcore10:00:232
249940,0watchdog/20-21swapper/207:05:242
188609943,0cyclictest3599-21lldpd09:30:112
188609943,0cyclictest3599-21lldpd09:20:102
188609943,0cyclictest3599-21lldpd08:09:322
188609943,0cyclictest3599-21lldpd07:00:522
188609943,0cyclictest3314-21awk11:35:212
188609943,0cyclictest3262-21irqbalance11:25:172
188609943,0cyclictest3262-21irqbalance08:45:072
188609943,0cyclictest3262-21irqbalance07:46:572
188609943,0cyclictest3262-21irqbalance07:35:072
188609943,0cyclictest3262-21irqbalance07:30:172
188609943,0cyclictest3262-21irqbalance07:25:172
188609943,0cyclictest3262-21irqbalance07:20:172
188609943,0cyclictest3262-21irqbalance07:11:072
188609943,0cyclictest3262-21irqbalance06:55:172
188609943,0cyclictest3262-21irqbalance06:50:072
188609943,0cyclictest3262-21irqbalance06:45:072
188609943,0cyclictest3262-21irqbalance06:40:172
188609943,0cyclictest3262-21irqbalance06:35:172
188609943,0cyclictest3262-21irqbalance06:25:082
188609943,0cyclictest2682-21awk08:00:192
188609943,0cyclictest2412-21awk07:15:202
188609943,0cyclictest20580-21awk07:50:222
188609940,0cyclictest24825-21munin-node08:55:242
188599943,0cyclictest7681-21awk08:25:201
188599943,0cyclictest401-21awk10:55:211
188599943,0cyclictest3599-21lldpd11:22:171
188599943,0cyclictest3599-21lldpd10:43:031
188599943,0cyclictest3599-21lldpd09:25:401
188599943,0cyclictest3599-21lldpd07:51:321
188599943,0cyclictest3417-21awk11:35:211
188599943,0cyclictest23871-21snmpd11:42:371
188599943,0cyclictest23871-21snmpd07:30:201
188599943,0cyclictest23871-21snmpd06:40:201
188599943,0cyclictest1839-21awk09:00:221
188589944,0cyclictest0-21swapper/008:47:480
188589944,0cyclictest0-21swapper/006:30:290
188589943,0cyclictest6176-21awk07:40:220
188589943,0cyclictest3262-21irqbalance09:20:070
188589943,0cyclictest3262-21irqbalance08:52:570
188589943,0cyclictest27871-21awk07:55:190
188589943,0cyclictest23871-21snmpd10:05:060
188589940,0cyclictest22810-21munin-run07:30:030
179940,0migration/18675-21fschecks_time08:45:181
179940,0migration/17895-21taskset11:00:091
179940,0migration/15223-21cpuspeed_turbos06:55:161
179940,0migration/132733-21timerwakeupswit06:25:481
179940,0migration/126976-21aten_r1power_en09:35:181
179940,0migration/125078-21kthreadcore10:30:241
179940,0migration/124443-21kthreadcore07:05:241
179940,0migration/123697-21kthreadcore06:20:221
179940,0migration/123348-21diskmemload08:51:191
179940,0migration/121479-21cpuspeed_turbos10:10:181
179940,0migration/121316-21kthreadcore09:30:231
179940,0migration/11948-21kthreadcore07:35:221
179940,0migration/117062-21hwlatdetect08:10:221
179940,0migration/117062-21hwlatdetect08:10:221
179940,0migration/114528-21fschecks_count10:25:211
16151240,0sleep20-21swapper/208:10:192
16151240,0sleep20-21swapper/208:10:192
169940,0watchdog/16457-21irqcore07:40:231
169940,0watchdog/130141-21aten_r1power_fr06:50:161
169940,0watchdog/128671-21kthreadcore08:35:231
169940,0watchdog/124823-21cut06:45:241
169940,0watchdog/12157-21kthreadcore10:35:231
169940,0watchdog/121016-21sed09:50:231
169940,0watchdog/120530-21kthreadcore11:05:231
169940,0watchdog/10-21swapper/108:15:191
149940,0migration/09308-21sh11:20:080
149940,0migration/017236-21timerandwakeup11:40:330
188599933,0cyclictest3599-21lldpd08:01:321
188599933,0cyclictest2278-21awk06:30:251
188599932,0cyclictest6386-21runrttasks10:01:011
188599932,0cyclictest2962-21awk09:20:221
188599932,0cyclictest12379-21basename07:45:151
188599932,0cyclictest12293-21diskmemload09:45:231
188599932,0cyclictest10429-21irqstats08:05:231
188599931,0cyclictest5240-21kthreadcore07:15:221
188589933,0cyclictest7572-21kworker/0:111:35:030
188589933,0cyclictest7572-21kworker/0:110:50:030
188589933,0cyclictest7572-21kworker/0:110:45:150
188589933,0cyclictest7572-21kworker/0:110:30:050
188589933,0cyclictest7572-21kworker/0:110:10:070
188589933,0cyclictest7572-21kworker/0:109:50:090
188589933,0cyclictest7572-21kworker/0:109:40:050
188589933,0cyclictest7572-21kworker/0:109:15:170
188589933,0cyclictest7572-21kworker/0:108:55:070
188589933,0cyclictest28906-21kworker/0:208:25:060
188589933,0cyclictest28906-21kworker/0:207:00:040
188589933,0cyclictest28906-21kworker/0:206:40:060
188589933,0cyclictest28906-21kworker/0:206:25:060
188589932,0cyclictest3599-21lldpd07:30:170
179930,0migration/12889-21kthreadcore08:40:221
179930,0migration/128818-21kthreadcore11:30:251
179930,0migration/124858-21aten_r1power_cu11:10:151
169930,0watchdog/10-21swapper/111:46:391
169930,0watchdog/10-21swapper/110:45:231
169930,0watchdog/10-21swapper/107:10:041
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional