You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-28 - 22:48
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Tue Oct 28, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24423995555,0cyclictest5104-21md1_resync08:49:412
2409924948,0sleep10-21swapper/106:18:311
2425123828,0sleep30-21swapper/306:20:133
2430923624,0sleep00-21swapper/006:20:540
2420423425,0sleep20-21swapper/206:19:442
8315070,0irq/26-0000:00:13329-21grep09:41:433
8315070,0irq/26-0000:00:0-21swapper/310:58:093
8315070,0irq/26-0000:00:0-21swapper/309:32:223
8315070,0irq/26-0000:00:0-21swapper/307:36:443
8315070,0irq/26-0000:00:0-21swapper/307:11:393
8315070,0irq/26-0000:00:0-21swapper/306:26:383
8315060,0irq/26-0000:00:9751-21ssh10:16:513
8315060,0irq/26-0000:00:5705-21memory09:36:443
8315060,0irq/26-0000:00:3932-21port_sshpriv10:51:453
8315060,0irq/26-0000:00:29300-21port_sshpriv11:06:453
8315060,0irq/26-0000:00:26265-21grep08:16:383
8315060,0irq/26-0000:00:18833-21perl08:11:363
8315060,0irq/26-0000:00:12033-21perl08:46:383
8315060,0irq/26-0000:00:10719-21perl08:01:373
8315060,0irq/26-0000:00:0-21swapper/311:22:403
8315060,0irq/26-0000:00:0-21swapper/311:11:373
8315060,0irq/26-0000:00:0-21swapper/310:36:403
8315060,0irq/26-0000:00:0-21swapper/310:34:483
8315060,0irq/26-0000:00:0-21swapper/310:06:273
8315060,0irq/26-0000:00:0-21swapper/309:23:433
8315060,0irq/26-0000:00:0-21swapper/309:15:223
8315060,0irq/26-0000:00:0-21swapper/309:02:383
8315060,0irq/26-0000:00:0-21swapper/308:41:393
8315060,0irq/26-0000:00:0-21swapper/308:21:413
8315060,0irq/26-0000:00:0-21swapper/307:56:413
8315060,0irq/26-0000:00:0-21swapper/307:55:373
8315060,0irq/26-0000:00:0-21swapper/307:47:333
8315060,0irq/26-0000:00:0-21swapper/307:01:443
8315060,0irq/26-0000:00:0-21swapper/306:56:333
8315060,0irq/26-0000:00:0-21swapper/306:52:483
8315060,0irq/26-0000:00:0-21swapper/306:21:423
4160,0ktimersoftd/00-21swapper/010:58:100
329960,0watchdog/328165-21sed06:41:513
329960,0watchdog/30-21swapper/311:46:383
329960,0watchdog/30-21swapper/311:40:063
329960,0watchdog/30-21swapper/311:27:593
329960,0watchdog/30-21swapper/310:48:393
329960,0watchdog/30-21swapper/310:26:433
329960,0watchdog/30-21swapper/310:11:313
329960,0watchdog/30-21swapper/310:01:353
329960,0watchdog/30-21swapper/309:52:193
329960,0watchdog/30-21swapper/309:47:273
329960,0watchdog/30-21swapper/309:26:273
329960,0watchdog/30-21swapper/309:17:313
329960,0watchdog/30-21swapper/308:56:553
329960,0watchdog/30-21swapper/308:51:233
329960,0watchdog/30-21swapper/308:28:433
329960,0watchdog/30-21swapper/308:07:233
329960,0watchdog/30-21swapper/307:17:393
329960,0watchdog/30-21swapper/307:11:113
329960,0watchdog/30-21swapper/306:47:433
329960,0watchdog/30-21swapper/306:37:553
329960,0watchdog/30-21swapper/306:31:353
29935060,0irq/33-eth0-tx-6972-21munin-node06:31:450
29935060,0irq/33-eth0-tx-12926-21munin-node07:41:450
29935060,0irq/33-eth0-tx-0-21swapper/008:16:420
29925060,0irq/32-eth0-rx-1940-21perl09:56:383
29925060,0irq/32-eth0-rx-14307-21hddtemp_smartct09:06:413
29925060,0irq/32-eth0-rx-0-21swapper/311:18:343
29925060,0irq/32-eth0-rx-0-21swapper/311:02:003
29925060,0irq/32-eth0-rx-0-21swapper/310:42:153
244239966,0cyclictest0-21swapper/210:12:312
244229966,0cyclictest0-21swapper/108:09:171
244219963,0cyclictest17956-21munin-run11:41:220
8315050,0irq/26-0000:00:0-21swapper/311:41:393
8315050,0irq/26-0000:00:0-21swapper/308:31:383
4150,0ktimersoftd/05105-21md0_resync09:21:350
4150,0ktimersoftd/00-21swapper/009:03:590
339950,0migration/33394-21kthreadcore11:31:423
339950,0migration/323205-21taskset07:26:323
329950,0watchdog/330845-21cpuspeed_turbos07:31:353
329950,0watchdog/313022-21ps10:21:433
329950,0watchdog/30-21swapper/308:36:273
329950,0watchdog/30-21swapper/307:41:233
329950,0watchdog/30-21swapper/307:21:313
29935050,0irq/33-eth0-tx-9904-21diskstats11:36:372
29935050,0irq/33-eth0-tx-290692sleep210:49:212
29935050,0irq/33-eth0-tx-16643-21munin-node09:46:472
29935050,0irq/33-eth0-tx-15115-21fschecks_count11:01:392
29935050,0irq/33-eth0-tx-1065-21munin-node07:11:470
29935050,0irq/33-eth0-tx-0-21swapper/208:56:562
29935050,0irq/33-eth0-tx-0-21swapper/208:32:432
29935050,0irq/33-eth0-tx-0-21swapper/008:21:430
27150,0ktimersoftd/20-21swapper/210:21:592
27150,0ktimersoftd/20-21swapper/206:54:112
259950,0migration/29251-21kthreadcore11:16:432
259950,0migration/28682-21needreboot07:56:422
259950,0migration/230617-21taskset11:09:282
259950,0migration/22980-21sh06:46:482
259950,0migration/225633-21sh11:43:292
259950,0migration/220653-21sendmail_mailst06:36:472
259950,0migration/219752-21kthreadcore08:11:402
259950,0migration/218728-21latency_hist07:01:232
259950,0migration/214130-21munin-node10:41:402
244239955,0cyclictest0-21swapper/211:47:452
244239955,0cyclictest0-21swapper/211:35:552
244239955,0cyclictest0-21swapper/211:21:252
244239955,0cyclictest0-21swapper/209:05:212
244239955,0cyclictest0-21swapper/208:03:332
244239955,0cyclictest0-21swapper/207:30:082
244239955,0cyclictest0-21swapper/207:19:342
244239955,0cyclictest0-21swapper/207:11:132
244239955,0cyclictest0-21swapper/206:23:082
244239954,0cyclictest6719-21munin-node09:21:362
244239954,0cyclictest22896-21sed06:41:432
244239950,0cyclictest0-21swapper/208:36:492
244229955,0cyclictest0-21swapper/111:46:131
244229955,0cyclictest0-21swapper/111:15:001
244229955,0cyclictest0-21swapper/110:50:431
244229955,0cyclictest0-21swapper/110:23:461
244229955,0cyclictest0-21swapper/110:03:441
244229955,0cyclictest0-21swapper/109:42:371
244229954,0cyclictest15635-21sh07:21:231
244229950,0cyclictest0-21swapper/110:34:011
244229950,0cyclictest0-21swapper/107:40:451
244229950,0cyclictest0-21swapper/107:07:481
244229950,0cyclictest0-21swapper/106:53:461
244229950,0cyclictest0-21swapper/106:29:571
244219955,0cyclictest807-21kworker/0:107:23:430
244219955,0cyclictest0-21swapper/011:46:330
244219955,0cyclictest0-21swapper/008:59:360
244219955,0cyclictest0-21swapper/007:10:490
244219950,0cyclictest0-21swapper/010:34:110
244219950,0cyclictest0-21swapper/009:13:470
244219950,0cyclictest0-21swapper/007:54:010
19150,0ktimersoftd/10-21swapper/111:23:031
19150,0ktimersoftd/10-21swapper/110:58:101
19150,0ktimersoftd/10-21swapper/109:02:531
179950,0migration/13020-21fschecks_count07:56:361
179950,0migration/126415-21irqcore09:51:401
179950,0migration/125200-21sh08:33:401
179950,0migration/120054-21taskset10:42:291
179950,0migration/118497-21kthreadcore09:26:401
179950,0migration/116030-21ksmtuned08:27:001
169950,0watchdog/12119-21ssh11:48:421
169950,0watchdog/110063-21ssh10:17:511
149950,0migration/08766-21kthreadcore10:36:400
149950,0migration/06013-21kthreadcore10:16:410
149950,0migration/05816-21kthreadcore08:41:410
149950,0migration/0373-21sendmail_mailst08:36:470
149950,0migration/031776-21irqcore07:31:390
149950,0migration/031669-21kthreadcore06:46:430
149950,0migration/028414-21taskset06:42:300
149950,0migration/028046-21kthreadcore06:21:420
149950,0migration/027829-21kthreadcore11:26:400
149950,0migration/027703-21kthreadcore11:06:410
149950,0migration/02691-21kthreadcore09:16:460
149950,0migration/025808-21kthreadcore07:26:410
149950,0migration/022151-21kthreadcore08:31:430
149950,0migration/021714-21kthreadcore11:41:400
149950,0migration/020195-21sh09:06:560
149950,0migration/02012-21sh10:14:320
149950,0migration/019693-21gltestperf07:01:370
149950,0migration/018824-21kthreadcore06:36:440
149950,0migration/017968-21sh10:06:220
149950,0migration/016107-21aten_r1power_po11:21:370
149950,0migration/016048-21kthreadcore10:21:440
149950,0migration/014691-21aten_r1power_vo11:01:370
149950,0migration/01272-21fschecks_count11:31:380
149950,0migration/010807-21fschecks_time08:01:370
29935040,0irq/33-eth0-tx-5105-21md0_resync09:38:282
29935040,0irq/33-eth0-tx-4642-21ssh10:52:302
29935040,0irq/33-eth0-tx-4030-21kthreadcore09:56:422
29935040,0irq/33-eth0-tx-2832-21ssh10:16:302
29935040,0irq/33-eth0-tx-0-21swapper/209:31:332
29935040,0irq/33-eth0-tx-0-21swapper/209:16:422
29935040,0irq/33-eth0-tx-0-21swapper/209:11:592
29935040,0irq/33-eth0-tx-0-21swapper/007:56:480
29935040,0irq/33-eth0-tx-0-21swapper/007:46:370
29935040,0irq/33-eth0-tx-0-21swapper/006:51:440
29935040,0irq/33-eth0-tx-0-21swapper/006:26:400
259940,0migration/28105-21latency_hist09:41:222
259940,0migration/27900-21kthreadcore10:56:432
259940,0migration/27464-21fschecks_count06:31:372
259940,0migration/25598-21kthreadcore08:41:412
259940,0migration/25281-21munin-node11:11:422
259940,0migration/232036-21aten_r1power_fr06:26:362
259940,0migration/228958-21kthreadcore08:16:432
259940,0migration/219770-21kthreadcore09:26:412
259940,0migration/218595-21latency_hist07:01:232
259940,0migration/216507-21fschecks_count07:21:372
259940,0migration/210655-21fschecks_time10:01:372
244239944,0cyclictest0-21swapper/210:31:552
244239944,0cyclictest0-21swapper/208:21:222
244239943,0cyclictest3599-21lldpd10:27:292
244239943,0cyclictest3599-21lldpd10:06:242
244239943,0cyclictest3599-21lldpd07:46:512
244239943,0cyclictest3262-21irqbalance11:26:562
244239943,0cyclictest3262-21irqbalance10:36:272
244239943,0cyclictest3262-21irqbalance07:51:572
244239943,0cyclictest23871-21snmpd08:51:492
244229944,0cyclictest0-21swapper/111:39:451
244229944,0cyclictest0-21swapper/108:46:581
244229944,0cyclictest0-21swapper/107:46:541
244229943,0cyclictest5768-21awk10:36:381
244229943,0cyclictest3599-21lldpd09:23:511
244229943,0cyclictest31248-21sadc11:11:221
244229943,0cyclictest26496-21sadc10:11:231
244229943,0cyclictest23871-21snmpd09:16:491
244229943,0cyclictest14457-21awk09:06:421
244219944,0cyclictest0-21swapper/010:41:450
244219943,0cyclictest3599-21lldpd11:16:440
244219943,0cyclictest3599-21lldpd08:51:530
244219943,0cyclictest26386-21awk09:51:390
244219943,0cyclictest24753-21awk09:31:410
244219943,0cyclictest19195-21awk08:11:380
244219943,0cyclictest12561-21awk06:56:400
244219940,0cyclictest0-21swapper/007:41:050
249940,0watchdog/25653-21kthreadcore07:11:432
249940,0watchdog/2299350irq/33-eth0-tx-09:54:312
249940,0watchdog/214001-21cut08:26:472
249940,0watchdog/20-21swapper/209:06:512
249940,0watchdog/20-21swapper/207:31:352
179940,0migration/17952-21runrttasks07:12:351
179940,0migration/1330-21mdstat10:11:461
179940,0migration/131544-21kernelversion10:51:411
179940,0migration/130600-21aten_r1power_en07:31:341
179940,0migration/129530-21sendmail_mailtr07:26:461
179940,0migration/129461-21fschecks_count06:46:381
179940,0migration/126928-21kthreadcore11:26:401
179940,0migration/126817-21kthreadcore06:21:411
179940,0migration/125884-21runrttasks07:06:141
179940,0migration/124507-21gltestperf09:31:391
179940,0migration/124356-21kthreadcore06:41:441
179940,0migration/121401-21munin-node10:26:491
179940,0migration/121097-21taskset06:37:341
179940,0migration/119069-21gltestperf08:11:371
179940,0migration/118322-21kthreadcore07:21:411
179940,0migration/112017-21kthreadcore08:01:411
169940,0watchdog/128519-21cut08:16:431
169940,0watchdog/126876-21ssh08:51:591
169940,0watchdog/122325-21df_inode09:11:391
169940,0watchdog/115967-21sh11:01:431
169940,0watchdog/11542-21kthreadcore09:36:431
169940,0watchdog/10-21swapper/109:56:431
169940,0watchdog/10-21swapper/108:36:431
169940,0watchdog/10-21swapper/107:54:511
149940,0migration/07144-21sh11:16:140
149940,0migration/027028-21kthreadcore10:46:420
149940,0migration/020666-21fschecks_time10:26:460
149940,0migration/011961-21fschecks_count08:46:380
244239932,0cyclictest5511-21cat07:36:232
244239932,0cyclictest3599-21lldpd08:08:032
244239932,0cyclictest16780-21grep07:41:422
244229933,0cyclictest3599-21lldpd08:22:021
244229932,0cyclictest8533-21awk11:16:411
244229932,0cyclictest17752-21sensors06:56:461
244229932,0cyclictest17427-21cut07:41:431
244219933,0cyclictest807-21kworker/0:108:06:370
244219933,0cyclictest807-21kworker/0:107:16:230
244219933,0cyclictest7012-21kworker/0:008:26:370
244219933,0cyclictest4288-21kworker/0:110:51:250
244219933,0cyclictest13175-21kworker/0:009:56:370
244219933,0cyclictest13175-21kworker/0:009:46:230
244219933,0cyclictest13175-21kworker/0:009:41:430
244219933,0cyclictest13175-21kworker/0:009:36:270
244219933,0cyclictest13175-21kworker/0:009:26:290
244219932,0cyclictest19401-21awk10:06:420
169930,0watchdog/15103-21kthreadcore11:31:431
169930,0watchdog/132594-21diskmemload09:47:031
169930,0watchdog/10-21swapper/108:56:391
169930,0watchdog/10-21swapper/108:41:271
169930,0watchdog/10-21swapper/106:31:431
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional