You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-29 - 17:22
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2.osadl.org (updated Sat Nov 29, 2025 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2362624847,0sleep10-21swapper/106:18:181
2360324233,0sleep30-21swapper/306:18:003
2340223928,0sleep00-21swapper/006:15:410
2361723727,0sleep20-21swapper/206:18:112
23795992121,0cyclictest31619-21sensors09:29:510
23795991515,0cyclictest18987-21sensors10:19:510
23795991313,0cyclictest3599-21lldpd08:55:150
2379599120,0cyclictest0-21swapper/008:20:510
339980,0migration/325364-21kthreadcore10:04:473
329970,0watchdog/311492-21taskset08:24:313
329970,0watchdog/30-21swapper/311:36:593
29925070,0irq/32-eth0-rx-0-21swapper/311:24:463
8315060,0irq/26-0000:00:30255-21awk08:54:272
4160,0ktimersoftd/00-21swapper/008:36:110
339960,0migration/35637-21sh08:57:333
339960,0migration/329968-21munin-node06:19:483
329960,0watchdog/325639-21kthreadcore08:09:473
329960,0watchdog/310657-21kthreadcore08:39:513
329960,0watchdog/30-21swapper/311:04:593
329960,0watchdog/30-21swapper/311:03:193
329960,0watchdog/30-21swapper/310:42:073
329960,0watchdog/30-21swapper/310:39:073
329960,0watchdog/30-21swapper/310:24:353
329960,0watchdog/30-21swapper/310:15:233
329960,0watchdog/30-21swapper/310:09:433
329960,0watchdog/30-21swapper/310:02:513
329960,0watchdog/30-21swapper/309:57:393
329960,0watchdog/30-21swapper/309:49:273
329960,0watchdog/30-21swapper/309:36:073
329960,0watchdog/30-21swapper/309:09:513
329960,0watchdog/30-21swapper/308:46:393
329960,0watchdog/30-21swapper/308:38:073
329960,0watchdog/30-21swapper/306:44:443
29935060,0irq/33-eth0-tx-0-21swapper/008:12:310
29935060,0irq/33-eth0-tx-0-21swapper/006:57:170
29925060,0irq/32-eth0-rx-26153-21perl09:29:433
29925060,0irq/32-eth0-rx-1454-21cut07:29:473
29925060,0irq/32-eth0-rx-0-21swapper/311:47:183
29925060,0irq/32-eth0-rx-0-21swapper/311:39:373
29925060,0irq/32-eth0-rx-0-21swapper/311:33:353
29925060,0irq/32-eth0-rx-0-21swapper/311:14:503
29925060,0irq/32-eth0-rx-0-21swapper/311:13:333
29925060,0irq/32-eth0-rx-0-21swapper/310:55:343
29925060,0irq/32-eth0-rx-0-21swapper/310:53:023
29925060,0irq/32-eth0-rx-0-21swapper/309:39:463
29925060,0irq/32-eth0-rx-0-21swapper/309:24:473
29925060,0irq/32-eth0-rx-0-21swapper/309:02:253
29925060,0irq/32-eth0-rx-0-21swapper/308:29:533
29925060,0irq/32-eth0-rx-0-21swapper/308:21:243
27160,0ktimersoftd/20-21swapper/210:33:192
259960,0migration/225681-21kthreadcore09:09:462
237979960,0cyclictest0-21swapper/211:45:272
149960,0migration/031731-21kthreadcore07:29:460
149960,0migration/019020-21sh11:38:440
339950,0migration/34784-21kthreadcore07:34:473
339950,0migration/323233-21kthreadcore07:24:463
339950,0migration/322035-21kthreadcore07:44:473
339950,0migration/320764-21kthreadcore09:04:493
339950,0migration/315372-21kthreadcore06:54:463
339950,0migration/314906-21irqcore08:04:443
339950,0migration/313833-21fschecks_count06:34:423
339950,0migration/311686-21kthreadcore07:59:473
329950,0watchdog/328098-21munin-node08:14:473
329950,0watchdog/325320-21munin-node07:49:433
329950,0watchdog/318521-21cpu09:44:433
329950,0watchdog/312544-21python07:14:523
329950,0watchdog/30-21swapper/311:19:273
329950,0watchdog/30-21swapper/310:31:233
329950,0watchdog/30-21swapper/309:14:313
329950,0watchdog/30-21swapper/308:49:433
329950,0watchdog/30-21swapper/307:55:233
329950,0watchdog/30-21swapper/307:20:283
329950,0watchdog/30-21swapper/307:10:323
329950,0watchdog/30-21swapper/307:00:163
329950,0watchdog/30-21swapper/306:50:563
329950,0watchdog/30-21swapper/306:39:403
329950,0watchdog/30-21swapper/306:31:483
329950,0watchdog/30-21swapper/306:25:323
29935050,0irq/33-eth0-tx-9261-21kthreadcore10:34:482
29935050,0irq/33-eth0-tx-8818-21munin-node11:14:492
29935050,0irq/33-eth0-tx-8019-21unixbench_multi09:34:562
29935050,0irq/33-eth0-tx-27132-21ssh10:06:542
29935050,0irq/33-eth0-tx-25303-21munin-node09:29:432
29935050,0irq/33-eth0-tx-23025-21sed10:44:412
29935050,0irq/33-eth0-tx-16153-21grep10:19:472
29935050,0irq/33-eth0-tx-1555-21ssh09:51:302
29935050,0irq/33-eth0-tx-13690-21munin-node08:04:530
29935050,0irq/33-eth0-tx-11534-21munin-node08:24:442
29935050,0irq/33-eth0-tx-0-21swapper/211:28:262
29935050,0irq/33-eth0-tx-0-21swapper/210:55:272
29935050,0irq/33-eth0-tx-0-21swapper/006:37:140
29935050,0irq/33-eth0-tx-0-21swapper/006:29:410
29925050,0irq/32-eth0-rx-0-21swapper/310:44:393
29925050,0irq/32-eth0-rx-0-21swapper/310:20:103
29925050,0irq/32-eth0-rx-0-21swapper/309:19:503
29925050,0irq/32-eth0-rx-0-21swapper/307:39:443
29925050,0irq/32-eth0-rx-0-21swapper/307:04:533
259950,0migration/28177-21sh11:13:282
259950,0migration/227507-21sh11:04:462
259950,0migration/222570-21sh11:00:482
259950,0migration/218109-21fschecks_time07:44:422
249950,0watchdog/23042-21aten_r1power_po08:19:392
237979955,0cyclictest0-21swapper/209:41:132
237979955,0cyclictest0-21swapper/209:14:342
237979955,0cyclictest0-21swapper/209:08:532
237979950,0cyclictest0-21swapper/209:24:492
237979950,0cyclictest0-21swapper/208:55:392
237969955,0cyclictest0-21swapper/109:46:321
237969955,0cyclictest0-21swapper/109:39:071
237969955,0cyclictest0-21swapper/109:30:361
237969955,0cyclictest0-21swapper/109:19:151
237969955,0cyclictest0-21swapper/109:11:511
237969950,0cyclictest0-21swapper/110:41:051
237969950,0cyclictest0-21swapper/109:23:501
237969950,0cyclictest0-21swapper/109:09:001
237959951,0cyclictest2572-21idleruntime-cro09:54:270
237959950,0cyclictest0-21swapper/010:08:300
237959950,0cyclictest0-21swapper/009:09:410
19150,0ktimersoftd/10-21swapper/110:33:191
19150,0ktimersoftd/10-21swapper/108:31:571
169950,0watchdog/14306-21kthreadcore07:09:481
149950,0migration/09977-21sh09:55:540
149950,0migration/08535-21kthreadcore08:39:510
149950,0migration/06013-21taskset07:13:330
149950,0migration/05267-21kthreadcore07:34:470
149950,0migration/03866-21sh10:14:090
149950,0migration/03811-21kthreadcore07:54:460
149950,0migration/03658-21taskset11:47:280
149950,0migration/030375-21fschecks_time10:29:420
149950,0migration/030295-21latency_hist08:54:270
149950,0migration/029544-21kthreadcore06:19:460
149950,0migration/029506-21kthreadcore06:44:490
149950,0migration/029037-21kthreadcore11:24:470
149950,0migration/026859-21kthreadcore10:24:470
149950,0migration/026105-21munin-node08:29:470
149950,0migration/025476-21kthreadcore10:44:460
149950,0migration/023355-21sh11:03:060
149950,0migration/022890-21sh09:08:170
149950,0migration/020769-21kthreadcore11:19:470
149950,0migration/014613-21hwlatdetect08:44:440
149950,0migration/01443-21kthreadcore06:24:450
149950,0migration/014323-21taskset10:56:250
149950,0migration/012834-21irqrtprio09:59:460
149950,0migration/011994-21kthreadcore07:14:470
149950,0migration/010848-21sh09:39:420
149950,0migration/010838-21kthreadcore08:59:470
149950,0migration/010546-21kthreadcore11:14:460
29935040,0irq/33-eth0-tx-4249-21munin-node10:14:422
29935040,0irq/33-eth0-tx-21950-21ssh10:24:422
29935040,0irq/33-eth0-tx-0-21swapper/210:39:382
29935040,0irq/33-eth0-tx-0-21swapper/210:09:272
29935040,0irq/33-eth0-tx-0-21swapper/209:59:522
29935040,0irq/33-eth0-tx-0-21swapper/209:54:402
29935040,0irq/33-eth0-tx-0-21swapper/209:44:482
29935040,0irq/33-eth0-tx-0-21swapper/209:19:312
29935040,0irq/33-eth0-tx-0-21swapper/209:00:292
29935040,0irq/33-eth0-tx-0-21swapper/208:44:312
29935040,0irq/33-eth0-tx-0-21swapper/208:39:512
29935040,0irq/33-eth0-tx-0-21swapper/008:14:510
259940,0migration/29022-21vmstat11:29:552
259940,0migration/232705-21irqrtprio06:24:452
259940,0migration/232014-21munin-node10:49:442
259940,0migration/229202-21kthreadcore06:19:462
259940,0migration/225735-21kthreadcore07:24:472
259940,0migration/225051-21kthreadcore08:29:472
259940,0migration/220701-21kthreadcore06:59:462
259940,0migration/216223-21sendmail_mailst07:39:492
259940,0migration/214517-21hwlatdetect07:19:452
259940,0migration/212617-21kthreadcore06:54:452
259940,0migration/211341-21fschecks_count11:34:422
259940,0migration/211289-21kthreadcore07:14:472
249940,0watchdog/26891-21perl06:29:442
249940,0watchdog/225914-21smartctl11:39:512
249940,0watchdog/224421-21munin-node07:49:432
249940,0watchdog/20-21swapper/208:09:472
249940,0watchdog/20-21swapper/208:04:432
237979943,0cyclictest7555-21awk07:59:442
237979943,0cyclictest32599-21awk07:09:452
237979943,0cyclictest29005-21awk07:29:432
237979943,0cyclictest23871-21snmpd07:58:492
237979943,0cyclictest23871-21snmpd07:37:182
237979943,0cyclictest23871-21snmpd06:49:332
237979943,0cyclictest23871-21snmpd06:42:182
237979943,0cyclictest23871-21snmpd06:34:482
237979943,0cyclictest18490-21awk11:19:452
237979942,0cyclictest0-21swapper/208:34:472
237969944,0cyclictest0-21swapper/111:13:051
237969944,0cyclictest0-21swapper/108:44:441
237969943,0cyclictest3599-21lldpd11:46:241
237969943,0cyclictest3599-21lldpd11:44:231
237969943,0cyclictest3599-21lldpd11:29:431
237969943,0cyclictest3599-21lldpd11:19:431
237969943,0cyclictest3599-21lldpd10:57:111
237969943,0cyclictest3599-21lldpd10:17:021
237969943,0cyclictest3599-21lldpd08:57:151
237969943,0cyclictest3599-21lldpd07:05:181
237969943,0cyclictest3599-21lldpd06:46:481
237969943,0cyclictest30772-21sadc10:49:261
237969943,0cyclictest29279-21sadc10:29:261
237969943,0cyclictest27909-21awk09:49:441
237969943,0cyclictest27205-21awk11:24:451
237969943,0cyclictest25227-21awk11:04:441
237969943,0cyclictest14823-21awk08:04:441
237969943,0cyclictest12051-21awk11:34:451
237969943,0cyclictest10966-21awk06:54:441
237959943,0cyclictest9375-21awk09:19:450
237959943,0cyclictest8625-21kworker/0:010:17:310
237959943,0cyclictest6697-21kworker/0:011:43:010
237959943,0cyclictest18141-21awk06:59:450
179940,0migration/19608-21fschecks_time11:14:431
179940,0migration/16730-21fschecks_time06:29:431
179940,0migration/14536-21kthreadcore07:34:471
179940,0migration/120809-21idleruntime10:04:431
179940,0migration/113458-21kthreadcore09:59:461
179940,0migration/113331-21kthreadcore08:24:451
179940,0migration/112070-21kthreadcore07:39:441
169940,0watchdog/16163-21taskset07:58:231
169940,0watchdog/16129-21diskstats10:34:431
169940,0watchdog/122858-21memory06:59:481
169940,0watchdog/122229-21cpu08:49:391
169940,0watchdog/118347-21kthreadcore09:24:471
169940,0watchdog/115700-21munin-node10:59:391
169940,0watchdog/11130-21kthreadcore08:34:471
169940,0watchdog/110386-21ssh08:22:071
169940,0watchdog/10-21swapper/109:54:471
169940,0watchdog/10-21swapper/108:09:471
169940,0watchdog/10-21swapper/107:49:471
149940,0migration/08164-21kthreadcore07:59:460
149940,0migration/0635-21idleruntime09:14:440
149940,0migration/03398-21if_eth1.211:29:450
149940,0migration/031309-21cpuspeed_turbos10:49:430
149940,0migration/02753-21irqcore09:34:470
149940,0migration/027297-21kthreadcore11:04:460
149940,0migration/02659-21kthreadcore11:09:460
149940,0migration/014698-21kthreadcore08:24:450
8315030,0irq/26-0000:00:0-21swapper/207:04:382
237979932,0cyclictest29275-21awk08:14:462
237979930,0cyclictest28656-21hddtemp_smartct06:44:462
237969933,0cyclictest3599-21lldpd09:39:491
237969932,0cyclictest7671-21awk08:59:441
237969932,0cyclictest7290-21grep08:39:501
237969932,0cyclictest3599-21lldpd06:26:111
237969932,0cyclictest2099-21processes10:09:501
237969932,0cyclictest18314-21awk07:44:441
237969932,0cyclictest13524-21grep10:19:431
237969931,0cyclictest32679-21kthreadcore10:49:481
237959933,0cyclictest8625-21kworker/0:009:44:310
237959933,0cyclictest8625-21kworker/0:009:24:290
237959933,0cyclictest8625-21kworker/0:007:49:280
237959933,0cyclictest8625-21kworker/0:007:44:280
237959933,0cyclictest8625-21kworker/0:007:39:280
237959933,0cyclictest8625-21kworker/0:007:24:280
237959933,0cyclictest8625-21kworker/0:007:19:420
237959933,0cyclictest8625-21kworker/0:007:04:320
237959933,0cyclictest8625-21kworker/0:006:49:320
237959933,0cyclictest8625-21kworker/0:006:39:280
237959933,0cyclictest6697-21kworker/0:010:39:290
237959933,0cyclictest20291-21kworker/0:210:34:270
179930,0migration/18544-21kthreadcore07:59:461
179930,0migration/15517-21kthreadcore06:49:461
179930,0migration/120954-21fschecks_count06:39:411
179930,0migration/115862-21kthreadcore06:34:461
169930,0watchdog/10-21swapper/108:14:351
169930,0watchdog/10-21swapper/107:29:271
169930,0watchdog/10-21swapper/107:24:471
169930,0watchdog/10-21swapper/107:19:441
169930,0watchdog/10-21swapper/107:15:001
237969920,0cyclictest0-21swapper/106:19:331
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional