You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-30 - 01:26
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2.osadl.org (updated Sun Nov 30, 2025 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1957225243,0sleep10-21swapper/118:14:261
2669923727,0sleep30-21swapper/318:19:213
2637523728,0sleep20-21swapper/218:15:352
2643523423,0sleep00-21swapper/018:16:130
329980,0watchdog/332585-21kthreadcore20:14:463
329980,0watchdog/332585-21kthreadcore20:14:463
329970,0watchdog/32197-21grep18:24:423
329970,0watchdog/30-21swapper/320:34:383
29925070,0irq/32-eth0-rx-15356-21kthreadcore22:54:443
8315060,0irq/26-0000:00:1841-21dump-pmu-power19:54:252
8315060,0irq/26-0000:00:10078-21crond20:59:242
8315060,0irq/26-0000:00:0-21swapper/318:39:253
339960,0migration/38843-21sh22:30:053
339960,0migration/321253-21sh21:42:273
329960,0watchdog/313471-21kthreadcore23:29:453
329960,0watchdog/30-21swapper/323:45:333
329960,0watchdog/30-21swapper/323:39:493
329960,0watchdog/30-21swapper/323:36:173
329960,0watchdog/30-21swapper/323:24:253
329960,0watchdog/30-21swapper/323:19:493
329960,0watchdog/30-21swapper/323:16:293
329960,0watchdog/30-21swapper/323:03:133
329960,0watchdog/30-21swapper/322:53:093
329960,0watchdog/30-21swapper/322:34:453
329960,0watchdog/30-21swapper/322:17:373
329960,0watchdog/30-21swapper/322:07:413
329960,0watchdog/30-21swapper/321:59:413
329960,0watchdog/30-21swapper/321:35:413
329960,0watchdog/30-21swapper/321:15:573
329960,0watchdog/30-21swapper/321:09:533
329960,0watchdog/30-21swapper/320:39:463
329960,0watchdog/30-21swapper/320:09:263
329960,0watchdog/30-21swapper/319:45:543
329960,0watchdog/30-21swapper/318:20:103
29935060,0irq/33-eth0-tx-5675-21munin-node19:34:440
29935060,0irq/33-eth0-tx-0-21swapper/020:10:290
29925060,0irq/32-eth0-rx-27089-21iostat_ios22:24:423
29925060,0irq/32-eth0-rx-21692-21kthreadcore22:19:433
29925060,0irq/32-eth0-rx-0-21swapper/322:47:073
29925060,0irq/32-eth0-rx-0-21swapper/322:41:343
29925060,0irq/32-eth0-rx-0-21swapper/322:12:593
29925060,0irq/32-eth0-rx-0-21swapper/321:58:563
29925060,0irq/32-eth0-rx-0-21swapper/321:48:033
29925060,0irq/32-eth0-rx-0-21swapper/321:34:113
29925060,0irq/32-eth0-rx-0-21swapper/321:24:463
29925060,0irq/32-eth0-rx-0-21swapper/321:04:593
29925060,0irq/32-eth0-rx-0-21swapper/320:51:143
29925060,0irq/32-eth0-rx-0-21swapper/320:44:423
29925060,0irq/32-eth0-rx-0-21swapper/320:31:393
29925060,0irq/32-eth0-rx-0-21swapper/320:25:023
269269960,0cyclictest0-21swapper/122:49:511
269209966,0cyclictest0-21swapper/023:21:130
269209960,0cyclictest0-21swapper/018:49:430
149960,0migration/09486-21sh22:31:590
149960,0migration/05105-21irqcore22:49:450
149960,0migration/030196-21kthreadcore22:24:440
149960,0migration/022922-21kthreadcore22:59:430
149960,0migration/019443-21sh22:55:540
8315050,0irq/26-0000:00:20843-21latency_hist22:59:252
339950,0migration/36908-21kthreadcore19:54:463
339950,0migration/36155-21cpuspeed_turbos20:19:413
339950,0migration/330361-21kthreadcore19:04:443
329950,0watchdog/38717-21taskset20:55:133
329950,0watchdog/30-21swapper/323:09:333
329950,0watchdog/30-21swapper/323:04:373
329950,0watchdog/30-21swapper/321:49:293
329950,0watchdog/30-21swapper/320:59:373
329950,0watchdog/30-21swapper/319:59:263
329950,0watchdog/30-21swapper/319:49:263
329950,0watchdog/30-21swapper/319:41:183
329950,0watchdog/30-21swapper/319:34:303
329950,0watchdog/30-21swapper/319:32:143
329950,0watchdog/30-21swapper/319:24:263
329950,0watchdog/30-21swapper/319:20:183
329950,0watchdog/30-21swapper/319:09:303
329950,0watchdog/30-21swapper/319:01:583
329950,0watchdog/30-21swapper/318:49:303
329950,0watchdog/30-21swapper/318:37:143
329950,0watchdog/30-21swapper/318:31:543
29935050,0irq/33-eth0-tx-6627-21ssh23:27:162
29935050,0irq/33-eth0-tx-3449-21munin-node21:14:532
29935050,0irq/33-eth0-tx-30729-21munin-node21:49:472
29935050,0irq/33-eth0-tx-29801-21ssh20:29:542
29935050,0irq/33-eth0-tx-23602-21munin-node19:24:480
29935050,0irq/33-eth0-tx-22958-21munin-node23:19:402
29935050,0irq/33-eth0-tx-2261-21cut20:34:432
29935050,0irq/33-eth0-tx-20742-21sendmail_mailqu20:24:482
29935050,0irq/33-eth0-tx-14346-21grep20:39:452
29935050,0irq/33-eth0-tx-0-21swapper/223:47:492
29935050,0irq/33-eth0-tx-0-21swapper/222:54:442
29935050,0irq/33-eth0-tx-0-21swapper/221:37:072
29935050,0irq/33-eth0-tx-0-21swapper/221:27:592
29935050,0irq/33-eth0-tx-0-21swapper/221:19:352
29935050,0irq/33-eth0-tx-0-21swapper/018:59:330
29935050,0irq/33-eth0-tx-0-21swapper/018:58:330
29925050,0irq/32-eth0-rx-2464-21kthreadcore18:44:463
29925050,0irq/32-eth0-rx-0-21swapper/321:19:273
29925050,0irq/32-eth0-rx-0-21swapper/319:14:393
269329954,0cyclictest8513-21sed19:09:542
269269955,0cyclictest0-21swapper/121:39:411
269269955,0cyclictest0-21swapper/121:14:471
269269950,0cyclictest0-21swapper/119:49:481
269209955,0cyclictest0-21swapper/021:42:590
269209955,0cyclictest0-21swapper/021:17:550
269209955,0cyclictest0-21swapper/020:52:420
269209954,0cyclictest15851-21idleruntime-cro23:34:240
269209950,0cyclictest0-21swapper/023:08:450
269209950,0cyclictest0-21swapper/022:16:060
269209950,0cyclictest0-21swapper/021:05:380
259950,0migration/26358-21fschecks_time19:34:402
259950,0migration/232349-21unixbench_singl20:49:532
259950,0migration/224674-21sh22:21:472
259950,0migration/223550-21timerandwakeup20:44:512
259950,0migration/223105-21kthreadcore18:59:452
259950,0migration/216788-21taskset22:18:452
259950,0migration/216129-21aten_r1power_cu23:34:382
179950,0migration/128911-21missed_timers23:19:461
179950,0migration/124151-21fschecks_time20:29:411
149950,0migration/08740-21taskset19:55:090
149950,0migration/07979-21kthreadcore20:19:460
149950,0migration/05157-21kthreadcore19:09:470
149950,0migration/05003-21taskset18:45:560
149950,0migration/032753-21kthreadcore19:04:450
149950,0migration/030760-21latency_hist19:29:250
149950,0migration/030142-21latency23:39:480
149950,0migration/027925-21kthreadcore18:39:460
149950,0migration/023055-21fschecks_time21:44:390
149950,0migration/022228-21ntp_kernel_pll_23:34:490
149950,0migration/021962-21kthreadcore19:44:430
149950,0migration/021370-21kthreadcore22:39:410
149950,0migration/01847-21kthreadcore20:34:430
149950,0migration/017562-21hddtemp_smartct20:04:410
149950,0migration/01668-21irqrtprio22:09:440
149950,0migration/015943-21kthreadcore21:19:440
149950,0migration/015760-21taskset18:31:310
149950,0migration/01567-21taskset19:51:250
149950,0migration/014938-21fschecks_count20:24:400
149950,0migration/014857-21kthreadcore22:34:470
149950,0migration/014668-21sh21:57:500
149950,0migration/014577-21kthreadcore19:14:460
149950,0migration/012837-21taskset23:10:290
149950,0migration/011909-21munin-node21:34:510
149950,0migration/01075-21kthreadcore22:44:430
149950,0migration/010608-21irqcore19:59:430
149950,0migration/010230-21latency_hist20:59:240
329940,0watchdog/30-21swapper/318:54:263
29935040,0irq/33-eth0-tx-30054-21ssh22:04:472
29935040,0irq/33-eth0-tx-2239-21kthreadcore23:04:442
29935040,0irq/33-eth0-tx-0-21swapper/223:14:402
29935040,0irq/33-eth0-tx-0-21swapper/223:09:442
29935040,0irq/33-eth0-tx-0-21swapper/222:49:322
29935040,0irq/33-eth0-tx-0-21swapper/222:44:432
29935040,0irq/33-eth0-tx-0-21swapper/222:30:512
29935040,0irq/33-eth0-tx-0-21swapper/222:09:412
29935040,0irq/33-eth0-tx-0-21swapper/221:39:372
29935040,0irq/33-eth0-tx-0-21swapper/221:29:502
29935040,0irq/33-eth0-tx-0-21swapper/221:04:382
29935040,0irq/33-eth0-tx-0-21swapper/220:20:262
29925040,0irq/32-eth0-rx-0-21swapper/320:04:393
269329943,0cyclictest3599-21lldpd19:54:332
269329943,0cyclictest3599-21lldpd19:44:332
269329943,0cyclictest3599-21lldpd18:43:592
269329943,0cyclictest3599-21lldpd18:19:292
269329943,0cyclictest31888-21awk19:29:432
269329943,0cyclictest27038-21awk22:24:422
269329943,0cyclictest23871-21snmpd20:05:342
269329943,0cyclictest23871-21snmpd18:33:042
269329943,0cyclictest20337-21awk22:39:402
269329943,0cyclictest11718-21awk22:34:442
269329940,0cyclictest0-21swapper/223:29:432
269269943,0cyclictest7059-21awk20:19:441
269269943,0cyclictest3599-21lldpd23:32:211
269269943,0cyclictest3599-21lldpd20:50:461
269269943,0cyclictest3599-21lldpd19:11:031
269269943,0cyclictest3599-21lldpd19:05:331
269269943,0cyclictest3599-21lldpd19:02:031
269269943,0cyclictest3599-21lldpd18:50:001
269269943,0cyclictest3599-21lldpd18:41:001
269269943,0cyclictest3262-21irqbalance22:54:391
269269943,0cyclictest3262-21irqbalance22:24:391
269269943,0cyclictest3262-21irqbalance21:04:291
269269943,0cyclictest3262-21irqbalance20:59:291
269269943,0cyclictest3262-21irqbalance20:34:291
269269943,0cyclictest32192-21awk20:14:441
269269943,0cyclictest32192-21awk20:14:441
269269943,0cyclictest3036-21awk20:54:431
269269943,0cyclictest28695-21awk22:44:401
269269943,0cyclictest23871-21snmpd23:35:051
269269943,0cyclictest17670-21awk20:04:421
269269943,0cyclictest17299-21awk18:34:421
269269943,0cyclictest1249-21awk22:09:421
269269943,0cyclictest1004-21grep21:09:491
269209944,0cyclictest0-21swapper/023:48:390
269209944,0cyclictest0-21swapper/022:06:480
269209944,0cyclictest0-21swapper/021:50:150
269209944,0cyclictest0-21swapper/021:25:110
269209943,0cyclictest28022-21awk18:19:410
259940,0migration/28207-21kthreadcore18:49:452
259940,0migration/229990-21kthreadcore19:04:442
259940,0migration/228046-21kthreadcore20:09:452
259940,0migration/227185-21kthreadcore21:44:442
259940,0migration/220571-21kthreadcore21:59:452
259940,0migration/215743-21kthreadcore19:39:452
259940,0migration/215613-21timerwakeupswit19:14:532
259940,0migration/210289-21kthreadcore21:54:452
249940,0watchdog/231437-21entropy20:14:422
249940,0watchdog/231437-21entropy20:14:422
249940,0watchdog/227607-21cut23:39:452
249940,0watchdog/227275-21ssh21:09:292
249940,0watchdog/216992-21fschecks_count19:19:422
249940,0watchdog/216003-21grep20:59:452
249940,0watchdog/20-21swapper/218:24:462
179940,0migration/16739-21fschecks_time23:09:411
179940,0migration/14408-21kthreadcore19:54:451
179940,0migration/13758-21kthreadcore23:44:441
179940,0migration/13127-21taskset22:29:421
179940,0migration/129527-21taskset23:04:321
179940,0migration/123305-21sh22:02:591
179940,0migration/12089-21kthreadcore21:49:441
179940,0migration/119875-21kthreadcore20:24:451
179940,0migration/119094-21diskmemload21:20:551
179940,0migration/117046-21switchtime22:34:541
179940,0migration/115045-21munin-node18:29:461
179940,0migration/113921-21idleruntime19:39:421
169940,0watchdog/127712-21ssh23:00:011
169940,0watchdog/125048-21sh23:39:411
169940,0watchdog/122900-21kthreadcore21:24:451
169940,0watchdog/12222-21cut18:44:461
169940,0watchdog/118358-21grep18:54:461
169940,0watchdog/112218-21cut21:54:451
169940,0watchdog/10-21swapper/123:14:291
169940,0watchdog/10-21swapper/122:39:451
169940,0watchdog/10-21swapper/119:19:461
169940,0watchdog/10-21swapper/119:14:461
169940,0watchdog/10-21swapper/118:24:461
149940,0migration/08814-21fschecks_count20:39:390
149940,0migration/0430-21irqrtprio23:24:430
149940,0migration/031261-21diskmemload20:33:140
149940,0migration/031000-21runrttasks20:14:280
149940,0migration/031000-21runrttasks20:14:280
149940,0migration/021200-21kthreadcore22:19:430
149940,0migration/020198-21kthreadcore20:44:430
149940,0migration/013432-21fschecks_count19:39:410
269329932,0cyclictest31715-21awk18:44:432
269329932,0cyclictest24638-21awk19:24:422
269329930,0cyclictest16783-21cut18:54:452
269329930,0cyclictest0-21swapper/219:59:252
269269933,0cyclictest3599-21lldpd20:03:051
269269932,0cyclictest8354-21munin-node20:39:381
269269932,0cyclictest3599-21lldpd19:34:331
269269932,0cyclictest3262-21irqbalance23:25:191
269269932,0cyclictest25210-21awk22:04:431
269269932,0cyclictest23871-21snmpd20:45:351
269209933,0cyclictest4701-21kworker/0:121:29:260
269209933,0cyclictest4701-21kworker/0:121:09:440
269209933,0cyclictest4701-21kworker/0:120:54:400
269209933,0cyclictest4701-21kworker/0:119:19:400
269209933,0cyclictest21932-21kworker/0:123:14:410
269209932,0cyclictest6218-21kworker/0:221:59:260
269209932,0cyclictest4701-21kworker/0:118:24:260
269209930,0cyclictest17123-21perl18:34:400
249930,0watchdog/20-21swapper/218:36:502
179930,0migration/126189-21kthreadcore19:24:441
179930,0migration/125802-21kthreadcore20:09:441
179930,0migration/120355-21cpuspeed_turbos19:44:391
169930,0watchdog/15627-21diskmemload21:45:131
169930,0watchdog/10-21swapper/122:19:291
169930,0watchdog/10-21swapper/122:15:491
169930,0watchdog/10-21swapper/121:35:251
169930,0watchdog/10-21swapper/121:30:011
169920,0watchdog/10-21swapper/119:29:261
169920,0watchdog/10-21swapper/118:19:261
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional