You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-09 - 12:06
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Wed Sep 09, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2429125352,0sleep20-21swapper/219:29:442
2413823828,0sleep30-21swapper/319:27:573
2420123725,0sleep10-21swapper/119:28:391
2426023127,0sleep00-21swapper/019:29:260
245759970,0cyclictest0-21swapper/021:02:350
8305060,0irq/26-0000:00:3222-17sedispatch19:42:172
339960,0migration/330910-21munin-node00:37:333
339960,0migration/329256-21sh00:36:183
339960,0migration/320255-21sh00:30:033
339960,0migration/310275-21sh22:48:163
29805060,0irq/28-eth0-rx-22087-21tr20:57:353
149960,0migration/04918-21munin-node20:02:350
149960,0migration/031076-21sh23:37:590
149960,0migration/029865-21sh00:37:260
149960,0migration/02769-21sh22:46:390
149960,0migration/023412-21aten_r1power_en22:02:260
149960,0migration/017856-21sh22:36:010
149960,0migration/011214-21taskset21:34:480
4150,0ktimersoftd/00-21swapper/022:55:100
339950,0migration/36912-21aten_r1power_cu19:42:273
339950,0migration/36669-21kthreadcore20:47:363
339950,0migration/35846-21sendmail_mailqu19:37:403
339950,0migration/35676-21kthreadcore21:32:353
339950,0migration/35554-21taskset21:51:243
339950,0migration/3484-21munin-node22:07:323
339950,0migration/34157-21kthreadcore00:02:363
339950,0migration/34005-21munin-node20:02:323
339950,0migration/3333-21kthreadcore23:02:373
339950,0migration/331597-21latency_hist00:57:183
339950,0migration/330041-21kernelversion19:57:343
339950,0migration/329990-21irqcore20:42:333
339950,0migration/325256-21cpuspeed_turbos20:17:303
339950,0migration/32402-21taskset00:18:133
339950,0migration/323669-21cpuspeed_turbos22:02:273
339950,0migration/321680-21taskset00:48:453
339950,0migration/319163-21kthreadcore19:47:343
339950,0migration/319104-21kthreadcore22:17:373
329950,0watchdog/38479-21tr22:12:323
329950,0watchdog/36828-21cut00:22:373
329950,0watchdog/33019-21cut20:22:363
329950,0watchdog/319018-21diskstats22:37:323
329950,0watchdog/318778-21smartctl20:12:323
329950,0watchdog/316831-21kthreadcore20:32:363
329950,0watchdog/313747-21ssh21:54:563
329950,0watchdog/313606-21python21:12:403
329950,0watchdog/313278-21tune2fs21:37:323
329950,0watchdog/30-21swapper/323:42:323
29815050,0irq/29-eth0-tx-7545-21nfs4_client22:27:362
29815050,0irq/29-eth0-tx-6841-21sendmail_mailtr23:42:422
29815050,0irq/29-eth0-tx-28959-21ssh21:45:492
29815050,0irq/29-eth0-tx-27093-21munin-node22:42:282
29815050,0irq/29-eth0-tx-23963-21ssh23:52:422
29815050,0irq/29-eth0-tx-17819-21munin-node20:12:382
29815050,0irq/29-eth0-tx-10553-21python00:22:422
29815050,0irq/29-eth0-tx-0-21swapper/223:38:222
29815050,0irq/29-eth0-tx-0-21swapper/223:02:442
29815050,0irq/29-eth0-tx-0-21swapper/222:58:212
29815050,0irq/29-eth0-tx-0-21swapper/221:59:082
29815050,0irq/29-eth0-tx-0-21swapper/221:52:382
29805050,0irq/28-eth0-rx-32165-21iostat_ios23:22:343
29805050,0irq/28-eth0-rx-30464-21pmu-power23:37:383
29805050,0irq/28-eth0-rx-24001-21kthreadcore22:57:353
29805050,0irq/28-eth0-rx-19920-21cut21:17:363
29805050,0irq/28-eth0-rx-16059-21ssh21:57:343
29805050,0irq/28-eth0-rx-0-21swapper/323:27:323
29805050,0irq/28-eth0-rx-0-21swapper/323:17:333
29805050,0irq/28-eth0-rx-0-21swapper/323:08:093
29805050,0irq/28-eth0-rx-0-21swapper/322:42:313
29805050,0irq/28-eth0-rx-0-21swapper/322:27:383
29805050,0irq/28-eth0-rx-0-21swapper/321:46:143
29805050,0irq/28-eth0-rx-0-21swapper/300:52:343
29805050,0irq/28-eth0-rx-0-21swapper/300:11:523
27150,0ktimersoftd/20-21swapper/223:29:342
259950,0migration/28315-21if_eth1.220:27:342
259950,0migration/25156-21sh00:40:312
259950,0migration/21816-21kthreadcore22:07:342
259950,0migration/216829-21munin-node20:07:382
245779950,0cyclictest0-21swapper/223:27:032
245779950,0cyclictest0-21swapper/222:24:182
245779950,0cyclictest0-21swapper/200:29:382
245769955,0cyclictest3662-21diskmemload00:06:041
245769955,0cyclictest0-21swapper/100:12:411
245769950,0cyclictest0-21swapper/121:57:251
245769950,0cyclictest0-21swapper/121:42:261
245769950,0cyclictest0-21swapper/121:40:521
245769950,0cyclictest0-21swapper/120:07:321
245769950,0cyclictest0-21swapper/100:09:221
249950,0watchdog/226611-21kthreadcore19:32:362
249950,0watchdog/225498-21ssh22:38:282
249950,0watchdog/213862-21aten2.4-expect23:12:322
249950,0watchdog/211641-21cut23:47:362
179950,0migration/118728-21missed_timers00:27:381
179950,0migration/117777-21kthreadcore23:32:341
169950,0watchdog/130768-21ssh22:04:401
169950,0watchdog/12457-21kthreadcore00:57:371
169950,0watchdog/124304-21aten2.4-expect23:37:281
169950,0watchdog/111642-21tr23:47:361
149950,0migration/09270-21sh00:03:220
149950,0migration/07303-21kthreadcore22:47:350
149950,0migration/0519-21fschecks_count21:07:320
149950,0migration/04532-21kthreadcore23:42:380
149950,0migration/032249-21aten_r1power_cu20:22:290
149950,0migration/031054-21aten_r1power_po23:22:290
149950,0migration/030462-21fschecks_count21:47:310
149950,0migration/029655-21fschecks_count23:02:330
149950,0migration/027776-21kernelversion23:57:360
149950,0migration/025468-21kthreadcore21:22:360
149950,0migration/024363-21kthreadcore21:42:370
149950,0migration/024191-21unixbench-2d23:52:430
149950,0migration/023215-21runrttasks00:52:290
149950,0migration/021539-21cpuspeed_turbos19:52:290
149950,0migration/021330-21taskset21:57:430
149950,0migration/020589-21munin-node21:17:370
149950,0migration/020325-21kthreadcore20:57:350
149950,0migration/019720-21kthreadcore20:32:370
149950,0migration/018222-21taskset00:09:300
149950,0migration/015959-21kthreadcore20:07:340
149950,0migration/015905-21kthreadcore23:12:390
149950,0migration/015860-21aten_r1power_fr23:32:280
149950,0migration/014968-21latency23:47:380
149950,0migration/014714-21fschecks_count19:47:290
149950,0migration/010061-21kthreadcore21:12:350
339940,0migration/332089-21taskset21:05:413
339940,0migration/324756-21latency_hist19:32:173
339940,0migration/317580-21diskmemload23:52:223
329940,0watchdog/328026-21kthreadcore23:57:373
329940,0watchdog/324320-21kthreadcore21:22:363
329940,0watchdog/320653-21ssh23:13:043
329940,0watchdog/30-21swapper/323:47:363
329940,0watchdog/30-21swapper/322:52:243
329940,0watchdog/30-21swapper/321:27:363
29815040,0irq/29-eth0-tx-4260-21ssh21:47:502
29815040,0irq/29-eth0-tx-3262-21ssh22:47:252
29815040,0irq/29-eth0-tx-15149-21ssh22:13:322
29815040,0irq/29-eth0-tx-0-21swapper/222:32:232
29815040,0irq/29-eth0-tx-0-21swapper/222:17:362
29815040,0irq/29-eth0-tx-0-21swapper/222:02:252
29815040,0irq/29-eth0-tx-0-21swapper/200:32:242
29815040,0irq/29-eth0-tx-0-21swapper/200:02:302
29805040,0irq/28-eth0-rx-17136-21ssh22:33:543
29805040,0irq/28-eth0-rx-0-21swapper/323:32:383
29805040,0irq/28-eth0-rx-0-21swapper/322:24:473
29805040,0irq/28-eth0-rx-0-21swapper/320:37:303
29805040,0irq/28-eth0-rx-0-21swapper/320:27:343
29805040,0irq/28-eth0-rx-0-21swapper/319:52:303
29805040,0irq/28-eth0-rx-0-21swapper/300:42:243
29805040,0irq/28-eth0-rx-0-21swapper/300:12:223
259940,0migration/29867-21kthreadcore19:42:342
259940,0migration/27473-21munin-node21:12:292
259940,0migration/232732-21fschecks_count20:22:322
259940,0migration/230801-21kthreadcore19:57:342
259940,0migration/23030-21runrttasks20:43:352
259940,0migration/223864-21kthreadcore19:52:342
259940,0migration/222417-21kthreadcore00:12:352
259940,0migration/22104-21kthreadcore00:57:372
259940,0migration/219486-21irqrtprio20:57:342
259940,0migration/217501-21kthreadcore23:32:342
259940,0migration/216357-21kthreadcore22:52:352
259940,0migration/215579-21fschecks_time00:47:322
259940,0migration/215133-21kthreadcore21:37:362
259940,0migration/214615-21fschecks_count19:47:292
245779943,0cyclictest7499-21ps21:32:352
245779943,0cyclictest7132-21awk00:42:332
245779943,0cyclictest4746-21awk20:47:342
245779943,0cyclictest3571-21lldpd21:20:282
245779943,0cyclictest3524-21snmpd20:40:022
245779943,0cyclictest3524-21snmpd00:52:212
245779943,0cyclictest26088-21awk20:17:332
245779943,0cyclictest15391-21awk20:32:342
245779940,0cyclictest0-21swapper/200:17:352
245769944,0cyclictest0-21swapper/123:23:201
245769944,0cyclictest0-21swapper/123:01:541
245769944,0cyclictest0-21swapper/100:37:361
245769944,0cyclictest0-21swapper/100:22:461
245769943,0cyclictest8303-21awk23:27:331
245769943,0cyclictest6292-21awk23:07:311
245769943,0cyclictest4751-21grep22:47:341
245769943,0cyclictest4447-21awk20:02:341
245769943,0cyclictest3571-21lldpd21:19:281
245769943,0cyclictest3571-21lldpd19:42:271
245769943,0cyclictest3571-21lldpd00:33:301
245769943,0cyclictest3250-21irqbalance23:12:241
245769943,0cyclictest3250-21irqbalance21:52:331
245769943,0cyclictest3250-21irqbalance21:33:031
245769943,0cyclictest3250-21irqbalance20:42:231
245769943,0cyclictest3250-21irqbalance20:37:231
245769943,0cyclictest3250-21irqbalance20:32:331
245769943,0cyclictest3250-21irqbalance20:27:331
245769943,0cyclictest3250-21irqbalance20:19:331
245769943,0cyclictest3250-21irqbalance19:32:231
245769943,0cyclictest3250-21irqbalance00:47:541
245769943,0cyclictest3250-21irqbalance00:17:241
245769943,0cyclictest29528-21awk19:57:321
245769943,0cyclictest2472-21awk22:27:321
245769943,0cyclictest17782-21awk22:17:351
245759944,0cyclictest0-21swapper/022:38:550
245759944,0cyclictest0-21swapper/000:19:020
245759943,0cyclictest8228-21awk20:27:320
245759943,0cyclictest3571-21lldpd21:56:290
245759943,0cyclictest3524-21snmpd20:38:320
245759943,0cyclictest26387-21awk22:22:340
245759943,0cyclictest25875-21awk19:32:340
245759943,0cyclictest18924-21awk20:12:330
245759943,0cyclictest15771-21awk00:47:340
249940,0watchdog/25193-21ssh23:07:242
249940,0watchdog/232584-21ls23:57:452
249940,0watchdog/229486-21kthreadcore21:02:362
249940,0watchdog/228329-21load23:17:362
249940,0watchdog/223389-21kthreadcore21:22:362
249940,0watchdog/215997-21kthreadcore00:07:372
179940,0migration/13109-21kthreadcore21:47:361
179940,0migration/130756-21kthreadcore22:42:351
179940,0migration/129783-21fschecks_time21:27:311
179940,0migration/127956-21kthreadcore22:22:361
179940,0migration/123083-21irqcore21:22:351
179940,0migration/123049-21latency_hist00:52:181
179940,0migration/122316-21latency_hist23:17:171
179940,0migration/12189-21kthreadcore19:37:361
179940,0migration/120458-21kthreadcore20:57:351
179940,0migration/117212-21timerandwakeup20:52:441
179940,0migration/116639-21kthreadcore19:47:331
179940,0migration/114301-21taskset21:16:441
179940,0migration/114205-21kthreadcore22:52:341
169940,0watchdog/18480-21cpuspeed_turbos22:12:321
169940,0watchdog/1519-21perl20:22:321
169940,0watchdog/128025-21kthreadcore23:57:371
169940,0watchdog/121452-21kthreadcore20:12:361
169940,0watchdog/117284-21ssh22:34:201
169940,0watchdog/11667-21irqcore23:42:361
169940,0watchdog/10-21swapper/123:55:571
169940,0watchdog/10-21swapper/121:02:361
169940,0watchdog/10-21swapper/100:42:371
149940,0migration/07990-21kthreadcore23:07:330
149940,0migration/06716-21sh22:08:540
149940,0migration/05824-21kthreadcore22:27:350
149940,0migration/05809-21kthreadcore00:22:360
149940,0migration/029406-21sh23:18:380
149940,0migration/028190-21diskmemload23:00:450
149940,0migration/027366-21kthreadcore20:17:350
149940,0migration/025941-21kthreadcore00:32:370
149940,0migration/01684-21kthreadcore00:57:360
149940,0migration/015473-21kthreadcore21:37:360
149940,0migration/011486-21kthreadcore19:42:340
8305030,0irq/26-0000:00:0-21swapper/220:02:282
339930,0migration/317680-21runrttasks20:55:373
329930,0watchdog/30-21swapper/321:07:283
329930,0watchdog/30-21swapper/320:07:283
245779933,0cyclictest3571-21lldpd21:27:282
245779933,0cyclictest0-21swapper/221:07:222
245779930,0cyclictest0-21swapper/220:52:332
245769933,0cyclictest3571-21lldpd23:02:291
245769930,0cyclictest742-21perl21:07:321
245769930,0cyclictest19360-21perl22:37:331
245759932,0cyclictest8410-21awk23:27:330
245759932,0cyclictest3571-21lldpd22:17:290
245759932,0cyclictest3524-21snmpd20:43:020
245759932,0cyclictest13929-21grep20:52:370
245759932,0cyclictest13924-21awk00:27:340
245759930,0cyclictest9185-21hwlatdetect22:12:350
245759930,0cyclictest6652-21munin-run19:42:170
245759930,0cyclictest22153-21cut00:12:350
169930,0watchdog/10-21swapper/122:07:201
169930,0watchdog/10-21swapper/120:47:401
245759921,0cyclictest28584-21aten2.4-expect19:57:290
245759920,0cyclictest5933-21latency_hist00:42:180
245759920,0cyclictest28944-21sort21:27:170
245759920,0cyclictest0-21swapper/020:47:170
169920,0watchdog/10-21swapper/119:52:201
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional