You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-29 - 04:25
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2.osadl.org (updated Mon Dec 29, 2025 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2458024443,0sleep10-21swapper/119:07:151
2473323627,0sleep20-21swapper/219:09:042
2461723423,0sleep00-21swapper/019:07:420
2470523026,0sleep30-21swapper/319:08:453
25004991413,0cyclictest83050irq/26-0000:00:21:30:272
339960,0migration/38714-21irqrtprio19:21:373
339960,0migration/34922-21sh22:01:573
339960,0migration/325330-21munin-node00:01:433
339960,0migration/324749-21runrttasks00:39:033
339960,0migration/322400-21sh22:50:283
339960,0migration/314121-21sh23:38:213
339960,0migration/311941-21idleruntime-cro23:21:223
339960,0migration/311624-21sh21:13:533
329960,0watchdog/3834-21ssh23:51:213
29815060,0irq/29-eth0-tx-22063-21munin-node21:01:500
29805060,0irq/28-eth0-rx-32638-21munin-node23:11:433
29805060,0irq/28-eth0-rx-0-21swapper/321:16:363
250049960,0cyclictest0-21swapper/221:49:342
250049960,0cyclictest0-21swapper/200:31:382
250039960,0cyclictest0-21swapper/121:58:381
179960,0migration/117224-21taskset23:57:301
149960,0migration/05006-21latency_hist22:41:220
149960,0migration/04761-21sh23:32:200
149960,0migration/031990-21sh22:17:290
149960,0migration/026292-21sh00:02:470
149960,0migration/017355-21sh22:26:510
8305050,0irq/26-0000:00:0-21swapper/220:36:212
339950,0migration/36335-21kernelversion00:11:413
339950,0migration/34451-21kthreadcore23:51:433
339950,0migration/33234-21kthreadcore23:31:443
339950,0migration/329046-21latency_hist20:21:213
339950,0migration/328984-21fschecks_time00:06:363
339950,0migration/323837-21if_eth1.300:21:403
339950,0migration/322080-21aten_r1power_cu21:01:353
339950,0migration/32058-21kthreadcore20:01:393
339950,0migration/31966-21kthreadcore19:36:383
339950,0migration/31966-21kthreadcore19:36:383
339950,0migration/317607-21latency_hist21:36:213
339950,0migration/313857-21kthreadcore23:21:413
339950,0migration/312174-21hddtemp_smartct20:31:353
339950,0migration/310194-21kthreadcore20:06:393
329950,0watchdog/33492-21ssh21:46:293
329950,0watchdog/328243-21ssh21:56:533
329950,0watchdog/325836-21entropy19:56:363
329950,0watchdog/325565-21kthreadcore21:21:403
329950,0watchdog/320598-21ssh21:55:013
329950,0watchdog/317488-21sleep23:58:093
329950,0watchdog/30-21swapper/320:56:483
329950,0watchdog/30-21swapper/319:16:443
29815050,0irq/29-eth0-tx-8729-21munin-node23:01:522
29815050,0irq/29-eth0-tx-4788-21ssh23:51:432
29815050,0irq/29-eth0-tx-3526-21ssh00:09:152
29815050,0irq/29-eth0-tx-26501-21ssh21:41:092
29815050,0irq/29-eth0-tx-25306-21sshd20:40:570
29815050,0irq/29-eth0-tx-0-21swapper/223:48:542
29815050,0irq/29-eth0-tx-0-21swapper/223:41:042
29815050,0irq/29-eth0-tx-0-21swapper/222:40:472
29815050,0irq/29-eth0-tx-0-21swapper/222:31:462
29815050,0irq/29-eth0-tx-0-21swapper/222:12:182
29815050,0irq/29-eth0-tx-0-21swapper/222:07:512
29815050,0irq/29-eth0-tx-0-21swapper/200:17:192
29805050,0irq/28-eth0-rx-8730-21ssh22:41:403
29805050,0irq/28-eth0-rx-6314-21runrttasks23:06:353
29805050,0irq/28-eth0-rx-5857-21netstat22:56:473
29805050,0irq/28-eth0-rx-3874-21kthreadcore20:46:413
29805050,0irq/28-eth0-rx-32134-21fschecks_count21:26:373
29805050,0irq/28-eth0-rx-27177-21cut22:51:383
29805050,0irq/28-eth0-rx-26120-21munin-node22:31:463
29805050,0irq/28-eth0-rx-16823-21if_eth023:41:393
29805050,0irq/28-eth0-rx-11180-21kthreadcore00:31:413
29805050,0irq/28-eth0-rx-0-21swapper/323:31:223
29805050,0irq/28-eth0-rx-0-21swapper/323:05:003
29805050,0irq/28-eth0-rx-0-21swapper/322:06:413
29805050,0irq/28-eth0-rx-0-21swapper/321:41:393
29805050,0irq/28-eth0-rx-0-21swapper/321:36:093
29805050,0irq/28-eth0-rx-0-21swapper/300:26:403
29805050,0irq/28-eth0-rx-0-21swapper/300:16:413
250049955,0cyclictest3927-21diskmemload21:19:082
250049955,0cyclictest0-21swapper/223:31:342
250049955,0cyclictest0-21swapper/222:30:052
250049955,0cyclictest0-21swapper/221:36:002
250049955,0cyclictest0-21swapper/200:27:422
250049950,0cyclictest0-21swapper/221:01:002
250049950,0cyclictest0-21swapper/200:06:062
250039955,0cyclictest0-21swapper/123:54:491
250039955,0cyclictest0-21swapper/123:43:091
250039955,0cyclictest0-21swapper/123:23:171
250039955,0cyclictest0-21swapper/121:01:241
250039955,0cyclictest0-21swapper/119:26:391
250039950,0cyclictest0-21swapper/122:27:571
250039950,0cyclictest0-21swapper/120:36:201
250039950,0cyclictest0-21swapper/100:23:121
250039950,0cyclictest0-21swapper/100:16:441
250029954,0cyclictest15721-21crond22:11:210
250029954,0cyclictest13505-21ntp_states20:06:420
250029950,0cyclictest0-21swapper/022:12:500
250029950,0cyclictest0-21swapper/000:31:440
259950,0migration/231396-21munin-node19:56:432
259950,0migration/228467-21taskset19:33:592
259950,0migration/227626-21kthreadcore21:21:412
259950,0migration/224037-21sh00:37:202
259950,0migration/221700-21missed_timers23:41:432
259950,0migration/22008-21kthreadcore22:56:452
259950,0migration/219838-21kthreadcore19:51:382
249950,0watchdog/25532-21irqcore21:11:402
249950,0watchdog/225305-21kthreadcore21:56:412
19150,0ktimersoftd/10-21swapper/123:32:541
19150,0ktimersoftd/10-21swapper/122:04:561
179950,0migration/130303-21kthreadcore19:56:411
179950,0migration/117291-21timerandwakeup19:46:471
179950,0migration/115461-21fschecks_count20:56:371
179950,0migration/114737-21proc_pri21:31:431
179950,0migration/112629-21sh00:14:351
179950,0migration/111214-21missed_timers22:41:411
169950,0watchdog/128404-21grep22:16:411
149950,0migration/08839-21sh23:53:450
149950,0migration/07745-21kthreadcore20:26:390
149950,0migration/07603-21missed_timers22:21:410
149950,0migration/07313-21latency_hist20:06:220
149950,0migration/05798-21kthreadcore20:46:410
149950,0migration/04818-21kthreadcore21:26:420
149950,0migration/04782-21fschecks_count21:11:370
149950,0migration/0390-21sh23:50:220
149950,0migration/032312-21taskset20:43:380
149950,0migration/031739-21sh22:56:170
149950,0migration/030665-21kthreadcore19:36:370
149950,0migration/030665-21kthreadcore19:36:370
149950,0migration/03038-21latency_hist23:16:220
149950,0migration/029967-21kthreadcore19:56:410
149950,0migration/02418-21sendmail_mailst22:36:440
149950,0migration/023121-21fschecks_count00:21:380
149950,0migration/022577-21cpuspeed_turbos21:21:350
149950,0migration/021886-21fschecks_time23:26:360
149950,0migration/021505-21kthreadcore00:36:440
149950,0migration/020270-21kthreadcore19:51:390
149950,0migration/01967-21sh21:43:310
149950,0migration/018510-21processes23:21:450
149950,0migration/018464-21fschecks_time23:06:360
149950,0migration/017143-21sh21:35:490
149950,0migration/016511-21fschecks_time23:41:370
149950,0migration/015206-21fschecks_time19:26:350
149950,0migration/014636-21kthreadcore19:46:400
149950,0migration/013600-21kthreadcore23:56:400
149950,0migration/011959-21kthreadcore20:51:380
149950,0migration/011467-21kthreadcore19:21:380
149950,0migration/010970-21taskset21:48:200
8305040,0irq/26-0000:00:0-21swapper/219:16:432
339940,0migration/35696-21kthreadcore22:21:403
339940,0migration/33730-21taskset21:09:183
339940,0migration/325082-21runrttasks20:38:183
339940,0migration/324771-21runrttasks19:51:483
29815040,0irq/29-eth0-tx-21629-21ssh23:26:362
29815040,0irq/29-eth0-tx-14082-21ssh22:46:232
29815040,0irq/29-eth0-tx-0-21swapper/223:56:312
29815040,0irq/29-eth0-tx-0-21swapper/223:21:322
29815040,0irq/29-eth0-tx-0-21swapper/223:16:232
29815040,0irq/29-eth0-tx-0-21swapper/222:51:382
29815040,0irq/29-eth0-tx-0-21swapper/222:41:282
29815040,0irq/29-eth0-tx-0-21swapper/222:22:062
29815040,0irq/29-eth0-tx-0-21swapper/222:01:242
29815040,0irq/29-eth0-tx-0-21swapper/221:43:312
29815040,0irq/29-eth0-tx-0-21swapper/200:11:242
29815040,0irq/29-eth0-tx-0-21swapper/021:06:470
29815040,0irq/29-eth0-tx-0-21swapper/020:31:360
29805040,0irq/28-eth0-rx-0-21swapper/322:36:333
29805040,0irq/28-eth0-rx-0-21swapper/322:26:343
29805040,0irq/28-eth0-rx-0-21swapper/322:16:293
29805040,0irq/28-eth0-rx-0-21swapper/322:13:483
250049943,0cyclictest26365-21awk19:11:372
250049943,0cyclictest15863-21awk20:11:392
250049943,0cyclictest13593-21awk21:51:382
250039944,0cyclictest0-21swapper/121:50:161
250039943,0cyclictest3524-21snmpd20:29:561
250039943,0cyclictest3524-21snmpd19:13:401
250039943,0cyclictest3524-21snmpd00:37:441
250039943,0cyclictest29993-21awk19:36:361
250039943,0cyclictest29993-21awk19:36:361
250039943,0cyclictest26635-21awk20:41:371
250039943,0cyclictest25806-21awk23:46:411
250039943,0cyclictest20623-21awk22:31:361
250039943,0cyclictest15099-21grep22:46:371
250029943,0cyclictest3571-21lldpd22:33:500
250029943,0cyclictest3571-21lldpd19:18:180
250029943,0cyclictest3524-21snmpd23:00:120
250029943,0cyclictest3524-21snmpd20:59:410
250029943,0cyclictest3524-21snmpd00:12:140
250029943,0cyclictest3524-21snmpd00:08:440
250029943,0cyclictest332-21awk00:26:390
250029943,0cyclictest31543-21awk22:01:380
250029943,0cyclictest13487-21gltestperf21:51:370
259940,0migration/27226-21runrttasks20:05:412
259940,0migration/230890-21kthreadcore23:11:422
259940,0migration/228906-21runrttasks19:36:272
259940,0migration/228906-21runrttasks19:36:272
259940,0migration/22581-21taskset20:21:432
259940,0migration/225243-21kthreadcore20:16:422
259940,0migration/220954-21kthreadcore23:06:402
259940,0migration/219475-21kthreadcore19:26:392
259940,0migration/214266-21turbostat.cron19:26:162
259940,0migration/212971-21kthreadcore19:46:392
249940,0watchdog/2891-21fschecks_count20:46:362
249940,0watchdog/224889-21ssh22:16:252
249940,0watchdog/224091-21irqrtprio00:21:412
179940,0migration/1791-21fschecks_count20:01:351
179940,0migration/16825-21kthreadcore22:21:401
179940,0migration/14990-21kthreadcore00:26:411
179940,0migration/131676-21aten_r1power_po21:26:351
179940,0migration/131669-21kthreadcore21:41:391
179940,0migration/130901-21kthreadcore23:11:421
179940,0migration/129653-21sh23:30:541
179940,0migration/129203-21fschecks_time22:36:361
179940,0migration/128749-21fschecks_count00:06:361
179940,0migration/125896-21diskmemload00:01:541
179940,0migration/122900-21munin-node19:31:371
179940,0migration/120830-21kthreadcore19:51:391
179940,0migration/118974-21irqcore21:36:391
179940,0migration/118185-21kthreadcore22:11:411
179940,0migration/113909-21diskmemload23:37:501
179940,0migration/112644-21kthreadcore19:21:391
179940,0migration/110544-21taskset19:43:431
179940,0migration/110506-21kthreadcore00:31:411
169940,0watchdog/18725-21ssh23:01:371
169940,0watchdog/15531-21irqcore21:11:401
169940,0watchdog/1333-21fschecks_time22:56:411
169940,0watchdog/122215-21kthreadcore23:06:411
169940,0watchdog/120359-21cat20:36:401
169940,0watchdog/115384-21perl20:11:361
169940,0watchdog/114835-21kthreadcore21:51:411
169940,0watchdog/111131-21cut22:06:411
169940,0watchdog/10-21swapper/121:16:361
169940,0watchdog/10-21swapper/120:46:361
169940,0watchdog/10-21swapper/120:06:401
169940,0watchdog/10-21swapper/119:16:441
149940,0migration/024537-21kthreadcore19:31:390
149940,0migration/011770-21kthreadcore23:36:410
149940,0migration/01084-21kthreadcore20:21:430
149940,0migration/010786-21kthreadcore23:01:440
329930,0watchdog/30-21swapper/320:51:403
329930,0watchdog/30-21swapper/320:41:443
329930,0watchdog/30-21swapper/320:26:363
329930,0watchdog/30-21swapper/320:11:363
329930,0watchdog/30-21swapper/319:46:403
329930,0watchdog/30-21swapper/319:31:283
329930,0watchdog/30-21swapper/319:14:443
29815030,0irq/29-eth0-tx-0-21swapper/019:11:320
29805030,0irq/28-eth0-rx-0-21swapper/320:16:503
29805030,0irq/28-eth0-rx-0-21swapper/319:41:513
250049932,0cyclictest4869-21awk19:41:392
250049932,0cyclictest3930-21sadc21:11:212
250049930,0cyclictest4741-21perl20:26:352
250049930,0cyclictest28799-21runrttasks21:03:492
250049930,0cyclictest13205-21grep20:51:392
250039932,0cyclictest3524-21snmpd23:16:281
250039932,0cyclictest3524-21snmpd21:09:411
250039932,0cyclictest27914-21tr22:51:381
250039930,0cyclictest8430-21perl20:51:341
250029933,0cyclictest5188-21kworker/0:121:56:370
250029933,0cyclictest5188-21kworker/0:121:36:350
250029933,0cyclictest5188-21kworker/0:121:16:350
250029933,0cyclictest4845-21kworker/0:023:11:390
250029933,0cyclictest4845-21kworker/0:000:16:390
250029933,0cyclictest29669-21kworker/0:020:16:320
250029933,0cyclictest29669-21kworker/0:020:11:300
250029932,0cyclictest8175-21cut19:41:420
250029932,0cyclictest15059-21grep22:46:370
249930,0watchdog/20-21swapper/220:36:402
249930,0watchdog/20-21swapper/220:06:242
179930,0migration/129588-21fschecks_count20:21:381
169930,0watchdog/10-21swapper/121:21:401
169930,0watchdog/10-21swapper/120:16:241
329920,0watchdog/30-21swapper/319:26:243
250049921,0cyclictest25567-21sed20:41:212
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional