You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-28 - 05:37
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack1slot2.osadl.org (updated Sun Dec 28, 2025 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1861624839,0sleep20-21swapper/219:06:432
1906124536,0sleep30-21swapper/319:07:403
1923123127,0sleep10-21swapper/119:09:381
1936623026,0sleep00-21swapper/019:11:100
339960,0migration/325157-21sh21:24:493
339960,0migration/311711-21sh23:05:103
339960,0migration/311005-21sh00:34:313
329960,0watchdog/32204-21grep19:21:313
29805060,0irq/28-eth0-rx-32207-21irqrtprio21:46:353
27160,0ktimersoftd/20-21swapper/222:13:112
194459960,0cyclictest0-21swapper/223:55:582
194459960,0cyclictest0-21swapper/223:29:482
194439966,0cyclictest0-21swapper/021:20:350
169960,0watchdog/127166-21fschecks_time00:26:321
149960,0migration/05224-21kthreadcore20:06:390
149960,0migration/018543-21sh00:01:360
8305050,0irq/26-0000:00:13952-21ssh00:01:192
4150,0ktimersoftd/00-21swapper/022:53:210
35150,0ktimersoftd/30-21swapper/322:53:213
339950,0migration/38856-21cpuspeed_turbos00:16:293
339950,0migration/36544-21kthreadcore20:51:353
339950,0migration/35304-21taskset20:29:393
339950,0migration/3442-21munin-node20:01:393
339950,0migration/33932-21munin-node21:31:323
339950,0migration/332568-21kthreadcore23:16:353
339950,0migration/332453-21mdstat00:26:373
339950,0migration/328800-21runrttasks20:21:353
339950,0migration/326253-21ksmtuned19:11:443
339950,0migration/325515-21kthreadcore19:36:343
339950,0migration/325119-21kthreadcore23:46:393
339950,0migration/32323-21fschecks_count22:06:323
339950,0migration/317447-21if_eth021:56:343
339950,0migration/315799-21latency_hist23:26:163
339950,0migration/31243-21taskset19:18:543
339950,0migration/311928-21idleruntime22:11:323
339950,0migration/311645-21munin-node23:41:333
329950,0watchdog/3834-21seq21:27:393
329950,0watchdog/329471-21if_err_eth1.323:51:323
329950,0watchdog/326817-21ssh22:17:163
329950,0watchdog/323417-21diskstats00:06:323
329950,0watchdog/323374-40yum-updatesd-he21:03:393
329950,0watchdog/321018-21kthreadcore19:31:353
329950,0watchdog/320750-21python22:31:403
329950,0watchdog/318191-21ntp_states00:36:403
329950,0watchdog/315247-21kthreadcore20:36:353
329950,0watchdog/30-21swapper/322:56:363
329950,0watchdog/30-21swapper/320:46:353
29815050,0irq/29-eth0-tx-21087-21kthreadcore00:21:352
29815050,0irq/29-eth0-tx-0-21swapper/221:45:282
29815050,0irq/29-eth0-tx-0-21swapper/020:52:480
29815050,0irq/29-eth0-tx-0-21swapper/020:26:160
29805050,0irq/28-eth0-rx-8698-21tr23:56:363
29805050,0irq/28-eth0-rx-8361-21idleruntime21:51:343
29805050,0irq/28-eth0-rx-7961-21munin-node21:16:293
29805050,0irq/28-eth0-rx-6016-21python00:11:423
29805050,0irq/28-eth0-rx-3285-21grep21:11:353
29805050,0irq/28-eth0-rx-28817-21ssh21:42:433
29805050,0irq/28-eth0-rx-26903-21kthreadcore22:01:383
29805050,0irq/28-eth0-rx-24054-21grep22:36:313
29805050,0irq/28-eth0-rx-0-21swapper/323:36:373
29805050,0irq/28-eth0-rx-0-21swapper/323:23:123
29805050,0irq/28-eth0-rx-0-21swapper/323:11:443
29805050,0irq/28-eth0-rx-0-21swapper/323:06:293
29805050,0irq/28-eth0-rx-0-21swapper/322:46:333
29805050,0irq/28-eth0-rx-0-21swapper/322:21:373
29805050,0irq/28-eth0-rx-0-21swapper/321:36:373
29805050,0irq/28-eth0-rx-0-21swapper/300:21:303
29805050,0irq/28-eth0-rx-0-21swapper/300:01:363
259950,0migration/29614-21munin-node20:11:312
259950,0migration/23727-21kernelversion20:51:342
249950,0watchdog/212187-21kthreadcore23:21:362
194459955,0cyclictest0-21swapper/223:06:282
194459955,0cyclictest0-21swapper/222:35:432
194459955,0cyclictest0-21swapper/222:17:072
194459955,0cyclictest0-21swapper/221:39:072
194459955,0cyclictest0-21swapper/200:08:062
194459954,0cyclictest10039-21grep00:16:332
194459950,0cyclictest0-21swapper/222:45:392
194459950,0cyclictest0-21swapper/221:23:212
194449955,0cyclictest30811-21diskmemload22:41:411
194449955,0cyclictest0-21swapper/123:21:341
194449955,0cyclictest0-21swapper/121:35:471
194449955,0cyclictest0-21swapper/100:38:581
194449950,0cyclictest0-21swapper/100:01:081
194439955,0cyclictest21360-21kworker/0:200:36:360
194439955,0cyclictest21070-21kworker/0:122:36:340
194439955,0cyclictest0-21swapper/023:07:090
194439954,0cyclictest24625-21basename23:31:160
194439954,0cyclictest21070-21kworker/0:122:01:580
194439950,0cyclictest0-21swapper/022:23:010
194439950,0cyclictest0-21swapper/021:36:210
19150,0ktimersoftd/10-21swapper/122:19:551
179950,0migration/19740-21munin-node20:11:311
179950,0migration/16813-21sh00:12:451
179950,0migration/14718-21taskset22:25:181
179950,0migration/129246-21taskset23:14:191
179950,0migration/128694-21taskset21:42:271
179950,0migration/126867-21kthreadcore22:36:351
179950,0migration/126424-21runrttasks22:55:541
179950,0migration/126344-21kthreadcore21:06:341
179950,0migration/125628-21munin-node22:01:321
179950,0migration/124012-21fschecks_time19:36:301
179950,0migration/123412-21kthreadcore19:56:351
179950,0migration/113941-21sh21:16:401
179950,0migration/112377-21kthreadcore22:46:331
179950,0migration/111133-21cpuspeed_turbos22:11:301
179950,0migration/110966-21kernelversion20:56:351
179950,0migration/110301-21latency_hist23:41:161
169950,0watchdog/112222-21rm21:36:191
149950,0migration/07830-21kthreadcore23:56:360
149950,0migration/07510-21kthreadcore20:31:370
149950,0migration/032659-21kthreadcore23:16:360
149950,0migration/032489-21fschecks_count00:11:320
149950,0migration/031170-21kthreadcore20:46:360
149950,0migration/031053-21sh21:11:230
149950,0migration/028655-21kthreadcore21:26:350
149950,0migration/027952-21gltestperf20:01:340
149950,0migration/027815-21kthreadcore19:36:350
149950,0migration/027350-21fschecks_count19:16:310
149950,0migration/027293-21kthreadcore23:31:360
149950,0migration/026774-21taskset20:01:050
149950,0migration/026736-21latency_hist22:56:170
149950,0migration/025484-21kthreadcore00:06:370
149950,0migration/022141-21kthreadcore00:21:350
149950,0migration/020255-21fschecks_count19:11:320
149950,0migration/020108-21kthreadcore22:31:350
149950,0migration/01816-21kthreadcore19:41:360
149950,0migration/017871-21kthreadcore19:31:340
149950,0migration/017092-21cpuspeed_turbos21:21:320
149950,0migration/017062-21sh00:20:520
149950,0migration/016814-21fschecks_count21:56:320
149950,0migration/015878-21sh22:47:120
149950,0migration/015087-21missed_timers20:11:370
149950,0migration/014207-21kthreadcore23:41:360
149950,0migration/011614-21sendmail_mailst19:46:410
149950,0migration/010889-21aten_r1power_cu22:11:280
149950,0migration/010402-21sh23:02:050
339940,0migration/36600-21runrttasks20:31:343
339940,0migration/33981-21kthreadcore20:06:383
339940,0migration/313894-21kthreadcore20:11:363
339940,0migration/313524-21runrttasks19:51:313
339940,0migration/310612-21interrupts20:56:343
329940,0watchdog/36314-21runrttasks19:58:433
329940,0watchdog/322601-21sensors20:16:393
329940,0watchdog/30-21swapper/323:31:363
329940,0watchdog/30-21swapper/320:41:353
329940,0watchdog/30-21swapper/319:41:353
329940,0watchdog/30-21swapper/319:30:073
29815040,0irq/29-eth0-tx-6673-21ssh22:26:342
29815040,0irq/29-eth0-tx-4103-21ssh21:31:332
29815040,0irq/29-eth0-tx-26652-21kthreadcore23:31:352
29815040,0irq/29-eth0-tx-25208-21ssh22:01:182
29815040,0irq/29-eth0-tx-0-21swapper/223:46:192
29815040,0irq/29-eth0-tx-0-21swapper/223:11:262
29815040,0irq/29-eth0-tx-0-21swapper/222:56:392
29815040,0irq/29-eth0-tx-0-21swapper/222:46:392
29815040,0irq/29-eth0-tx-0-21swapper/221:56:452
29815040,0irq/29-eth0-tx-0-21swapper/221:11:332
29815040,0irq/29-eth0-tx-0-21swapper/021:06:280
29805040,0irq/28-eth0-rx-0-21swapper/322:42:123
29805040,0irq/28-eth0-rx-0-21swapper/322:26:193
259940,0migration/29440-21kthreadcore21:16:332
259940,0migration/29142-21kthreadcore21:51:372
259940,0migration/27031-21kthreadcore19:46:362
259940,0migration/232507-21kthreadcore20:26:362
259940,0migration/228307-21irqcore20:01:372
259940,0migration/225981-21kthreadcore22:36:352
259940,0migration/22488-21kthreadcore19:41:362
259940,0migration/223942-21munin-node22:51:392
259940,0migration/222576-21munin-node20:16:392
259940,0migration/21987-21cpu22:06:302
259940,0migration/212124-21aten_r1power_fr00:36:302
249940,0watchdog/29054-21cut19:26:272
249940,0watchdog/25691-21tune2fs23:56:322
249940,0watchdog/23054-21ssh00:31:282
249940,0watchdog/230000-21grep21:26:352
249940,0watchdog/227836-21grep20:46:312
249940,0watchdog/227165-21fschecks_time00:26:322
249940,0watchdog/220168-21kthreadcore21:01:352
249940,0watchdog/215307-21kthreadcore23:41:362
249940,0watchdog/21191-21kthreadcore23:16:362
194459943,0cyclictest4572-21awk23:01:312
194459943,0cyclictest3571-21lldpd21:07:022
194459943,0cyclictest3571-21lldpd20:41:322
194459943,0cyclictest3571-21lldpd20:07:322
194459943,0cyclictest3571-21lldpd19:55:012
194459943,0cyclictest3571-21lldpd19:37:012
194459943,0cyclictest3571-21lldpd19:35:012
194459943,0cyclictest3524-21snmpd20:56:232
194459943,0cyclictest3524-21snmpd19:58:372
194459943,0cyclictest3250-21irqbalance23:36:232
194459943,0cyclictest3250-21irqbalance20:31:232
194459943,0cyclictest3250-21irqbalance00:11:232
194459943,0cyclictest31798-21awk21:46:342
194459943,0cyclictest29978-21awk22:21:362
194459943,0cyclictest28005-21awk19:16:332
194459943,0cyclictest13931-21awk20:36:332
194449944,0cyclictest0-21swapper/123:48:501
194449944,0cyclictest0-21swapper/122:26:511
194449944,0cyclictest0-21swapper/100:17:221
194449943,0cyclictest6787-21awk20:31:351
194449943,0cyclictest4243-21awk00:31:341
194449943,0cyclictest3571-21lldpd23:19:031
194449943,0cyclictest3571-21lldpd23:01:331
194449943,0cyclictest3571-21lldpd21:59:031
194449943,0cyclictest2951-21awk20:06:361
194449943,0cyclictest28116-21awk20:01:361
194449943,0cyclictest27981-21awk19:16:331
194449943,0cyclictest27945-21awk22:56:331
194449943,0cyclictest21135-21awk20:41:321
194449943,0cyclictest17442-21awk20:16:321
194439944,0cyclictest21360-21kworker/0:223:46:380
194439944,0cyclictest21360-21kworker/0:223:21:180
194439944,0cyclictest21360-21kworker/0:200:32:020
194439944,0cyclictest21360-21kworker/0:200:26:260
194439944,0cyclictest21070-21kworker/0:122:41:320
194439944,0cyclictest21070-21kworker/0:122:16:480
194439944,0cyclictest21070-21kworker/0:122:06:440
194439944,0cyclictest21070-21kworker/0:121:52:380
194439944,0cyclictest21070-21kworker/0:121:46:580
194439944,0cyclictest21070-21kworker/0:121:42:120
194439944,0cyclictest21070-21kworker/0:121:03:400
194439944,0cyclictest21070-21kworker/0:120:41:370
194439944,0cyclictest18632-21kworker/0:019:21:210
194439943,0cyclictest3250-21irqbalance21:31:230
194439943,0cyclictest31999-21awk20:26:350
194439943,0cyclictest29437-21awk23:51:320
194439943,0cyclictest2498-21awk23:36:320
194439943,0cyclictest22600-21awk23:11:320
194439943,0cyclictest10101-21awk19:26:310
194439940,0cyclictest12897-21munin-node20:36:410
179940,0migration/18219-21kthreadcore19:46:361
179940,0migration/16127-21kthreadcore20:51:351
179940,0migration/14646-21processes21:46:401
179940,0migration/131004-21fschecks_count19:41:311
179940,0migration/129400-21idleruntime23:51:311
179940,0migration/124498-21taskset21:23:311
169940,0watchdog/19052-21aten_r1power_fr19:26:271
169940,0watchdog/17338-21munin-node21:51:401
169940,0watchdog/15792-21ssh21:13:111
169940,0watchdog/13488-21kthreadcore22:06:361
169940,0watchdog/130915-21ssh00:09:401
169940,0watchdog/129637-21kthreadcore20:46:351
169940,0watchdog/128423-21kthreadcore23:31:361
169940,0watchdog/123670-21ssh23:29:041
169940,0watchdog/121019-21kthreadcore19:31:351
169940,0watchdog/119394-21ssh23:07:121
169940,0watchdog/114930-21cut00:01:321
169940,0watchdog/10-21swapper/123:41:361
169940,0watchdog/10-21swapper/121:26:351
169940,0watchdog/10-21swapper/120:36:311
149940,0migration/011025-21kthreadcore22:26:360
29805030,0irq/28-eth0-rx-0-21swapper/321:06:243
29805030,0irq/28-eth0-rx-0-21swapper/319:46:473
194459933,0cyclictest3571-21lldpd20:23:322
194459932,0cyclictest3571-21lldpd19:21:302
194459932,0cyclictest3571-21lldpd19:11:302
194449933,0cyclictest3571-21lldpd20:28:021
194449933,0cyclictest13758-21awk19:51:331
194449932,0cyclictest19106-21irqstats00:21:341
194449932,0cyclictest15526-21awk22:31:321
194439933,0cyclictest18632-21kworker/0:020:16:170
194439933,0cyclictest18632-21kworker/0:019:51:170
194439930,0cyclictest10184-21perl20:56:320
179930,0migration/130293-21runrttasks20:22:541
169930,0watchdog/10-21swapper/121:01:351
169930,0watchdog/10-21swapper/119:21:311
169930,0watchdog/10-21swapper/119:11:311
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional