You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-09 - 11:17
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2.osadl.org (updated Fri Jan 09, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2053224131,0sleep30-21swapper/319:11:293
2055623626,0sleep20-21swapper/219:11:482
2048123626,0sleep10-21swapper/119:10:551
2032322823,0sleep00-21swapper/019:09:020
339960,0migration/33688-21sh22:23:543
339960,0migration/331681-21munin-node00:27:443
339960,0migration/329161-21sh21:43:063
339960,0migration/324752-21sh22:54:043
339960,0migration/320843-21sh21:39:393
339960,0migration/311983-21munin-node22:12:303
329960,0watchdog/34904-21sed00:12:443
29805060,0irq/28-eth0-rx-8316-21ssh22:47:123
29805060,0irq/28-eth0-rx-22003-21ssh23:27:483
29805060,0irq/28-eth0-rx-18088-21grep22:32:353
29805060,0irq/28-eth0-rx-15276-21cut19:52:343
206759960,0cyclictest0-21swapper/223:57:482
206749966,0cyclictest31899-21diskmemload23:22:091
206749966,0cyclictest31899-21diskmemload23:01:201
206749960,0cyclictest0-21swapper/121:14:361
149960,0migration/05547-21sh21:47:440
149960,0migration/05547-21sh21:47:440
149960,0migration/05310-21latency_hist22:27:160
149960,0migration/030948-21irqcore23:17:340
149960,0migration/015786-21kthreadcore00:37:350
35150,0ktimersoftd/30-21swapper/322:19:473
339950,0migration/39936-21kthreadcore22:27:353
339950,0migration/39082-21munin-node19:22:363
339950,0migration/35335-21sh23:20:503
339950,0migration/328354-21taskset21:27:323
339950,0migration/327000-21fschecks_time22:57:293
339950,0migration/326903-21munin-node23:12:363
339950,0migration/324012-21kthreadcore19:57:333
339950,0migration/322128-21fschecks_time20:42:293
339950,0migration/320030-21sh00:03:493
339950,0migration/310968-21munin-node20:57:283
339950,0migration/310686-21kthreadcore21:17:333
329950,0watchdog/38084-21perl00:17:323
329950,0watchdog/332180-21cpu20:27:273
329950,0watchdog/30-21swapper/322:47:203
329950,0watchdog/30-21swapper/300:08:043
29815050,0irq/29-eth0-tx-9982-21ssh23:03:212
29815050,0irq/29-eth0-tx-632-21ssh22:04:382
29815050,0irq/29-eth0-tx-28823-21munin-node22:22:402
29815050,0irq/29-eth0-tx-2753-21kthreadcore21:47:352
29815050,0irq/29-eth0-tx-2753-21kthreadcore21:47:352
29815050,0irq/29-eth0-tx-24556-21munin-node19:37:430
29815050,0irq/29-eth0-tx-21140-21munin-node22:17:332
29815050,0irq/29-eth0-tx-18148-21cpu21:22:292
29815050,0irq/29-eth0-tx-17726-21munin-node21:02:340
29815050,0irq/29-eth0-tx-16823-21kthreadcore23:27:352
29815050,0irq/29-eth0-tx-15135-21ssh00:20:342
29815050,0irq/29-eth0-tx-14486-21tr21:17:342
29815050,0irq/29-eth0-tx-13124-21iostat_ios23:07:332
29815050,0irq/29-eth0-tx-11046-21munin-node22:12:332
29815050,0irq/29-eth0-tx-0-21swapper/223:25:552
29815050,0irq/29-eth0-tx-0-21swapper/222:44:432
29815050,0irq/29-eth0-tx-0-21swapper/222:08:512
29815050,0irq/29-eth0-tx-0-21swapper/221:57:312
29815050,0irq/29-eth0-tx-0-21swapper/221:33:452
29815050,0irq/29-eth0-tx-0-21swapper/221:15:092
29815050,0irq/29-eth0-tx-0-21swapper/020:02:140
29805050,0irq/28-eth0-rx-6226-21munin-node23:22:263
29805050,0irq/28-eth0-rx-4041-21if_eth023:02:313
29805050,0irq/28-eth0-rx-25987-21diskstats22:02:293
29805050,0irq/28-eth0-rx-22377-21cut21:22:353
29805050,0irq/28-eth0-rx-18546-21kthreadcore21:57:333
29805050,0irq/28-eth0-rx-0-21swapper/323:42:313
29805050,0irq/28-eth0-rx-0-21swapper/323:38:023
29805050,0irq/28-eth0-rx-0-21swapper/323:33:583
29805050,0irq/28-eth0-rx-0-21swapper/323:08:403
29805050,0irq/28-eth0-rx-0-21swapper/322:07:323
29805050,0irq/28-eth0-rx-0-21swapper/321:56:053
29805050,0irq/28-eth0-rx-0-21swapper/321:47:353
29805050,0irq/28-eth0-rx-0-21swapper/321:47:353
29805050,0irq/28-eth0-rx-0-21swapper/321:35:133
29805050,0irq/28-eth0-rx-0-21swapper/321:17:143
29805050,0irq/28-eth0-rx-0-21swapper/321:07:343
29805050,0irq/28-eth0-rx-0-21swapper/300:40:523
29805050,0irq/28-eth0-rx-0-21swapper/300:34:163
29805050,0irq/28-eth0-rx-0-21swapper/300:22:253
259950,0migration/28335-21sh23:40:022
259950,0migration/27681-21munin-node00:32:372
259950,0migration/231629-21unixbench_singl00:27:432
259950,0migration/227888-21sh23:13:342
259950,0migration/220473-21munin-node22:32:382
259950,0migration/212249-21kthreadcore00:37:342
249950,0watchdog/231925-21kthreadcore20:47:352
249950,0watchdog/219433-21kthreadcore19:52:352
206759950,0cyclictest0-21swapper/223:01:492
206749955,0cyclictest0-21swapper/123:27:211
206749955,0cyclictest0-21swapper/119:57:401
206749955,0cyclictest0-21swapper/100:17:191
206749955,0cyclictest0-21swapper/100:16:131
206749950,0cyclictest0-21swapper/122:24:551
206749950,0cyclictest0-21swapper/121:48:111
206749950,0cyclictest0-21swapper/121:48:111
206739955,0cyclictest0-21swapper/023:56:100
206739955,0cyclictest0-21swapper/023:08:140
206739955,0cyclictest0-21swapper/021:24:200
206739950,0cyclictest0-21swapper/023:49:510
206739950,0cyclictest0-21swapper/023:43:330
179950,0migration/19182-21munin-node21:17:281
179950,0migration/131743-21taskset19:42:181
179950,0migration/118477-21munin-node19:32:301
179950,0migration/11824-21kthreadcore22:42:341
179950,0migration/116413-21kthreadcore20:37:341
149950,0migration/09538-21timerwakeupswit23:02:420
149950,0migration/06065-21munin-node20:27:410
149950,0migration/04396-21taskset22:25:370
149950,0migration/032560-21ksmtuned23:37:020
149950,0migration/032055-21timerandwakeup22:02:420
149950,0migration/029731-21kthreadcore21:07:340
149950,0migration/02956-21fschecks_time22:07:300
149950,0migration/024929-21kthreadcore00:07:340
149950,0migration/024746-21latency_hist20:22:160
149950,0migration/024396-21munin-node22:37:330
149950,0migration/023664-21kthreadcore23:12:330
149950,0migration/019664-21kthreadcore22:32:350
149950,0migration/019065-21kthreadcore00:22:340
149950,0migration/018429-21fschecks_time20:17:300
149950,0migration/017579-21cpuspeed_turbos19:32:260
149950,0migration/015612-21sh00:21:450
149950,0migration/015224-21sh23:27:280
149950,0migration/014266-21timerandwakeup21:52:430
149950,0migration/013980-21kthreadcore21:17:340
149950,0migration/01202-21kthreadcore20:47:350
149950,0migration/010520-21latency_hist20:57:170
8305040,0irq/26-0000:00:20440-21sort20:37:422
8305040,0irq/26-0000:00:0-21swapper/223:42:172
339940,0migration/35062-21kthreadcore20:52:333
339940,0migration/33832-21sh23:57:203
339940,0migration/325471-21fschecks_time20:22:283
339940,0migration/324793-21latency_hist21:07:173
339940,0migration/324492-21latency_hist19:37:163
339940,0migration/323981-21sh22:37:303
329940,0watchdog/36314-21runrttasks19:50:433
329940,0watchdog/31465-21sort19:17:353
329940,0watchdog/30-21swapper/323:52:283
329940,0watchdog/30-21swapper/320:32:353
329940,0watchdog/30-21swapper/320:18:513
329940,0watchdog/30-21swapper/320:12:273
329940,0watchdog/30-21swapper/320:02:313
29815040,0irq/29-eth0-tx-31845-21telnet19:42:270
29815040,0irq/29-eth0-tx-0-21swapper/223:52:222
29815040,0irq/29-eth0-tx-0-21swapper/222:47:352
29815040,0irq/29-eth0-tx-0-21swapper/222:37:282
29815040,0irq/29-eth0-tx-0-21swapper/200:08:132
29815040,0irq/29-eth0-tx-0-21swapper/020:32:400
29815040,0irq/29-eth0-tx-0-21swapper/019:27:370
29805040,0irq/28-eth0-rx-10959-21fschecks_time19:27:293
29805040,0irq/28-eth0-rx-0-21swapper/323:47:173
259940,0migration/25721-21kthreadcore19:22:322
259940,0migration/25322-21sh22:27:192
259940,0migration/25230-21kthreadcore20:07:332
259940,0migration/231477-21taskset19:41:552
259940,0migration/227064-21latency_hist21:27:172
259940,0migration/226168-21munin-node20:22:322
259940,0migration/224258-21aten_r1power_po23:32:282
259940,0migration/221698-21kthreadcore21:02:332
259940,0migration/219754-21kthreadcore22:52:342
259940,0migration/21731-21kthreadcore19:42:332
259940,0migration/210912-21kthreadcore19:47:342
259940,0migration/210846-21kthreadcore21:52:352
249940,0watchdog/28494-21kthreadcore20:32:352
249940,0watchdog/2703-21iostat20:27:312
249940,0watchdog/231405-21tune2fs00:12:322
249940,0watchdog/213905-21if_err_eth1.200:02:322
249940,0watchdog/211704-21if_eth1.320:57:312
249940,0watchdog/20-21swapper/223:47:242
249940,0watchdog/20-21swapper/200:22:282
206759943,0cyclictest3571-21lldpd21:09:382
206759943,0cyclictest3571-21lldpd20:55:382
206759943,0cyclictest3571-21lldpd20:45:082
206759943,0cyclictest3571-21lldpd20:20:082
206759943,0cyclictest3571-21lldpd20:15:382
206759943,0cyclictest3571-21lldpd20:05:082
206759943,0cyclictest3571-21lldpd19:30:062
206759943,0cyclictest3571-21lldpd19:19:352
206759943,0cyclictest3571-21lldpd19:17:062
206759943,0cyclictest3524-21snmpd19:32:402
206759943,0cyclictest29706-21munin-node23:17:312
206759943,0cyclictest23364-21awk21:42:302
206759943,0cyclictest22115-21awk19:57:312
206759943,0cyclictest14215-21awk21:37:322
206749944,0cyclictest0-21swapper/122:13:241
206749943,0cyclictest4098-21awk19:22:311
206749943,0cyclictest3571-21lldpd23:48:131
206749943,0cyclictest3571-21lldpd23:13:421
206749943,0cyclictest3571-21lldpd23:04:431
206749943,0cyclictest3571-21lldpd22:37:431
206749943,0cyclictest3571-21lldpd22:03:091
206749943,0cyclictest3571-21lldpd21:43:391
206749943,0cyclictest3571-21lldpd21:41:391
206749943,0cyclictest3571-21lldpd21:34:391
206749943,0cyclictest3571-21lldpd21:23:381
206749943,0cyclictest25953-21awk20:22:311
206749943,0cyclictest22848-21awk00:07:321
206749943,0cyclictest22318-21awk20:42:311
206749943,0cyclictest17869-21awk21:57:321
206749943,0cyclictest17251-21awk00:22:311
206749943,0cyclictest13099-21awk23:07:331
206749943,0cyclictest11545-21awk00:37:321
206749943,0cyclictest11386-21awk20:12:301
206749942,0cyclictest2307-21kthreadcore23:37:341
206739944,0cyclictest4272-21kworker/0:121:36:520
206739944,0cyclictest0-21swapper/023:38:210
206739944,0cyclictest0-21swapper/000:02:280
206739943,0cyclictest698-21awk22:42:320
206739943,0cyclictest3571-21lldpd20:42:080
206739943,0cyclictest3524-21snmpd19:52:400
206739943,0cyclictest3524-21snmpd19:50:400
206739943,0cyclictest3250-21irqbalance23:23:030
206739943,0cyclictest3250-21irqbalance22:17:330
206739943,0cyclictest3250-21irqbalance21:43:130
206739943,0cyclictest3250-21irqbalance20:13:030
206739943,0cyclictest3250-21irqbalance20:08:130
206739943,0cyclictest21917-21awk19:12:300
206739943,0cyclictest18694-21awk22:52:330
206739943,0cyclictest14272-21awk21:37:330
179940,0migration/17417-21kthreadcore22:27:341
179940,0migration/13999-21fschecks_time20:52:291
179940,0migration/130804-21missed_timers21:07:351
179940,0migration/12336-21kthreadcore20:27:321
179940,0migration/120400-21ntp_kernel_pll_22:32:371
179940,0migration/117231-21kthreadcore00:02:351
179940,0migration/113802-21missed_timers21:52:361
179940,0migration/113273-21taskset19:48:191
179940,0migration/112382-21kthreadcore20:57:321
179940,0migration/110201-21sh22:10:581
179940,0migration/110196-21aten_r1power_cu19:27:271
179940,0migration/110133-21fschecks_count23:42:311
169940,0watchdog/18588-21ssh22:47:201
169940,0watchdog/18350-21cut23:22:321
169940,0watchdog/13916-21ssh23:57:281
169940,0watchdog/130149-21netstat23:32:361
169940,0watchdog/128892-21fschecks_time20:02:271
169940,0watchdog/124237-21chrt20:18:511
169940,0watchdog/12309-21ssh21:29:351
169940,0watchdog/11425-21ssh00:32:241
169940,0watchdog/10-21swapper/122:17:441
169940,0watchdog/10-21swapper/119:52:351
149940,0migration/025981-21kthreadcore20:42:340
149940,0migration/025299-21cpuspeed_turbos00:27:290
149940,0migration/018546-21sh22:14:400
149940,0migration/01718-21diskmemload23:00:150
339930,0migration/317512-21munin-node19:32:263
329930,0watchdog/30-21swapper/320:47:353
329930,0watchdog/30-21swapper/320:37:353
329930,0watchdog/30-21swapper/319:42:273
29805030,0irq/28-eth0-rx-0-21swapper/320:07:273
29805030,0irq/28-eth0-rx-0-21swapper/319:12:363
206749932,0cyclictest7913-21awk20:32:331
206749932,0cyclictest3524-21snmpd20:47:221
206749932,0cyclictest21061-21tr22:52:341
206749930,0cyclictest4349-21irqcore20:07:321
206739932,0cyclictest9831-21awk22:47:340
206739932,0cyclictest6678-21kthreadcore19:22:320
206739932,0cyclictest4334-21awk20:52:310
206739932,0cyclictest3571-21lldpd00:33:430
206739932,0cyclictest29382-21awk20:02:290
206739932,0cyclictest28243-21awk21:27:310
206739932,0cyclictest12532-21sadc00:02:170
206739931,0cyclictest17224-21fschecks_count21:57:300
206739930,0cyclictest397-21perl21:12:290
206739930,0cyclictest31524-21perl00:12:320
206739930,0cyclictest0-21swapper/019:17:340
179930,0migration/131214-21runrttasks19:38:511
169930,0watchdog/131899-21diskmemload00:30:081
169930,0watchdog/10-21swapper/123:52:281
169930,0watchdog/10-21swapper/121:02:311
169930,0watchdog/10-21swapper/119:17:351
169920,0watchdog/10-21swapper/119:12:191
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional