You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-29 - 02:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Tue Sep 29, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450970,0irq/25-eth10-21swapper/318:41:573
1780523624,0sleep20-21swapper/218:42:282
1779223522,0sleep00-21swapper/018:42:170
1772323430,0sleep10-21swapper/118:41:301
339960,0migration/330887-21sh20:49:043
339960,0migration/330768-21sh21:30:093
339960,0migration/328442-21sh23:11:223
339960,0migration/325969-21sh00:08:423
339960,0migration/314487-21sh21:40:383
29715060,0irq/26-eth0-rx-27102-21cat21:07:493
178879960,0cyclictest0-21swapper/221:10:202
178869966,0cyclictest16014-21diskmemload21:01:151
178869960,0cyclictest16014-21diskmemload22:56:111
178859966,0cyclictest0-21swapper/022:47:150
178859960,0cyclictest0-21swapper/022:13:220
149960,0migration/022705-21sh21:03:310
149960,0migration/015236-21sh21:21:140
8215050,0irq/24-0000:00:31572-21ls21:32:332
4150,0ktimersoftd/00-21swapper/023:18:190
35150,0ktimersoftd/30-21swapper/321:36:483
339950,0migration/37854-21aten_r1power_vo19:47:453
339950,0migration/331053-21taskset18:50:583
339950,0migration/330082-21latency_hist22:32:333
339950,0migration/323645-21kthreadcore23:47:523
339950,0migration/321726-21idleruntime-cro20:22:333
339950,0migration/321718-21missed_timers20:42:553
339950,0migration/321525-21fschecks_count19:57:473
339950,0migration/319592-21taskset19:30:123
339950,0migration/319460-21ksmtuned23:24:393
339950,0migration/31690-21kthreadcore21:52:513
339950,0migration/316776-21idleruntime21:22:463
339950,0migration/314593-21cpuspeed_turbos19:52:473
29725050,0irq/27-eth0-tx-7355-21kthreadcore22:57:482
29725050,0irq/27-eth0-tx-5734-21munin-node18:57:570
29725050,0irq/27-eth0-tx-22203-21munin-node22:27:532
29725050,0irq/27-eth0-tx-22203-21munin-node22:27:532
29725050,0irq/27-eth0-tx-20986-21munin-node19:57:500
29725050,0irq/27-eth0-tx-19168-21ssh00:07:042
29725050,0irq/27-eth0-tx-16327-21munin-node21:02:482
29725050,0irq/27-eth0-tx-15126-21ssh21:00:022
29725050,0irq/27-eth0-tx-14852-21munin-node22:02:582
29725050,0irq/27-eth0-tx-0-21swapper/223:50:332
29725050,0irq/27-eth0-tx-0-21swapper/222:18:212
29725050,0irq/27-eth0-tx-0-21swapper/222:07:402
29725050,0irq/27-eth0-tx-0-21swapper/221:47:502
29725050,0irq/27-eth0-tx-0-21swapper/200:09:502
29715050,0irq/26-eth0-rx-9344-21iostat_ios20:57:493
29715050,0irq/26-eth0-rx-9180-21kthreadcore22:57:493
29715050,0irq/26-eth0-rx-8520-21grep21:17:453
29715050,0irq/26-eth0-rx-31970-21cut23:12:533
29715050,0irq/26-eth0-rx-30327-21grep22:52:473
29715050,0irq/26-eth0-rx-29630-21ssh21:48:323
29715050,0irq/26-eth0-rx-27993-21munin-node22:27:533
29715050,0irq/26-eth0-rx-27993-21munin-node22:27:533
29715050,0irq/26-eth0-rx-23779-21iostat_ios22:07:493
29715050,0irq/26-eth0-rx-22046-21kthreadcore23:27:483
29715050,0irq/26-eth0-rx-17196-21if_err_eth1.221:02:483
29715050,0irq/26-eth0-rx-14038-21cpuspeed_turbos22:42:453
29715050,0irq/26-eth0-rx-14020-21ssh23:02:473
29715050,0irq/26-eth0-rx-0-21swapper/323:53:103
29715050,0irq/26-eth0-rx-0-21swapper/323:36:283
29715050,0irq/26-eth0-rx-0-21swapper/323:17:423
29715050,0irq/26-eth0-rx-0-21swapper/322:50:183
29715050,0irq/26-eth0-rx-0-21swapper/322:16:143
29715050,0irq/26-eth0-rx-0-21swapper/322:03:023
29715050,0irq/26-eth0-rx-0-21swapper/321:57:523
29715050,0irq/26-eth0-rx-0-21swapper/321:14:563
29715050,0irq/26-eth0-rx-0-21swapper/320:52:423
29715050,0irq/26-eth0-rx-0-21swapper/318:57:343
27150,0ktimersoftd/20-21swapper/222:34:412
259950,0migration/29810-21kthreadcore21:37:502
259950,0migration/22362-21kthreadcore20:52:502
259950,0migration/220905-21latency_hist19:57:342
259950,0migration/220854-21aten_r1power_vo23:27:432
259950,0migration/214252-21sh22:01:402
249950,0watchdog/215258-21cut23:42:512
249950,0watchdog/20-21swapper/220:17:522
19150,0ktimersoftd/10-21swapper/120:48:481
178879955,0cyclictest0-21swapper/221:22:482
178879955,0cyclictest0-21swapper/220:46:132
178879950,0cyclictest0-21swapper/223:18:592
178879950,0cyclictest0-21swapper/222:40:032
178879950,0cyclictest0-21swapper/222:40:032
178869955,0cyclictest0-21swapper/122:14:531
178869954,0cyclictest12840-21cut00:02:481
178869950,0cyclictest0-21swapper/123:54:501
178869950,0cyclictest0-21swapper/121:19:421
178859950,0cyclictest0-21swapper/023:30:560
178859950,0cyclictest0-21swapper/021:15:300
178859950,0cyclictest0-21swapper/019:56:400
179950,0migration/19310-21kthreadcore18:57:521
179950,0migration/18355-21ntp_kernel_pll_20:32:531
179950,0migration/18039-21df_abs19:47:461
179950,0migration/16393-21taskset21:57:041
179950,0migration/15346-21taskset18:55:521
179950,0migration/122124-21rt-features21:02:541
179950,0migration/121294-21diskmemload00:07:501
179950,0migration/121286-21cpuspeed_turbos19:57:461
179950,0migration/110867-21taskset23:39:151
169950,0watchdog/128714-21ssh22:50:351
169950,0watchdog/112524-21port_sshpriv21:57:511
149950,0migration/09903-21kthreadcore22:37:510
149950,0migration/09903-21kthreadcore22:37:510
149950,0migration/07801-21kthreadcore23:57:490
149950,0migration/07622-21kthreadcore23:37:510
149950,0migration/07367-21latency_hist21:37:330
149950,0migration/07016-21sh21:57:420
149950,0migration/06142-21taskset19:18:570
149950,0migration/05978-21gltestperf22:57:450
149950,0migration/032484-21irqrtprio21:52:500
149950,0migration/032256-21munin-node18:52:480
149950,0migration/030935-21kthreadcore23:12:530
149950,0migration/030357-21kthreadcore23:52:490
149950,0migration/024096-21kthreadcore22:47:500
149950,0migration/023861-21sh21:27:350
149950,0migration/023779-21munin-node22:07:490
149950,0migration/022367-21fschecks_count20:22:450
149950,0migration/021129-21irqcore19:32:510
149950,0migration/020651-21kthreadcore21:22:500
149950,0migration/018387-21kthreadcore21:42:510
149950,0migration/018083-21taskset18:42:350
149950,0migration/017150-21kthreadcore20:17:510
149950,0migration/015655-21kthreadcore23:42:510
149950,0migration/014722-21sh21:41:210
149950,0migration/014553-21irqrtprio19:27:500
149950,0migration/011305-21kthreadcore20:12:510
149950,0migration/010934-21kthreadcore20:37:520
149950,0migration/010771-21kthreadcore22:17:510
339940,0migration/36800-21irqcore18:57:503
339940,0migration/34339-21kthreadcore19:42:513
339940,0migration/329058-21kthreadcore20:02:503
339940,0migration/328327-21taskset20:27:133
339940,0migration/323537-21kthreadcore19:07:513
339940,0migration/322479-21kthreadcore18:42:513
29725040,0irq/27-eth0-tx-31248-21munin-node21:52:482
29725040,0irq/27-eth0-tx-27651-21ssh23:08:592
29725040,0irq/27-eth0-tx-18292-21ssh23:22:502
29725040,0irq/27-eth0-tx-0-21swapper/223:53:282
29725040,0irq/27-eth0-tx-0-21swapper/223:12:492
29725040,0irq/27-eth0-tx-0-21swapper/222:52:372
29725040,0irq/27-eth0-tx-0-21swapper/222:44:382
29725040,0irq/27-eth0-tx-0-21swapper/221:42:532
29725040,0irq/27-eth0-tx-0-21swapper/221:32:352
29725040,0irq/27-eth0-tx-0-21swapper/221:18:072
29725040,0irq/27-eth0-tx-0-21swapper/220:47:432
29725040,0irq/27-eth0-tx-0-21swapper/020:02:450
29715040,0irq/26-eth0-rx-9114-40yum-updatesd-he20:34:543
29715040,0irq/26-eth0-rx-16747-21kthreadcore21:42:513
29715040,0irq/26-eth0-rx-0-21swapper/323:57:393
29715040,0irq/26-eth0-rx-0-21swapper/323:42:513
29715040,0irq/26-eth0-rx-0-21swapper/323:37:443
29715040,0irq/26-eth0-rx-0-21swapper/322:37:453
29715040,0irq/26-eth0-rx-0-21swapper/322:37:453
29715040,0irq/26-eth0-rx-0-21swapper/322:23:273
29715040,0irq/26-eth0-rx-0-21swapper/322:19:493
29715040,0irq/26-eth0-rx-0-21swapper/300:05:313
259940,0migration/29497-21kthreadcore19:22:512
259940,0migration/24726-21fschecks_count23:37:472
259940,0migration/229653-21irqcore20:27:502
259940,0migration/222746-21idleruntime20:22:462
259940,0migration/22077-21kthreadcore19:17:502
259940,0migration/219548-21fschecks_count19:07:472
259940,0migration/219155-21kthreadcore22:22:542
259940,0migration/217368-21kthreadcore19:02:512
259940,0migration/216768-21kthreadcore23:02:502
259940,0migration/214849-21kthreadcore19:27:512
249940,0watchdog/26074-21ssh21:12:552
249940,0watchdog/25967-21kthreadcore20:32:522
249940,0watchdog/230704-21kthreadcore19:12:522
249940,0watchdog/21711-21munin-node19:42:482
249940,0watchdog/20-21swapper/222:12:472
249940,0watchdog/20-21swapper/219:52:522
178879943,0cyclictest3556-21lldpd23:58:402
178879943,0cyclictest3556-21lldpd22:47:402
178879943,0cyclictest3556-21lldpd19:37:392
178879943,0cyclictest29545-21awk23:32:492
178879943,0cyclictest28457-21awk20:02:482
178879943,0cyclictest25734-21awk18:47:492
178879943,0cyclictest20995-21awk19:32:512
178869944,0cyclictest0-21swapper/121:49:011
178869943,0cyclictest5686-21awk23:17:471
178869943,0cyclictest3556-21lldpd21:37:411
178869943,0cyclictest3514-21snmpd20:26:381
178869943,0cyclictest29508-21awk20:27:491
178869943,0cyclictest2802-21awk20:07:481
178869943,0cyclictest26595-21awk19:12:491
178869943,0cyclictest24792-21awk21:27:481
178869943,0cyclictest19806-21sadc19:32:341
178869943,0cyclictest16420-21awk21:42:501
178869943,0cyclictest16149-21awk20:17:491
178859944,0cyclictest0-21swapper/023:07:170
178859943,0cyclictest9267-21awk20:57:480
178859943,0cyclictest8632-21awk19:47:490
178859943,0cyclictest30142-21kworker/0:223:10:140
178859943,0cyclictest29532-21awk20:27:490
178859943,0cyclictest2772-21awk20:07:480
178859943,0cyclictest13044-21kworker/0:019:07:570
178859943,0cyclictest1053-21awk20:52:490
178859942,0cyclictest29936-21kworker/0:123:24:380
178859940,0cyclictest19342-21basename00:07:320
179940,0migration/17872-21kthreadcore22:37:511
179940,0migration/17872-21kthreadcore22:37:511
179940,0migration/15653-21processes19:17:541
179940,0migration/13792-21kthreadcore21:32:491
179940,0migration/128578-21kthreadcore21:07:501
179940,0migration/126417-21kthreadcore23:07:511
179940,0migration/126250-21kthreadcore22:07:501
179940,0migration/124679-21kthreadcore23:27:491
179940,0migration/119083-21kthreadcore20:42:531
179940,0migration/118509-21kthreadcore19:52:521
179940,0migration/116412-21kthreadcore22:02:551
179940,0migration/114643-21kthreadcore23:22:471
179940,0migration/114411-21kthreadcore23:42:511
179940,0migration/113185-21hwlatdetect19:02:481
179940,0migration/112857-21taskset19:24:181
179940,0migration/111037-21runrttasks00:01:351
169940,0watchdog/16500-21ssh20:53:321
169940,0watchdog/129206-21aten2.4-expect23:12:471
169940,0watchdog/123488-21ssh21:27:071
169940,0watchdog/122350-21ssh22:27:431
169940,0watchdog/122350-21ssh22:27:431
169940,0watchdog/120387-21ssh23:06:031
169940,0watchdog/113961-21ssh22:22:311
169940,0watchdog/110860-21open_inodes22:57:511
169940,0watchdog/10-21swapper/123:50:071
169940,0watchdog/10-21swapper/123:32:511
169940,0watchdog/10-21swapper/119:32:521
149940,0migration/030090-21missed_timers18:47:520
149940,0migration/026509-21kthreadcore22:27:490
149940,0migration/026509-21kthreadcore22:27:490
149940,0migration/025876-21kthreadcore21:07:490
149940,0migration/018873-21kthreadcore22:22:540
149940,0migration/01047-21kthreadcore22:52:510
29715030,0irq/26-eth0-rx-0-21swapper/320:37:513
29715030,0irq/26-eth0-rx-0-21swapper/320:12:523
29715030,0irq/26-eth0-rx-0-21swapper/319:37:453
29715030,0irq/26-eth0-rx-0-21swapper/319:32:503
29715030,0irq/26-eth0-rx-0-21swapper/319:20:383
29715030,0irq/26-eth0-rx-0-21swapper/319:15:023
29715030,0irq/26-eth0-rx-0-21swapper/319:06:313
249930,0watchdog/210938-21tr18:57:522
249930,0watchdog/20-21swapper/220:12:562
249930,0watchdog/20-21swapper/220:07:562
178879932,0cyclictest4060-21kthreadcore18:52:512
178879932,0cyclictest3556-21lldpd19:50:092
178879932,0cyclictest19009-21awk18:42:492
178879932,0cyclictest10441-21awk20:37:502
178869932,0cyclictest9568-21awk20:12:491
178869932,0cyclictest3809-21grep19:42:511
178869932,0cyclictest3241-21irqbalance20:03:021
178869932,0cyclictest27616-21awk19:37:481
178869932,0cyclictest18953-21awk18:42:481
178869932,0cyclictest14277-21aten_r1power_cu22:22:471
178869930,0cyclictest5837-21python21:12:541
178859933,0cyclictest9412-21kworker/0:120:47:370
178859933,0cyclictest9412-21kworker/0:120:42:350
178859933,0cyclictest9412-21kworker/0:120:32:350
178859933,0cyclictest30142-21kworker/0:222:32:520
178859933,0cyclictest30142-21kworker/0:222:02:360
178859933,0cyclictest29936-21kworker/0:123:47:340
178859933,0cyclictest29936-21kworker/0:123:32:360
178859933,0cyclictest29936-21kworker/0:100:07:340
178859933,0cyclictest13044-21kworker/0:019:42:410
178859933,0cyclictest13044-21kworker/0:019:22:350
178859933,0cyclictest13044-21kworker/0:019:12:350
178859933,0cyclictest13044-21kworker/0:019:02:350
178859932,0cyclictest3514-21snmpd21:47:380
178859932,0cyclictest27664-21awk19:37:490
169930,0watchdog/119906-21munin-node19:07:481
169930,0watchdog/10-21swapper/122:42:551
169930,0watchdog/10-21swapper/122:32:431
169930,0watchdog/10-21swapper/120:37:521
169930,0watchdog/10-21swapper/118:47:561
29715020,0irq/26-eth0-rx-0-21swapper/320:27:383
29715020,0irq/26-eth0-rx-0-21swapper/320:07:493
29715020,0irq/26-eth0-rx-0-21swapper/319:23:233
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional