You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-15 - 01:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Fri May 15, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
328525049,0sleep20-21swapper/218:50:532
304450460,0irq/25-eth10-21swapper/318:51:113
849323624,0sleep10-21swapper/118:51:521
855623524,0sleep00-21swapper/018:52:350
88889988,0cyclictest3514-21snmpd22:35:561
88899960,0cyclictest0-21swapper/223:03:242
88889965,0cyclictest6320-21sadc22:55:361
88889960,0cyclictest6928-21diskmemload21:08:411
88879965,0cyclictest25052-21ps21:25:520
88879960,0cyclictest0-21swapper/023:29:410
339960,0migration/34990-21sh23:54:063
339960,0migration/34638-21munin-node22:10:543
339960,0migration/332527-21kthreadcore19:35:533
339960,0migration/322779-21aten_r1power_vo23:05:483
339960,0migration/322050-21sh21:02:453
339960,0migration/319494-21sh23:40:563
29725060,0irq/27-eth0-tx-19877-21sensors_volt23:41:002
29725060,0irq/27-eth0-tx-10027-21munin-node19:20:550
29725060,0irq/27-eth0-tx-0-21swapper/221:37:292
29725060,0irq/27-eth0-tx-0-21swapper/221:37:292
29715060,0irq/26-eth0-rx-31062-21perl23:30:493
29715060,0irq/26-eth0-rx-30239-21diskstats00:10:493
29715060,0irq/26-eth0-rx-21153-21ntpq21:40:553
29715060,0irq/26-eth0-rx-0-21swapper/323:58:433
179960,0migration/129655-21sh22:06:131
149960,0migration/03149-21missed_timers23:50:570
149960,0migration/014347-21sh22:19:030
149960,0migration/013614-21timerwakeupswit21:16:010
149960,0migration/013430-21sh23:39:290
149960,0migration/012391-21sh23:57:130
88899955,0cyclictest0-21swapper/222:27:372
88899955,0cyclictest0-21swapper/200:07:152
88899950,0cyclictest0-21swapper/223:20:452
88899950,0cyclictest0-21swapper/222:08:042
88889950,0cyclictest0-21swapper/123:06:461
88889950,0cyclictest0-21swapper/121:15:091
88889950,0cyclictest0-21swapper/100:05:441
88879955,0cyclictest0-21swapper/000:21:330
88879954,0cyclictest8033-21awk20:55:520
88879950,0cyclictest0-21swapper/022:56:060
8215050,0irq/24-0000:00:6928-21diskmemload21:15:372
8215050,0irq/24-0000:00:15518-21latency_hist19:00:372
339950,0migration/39381-21kthreadcore21:35:543
339950,0migration/39381-21kthreadcore21:35:543
339950,0migration/38799-21kthreadcore20:05:523
339950,0migration/38300-21kthreadcore22:35:533
339950,0migration/37431-21aten_r1power_cu21:55:473
339950,0migration/36837-21diskmemload21:14:213
339950,0migration/36516-21taskset20:51:113
339950,0migration/35402-21munin-node19:40:523
339950,0migration/3379-21munin-node20:25:513
339950,0migration/331566-21fschecks_time20:00:493
339950,0migration/329305-21kthreadcore20:20:553
339950,0migration/328393-21needreboot22:45:563
339950,0migration/325803-21kthreadcore22:25:543
339950,0migration/324956-21kthreadcore23:45:523
339950,0migration/324591-21taskset20:16:083
339950,0migration/324066-21kthreadcore00:05:523
339950,0migration/323516-21irqcore19:05:513
339950,0migration/320533-21irqcore20:40:523
339950,0migration/317728-21kthreadcore23:20:543
339950,0migration/317585-21latency_hist19:50:373
339950,0migration/315674-21latency_hist19:00:373
339950,0migration/312947-21kthreadcore19:45:533
339950,0migration/312896-21sh23:17:103
339950,0migration/312245-21taskset20:31:263
29725050,0irq/27-eth0-tx-29363-21ssh23:28:302
29725050,0irq/27-eth0-tx-26546-21grep21:45:532
29725050,0irq/27-eth0-tx-15456-21munin-node22:00:582
29725050,0irq/27-eth0-tx-14138-21ssh21:57:362
29725050,0irq/27-eth0-tx-13758-21ssh22:58:452
29725050,0irq/27-eth0-tx-12876-21ssh00:19:302
29725050,0irq/27-eth0-tx-0-21swapper/223:53:542
29725050,0irq/27-eth0-tx-0-21swapper/222:15:572
29725050,0irq/27-eth0-tx-0-21swapper/221:31:152
29725050,0irq/27-eth0-tx-0-21swapper/200:10:372
29725050,0irq/27-eth0-tx-0-21swapper/020:15:310
29725050,0irq/27-eth0-tx-0-21swapper/019:50:540
29725050,0irq/27-eth0-tx-0-21swapper/019:50:260
29715050,0irq/26-eth0-rx-4777-21kthreadcore19:15:523
29715050,0irq/26-eth0-rx-32176-21fschecks_count21:50:493
29715050,0irq/26-eth0-rx-29606-21ssh22:06:053
29715050,0irq/26-eth0-rx-28100-21which23:25:573
29715050,0irq/26-eth0-rx-16266-21idleruntime22:00:503
29715050,0irq/26-eth0-rx-15719-21if_err_eth022:40:513
29715050,0irq/26-eth0-rx-15097-21entropy23:00:493
29715050,0irq/26-eth0-rx-13423-21ssh20:56:273
29715050,0irq/26-eth0-rx-13405-21runrttasks22:16:203
29715050,0irq/26-eth0-rx-11940-21kthreadcore21:15:553
29715050,0irq/26-eth0-rx-0-21swapper/323:40:303
29715050,0irq/26-eth0-rx-0-21swapper/323:10:483
29715050,0irq/26-eth0-rx-0-21swapper/322:55:563
29715050,0irq/26-eth0-rx-0-21swapper/322:50:523
29715050,0irq/26-eth0-rx-0-21swapper/321:25:573
29715050,0irq/26-eth0-rx-0-21swapper/321:20:433
29715050,0irq/26-eth0-rx-0-21swapper/321:05:543
29715050,0irq/26-eth0-rx-0-21swapper/300:20:553
29715050,0irq/26-eth0-rx-0-21swapper/300:15:503
259950,0migration/25530-21sh22:32:372
259950,0migration/231989-21irqcore22:50:522
259950,0migration/23185-21munin-node19:35:572
259950,0migration/229339-21sh23:49:022
259950,0migration/22831-21kthreadcore20:50:532
259950,0migration/224772-21fschecks_count19:55:482
259950,0migration/224090-21munin-node19:30:512
259950,0migration/222180-21sh21:21:392
259950,0migration/221983-21sh22:23:052
259950,0migration/217266-21kthreadcore00:00:542
259950,0migration/213495-21sh00:20:372
249950,0watchdog/212185-21latency22:35:542
179950,0migration/17226-21latency_hist21:55:371
179950,0migration/16154-21sh22:34:211
179950,0migration/131250-21munin-node23:30:501
179950,0migration/129720-21taskset19:35:361
179950,0migration/122079-21kthreadcore20:40:531
179950,0migration/122012-21sh21:21:101
179950,0migration/119284-21gltestperf20:15:491
179950,0migration/113267-21sh22:16:001
179950,0migration/111665-21kthreadcore22:55:571
169950,0watchdog/121856-21ssh23:25:021
169950,0watchdog/119147-21cut19:50:551
149950,0migration/07306-21turbostat21:15:370
149950,0migration/06582-21sh21:33:300
149950,0migration/05473-21kthreadcore20:50:530
149950,0migration/031877-21kthreadcore00:10:540
149950,0migration/031489-21aten_r1power_en22:10:450
149950,0migration/031432-21kthreadcore19:35:530
149950,0migration/029270-21pmu-power21:45:560
149950,0migration/029226-21sh23:48:430
149950,0migration/028595-21munin-node19:30:550
149950,0migration/027701-21missed_timers19:05:530
149950,0migration/026806-21fschecks_time20:45:500
149950,0migration/025664-21kthreadcore22:25:530
149950,0migration/025313-21irqcore19:55:510
149950,0migration/023580-21kthreadcore20:15:540
149950,0migration/020463-21switchtime23:00:590
149950,0migration/019662-21aten_r1power_vo20:40:480
149950,0migration/018709-21kthreadcore22:20:540
149950,0migration/018445-21kthreadcore19:00:530
149950,0migration/017551-21kthreadcore00:00:540
149950,0migration/016349-21kthreadcore22:40:530
149950,0migration/012778-21sh23:16:500
149950,0migration/012406-21taskset20:33:190
88899943,0cyclictest3241-21irqbalance21:05:452
88899943,0cyclictest3241-21irqbalance19:50:552
88899943,0cyclictest13749-21awk20:35:512
88899943,0cyclictest10895-21awk19:20:512
88889944,0cyclictest0-21swapper/123:36:341
88889944,0cyclictest0-21swapper/121:33:161
88889943,0cyclictest8079-21awk20:55:531
88889943,0cyclictest7383-21ps23:15:531
88889943,0cyclictest3556-21lldpd22:47:571
88889943,0cyclictest3556-21lldpd19:20:541
88889943,0cyclictest3514-21snmpd21:48:021
88889943,0cyclictest3514-21snmpd20:52:321
88889943,0cyclictest3514-21snmpd20:47:321
88889943,0cyclictest3514-21snmpd20:36:321
88889943,0cyclictest3514-21snmpd20:34:021
88889943,0cyclictest3514-21snmpd20:01:171
88889943,0cyclictest3514-21snmpd19:55:471
88889943,0cyclictest3514-21snmpd19:10:021
88889943,0cyclictest24088-21awk22:25:511
88889943,0cyclictest12784-21awk20:10:491
88889940,0cyclictest7827-21munin-node21:15:491
88879943,0cyclictest7942-21kworker/0:122:40:270
88879943,0cyclictest3514-21snmpd23:10:470
88879943,0cyclictest32615-21awk21:50:510
88879943,0cyclictest31838-21awk20:00:510
88879943,0cyclictest31693-21kworker/0:121:21:060
88879943,0cyclictest16242-21kworker/0:222:09:220
88879943,0cyclictest15029-21awk23:40:520
8215040,0irq/24-0000:00:11795-21cat20:10:372
339940,0migration/329079-21sh19:10:473
339940,0migration/326324-21kthreadcore19:55:523
339940,0migration/315145-21sh22:20:463
339940,0migration/314676-21kthreadcore20:35:523
339940,0migration/310372-21kernelversion18:55:533
29725040,0irq/27-eth0-tx-30347-21ssh22:49:432
29725040,0irq/27-eth0-tx-18032-21cut21:40:532
29725040,0irq/27-eth0-tx-0-21swapper/223:55:372
29725040,0irq/27-eth0-tx-0-21swapper/223:30:532
29725040,0irq/27-eth0-tx-0-21swapper/223:12:032
29725040,0irq/27-eth0-tx-0-21swapper/222:43:542
29725040,0irq/27-eth0-tx-0-21swapper/222:11:032
29725040,0irq/27-eth0-tx-0-21swapper/221:25:502
29725040,0irq/27-eth0-tx-0-21swapper/221:00:572
29725040,0irq/27-eth0-tx-0-21swapper/020:05:550
29725040,0irq/27-eth0-tx-0-21swapper/019:10:590
29725040,0irq/27-eth0-tx-0-21swapper/018:55:540
29715040,0irq/26-eth0-rx-0-21swapper/322:30:493
29715040,0irq/26-eth0-rx-0-21swapper/321:45:543
29715040,0irq/26-eth0-rx-0-21swapper/321:30:443
29715040,0irq/26-eth0-rx-0-21swapper/319:20:533
29715040,0irq/26-eth0-rx-0-21swapper/300:02:213
259940,0migration/27866-21kthreadcore20:30:532
259940,0migration/27721-21aten_r1power_po21:15:482
259940,0migration/26630-21kthreadcore19:40:532
259940,0migration/26006-21runrttasks23:35:422
259940,0migration/25752-21kthreadcore19:15:532
259940,0migration/24964-21pmu-power21:50:562
259940,0migration/23406-21kthreadcore20:00:532
259940,0migration/227144-21kthreadcore23:05:532
259940,0migration/221306-21kthreadcore19:25:542
259940,0migration/215222-21kthreadcore19:45:542
259940,0migration/213632-21kthreadcore20:10:512
249940,0watchdog/215782-21aten_r1power_en19:00:472
249940,0watchdog/20-21swapper/223:15:542
249940,0watchdog/20-21swapper/220:55:552
179940,0migration/19586-21latency19:40:541
179940,0migration/17552-21kthreadcore00:15:531
179940,0migration/13727-21kthreadcore20:25:531
179940,0migration/130667-21aten_r1power_cu23:10:471
179940,0migration/124455-21kthreadcore23:25:531
179940,0migration/121359-21taskset23:44:171
179940,0migration/121357-21taskset00:05:011
179940,0migration/119370-21kthreadcore22:00:531
179940,0migration/117116-21kthreadcore23:00:531
179940,0migration/116673-21kthreadcore21:00:561
179940,0migration/116611-21latency_hist19:25:371
179940,0migration/116413-21kthreadcore00:20:551
179940,0migration/116010-21if_err_eth022:20:511
169940,0watchdog/129723-21switchtime21:25:591
169940,0watchdog/114876-21ssh22:40:461
169940,0watchdog/10-21swapper/121:35:551
169940,0watchdog/10-21swapper/121:35:551
169940,0watchdog/10-21swapper/120:05:511
149940,0migration/04073-21kthreadcore20:25:530
149940,0migration/029798-21sh21:06:240
149940,0migration/029317-21runrttasks22:46:450
149940,0migration/026390-21kthreadcore00:05:530
88899932,0cyclictest23307-21awk19:05:502
88899932,0cyclictest20386-21awk20:40:512
88899932,0cyclictest19400-21awk20:15:502
88889932,0cyclictest6092-21cut23:55:481
88889932,0cyclictest4186-21awk19:15:501
88889932,0cyclictest3514-21snmpd22:10:501
88889932,0cyclictest3514-21snmpd21:55:471
88889932,0cyclictest31051-21awk23:50:531
88889932,0cyclictest30890-21awk19:35:511
88889932,0cyclictest29488-21grep19:10:491
88889932,0cyclictest25753-21fschecks_time20:20:511
88889932,0cyclictest25573-21grep23:45:521
88879933,0cyclictest7942-21kworker/0:123:30:390
88879933,0cyclictest7942-21kworker/0:123:20:390
88879933,0cyclictest7942-21kworker/0:123:05:410
88879933,0cyclictest7942-21kworker/0:122:50:410
88879933,0cyclictest7942-21kworker/0:122:30:380
88879933,0cyclictest7942-21kworker/0:100:15:450
88879933,0cyclictest32189-21kworker/0:221:00:440
88879933,0cyclictest32189-21kworker/0:219:25:380
88879933,0cyclictest32189-21kworker/0:219:15:420
88879933,0cyclictest31693-21kworker/0:122:00:380
88879933,0cyclictest31693-21kworker/0:121:55:400
88879933,0cyclictest31693-21kworker/0:121:40:400
88879933,0cyclictest31693-21kworker/0:121:35:380
88879933,0cyclictest31693-21kworker/0:121:35:380
88879933,0cyclictest0-21swapper/020:20:390
88879932,0cyclictest15040-21grep20:35:520
88879930,0cyclictest4243-21munin-node19:40:490
8215030,0irq/24-0000:00:0-21swapper/220:26:252
29715030,0irq/26-eth0-rx-0-21swapper/320:45:513
29715030,0irq/26-eth0-rx-0-21swapper/320:12:183
29715030,0irq/26-eth0-rx-0-21swapper/319:30:533
249930,0watchdog/20-21swapper/220:20:512
169930,0watchdog/10-21swapper/121:40:471
169930,0watchdog/10-21swapper/119:45:511
169930,0watchdog/10-21swapper/119:00:511
169930,0watchdog/10-21swapper/100:10:541
88899921,0cyclictest26385-21cpuspeed_turbos20:45:492
35120,0ktimersoftd/322276-21processes19:25:573
249920,0watchdog/20-21swapper/219:10:392
169920,0watchdog/10-21swapper/118:55:431
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional