You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-16 - 10:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sat May 16, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3094725857,0sleep20-21swapper/218:55:012
3076624738,0sleep10-21swapper/118:52:511
3078023523,0sleep00-21swapper/018:53:020
304450350,0irq/25-eth10-21swapper/318:51:103
310679970,0cyclictest0-21swapper/100:05:431
35160,0ktimersoftd/30-21swapper/323:59:033
339960,0migration/326482-21munin-node20:00:523
339960,0migration/31949-21sh00:16:563
339960,0migration/310377-21sh23:41:333
310679960,0cyclictest29235-21diskmemload23:50:541
29715060,0irq/26-eth0-rx-5043-21if_eth023:20:443
29715060,0irq/26-eth0-rx-4761-21diskstats23:00:423
29715060,0irq/26-eth0-rx-30247-21perl21:35:433
29715060,0irq/26-eth0-rx-18350-21yum23:25:543
259960,0migration/227201-21ntp_kernel_err21:10:492
249960,0watchdog/232456-21kthreadcore23:15:482
149960,0migration/02454-21sh23:57:440
149960,0migration/020532-21sh22:30:170
149960,0migration/020197-21sh23:30:330
149960,0migration/019226-21sh00:09:420
149960,0migration/013721-21latency20:15:490
149960,0migration/011569-21sh22:43:190
8215050,0irq/24-0000:00:4974-21perf21:00:302
8215050,0irq/24-0000:00:29396-21ls21:35:302
8215050,0irq/24-0000:00:18707-21perf19:10:322
339950,0migration/39535-21kthreadcore19:25:473
339950,0migration/34533-21hddtemp_smartct00:00:433
339950,0migration/34027-21kthreadcore18:55:493
339950,0migration/34027-21kthreadcore18:55:493
339950,0migration/331945-21kthreadcore21:15:473
339950,0migration/33127-21runrttasks22:57:523
339950,0migration/33021-21irqrtprio20:10:473
339950,0migration/32785-21taskset22:36:233
339950,0migration/327555-21timerandwakeup20:25:533
339950,0migration/326687-21latency21:50:483
339950,0migration/326567-21taskset19:40:423
339950,0migration/321899-21munin-node22:10:443
339950,0migration/321089-21kernelversion00:10:473
339950,0migration/320781-21munin-node19:35:463
339950,0migration/320389-21runrttasks22:29:533
339950,0migration/317496-21kthreadcore22:45:473
339950,0migration/315639-21kthreadcore21:45:463
339950,0migration/313795-21taskset19:53:273
339950,0migration/313681-21kthreadcore19:05:483
310699955,0cyclictest13228-21irqrtprio23:45:462
310699955,0cyclictest0-21swapper/223:41:212
310699955,0cyclictest0-21swapper/222:56:252
310699954,0cyclictest5621-21awk22:20:452
310699954,0cyclictest14133-21awk19:30:462
310699950,0cyclictest0-21swapper/221:27:012
310669955,0cyclictest0-21swapper/023:08:190
310669955,0cyclictest0-21swapper/019:15:470
310669950,0cyclictest0-21swapper/023:26:180
310669950,0cyclictest0-21swapper/021:21:350
310669950,0cyclictest0-21swapper/021:17:480
29725050,0irq/27-eth0-tx-3524-21sshd19:26:140
29725050,0irq/27-eth0-tx-29664-21munin-node21:35:452
29725050,0irq/27-eth0-tx-29235-21diskmemload21:58:492
29725050,0irq/27-eth0-tx-20230-21munin-node23:30:462
29725050,0irq/27-eth0-tx-0-21swapper/223:55:422
29725050,0irq/27-eth0-tx-0-21swapper/223:52:002
29725050,0irq/27-eth0-tx-0-21swapper/223:35:332
29725050,0irq/27-eth0-tx-0-21swapper/223:07:152
29725050,0irq/27-eth0-tx-0-21swapper/222:14:002
29725050,0irq/27-eth0-tx-0-21swapper/221:42:562
29725050,0irq/27-eth0-tx-0-21swapper/200:11:302
29725050,0irq/27-eth0-tx-0-21swapper/020:55:430
29725050,0irq/27-eth0-tx-0-21swapper/020:54:130
29715050,0irq/26-eth0-rx-7551-21grep22:00:493
29715050,0irq/26-eth0-rx-6559-21runrttasks23:47:413
29715050,0irq/26-eth0-rx-30117-21interrupts22:15:463
29715050,0irq/26-eth0-rx-26913-21kthreadcore21:10:483
29715050,0irq/26-eth0-rx-22831-21kthreadcore22:50:463
29715050,0irq/26-eth0-rx-1976-21unixbench_singl23:35:533
29715050,0irq/26-eth0-rx-18621-21sh23:06:133
29715050,0irq/26-eth0-rx-16816-21kthreadcore00:05:473
29715050,0irq/26-eth0-rx-15902-21kthreadcore22:05:473
29715050,0irq/26-eth0-rx-15039-21cut21:05:483
29715050,0irq/26-eth0-rx-0-21swapper/323:15:423
29715050,0irq/26-eth0-rx-0-21swapper/323:12:143
29715050,0irq/26-eth0-rx-0-21swapper/322:40:473
29715050,0irq/26-eth0-rx-0-21swapper/322:30:483
29715050,0irq/26-eth0-rx-0-21swapper/321:43:453
29715050,0irq/26-eth0-rx-0-21swapper/321:30:543
29715050,0irq/26-eth0-rx-0-21swapper/321:25:333
29715050,0irq/26-eth0-rx-0-21swapper/321:01:533
29715050,0irq/26-eth0-rx-0-21swapper/320:55:453
29715050,0irq/26-eth0-rx-0-21swapper/320:20:493
29715050,0irq/26-eth0-rx-0-21swapper/319:45:483
29715050,0irq/26-eth0-rx-0-21swapper/300:21:063
27150,0ktimersoftd/20-21swapper/221:08:252
259950,0migration/27464-21kthreadcore21:00:482
259950,0migration/25848-21idleruntime22:00:462
259950,0migration/23376-21sh22:38:012
259950,0migration/221478-21fschecks_time22:30:442
259950,0migration/212616-21aten_r1power_cu22:45:412
259950,0migration/210711-21sh23:01:292
249950,0watchdog/2885-21timerandwakeup20:05:532
249950,0watchdog/213073-21aten_r1power_cu22:05:412
179950,0migration/16182-21taskset19:24:031
179950,0migration/13506-21sh21:57:011
179950,0migration/129938-21kthreadcore20:30:461
179950,0migration/127140-21sh22:51:541
179950,0migration/127108-21taskset00:14:121
179950,0migration/126227-21taskset19:39:201
179950,0migration/124956-21taskset19:11:061
179950,0migration/123037-21cpuspeed_turbos20:50:421
179950,0migration/117790-21munin-node23:25:491
169950,0watchdog/16259-21tail19:45:491
169950,0watchdog/15660-21tune2fs22:00:451
149950,0migration/07696-21aten_r1power_po19:50:430
149950,0migration/06370-21kthreadcore20:10:480
149950,0migration/04618-21latency_hist22:20:300
149950,0migration/032065-21kthreadcore22:35:460
149950,0migration/028988-21kthreadcore20:05:470
149950,0migration/026961-21taskset22:51:210
149950,0migration/026802-21cpuspeed_turbos19:40:430
149950,0migration/022300-21if_eth121:50:450
149950,0migration/020934-21fschecks_count23:10:430
149950,0migration/020210-21aten_r1power_vo23:50:420
149950,0migration/019945-21munin-node19:35:420
149950,0migration/017372-21kthreadcore19:30:480
149950,0migration/015298-21aten_r1power_vo20:20:420
149950,0migration/013076-21aten_r1power_cu22:05:410
149950,0migration/010185-21taskset23:40:590
8215040,0irq/24-0000:00:0-21swapper/222:15:302
339940,0migration/322776-21kthreadcore19:10:483
339940,0migration/319085-21kthreadcore19:55:473
339940,0migration/31372-21kthreadcore19:20:453
339940,0migration/312116-21sh21:21:543
339940,0migration/310068-21kthreadcore20:15:483
310699944,0cyclictest0-21swapper/221:50:312
310699943,0cyclictest6507-21awk19:00:462
310699943,0cyclictest3514-21snmpd19:52:482
310699943,0cyclictest3514-21snmpd19:38:182
310699943,0cyclictest2682-21awk20:10:462
310699940,0cyclictest0-21swapper/220:30:162
310679944,0cyclictest0-21swapper/123:59:151
310679944,0cyclictest0-21swapper/123:30:531
310679944,0cyclictest0-21swapper/121:36:061
310679944,0cyclictest0-21swapper/100:15:581
310679943,0cyclictest9362-21awk20:15:461
310679943,0cyclictest8562-21awk19:50:461
310679943,0cyclictest3556-21lldpd22:23:141
310679943,0cyclictest3514-21snmpd18:56:031
310679943,0cyclictest3514-21snmpd18:56:031
310679943,0cyclictest3241-21irqbalance22:14:291
310679943,0cyclictest3241-21irqbalance21:02:291
310679943,0cyclictest30693-21awk21:15:441
310679943,0cyclictest30345-21awk20:55:451
310679943,0cyclictest12937-21awk23:45:441
310679940,0cyclictest30430-21kthreadcore23:35:461
310679940,0cyclictest0-21swapper/121:43:211
310669944,0cyclictest0-21swapper/021:41:180
310669944,0cyclictest0-21swapper/021:39:240
310669944,0cyclictest0-21swapper/021:09:270
310669943,0cyclictest4471-21awk00:20:460
310669943,0cyclictest3556-21lldpd22:01:140
310669943,0cyclictest3556-21lldpd19:07:440
310669943,0cyclictest3514-21snmpd23:46:040
310669943,0cyclictest3514-21snmpd22:18:490
310669943,0cyclictest3514-21snmpd21:26:340
310669943,0cyclictest3241-21irqbalance22:45:390
310669943,0cyclictest28832-21sadc22:15:300
310669943,0cyclictest12092-21sadc23:05:300
310669940,0cyclictest18926-21munin-node19:10:470
29725040,0irq/27-eth0-tx-29235-21diskmemload00:05:402
29725040,0irq/27-eth0-tx-0-21swapper/223:26:232
29725040,0irq/27-eth0-tx-0-21swapper/223:20:402
29725040,0irq/27-eth0-tx-0-21swapper/223:11:082
29725040,0irq/27-eth0-tx-0-21swapper/222:40:452
29725040,0irq/27-eth0-tx-0-21swapper/222:25:352
29725040,0irq/27-eth0-tx-0-21swapper/221:15:392
29725040,0irq/27-eth0-tx-0-21swapper/200:15:312
29715040,0irq/26-eth0-rx-0-21swapper/323:30:333
29715040,0irq/26-eth0-rx-0-21swapper/322:21:203
29715040,0irq/26-eth0-rx-0-21swapper/321:55:573
29715040,0irq/26-eth0-rx-0-21swapper/320:54:293
29715040,0irq/26-eth0-rx-0-21swapper/320:40:473
29715040,0irq/26-eth0-rx-0-21swapper/320:30:463
29715040,0irq/26-eth0-rx-0-21swapper/320:05:543
29715040,0irq/26-eth0-rx-0-21swapper/319:00:553
259940,0migration/29844-21kthreadcore20:15:472
259940,0migration/28174-21needreboot20:35:502
259940,0migration/27977-21kthreadcore00:20:492
259940,0migration/27867-21kthreadcore21:20:472
259940,0migration/26378-21kthreadcore00:00:472
259940,0migration/230891-21munin-node19:15:492
259940,0migration/228114-21missed_timers20:50:482
259940,0migration/224204-21kthreadcore20:00:492
259940,0migration/217518-21kthreadcore20:20:462
259940,0migration/214341-21aten_r1power_po19:55:422
259940,0migration/214174-21idleruntime21:45:432
259940,0migration/211553-21kthreadcore20:40:462
249940,0watchdog/26260-21head19:45:492
249940,0watchdog/229527-21irqcore20:30:452
249940,0watchdog/20-21swapper/222:50:412
249940,0watchdog/20-21swapper/220:45:492
249940,0watchdog/20-21swapper/218:55:462
249940,0watchdog/20-21swapper/218:55:462
21085240,0sleep321572-21kthreadcore23:50:473
179940,0migration/19896-21sh23:00:481
179940,0migration/19868-21forks20:40:431
179940,0migration/14748-21irqrtprio00:20:471
179940,0migration/13367-21fschecks_time20:35:451
179940,0migration/131584-21kthreadcore22:15:471
179940,0migration/128871-21aten_r1power_cu22:35:401
179940,0migration/127552-21timerandwakeup20:25:531
179940,0migration/127448-21smart_sda21:10:521
179940,0migration/126821-21unixbench_singl20:00:541
179940,0migration/125372-21kthreadcore21:50:481
179940,0migration/119583-21runrttasks22:48:151
179940,0migration/116349-21kthreadcore21:25:471
179940,0migration/113945-21fschecks_count21:05:451
179940,0migration/113711-21fschecks_count19:30:441
179940,0migration/110730-21kthreadcore19:00:491
179940,0migration/110499-21timerandwakeup22:40:521
169940,0watchdog/14202-21aten_r1power_fr23:20:401
169940,0watchdog/129235-21diskmemload21:32:251
169940,0watchdog/127269-21ssh23:13:011
169940,0watchdog/117540-21kthreadcore22:25:491
169940,0watchdog/113072-21aten_r1power_cu22:05:411
149940,0migration/032157-21kthreadcore23:15:480
149940,0migration/030606-21kthreadcore22:55:460
149940,0migration/022998-21irqrtprio20:25:470
310699932,0cyclictest7214-21perl19:25:432
310699932,0cyclictest666-21awk19:20:432
310699932,0cyclictest32278-21switchtime19:40:532
310679932,0cyclictest9817-21grep21:20:481
310679932,0cyclictest4038-21grep20:10:471
310679932,0cyclictest30148-21tr19:15:491
310679932,0cyclictest29235-21diskmemload23:05:411
310679932,0cyclictest28571-21date22:55:301
310679932,0cyclictest20713-21cat20:00:311
310679932,0cyclictest16365-21grep20:45:431
310679932,0cyclictest13230-21awk19:05:471
310679932,0cyclictest11104-21cut19:25:471
310669933,0cyclictest30264-21awk21:55:450
310669933,0cyclictest25386-21kworker/0:020:45:320
310669933,0cyclictest25386-21kworker/0:020:40:580
310669933,0cyclictest25386-21kworker/0:020:35:380
310669933,0cyclictest25386-21kworker/0:020:30:340
310669933,0cyclictest25386-21kworker/0:020:00:320
310669933,0cyclictest25386-21kworker/0:019:55:340
310669933,0cyclictest25386-21kworker/0:019:45:350
310669933,0cyclictest25386-21kworker/0:019:20:350
310669933,0cyclictest25386-21kworker/0:019:00:350
310669933,0cyclictest25386-21kworker/0:018:55:370
310669933,0cyclictest25386-21kworker/0:018:55:370
310669933,0cyclictest23581-21kworker/0:221:45:460
310669933,0cyclictest23581-21kworker/0:221:30:360
310669933,0cyclictest23581-21kworker/0:221:11:060
310669933,0cyclictest21429-21kworker/0:123:35:360
310669933,0cyclictest21429-21kworker/0:123:20:320
310669933,0cyclictest21429-21kworker/0:100:10:330
310669933,0cyclictest21429-21kworker/0:100:00:470
310669933,0cyclictest13588-21kworker/0:022:30:400
310669932,0cyclictest6043-21hwlatdetect21:00:450
310669932,0cyclictest28715-21awk00:15:430
29715030,0irq/26-eth0-rx-0-21swapper/320:36:083
29715030,0irq/26-eth0-rx-0-21swapper/319:30:423
29715030,0irq/26-eth0-rx-0-21swapper/319:15:423
179930,0migration/116749-21kthreadcore20:20:461
169930,0watchdog/129235-21diskmemload00:00:441
169930,0watchdog/10-21swapper/123:40:481
169930,0watchdog/10-21swapper/123:15:481
169930,0watchdog/10-21swapper/122:30:491
169930,0watchdog/10-21swapper/121:46:531
169930,0watchdog/10-21swapper/120:05:491
310679921,0cyclictest26333-21crond19:40:311
29715020,0irq/26-eth0-rx-0-21swapper/320:45:433
249920,0watchdog/20-21swapper/219:10:412
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional