You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-12 - 02:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Thu Mar 12, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2558425350,0sleep10-21swapper/118:57:591
304450350,0irq/25-eth10-21swapper/318:57:013
2571923423,0sleep00-21swapper/018:59:380
2589123228,0sleep20-21swapper/219:01:362
29715070,0irq/26-eth0-rx-69262sleep321:28:123
339960,0migration/37698-21aten_r1power_cu22:11:493
339960,0migration/33399-21sh00:07:233
29715060,0irq/26-eth0-rx-29946-21wc22:02:003
29715060,0irq/26-eth0-rx-12725-21mdstat22:31:563
259659960,0cyclictest0-21swapper/222:36:362
259659960,0cyclictest0-21swapper/221:11:562
259649960,0cyclictest0-21swapper/121:55:141
8215050,0irq/24-0000:00:26898-21ssh00:21:592
339950,0migration/39796-21kthreadcore23:31:553
339950,0migration/34348-21kthreadcore22:26:553
339950,0migration/332570-21kthreadcore20:16:553
339950,0migration/33182-21kthreadcore21:46:573
339950,0migration/331545-21munin-node23:06:533
339950,0migration/331169-21kernelversion23:46:533
339950,0migration/328355-21unixbench-2d23:22:013
339950,0migration/328316-21kthreadcore21:41:563
339950,0migration/327894-21kthreadcore21:01:543
339950,0migration/326382-21unixbench-2d19:22:013
339950,0migration/324296-21kthreadcore00:01:543
339950,0migration/323665-21munin-node22:41:513
339950,0migration/322750-21sh21:58:323
339950,0migration/322326-21irqrtprio00:21:533
339950,0migration/32078-21dump-pmu-power19:56:393
339950,0migration/317843-21kthreadcore20:06:573
339950,0migration/317817-21kthreadcore22:36:553
339950,0migration/315209-21taskset20:03:183
339950,0migration/314781-21munin-node23:16:503
339950,0migration/314138-21sh22:55:303
339950,0migration/313263-21aten_r1power_fr00:16:513
339950,0migration/311916-21runrttasks23:52:453
29725050,0irq/27-eth0-tx-8254-21telnet19:36:510
29725050,0irq/27-eth0-tx-5892-21ssh22:28:192
29725050,0irq/27-eth0-tx-26054-21kthreadcore23:41:552
29725050,0irq/27-eth0-tx-23260-21munin-node22:41:582
29725050,0irq/27-eth0-tx-15698-21if_eth022:56:522
29725050,0irq/27-eth0-tx-15631-21ssh21:56:062
29725050,0irq/27-eth0-tx-0-21swapper/019:46:510
29715050,0irq/26-eth0-rx-9273-21kthreadcore00:11:533
29715050,0irq/26-eth0-rx-6498-21ssh21:08:073
29715050,0irq/26-eth0-rx-21791-21latency_hist23:41:383
29715050,0irq/26-eth0-rx-2048-21cut22:06:553
29715050,0irq/26-eth0-rx-19610-21tr21:16:543
29715050,0irq/26-eth0-rx-17413-21grep21:36:543
29715050,0irq/26-eth0-rx-0-21swapper/323:36:573
29715050,0irq/26-eth0-rx-0-21swapper/323:26:503
29715050,0irq/26-eth0-rx-0-21swapper/323:12:253
29715050,0irq/26-eth0-rx-0-21swapper/323:00:293
29715050,0irq/26-eth0-rx-0-21swapper/322:47:153
29715050,0irq/26-eth0-rx-0-21swapper/321:55:273
29715050,0irq/26-eth0-rx-0-21swapper/321:33:463
29715050,0irq/26-eth0-rx-0-21swapper/321:21:543
29715050,0irq/26-eth0-rx-0-21swapper/321:12:003
29715050,0irq/26-eth0-rx-0-21swapper/320:46:483
29715050,0irq/26-eth0-rx-0-21swapper/319:06:503
27150,0ktimersoftd/20-21swapper/222:21:472
27150,0ktimersoftd/20-21swapper/221:06:162
259659955,0cyclictest0-21swapper/223:12:332
259659955,0cyclictest0-21swapper/222:56:092
259659955,0cyclictest0-21swapper/221:20:372
259659954,0cyclictest6076-21sh23:11:382
259659954,0cyclictest17074-21ps23:36:572
259659950,0cyclictest0-21swapper/222:12:192
259649955,0cyclictest0-21swapper/123:15:471
259649955,0cyclictest0-21swapper/122:17:461
259649955,0cyclictest0-21swapper/121:25:451
259649955,0cyclictest0-21swapper/121:11:241
259649950,0cyclictest0-21swapper/122:36:511
259649950,0cyclictest0-21swapper/122:01:411
259649950,0cyclictest0-21swapper/120:37:021
259629954,0cyclictest3514-21snmpd21:25:540
259629950,0cyclictest0-21swapper/022:01:510
259950,0migration/232282-21kthreadcore23:46:532
259950,0migration/229888-21latency_hist20:41:392
259950,0migration/218953-21munin-node23:56:552
259950,0migration/218444-21kthreadcore22:16:552
259950,0migration/21772-21kthreadcore19:06:552
259950,0migration/215521-21idleruntime-cro20:06:392
259950,0migration/212940-21sh23:34:342
249950,0watchdog/212894-21grep21:31:572
179950,0migration/19886-21kthreadcore22:51:551
179950,0migration/18770-21kthreadcore22:31:551
179950,0migration/17974-21idleruntime19:11:511
179950,0migration/15641-21taskset22:27:351
179950,0migration/132288-21irqcore22:46:521
179950,0migration/130916-21kthreadcore19:51:571
179950,0migration/126411-21kthreadcore22:21:541
179950,0migration/116462-21aten_r1power_fr21:36:491
179950,0migration/110655-21kthreadcore22:11:561
149950,0migration/028247-21sh00:24:590
149950,0migration/028138-21sh00:03:250
149950,0migration/021234-21idleruntime-cro00:01:380
149950,0migration/015567-21sh21:15:100
8215040,0irq/24-0000:00:4122-21latency_hist20:46:392
339940,0migration/328667-21taskset20:14:593
339940,0migration/321453-21runrttasks19:46:493
339940,0migration/318070-21df_abs20:56:513
339940,0migration/314843-21aten2.4-expect19:41:503
339940,0migration/312206-21kthreadcore19:11:543
29725040,0irq/27-eth0-tx-5688-21grep21:47:012
29725040,0irq/27-eth0-tx-5074-21ssh22:47:252
29725040,0irq/27-eth0-tx-30271-21ssh23:26:422
29725040,0irq/27-eth0-tx-22003-21ssh22:38:432
29725040,0irq/27-eth0-tx-21451-21ssh00:01:412
29725040,0irq/27-eth0-tx-0-21swapper/223:51:412
29725040,0irq/27-eth0-tx-0-21swapper/222:06:392
29725040,0irq/27-eth0-tx-0-21swapper/222:02:062
29725040,0irq/27-eth0-tx-0-21swapper/221:58:382
29725040,0irq/27-eth0-tx-0-21swapper/221:36:472
29725040,0irq/27-eth0-tx-0-21swapper/221:06:582
29725040,0irq/27-eth0-tx-0-21swapper/200:26:462
29725040,0irq/27-eth0-tx-0-21swapper/019:52:000
29725040,0irq/27-eth0-tx-0-21swapper/019:21:590
29715040,0irq/26-eth0-rx-27712-21kthreadcore23:01:543
29715040,0irq/26-eth0-rx-0-21swapper/323:56:503
29715040,0irq/26-eth0-rx-0-21swapper/322:22:463
29715040,0irq/26-eth0-rx-0-21swapper/322:16:463
29715040,0irq/26-eth0-rx-0-21swapper/320:44:373
29715040,0irq/26-eth0-rx-0-21swapper/319:31:543
29715040,0irq/26-eth0-rx-0-21swapper/300:27:483
259659944,0cyclictest0-21swapper/223:16:392
259659944,0cyclictest0-21swapper/200:11:502
259659943,0cyclictest8099-21awk19:11:522
259659943,0cyclictest3556-21lldpd23:01:492
259659943,0cyclictest3556-21lldpd19:54:192
259659943,0cyclictest3556-21lldpd00:06:502
259659943,0cyclictest3514-21snmpd23:21:402
259659943,0cyclictest3514-21snmpd19:24:252
259659943,0cyclictest27111-21awk19:01:532
259659943,0cyclictest2309-21awk19:31:522
259649943,0cyclictest3514-21snmpd21:48:401
259649943,0cyclictest3514-21snmpd21:45:401
259649943,0cyclictest3514-21snmpd20:31:401
259649943,0cyclictest3514-21snmpd20:06:401
259649943,0cyclictest3514-21snmpd20:01:551
259649943,0cyclictest3241-21irqbalance23:33:561
259649943,0cyclictest30750-21awk23:46:511
259649943,0cyclictest25356-21awk21:01:521
259649943,0cyclictest24241-21awk22:41:531
259649943,0cyclictest22307-21awk19:46:521
259649943,0cyclictest15652-21awk19:41:531
259649943,0cyclictest14170-21awk00:16:561
259629944,0cyclictest0-21swapper/023:15:570
259629943,0cyclictest8645-21awk22:11:540
259629943,0cyclictest734-21awk21:06:530
259629943,0cyclictest6375-21awk23:51:510
259629943,0cyclictest3514-21snmpd22:19:400
259629943,0cyclictest3514-21snmpd20:28:100
259629943,0cyclictest3514-21snmpd20:28:100
259629943,0cyclictest3514-21snmpd19:43:250
259629943,0cyclictest3309-21munin-node20:21:490
259629943,0cyclictest3241-21irqbalance23:42:460
259629943,0cyclictest3241-21irqbalance23:37:560
259629943,0cyclictest3241-21irqbalance23:27:260
259629943,0cyclictest3241-21irqbalance23:21:560
259629943,0cyclictest3241-21irqbalance22:44:460
259629943,0cyclictest3241-21irqbalance21:47:070
259629943,0cyclictest3241-21irqbalance21:42:370
259629943,0cyclictest3241-21irqbalance21:26:470
259629943,0cyclictest3241-21irqbalance20:36:470
259629943,0cyclictest3241-21irqbalance19:16:470
259629943,0cyclictest3241-21irqbalance19:11:570
259629943,0cyclictest3241-21irqbalance19:10:570
259629943,0cyclictest30935-21awk20:41:530
259629943,0cyclictest29915-21awk20:16:530
259629943,0cyclictest27017-21awk19:01:520
259629943,0cyclictest17240-21awk21:36:530
259629943,0cyclictest16982-21awk21:56:510
259629940,0cyclictest0-21swapper/022:08:190
259940,0migration/29324-21irqrtprio19:36:562
259940,0migration/24042-21runrttasks20:45:492
259940,0migration/23322-21aten_r1power_cu20:21:472
259940,0migration/224732-21cpu21:21:502
259940,0migration/222212-21hddtemp_smartct19:46:512
259940,0migration/219118-21kthreadcore20:06:582
259940,0migration/217695-21latency_hist20:56:392
259940,0migration/215571-21hwlatdetect19:41:532
259940,0migration/213297-21aten2.4-expect00:16:522
249940,0watchdog/26794-21ssh21:27:492
249940,0watchdog/229153-21grep19:26:532
249940,0watchdog/211199-21basename20:51:492
249940,0watchdog/20-21swapper/221:41:562
179940,0migration/18093-21latency_hist21:11:381
179940,0migration/17091-21rt-features19:31:581
179940,0migration/1683-21kthreadcore00:26:551
179940,0migration/15665-21timerwakeupswit22:07:011
179940,0migration/15479-21kthreadcore19:06:561
179940,0migration/13954-21ksmtuned20:44:481
179940,0migration/13442-21kthreadcore21:26:541
179940,0migration/132539-21kthreadcore23:06:561
179940,0migration/126945-21munin-node19:01:511
179940,0migration/119988-21kthreadcore21:16:541
179940,0migration/119114-21load19:16:571
179940,0migration/118921-21kthreadcore22:56:541
179940,0migration/115683-21kthreadcore23:56:541
179940,0migration/112867-21kthreadcore20:26:541
179940,0migration/112867-21kthreadcore20:26:541
169940,0watchdog/131131-21basename23:26:521
169940,0watchdog/129236-21ssh23:24:161
169940,0watchdog/129109-21ssh23:02:401
169940,0watchdog/129031-21ssh00:06:041
169940,0watchdog/121179-21ssh00:21:481
169940,0watchdog/12063-21kthreadcore00:06:561
169940,0watchdog/116049-21ssh21:56:401
169940,0watchdog/111200-21aten_r1power_vo20:51:491
169940,0watchdog/10-21swapper/121:31:571
149940,0migration/09794-21gltestperf20:01:530
149940,0migration/09186-21munin-node21:31:530
149940,0migration/08224-21aten_r1power_vo21:51:500
149940,0migration/07398-21unixbench_multi19:32:000
149940,0migration/06734-21latency_hist22:51:380
149940,0migration/06657-21gltestperf23:31:510
149940,0migration/05155-21sh23:51:260
149940,0migration/029793-21missed_timers21:01:560
149940,0migration/023371-21fschecks_time23:01:500
149940,0migration/022089-21missed_timers20:31:550
149940,0migration/019620-21kthreadcore20:56:550
149940,0migration/017309-21kthreadcore22:56:540
149940,0migration/017089-21taskset20:52:230
149940,0migration/016951-21fschecks_count21:16:500
149940,0migration/01230-21latency_hist19:31:390
29715030,0irq/26-eth0-rx-0-21swapper/320:38:463
29715030,0irq/26-eth0-rx-0-21swapper/320:31:483
29715030,0irq/26-eth0-rx-0-21swapper/320:22:293
29715030,0irq/26-eth0-rx-0-21swapper/319:56:493
29715030,0irq/26-eth0-rx-0-21swapper/319:37:113
29715030,0irq/26-eth0-rx-0-21swapper/319:26:483
259659933,0cyclictest3556-21lldpd20:33:482
259659933,0cyclictest0-21swapper/220:16:272
259659932,0cyclictest3556-21lldpd20:30:492
259659932,0cyclictest3556-21lldpd20:30:492
259659930,0cyclictest0-21swapper/220:17:142
259649932,0cyclictest24132-21sadc21:01:381
259649932,0cyclictest23371-21awk20:11:541
259649932,0cyclictest22452-21sh23:41:511
259649932,0cyclictest0-21swapper/100:12:521
259649931,0cyclictest1020-21cut20:16:561
259649930,0cyclictest0-21swapper/119:56:551
259629932,0cyclictest3514-21snmpd22:36:550
259629932,0cyclictest3514-21snmpd22:32:250
259629932,0cyclictest32128-21awk22:46:510
259629932,0cyclictest28052-21sensors20:12:000
259629932,0cyclictest24590-21awk22:21:520
259629932,0cyclictest16609-21awk20:06:550
259629932,0cyclictest15127-21tail23:16:510
259629932,0cyclictest13292-21aten_r1power_po00:16:520
259629932,0cyclictest1162-21cut00:26:550
259629931,0cyclictest0-21swapper/023:06:550
259629930,0cyclictest6177-21irqcore00:11:520
259629930,0cyclictest4347-21munin-node20:47:000
259930,0migration/215761-21kthreadcore19:16:562
249930,0watchdog/20-21swapper/219:56:492
169930,0watchdog/114343-21munin-node23:36:521
169930,0watchdog/10-21swapper/123:51:561
169930,0watchdog/10-21swapper/123:16:481
169930,0watchdog/10-21swapper/120:46:491
169930,0watchdog/10-21swapper/119:36:571
169930,0watchdog/10-21swapper/119:26:531
169930,0watchdog/10-21swapper/119:22:011
149930,0migration/06042-21runrttasks22:28:470
339920,0migration/311464-21diskstats20:51:503
29715020,0irq/26-eth0-rx-0-21swapper/320:26:573
29715020,0irq/26-eth0-rx-0-21swapper/320:26:573
29715020,0irq/26-eth0-rx-0-21swapper/319:16:553
29715020,0irq/26-eth0-rx-0-21swapper/319:01:563
259649921,0cyclictest0-21swapper/120:21:481
259629920,0cyclictest29359-21latency_hist00:06:380
259629920,0cyclictest0-21swapper/019:56:490
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional