You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-22 - 06:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Sun Mar 22, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450820,0irq/25-eth10-21swapper/319:00:093
1461226059,0sleep20-21swapper/218:56:212
1492923430,0sleep10-21swapper/119:00:001
1469123423,0sleep00-21swapper/018:57:150
150669970,0cyclictest23927-21wc20:16:041
339960,0migration/39243-21sh23:56:003
339960,0migration/33795-21sh22:34:203
339960,0migration/33770-21perf23:15:413
339960,0migration/328729-21aten_r1power_vo22:50:523
339960,0migration/327225-21sh22:26:353
339960,0migration/325900-21sh00:06:133
339960,0migration/317981-21sh00:22:253
339960,0migration/317624-21munin-node23:00:583
339960,0migration/316035-21sh19:46:013
29725060,0irq/27-eth0-tx-29600-21munin-node19:36:030
29715060,0irq/26-eth0-rx-5838-21if_eth022:15:573
29715060,0irq/26-eth0-rx-18462-21netstat22:21:003
29715060,0irq/26-eth0-rx-0-21swapper/300:13:293
150679966,0cyclictest13210-21diskmemload23:17:012
150679966,0cyclictest13210-21diskmemload00:03:412
150679960,0cyclictest0-21swapper/200:14:342
150669960,0cyclictest0-21swapper/123:30:561
4150,0ktimersoftd/00-21swapper/023:42:190
4150,0ktimersoftd/00-21swapper/021:19:100
4150,0ktimersoftd/00-21swapper/000:27:400
339950,0migration/3985-21kthreadcore20:25:593
339950,0migration/38534-21kthreadcore20:56:003
339950,0migration/34644-21hwlatdetect23:35:553
339950,0migration/34177-21taskset22:14:193
339950,0migration/33995-21timerandwakeup20:01:053
339950,0migration/329729-21aten_r1power_po21:50:543
339950,0migration/329591-21switchtime19:56:063
339950,0migration/328025-21aten_r1power_en23:50:523
339950,0migration/327833-21runrttasks23:30:273
339950,0migration/327282-21runrttasks22:47:433
339950,0migration/326225-21kthreadcore20:20:583
339950,0migration/325770-21munin-node20:45:533
339950,0migration/324936-21kthreadcore22:06:003
339950,0migration/324482-21kthreadcore23:05:593
339950,0migration/324408-21munin-node20:40:593
339950,0migration/323722-21kthreadcore23:45:583
339950,0migration/323452-21taskset19:53:183
339950,0migration/32325-21kthreadcore21:30:593
339950,0migration/322751-21latency_hist19:30:433
339950,0migration/319173-21munin-node21:01:033
339950,0migration/313219-21latency_hist22:00:413
339950,0migration/310106-21kthreadcore20:30:563
29725050,0irq/27-eth0-tx-2542-21logger22:51:412
29725050,0irq/27-eth0-tx-13210-21diskmemload21:17:062
29725050,0irq/27-eth0-tx-12581-21munin-node22:40:542
29725050,0irq/27-eth0-tx-0-21swapper/222:10:552
29725050,0irq/27-eth0-tx-0-21swapper/221:05:562
29725050,0irq/27-eth0-tx-0-21swapper/021:00:570
29725050,0irq/27-eth0-tx-0-21swapper/020:35:520
29725050,0irq/27-eth0-tx-0-21swapper/020:28:200
29725050,0irq/27-eth0-tx-0-21swapper/019:40:550
29725050,0irq/27-eth0-tx-0-21swapper/019:33:430
29725050,0irq/27-eth0-tx-0-21swapper/019:15:560
29715050,0irq/26-eth0-rx-4019-21fschecks_time00:15:553
29715050,0irq/26-eth0-rx-31479-21tr23:30:583
29715050,0irq/26-eth0-rx-26529-21kthreadcore21:45:593
29715050,0irq/26-eth0-rx-25799-21sh00:27:003
29715050,0irq/26-eth0-rx-19247-21ssh22:42:303
29715050,0irq/26-eth0-rx-18504-21ssh23:42:213
29715050,0irq/26-eth0-rx-14749-21kthreadcore23:20:593
29715050,0irq/26-eth0-rx-11596-21ssh21:56:313
29715050,0irq/26-eth0-rx-0-21swapper/323:19:453
29715050,0irq/26-eth0-rx-0-21swapper/322:57:113
29715050,0irq/26-eth0-rx-0-21swapper/321:42:093
29715050,0irq/26-eth0-rx-0-21swapper/321:35:593
29715050,0irq/26-eth0-rx-0-21swapper/321:27:093
29715050,0irq/26-eth0-rx-0-21swapper/321:17:143
29715050,0irq/26-eth0-rx-0-21swapper/321:10:593
29715050,0irq/26-eth0-rx-0-21swapper/321:07:273
29715050,0irq/26-eth0-rx-0-21swapper/300:04:553
27150,0ktimersoftd/20-21swapper/219:05:432
27150,0ktimersoftd/20-21swapper/200:27:402
259950,0migration/24185-21latency_hist22:55:412
259950,0migration/24131-21sh21:12:472
259950,0migration/227393-21sh22:48:032
259950,0migration/226495-21sh00:07:582
259950,0migration/223798-21kthreadcore22:26:002
259950,0migration/219070-21sh23:43:582
19150,0ktimersoftd/10-21swapper/122:25:311
179950,0migration/130276-21munin-node22:10:571
179950,0migration/116965-21munin-node00:21:021
179950,0migration/116735-21kthreadcore21:20:571
150679955,0cyclictest0-21swapper/223:10:522
150679955,0cyclictest0-21swapper/222:35:522
150679955,0cyclictest0-21swapper/222:05:172
150679954,0cyclictest0-21swapper/223:04:432
150679950,0cyclictest0-21swapper/223:57:512
150679950,0cyclictest0-21swapper/221:33:262
150669955,0cyclictest0-21swapper/122:01:231
150669950,0cyclictest0-21swapper/123:14:231
150669950,0cyclictest0-21swapper/122:53:161
150669950,0cyclictest0-21swapper/122:34:201
150669950,0cyclictest0-21swapper/122:08:481
150669950,0cyclictest0-21swapper/100:00:161
150659955,0cyclictest28095-21ssh21:26:250
150659954,0cyclictest13985-21tr22:20:570
150659950,0cyclictest13210-21diskmemload00:07:230
150659950,0cyclictest0-21swapper/023:03:420
150659950,0cyclictest0-21swapper/022:57:330
150659950,0cyclictest0-21swapper/022:02:120
150659950,0cyclictest0-21swapper/021:43:080
150659950,0cyclictest0-21swapper/021:13:110
150659950,0cyclictest0-21swapper/000:25:410
149950,0migration/04434-21taskset20:03:320
149950,0migration/03163-21timerandwakeup21:51:050
149950,0migration/029413-21aten_r1power_vo22:10:530
149950,0migration/028893-21fschecks_time23:30:540
149950,0migration/028470-21latency_hist22:50:410
149950,0migration/023807-21kthreadcore22:26:000
149950,0migration/021842-21kthreadcore20:15:590
149950,0migration/01680-21munin-node23:51:050
149950,0migration/014468-21munin-node21:20:540
149950,0migration/01321-21latency23:11:000
149950,0migration/010526-21munin-node20:06:030
149950,0migration/010283-21rt-features22:36:000
4140,0ktimersoftd/00-21swapper/022:09:430
4140,0ktimersoftd/00-21swapper/000:17:530
339940,0migration/39865-21kthreadcore22:35:563
339940,0migration/35874-21timerandwakeup20:51:043
339940,0migration/315159-21kthreadcore20:35:583
339940,0migration/310035-21taskset19:42:183
29725040,0irq/27-eth0-tx-28706-21gltestperf23:50:552
29725040,0irq/27-eth0-tx-21976-21aten_r1power_vo21:25:522
29725040,0irq/27-eth0-tx-20047-21ssh21:02:242
29725040,0irq/27-eth0-tx-13403-21latency_hist21:20:412
29725040,0irq/27-eth0-tx-0-21swapper/223:46:042
29725040,0irq/27-eth0-tx-0-21swapper/223:21:292
29725040,0irq/27-eth0-tx-0-21swapper/222:31:032
29725040,0irq/27-eth0-tx-0-21swapper/222:18:392
29725040,0irq/27-eth0-tx-0-21swapper/222:06:222
29725040,0irq/27-eth0-tx-0-21swapper/221:45:542
29725040,0irq/27-eth0-tx-0-21swapper/221:35:542
29725040,0irq/27-eth0-tx-0-21swapper/200:16:102
29725040,0irq/27-eth0-tx-0-21swapper/020:10:540
29725040,0irq/27-eth0-tx-0-21swapper/019:55:580
29715040,0irq/26-eth0-rx-0-21swapper/321:20:513
29715040,0irq/26-eth0-rx-0-21swapper/319:16:003
29715040,0irq/26-eth0-rx-0-21swapper/319:05:593
259940,0migration/27190-21kthreadcore20:30:552
259940,0migration/26091-21hddtemp_smartct21:55:562
259940,0migration/2603-21kthreadcore19:35:592
259940,0migration/25558-21turbostat20:30:422
259940,0migration/24654-21hwlatdetect23:35:552
259940,0migration/228597-21munin-node23:30:532
259940,0migration/22806-21kthreadcore20:50:582
259940,0migration/226385-21sh23:26:162
259940,0migration/224630-21pmu-power20:41:012
259940,0migration/220351-21aten_r1power_en23:05:532
259940,0migration/218837-21fschecks_time20:15:562
259940,0migration/217354-21kthreadcore21:40:592
259940,0migration/213390-21kthreadcore19:45:582
249940,0watchdog/230191-21fschecks_time21:50:562
249940,0watchdog/225689-21if_err_eth1.220:20:562
249940,0watchdog/216952-21perl19:25:572
249940,0watchdog/20-21swapper/219:55:562
179940,0migration/19876-21kthreadcore21:36:001
179940,0migration/17786-21kthreadcore20:05:581
179940,0migration/132616-21kthreadcore19:35:591
179940,0migration/1308-21kthreadcore23:50:591
179940,0migration/129243-21fschecks_time19:10:551
179940,0migration/128658-21kthreadcore20:45:591
179940,0migration/127751-21taskset23:30:131
179940,0migration/119031-21kthreadcore19:51:001
179940,0migration/114413-21kthreadcore23:20:591
179940,0migration/112158-21cpuspeed_turbos23:40:541
169940,0watchdog/14011-21ssh21:32:081
169940,0watchdog/118988-21ssh23:02:401
169940,0watchdog/116585-21kthreadcore22:41:001
169940,0watchdog/113210-21diskmemload00:27:471
169940,0watchdog/111721-21ssh21:56:521
169940,0watchdog/110934-21load21:16:001
169940,0watchdog/110183-21ssh00:17:551
169940,0watchdog/10-21swapper/123:10:201
169940,0watchdog/10-21swapper/122:35:441
169940,0watchdog/10-21swapper/100:05:471
150679943,0cyclictest7682-21awk20:55:582
150679943,0cyclictest4742-21awk19:40:562
150679943,0cyclictest3556-21lldpd20:05:142
150679943,0cyclictest3556-21lldpd19:15:432
150679943,0cyclictest3514-21snmpd19:51:052
150679943,0cyclictest29020-21sort19:10:552
150679943,0cyclictest28459-21grep19:31:022
150679943,0cyclictest26604-21awk20:45:572
150679943,0cyclictest13864-21awk22:20:562
150679943,0cyclictest13406-21awk20:35:562
150679943,0cyclictest12273-21awk00:20:552
150679940,0cyclictest0-21swapper/219:02:342
150669944,0cyclictest0-21swapper/122:55:561
150669944,0cyclictest0-21swapper/121:43:061
150669943,0cyclictest961-21awk20:50:571
150669943,0cyclictest702-21sort00:11:011
150669943,0cyclictest5822-21awk22:15:571
150669943,0cyclictest3795-21awk19:15:571
150669943,0cyclictest3514-21snmpd21:55:351
150669943,0cyclictest3514-21snmpd21:08:501
150669943,0cyclictest3514-21snmpd21:01:051
150669943,0cyclictest3514-21snmpd20:55:501
150669943,0cyclictest3514-21snmpd20:00:501
150669943,0cyclictest3514-21snmpd19:58:351
150669943,0cyclictest3514-21snmpd19:42:501
150669943,0cyclictest3514-21snmpd00:03:211
150669943,0cyclictest32463-21awk20:25:581
150669943,0cyclictest3241-21irqbalance23:36:301
150669943,0cyclictest3241-21irqbalance22:26:101
150669943,0cyclictest3241-21irqbalance21:26:401
150669943,0cyclictest3241-21irqbalance20:40:501
150669943,0cyclictest3241-21irqbalance20:36:101
150669943,0cyclictest3241-21irqbalance20:30:501
150669943,0cyclictest3241-21irqbalance20:20:511
150669943,0cyclictest3241-21irqbalance20:11:011
150669943,0cyclictest3241-21irqbalance19:31:011
150669943,0cyclictest3241-21irqbalance19:26:211
150669943,0cyclictest3241-21irqbalance19:22:111
150669943,0cyclictest3241-21irqbalance19:06:011
150669943,0cyclictest3241-21irqbalance19:00:511
150669943,0cyclictest22466-21awk21:45:561
150669943,0cyclictest21541-21awk22:45:561
150669943,0cyclictest11457-21awk19:45:561
150659944,0cyclictest0-21swapper/022:18:080
150659944,0cyclictest0-21swapper/000:00:460
150659943,0cyclictest6738-21awk20:30:540
150659943,0cyclictest6201-21awk21:55:570
150659943,0cyclictest3514-21snmpd20:45:200
150659943,0cyclictest29408-21sort19:10:560
150659943,0cyclictest28364-21sadc19:10:430
150659943,0cyclictest26699-21awk20:45:570
150659943,0cyclictest22497-21awk21:45:560
150659943,0cyclictest21233-21awk23:05:560
150659943,0cyclictest12994-21awk23:20:570
150659943,0cyclictest11829-21sadc23:20:410
150659942,0cyclictest13205-21sadc21:00:410
149940,0migration/08814-21kthreadcore21:35:590
149940,0migration/031240-21kthreadcore00:10:590
149940,0migration/027458-21kthreadcore20:20:590
149940,0migration/021128-21fschecks_count22:45:540
149940,0migration/013954-21kthreadcore22:40:590
149940,0migration/012730-21kthreadcore19:20:580
29725030,0irq/27-eth0-tx-0-21swapper/019:00:540
29715030,0irq/26-eth0-rx-0-21swapper/320:16:003
29715030,0irq/26-eth0-rx-0-21swapper/319:20:533
29715030,0irq/26-eth0-rx-0-21swapper/319:00:533
249930,0watchdog/20-21swapper/220:05:522
179930,0migration/14541-21fschecks_count23:15:541
169930,0watchdog/113210-21diskmemload23:48:001
150679932,0cyclictest6559-21runrttasks20:10:572
150679932,0cyclictest3556-21lldpd19:20:442
150669932,0cyclictest1900-21kthreadcore21:11:001
150659932,0cyclictest935-21awk20:50:570
150659932,0cyclictest8572-21grep23:35:590
150659932,0cyclictest5560-21grep23:55:580
150659932,0cyclictest3514-21snmpd19:52:050
150659932,0cyclictest3514-21snmpd19:25:530
150659932,0cyclictest30716-21if_err_eth121:30:560
150659932,0cyclictest29457-21grep22:30:540
150659932,0cyclictest22420-21if_err_eth1.221:05:550
150659932,0cyclictest21945-21grep23:25:590
150659932,0cyclictest20848-21awk23:45:550
339920,0migration/35252-21diskstats20:05:543
339920,0migration/312527-21iostat_ios20:10:573
29715020,0irq/26-eth0-rx-0-21swapper/319:35:583
29715020,0irq/26-eth0-rx-0-21swapper/319:31:023
29715020,0irq/26-eth0-rx-0-21swapper/319:10:513
150659921,0cyclictest10537-21sed19:45:430
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional