You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-30 - 23:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Mon Mar 30, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
412224333,0sleep30-21swapper/306:56:493
419523324,0sleep10-21swapper/106:57:391
403123228,0sleep20-21swapper/206:55:442
405022723,0sleep00-21swapper/006:55:580
44619990,0cyclictest0-21swapper/112:25:291
44599988,0cyclictest0-21swapper/009:00:130
44619976,0cyclictest26261-21ssh09:53:581
44619975,0cyclictest0-21swapper/111:19:061
44619972,0cyclictest11409-21awk10:05:101
44619970,0cyclictest0-21swapper/109:44:501
29715070,0irq/26-eth0-rx-22546-21unixbench_singl12:10:173
44699960,0cyclictest0-21swapper/210:39:112
44619964,0cyclictest0-21swapper/111:46:231
44619962,0cyclictest30663-21grep10:35:121
44619962,0cyclictest27402-21awk08:05:091
44619962,0cyclictest24186-21grep08:25:111
44619962,0cyclictest16158-21awk08:45:111
44619961,0cyclictest8303-21netstat09:00:131
44619961,0cyclictest32264-21munin-node07:45:131
44619961,0cyclictest30535-21netstat11:35:151
44619961,0cyclictest27510-21aten_r1power_en08:30:041
44619961,0cyclictest17079-21hostname11:30:041
44619961,0cyclictest1623-21awk08:10:101
44619961,0cyclictest14478-21kthreadcore09:45:121
44619961,0cyclictest0-21swapper/110:44:571
44619960,0cyclictest9128-21aten_r1power_en11:25:051
44619960,0cyclictest835-21idleruntime12:20:081
44619960,0cyclictest7964-21missed_timers10:00:131
44619960,0cyclictest7537-21ssh11:42:071
44619960,0cyclictest5696-21sh07:25:061
44619960,0cyclictest31696-21diskstats07:20:071
44619960,0cyclictest30053-21kthreadcore08:55:111
44619960,0cyclictest29240-21kthreadcore09:35:121
44619960,0cyclictest26141-21tune2fs07:40:081
44619960,0cyclictest25423-21sh10:33:491
44619960,0cyclictest25211-21ssh10:12:061
44619960,0cyclictest20595-21idleruntime-cro08:24:541
44619960,0cyclictest2024-21munin-node11:20:081
44619960,0cyclictest18810-21kthreadcore11:10:101
44619960,0cyclictest17000-21ssh10:27:341
44619960,0cyclictest15170-21ssh12:06:541
44619960,0cyclictest11147-21kthreadcore11:05:121
44619960,0cyclictest0-21swapper/109:34:441
44619960,0cyclictest0-21swapper/109:20:061
44619960,0cyclictest0-21swapper/109:10:181
35160,0ktimersoftd/30-21swapper/309:07:003
339960,0migration/39398-21sh11:04:573
339960,0migration/34345-21needreboot07:20:133
339960,0migration/327244-21latency_hist09:14:543
339960,0migration/32573-21munin-node10:40:093
339960,0migration/324663-21sh11:34:383
339960,0migration/322131-21latency11:50:123
339960,0migration/320482-21munin-node08:45:133
339960,0migration/31806-21sh09:59:003
339960,0migration/316920-21sh10:48:173
339960,0migration/316691-21sh10:05:293
29715060,0irq/26-eth0-rx-2359-21iostat_ios11:00:103
29715060,0irq/26-eth0-rx-2233-21cut08:10:113
29715060,0irq/26-eth0-rx-0-21swapper/311:26:083
149960,0migration/09747-21sh10:24:390
149960,0migration/08947-21sh10:43:210
149960,0migration/028354-21sh09:15:100
149960,0migration/026392-21fschecks_time10:55:070
149960,0migration/017862-21sh09:07:250
44699955,0cyclictest0-21swapper/211:20:292
44699955,0cyclictest0-21swapper/210:59:032
44699955,0cyclictest0-21swapper/209:44:002
44699955,0cyclictest0-21swapper/209:15:092
44699955,0cyclictest0-21swapper/209:10:442
44699955,0cyclictest0-21swapper/207:50:202
44699950,0cyclictest0-21swapper/212:11:342
44699950,0cyclictest0-21swapper/211:39:342
44699950,0cyclictest0-21swapper/208:04:502
44619951,0cyclictest25789-21aten_r1power_cu10:55:051
44619951,0cyclictest0-21swapper/107:59:541
44619951,0cyclictest0-21swapper/106:59:551
44619950,0cyclictest8407-21cat08:39:541
44619950,0cyclictest7542-21munin-node08:15:031
44619950,0cyclictest6329-21latency_hist07:49:541
44619950,0cyclictest32755-21ssh11:59:571
44619950,0cyclictest27246-21date09:14:541
44619950,0cyclictest26754-21latency_hist09:54:541
44619950,0cyclictest26300-21cat10:14:541
44619950,0cyclictest24749-21ssh11:54:541
44619950,0cyclictest24674-21hostname07:15:041
44619950,0cyclictest24355-21cut12:14:541
44619950,0cyclictest21722-21cat08:49:541
44619950,0cyclictest18980-21cut07:35:051
44619950,0cyclictest1848-21rm10:19:541
44619950,0cyclictest18027-21chrt07:09:591
44619950,0cyclictest17781-21aten2.4-expect10:50:041
44619950,0cyclictest1760-21aten2.4-expect08:35:051
44619950,0cyclictest16741-21rm11:49:571
44619950,0cyclictest16417-21ssh12:09:551
44619950,0cyclictest1624-21ssh10:39:561
44619950,0cyclictest13509-21grep07:55:061
44619950,0cyclictest1269-21ssh10:59:551
44619950,0cyclictest12264-21sed07:29:541
44619950,0cyclictest11390-21munin-node07:05:061
44619950,0cyclictest11207-21grep09:24:541
44619950,0cyclictest10819-21sh09:04:581
339950,0migration/35688-21ntp_offset12:20:143
339950,0migration/33888-21kthreadcore08:35:113
339950,0migration/33475-21kthreadcore11:20:093
339950,0migration/32982-21munin-node09:00:073
339950,0migration/327640-21munin-node08:50:163
339950,0migration/327561-21irqcore08:05:103
339950,0migration/325383-21fschecks_time11:55:083
339950,0migration/324119-21kthreadcore09:30:113
339950,0migration/32208-21taskset08:57:063
339950,0migration/321315-21diskmemload10:30:103
339950,0migration/319196-21kthreadcore10:50:103
339950,0migration/318710-21cpuspeed_turbos10:10:053
339950,0migration/315793-21kthreadcore07:05:123
339950,0migration/310031-21latency_hist10:24:543
29725050,0irq/27-eth0-tx-3593-21kthreadcore10:20:112
29725050,0irq/27-eth0-tx-28171-21kernelversion09:55:112
29725050,0irq/27-eth0-tx-27843-21perl09:35:092
29725050,0irq/27-eth0-tx-14846-21ssh11:25:162
29725050,0irq/27-eth0-tx-0-21swapper/211:10:542
29725050,0irq/27-eth0-tx-0-21swapper/209:22:492
29715050,0irq/26-eth0-rx-2746-21cat09:39:543
29715050,0irq/26-eth0-rx-25731-21cpuspeed_turbos11:15:053
29715050,0irq/26-eth0-rx-0-21swapper/312:05:113
29715050,0irq/26-eth0-rx-0-21swapper/312:03:103
29715050,0irq/26-eth0-rx-0-21swapper/311:41:013
29715050,0irq/26-eth0-rx-0-21swapper/311:35:133
29715050,0irq/26-eth0-rx-0-21swapper/311:10:113
29715050,0irq/26-eth0-rx-0-21swapper/310:20:023
29715050,0irq/26-eth0-rx-0-21swapper/310:15:093
29715050,0irq/26-eth0-rx-0-21swapper/310:03:483
29715050,0irq/26-eth0-rx-0-21swapper/309:51:213
29715050,0irq/26-eth0-rx-0-21swapper/309:48:063
29715050,0irq/26-eth0-rx-0-21swapper/309:20:373
29715050,0irq/26-eth0-rx-0-21swapper/309:10:093
27150,0ktimersoftd/20-21swapper/211:54:422
27150,0ktimersoftd/20-21swapper/211:33:152
259950,0migration/226728-21cpuspeed_turbos10:15:062
259950,0migration/223816-21latency10:10:112
259950,0migration/219103-21munin-node10:30:092
259950,0migration/218159-21munin-node07:30:172
249950,0watchdog/218543-21ssh09:28:282
249950,0watchdog/215071-21pmu-power11:05:162
149950,0migration/08442-21taskset12:04:570
149950,0migration/07358-21hddtemp_smartct07:50:070
149950,0migration/03390-21kthreadcore11:20:090
149950,0migration/032156-21kthreadcore08:55:110
149950,0migration/032083-21taskset12:19:380
149950,0migration/031607-21kthreadcore08:05:120
149950,0migration/031526-21unixbench_multi07:40:180
149950,0migration/028842-21kthreadcore11:35:140
149950,0migration/027581-21aten_r1power_po08:30:050
149950,0migration/027173-21irqcore10:35:100
149950,0migration/026913-21fschecks_count10:15:070
149950,0migration/02484-21kthreadcore08:10:110
149950,0migration/023424-21kthreadcore09:10:120
149950,0migration/022022-21kthreadcore08:00:120
149950,0migration/021010-21kthreadcore07:10:130
149950,0migration/020126-21hwlatdetect09:30:070
149950,0migration/018339-21latency_hist10:09:540
149950,0migration/018156-21kthreadcore11:50:100
149950,0migration/017876-21aten_r1power_po10:50:050
149950,0migration/015999-21idleruntime08:45:090
149950,0migration/01516-21irqcore12:00:100
149950,0migration/013965-21ntp_offset08:40:140
149950,0migration/013842-21kthreadcore09:25:100
149950,0migration/013710-21ntp_offset12:25:150
149950,0migration/012614-21cpuspeed_turbos07:30:070
149950,0migration/011356-21latency_hist07:04:550
149950,0migration/010236-21pmu-power07:00:140
8215040,0irq/24-0000:00:13890-21cat08:19:492
8215040,0irq/24-0000:00:0-21swapper/208:09:542
44699943,0cyclictest3514-21snmpd11:55:182
44699943,0cyclictest3514-21snmpd10:50:182
44699943,0cyclictest25621-21awk07:15:082
44599943,0cyclictest803-21awk07:45:110
44599943,0cyclictest3514-21snmpd10:03:330
44599943,0cyclictest3241-21irqbalance11:18:120
44599943,0cyclictest19994-21ps07:35:110
44599943,0cyclictest17474-21sadc10:49:540
44599943,0cyclictest10766-21cut10:25:080
44599940,0cyclictest0-21swapper/009:42:380
339940,0migration/38960-21kthreadcore07:00:113
339940,0migration/35971-21ksmtuned07:46:003
339940,0migration/330045-21kthreadcore10:35:113
339940,0migration/327979-21fschecks_count08:30:073
339940,0migration/322800-21kthreadcore08:00:123
339940,0migration/321970-21kthreadcore07:10:133
339940,0migration/317075-21kthreadcore07:55:123
339940,0migration/312023-21kthreadcore12:25:123
29725040,0irq/27-eth0-tx-8488-21munin-node12:05:052
29725040,0irq/27-eth0-tx-25403-21ssh11:14:592
29725040,0irq/27-eth0-tx-0-21swapper/212:25:282
29725040,0irq/27-eth0-tx-0-21swapper/212:15:092
29725040,0irq/27-eth0-tx-0-21swapper/212:00:122
29725040,0irq/27-eth0-tx-0-21swapper/211:40:132
29725040,0irq/27-eth0-tx-0-21swapper/211:00:292
29725040,0irq/27-eth0-tx-0-21swapper/210:46:192
29725040,0irq/27-eth0-tx-0-21swapper/210:40:382
29725040,0irq/27-eth0-tx-0-21swapper/210:25:062
29725040,0irq/27-eth0-tx-0-21swapper/210:05:202
29725040,0irq/27-eth0-tx-0-21swapper/209:50:042
29725040,0irq/27-eth0-tx-0-21swapper/209:04:572
29725040,0irq/27-eth0-tx-0-21swapper/209:00:472
29725040,0irq/27-eth0-tx-0-21swapper/008:35:090
29725040,0irq/27-eth0-tx-0-21swapper/008:15:030
29725040,0irq/27-eth0-tx-0-21swapper/007:55:060
29725040,0irq/27-eth0-tx-0-21swapper/007:25:100
29715040,0irq/26-eth0-rx-27613-21kthreadcore12:15:123
29715040,0irq/26-eth0-rx-14114-21kthreadcore07:30:113
29715040,0irq/26-eth0-rx-0-21swapper/311:45:043
29715040,0irq/26-eth0-rx-0-21swapper/310:55:013
29715040,0irq/26-eth0-rx-0-21swapper/309:36:543
29715040,0irq/26-eth0-rx-0-21swapper/309:25:143
29715040,0irq/26-eth0-rx-0-21swapper/307:50:053
29715040,0irq/26-eth0-rx-0-21swapper/307:35:063
259940,0migration/29608-21idleruntime11:45:102
259940,0migration/28226-21kthreadcore07:25:122
259940,0migration/24736-21kthreadcore10:00:112
259940,0migration/230738-21kthreadcore08:30:102
259940,0migration/22514-21hddtemp_smartct08:35:082
259940,0migration/219564-21cpuspeed_turbos09:30:052
259940,0migration/219109-21aten_r1power_vo07:35:062
259940,0migration/216477-21kernelversion08:45:112
259940,0migration/21592-21kthreadcore07:20:112
259940,0migration/211245-21latency_hist07:04:552
249940,0watchdog/232424-21ssh12:20:002
249940,0watchdog/221686-21if_err_eth1.208:25:092
249940,0watchdog/215546-21kthreadcore08:20:132
249940,0watchdog/20-21swapper/209:45:122
249940,0watchdog/20-21swapper/207:10:092
149940,0migration/07145-21sh11:40:570
149940,0migration/030323-21kthreadcore09:35:130
149940,0migration/023675-21runrttasks11:10:320
149940,0migration/021948-21irqcore08:25:100
149940,0migration/020442-21kthreadcore11:30:110
149940,0migration/02013-21kthreadcore12:20:110
149940,0migration/019256-21kthreadcore12:10:110
149940,0migration/016574-21kthreadcore08:20:130
44699932,0cyclictest22656-21awk08:50:082
44699932,0cyclictest12243-21awk07:05:092
44699930,0cyclictest18898-21python07:55:162
44699930,0cyclictest12604-21grep08:40:122
44699930,0cyclictest0-21swapper/207:44:572
44599933,0cyclictest9874-21kworker/0:111:44:550
44599933,0cyclictest9576-21kworker/0:211:54:570
44599933,0cyclictest4106-21awk09:20:080
44599933,0cyclictest22746-21kworker/0:211:05:090
44599933,0cyclictest22746-21kworker/0:210:59:550
44599933,0cyclictest22746-21kworker/0:210:29:570
44599933,0cyclictest22746-21kworker/0:210:05:150
44599933,0cyclictest22746-21kworker/0:209:54:570
44599933,0cyclictest22746-21kworker/0:209:50:150
44599933,0cyclictest22746-21kworker/0:209:45:090
44599933,0cyclictest22746-21kworker/0:208:50:040
44599933,0cyclictest22746-21kworker/0:207:20:020
44599933,0cyclictest22746-21kworker/0:207:14:580
44599932,0cyclictest10008-21awk11:25:090
339930,0migration/312242-21kthreadcore08:15:103
29715030,0irq/26-eth0-rx-0-21swapper/308:26:163
29715030,0irq/26-eth0-rx-0-21swapper/308:20:133
29715030,0irq/26-eth0-rx-0-21swapper/307:25:123
259930,0migration/229323-21if_err_eth108:55:092
259930,0migration/226444-21runrttasks07:40:092
249930,0watchdog/20-21swapper/208:10:172
29715020,0irq/26-eth0-rx-0-21swapper/308:40:043
29715020,0irq/26-eth0-rx-0-21swapper/307:40:123
29715020,0irq/26-eth0-rx-0-21swapper/307:15:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional