You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-19 - 06:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Thu Mar 19, 2026 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
351710,0ktimersoftd/330439-21perf19:00:503
435725452,0sleep20-21swapper/218:57:292
437023624,0sleep00-21swapper/018:57:390
455323223,0sleep10-21swapper/118:59:481
472599140,0cyclictest0-21swapper/021:01:280
47269976,0cyclictest0-21swapper/121:58:181
47259960,0cyclictest0-21swapper/021:18:330
47259960,0cyclictest0-21swapper/000:05:300
339960,0migration/39397-21latency_hist23:45:583
339960,0migration/331067-21munin-node20:31:133
339960,0migration/32513-21sh22:00:363
339960,0migration/324901-21sh22:31:233
339960,0migration/323989-21sh23:52:103
339960,0migration/31399-21sa123:20:583
339960,0migration/31366-21sh22:18:063
339960,0migration/311694-21sh21:26:083
29725060,0irq/27-eth0-tx-2214-21munin-node20:36:140
29715060,0irq/26-eth0-rx-3201-13audispd23:00:583
29715060,0irq/26-eth0-rx-0-21swapper/323:56:043
149960,0migration/08362-21sh23:22:320
149960,0migration/018568-21sh22:10:260
47289955,0cyclictest2990-21diskmemload23:23:362
47289955,0cyclictest2990-21diskmemload00:06:482
47289954,0cyclictest0-21swapper/222:21:482
47289950,0cyclictest2990-21diskmemload22:04:372
47269955,0cyclictest0-21swapper/122:06:491
47269955,0cyclictest0-21swapper/121:12:251
47269955,0cyclictest0-21swapper/119:16:231
47269950,0cyclictest0-21swapper/122:38:401
47269950,0cyclictest0-21swapper/121:24:531
47259955,0cyclictest0-21swapper/023:48:190
47259955,0cyclictest0-21swapper/023:10:390
47259955,0cyclictest0-21swapper/000:21:540
47259952,0cyclictest1415-21munin-run23:20:580
4150,0ktimersoftd/00-21swapper/023:44:230
4150,0ktimersoftd/00-21swapper/021:07:120
339950,0migration/37645-21taskset00:01:563
339950,0migration/37435-21taskset20:12:013
339950,0migration/37435-21taskset20:12:013
339950,0migration/37281-21cpuspeed_turbos19:51:123
339950,0migration/36442-21taskset19:47:023
339950,0migration/34868-21latency_hist19:01:003
339950,0migration/34318-21kthreadcore23:01:153
339950,0migration/323981-21sendmail_mailtr19:11:193
339950,0migration/322260-21taskset20:51:073
339950,0migration/321226-21munin-node20:46:203
339950,0migration/319011-21latency19:56:163
339950,0migration/316511-21ksmtuned23:07:023
339950,0migration/311090-21aten_r1power_cu22:06:073
339950,0migration/310277-21kthreadcore19:26:163
29725050,0irq/27-eth0-tx-27513-21taskset21:56:112
29725050,0irq/27-eth0-tx-25859-21munin-node23:36:112
29725050,0irq/27-eth0-tx-24985-21munin-node19:16:160
29725050,0irq/27-eth0-tx-19948-21diskstats21:11:122
29725050,0irq/27-eth0-tx-1907-21ssh22:19:422
29725050,0irq/27-eth0-tx-19045-21munin-node22:11:202
29725050,0irq/27-eth0-tx-16796-21ssh23:07:532
29725050,0irq/27-eth0-tx-15070-21ssh00:26:232
29725050,0irq/27-eth0-tx-14670-21munin-node20:21:190
29725050,0irq/27-eth0-tx-1227-21aten2.4-expect00:01:072
29725050,0irq/27-eth0-tx-0-21swapper/221:43:132
29715050,0irq/26-eth0-rx-9007-21munin-node22:01:213
29715050,0irq/26-eth0-rx-8693-21kthreadcore21:41:153
29715050,0irq/26-eth0-rx-31975-21ssh00:18:063
29715050,0irq/26-eth0-rx-31956-21cut21:16:153
29715050,0irq/26-eth0-rx-21681-21kthreadcore21:31:153
29715050,0irq/26-eth0-rx-21677-21cat21:51:163
29715050,0irq/26-eth0-rx-17381-21cat23:50:583
29715050,0irq/26-eth0-rx-16402-21kthreadcore19:31:163
29715050,0irq/26-eth0-rx-0-21swapper/323:26:113
29715050,0irq/26-eth0-rx-0-21swapper/322:46:163
29715050,0irq/26-eth0-rx-0-21swapper/322:36:163
29715050,0irq/26-eth0-rx-0-21swapper/322:26:143
29715050,0irq/26-eth0-rx-0-21swapper/322:25:553
29715050,0irq/26-eth0-rx-0-21swapper/322:15:073
29715050,0irq/26-eth0-rx-0-21swapper/321:38:193
29715050,0irq/26-eth0-rx-0-21swapper/321:23:523
29715050,0irq/26-eth0-rx-0-21swapper/321:13:073
29715050,0irq/26-eth0-rx-0-21swapper/321:02:093
29715050,0irq/26-eth0-rx-0-21swapper/300:11:113
27150,0ktimersoftd/20-21swapper/223:59:592
259950,0migration/28899-21sh23:44:522
259950,0migration/28524-21kthreadcore19:51:172
259950,0migration/28235-21unixbench_singl23:01:212
259950,0migration/23382-21latency_hist21:20:582
259950,0migration/227243-21idleruntime22:56:132
259950,0migration/224924-21sh22:31:272
259950,0migration/220910-21kthreadcore00:11:152
259950,0migration/219833-21fschecks_count19:36:122
259950,0migration/217618-21sh23:51:102
259950,0migration/216914-21open_files21:06:182
259950,0migration/211626-21latency_hist21:25:582
249950,0watchdog/210841-21sh22:26:092
19150,0ktimersoftd/10-21swapper/123:59:591
19150,0ktimersoftd/10-21swapper/122:25:341
179950,0migration/131859-21missed_timers22:56:171
169950,0watchdog/119749-21iostat_ios22:31:131
149950,0migration/08381-21kthreadcore19:26:150
149950,0migration/06788-21munin-node20:11:170
149950,0migration/06788-21munin-node20:11:170
149950,0migration/0635-21kthreadcore20:31:130
149950,0migration/04290-21kthreadcore19:21:160
149950,0migration/04159-21runrttasks22:41:140
149950,0migration/03641-21fschecks_time22:01:110
149950,0migration/031132-21munin-node23:56:160
149950,0migration/02988-21sh22:21:100
149950,0migration/029486-21kthreadcore20:06:150
149950,0migration/027053-21irqcore19:41:150
149950,0migration/024671-21taskset19:15:090
149950,0migration/023693-21kthreadcore20:51:150
149950,0migration/02274-21cpuspeed_turbos23:01:100
149950,0migration/020852-21kthreadcore22:11:160
149950,0migration/020331-21kthreadcore22:51:150
149950,0migration/019553-21timerwakeupswit19:56:210
149950,0migration/019525-21latency_hist21:30:580
149950,0migration/018442-21irqcore00:11:130
149950,0migration/018419-21idleruntime23:51:140
149950,0migration/017504-21timerwakeupswit19:06:220
149950,0migration/016001-21kthreadcore21:46:150
149950,0migration/014654-21rt-features00:26:190
149950,0migration/011497-21kthreadcore19:51:180
8215040,0irq/24-0000:00:20150-21idleruntime-cro20:00:592
47289943,0cyclictest3556-21lldpd23:16:032
47289943,0cyclictest3556-21lldpd22:51:032
47289943,0cyclictest3556-21lldpd19:48:332
47289943,0cyclictest3556-21lldpd19:26:332
47289943,0cyclictest3556-21lldpd19:13:032
47289943,0cyclictest26879-21awk19:41:142
47289943,0cyclictest18841-21awk23:31:142
47289943,0cyclictest1724-21awk00:21:122
47289940,0cyclictest0-21swapper/223:11:112
47269944,0cyclictest0-21swapper/121:47:061
47269943,0cyclictest4251-21awk21:01:141
47269943,0cyclictest3997-21awk22:01:131
47269943,0cyclictest3556-21lldpd23:38:041
47269943,0cyclictest3556-21lldpd23:35:331
47269943,0cyclictest3556-21lldpd22:54:321
47269943,0cyclictest3556-21lldpd22:41:031
47269943,0cyclictest3556-21lldpd21:41:031
47269943,0cyclictest3556-21lldpd21:33:031
47269943,0cyclictest3556-21lldpd21:29:031
47269943,0cyclictest3556-21lldpd21:10:031
47269943,0cyclictest3556-21lldpd20:37:031
47269943,0cyclictest3556-21lldpd19:58:331
47269943,0cyclictest3556-21lldpd19:49:331
47269943,0cyclictest3556-21lldpd19:03:031
47269943,0cyclictest3556-21lldpd00:06:331
47269943,0cyclictest32522-21awk19:21:131
47269943,0cyclictest28950-21awk20:31:111
47269943,0cyclictest2294-21awk23:41:111
47269943,0cyclictest22216-21awk20:26:141
47269943,0cyclictest20271-21awk19:36:141
47269943,0cyclictest18565-21awk23:51:141
47269943,0cyclictest1774-21awk00:21:131
47269943,0cyclictest16517-21awk20:46:141
47269943,0cyclictest12524-21awk19:06:141
47269943,0cyclictest10762-21awk23:26:141
47259944,0cyclictest0-21swapper/023:30:490
47259944,0cyclictest0-21swapper/022:17:310
47259943,0cyclictest6023-21kworker/0:022:39:460
47259943,0cyclictest3514-21snmpd21:15:010
47259943,0cyclictest3514-21snmpd20:59:010
47259943,0cyclictest3241-21irqbalance21:36:070
47259943,0cyclictest20295-21awk19:36:140
47259943,0cyclictest19084-21awk23:11:130
47259943,0cyclictest16589-21awk20:46:140
47259943,0cyclictest11126-21idleruntime-cro21:45:580
47259943,0cyclictest10310-21awk00:06:120
47259940,0cyclictest0-21swapper/019:49:470
339940,0migration/332379-21kthreadcore20:56:163
339940,0migration/331193-21taskset19:18:593
339940,0migration/321830-21kthreadcore23:31:163
339940,0migration/315865-21kthreadcore21:06:153
339940,0migration/315839-21irqrtprio20:21:123
339940,0migration/313053-21kthreadcore00:26:153
29725040,0irq/27-eth0-tx-9034-21telnet20:41:100
29725040,0irq/27-eth0-tx-8015-21munin-node20:16:100
29725040,0irq/27-eth0-tx-2990-21diskmemload21:46:402
29725040,0irq/27-eth0-tx-27685-21munin-node21:36:122
29725040,0irq/27-eth0-tx-0-21swapper/223:26:012
29725040,0irq/27-eth0-tx-0-21swapper/222:41:332
29725040,0irq/27-eth0-tx-0-21swapper/222:06:052
29725040,0irq/27-eth0-tx-0-21swapper/221:51:052
29725040,0irq/27-eth0-tx-0-21swapper/200:16:022
29715040,0irq/26-eth0-rx-21854-21kthreadcore23:11:153
29715040,0irq/26-eth0-rx-21392-21munin-node20:26:103
29715040,0irq/26-eth0-rx-1717-21if_err_eth1.300:21:123
29715040,0irq/26-eth0-rx-0-21swapper/323:36:033
29715040,0irq/26-eth0-rx-0-21swapper/323:21:093
29715040,0irq/26-eth0-rx-0-21swapper/322:51:163
29715040,0irq/26-eth0-rx-0-21swapper/322:43:383
29715040,0irq/26-eth0-rx-0-21swapper/321:46:103
29715040,0irq/26-eth0-rx-0-21swapper/319:41:193
29715040,0irq/26-eth0-rx-0-21swapper/319:06:103
29715040,0irq/26-eth0-rx-0-21swapper/300:06:083
259940,0migration/25524-21kthreadcore20:11:162
259940,0migration/25524-21kthreadcore20:11:162
259940,0migration/230701-21kthreadcore20:31:132
259940,0migration/22530-21kthreadcore19:21:152
259940,0migration/22455-21cpuspeed_turbos20:36:102
259940,0migration/221367-21munin-node20:01:142
259940,0migration/220989-21kernelversion21:31:152
259940,0migration/216489-21kthreadcore19:31:162
259940,0migration/211659-21irqrtprio22:46:142
249940,0watchdog/29111-21python21:01:212
249940,0watchdog/2894-21ssh22:37:452
249940,0watchdog/215075-21entropy20:21:092
249940,0watchdog/21336-21munin-node21:16:212
249940,0watchdog/20-21swapper/223:46:202
249940,0watchdog/20-21swapper/220:26:132
249940,0watchdog/20-21swapper/219:16:222
249940,0watchdog/20-21swapper/219:01:142
179940,0migration/19852-21kthreadcore19:51:171
179940,0migration/1969-21taskset22:16:581
179940,0migration/17714-21timerandwakeup23:21:211
179940,0migration/129085-21kthreadcore20:06:151
179940,0migration/128020-21df21:36:111
179940,0migration/124874-21diskmemload21:51:181
179940,0migration/122936-21kthreadcore22:11:171
179940,0migration/121757-21kthreadcore19:11:151
179940,0migration/111397-21kthreadcore23:46:151
179940,0migration/111069-21fschecks_time22:46:121
169940,0watchdog/12990-21diskmemload22:26:091
169940,0watchdog/125261-21ssh00:16:041
169940,0watchdog/124143-21unixbench_multi23:11:211
169940,0watchdog/110223-21aten2.4-expect23:06:091
169940,0watchdog/10-21swapper/123:16:131
169940,0watchdog/10-21swapper/120:51:171
169940,0watchdog/10-21swapper/120:21:131
169940,0watchdog/10-21swapper/119:41:211
169940,0watchdog/10-21swapper/100:11:201
169940,0watchdog/10-21swapper/100:02:561
149940,0migration/0420-21sh00:20:480
149940,0migration/023238-21kthreadcore22:31:150
149940,0migration/018963-21irqcore23:31:150
149940,0migration/014957-21kthreadcore22:26:150
47289932,0cyclictest3556-21lldpd20:16:032
47289932,0cyclictest3556-21lldpd19:06:032
47289930,0cyclictest27365-21df20:06:112
47269933,0cyclictest3556-21lldpd20:01:031
47269932,0cyclictest9954-21awk20:41:141
47269932,0cyclictest7994-21cut20:16:001
47269932,0cyclictest3556-21lldpd19:27:031
47269932,0cyclictest3556-21lldpd00:26:031
47269932,0cyclictest30217-21tr21:16:151
47269932,0cyclictest29879-21ethtool20:56:141
47269932,0cyclictest2126-21awk20:11:131
47269932,0cyclictest2126-21awk20:11:131
47269932,0cyclictest13550-21awk19:31:141
47259933,0cyclictest6023-21kworker/0:022:56:000
47259933,0cyclictest6023-21kworker/0:022:46:040
47259933,0cyclictest6023-21kworker/0:021:56:100
47259933,0cyclictest6023-21kworker/0:021:51:020
47259933,0cyclictest6023-21kworker/0:021:26:060
47259933,0cyclictest6023-21kworker/0:021:21:020
47259933,0cyclictest6023-21kworker/0:020:26:020
47259933,0cyclictest6023-21kworker/0:020:01:010
47259933,0cyclictest28592-21kworker/0:219:31:010
47259933,0cyclictest28592-21kworker/0:219:01:030
47259933,0cyclictest18628-21kworker/0:123:36:040
29715030,0irq/26-eth0-rx-0-21swapper/320:41:103
29715030,0irq/26-eth0-rx-0-21swapper/320:16:153
29715030,0irq/26-eth0-rx-0-21swapper/320:09:493
29715030,0irq/26-eth0-rx-0-21swapper/320:01:103
29715030,0irq/26-eth0-rx-0-21swapper/319:36:093
259930,0migration/225115-21kthreadcore20:51:152
249930,0watchdog/20-21swapper/220:46:172
249930,0watchdog/20-21swapper/220:41:172
169930,0watchdog/10-21swapper/123:01:131
29715020,0irq/26-eth0-rx-0-21swapper/320:36:163
29715020,0irq/26-eth0-rx-0-21swapper/319:21:353
249920,0watchdog/20-21swapper/220:56:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional