You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-12 - 13:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Fri Jun 12, 2026 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450550,0irq/25-eth10-21swapper/318:50:083
338224738,0sleep10-21swapper/118:51:531
2968023624,0sleep20-21swapper/218:48:102
2945323329,0sleep00-21swapper/018:48:000
8215060,0irq/24-0000:00:8913-21ssh23:37:562
35679965,0cyclictest30742-21ntp_states23:48:161
35679960,0cyclictest0-21swapper/123:03:021
35669960,0cyclictest0-21swapper/022:36:550
35669960,0cyclictest0-21swapper/021:52:550
339960,0migration/3689-21sh21:50:573
29725060,0irq/27-eth0-tx-0-21swapper/222:48:302
29715060,0irq/26-eth0-rx-24699-21ssh21:04:533
29715060,0irq/26-eth0-rx-19240-21iostat21:23:123
29715060,0irq/26-eth0-rx-18685-21fschecks_time22:03:093
29715060,0irq/26-eth0-rx-0-21swapper/323:58:003
4150,0ktimersoftd/00-21swapper/023:53:220
4150,0ktimersoftd/00-21swapper/022:55:030
4150,0ktimersoftd/00-21swapper/021:43:000
4150,0ktimersoftd/00-21swapper/021:29:230
35739955,0cyclictest0-21swapper/221:35:172
35739955,0cyclictest0-21swapper/200:08:412
35739950,0cyclictest0-21swapper/221:25:402
35679955,0cyclictest0-21swapper/123:43:041
35679955,0cyclictest0-21swapper/122:13:411
35679955,0cyclictest0-21swapper/121:40:341
35679955,0cyclictest0-21swapper/120:58:101
35679955,0cyclictest0-21swapper/100:21:511
35679950,0cyclictest0-21swapper/123:12:101
35679950,0cyclictest0-21swapper/122:54:591
35679950,0cyclictest0-21swapper/122:48:031
35669955,0cyclictest0-21swapper/023:03:330
35669955,0cyclictest0-21swapper/021:03:060
35669955,0cyclictest0-21swapper/019:52:570
35669955,0cyclictest0-21swapper/000:16:050
35669955,0cyclictest0-21swapper/000:04:530
35669950,0cyclictest0-21swapper/023:16:010
339950,0migration/38578-21taskset21:55:003
339950,0migration/37520-21runrttasks23:13:483
339950,0migration/3630-21aten_r1power_en00:13:073
339950,0migration/332731-21taskset00:12:513
339950,0migration/332154-21hwlatdetect19:38:103
339950,0migration/331770-21munin-node21:08:163
339950,0migration/329652-21kthreadcore23:08:133
339950,0migration/328985-21kthreadcore20:48:113
339950,0migration/325976-21munin-node19:33:123
339950,0migration/325730-21kthreadcore19:08:133
339950,0migration/32566-21kthreadcore23:53:123
339950,0migration/325630-21munin-node20:18:183
339950,0migration/322578-21kthreadcore23:03:143
339950,0migration/321007-21kthreadcore19:53:133
339950,0migration/320784-21latency_hist20:42:573
339950,0migration/319542-21kthreadcore21:43:143
339950,0migration/319333-21taskset20:13:253
339950,0migration/317804-21fschecks_time19:03:103
339950,0migration/31704-21cpu22:33:093
339950,0migration/316913-21latency_hist00:02:563
339950,0migration/31657-21idleruntime20:28:113
339950,0migration/314720-21missed_timers23:18:153
339950,0migration/313340-21kthreadcore22:18:143
339950,0migration/31295-21cpuspeed_turbos23:33:083
339950,0migration/311513-21timerandwakeup19:43:203
29725050,0irq/27-eth0-tx-9861-21munin-node20:58:122
29725050,0irq/27-eth0-tx-7249-21ssh00:14:232
29725050,0irq/27-eth0-tx-31362-21munin-node19:38:180
29725050,0irq/27-eth0-tx-0-21swapper/223:27:212
29725050,0irq/27-eth0-tx-0-21swapper/222:35:472
29725050,0irq/27-eth0-tx-0-21swapper/221:50:492
29715050,0irq/26-eth0-rx-8918-21sh21:35:403
29715050,0irq/26-eth0-rx-2564-21kernelversion22:53:123
29715050,0irq/26-eth0-rx-17367-21cpu23:43:083
29715050,0irq/26-eth0-rx-14868-21cut19:23:143
29715050,0irq/26-eth0-rx-1202-21kthreadcore20:03:133
29715050,0irq/26-eth0-rx-11501-21kthreadcore20:58:153
29715050,0irq/26-eth0-rx-0-21swapper/323:48:103
29715050,0irq/26-eth0-rx-0-21swapper/323:30:183
29715050,0irq/26-eth0-rx-0-21swapper/323:23:153
29715050,0irq/26-eth0-rx-0-21swapper/322:58:113
29715050,0irq/26-eth0-rx-0-21swapper/322:31:053
29715050,0irq/26-eth0-rx-0-21swapper/322:27:333
29715050,0irq/26-eth0-rx-0-21swapper/322:11:443
29715050,0irq/26-eth0-rx-0-21swapper/321:58:423
29715050,0irq/26-eth0-rx-0-21swapper/321:40:503
29715050,0irq/26-eth0-rx-0-21swapper/321:28:113
29715050,0irq/26-eth0-rx-0-21swapper/321:13:133
29715050,0irq/26-eth0-rx-0-21swapper/318:53:133
27150,0ktimersoftd/21582-21diskmemload23:53:222
27150,0ktimersoftd/20-21swapper/222:55:032
27150,0ktimersoftd/20-21swapper/221:29:232
259950,0migration/231862-21munin-node22:08:252
259950,0migration/231575-21latency21:08:152
259950,0migration/228228-21kthreadcore20:23:122
259950,0migration/225598-21aten_r1power_po23:08:072
259950,0migration/220134-21kthreadcore22:23:152
259950,0migration/220064-21timerwakeupswit20:38:192
259950,0migration/211891-21fschecks_count19:23:102
249950,0watchdog/219988-21kthreadcore22:03:152
19150,0ktimersoftd/10-21swapper/122:19:021
179950,0migration/129006-21latency19:08:141
179950,0migration/128484-21kthreadcore22:08:191
179950,0migration/125567-21latency_hist19:57:571
179950,0migration/12302-21fschecks_time21:53:091
179950,0migration/122031-21irqcore20:43:121
179950,0migration/120419-21kthreadcore19:53:131
179950,0migration/116609-21sh21:59:511
179950,0migration/116182-21unixbench-2d21:18:211
179950,0migration/115261-21munin-node23:18:191
179950,0migration/111374-21kthreadcore22:58:141
169950,0watchdog/16315-21cut20:53:151
169950,0watchdog/120075-21kthreadcore22:23:151
149950,0migration/08408-21munin-node20:33:100
149950,0migration/07762-21irqcore20:08:110
149950,0migration/06581-21munin-node19:43:120
149950,0migration/031706-21switchtime22:28:200
149950,0migration/025204-21taskset22:47:300
149950,0migration/02318-21fschecks_count21:33:090
149950,0migration/019863-21latency_hist20:17:570
149950,0migration/018829-21sh22:23:130
149950,0migration/016317-21sh22:40:020
149950,0migration/014761-21munin-node23:58:190
149950,0migration/013932-21kthreadcore23:18:140
8215040,0irq/24-0000:00:0-21swapper/220:42:572
35739944,0cyclictest0-21swapper/222:28:062
35739943,0cyclictest6628-21awk19:43:122
35739943,0cyclictest5668-21awk19:18:122
35739943,0cyclictest3556-21lldpd19:57:392
35739943,0cyclictest3556-21lldpd19:14:392
35739943,0cyclictest3514-21snmpd20:17:352
35739943,0cyclictest3514-21snmpd20:00:502
35739943,0cyclictest3514-21snmpd18:53:192
35739943,0cyclictest2692-21awk20:53:122
35739943,0cyclictest24755-21awk19:08:112
35739943,0cyclictest2052-21munin-node21:13:192
35739943,0cyclictest1783-21awk20:28:122
35739943,0cyclictest10014-21awk23:38:112
35679944,0cyclictest3556-21lldpd20:33:381
35679944,0cyclictest0-21swapper/120:03:081
35679944,0cyclictest0-21swapper/100:06:341
35679943,0cyclictest3556-21lldpd23:39:101
35679943,0cyclictest3556-21lldpd23:28:101
35679943,0cyclictest3556-21lldpd22:46:101
35679943,0cyclictest3556-21lldpd20:31:391
35679943,0cyclictest3514-21snmpd21:14:351
35679943,0cyclictest3514-21snmpd20:13:051
35679943,0cyclictest2382-21awk22:33:121
35679943,0cyclictest18190-21awk23:23:131
35669943,0cyclictest5644-21awk19:18:120
35669943,0cyclictest3556-21lldpd19:13:090
35669943,0cyclictest3556-21lldpd00:18:110
35669943,0cyclictest3514-21snmpd21:11:200
35669943,0cyclictest3241-21irqbalance23:38:550
35669943,0cyclictest3241-21irqbalance23:34:450
35669943,0cyclictest3241-21irqbalance22:50:450
35669943,0cyclictest3241-21irqbalance22:03:150
35669943,0cyclictest3241-21irqbalance22:02:350
35669943,0cyclictest3241-21irqbalance21:25:550
35669943,0cyclictest3241-21irqbalance18:53:060
35669943,0cyclictest30178-21sadc19:12:570
35669943,0cyclictest2950-21awk21:13:110
35669943,0cyclictest2709-21awk21:53:110
35669943,0cyclictest2707-21awk20:53:120
35669943,0cyclictest26979-21awk22:08:170
35669943,0cyclictest25930-21awk00:08:100
35669943,0cyclictest18153-21awk23:43:110
35669943,0cyclictest11406-21awk18:58:120
35669943,0cyclictest11071-21awk21:18:120
339940,0migration/332388-21kthreadcore19:13:133
339940,0migration/329861-21kthreadcore19:58:143
339940,0migration/320359-21kthreadcore19:28:143
339940,0migration/312429-21kthreadcore20:33:133
339940,0migration/310099-21fschecks_count22:38:103
29725040,0irq/27-eth0-tx-9670-21df_inode23:18:092
29725040,0irq/27-eth0-tx-23059-21ssh23:03:172
29725040,0irq/27-eth0-tx-17174-21munin-node23:43:082
29725040,0irq/27-eth0-tx-1582-21diskmemload21:06:042
29725040,0irq/27-eth0-tx-10372-21fschecks_time22:18:102
29725040,0irq/27-eth0-tx-10008-21munin-node21:38:122
29725040,0irq/27-eth0-tx-0-21swapper/223:48:392
29725040,0irq/27-eth0-tx-0-21swapper/223:28:112
29725040,0irq/27-eth0-tx-0-21swapper/222:58:372
29725040,0irq/27-eth0-tx-0-21swapper/222:38:092
29725040,0irq/27-eth0-tx-0-21swapper/222:13:032
29725040,0irq/27-eth0-tx-0-21swapper/221:58:102
29725040,0irq/27-eth0-tx-0-21swapper/221:43:062
29725040,0irq/27-eth0-tx-0-21swapper/221:18:152
29725040,0irq/27-eth0-tx-0-21swapper/200:18:002
29725040,0irq/27-eth0-tx-0-21swapper/200:03:102
29725040,0irq/27-eth0-tx-0-21swapper/020:32:480
29725040,0irq/27-eth0-tx-0-21swapper/020:03:070
29715040,0irq/26-eth0-rx-0-21swapper/323:38:073
29715040,0irq/26-eth0-rx-0-21swapper/322:49:123
29715040,0irq/26-eth0-rx-0-21swapper/322:43:023
29715040,0irq/26-eth0-rx-0-21swapper/322:16:083
29715040,0irq/26-eth0-rx-0-21swapper/321:17:583
29715040,0irq/26-eth0-rx-0-21swapper/320:53:103
29715040,0irq/26-eth0-rx-0-21swapper/320:43:133
29715040,0irq/26-eth0-rx-0-21swapper/320:23:073
29715040,0irq/26-eth0-rx-0-21swapper/300:18:133
259940,0migration/24979-21kthreadcore21:53:132
259940,0migration/24963-21kthreadcore23:13:132
259940,0migration/227967-21kthreadcore19:33:132
259940,0migration/224335-21kthreadcore20:18:142
259940,0migration/223308-21runrttasks19:05:162
259940,0migration/22105-21kthreadcore19:38:132
259940,0migration/220624-21kthreadcore19:28:142
259940,0migration/217682-21aten2.4-expect22:43:072
259940,0migration/215747-21latency18:58:152
259940,0migration/213657-21kernelversion19:48:142
259940,0migration/211672-21kthreadcore20:33:132
249940,0watchdog/27880-21cut20:08:112
249940,0watchdog/216076-21sh00:01:102
179940,0migration/1492-21latency_hist00:12:561
179940,0migration/13610-21munin-node19:13:171
179940,0migration/131613-21cpuspeed_turbos19:38:081
179940,0migration/126709-21kthreadcore00:08:121
179940,0migration/126362-21cpuspeed_turbos21:48:091
179940,0migration/126226-21taskset21:08:071
179940,0migration/125758-21timerandwakeup20:18:191
179940,0migration/124078-21taskset19:28:251
179940,0migration/122428-21kthreadcore21:23:141
179940,0migration/114857-21fschecks_time20:38:091
179940,0migration/11106-21runrttasks21:31:361
169940,0watchdog/17857-21ssh23:55:301
169940,0watchdog/125003-21ssh21:05:351
169940,0watchdog/124381-21ssh22:04:071
169940,0watchdog/124206-21ssh21:43:231
169940,0watchdog/10-21swapper/120:48:111
149940,0migration/031576-21taskset23:28:360
149940,0migration/028810-21kthreadcore19:33:130
149940,0migration/027918-21kthreadcore19:58:130
149940,0migration/027042-21fschecks_count20:23:090
149940,0migration/025428-21cpuspeed_turbos23:48:090
149940,0migration/024770-21kthreadcore20:18:140
149940,0migration/021314-21kthreadcore19:28:140
149940,0migration/017777-21kthreadcore20:38:130
149940,0migration/010786-21hddtemp_smartct21:38:100
149940,0migration/010113-21df_abs22:18:090
4130,0ktimersoftd/023766-21sh23:24:140
35739932,0cyclictest913-21grep20:03:122
35739932,0cyclictest28452-21awk20:48:092
35679933,0cyclictest0-21swapper/123:59:281
35679932,0cyclictest9473-21sensors18:53:171
35679932,0cyclictest6521-21awk19:43:111
35679932,0cyclictest5974-21grep23:13:131
35679932,0cyclictest3556-21lldpd22:38:101
35679932,0cyclictest27452-21awk20:23:101
35679932,0cyclictest26727-21awk22:28:121
35679932,0cyclictest25876-21irqrtprio19:33:121
35679932,0cyclictest17349-21cpuspeed_turbos19:03:091
35679932,0cyclictest13336-21awk19:48:131
35679931,0cyclictest17026-21processes19:23:181
35679930,0cyclictest1561-21perl23:33:091
35679930,0cyclictest0-21swapper/118:59:551
35669932,0cyclictest3241-21irqbalance22:58:050
35669932,0cyclictest25552-21telnet23:08:070
35669932,0cyclictest2521-21awk22:13:120
35669932,0cyclictest18744-21kthreadcore19:03:130
35669932,0cyclictest14127-21grep20:58:160
35669930,0cyclictest12381-21munin-node19:48:170
35669930,0cyclictest0-21swapper/020:48:080
29725030,0irq/27-eth0-tx-0-21swapper/019:23:080
29715030,0irq/26-eth0-rx-0-21swapper/320:09:303
29715030,0irq/26-eth0-rx-0-21swapper/319:20:363
29715030,0irq/26-eth0-rx-0-21swapper/318:58:353
169930,0watchdog/10-21swapper/120:08:111
149930,0migration/022637-21kthreadcore20:43:130
35679921,0cyclictest4674-21latency_hist19:17:571
35679921,0cyclictest1596-21latency_hist21:32:571
35120,0ktimersoftd/30-21swapper/319:48:083
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional