You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-19 - 15:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Sun Apr 19, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1872423627,0sleep30-21swapper/306:55:303
1855723425,0sleep20-21swapper/206:53:332
1862923322,0sleep00-21swapper/006:54:270
1854522622,0sleep10-21swapper/106:53:281
190099977,0cyclictest15215-21turbostat08:08:010
35160,0ktimersoftd/30-21swapper/312:27:073
339960,0migration/38741-21dump-pmu-power10:13:013
339960,0migration/36663-21latency09:08:173
339960,0migration/35293-21kthreadcore08:48:173
339960,0migration/324209-21sh10:21:243
339960,0migration/315184-21sh10:13:313
339960,0migration/313316-21latency_hist07:18:013
29715060,0irq/26-eth0-rx-26433-21iostat09:03:163
29715060,0irq/26-eth0-rx-10384-21sed09:13:153
249960,0watchdog/20-21swapper/209:43:192
190099965,0cyclictest5007-21grep08:48:170
190099965,0cyclictest14752-21sh12:18:000
149960,0migration/024234-21sh09:19:250
149960,0migration/023742-21sh08:59:180
149960,0migration/015327-21sh11:16:280
149960,0migration/014873-21sh10:54:210
339950,0migration/37022-21taskset12:13:023
339950,0migration/36229-21kthreadcore07:33:203
339950,0migration/34365-21kthreadcore07:08:183
339950,0migration/332406-21munin-node12:08:153
339950,0migration/330728-21kthreadcore07:28:193
339950,0migration/329949-21cpuspeed_turbos08:43:133
339950,0migration/327604-21kthreadcore09:23:173
339950,0migration/32600-21taskset08:21:443
339950,0migration/325068-21kthreadcore08:38:173
339950,0migration/323504-21taskset09:58:263
339950,0migration/323224-21munin-node09:38:203
339950,0migration/322470-21sh11:40:043
339950,0migration/322332-21cpuspeed_turbos08:13:133
339950,0migration/321985-21munin-node10:58:203
339950,0migration/321783-21kthreadcore06:58:183
339950,0migration/321508-21kthreadcore08:33:213
339950,0migration/31864-21fschecks_time09:48:123
339950,0migration/318298-21kthreadcore10:38:173
339950,0migration/316764-21taskset08:54:073
339950,0migration/315889-21turbostat.cron11:18:013
339950,0migration/31504-21idleruntime10:28:143
339950,0migration/314747-21sh10:53:583
339950,0migration/314519-21munin-node07:18:163
339950,0migration/310413-21kthreadcore10:33:173
339950,0migration/310208-21idleruntime09:53:153
29725050,0irq/27-eth0-tx-9687-21perl10:13:152
29725050,0irq/27-eth0-tx-30570-21ssh11:24:122
29725050,0irq/27-eth0-tx-24458-21iostat_ios12:03:182
29725050,0irq/27-eth0-tx-23998-21ssh09:00:052
29725050,0irq/27-eth0-tx-2038-21cut12:08:172
29725050,0irq/27-eth0-tx-17445-21perl10:38:142
29725050,0irq/27-eth0-tx-1715-21hostname10:08:152
29725050,0irq/27-eth0-tx-0-21swapper/211:15:352
29725050,0irq/27-eth0-tx-0-21swapper/008:27:050
29715050,0irq/26-eth0-rx-9606-21ssh09:33:133
29715050,0irq/26-eth0-rx-77332sleep311:12:423
29715050,0irq/26-eth0-rx-6801-21grep09:28:183
29715050,0irq/26-eth0-rx-30596-21netstat10:23:203
29715050,0irq/26-eth0-rx-26323-21perl09:43:153
29715050,0irq/26-eth0-rx-25738-21irqrtprio11:03:173
29715050,0irq/26-eth0-rx-1650-21kthreadcore10:48:173
29715050,0irq/26-eth0-rx-15774-21kthreadcore07:43:183
29715050,0irq/26-eth0-rx-0-21swapper/312:18:193
29715050,0irq/26-eth0-rx-0-21swapper/311:58:233
29715050,0irq/26-eth0-rx-0-21swapper/311:53:063
29715050,0irq/26-eth0-rx-0-21swapper/311:49:093
29715050,0irq/26-eth0-rx-0-21swapper/311:33:193
29715050,0irq/26-eth0-rx-0-21swapper/311:25:203
29715050,0irq/26-eth0-rx-0-21swapper/311:18:313
29715050,0irq/26-eth0-rx-0-21swapper/311:18:313
29715050,0irq/26-eth0-rx-0-21swapper/310:46:553
29715050,0irq/26-eth0-rx-0-21swapper/310:03:193
29715050,0irq/26-eth0-rx-0-21swapper/308:58:433
29715050,0irq/26-eth0-rx-0-21swapper/307:38:153
259950,0migration/28938-21aten_r1power_vo08:03:122
259950,0migration/28480-21kthreadcore12:13:172
259950,0migration/25944-21switchtime11:08:242
259950,0migration/231816-21sh10:25:112
259950,0migration/229820-21sh11:03:192
259950,0migration/226515-21irqcore09:03:172
259950,0migration/223508-21timerandwakeup09:38:232
259950,0migration/220462-21latency12:18:192
259950,0migration/220093-21kthreadcore10:18:192
259950,0migration/218034-21kthreadcore07:43:182
259950,0migration/214083-21fschecks_time07:18:142
249950,0watchdog/223500-21python09:18:232
190149955,0cyclictest0-21swapper/211:59:062
190149955,0cyclictest0-21swapper/209:08:222
190149954,0cyclictest3514-21snmpd07:04:502
190149950,0cyclictest0-21swapper/211:42:422
190149950,0cyclictest0-21swapper/211:33:242
190149950,0cyclictest0-21swapper/209:52:022
190109955,0cyclictest0-21swapper/111:03:151
190099955,0cyclictest0-21swapper/011:52:090
190099954,0cyclictest6559-21runrttasks11:28:200
190099950,0cyclictest0-21swapper/010:23:420
179950,0migration/114778-21kthreadcore08:53:181
169950,0watchdog/124178-21ssh09:19:151
149950,0migration/08098-21unixbench_singl07:58:240
149950,0migration/0493-21fschecks_count07:08:140
149950,0migration/031945-21kthreadcore08:43:180
149950,0migration/031008-21taskset12:07:430
149950,0migration/029682-21timerandwakeup11:43:240
149950,0migration/02968-21kthreadcore10:08:180
149950,0migration/029390-21kthreadcore09:23:180
149950,0migration/028497-21kthreadcore07:28:180
149950,0migration/028111-21taskset08:13:380
149950,0migration/026608-21if_eth1.209:43:170
149950,0migration/024032-21munin-node12:23:170
149950,0migration/023364-21df_abs08:38:130
149950,0migration/022145-21timerwakeupswit11:18:250
149950,0migration/022145-21timerwakeupswit11:18:250
149950,0migration/018667-21kthreadcore10:38:170
149950,0migration/018484-21kthreadcore11:38:170
149950,0migration/018189-21kthreadcore12:18:180
149950,0migration/017010-21fschecks_time08:33:170
149950,0migration/016625-21kthreadcore07:43:180
149950,0migration/012770-21kthreadcore08:28:160
149950,0migration/012188-21kthreadcore07:38:190
149950,0migration/011340-21kthreadcore10:33:180
149950,0migration/011299-21kthreadcore11:33:190
149950,0migration/010660-21kthreadcore11:53:170
339940,0migration/34293-21kthreadcore11:28:173
339940,0migration/311769-21kthreadcore08:28:163
29725040,0irq/27-eth0-tx-25372-21munin-node10:03:122
29725040,0irq/27-eth0-tx-24714-21ssh10:43:022
29725040,0irq/27-eth0-tx-0-21swapper/211:53:352
29725040,0irq/27-eth0-tx-0-21swapper/211:48:022
29725040,0irq/27-eth0-tx-0-21swapper/211:43:052
29725040,0irq/27-eth0-tx-0-21swapper/210:58:082
29725040,0irq/27-eth0-tx-0-21swapper/210:53:122
29725040,0irq/27-eth0-tx-0-21swapper/210:34:312
29725040,0irq/27-eth0-tx-0-21swapper/210:28:192
29725040,0irq/27-eth0-tx-0-21swapper/210:00:162
29725040,0irq/27-eth0-tx-0-21swapper/209:53:342
29725040,0irq/27-eth0-tx-0-21swapper/209:33:062
29725040,0irq/27-eth0-tx-0-21swapper/209:23:052
29715040,0irq/26-eth0-rx-0-21swapper/312:03:203
29715040,0irq/26-eth0-rx-0-21swapper/311:43:103
29715040,0irq/26-eth0-rx-0-21swapper/309:22:423
259940,0migration/29456-21munin-node08:48:202
259940,0migration/2731-21fschecks_time07:08:142
259940,0migration/232470-21munin-node11:28:122
259940,0migration/23056-21kthreadcore10:48:172
259940,0migration/227786-21runrttasks07:53:022
259940,0migration/22351-21idleruntime09:28:152
259940,0migration/220140-21runrttasks07:23:042
259940,0migration/216914-21gltestperf11:18:152
259940,0migration/216914-21gltestperf11:18:152
259940,0migration/213149-21kthreadcore08:28:162
259940,0migration/210586-21kthreadcore07:38:182
249940,0watchdog/20-21swapper/209:14:032
249940,0watchdog/20-21swapper/208:08:152
190149943,0cyclictest3556-21lldpd08:33:052
190149943,0cyclictest3556-21lldpd08:23:052
190149943,0cyclictest3556-21lldpd07:58:052
190149943,0cyclictest3556-21lldpd07:14:052
190149943,0cyclictest3514-21snmpd08:46:502
190149943,0cyclictest3514-21snmpd08:16:062
190149943,0cyclictest3514-21snmpd07:50:202
190149943,0cyclictest3514-21snmpd07:33:052
190149943,0cyclictest3514-21snmpd06:58:052
190149943,0cyclictest32219-21netstat07:28:202
190149943,0cyclictest29662-21awk08:18:182
190149943,0cyclictest23983-21awk12:23:172
190149943,0cyclictest23837-21awk08:38:152
190109944,0cyclictest0-21swapper/110:54:351
190109944,0cyclictest0-21swapper/109:00:351
190109943,0cyclictest3556-21lldpd12:18:051
190109943,0cyclictest3556-21lldpd12:16:051
190109943,0cyclictest3556-21lldpd12:10:051
190109943,0cyclictest3556-21lldpd11:58:051
190109943,0cyclictest3556-21lldpd11:15:351
190109943,0cyclictest3556-21lldpd11:00:351
190109943,0cyclictest3556-21lldpd09:40:351
190109943,0cyclictest3556-21lldpd08:48:051
190109943,0cyclictest3556-21lldpd08:13:351
190109943,0cyclictest3556-21lldpd08:12:351
190109943,0cyclictest3556-21lldpd07:48:061
190109943,0cyclictest3556-21lldpd07:44:051
190109943,0cyclictest3556-21lldpd07:38:051
190109943,0cyclictest3556-21lldpd07:21:361
190109943,0cyclictest3514-21snmpd07:54:061
190109943,0cyclictest3514-21snmpd07:35:351
190109943,0cyclictest27781-21awk07:28:171
190109943,0cyclictest17906-21awk10:18:161
190109943,0cyclictest1191-21awk10:48:151
190109940,0cyclictest0-21swapper/111:31:001
190109940,0cyclictest0-21swapper/109:48:411
190109940,0cyclictest0-21swapper/109:08:571
190099944,0cyclictest0-21swapper/011:06:440
190099944,0cyclictest0-21swapper/010:19:360
190099943,0cyclictest3556-21lldpd09:12:360
190099943,0cyclictest3556-21lldpd07:50:350
190099943,0cyclictest3514-21snmpd07:23:500
190099943,0cyclictest3241-21irqbalance09:04:390
190099943,0cyclictest20798-21kworker/0:006:58:190
190099943,0cyclictest1960-21kworker/0:110:32:160
190099943,0cyclictest18612-21awk09:38:150
190099943,0cyclictest1807-21sadc07:58:010
190099943,0cyclictest17291-21awk10:58:150
190099943,0cyclictest16425-21awk08:08:170
190099942,0cyclictest20798-21kworker/0:009:36:380
190099940,0cyclictest0-21swapper/010:52:520
179940,0migration/15703-21kthreadcore10:28:181
179940,0migration/15022-21kthreadcore08:23:181
179940,0migration/14679-21kthreadcore09:28:181
179940,0migration/13445-21kthreadcore07:58:171
179940,0migration/132348-21latency_hist11:08:011
179940,0migration/130931-21irqrtprio08:43:171
179940,0migration/129703-21taskset12:03:521
179940,0migration/128716-21taskset08:18:031
179940,0migration/12736-21kthreadcore07:08:171
179940,0migration/126870-21kthreadcore10:23:181
179940,0migration/125304-21latency_hist10:03:011
179940,0migration/124793-21kthreadcore08:38:171
179940,0migration/123659-21latency_hist11:43:011
179940,0migration/122628-21kthreadcore07:23:181
179940,0migration/119042-21kthreadcore08:33:201
179940,0migration/117635-21kthreadcore11:18:181
179940,0migration/117635-21kthreadcore11:18:181
179940,0migration/114604-21munin-node10:13:221
179940,0migration/113847-21kthreadcore08:28:171
179940,0migration/110188-21idleruntime09:53:151
169940,0watchdog/132549-21ssh11:48:151
169940,0watchdog/125872-21rm09:23:071
169940,0watchdog/125211-21kthreadcore12:23:181
169940,0watchdog/124820-21fschecks_time11:23:151
169940,0watchdog/1221112sleep111:38:591
169940,0watchdog/117316-21ssh09:58:031
169940,0watchdog/10-21swapper/111:33:191
169940,0watchdog/10-21swapper/108:03:111
169940,0watchdog/10-21swapper/106:58:201
149940,0migration/09605-21kthreadcore07:13:170
149940,0migration/0388-21sh10:07:290
149940,0migration/031269-21runrttasks10:44:330
149940,0migration/023708-21sh09:59:000
149940,0migration/01947-21sh09:48:120
149940,0migration/016874-21ksmtuned09:16:290
149940,0migration/01651-21aten_r1power_vo09:28:120
149940,0migration/015396-21kthreadcore08:53:180
339930,0migration/312712-21kthreadcore08:03:183
29725030,0irq/27-eth0-tx-0-21swapper/007:03:130
29715030,0irq/26-eth0-rx-0-21swapper/308:23:133
29715030,0irq/26-eth0-rx-0-21swapper/308:08:063
29715030,0irq/26-eth0-rx-0-21swapper/307:48:123
29715030,0irq/26-eth0-rx-0-21swapper/307:23:123
190109933,0cyclictest0-21swapper/111:53:041
190109932,0cyclictest9405-21fschecks_time10:33:141
190109932,0cyclictest931-21basename10:08:121
190109932,0cyclictest7749-21awk07:13:141
190109932,0cyclictest3556-21lldpd10:43:061
190109932,0cyclictest3241-21irqbalance09:13:091
190099933,0cyclictest20798-21kworker/0:008:18:030
190099933,0cyclictest20798-21kworker/0:007:33:030
190099933,0cyclictest20798-21kworker/0:007:18:070
190099933,0cyclictest1960-21kworker/0:112:08:140
190099933,0cyclictest1960-21kworker/0:111:58:380
190099933,0cyclictest1960-21kworker/0:111:08:220
190099933,0cyclictest1960-21kworker/0:109:53:140
190099933,0cyclictest0-21swapper/011:23:080
190099932,0cyclictest9871-21awk10:13:160
19130,0ktimersoftd/10-21swapper/107:05:391
169930,0watchdog/10-21swapper/110:38:151
169930,0watchdog/10-21swapper/109:43:191
169930,0watchdog/10-21swapper/109:33:151
169930,0watchdog/10-21swapper/109:03:271
29715020,0irq/26-eth0-rx-0-21swapper/307:58:073
29715020,0irq/26-eth0-rx-0-21swapper/307:53:283
29715020,0irq/26-eth0-rx-0-21swapper/307:03:123
249920,0watchdog/20-21swapper/208:53:112
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional