You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-10 - 23:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Fri Apr 10, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450560,0irq/25-eth10-21swapper/306:57:053
3244224132,0sleep10-21swapper/106:57:231
3184723531,0sleep00-21swapper/006:54:170
3233723127,0sleep20-21swapper/206:56:072
29725070,0irq/27-eth0-tx-13850-21munin-node07:09:070
339960,0migration/36598-21munin-node10:24:073
339960,0migration/34534-21sh11:03:013
339960,0migration/329665-21sh09:57:433
339960,0migration/327965-21sh11:57:503
339960,0migration/319465-21sh11:51:253
339960,0migration/313896-21runrttasks09:48:263
326399960,0cyclictest0-21swapper/208:44:132
326389966,0cyclictest0-21swapper/109:16:501
326389960,0cyclictest0-21swapper/108:49:051
29715060,0irq/26-eth0-rx-0-21swapper/309:14:153
249960,0watchdog/27538-21kthreadcore11:44:062
179960,0migration/120651-21unixbench-2d09:29:141
149960,0migration/030086-21fschecks_time10:59:050
8215050,0irq/24-0000:00:8900-21munin-run07:53:522
8215050,0irq/24-0000:00:21730-21latency_hist10:13:522
8215050,0irq/24-0000:00:0-21swapper/212:03:522
339950,0migration/37991-21kernelversion09:24:083
339950,0migration/34713-21kthreadcore07:49:093
339950,0migration/34635-21timerwakeupswit09:19:153
339950,0migration/34052-21munin-node09:39:123
339950,0migration/332040-21kthreadcore08:34:083
339950,0migration/331078-21taskset08:59:023
339950,0migration/329926-21rt-features08:54:123
339950,0migration/329839-21hddtemp_smartct08:09:063
339950,0migration/329054-21cpu11:39:023
339950,0migration/323539-21irqcore08:04:093
339950,0migration/323035-21df_abs09:34:033
339950,0migration/321499-21fschecks_time11:34:043
339950,0migration/321345-21taskset09:51:453
339950,0migration/318669-21kthreadcore07:34:083
339950,0migration/317071-21kthreadcore09:09:073
339950,0migration/316158-21kthreadcore08:44:093
339950,0migration/314446-21kthreadcore12:09:073
339950,0migration/313721-21aten_r1power_cu10:29:023
339950,0migration/313507-21taskset07:07:453
339950,0migration/310483-21munin-node08:19:043
326399955,0cyclictest0-21swapper/209:18:262
326399950,0cyclictest0-21swapper/209:33:532
326389955,0cyclictest0-21swapper/111:44:241
326389955,0cyclictest0-21swapper/110:05:141
326389950,0cyclictest0-21swapper/111:30:041
326389950,0cyclictest0-21swapper/110:56:101
326389950,0cyclictest0-21swapper/110:12:011
326389950,0cyclictest0-21swapper/109:52:471
326389950,0cyclictest0-21swapper/108:58:521
326379955,0cyclictest0-21swapper/012:22:020
326379955,0cyclictest0-21swapper/010:34:120
326379950,0cyclictest0-21swapper/011:35:510
326379950,0cyclictest0-21swapper/011:32:040
326379950,0cyclictest0-21swapper/010:25:030
29725050,0irq/27-eth0-tx-386-21munin-node06:59:040
29725050,0irq/27-eth0-tx-30980-21munin-node08:59:132
29725050,0irq/27-eth0-tx-29710-21munin-node10:39:142
29725050,0irq/27-eth0-tx-29142-21munin-node11:19:062
29725050,0irq/27-eth0-tx-20858-21munin-node11:34:042
29725050,0irq/27-eth0-tx-19725-21ssh10:49:392
29725050,0irq/27-eth0-tx-12303-21munin-node12:09:132
29725050,0irq/27-eth0-tx-11625-21ssh10:44:242
29725050,0irq/27-eth0-tx-0-21swapper/008:43:200
29715050,0irq/26-eth0-rx-9057-21ssh10:44:103
29715050,0irq/26-eth0-rx-6604-21tr11:24:083
29715050,0irq/26-eth0-rx-6559-21runrttasks10:40:263
29715050,0irq/26-eth0-rx-30292-21irqcore11:19:063
29715050,0irq/26-eth0-rx-24582-21cut10:14:093
29715050,0irq/26-eth0-rx-20930-21latency_hist11:13:523
29715050,0irq/26-eth0-rx-20032-21grep10:09:143
29715050,0irq/26-eth0-rx-14380-21ssh09:08:393
29715050,0irq/26-eth0-rx-1397-21awk12:19:103
29715050,0irq/26-eth0-rx-0-21swapper/312:14:053
29715050,0irq/26-eth0-rx-0-21swapper/312:04:153
29715050,0irq/26-eth0-rx-0-21swapper/311:44:003
29715050,0irq/26-eth0-rx-0-21swapper/311:31:423
29715050,0irq/26-eth0-rx-0-21swapper/311:04:123
29715050,0irq/26-eth0-rx-0-21swapper/310:49:143
29715050,0irq/26-eth0-rx-0-21swapper/310:33:543
29715050,0irq/26-eth0-rx-0-21swapper/310:00:103
29715050,0irq/26-eth0-rx-0-21swapper/307:44:103
29715050,0irq/26-eth0-rx-0-21swapper/307:25:513
259950,0migration/29361-21missed_timers12:24:092
259950,0migration/23818-21sh11:00:552
259950,0migration/213035-21latency_hist11:08:522
249950,0watchdog/24248-21sh12:03:542
179950,0migration/1629-21cpuspeed_turbos06:59:031
179950,0migration/15737-21kthreadcore12:24:081
179950,0migration/15349-21taskset09:41:571
179950,0migration/13547-21aten_r1power_vo08:14:051
179950,0migration/132750-21kthreadcore08:59:071
179950,0migration/13243-21latency_hist08:13:521
179950,0migration/128904-21sh10:16:351
179950,0migration/128409-21sendmail_mailqu09:34:111
179950,0migration/126269-21timerwakeupswit12:14:161
179950,0migration/124365-21kthreadcore07:39:091
179950,0migration/115268-21latency08:19:091
179950,0migration/113563-21taskset09:26:311
179950,0migration/11277-21kthreadcore09:19:091
179950,0migration/110503-21taskset12:04:251
169950,0watchdog/15785-21diskstats10:44:061
149950,0migration/08988-21latency_hist07:53:520
149950,0migration/05044-21gltestperf12:04:040
149950,0migration/04486-21fschecks_time12:24:040
149950,0migration/03506-21kthreadcore07:24:090
149950,0migration/031418-21gltestperf09:39:040
149950,0migration/03123-21sh11:19:510
149950,0migration/026277-21kthreadcore09:54:080
149950,0migration/025873-21kthreadcore09:14:090
149950,0migration/024402-21kthreadcore09:34:070
149950,0migration/022689-21latency_hist09:13:520
149950,0migration/022484-21kthreadcore11:54:060
149950,0migration/021527-21latency_hist10:33:520
149950,0migration/020020-21sh11:53:020
149950,0migration/019504-21kthreadcore07:59:090
149950,0migration/019250-21kthreadcore08:24:090
149950,0migration/018079-21kthreadcore10:49:080
149950,0migration/01657-21missed_timers11:59:090
149950,0migration/015875-21kthreadcore10:09:080
149950,0migration/01508-21kthreadcore08:09:100
149950,0migration/014734-21latency_hist07:33:520
339940,0migration/39503-21df_abs07:54:043
339940,0migration/37119-21irqcore10:04:063
339940,0migration/326099-21kthreadcore08:29:073
339940,0migration/323748-21taskset08:50:093
326399944,0cyclictest0-21swapper/208:29:052
326399943,0cyclictest3556-21lldpd09:20:202
326399943,0cyclictest3556-21lldpd07:55:212
326399943,0cyclictest3556-21lldpd07:37:502
326399943,0cyclictest3556-21lldpd07:25:512
326399943,0cyclictest23239-21awk09:54:052
326399940,0cyclictest13686-21munin-node10:29:042
326389944,0cyclictest0-21swapper/111:12:431
326389944,0cyclictest0-21swapper/110:29:031
326389943,0cyclictest6063-21awk11:04:061
326389943,0cyclictest5734-21awk11:24:061
326389943,0cyclictest3556-21lldpd12:19:211
326389943,0cyclictest3514-21snmpd12:03:301
326389943,0cyclictest3328-21awk07:49:081
326389943,0cyclictest3241-21irqbalance10:35:401
326389943,0cyclictest3241-21irqbalance09:11:201
326389943,0cyclictest30145-21awk11:19:061
326389943,0cyclictest21449-21awk11:54:041
326389943,0cyclictest13565-21awk11:49:041
326389943,0cyclictest11868-21awk08:44:071
326379944,0cyclictest0-21swapper/009:30:120
326379943,0cyclictest5783-21awk11:24:070
326379943,0cyclictest3556-21lldpd11:48:210
326379943,0cyclictest3556-21lldpd11:40:510
326379943,0cyclictest3556-21lldpd11:08:210
326379943,0cyclictest3514-21snmpd10:02:000
326379943,0cyclictest3514-21snmpd09:04:000
326379943,0cyclictest31994-21awk09:19:070
326379943,0cyclictest30849-21awk08:34:060
326379943,0cyclictest23270-21awk08:04:070
326379943,0cyclictest21147-21awk12:14:070
326379943,0cyclictest11823-21awk08:44:060
326379943,0cyclictest11048-21awk08:19:070
326379940,0cyclictest9128-21munin-node07:54:040
29725040,0irq/27-eth0-tx-5104-21munin-node11:04:042
29725040,0irq/27-eth0-tx-31141-21ssh08:59:020
29725040,0irq/27-eth0-tx-26692-21ssh12:15:052
29725040,0irq/27-eth0-tx-21980-21ssh10:13:572
29725040,0irq/27-eth0-tx-21673-21munin-node10:34:042
29725040,0irq/27-eth0-tx-0-21swapper/212:18:582
29725040,0irq/27-eth0-tx-0-21swapper/211:54:052
29725040,0irq/27-eth0-tx-0-21swapper/211:38:572
29725040,0irq/27-eth0-tx-0-21swapper/211:29:442
29725040,0irq/27-eth0-tx-0-21swapper/211:25:372
29725040,0irq/27-eth0-tx-0-21swapper/210:54:122
29725040,0irq/27-eth0-tx-0-21swapper/210:24:112
29725040,0irq/27-eth0-tx-0-21swapper/210:20:392
29725040,0irq/27-eth0-tx-0-21swapper/210:03:572
29725040,0irq/27-eth0-tx-0-21swapper/209:59:112
29725040,0irq/27-eth0-tx-0-21swapper/209:49:072
29725040,0irq/27-eth0-tx-0-21swapper/209:45:532
29725040,0irq/27-eth0-tx-0-21swapper/209:39:072
29725040,0irq/27-eth0-tx-0-21swapper/209:28:592
29725040,0irq/27-eth0-tx-0-21swapper/209:24:352
29725040,0irq/27-eth0-tx-0-21swapper/209:04:082
29725040,0irq/27-eth0-tx-0-21swapper/008:14:070
29725040,0irq/27-eth0-tx-0-21swapper/007:44:050
29725040,0irq/27-eth0-tx-0-21swapper/007:39:070
29725040,0irq/27-eth0-tx-0-21swapper/007:30:400
29715040,0irq/26-eth0-rx-6497-21kthreadcore12:24:083
29715040,0irq/26-eth0-rx-0-21swapper/312:02:343
29715040,0irq/26-eth0-rx-0-21swapper/311:14:073
29715040,0irq/26-eth0-rx-0-21swapper/310:53:553
29715040,0irq/26-eth0-rx-0-21swapper/310:19:013
29715040,0irq/26-eth0-rx-0-21swapper/309:29:363
29715040,0irq/26-eth0-rx-0-21swapper/307:39:133
29715040,0irq/26-eth0-rx-0-21swapper/307:14:053
259940,0migration/27439-21kthreadcore08:14:102
259940,0migration/228080-21runrttasks07:43:562
259940,0migration/225099-21kthreadcore08:04:102
259940,0migration/221472-21kthreadcore08:49:092
259940,0migration/216611-21kthreadcore09:09:072
259940,0migration/213833-21runrttasks07:08:542
259940,0migration/21255-21idleruntime06:59:062
259940,0migration/211358-21kernelversion08:19:072
259940,0migration/211185-21kthreadcore07:04:092
259940,0migration/21112-21kthreadcore08:09:102
249940,0watchdog/226770-21grep07:39:112
249940,0watchdog/224043-21kthreadcore11:14:102
249940,0watchdog/212562-21munin-node11:49:062
249940,0watchdog/20-21swapper/207:19:072
19140,0ktimersoftd/10-21swapper/109:05:301
179940,0migration/19282-21kthreadcore10:24:091
179940,0migration/13568-21kthreadcore07:24:091
179940,0migration/132413-21latency07:19:081
179940,0migration/131795-21kthreadcore08:34:081
179940,0migration/130228-21kthreadcore11:39:071
179940,0migration/130071-21kthreadcore07:44:091
179940,0migration/128353-21kthreadcore08:29:081
179940,0migration/122839-21fschecks_count09:54:041
179940,0migration/117755-21kthreadcore10:49:081
179940,0migration/116224-21kthreadcore07:09:091
179940,0migration/111508-21kthreadcore07:54:081
179940,0migration/110230-21kthreadcore07:29:101
169940,0watchdog/15251-21ssh10:02:301
169940,0watchdog/13893-21ssh10:40:221
169940,0watchdog/124042-21kthreadcore11:14:101
169940,0watchdog/1204622sleep107:13:551
169940,0watchdog/113191-21users07:04:151
149940,0migration/07638-21fschecks_count07:04:050
149940,0migration/04124-21sh10:20:100
149940,0migration/024042-21taskset08:53:380
149940,0migration/023519-21kthreadcore10:14:090
149940,0migration/021201-21sh09:51:190
149940,0migration/015646-21kthreadcore12:09:070
149940,0migration/012309-21runrttasks10:46:220
149940,0migration/010138-21kthreadcore09:24:090
339930,0migration/35825-21missed_timers06:59:103
326399932,0cyclictest9977-21sensors08:39:132
326399930,0cyclictest0-21swapper/208:58:242
326389932,0cyclictest3556-21lldpd08:02:501
326389932,0cyclictest3514-21snmpd08:05:451
326389932,0cyclictest2849-21sensors10:59:131
326379933,0cyclictest3556-21lldpd11:09:210
326379932,0cyclictest7233-21awk09:44:060
326379932,0cyclictest3128-21swap10:39:130
326379932,0cyclictest29176-21sadc10:58:520
326379932,0cyclictest21350-21awk07:14:080
326379930,0cyclictest27152-21munin-node07:19:050
29725030,0irq/27-eth0-tx-0-21swapper/008:54:020
29725030,0irq/27-eth0-tx-0-21swapper/008:29:020
29715030,0irq/26-eth0-rx-0-21swapper/308:39:023
29715030,0irq/26-eth0-rx-0-21swapper/308:14:053
29715030,0irq/26-eth0-rx-0-21swapper/307:59:043
29715030,0irq/26-eth0-rx-0-21swapper/307:19:133
29715030,0irq/26-eth0-rx-0-21swapper/307:09:033
249930,0watchdog/20-21swapper/208:24:062
249930,0watchdog/20-21swapper/207:29:112
249930,0watchdog/20-21swapper/207:13:552
169930,0watchdog/10-21swapper/112:09:021
169930,0watchdog/10-21swapper/111:33:581
169930,0watchdog/10-21swapper/110:20:261
169930,0watchdog/10-21swapper/109:44:181
326399920,0cyclictest29912-21cat08:33:522
326379920,0cyclictest0-21swapper/011:13:550
326379920,0cyclictest0-21swapper/010:04:010
29715020,0irq/26-eth0-rx-0-21swapper/308:24:463
29715020,0irq/26-eth0-rx-0-21swapper/307:29:083
249920,0watchdog/20-21swapper/207:58:592
169920,0watchdog/10-21swapper/108:38:541
169920,0watchdog/10-21swapper/108:24:021
169920,0watchdog/10-21swapper/107:33:591
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional