You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-15 - 14:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Tue Sep 15, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1010323726,0sleep30-21swapper/306:41:043
1026423425,0sleep10-21swapper/106:42:541
1020123423,0sleep00-21swapper/006:42:130
1005323228,0sleep20-21swapper/206:40:272
10415991212,0cyclictest0-21swapper/008:45:190
339960,0migration/38006-21fschecks_count09:44:023
339960,0migration/35928-21munin-node10:44:033
339960,0migration/330957-21sh09:58:273
339960,0migration/324168-21sh08:52:573
339960,0migration/314627-21sh10:08:453
339960,0migration/310045-21munin-node11:24:103
149960,0migration/018939-21needreboot07:59:100
149960,0migration/015928-21sh09:07:580
104189965,0cyclictest7246-21crond09:43:502
104189960,0cyclictest0-21swapper/211:50:512
104169965,0cyclictest0-21swapper/110:48:521
104159965,0cyclictest7134-21ssh09:43:300
8215050,0irq/24-0000:00:4192-21python10:19:112
8215050,0irq/24-0000:00:25678-21latency_hist07:43:502
8215050,0irq/24-0000:00:16650-21cat08:48:502
8215050,0irq/24-0000:00:12500-21cat07:33:512
4150,0ktimersoftd/00-21swapper/012:10:430
4150,0ktimersoftd/00-21swapper/011:20:460
35150,0ktimersoftd/30-21swapper/311:01:443
339950,0migration/3967-21needreboot11:59:073
339950,0migration/39329-21kthreadcore07:29:103
339950,0migration/37095-21taskset09:00:383
339950,0migration/35242-21sh10:43:043
339950,0migration/33583-21munin-node12:04:033
339950,0migration/331921-21taskset09:18:243
339950,0migration/330932-21kthreadcore08:09:093
339950,0migration/330466-21runrttasks09:35:223
339950,0migration/327178-21taskset08:29:443
339950,0migration/32500-21kthreadcore09:59:073
339950,0migration/324496-21kthreadcore11:34:073
339950,0migration/321222-21irqcore08:04:083
339950,0migration/319196-21munin-node10:49:093
339950,0migration/318400-21irqcore06:49:063
339950,0migration/318214-21latency_hist07:13:513
339950,0migration/316939-21kthreadcore11:29:073
339950,0migration/31599-21kthreadcore07:24:073
339950,0migration/315237-21taskset09:05:513
339950,0migration/315204-21kthreadcore07:34:083
339950,0migration/313848-21kthreadcore06:44:083
339950,0migration/310781-21runrttasks11:45:483
29725050,0irq/27-eth0-tx-4882-21basename11:44:082
29725050,0irq/27-eth0-tx-20683-21ssh11:13:232
29725050,0irq/27-eth0-tx-14369-21munin-node10:29:082
29725050,0irq/27-eth0-tx-11878-21diskstats12:09:022
29725050,0irq/27-eth0-tx-0-21swapper/211:59:412
29725050,0irq/27-eth0-tx-0-21swapper/210:44:582
29725050,0irq/27-eth0-tx-0-21swapper/210:24:092
29725050,0irq/27-eth0-tx-0-21swapper/210:08:342
29725050,0irq/27-eth0-tx-0-21swapper/209:52:202
29725050,0irq/27-eth0-tx-0-21swapper/209:04:072
29725050,0irq/27-eth0-tx-0-21swapper/209:03:072
29725050,0irq/27-eth0-tx-0-21swapper/208:45:592
29725050,0irq/27-eth0-tx-0-21swapper/007:15:430
29725050,0irq/27-eth0-tx-0-21swapper/007:10:420
29715050,0irq/26-eth0-rx-28570-21sh11:39:013
29715050,0irq/26-eth0-rx-28360-21kthreadcore08:54:063
29715050,0irq/26-eth0-rx-28193-21sh11:17:073
29715050,0irq/26-eth0-rx-27244-21kthreadcore06:54:093
29715050,0irq/26-eth0-rx-20077-21aten2.4-expect11:54:013
29715050,0irq/26-eth0-rx-18167-21kthreadcore11:09:083
29715050,0irq/26-eth0-rx-18144-21users11:49:133
29715050,0irq/26-eth0-rx-16461-21iostat_ios09:49:053
29715050,0irq/26-eth0-rx-14937-21munin-node10:09:033
29715050,0irq/26-eth0-rx-0-21swapper/312:08:493
29715050,0irq/26-eth0-rx-0-21swapper/310:58:273
29715050,0irq/26-eth0-rx-0-21swapper/310:34:103
29715050,0irq/26-eth0-rx-0-21swapper/310:24:513
29715050,0irq/26-eth0-rx-0-21swapper/309:19:063
29715050,0irq/26-eth0-rx-0-21swapper/309:08:553
259950,0migration/24726-21munin-node09:59:112
259950,0migration/227012-21timerandwakeup11:14:122
259950,0migration/224480-21aten_r1power_fr09:14:012
259950,0migration/222894-21sh09:10:102
259950,0migration/219375-21pmu-power10:49:112
259950,0migration/213019-21munin-node09:44:082
249950,0watchdog/22897-21kthreadcore10:39:082
249950,0watchdog/224355-21fschecks_count06:54:052
249950,0watchdog/20-21swapper/211:29:442
249950,0watchdog/20-21swapper/209:19:362
249950,0watchdog/20-21swapper/208:04:092
19150,0ktimersoftd/10-21swapper/112:10:431
19150,0ktimersoftd/10-21swapper/111:46:441
179950,0migration/15779-21taskset09:39:291
179950,0migration/130326-21ntp_offset07:19:071
179950,0migration/127655-21munin-node10:34:091
179950,0migration/125962-21timerwakeupswit11:54:121
179950,0migration/125408-21kthreadcore11:14:071
179950,0migration/124020-21kthreadcore11:34:071
179950,0migration/121723-21taskset10:33:031
179950,0migration/11282-21smart_sdb11:59:101
179950,0migration/11064-21kthreadcore06:59:071
169950,0watchdog/14519-21missed_timers07:24:091
169950,0watchdog/131268-21needreboot07:44:091
149950,0migration/08411-21taskset08:42:060
149950,0migration/04801-21kthreadcore08:59:070
149950,0migration/032667-21munin-node09:19:010
149950,0migration/031819-21kthreadcore08:34:080
149950,0migration/02862-21kthreadcore08:14:070
149950,0migration/025723-21idleruntime07:19:020
149950,0migration/025668-21kthreadcore11:14:070
149950,0migration/024981-21kthreadcore10:54:070
149950,0migration/024181-21cpuspeed_turbos09:34:030
149950,0migration/023407-21taskset08:50:510
149950,0migration/022592-21kthreadcore08:04:090
149950,0migration/020811-21taskset08:28:050
149950,0migration/020211-21kthreadcore09:09:080
149950,0migration/019240-21kthreadcore06:49:070
149950,0migration/014782-21munin-node10:29:030
149950,0migration/014255-21kthreadcore11:49:060
149950,0migration/012705-21taskset11:08:170
149950,0migration/01099-21kthreadcore07:24:070
149950,0migration/010761-21kthreadcore07:54:100
149950,0migration/010133-21sh12:05:350
104189955,0cyclictest0-21swapper/211:57:092
104189955,0cyclictest0-21swapper/207:19:122
104189950,0cyclictest0-21swapper/211:06:422
104189950,0cyclictest0-21swapper/210:54:152
104169955,0cyclictest0-21swapper/111:24:021
104169955,0cyclictest0-21swapper/110:17:201
104169955,0cyclictest0-21swapper/109:00:521
104169954,0cyclictest3514-21snmpd11:08:091
104169950,0cyclictest0-21swapper/111:11:341
104169950,0cyclictest0-21swapper/110:04:341
104169950,0cyclictest0-21swapper/110:03:091
104159955,0cyclictest0-21swapper/009:48:130
104159950,0cyclictest0-21swapper/011:40:290
104159950,0cyclictest0-21swapper/010:11:520
104159950,0cyclictest0-21swapper/009:28:590
339940,0migration/39671-21kthreadcore07:04:073
339940,0migration/37819-21taskset08:18:543
339940,0migration/37520-21kthreadcore11:04:093
339940,0migration/34793-21latency_hist07:03:513
339940,0migration/330943-21taskset07:19:463
339940,0migration/322808-21latency_hist10:13:493
339940,0migration/317582-21kthreadcore10:29:083
339940,0migration/317006-21iostat09:29:083
339940,0migration/313903-21kthreadcore07:09:083
29725040,0irq/27-eth0-tx-28634-21telnet11:39:012
29725040,0irq/27-eth0-tx-0-21swapper/212:04:042
29725040,0irq/27-eth0-tx-0-21swapper/211:34:022
29725040,0irq/27-eth0-tx-0-21swapper/211:25:062
29725040,0irq/27-eth0-tx-0-21swapper/211:19:072
29725040,0irq/27-eth0-tx-0-21swapper/210:08:532
29725040,0irq/27-eth0-tx-0-21swapper/209:23:552
29725040,0irq/27-eth0-tx-0-21swapper/208:53:512
29725040,0irq/27-eth0-tx-0-21swapper/007:04:060
29715040,0irq/26-eth0-rx-30365-21kthreadcore11:19:073
29715040,0irq/26-eth0-rx-0-21swapper/310:18:533
29715040,0irq/26-eth0-rx-0-21swapper/309:41:013
29715040,0irq/26-eth0-rx-0-21swapper/309:26:063
29715040,0irq/26-eth0-rx-0-21swapper/308:46:093
29715040,0irq/26-eth0-rx-0-21swapper/307:39:033
259940,0migration/26822-21latency_hist07:53:512
259940,0migration/228163-21kthreadcore10:14:072
259940,0migration/224221-21kthreadcore10:34:072
259940,0migration/223897-21kthreadcore08:29:092
259940,0migration/218857-21kthreadcore06:49:072
259940,0migration/218618-21kthreadcore09:29:092
259940,0migration/218389-21kthreadcore08:24:082
259940,0migration/213278-21ntp_offset08:19:092
249940,0watchdog/24096-21munin-node06:59:132
249940,0watchdog/228395-21kthreadcore08:09:092
249940,0watchdog/20-21swapper/208:34:082
179940,0migration/15705-21latency_hist10:43:491
179940,0migration/131611-21kthreadcore11:39:071
179940,0migration/128947-21kthreadcore09:14:071
179940,0migration/126946-21aten_r1power_fr08:09:031
179940,0migration/122437-21gltestperf10:54:031
179940,0migration/121768-21munin-node09:29:141
179940,0migration/120285-21timerwakeupswit08:24:131
179940,0migration/119837-21latency_hist11:53:491
179940,0migration/113731-21irqcore11:29:051
179940,0migration/111541-21kthreadcore10:24:071
169940,0watchdog/128245-21kthreadcore09:34:081
169940,0watchdog/127146-21kthreadcore06:54:091
169940,0watchdog/121821-21kthreadcore08:04:091
169940,0watchdog/119622-21kthreadcore09:09:081
169940,0watchdog/117517-21tr07:59:091
169940,0watchdog/114737-21ssh09:04:241
169940,0watchdog/112511-21kthreadcore08:44:081
169940,0watchdog/10-21swapper/111:19:081
149940,0migration/09119-21kthreadcore10:24:060
149940,0migration/08532-21kthreadcore10:04:060
149940,0migration/0690-21kthreadcore10:19:060
149940,0migration/030772-21fschecks_time10:39:030
149940,0migration/029642-21kthreadcore11:59:050
149940,0migration/01826-21kthreadcore09:59:070
104189943,0cyclictest3556-21lldpd07:47:322
104189943,0cyclictest30502-21awk10:59:042
104189943,0cyclictest24772-21awk09:34:052
104189943,0cyclictest24312-21awk09:54:052
104189943,0cyclictest12329-21sadc07:33:502
104189943,0cyclictest11584-21awk06:44:072
104169943,0cyclictest3556-21lldpd09:24:021
104169943,0cyclictest3556-21lldpd09:19:021
104169943,0cyclictest3556-21lldpd07:39:021
104169943,0cyclictest3514-21snmpd07:53:531
104169943,0cyclictest3514-21snmpd07:29:531
104169943,0cyclictest3241-21irqbalance12:04:581
104169943,0cyclictest3241-21irqbalance10:09:481
104169943,0cyclictest3241-21irqbalance09:43:581
104169943,0cyclictest3241-21irqbalance08:48:591
104169943,0cyclictest3241-21irqbalance07:09:091
104169943,0cyclictest31707-21awk10:19:041
104169943,0cyclictest30486-21awk10:59:041
104169943,0cyclictest28611-21awk08:34:051
104169943,0cyclictest25645-21awk08:54:031
104169943,0cyclictest13493-21awk07:34:061
104169943,0cyclictest11608-21awk06:44:071
104169943,0cyclictest1099-21awk07:49:061
104159944,0cyclictest0-21swapper/010:14:430
104159943,0cyclictest6857-21awk07:29:080
104159943,0cyclictest6755-21awk10:44:060
104159943,0cyclictest3556-21lldpd09:56:330
104159943,0cyclictest3241-21irqbalance11:28:580
104159943,0cyclictest3241-21irqbalance11:26:580
104159943,0cyclictest27819-21awk08:09:070
104159943,0cyclictest23136-21awk10:34:050
104159943,0cyclictest16387-21awk09:49:040
104159943,0cyclictest12563-21munin-node07:34:040
339930,0migration/315029-21kthreadcore07:59:083
29725030,0irq/27-eth0-tx-0-21swapper/007:39:030
29715030,0irq/26-eth0-rx-0-21swapper/308:40:163
29715030,0irq/26-eth0-rx-0-21swapper/308:34:033
29715030,0irq/26-eth0-rx-0-21swapper/308:24:113
29715030,0irq/26-eth0-rx-0-21swapper/308:14:023
29715030,0irq/26-eth0-rx-0-21swapper/307:54:023
29715030,0irq/26-eth0-rx-0-21swapper/307:49:153
29715030,0irq/26-eth0-rx-0-21swapper/307:44:093
249930,0watchdog/2323382chrt07:24:052
249930,0watchdog/20-21swapper/207:09:092
179930,0migration/132246-21kthreadcore10:39:071
179930,0migration/127560-21kthreadcore09:54:081
179930,0migration/122220-21irqcore08:29:081
169930,0watchdog/16201-21cut08:14:081
104189932,0cyclictest4110-21kthreadcore07:49:082
104189932,0cyclictest3586-21grep08:39:062
104189932,0cyclictest1982-21awk08:14:062
104189932,0cyclictest18028-21latency_hist07:13:512
104189930,0cyclictest19358-21sed07:59:142
104169933,0cyclictest5865-21awk07:04:051
104169933,0cyclictest3556-21lldpd08:19:021
104169932,0cyclictest3241-21irqbalance09:48:581
104169932,0cyclictest3241-21irqbalance07:14:191
104169932,0cyclictest20645-21kthreadcore06:49:071
104169932,0cyclictest2003-21cat08:38:501
104159933,0cyclictest30990-21kworker/0:011:53:570
104159933,0cyclictest30990-21kworker/0:011:43:510
104159933,0cyclictest30990-21kworker/0:011:34:010
104159933,0cyclictest30990-21kworker/0:010:58:510
104159933,0cyclictest27638-21kworker/0:109:23:590
104159933,0cyclictest27638-21kworker/0:109:14:190
104159933,0cyclictest27638-21kworker/0:108:54:060
104159933,0cyclictest27638-21kworker/0:108:29:000
104159933,0cyclictest27638-21kworker/0:108:18:580
104159933,0cyclictest20449-21kworker/0:007:48:540
104159933,0cyclictest20449-21kworker/0:007:43:520
104159933,0cyclictest20449-21kworker/0:006:58:560
104159933,0cyclictest20449-21kworker/0:006:53:540
104159933,0cyclictest20449-21kworker/0:006:43:540
104159932,0cyclictest19529-21sensors10:49:120
104159932,0cyclictest18935-21sensors11:09:120
104189921,0cyclictest0-21swapper/207:03:512
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional