You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-25 - 15:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Thu Dec 25, 2025 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450650,0irq/25-eth10-21swapper/307:07:003
319524637,0sleep20-21swapper/207:05:352
338023823,0sleep00-21swapper/007:07:340
309322925,0sleep10-21swapper/107:04:221
339970,0migration/313789-21sh09:51:243
35929966,0cyclictest0-21swapper/212:18:492
35889966,0cyclictest1896-21diskmemload09:11:151
35889965,0cyclictest21215-21sort08:34:041
35889960,0cyclictest0-21swapper/111:03:111
35889960,0cyclictest0-21swapper/107:19:091
35879960,0cyclictest1896-21diskmemload10:59:010
35879960,0cyclictest0-21swapper/010:21:490
339960,0migration/39364-21sh09:11:513
339960,0migration/37603-21kthreadcore11:44:073
339960,0migration/37193-21sh10:24:243
339960,0migration/35525-21sh11:59:423
339960,0migration/34740-21sh09:44:563
339960,0migration/331800-21sh12:17:093
339960,0migration/329505-21sh09:41:523
339960,0migration/329306-21irqcore09:24:063
339960,0migration/328727-21irqrtprio10:59:073
339960,0migration/326934-21munin-node08:39:023
339960,0migration/325790-21aten_r1power_cu12:34:003
339960,0migration/321057-21sh09:37:263
29715060,0irq/26-eth0-rx-26152-21irqstats10:19:083
8215050,0irq/24-0000:00:26634-21munin-run08:38:502
4150,0ktimersoftd/00-21swapper/009:15:170
35929950,0cyclictest0-21swapper/212:00:132
35929950,0cyclictest0-21swapper/211:40:302
35889955,0cyclictest0-21swapper/112:34:091
35889955,0cyclictest0-21swapper/111:42:551
35879955,0cyclictest0-21swapper/009:48:430
35879955,0cyclictest0-21swapper/007:24:080
35879950,0cyclictest0-21swapper/010:35:130
339950,0migration/37170-21kthreadcore11:04:073
339950,0migration/3533-21kthreadcore07:29:093
339950,0migration/35117-21missed_timers10:04:103
339950,0migration/330365-21pmu-power07:49:103
339950,0migration/330365-21pmu-power07:49:103
339950,0migration/327334-21kthreadcore07:24:073
339950,0migration/327269-21kthreadcore08:14:073
339950,0migration/325055-21timerandwakeup10:54:133
339950,0migration/324597-21runrttasks08:09:093
339950,0migration/323130-21diskmemload10:15:283
339950,0migration/321337-21munin-node11:14:053
339950,0migration/321159-21taskset08:58:573
339950,0migration/317439-21aten_r1power_fr12:29:023
339950,0migration/31652-21fschecks_time08:44:033
339950,0migration/316260-21kthreadcore08:04:063
339950,0migration/314802-21aten_r1power_po11:49:023
339950,0migration/312165-21taskset08:02:333
339950,0migration/310431-21latency_hist07:13:513
29725050,0irq/27-eth0-tx-9027-21kthreadcore09:49:062
29725050,0irq/27-eth0-tx-8951-21munin-node12:24:052
29725050,0irq/27-eth0-tx-3804-21munin-node07:09:052
29725050,0irq/27-eth0-tx-30673-21munin-node09:44:022
29725050,0irq/27-eth0-tx-27853-21munin-node09:04:022
29725050,0irq/27-eth0-tx-25845-21munin-node08:14:082
29725050,0irq/27-eth0-tx-23166-21kthreadcore11:14:072
29725050,0irq/27-eth0-tx-22991-21kthreadcore10:34:062
29725050,0irq/27-eth0-tx-18150-21munin-node07:44:042
29725050,0irq/27-eth0-tx-14488-21munin-node08:54:072
29725050,0irq/27-eth0-tx-12033-21munin-node11:09:062
29725050,0irq/27-eth0-tx-11503-21if_err_eth010:49:032
29725050,0irq/27-eth0-tx-11497-21munin-node07:39:082
29725050,0irq/27-eth0-tx-0-21swapper/212:11:092
29725050,0irq/27-eth0-tx-0-21swapper/212:07:322
29725050,0irq/27-eth0-tx-0-21swapper/211:34:352
29725050,0irq/27-eth0-tx-0-21swapper/211:28:592
29725050,0irq/27-eth0-tx-0-21swapper/211:24:422
29725050,0irq/27-eth0-tx-0-21swapper/211:19:062
29725050,0irq/27-eth0-tx-0-21swapper/210:28:282
29725050,0irq/27-eth0-tx-0-21swapper/210:14:552
29725050,0irq/27-eth0-tx-0-21swapper/210:11:522
29725050,0irq/27-eth0-tx-0-21swapper/209:14:062
29715050,0irq/26-eth0-rx-9285-21irqrtprio10:09:073
29715050,0irq/26-eth0-rx-227242sleep312:10:323
29715050,0irq/26-eth0-rx-21309-21kthreadcore10:34:063
29715050,0irq/26-eth0-rx-20068-21ssh11:30:303
29715050,0irq/26-eth0-rx-16491-21kthreadcore11:09:063
29715050,0irq/26-eth0-rx-16490-21grep07:39:063
29715050,0irq/26-eth0-rx-15985-21munin-node09:54:043
29715050,0irq/26-eth0-rx-11174-21tune2fs09:14:033
29715050,0irq/26-eth0-rx-0-21swapper/312:24:043
29715050,0irq/26-eth0-rx-0-21swapper/312:18:583
29715050,0irq/26-eth0-rx-0-21swapper/312:04:023
29715050,0irq/26-eth0-rx-0-21swapper/311:57:443
29715050,0irq/26-eth0-rx-0-21swapper/311:21:323
29715050,0irq/26-eth0-rx-0-21swapper/310:51:103
29715050,0irq/26-eth0-rx-0-21swapper/310:44:543
259950,0migration/231600-21sh10:20:392
259950,0migration/229573-21kthreadcore08:39:072
259950,0migration/222998-21latency_hist11:53:502
259950,0migration/220276-21sh09:19:042
259950,0migration/218998-21latency_hist10:53:502
259950,0migration/21775-21timerwakeupswit09:24:132
259950,0migration/21624-21kthreadcore08:19:092
259950,0migration/214326-21munin-node09:34:052
259950,0migration/21237-21munin-node08:44:022
259950,0migration/210261-21kthreadcore11:44:082
249950,0watchdog/226181-21basename10:39:002
179950,0migration/14534-21kthreadcore10:04:091
179950,0migration/132311-21kthreadcore09:24:081
179950,0migration/125651-21taskset09:19:141
179950,0migration/122141-21aten_r1power_po09:39:011
179950,0migration/121391-21kthreadcore10:54:071
149950,0migration/08229-21fschecks_time12:04:020
149950,0migration/05745-21hwlatdetect11:24:020
149950,0migration/04802-21kthreadcore09:09:070
149950,0migration/03581-21kthreadcore07:54:060
149950,0migration/032549-21runrttasks11:59:060
149950,0migration/031174-21kthreadcore09:04:060
149950,0migration/031155-21kthreadcore11:19:060
149950,0migration/030460-21latency_hist09:43:500
149950,0migration/02649-21fschecks_count10:44:030
149950,0migration/026125-21kthreadcore11:34:060
149950,0migration/024016-21kthreadcore09:19:070
149950,0migration/021498-21sh11:50:450
149950,0migration/019835-21kthreadcore10:14:060
149950,0migration/019258-21kthreadcore09:54:070
149950,0migration/018425-21sh10:52:560
149950,0migration/017896-21ksmtuned07:43:410
149950,0migration/016768-21kthreadcore11:29:070
149950,0migration/013725-21kthreadcore10:29:090
149950,0migration/013674-21timerwakeupswit08:49:130
149950,0migration/011707-21missed_timers11:44:090
149950,0migration/01159-21taskset12:38:060
149950,0migration/010853-21kthreadcore09:49:070
8215040,0irq/24-0000:00:0-21swapper/209:53:502
35929943,0cyclictest5789-21awk07:34:052
35929943,0cyclictest3514-21snmpd08:24:082
35929943,0cyclictest3514-21snmpd08:05:382
35929943,0cyclictest32457-21awk07:54:042
35929943,0cyclictest3241-21irqbalance12:34:182
35929943,0cyclictest3241-21irqbalance11:03:582
35929943,0cyclictest3241-21irqbalance08:49:092
35929943,0cyclictest3241-21irqbalance08:29:092
35929943,0cyclictest31484-21awk07:29:082
35929943,0cyclictest30269-21turbostat.cron07:28:512
35889943,0cyclictest8741-21awk08:49:051
35889943,0cyclictest7193-21awk11:44:051
35889943,0cyclictest5670-21awk07:34:051
35889943,0cyclictest3556-21lldpd11:08:591
35889943,0cyclictest3556-21lldpd10:40:291
35889943,0cyclictest3514-21snmpd12:22:091
35889943,0cyclictest3514-21snmpd12:17:091
35889943,0cyclictest3514-21snmpd12:09:091
35889943,0cyclictest3514-21snmpd11:05:091
35889943,0cyclictest3514-21snmpd10:48:091
35889943,0cyclictest3514-21snmpd07:59:081
35889943,0cyclictest3514-21snmpd07:48:231
35889943,0cyclictest24029-21awk11:54:031
35889943,0cyclictest22070-21awk08:59:041
35889943,0cyclictest1771-21awk10:24:041
35889943,0cyclictest15736-21irqcore11:49:061
35889943,0cyclictest13423-21awk08:04:041
35889943,0cyclictest11643-21awk10:49:041
35879943,0cyclictest3556-21lldpd12:31:300
35879943,0cyclictest3556-21lldpd08:33:580
35879943,0cyclictest25760-21awk07:49:050
35879943,0cyclictest25760-21awk07:49:050
35879943,0cyclictest24597-21sadc07:48:510
35879943,0cyclictest2027-21awk08:44:050
35879943,0cyclictest15444-21awk08:54:060
35879943,0cyclictest11402-21awk07:14:050
339940,0migration/36322-21kthreadcore07:09:063
339940,0migration/324372-21taskset07:46:203
339940,0migration/324163-21kthreadcore08:34:063
339940,0migration/319990-21perf08:33:503
339940,0migration/319839-21kthreadcore07:19:093
339940,0migration/312713-21timerwakeupswit08:24:123
29725040,0irq/27-eth0-tx-2161-21munin-node09:09:062
29725040,0irq/27-eth0-tx-14652-21ssh11:48:552
29725040,0irq/27-eth0-tx-1279-21ssh12:19:052
29725040,0irq/27-eth0-tx-0-21swapper/212:28:552
29725040,0irq/27-eth0-tx-0-21swapper/210:44:192
29725040,0irq/27-eth0-tx-0-21swapper/210:04:192
29725040,0irq/27-eth0-tx-0-21swapper/209:38:572
29725040,0irq/27-eth0-tx-0-21swapper/209:28:512
29715040,0irq/26-eth0-rx-0-21swapper/311:39:013
29715040,0irq/26-eth0-rx-0-21swapper/311:34:053
29715040,0irq/26-eth0-rx-0-21swapper/311:24:273
29715040,0irq/26-eth0-rx-0-21swapper/310:39:003
29715040,0irq/26-eth0-rx-0-21swapper/310:29:113
29715040,0irq/26-eth0-rx-0-21swapper/310:01:093
29715040,0irq/26-eth0-rx-0-21swapper/309:29:033
29715040,0irq/26-eth0-rx-0-21swapper/309:18:503
29715040,0irq/26-eth0-rx-0-21swapper/308:54:023
29715040,0irq/26-eth0-rx-0-21swapper/308:49:083
259940,0migration/29231-21latency_hist10:28:502
259940,0migration/224534-21kthreadcore08:59:072
259940,0migration/219584-21munin-node08:09:032
249940,0watchdog/226896-21kthreadcore09:59:082
249940,0watchdog/220177-21kthreadcore07:19:092
249940,0watchdog/20-21swapper/210:59:082
179940,0migration/19557-21kthreadcore09:49:071
179940,0migration/19341-21kthreadcore12:04:051
179940,0migration/15444-21kthreadcore07:09:061
179940,0migration/15395-21kthreadcore08:44:071
179940,0migration/132466-21kthreadcore09:04:061
179940,0migration/130955-21sshd07:50:131
179940,0migration/130955-21sshd07:50:131
179940,0migration/129363-21sendmail_mailst09:59:121
179940,0migration/129236-21df_abs11:19:021
179940,0migration/123304-21kthreadcore11:34:051
179940,0migration/119733-21sh11:29:421
179940,0migration/118400-21munin-node12:29:071
179940,0migration/116810-21kthreadcore08:54:081
179940,0migration/115533-21kthreadcore09:14:081
179940,0migration/110696-21kernelversion10:29:081
179940,0migration/110454-21needreboot11:24:071
169940,0watchdog/15188-21ssh09:46:001
169940,0watchdog/124729-21munin-node07:24:051
169940,0watchdog/124721-21kthreadcore11:14:081
169940,0watchdog/124505-21ssh10:18:441
169940,0watchdog/117705-21ssh10:33:561
169940,0watchdog/115946-21chrt12:26:121
169940,0watchdog/112304-21ssh09:32:161
169940,0watchdog/111239-21kthreadcore10:09:081
169940,0watchdog/10-21swapper/109:34:081
169940,0watchdog/10-21swapper/108:39:051
149940,0migration/05912-21kthreadcore10:24:060
149940,0migration/05302-21kthreadcore11:04:060
149940,0migration/04461-21fschecks_time07:09:030
149940,0migration/032419-21kthreadcore09:24:080
149940,0migration/03009-21kthreadcore07:29:100
149940,0migration/027456-21kthreadcore11:54:060
149940,0migration/027024-21cpu08:39:020
149940,0migration/025371-21if_eth112:14:040
149940,0migration/024215-21kthreadcore08:59:060
149940,0migration/023630-21kthreadcore11:14:080
149940,0migration/018573-21kthreadcore08:29:070
149940,0migration/016046-21sh10:13:270
149940,0migration/012751-21cpuspeed_turbos08:04:010
149940,0migration/011509-21kthreadcore08:24:070
149940,0migration/010010-21kthreadcore07:59:090
35929932,0cyclictest3241-21irqbalance07:17:492
35929932,0cyclictest25736-21awk07:49:052
35929932,0cyclictest25736-21awk07:49:052
35889932,0cyclictest3514-21snmpd10:20:081
35889932,0cyclictest3514-21snmpd08:16:421
35889932,0cyclictest31414-21sort11:58:501
35889932,0cyclictest2130-21cut07:29:101
35889932,0cyclictest16040-21awk09:54:041
35889932,0cyclictest11346-21awk07:14:051
35889930,0cyclictest0-21swapper/107:54:471
35879933,0cyclictest28456-21kworker/0:009:59:130
35879933,0cyclictest28456-21kworker/0:008:13:540
35879933,0cyclictest28456-21kworker/0:008:08:560
35879933,0cyclictest28456-21kworker/0:007:33:560
35879933,0cyclictest28456-21kworker/0:007:18:540
35879933,0cyclictest11451-21kworker/0:012:19:030
35879933,0cyclictest11451-21kworker/0:012:08:530
35879933,0cyclictest11451-21kworker/0:011:38:590
35879933,0cyclictest11451-21kworker/0:011:09:270
35879933,0cyclictest11451-21kworker/0:010:54:310
35879933,0cyclictest0-21swapper/009:33:530
35879932,0cyclictest3556-21lldpd09:28:580
35879932,0cyclictest31908-21sed10:03:500
35879932,0cyclictest31896-21sensors10:39:100
35879932,0cyclictest28456-21kworker/0:008:18:520
35879930,0cyclictest9667-21perl12:24:040
29725030,0irq/27-eth0-tx-0-21swapper/207:59:032
29715030,0irq/26-eth0-rx-0-21swapper/309:04:013
29715030,0irq/26-eth0-rx-0-21swapper/307:56:493
29715030,0irq/26-eth0-rx-0-21swapper/307:33:553
169930,0watchdog/119193-21munin-node08:09:131
169930,0watchdog/10-21swapper/108:24:131
169930,0watchdog/10-21swapper/108:22:051
35889920,0cyclictest13503-21latency_hist08:28:501
29715020,0irq/26-eth0-rx-0-21swapper/308:18:523
169920,0watchdog/10-21swapper/107:38:531
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional