You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-26 - 23:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack1slot2s.osadl.org (updated Wed Aug 26, 2026 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
304450910,0irq/25-eth10-21swapper/306:42:093
1328527372,0sleep00-21swapper/006:43:230
1347926968,0sleep10-21swapper/106:45:401
1331722723,0sleep20-21swapper/206:43:452
339970,0migration/330486-21runrttasks12:00:083
339970,0migration/312603-21kthreadcore09:26:013
27170,0ktimersoftd/20-21swapper/209:52:302
339960,0migration/3442-21sh10:40:263
339960,0migration/318438-21taskset08:47:333
29725060,0irq/27-eth0-tx-17671-21munin-node08:26:060
29715060,0irq/26-eth0-rx-25636-21python08:51:063
29715060,0irq/26-eth0-rx-13342-21tr10:26:023
259960,0migration/232606-21fschecks_time11:20:582
149960,0migration/0308-21munin-node09:36:050
149960,0migration/024836-21sh10:13:240
149960,0migration/015129-21kthreadcore08:46:010
149960,0migration/011201-21kthreadcore11:46:020
135639960,0cyclictest0-21swapper/209:10:432
135599965,0cyclictest9917-21sed10:45:591
135589960,0cyclictest0-21swapper/011:59:110
8215050,0irq/24-0000:00:14892-21grep10:46:072
4150,0ktimersoftd/00-21swapper/011:19:180
35150,0ktimersoftd/30-21swapper/309:30:593
339950,0migration/39934-21irqcore10:45:593
339950,0migration/37857-21munin-node10:01:073
339950,0migration/37211-21aten_r1power_en11:45:553
339950,0migration/36015-21diskmemload11:43:093
339950,0migration/34362-21smart_sdb12:01:053
339950,0migration/34050-21kthreadcore11:21:023
339950,0migration/332730-21kthreadcore07:46:023
339950,0migration/328247-21timerwakeupswit07:41:073
339950,0migration/32385-21sh09:19:193
339950,0migration/32255-21kthreadcore08:36:013
339950,0migration/320626-21kthreadcore10:51:013
339950,0migration/318063-21munin-node08:25:573
339950,0migration/310907-21munin-node07:55:593
339950,0migration/310715-21munin-node10:05:583
339950,0migration/310635-21kthreadcore07:31:013
29725050,0irq/27-eth0-tx-9119-21munin-node07:30:570
29725050,0irq/27-eth0-tx-8382-21aten_r1power_vo11:05:542
29725050,0irq/27-eth0-tx-471-21ssh09:57:152
29725050,0irq/27-eth0-tx-3374-21munin-node07:51:010
29725050,0irq/27-eth0-tx-25888-21munin-node10:15:552
29725050,0irq/27-eth0-tx-24479-21if_err_eth1.311:35:572
29725050,0irq/27-eth0-tx-21589-21grep09:11:022
29725050,0irq/27-eth0-tx-1967-21munin-node10:01:042
29725050,0irq/27-eth0-tx-19095-21ssh09:30:532
29725050,0irq/27-eth0-tx-11534-21latency_hist09:05:452
29725050,0irq/27-eth0-tx-0-21swapper/211:58:462
29725050,0irq/27-eth0-tx-0-21swapper/211:51:072
29725050,0irq/27-eth0-tx-0-21swapper/211:46:012
29725050,0irq/27-eth0-tx-0-21swapper/211:10:442
29725050,0irq/27-eth0-tx-0-21swapper/210:32:422
29725050,0irq/27-eth0-tx-0-21swapper/210:09:352
29725050,0irq/27-eth0-tx-0-21swapper/208:56:532
29725050,0irq/27-eth0-tx-0-21swapper/008:01:020
29725050,0irq/27-eth0-tx-0-21swapper/007:26:040
29725050,0irq/27-eth0-tx-0-21swapper/007:17:150
29725050,0irq/27-eth0-tx-0-21swapper/007:07:140
29725050,0irq/27-eth0-tx-0-21swapper/006:45:570
29715050,0irq/26-eth0-rx-9952-21ssh09:01:293
29715050,0irq/26-eth0-rx-7994-21ssh10:43:553
29715050,0irq/26-eth0-rx-7881-21cpu11:25:573
29715050,0irq/26-eth0-rx-7120-21ssh11:03:123
29715050,0irq/26-eth0-rx-29830-21kthreadcore09:56:013
29715050,0irq/26-eth0-rx-24745-21fschecks_time11:15:583
29715050,0irq/26-eth0-rx-19694-21ssh09:10:543
29715050,0irq/26-eth0-rx-16717-21kthreadcore11:51:003
29715050,0irq/26-eth0-rx-15790-21iostat07:11:003
29715050,0irq/26-eth0-rx-15102-21fschecks_time12:10:573
29715050,0irq/26-eth0-rx-11014-21ssh09:45:563
29715050,0irq/26-eth0-rx-0-21swapper/310:15:443
29715050,0irq/26-eth0-rx-0-21swapper/309:42:523
29715050,0irq/26-eth0-rx-0-21swapper/308:58:023
29715050,0irq/26-eth0-rx-0-21swapper/307:21:023
259950,0migration/29433-21sh09:43:002
259950,0migration/227917-21latency_hist07:20:452
259950,0migration/226905-21latency_hist06:55:452
259950,0migration/222415-21latency_hist07:40:452
259950,0migration/210702-21missed_timers08:41:022
259950,0migration/210702-21missed_timers08:41:022
259950,0migration/210078-21aten_r1power_cu07:55:552
249950,0watchdog/225923-21kthreadcore08:31:032
179950,0migration/17752-21aten2.4-expect11:25:561
179950,0migration/14174-21diskmemload12:01:031
179950,0migration/122559-21kthreadcore10:11:021
179950,0migration/117940-21sh09:49:421
179950,0migration/117407-21sh09:26:281
179950,0migration/110699-21sh09:03:421
149950,0migration/09854-21df_abs10:25:570
149950,0migration/0854-21irqrtprio11:21:000
149950,0migration/06311-21kthreadcore08:16:010
149950,0migration/05720-21munin-node08:40:570
149950,0migration/05720-21munin-node08:40:570
149950,0migration/032613-21munin-node06:56:040
149950,0migration/030650-21taskset08:34:320
149950,0migration/026401-21irqcore10:36:010
149950,0migration/025821-21taskset09:32:570
149950,0migration/025066-21kthreadcore11:35:590
149950,0migration/024878-21kthreadcore06:51:020
149950,0migration/022088-21kthreadcore09:11:020
149950,0migration/021847-21kthreadcore10:51:010
149950,0migration/017346-21kthreadcore11:51:000
149950,0migration/011175-21aten_r1power_en09:25:550
149950,0migration/010464-21kthreadcore11:26:020
135639955,0cyclictest0-21swapper/212:11:332
135639955,0cyclictest0-21swapper/210:30:012
135639955,0cyclictest0-21swapper/209:16:332
135639955,0cyclictest0-21swapper/208:53:042
135639955,0cyclictest0-21swapper/208:48:582
135639954,0cyclictest8421-21grep11:25:592
135639950,0cyclictest0-21swapper/210:37:352
135599955,0cyclictest0-21swapper/111:47:511
135599950,0cyclictest0-21swapper/111:53:321
135589955,0cyclictest0-21swapper/007:45:500
135589950,0cyclictest0-21swapper/010:05:500
339940,0migration/39949-21kthreadcore07:06:013
339940,0migration/37517-21kthreadcore07:51:023
339940,0migration/35407-21aten_r1power_fr08:40:553
339940,0migration/35407-21aten_r1power_fr08:40:553
339940,0migration/35157-21kthreadcore10:21:043
339940,0migration/329004-21kthreadcore09:36:013
339940,0migration/322911-21kthreadcore07:16:013
339940,0migration/320058-21kthreadcore11:31:013
339940,0migration/319157-21fschecks_time09:50:563
339940,0migration/318905-21kthreadcore11:11:013
339940,0migration/312886-21kthreadcore08:21:003
29725040,0irq/27-eth0-tx-6529-21munin-node12:06:052
29725040,0irq/27-eth0-tx-32761-21ssh09:36:052
29725040,0irq/27-eth0-tx-32608-21ssh11:00:512
29725040,0irq/27-eth0-tx-24665-21munin-node10:55:572
29725040,0irq/27-eth0-tx-17936-21ssh10:10:502
29725040,0irq/27-eth0-tx-11137-21munin-node09:26:042
29725040,0irq/27-eth0-tx-0-21swapper/211:40:552
29725040,0irq/27-eth0-tx-0-21swapper/211:30:572
29725040,0irq/27-eth0-tx-0-21swapper/210:50:542
29725040,0irq/27-eth0-tx-0-21swapper/209:45:442
29725040,0irq/27-eth0-tx-0-21swapper/209:20:582
29725040,0irq/27-eth0-tx-0-21swapper/007:10:570
29715040,0irq/26-eth0-rx-3392-21cpu09:20:573
29715040,0irq/26-eth0-rx-27176-21cut11:36:003
29715040,0irq/26-eth0-rx-12376-21python12:06:053
29715040,0irq/26-eth0-rx-0-21swapper/311:06:593
29715040,0irq/26-eth0-rx-0-21swapper/310:57:293
29715040,0irq/26-eth0-rx-0-21swapper/310:31:383
29715040,0irq/26-eth0-rx-0-21swapper/310:11:063
29715040,0irq/26-eth0-rx-0-21swapper/309:05:573
29715040,0irq/26-eth0-rx-0-21swapper/308:00:563
29715040,0irq/26-eth0-rx-0-21swapper/307:35:553
29715040,0irq/26-eth0-rx-0-21swapper/307:01:013
29715040,0irq/26-eth0-rx-0-21swapper/306:55:553
29715040,0irq/26-eth0-rx-0-21swapper/306:46:033
259940,0migration/232055-21kthreadcore07:46:022
259940,0migration/230569-21fschecks_count08:10:592
259940,0migration/229690-21runrttasks08:09:202
259940,0migration/226322-21kthreadcore11:16:012
259940,0migration/223408-21switchtime08:26:072
259940,0migration/222151-21runrttasks07:40:112
259940,0migration/217529-21kthreadcore07:11:012
259940,0migration/215151-21runrttasks07:31:182
259940,0migration/214174-21munin-node06:45:582
259940,0migration/212367-21kthreadcore08:21:002
249940,0watchdog/210228-21timerwakeupswit08:16:072
249940,0watchdog/20-21swapper/212:01:062
249940,0watchdog/20-21swapper/210:20:582
179940,0migration/19234-21kthreadcore12:06:001
179940,0migration/16859-21kthreadcore10:01:021
179940,0migration/128843-21kthreadcore10:36:021
179940,0migration/127887-21latency_hist08:55:451
179940,0migration/127844-21kthreadcore10:15:591
179940,0migration/126778-21cpuspeed_turbos09:55:561
179940,0migration/12649-21kthreadcore11:41:011
179940,0migration/124004-21kthreadcore07:41:011
179940,0migration/122331-21sh11:32:451
179940,0migration/121901-21kthreadcore09:31:021
179940,0migration/121243-21kthreadcore09:11:011
179940,0migration/118853-21kthreadcore11:11:011
179940,0migration/118053-21kthreadcore12:11:011
179940,0migration/110183-21fschecks_time10:25:581
169940,0watchdog/16708-21python10:41:061
169940,0watchdog/16341-21kthreadcore09:21:021
169940,0watchdog/127635-21munin-node09:15:591
169940,0watchdog/1220932sleep107:39:311
169940,0watchdog/115808-21sensors10:06:061
169940,0watchdog/111790-21diskmemload11:37:021
169940,0watchdog/10-21swapper/111:21:021
169940,0watchdog/10-21swapper/109:41:021
169940,0watchdog/10-21swapper/108:20:551
149940,0migration/031927-21kthreadcore07:21:020
149940,0migration/02759-21kthreadcore11:41:010
149940,0migration/02744-21sh08:59:130
149940,0migration/026952-21kthreadcore10:56:010
149940,0migration/025354-21kthreadcore08:06:000
149940,0migration/021180-21kthreadcore11:11:020
149940,0migration/014708-21kthreadcore08:21:010
135639943,0cyclictest3556-21lldpd06:58:432
135639943,0cyclictest1889-21awk10:41:002
135639943,0cyclictest17656-21awk08:00:592
135639940,0cyclictest0-21swapper/208:36:012
135599944,0cyclictest0-21swapper/106:50:591
135599943,0cyclictest9006-21awk07:05:591
135599943,0cyclictest4336-21awk07:51:001
135599943,0cyclictest3556-21lldpd08:37:141
135599943,0cyclictest3514-21snmpd11:16:031
135599943,0cyclictest3514-21snmpd08:46:031
135599943,0cyclictest3514-21snmpd08:29:031
135599943,0cyclictest3514-21snmpd08:03:331
135599943,0cyclictest3514-21snmpd07:56:031
135599943,0cyclictest3241-21irqbalance11:08:421
135599943,0cyclictest3241-21irqbalance10:52:121
135599943,0cyclictest31011-21awk08:11:011
135599943,0cyclictest28936-21awk07:20:591
135599943,0cyclictest28018-21awk09:35:591
135599943,0cyclictest27941-21awk06:55:591
135599943,0cyclictest25619-21awk10:55:591
135599943,0cyclictest24284-21awk08:05:591
135599940,0cyclictest0-21swapper/110:33:071
135589944,0cyclictest0-21swapper/012:06:550
135589944,0cyclictest0-21swapper/010:41:280
135589944,0cyclictest0-21swapper/009:51:480
135589943,0cyclictest8746-21kworker/0:012:10:590
135589943,0cyclictest3514-21snmpd07:56:480
135589943,0cyclictest3514-21snmpd07:45:030
135589943,0cyclictest3241-21irqbalance10:49:320
135589943,0cyclictest3241-21irqbalance10:18:320
135589943,0cyclictest30938-21awk08:11:010
135589943,0cyclictest27357-21awk09:55:580
135589943,0cyclictest2495-21awk10:21:020
135589943,0cyclictest18415-21awk10:30:590
135589943,0cyclictest15778-21munin-node07:36:000
8215030,0irq/24-0000:00:0-21swapper/207:05:472
339930,0migration/330782-21munin-run08:35:453
339930,0migration/323412-21aten_r1power_en08:05:553
29715030,0irq/26-eth0-rx-0-21swapper/308:16:073
29715030,0irq/26-eth0-rx-0-21swapper/308:10:573
29715030,0irq/26-eth0-rx-0-21swapper/307:26:163
29715030,0irq/26-eth0-rx-0-21swapper/306:52:113
249930,0watchdog/20-21swapper/207:50:592
249930,0watchdog/20-21swapper/207:29:192
179930,0migration/129722-21sh11:57:481
169930,0watchdog/10-21swapper/110:20:581
169930,0watchdog/10-21swapper/108:41:031
169930,0watchdog/10-21swapper/108:41:031
169930,0watchdog/10-21swapper/107:00:591
135639932,0cyclictest29078-21awk07:21:002
135639932,0cyclictest1444-21aten_r1power_po07:00:562
135599933,0cyclictest0-21swapper/107:16:131
135599932,0cyclictest5339-21awk08:15:591
135599932,0cyclictest3514-21snmpd08:51:181
135599932,0cyclictest3204-21awk07:25:591
135599932,0cyclictest29780-21grep07:45:581
135599932,0cyclictest25219-21awk08:31:011
135599932,0cyclictest19883-21kthreadcore09:51:001
135599932,0cyclictest10050-21awk07:31:001
135599930,0cyclictest647-21munin-node11:00:561
135599930,0cyclictest16098-21kthreadcore07:11:011
135599930,0cyclictest13806-21sh06:45:561
135599930,0cyclictest0-21swapper/109:06:011
135589933,0cyclictest8746-21kworker/0:012:00:490
135589933,0cyclictest8746-21kworker/0:011:30:450
135589933,0cyclictest14384-21kworker/0:111:05:470
135589933,0cyclictest14384-21kworker/0:111:00:510
135589933,0cyclictest14384-21kworker/0:110:00:480
135589933,0cyclictest14384-21kworker/0:109:45:460
135589933,0cyclictest14384-21kworker/0:109:40:460
135589933,0cyclictest14384-21kworker/0:109:20:460
135589933,0cyclictest14384-21kworker/0:109:15:460
135589933,0cyclictest14384-21kworker/0:109:05:460
135589933,0cyclictest14384-21kworker/0:109:01:000
135589933,0cyclictest14384-21kworker/0:108:50:460
135589933,0cyclictest14384-21kworker/0:108:35:540
135589933,0cyclictest14384-21kworker/0:107:00:500
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional