You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-13 - 14:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Wed May 13, 2026 12:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1367024738,0sleep20-21swapper/206:50:592
1950923823,0sleep00-21swapper/006:52:250
1942723626,0sleep30-21swapper/306:51:283
1960423329,0sleep10-21swapper/106:53:301
198729977,0cyclictest8723-21sensors07:56:060
339960,0migration/332129-21sh12:03:183
339960,0migration/325337-21sh11:20:243
339960,0migration/323769-21sh11:36:333
339960,0migration/318432-21sh09:14:493
339960,0migration/318229-21sh09:54:563
339960,0migration/318229-21sh09:54:563
339960,0migration/311020-21sh09:30:543
29715060,0irq/26-eth0-rx-5310-21cut08:45:573
29715060,0irq/26-eth0-rx-20919-21grep08:56:023
29715060,0irq/26-eth0-rx-1828-21sed10:45:443
29715060,0irq/26-eth0-rx-13222-21grep08:01:023
198739960,0cyclictest8546-21smartctl07:10:591
149960,0migration/09785-21sh10:09:120
149960,0migration/028766-21kthreadcore12:21:000
149960,0migration/025052-21latency_hist12:00:440
149960,0migration/01888-21sh10:25:440
4150,0ktimersoftd/00-21swapper/009:18:230
339950,0migration/37513-21kthreadcore09:26:023
339950,0migration/36902-21timerandwakeup07:06:083
339950,0migration/34534-21kthreadcore07:31:003
339950,0migration/331614-21kthreadcore10:01:013
339950,0migration/329671-21kthreadcore11:41:013
339950,0migration/329382-21taskset08:14:463
339950,0migration/326871-21timerandwakeup07:21:093
339950,0migration/326343-21aten_r1power_en10:40:553
339950,0migration/324404-21munin-node10:16:073
339950,0migration/318179-21kthreadcore07:41:023
339950,0migration/317809-21kthreadcore07:16:033
339950,0migration/317211-21diskmemload10:33:273
339950,0migration/316769-21munin-node08:05:583
339950,0migration/314785-21latency11:31:013
339950,0migration/312362-21kthreadcore07:36:033
339950,0migration/311165-21fschecks_count08:25:583
339950,0migration/310806-21sh10:50:583
339950,0migration/310342-21irqcore11:51:003
29725050,0irq/27-eth0-tx-429-21unixbench_multi10:01:072
29725050,0irq/27-eth0-tx-3726-21if_eth1.309:46:002
29725050,0irq/27-eth0-tx-3726-21if_eth1.309:46:002
29725050,0irq/27-eth0-tx-25744-21munin-node11:21:052
29725050,0irq/27-eth0-tx-19439-21kthreadcore10:56:032
29725050,0irq/27-eth0-tx-18305-21munin-node10:36:032
29725050,0irq/27-eth0-tx-17440-21munin-node11:36:032
29725050,0irq/27-eth0-tx-1665-21munin-node11:05:592
29725050,0irq/27-eth0-tx-11386-21ssh10:31:012
29725050,0irq/27-eth0-tx-11310-21perl10:10:572
29725050,0irq/27-eth0-tx-0-21swapper/212:21:202
29725050,0irq/27-eth0-tx-0-21swapper/211:12:472
29725050,0irq/27-eth0-tx-0-21swapper/209:36:012
29725050,0irq/27-eth0-tx-0-21swapper/008:32:180
29725050,0irq/27-eth0-tx-0-21swapper/007:51:000
29715050,0irq/26-eth0-rx-8145-21ntp_offset10:06:033
29715050,0irq/26-eth0-rx-21891-21cut11:56:023
29715050,0irq/26-eth0-rx-0-21swapper/312:14:273
29715050,0irq/26-eth0-rx-0-21swapper/312:09:483
29715050,0irq/26-eth0-rx-0-21swapper/311:47:153
29715050,0irq/26-eth0-rx-0-21swapper/311:25:583
29715050,0irq/26-eth0-rx-0-21swapper/311:21:003
29715050,0irq/26-eth0-rx-0-21swapper/311:12:453
29715050,0irq/26-eth0-rx-0-21swapper/311:06:003
29715050,0irq/26-eth0-rx-0-21swapper/311:00:573
29715050,0irq/26-eth0-rx-0-21swapper/310:38:573
29715050,0irq/26-eth0-rx-0-21swapper/310:28:363
29715050,0irq/26-eth0-rx-0-21swapper/310:20:503
29715050,0irq/26-eth0-rx-0-21swapper/310:13:503
29715050,0irq/26-eth0-rx-0-21swapper/309:46:043
29715050,0irq/26-eth0-rx-0-21swapper/309:46:043
29715050,0irq/26-eth0-rx-0-21swapper/309:41:113
29715050,0irq/26-eth0-rx-0-21swapper/309:35:473
29715050,0irq/26-eth0-rx-0-21swapper/309:22:253
29715050,0irq/26-eth0-rx-0-21swapper/309:17:473
29715050,0irq/26-eth0-rx-0-21swapper/309:05:193
259950,0migration/28457-21taskset11:28:392
259950,0migration/27914-21latency09:06:032
259950,0migration/27449-21missed_timers10:46:032
259950,0migration/24014-21munin-node07:56:002
259950,0migration/232664-21missed_timers09:21:042
259950,0migration/231845-21kthreadcore08:16:012
259950,0migration/225944-21sh10:19:352
259950,0migration/223345-21cpuspeed_turbos08:10:572
198749955,0cyclictest0-21swapper/212:07:122
198749955,0cyclictest0-21swapper/211:16:552
198749955,0cyclictest0-21swapper/210:52:102
198749955,0cyclictest0-21swapper/210:21:062
198749955,0cyclictest0-21swapper/209:15:402
198749954,0cyclictest3514-21snmpd07:08:342
198749950,0cyclictest0-21swapper/209:55:152
198749950,0cyclictest0-21swapper/209:55:152
198729953,0cyclictest20419-21kworker/0:209:03:080
179950,0migration/1657-21latency_hist12:05:441
179950,0migration/13185-21kthreadcore11:26:011
179950,0migration/126394-21taskset09:39:291
179950,0migration/121078-21latency_hist07:20:451
179950,0migration/117528-21sh10:55:121
179950,0migration/116502-21timerandwakeup10:11:071
179950,0migration/110560-21taskset09:30:161
169950,0watchdog/123171-21kthreadcore06:56:041
149950,0migration/09958-21gltestperf11:50:570
149950,0migration/09820-21timerandwakeup08:21:050
149950,0migration/0978-21aten_r1power_po12:05:550
149950,0migration/08411-21pmu-power09:46:050
149950,0migration/08411-21pmu-power09:46:050
149950,0migration/08354-21sh11:07:310
149950,0migration/05119-21kthreadcore10:26:010
149950,0migration/030616-21kthreadcore07:26:030
149950,0migration/028248-21kthreadcore11:41:000
149950,0migration/026928-21munin-node11:00:590
149950,0migration/026661-21sh09:40:140
149950,0migration/025119-21kthreadcore07:46:030
149950,0migration/023689-21fschecks_time08:10:580
149950,0migration/021109-21kthreadcore11:36:000
149950,0migration/020565-21munin-node06:56:000
149950,0migration/019970-21irqcore09:56:000
149950,0migration/01887-21latency_hist07:30:450
149950,0migration/018649-21cpuspeed_turbos08:55:570
149950,0migration/017298-21proc_pri08:51:030
149950,0migration/01647-21taskset09:22:550
149950,0migration/015529-21aten2.4-expect07:40:560
149950,0migration/014355-21kthreadcore11:31:010
149950,0migration/013804-21taskset07:11:400
149950,0migration/011858-21kthreadcore11:11:010
149950,0migration/011573-21latency_hist08:50:450
149950,0migration/010870-21cpuspeed_turbos10:10:560
339940,0migration/332734-21kthreadcore08:41:013
339940,0migration/331588-21kthreadcore07:26:033
339940,0migration/327768-21irqcore07:01:013
339940,0migration/321935-21kthreadcore10:56:043
339940,0migration/317585-21timerwakeupswit08:51:053
329940,0watchdog/329996-21irqstats07:51:003
29725040,0irq/27-eth0-tx-8714-21ssh10:26:592
29725040,0irq/27-eth0-tx-27487-21munin-node09:01:002
29725040,0irq/27-eth0-tx-1638-21ssh11:45:562
29725040,0irq/27-eth0-tx-0-21swapper/212:11:002
29725040,0irq/27-eth0-tx-0-21swapper/212:00:562
29725040,0irq/27-eth0-tx-0-21swapper/211:56:312
29725040,0irq/27-eth0-tx-0-21swapper/211:51:282
29725040,0irq/27-eth0-tx-0-21swapper/211:40:542
29725040,0irq/27-eth0-tx-0-21swapper/209:30:492
29725040,0irq/27-eth0-tx-0-21swapper/209:25:562
29725040,0irq/27-eth0-tx-0-21swapper/208:56:352
29725040,0irq/27-eth0-tx-0-21swapper/007:35:590
29715040,0irq/26-eth0-rx-0-21swapper/312:21:363
29715040,0irq/26-eth0-rx-0-21swapper/312:16:113
29715040,0irq/26-eth0-rx-0-21swapper/309:56:013
29715040,0irq/26-eth0-rx-0-21swapper/309:07:473
29715040,0irq/26-eth0-rx-0-21swapper/308:20:543
259940,0migration/23273-21hddtemp_smartct10:05:582
259940,0migration/229072-21kthreadcore11:01:012
259940,0migration/227538-21munin-node09:40:582
259940,0migration/224571-21fschecks_count08:35:582
259940,0migration/221705-21kthreadcore12:16:002
259940,0migration/220247-21kthreadcore09:56:012
259940,0migration/217579-21aten_r1power_vo08:30:562
259940,0migration/215982-21kthreadcore07:16:032
259940,0migration/214890-21kthreadcore08:50:592
259940,0migration/212034-21kthreadcore11:31:002
259940,0migration/210353-21latency08:46:032
249940,0watchdog/224953-21sed09:16:072
249940,0watchdog/224459-21kthreadcore07:21:042
249940,0watchdog/213825-21cut07:36:042
198749944,0cyclictest0-21swapper/210:41:552
198749943,0cyclictest4870-21awk08:20:572
198749943,0cyclictest3556-21lldpd08:06:072
198749943,0cyclictest3556-21lldpd07:03:072
198749943,0cyclictest3514-21snmpd08:02:342
198749943,0cyclictest3514-21snmpd07:35:342
198749943,0cyclictest3514-21snmpd07:30:342
198749943,0cyclictest31686-21awk08:41:002
198749943,0cyclictest29567-21awk07:50:592
198749943,0cyclictest21065-21awk06:56:022
198749942,0cyclictest16336-21awk07:41:002
198739944,0cyclictest0-21swapper/111:04:191
198739944,0cyclictest0-21swapper/110:26:471
198739943,0cyclictest3556-21lldpd08:32:071
198739943,0cyclictest3556-21lldpd08:21:371
198739943,0cyclictest3556-21lldpd08:12:371
198739943,0cyclictest3556-21lldpd07:58:371
198739943,0cyclictest3514-21snmpd10:16:501
198739943,0cyclictest3514-21snmpd09:08:501
198739943,0cyclictest28083-21awk09:41:001
198739943,0cyclictest2659-21sadc09:25:451
198739943,0cyclictest25992-21awk12:20:581
198739943,0cyclictest23206-21ps07:46:021
198739943,0cyclictest15242-21awk07:16:001
198739943,0cyclictest12627-21awk08:50:571
198739943,0cyclictest11746-21awk09:10:591
198739943,0cyclictest10069-21awk11:50:581
198739940,0cyclictest0-21swapper/112:10:511
198729944,0cyclictest0-21swapper/011:21:290
198729944,0cyclictest0-21swapper/009:12:130
198729943,0cyclictest3840-21awk09:25:590
198729943,0cyclictest3514-21snmpd10:01:200
198729943,0cyclictest3241-21irqbalance12:04:430
198729943,0cyclictest3241-21irqbalance10:52:230
198729943,0cyclictest3241-21irqbalance09:51:330
198729943,0cyclictest3241-21irqbalance09:51:330
198729943,0cyclictest3241-21irqbalance09:09:230
198729940,0cyclictest0-21swapper/012:16:120
19140,0ktimersoftd/10-21swapper/109:51:291
19140,0ktimersoftd/10-21swapper/109:51:291
179940,0migration/16379-21kthreadcore08:46:021
179940,0migration/14749-21kthreadcore11:46:001
179940,0migration/132662-21kthreadcore08:41:011
179940,0migration/130919-21kthreadcore07:51:011
179940,0migration/126830-21latency_hist10:00:451
179940,0migration/124319-21cpuspeed_turbos08:35:571
179940,0migration/123220-21unixbench-2d11:56:081
179940,0migration/119978-21irqcore09:56:001
179940,0migration/119289-21kthreadcore11:36:001
179940,0migration/115580-21timerandwakeup11:11:071
179940,0migration/114498-21kthreadcore08:26:031
179940,0migration/11210-21runrttasks10:23:461
179940,0migration/110210-21gltestperf11:30:571
169940,0watchdog/126461-21ssh09:19:151
169940,0watchdog/124807-21ssh11:18:511
169940,0watchdog/118286-21ssh10:35:511
169940,0watchdog/11773-21munin-node07:06:001
169940,0watchdog/10-21swapper/112:06:021
169940,0watchdog/10-21swapper/111:21:031
169940,0watchdog/10-21swapper/110:46:471
169940,0watchdog/10-21swapper/109:01:031
169940,0watchdog/10-21swapper/107:36:001
169940,0watchdog/10-21swapper/107:31:001
149940,0migration/06063-21kthreadcore11:26:010
149940,0migration/023503-21kthreadcore10:16:020
149940,0migration/021809-21gltestperf07:21:000
149940,0migration/018400-21kthreadcore07:16:030
149940,0migration/012906-21kthreadcore08:26:020
11334240,0sleep112881-21kthreadcore10:31:021
29725030,0irq/27-eth0-tx-0-21swapper/008:40:550
29715030,0irq/26-eth0-rx-0-21swapper/308:36:023
29715030,0irq/26-eth0-rx-0-21swapper/308:32:253
29715030,0irq/26-eth0-rx-0-21swapper/308:15:553
29715030,0irq/26-eth0-rx-0-21swapper/307:56:103
29715030,0irq/26-eth0-rx-0-21swapper/306:55:583
198749932,0cyclictest8654-21awk07:11:002
198749932,0cyclictest22999-21awk07:46:012
198749932,0cyclictest11581-21awk08:26:002
198739933,0cyclictest3556-21lldpd07:45:071
198739933,0cyclictest3556-21lldpd07:28:071
198739932,0cyclictest3632-21awk09:45:591
198739932,0cyclictest3632-21awk09:45:591
198739932,0cyclictest27642-21awk07:01:001
198739932,0cyclictest27180-21awk10:40:591
198739932,0cyclictest26491-21awk11:40:591
198739932,0cyclictest22059-21sensors08:06:071
198739932,0cyclictest21290-21tr12:16:001
198739932,0cyclictest10576-21awk08:00:591
198729933,0cyclictest20419-21kworker/0:209:40:460
198729933,0cyclictest20419-21kworker/0:208:36:000
198729933,0cyclictest20419-21kworker/0:208:17:490
198729933,0cyclictest19581-21kworker/0:111:46:100
198729933,0cyclictest19581-21kworker/0:111:17:320
198729933,0cyclictest19581-21kworker/0:110:57:040
198729933,0cyclictest19581-21kworker/0:110:46:020
198729933,0cyclictest19581-21kworker/0:110:41:420
198729933,0cyclictest19581-21kworker/0:110:30:500
198729932,0cyclictest9101-21aten_r1power_vo12:10:550
198729932,0cyclictest1952-21awk07:06:010
198729932,0cyclictest11605-21grep09:30:580
198729932,0cyclictest10561-21awk08:00:590
198729932,0cyclictest0-21swapper/008:06:050
198729930,0cyclictest23672-21memory10:36:010
198729930,0cyclictest0-21swapper/007:00:590
169930,0watchdog/10-21swapper/110:55:511
169930,0watchdog/10-21swapper/109:31:551
169930,0watchdog/10-21swapper/108:59:111
29715020,0irq/26-eth0-rx-0-21swapper/307:45:513
29715020,0irq/26-eth0-rx-0-21swapper/307:10:493
169920,0watchdog/10-21swapper/111:05:471
169920,0watchdog/10-21swapper/110:05:471
169920,0watchdog/10-21swapper/108:15:551
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional