You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-14 - 08:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Sun Dec 14, 2025 00:43:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1699623829,0sleep20-21swapper/219:07:442
1711823430,0sleep00-21swapper/019:09:060
304450320,0irq/25-eth10-21swapper/319:08:333
1686223127,0sleep10-21swapper/119:06:071
339960,0migration/35341-21munin-node22:55:073
339960,0migration/330652-21munin-node21:55:043
339960,0migration/318542-21sh22:42:033
29725060,0irq/27-eth0-tx-13716-21munin-node20:20:170
29715060,0irq/26-eth0-rx-9576-21unixbench_multi23:50:163
29715060,0irq/26-eth0-rx-23967-21crond22:09:553
29715060,0irq/26-eth0-rx-21859-21sh21:50:053
29715060,0irq/26-eth0-rx-14384-21sh22:02:583
19160,0ktimersoftd/10-21swapper/119:45:101
172649966,0cyclictest0-21swapper/223:19:552
172649960,0cyclictest0-21swapper/223:09:312
172649960,0cyclictest0-21swapper/222:51:142
172649960,0cyclictest0-21swapper/222:47:262
172629966,0cyclictest0-21swapper/121:18:011
149960,0migration/03904-21sh00:06:390
149960,0migration/028919-21sh23:05:480
149960,0migration/028441-21sh00:03:200
149960,0migration/024432-21timerandwakeup23:40:160
149960,0migration/022222-21sh22:06:160
339950,0migration/36140-21kthreadcore21:40:113
339950,0migration/34469-21sh00:07:513
339950,0migration/332732-21munin-node20:30:193
339950,0migration/332522-21timerandwakeup22:30:183
339950,0migration/329199-21taskset23:06:233
339950,0migration/326654-21kthreadcore19:15:183
339950,0migration/324558-21kthreadcore20:00:123
339950,0migration/321015-21kthreadcore00:35:133
339950,0migration/320453-21aten2.4-expect20:25:043
339950,0migration/319138-21cpuspeed_turbos21:30:053
339950,0migration/317955-21aten_r1power_cu22:25:053
339950,0migration/31791-21runrttasks21:35:353
339950,0migration/317797-21kthreadcore00:15:123
339950,0migration/316827-21kthreadcore19:30:133
339950,0migration/314049-21kthreadcore22:20:133
29725050,0irq/27-eth0-tx-7970-21sh23:31:292
29725050,0irq/27-eth0-tx-7970-21sh23:31:292
29725050,0irq/27-eth0-tx-5059-21sh23:10:222
29725050,0irq/27-eth0-tx-29794-21kthreadcore21:35:112
29725050,0irq/27-eth0-tx-25073-21munin-node19:40:130
29725050,0irq/27-eth0-tx-23041-21munin-node00:20:102
29725050,0irq/27-eth0-tx-16641-21iostat22:05:112
29725050,0irq/27-eth0-tx-0-21swapper/200:35:132
29725050,0irq/27-eth0-tx-0-21swapper/021:10:060
29725050,0irq/27-eth0-tx-0-21swapper/021:06:190
29725050,0irq/27-eth0-tx-0-21swapper/020:40:160
29725050,0irq/27-eth0-tx-0-21swapper/020:25:170
29715050,0irq/26-eth0-rx-9819-21ssh21:24:583
29715050,0irq/26-eth0-rx-8575-21tr19:50:123
29715050,0irq/26-eth0-rx-6478-21chrt22:15:173
29715050,0irq/26-eth0-rx-311-21ntp_states23:45:143
29715050,0irq/26-eth0-rx-31052-21grep20:55:123
29715050,0irq/26-eth0-rx-2967-21ssh22:51:013
29715050,0irq/26-eth0-rx-23493-21tune2fs00:20:073
29715050,0irq/26-eth0-rx-21794-21diskstats20:50:083
29715050,0irq/26-eth0-rx-20243-21aten_r1power_en22:45:053
29715050,0irq/26-eth0-rx-13791-21seq23:15:273
29715050,0irq/26-eth0-rx-0-21swapper/323:58:583
29715050,0irq/26-eth0-rx-0-21swapper/323:39:573
29715050,0irq/26-eth0-rx-0-21swapper/323:25:293
29715050,0irq/26-eth0-rx-0-21swapper/323:01:033
29715050,0irq/26-eth0-rx-0-21swapper/322:11:183
29715050,0irq/26-eth0-rx-0-21swapper/321:48:233
29715050,0irq/26-eth0-rx-0-21swapper/321:20:083
29715050,0irq/26-eth0-rx-0-21swapper/321:11:543
29715050,0irq/26-eth0-rx-0-21swapper/319:20:123
29715050,0irq/26-eth0-rx-0-21swapper/300:30:123
29715050,0irq/26-eth0-rx-0-21swapper/300:25:113
29715050,0irq/26-eth0-rx-0-21swapper/300:14:033
259950,0migration/27071-21if_eth1.219:50:112
259950,0migration/220027-21sh23:00:192
259950,0migration/211946-21kthreadcore20:40:102
259950,0migration/211766-21latency_hist19:29:552
249950,0watchdog/231859-21kthreadcore21:55:102
19150,0ktimersoftd/10-21swapper/122:29:451
172649955,0cyclictest0-21swapper/221:05:172
172649950,0cyclictest0-21swapper/223:15:112
172629955,0cyclictest0-21swapper/122:36:041
172629950,0cyclictest0-21swapper/123:56:001
179950,0migration/16292-21irqrtprio19:25:111
179950,0migration/13681-21latency19:20:121
179950,0migration/131193-21kthreadcore20:05:131
179950,0migration/120409-21kthreadcore00:35:131
179950,0migration/116805-21munin-node22:05:121
179950,0migration/113999-21sh22:02:111
179950,0migration/113118-21sh00:12:521
149950,0migration/08373-21sh21:22:380
149950,0migration/08301-21kthreadcore22:55:120
149950,0migration/08037-21kthreadcore23:50:110
149950,0migration/0628-21taskset20:30:240
149950,0migration/05972-21latency_hist19:49:550
149950,0migration/04804-21kthreadcore20:35:120
149950,0migration/032672-21munin-node22:30:180
149950,0migration/032238-21kthreadcore22:50:100
149950,0migration/032217-21runrttasks23:10:110
149950,0migration/032074-21kthreadcore21:55:090
149950,0migration/03037-21sh23:49:410
149950,0migration/030163-21kthreadcore21:35:110
149950,0migration/028960-21sh21:52:150
149950,0migration/02834-21fschecks_time21:00:080
149950,0migration/027982-21latency_hist20:54:550
149950,0migration/027721-21kthreadcore20:05:120
149950,0migration/027478-21sh22:47:400
149950,0migration/027089-21switchtime20:50:180
149950,0migration/024834-21sh23:25:010
149950,0migration/018906-21aten_r1power_cu21:30:040
149950,0migration/017032-21taskset22:23:410
149950,0migration/01681-21kthreadcore19:20:120
149950,0migration/016781-21kthreadcore19:30:130
149950,0migration/016615-21kthreadcore23:00:110
149950,0migration/016217-21kthreadcore21:45:120
149950,0migration/015595-21proc_pri21:25:160
149950,0migration/014391-21sh00:15:050
149950,0migration/012153-21taskset19:50:380
149950,0migration/011613-21kthreadcore23:35:110
8215040,0irq/24-0000:00:0-21swapper/220:09:552
339940,0migration/36115-21irqcore19:25:113
339940,0migration/332540-21sh19:45:103
339940,0migration/319700-21kthreadcore23:20:113
339940,0migration/319517-21kthreadcore19:10:083
339940,0migration/318939-21taskset19:57:203
339940,0migration/316958-21kthreadcore20:20:123
339940,0migration/313432-21kthreadcore21:05:143
29725040,0irq/27-eth0-tx-2833-21ssh22:35:032
29725040,0irq/27-eth0-tx-18963-21chrt21:30:042
29725040,0irq/27-eth0-tx-18561-21munin-node23:40:082
29725040,0irq/27-eth0-tx-13071-21munin-node22:00:182
29725040,0irq/27-eth0-tx-11816-21grep00:10:172
29725040,0irq/27-eth0-tx-0-21swapper/223:55:242
29725040,0irq/27-eth0-tx-0-21swapper/223:50:162
29725040,0irq/27-eth0-tx-0-21swapper/223:45:282
29725040,0irq/27-eth0-tx-0-21swapper/223:35:492
29725040,0irq/27-eth0-tx-0-21swapper/223:25:142
29725040,0irq/27-eth0-tx-0-21swapper/222:55:452
29725040,0irq/27-eth0-tx-0-21swapper/222:39:562
29725040,0irq/27-eth0-tx-0-21swapper/222:29:582
29725040,0irq/27-eth0-tx-0-21swapper/222:25:182
29725040,0irq/27-eth0-tx-0-21swapper/222:20:052
29725040,0irq/27-eth0-tx-0-21swapper/222:14:572
29725040,0irq/27-eth0-tx-0-21swapper/221:45:042
29725040,0irq/27-eth0-tx-0-21swapper/221:20:012
29725040,0irq/27-eth0-tx-0-21swapper/221:15:012
29725040,0irq/27-eth0-tx-0-21swapper/200:32:232
29725040,0irq/27-eth0-tx-0-21swapper/200:05:232
29725040,0irq/27-eth0-tx-0-21swapper/200:00:142
29725040,0irq/27-eth0-tx-0-21swapper/020:10:090
29725040,0irq/27-eth0-tx-0-21swapper/019:55:130
29725040,0irq/27-eth0-tx-0-21swapper/019:15:210
29715040,0irq/26-eth0-rx-24449-21taskset21:15:023
29715040,0irq/26-eth0-rx-0-21swapper/323:35:123
29715040,0irq/26-eth0-rx-0-21swapper/323:30:043
29715040,0irq/26-eth0-rx-0-21swapper/323:30:043
29715040,0irq/26-eth0-rx-0-21swapper/323:10:263
29715040,0irq/26-eth0-rx-0-21swapper/322:35:293
29715040,0irq/26-eth0-rx-0-21swapper/320:05:133
29715040,0irq/26-eth0-rx-0-21swapper/319:40:073
29715040,0irq/26-eth0-rx-0-21swapper/300:00:103
259940,0migration/27875-21latency_hist20:39:552
259940,0migration/24234-21kthreadcore00:25:122
259940,0migration/230323-21runrttasks19:17:342
259940,0migration/225619-21runrttasks20:02:122
259940,0migration/221773-21df_abs20:50:082
259940,0migration/212353-21kthreadcore21:25:122
249940,0watchdog/231860-21timerandwakeup20:05:182
249940,0watchdog/222725-21kthreadcore20:25:102
249940,0watchdog/212806-21munin-node20:15:182
249940,0watchdog/20-21swapper/219:35:142
172649943,0cyclictest5411-21awk21:40:102
172649943,0cyclictest3556-21lldpd20:20:492
172649943,0cyclictest3556-21lldpd19:56:192
172649943,0cyclictest3514-21snmpd20:32:292
172649943,0cyclictest3514-21snmpd19:46:292
172649943,0cyclictest3202-21awk21:00:102
172649943,0cyclictest31623-21awk19:20:092
172649943,0cyclictest28903-21awk20:55:092
172649943,0cyclictest26029-21awk19:40:112
172649943,0cyclictest25157-21awk22:10:082
172649943,0cyclictest22800-21awk21:50:102
172649943,0cyclictest18376-21awk19:10:072
172649943,0cyclictest16742-21awk21:10:092
172649943,0cyclictest0-21swapper/200:14:542
172629943,0cyclictest5511-21awk21:40:101
172629943,0cyclictest3556-21lldpd21:04:191
172629943,0cyclictest3514-21snmpd23:10:311
172629943,0cyclictest3514-21snmpd22:55:311
172629943,0cyclictest3514-21snmpd21:46:301
172629943,0cyclictest3514-21snmpd21:24:301
172629943,0cyclictest3514-21snmpd20:19:591
172629943,0cyclictest3514-21snmpd20:00:441
172629943,0cyclictest3514-21snmpd19:30:291
172629943,0cyclictest3514-21snmpd19:11:291
172629943,0cyclictest3514-21snmpd00:16:311
172629943,0cyclictest28799-21awk21:35:091
172629943,0cyclictest27984-21awk20:30:121
172629943,0cyclictest23917-21awk00:20:091
172629940,0cyclictest0-21swapper/120:17:191
172619943,0cyclictest8067-21awk20:15:110
172619943,0cyclictest3556-21lldpd22:44:180
172619943,0cyclictest3514-21snmpd22:19:000
172619943,0cyclictest3514-21snmpd19:27:590
172619943,0cyclictest1998-21awk23:30:070
172619943,0cyclictest1998-21awk23:30:070
172619943,0cyclictest18912-21awk22:25:090
172619943,0cyclictest14719-21munin-node20:45:110
172619940,0cyclictest0-21swapper/020:04:550
179940,0migration/1967-21missed_timers20:55:131
179940,0migration/18673-21gltestperf00:30:091
179940,0migration/17398-21latency_hist23:14:551
179940,0migration/15662-21latency20:10:121
179940,0migration/15368-21kthreadcore23:50:101
179940,0migration/15088-21sh00:25:171
179940,0migration/130663-21runrttasks22:10:501
179940,0migration/130373-21kthreadcore22:30:121
179940,0migration/128494-21kernelversion23:45:111
179940,0migration/127151-21kthreadcore23:05:121
179940,0migration/126868-21kthreadcore19:40:121
179940,0migration/121775-21latency_hist21:49:551
179940,0migration/121746-21kthreadcore20:25:101
179940,0migration/121607-21switchtime21:10:171
179940,0migration/120585-21kthreadcore19:35:131
179940,0migration/119971-21latency20:45:161
179940,0migration/11602-21irqcore22:15:111
179940,0migration/115672-21sendmail_mailst23:35:161
179940,0migration/115291-21unixbench-2d22:20:181
169940,0watchdog/16559-21runrttasks22:40:131
169940,0watchdog/131334-21ssh23:25:571
169940,0watchdog/130409-21kthreadcore22:50:091
169940,0watchdog/129093-21kthreadcore19:15:181
169940,0watchdog/12440-21latency00:05:131
169940,0watchdog/123616-21ssh23:22:531
169940,0watchdog/12245-21awk20:35:101
169940,0watchdog/118457-21ssh21:29:421
169940,0watchdog/112764-21kthreadcore21:05:141
169940,0watchdog/110118-21cat20:40:101
169940,0watchdog/10-21swapper/122:59:571
169940,0watchdog/10-21swapper/121:55:101
149940,0migration/06222-21fschecks_count00:10:070
149940,0migration/024698-21latency_hist23:24:550
149940,0migration/019956-21kthreadcore19:35:130
149940,0migration/019405-21kthreadcore00:35:120
149940,0migration/015572-21sh00:33:270
149940,0migration/013390-21sh22:00:550
339930,0migration/3594-21cpuspeed_turbos20:10:063
29725030,0irq/27-eth0-tx-0-21swapper/019:10:030
29715030,0irq/26-eth0-rx-0-21swapper/321:00:163
29715030,0irq/26-eth0-rx-0-21swapper/320:15:073
29715030,0irq/26-eth0-rx-0-21swapper/319:35:083
172649932,0cyclictest3514-21snmpd19:25:442
172629933,0cyclictest3241-21irqbalance23:30:041
172629933,0cyclictest3241-21irqbalance23:30:041
172629932,0cyclictest22305-21awk20:50:111
172629932,0cyclictest19620-21irqcore23:40:101
172619933,0cyclictest907-21kworker/0:021:15:550
172619933,0cyclictest21165-21kworker/0:223:55:100
172619932,0cyclictest8542-21awk23:15:070
172619932,0cyclictest4509-21munin-node21:40:090
172619932,0cyclictest32607-21awk00:25:100
172619932,0cyclictest23996-21ethtool00:20:090
172619930,0cyclictest15514-21diskmemload22:11:330
179930,0migration/127505-21sh22:47:431
169930,0watchdog/129142-21ssh00:04:491
169930,0watchdog/112158-40yum-updatesd-he19:50:421
169930,0watchdog/10-21swapper/121:30:061
169930,0watchdog/10-21swapper/119:55:101
329920,0watchdog/30-21swapper/320:35:113
29715020,0irq/26-eth0-rx-0-21swapper/320:45:243
29715020,0irq/26-eth0-rx-0-21swapper/320:40:053
249920,0watchdog/20-21swapper/220:44:582
172619921,0cyclictest28062chrt22:35:000
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional