You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-06-06 - 05:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.4.229-rt200 (Profile)

Latency plot of shadow in rack #1, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -n -a -t3 -p99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Tue Jun 06, 2023 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
295750410,0irq/25-eth10-21swapper/318:45:223
1499130,0migration/012783-21kthreadcore18:43:180
271100,0ktimersoftd/20-21swapper/218:43:122
191100,0ktimersoftd/10-21swapper/118:43:111
156739977,0cyclictest0-21swapper/220:48:192
339960,0migration/37999-21taskset23:00:363
339960,0migration/37362-21sh22:41:053
339960,0migration/332379-21sh23:14:463
339960,0migration/327576-21sh20:54:463
339960,0migration/320014-21sh20:51:253
339960,0migration/316529-21sh23:06:183
339960,0migration/31189-21sh23:57:233
28855060,0irq/27-eth0-tx-0-21swapper/223:54:582
28845060,0irq/26-eth0-rx-0-21swapper/321:41:513
156689960,0cyclictest6039-21avahi-daemon22:59:171
156689960,0cyclictest0-21swapper/123:20:051
156689960,0cyclictest0-21swapper/121:20:541
149960,0migration/06454-21df_abs20:43:120
149960,0migration/025592-21fschecks_count20:33:130
149960,0migration/022910-21latency_hist21:53:000
149960,0migration/014126-21sh21:47:030
8225050,0irq/24-0000:00:3063-21munin-run19:03:002
8225050,0irq/24-0000:00:30404-21cat19:48:002
8225050,0irq/24-0000:00:17517-21latency_hist00:07:582
4150,0ktimersoftd/00-21swapper/023:27:220
339950,0migration/39733-21munin-node22:43:223
339950,0migration/38736-21munin-node19:03:263
339950,0migration/36548-21kthreadcore23:38:253
339950,0migration/34285-21cpuspeed_turbos19:28:133
339950,0migration/332558-21munin-node20:38:173
339950,0migration/331989-21kthreadcore20:58:253
339950,0migration/322392-21munin-node21:33:163
339950,0migration/321646-21timerwakeupswit18:48:253
339950,0migration/319865-21sh21:09:253
339950,0migration/318964-21irqcore00:08:273
339950,0migration/318098-21fschecks_time23:08:193
339950,0migration/317971-21munin-node19:38:163
339950,0migration/317269-21hddtemp_smartct19:13:193
339950,0migration/315841-21munin-node22:08:193
339950,0migration/315841-21munin-node22:08:193
339950,0migration/314337-21cpuspeed_turbos21:28:183
339950,0migration/312108-21aten2_r1power_p20:23:153
339950,0migration/310238-21munin-node00:03:183
28855050,0irq/27-eth0-tx-7707-21munin-node22:23:212
28855050,0irq/27-eth0-tx-32728-21kernelversion21:58:212
28855050,0irq/27-eth0-tx-31826-21ssh22:37:162
28855050,0irq/27-eth0-tx-30677-21if_err_eth121:18:202
28855050,0irq/27-eth0-tx-22249-21kthreadcore20:53:212
28855050,0irq/27-eth0-tx-21439-21ssh21:49:342
28855050,0irq/27-eth0-tx-10036-21awk23:23:162
28855050,0irq/27-eth0-tx-0-21swapper/223:50:312
28855050,0irq/27-eth0-tx-0-21swapper/223:40:502
28855050,0irq/27-eth0-tx-0-21swapper/223:36:172
28855050,0irq/27-eth0-tx-0-21swapper/223:08:022
28855050,0irq/27-eth0-tx-0-21swapper/223:03:042
28855050,0irq/27-eth0-tx-0-21swapper/222:59:212
28855050,0irq/27-eth0-tx-0-21swapper/222:51:352
28855050,0irq/27-eth0-tx-0-21swapper/222:47:222
28855050,0irq/27-eth0-tx-0-21swapper/221:28:182
28855050,0irq/27-eth0-tx-0-21swapper/020:48:030
28845050,0irq/26-eth0-rx-7256-21cut19:53:173
28845050,0irq/26-eth0-rx-6039-21avahi-daemon23:23:223
28845050,0irq/26-eth0-rx-32506-21ssh00:14:353
28845050,0irq/26-eth0-rx-2686-21irqcore23:18:223
28845050,0irq/26-eth0-rx-0-21swapper/323:58:253
28845050,0irq/26-eth0-rx-0-21swapper/323:43:203
28845050,0irq/26-eth0-rx-0-21swapper/323:36:073
28845050,0irq/26-eth0-rx-0-21swapper/323:28:043
28845050,0irq/26-eth0-rx-0-21swapper/322:57:203
28845050,0irq/26-eth0-rx-0-21swapper/322:48:323
28845050,0irq/26-eth0-rx-0-21swapper/322:33:223
28845050,0irq/26-eth0-rx-0-21swapper/322:28:383
28845050,0irq/26-eth0-rx-0-21swapper/322:23:133
28845050,0irq/26-eth0-rx-0-21swapper/322:19:383
28845050,0irq/26-eth0-rx-0-21swapper/322:17:393
28845050,0irq/26-eth0-rx-0-21swapper/322:03:263
28845050,0irq/26-eth0-rx-0-21swapper/321:43:243
28845050,0irq/26-eth0-rx-0-21swapper/321:24:023
28845050,0irq/26-eth0-rx-0-21swapper/321:21:153
259950,0migration/210839-21munin-node21:03:292
249950,0watchdog/26079-21sed20:18:142
249950,0watchdog/226949-21ssh00:13:252
249950,0watchdog/216880-21ssh23:45:372
249950,0watchdog/20-21swapper/219:08:272
179950,0migration/14722-21mdstat20:38:221
179950,0migration/126120-21munin-node23:53:151
179950,0migration/123811-21latency_hist19:43:001
179950,0migration/123132-21latency_hist19:18:001
179950,0migration/121947-21taskset20:53:201
179950,0migration/112795-21idleruntime20:23:191
156689955,0cyclictest6039-21avahi-daemon22:39:061
156689955,0cyclictest6039-21avahi-daemon21:35:151
156689955,0cyclictest6039-21avahi-daemon18:48:271
156689955,0cyclictest6039-21avahi-daemon00:11:481
156689955,0cyclictest0-21swapper/123:13:091
156689955,0cyclictest0-21swapper/122:01:351
156689955,0cyclictest0-21swapper/120:13:161
156689953,0cyclictest3350-21kworker/1:220:48:571
156689953,0cyclictest3154-21irqbalance00:04:481
156689950,0cyclictest6039-21avahi-daemon00:17:101
156689950,0cyclictest0-21swapper/123:35:321
156649955,0cyclictest0-21swapper/021:53:520
156649952,0cyclictest0-21swapper/019:13:000
156649950,0cyclictest0-21swapper/023:34:090
156649950,0cyclictest0-21swapper/023:28:380
156649950,0cyclictest0-21swapper/021:39:130
156649950,0cyclictest0-21swapper/020:58:130
156649950,0cyclictest0-21swapper/000:00:380
149950,0migration/08139-21idleruntime22:03:220
149950,0migration/06529-21sh22:20:450
149950,0migration/04651-21sh21:20:580
149950,0migration/03763-21fschecks_count19:03:190
149950,0migration/029846-21missed_timers20:08:220
149950,0migration/028232-21kthreadcore22:53:230
149950,0migration/027795-21needreboot18:53:220
149950,0migration/024296-21kthreadcore21:13:230
149950,0migration/02411-21sh23:18:200
149950,0migration/023675-21fschecks_time19:18:170
149950,0migration/021944-21ntp_kernel_pll_19:13:270
149950,0migration/018351-21fschecks_count00:08:220
149950,0migration/017968-21fschecks_count23:08:190
149950,0migration/017693-21ksmtuned20:02:430
149950,0migration/017469-21munin-node20:23:260
149950,0migration/016661-21sh22:48:010
149950,0migration/016349-21idleruntime22:08:230
149950,0migration/016349-21idleruntime22:08:230
149950,0migration/01288-21latency_hist23:17:580
149950,0migration/012067-21taskset21:23:500
149950,0migration/011906-21munin-node19:33:200
149950,0migration/010464-21runrttasks21:03:250
339940,0migration/36680-21fschecks_time20:43:133
339940,0migration/330072-21df19:23:163
28855040,0irq/27-eth0-tx-14032-21ssh21:08:212
28855040,0irq/27-eth0-tx-0-21swapper/223:28:242
28855040,0irq/27-eth0-tx-0-21swapper/223:18:162
28855040,0irq/27-eth0-tx-0-21swapper/223:16:292
28855040,0irq/27-eth0-tx-0-21swapper/222:53:192
28855040,0irq/27-eth0-tx-0-21swapper/222:29:522
28855040,0irq/27-eth0-tx-0-21swapper/222:08:182
28855040,0irq/27-eth0-tx-0-21swapper/222:08:182
28855040,0irq/27-eth0-tx-0-21swapper/222:03:102
28855040,0irq/27-eth0-tx-0-21swapper/221:43:122
28855040,0irq/27-eth0-tx-0-21swapper/221:33:222
28855040,0irq/27-eth0-tx-0-21swapper/221:23:202
28855040,0irq/27-eth0-tx-0-21swapper/221:14:102
28855040,0irq/27-eth0-tx-0-21swapper/200:05:102
28845040,0irq/26-eth0-rx-6039-21avahi-daemon21:05:143
28845040,0irq/26-eth0-rx-0-21swapper/323:48:513
28845040,0irq/26-eth0-rx-0-21swapper/321:58:133
28845040,0irq/26-eth0-rx-0-21swapper/321:53:103
28845040,0irq/26-eth0-rx-0-21swapper/321:48:003
28845040,0irq/26-eth0-rx-0-21swapper/321:13:173
28845040,0irq/26-eth0-rx-0-21swapper/320:13:213
259940,0migration/2559-21fschecks_time22:38:192
259940,0migration/231874-21taskset22:18:102
259940,0migration/229835-21fschecks_time20:58:202
249940,0watchdog/230551-21ssh21:38:022
249940,0watchdog/219233-21kthreadcore20:03:342
249940,0watchdog/21543-21grep19:23:232
249940,0watchdog/20-21swapper/220:23:222
249940,0watchdog/20-21swapper/219:28:192
179940,0migration/15657-21latency_hist21:23:001
179940,0migration/126339-21irqrtprio20:33:171
179940,0migration/124598-21aten2_r1power_e20:08:141
179940,0migration/117894-21latency_hist20:03:001
169940,0watchdog/17348-21munin-node22:03:341
169940,0watchdog/16101-21irqcore21:03:221
169940,0watchdog/16080-21cut20:18:141
169940,0watchdog/124780-21cut21:53:221
169940,0watchdog/122492-21grep23:48:251
169940,0watchdog/113410-21ssh21:08:181
169940,0watchdog/10-21swapper/123:23:221
169940,0watchdog/10-21swapper/122:43:061
169940,0watchdog/10-21swapper/122:08:181
169940,0watchdog/10-21swapper/122:08:181
169940,0watchdog/10-21swapper/119:28:191
156739944,0cyclictest6039-21avahi-daemon23:58:152
156739943,0cyclictest3427-21snmpd20:39:582
156739943,0cyclictest3427-21snmpd20:35:582
156739943,0cyclictest3427-21snmpd20:14:582
156739943,0cyclictest3427-21snmpd19:55:282
156739943,0cyclictest3427-21snmpd19:42:432
156739943,0cyclictest3154-21irqbalance21:53:092
156739943,0cyclictest3154-21irqbalance20:43:192
156739943,0cyclictest3154-21irqbalance20:29:292
156739943,0cyclictest3154-21irqbalance19:13:292
156739943,0cyclictest25489-21awk20:08:182
156739943,0cyclictest24729-21awk19:43:192
156739943,0cyclictest24713-21awk22:13:202
156739943,0cyclictest16751-21awk18:48:172
156739943,0cyclictest12350-21grep19:58:202
156689944,0cyclictest6039-21avahi-daemon23:39:481
156689944,0cyclictest6039-21avahi-daemon22:19:051
156689944,0cyclictest6039-21avahi-daemon20:43:131
156689943,0cyclictest4264-21awk19:03:211
156689943,0cyclictest3469-21lldpd22:24:031
156689943,0cyclictest3427-21snmpd21:30:431
156689943,0cyclictest3427-21snmpd19:23:131
156689943,0cyclictest3427-21snmpd19:00:281
156689943,0cyclictest3350-21kworker/1:222:36:291
156689943,0cyclictest3350-21kworker/1:221:52:371
156689943,0cyclictest3350-21kworker/1:219:47:421
156689943,0cyclictest3154-21irqbalance22:50:181
156689943,0cyclictest3154-21irqbalance22:13:191
156689943,0cyclictest3154-21irqbalance19:53:391
156689943,0cyclictest3154-21irqbalance19:48:191
156689943,0cyclictest3154-21irqbalance19:13:191
156689943,0cyclictest30031-21awk20:58:211
156689943,0cyclictest29538-21sadc21:18:001
156689943,0cyclictest26061-21awk22:53:211
156689943,0cyclictest2580-21awk23:58:201
156689943,0cyclictest1690-21kworker/1:123:32:191
156689943,0cyclictest1690-21kworker/1:123:08:351
156689943,0cyclictest11502-21awk19:33:171
156689943,0cyclictest10942-21awk23:43:211
156689943,0cyclictest10195-21awk23:03:221
156649944,0cyclictest14539-21cut21:28:200
156649944,0cyclictest0-21swapper/022:35:390
156649944,0cyclictest0-21swapper/022:28:050
156649944,0cyclictest0-21swapper/022:17:310
156649943,0cyclictest8510-21awk22:23:140
156649943,0cyclictest3469-21lldpd20:29:010
156649943,0cyclictest3427-21snmpd23:54:440
156649943,0cyclictest3427-21snmpd23:43:140
156649943,0cyclictest3427-21snmpd21:08:280
156649943,0cyclictest3427-21snmpd19:50:430
156649943,0cyclictest3427-21snmpd19:40:430
156649943,0cyclictest3154-21irqbalance23:38:180
156649943,0cyclictest3154-21irqbalance23:03:080
156649943,0cyclictest3154-21irqbalance21:58:090
156649943,0cyclictest3154-21irqbalance20:55:090
156649943,0cyclictest3154-21irqbalance20:18:090
156649943,0cyclictest3154-21irqbalance20:14:290
156649943,0cyclictest3154-21irqbalance20:06:390
156649943,0cyclictest3154-21irqbalance19:32:490
156649943,0cyclictest3154-21irqbalance18:58:390
156649943,0cyclictest3154-21irqbalance18:48:200
156649943,0cyclictest3154-21irqbalance00:04:180
156649943,0cyclictest26952-21awk00:13:260
156649943,0cyclictest24744-21awk19:43:190
156649943,0cyclictest18898-21awk23:48:210
156649943,0cyclictest1770-21awk22:58:220
149940,0migration/023094-21sh21:33:190
149940,0migration/010044-21missed_timers19:53:190
8225030,0irq/24-0000:00:0-21swapper/219:33:002
339930,0migration/325295-21kthreadcore19:43:213
28845030,0irq/26-eth0-rx-6039-21avahi-daemon19:52:093
28845030,0irq/26-eth0-rx-0-21swapper/319:59:193
156739932,0cyclictest3427-21snmpd19:18:132
156739930,0cyclictest8960-21python19:03:282
156739930,0cyclictest6039-21avahi-daemon18:56:012
156689933,0cyclictest3350-21kworker/1:222:28:171
156689933,0cyclictest3350-21kworker/1:221:43:271
156689933,0cyclictest3350-21kworker/1:221:38:091
156689933,0cyclictest3350-21kworker/1:220:28:081
156689933,0cyclictest3350-21kworker/1:219:58:021
156689933,0cyclictest3350-21kworker/1:219:08:001
156689933,0cyclictest3350-21kworker/1:218:53:061
156649933,0cyclictest0-21swapper/022:40:140
156649932,0cyclictest3427-21snmpd19:24:200
156649932,0cyclictest3154-21irqbalance20:38:190
156649930,0cyclictest8727-21df22:43:170
329920,0watchdog/312691-21kthreadcore19:08:213
329920,0watchdog/30-21swapper/318:58:173
28845020,0irq/26-eth0-rx-6039-21avahi-daemon20:18:243
28845020,0irq/26-eth0-rx-0-21swapper/320:33:003
28845020,0irq/26-eth0-rx-0-21swapper/320:30:453
28845020,0irq/26-eth0-rx-0-21swapper/320:09:293
28845020,0irq/26-eth0-rx-0-21swapper/320:03:263
28845020,0irq/26-eth0-rx-0-21swapper/319:35:233
28845020,0irq/26-eth0-rx-0-21swapper/319:18:143
28845020,0irq/26-eth0-rx-0-21swapper/318:53:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional