You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-06-02 - 21:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.4.229-rt200 (Profile)

Latency plot of shadow in rack #1, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -n -a -t3 -p99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Fri Jun 02, 2023 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
295750440,0irq/25-eth10-21swapper/306:47:093
1799120,0migration/110045-21fschecks_time06:43:441
288550100,0irq/27-eth0-tx-0-21swapper/006:43:490
271100,0ktimersoftd/20-21swapper/206:43:392
158339970,0cyclictest0-21swapper/009:13:400
149970,0migration/019428-21sh09:10:210
35160,0ktimersoftd/30-21swapper/311:00:223
339960,0migration/322999-21sh11:49:033
28855060,0irq/27-eth0-tx-3935-21ssh10:03:282
28845060,0irq/26-eth0-rx-0-21swapper/309:49:093
158429960,0cyclictest6039-21avahi-daemon11:16:532
158339960,0cyclictest21545-21kworker/0:010:33:550
149960,0migration/030314-21sh11:17:350
149960,0migration/027328-21sh08:54:350
149960,0migration/02516-21taskset09:18:570
149960,0migration/021814-21sh11:29:440
149960,0migration/020575-21sh11:09:090
8225050,0irq/24-0000:00:21371-21latency_hist10:53:222
339950,0migration/39728-21idleruntime-cro12:03:223
339950,0migration/38954-21munin-node07:03:443
339950,0migration/34789-21munin-node09:43:443
339950,0migration/33665-21taskset07:24:423
339950,0migration/332243-21taskset11:38:413
339950,0migration/330053-21irqrtprio09:18:473
339950,0migration/329042-21switchtime07:18:513
339950,0migration/329035-21taskset10:38:263
339950,0migration/329012-21runrttasks06:58:073
339950,0migration/328558-21runrttasks11:13:543
339950,0migration/323099-21munin-node07:38:493
339950,0migration/319120-21cpuspeed_turbos08:28:403
339950,0migration/318514-21taskset08:27:033
339950,0migration/317914-21switchtime10:08:523
339950,0migration/316976-21if_eth1.306:48:453
339950,0migration/313171-21sh11:23:593
339950,0migration/311098-21ntp_offset08:18:473
339950,0migration/311001-21aten2_r1power_v07:33:363
339950,0migration/310511-21taskset07:31:353
28855050,0irq/27-eth0-tx-31459-21kthreadcore09:18:482
28855050,0irq/27-eth0-tx-2526-21cat10:38:552
28855050,0irq/27-eth0-tx-20978-21munin-node08:53:522
28855050,0irq/27-eth0-tx-18177-21munin-node08:04:020
28855050,0irq/27-eth0-tx-17516-21munin-node07:38:460
28855050,0irq/27-eth0-tx-14504-21ssh08:48:460
28855050,0irq/27-eth0-tx-12722-21diskmemload11:28:412
28855050,0irq/27-eth0-tx-10627-21open_inodes10:43:462
28855050,0irq/27-eth0-tx-0-21swapper/212:10:392
28855050,0irq/27-eth0-tx-0-21swapper/211:53:322
28855050,0irq/27-eth0-tx-0-21swapper/211:45:572
28855050,0irq/27-eth0-tx-0-21swapper/210:48:442
28855050,0irq/27-eth0-tx-0-21swapper/210:34:482
28855050,0irq/27-eth0-tx-0-21swapper/209:58:492
28855050,0irq/27-eth0-tx-0-21swapper/209:53:392
28855050,0irq/27-eth0-tx-0-21swapper/208:58:242
28855050,0irq/27-eth0-tx-0-21swapper/008:38:520
28855050,0irq/27-eth0-tx-0-21swapper/007:43:450
28845050,0irq/26-eth0-rx-4852-21grep10:23:393
28845050,0irq/26-eth0-rx-27545-21munin-node10:53:573
28845050,0irq/26-eth0-rx-25936-21smartctl09:53:493
28845050,0irq/26-eth0-rx-20781-21grep07:13:433
28845050,0irq/26-eth0-rx-18785-21missed_timers10:48:503
28845050,0irq/26-eth0-rx-13818-21ntp_offset11:43:473
28845050,0irq/26-eth0-rx-11208-21irqrtprio12:03:463
28845050,0irq/26-eth0-rx-10317-21taskset10:04:283
28845050,0irq/26-eth0-rx-0-21swapper/312:13:463
28845050,0irq/26-eth0-rx-0-21swapper/312:10:593
28845050,0irq/26-eth0-rx-0-21swapper/311:53:443
28845050,0irq/26-eth0-rx-0-21swapper/311:33:343
28845050,0irq/26-eth0-rx-0-21swapper/311:28:463
28845050,0irq/26-eth0-rx-0-21swapper/310:43:423
28845050,0irq/26-eth0-rx-0-21swapper/310:30:463
28845050,0irq/26-eth0-rx-0-21swapper/310:22:333
28845050,0irq/26-eth0-rx-0-21swapper/309:59:593
28845050,0irq/26-eth0-rx-0-21swapper/309:38:443
28845050,0irq/26-eth0-rx-0-21swapper/309:34:293
28845050,0irq/26-eth0-rx-0-21swapper/309:28:303
28845050,0irq/26-eth0-rx-0-21swapper/309:08:563
28845050,0irq/26-eth0-rx-0-21swapper/309:05:223
28845050,0irq/26-eth0-rx-0-21swapper/309:01:193
28845050,0irq/26-eth0-rx-0-21swapper/308:55:103
259950,0migration/2558-21kthreadcore10:58:482
259950,0migration/211793-21kthreadcore11:23:502
259950,0migration/210134-21mdstat09:44:012
249950,0watchdog/26652-21kthreadcore09:03:492
179950,0migration/128329-21kernelversion12:13:451
179950,0migration/127293-21taskset09:56:351
179950,0migration/126900-21taskset07:18:461
169950,0watchdog/11324-21grep09:18:491
158429955,0cyclictest6039-21avahi-daemon07:13:462
158429955,0cyclictest12722-21diskmemload11:49:502
158429955,0cyclictest0-21swapper/210:10:302
158429955,0cyclictest0-21swapper/209:40:522
158429950,0cyclictest6039-21avahi-daemon12:14:452
158429950,0cyclictest6039-21avahi-daemon10:29:162
158429950,0cyclictest0-21swapper/211:43:132
158429950,0cyclictest0-21swapper/209:36:462
158349955,0cyclictest6039-21avahi-daemon10:25:561
158349955,0cyclictest6039-21avahi-daemon10:16:291
158349955,0cyclictest6039-21avahi-daemon09:25:521
158349955,0cyclictest6039-21avahi-daemon09:12:001
158349955,0cyclictest6039-21avahi-daemon08:51:111
158349955,0cyclictest0-21swapper/108:08:371
158349950,0cyclictest6039-21avahi-daemon11:37:311
158349950,0cyclictest0-21swapper/111:16:231
158339955,0cyclictest29852-21awk09:38:420
158339955,0cyclictest0-21swapper/009:08:090
158339950,0cyclictest0-21swapper/012:02:310
158339950,0cyclictest0-21swapper/011:50:410
149950,0migration/09831-21taskset07:07:180
149950,0migration/09708-21munin-node07:29:040
149950,0migration/07053-21kthreadcore09:43:590
149950,0migration/04124-21sh09:02:420
149950,0migration/03036-21sh10:39:210
149950,0migration/02883-21sh10:01:280
149950,0migration/028269-21sendmail_mailst06:53:530
149950,0migration/026221-21taskset12:12:190
149950,0migration/021074-21if_eth009:53:410
149950,0migration/019674-21sh10:49:180
149950,0migration/018246-21taskset08:23:530
149950,0migration/018035-21latency_hist08:03:230
149950,0migration/01790-21taskset12:16:060
149950,0migration/013661-21munin-node09:28:430
149950,0migration/012742-21taskset11:05:270
149950,0migration/010967-21timerandwakeup10:43:490
149950,0migration/010428-21timerandwakeup09:23:470
8305240,0sleep00-21swapper/011:43:380
339940,0migration/310688-21diskmemload09:23:513
28855040,0irq/27-eth0-tx-6039-21avahi-daemon10:23:312
28855040,0irq/27-eth0-tx-4816-21ssh11:19:022
28855040,0irq/27-eth0-tx-4508-21ssh09:23:282
28855040,0irq/27-eth0-tx-12684-21munin-node09:28:402
28855040,0irq/27-eth0-tx-0-21swapper/212:03:412
28855040,0irq/27-eth0-tx-0-21swapper/211:08:522
28855040,0irq/27-eth0-tx-0-21swapper/211:03:222
28855040,0irq/27-eth0-tx-0-21swapper/210:18:562
28855040,0irq/27-eth0-tx-0-21swapper/209:48:472
28855040,0irq/27-eth0-tx-0-21swapper/008:18:400
28855040,0irq/27-eth0-tx-0-21swapper/007:55:390
28855040,0irq/27-eth0-tx-0-21swapper/006:48:500
28845040,0irq/26-eth0-rx-6039-21avahi-daemon08:48:433
28845040,0irq/26-eth0-rx-0-21swapper/311:18:373
28845040,0irq/26-eth0-rx-0-21swapper/311:08:533
28845040,0irq/26-eth0-rx-0-21swapper/311:03:263
28845040,0irq/26-eth0-rx-0-21swapper/310:34:023
28845040,0irq/26-eth0-rx-0-21swapper/310:14:123
28845040,0irq/26-eth0-rx-0-21swapper/309:16:003
259940,0migration/23946-21munin-node07:48:522
259940,0migration/231927-21fschecks_time08:13:402
259940,0migration/224682-21fschecks_time07:43:432
259940,0migration/222881-21kthreadcore10:14:012
249940,0watchdog/218570-21grep08:03:492
249940,0watchdog/20-21swapper/207:23:542
179940,0migration/125249-21irqrtprio07:43:461
179940,0migration/121501-21kthreadcore12:08:471
179940,0migration/115580-21runrttasks11:46:261
179940,0migration/111728-21taskset10:08:171
169940,0watchdog/16924-21kthreadcore07:53:451
169940,0watchdog/16039-21avahi-daemon07:29:101
169940,0watchdog/127009-21ssh09:34:131
169940,0watchdog/117090-21sed07:13:381
169940,0watchdog/115966-21cut11:28:441
169940,0watchdog/113212-21kthreadcore07:08:421
169940,0watchdog/112722-21diskmemload10:49:571
169940,0watchdog/10-21swapper/111:18:361
169940,0watchdog/10-21swapper/111:08:451
169940,0watchdog/10-21swapper/110:58:491
169940,0watchdog/10-21swapper/108:23:371
169940,0watchdog/10-21swapper/107:33:411
158429943,0cyclictest6510-21awk08:18:422
158429943,0cyclictest5037-21awk07:28:552
158429943,0cyclictest3469-21lldpd08:50:122
158429943,0cyclictest3469-21lldpd08:28:112
158429943,0cyclictest3427-21snmpd11:58:422
158429943,0cyclictest3427-21snmpd09:09:122
158429943,0cyclictest3427-21snmpd08:38:272
158429943,0cyclictest3427-21snmpd08:28:422
158429943,0cyclictest3427-21snmpd08:11:122
158429943,0cyclictest3427-21snmpd07:59:422
158429943,0cyclictest3427-21snmpd07:55:422
158429943,0cyclictest3427-21snmpd07:43:122
158429943,0cyclictest26263-21awk08:33:382
158429943,0cyclictest24391-21awk11:33:412
158429943,0cyclictest21649-21awk09:13:442
158429943,0cyclictest11693-21awk07:33:392
158349944,0cyclictest6039-21avahi-daemon10:35:141
158349944,0cyclictest345-21kworker/1:108:42:561
158349944,0cyclictest12722-21diskmemload09:32:481
158349944,0cyclictest0-21swapper/110:44:321
158349943,0cyclictest475-21awk11:38:451
158349943,0cyclictest3649-21grep07:48:491
158349943,0cyclictest3469-21lldpd12:05:441
158349943,0cyclictest345-21kworker/1:108:13:421
158349943,0cyclictest3427-21snmpd11:06:421
158349943,0cyclictest3427-21snmpd09:59:271
158349943,0cyclictest31824-21sadc08:38:221
158349943,0cyclictest3154-21irqbalance11:27:311
158349943,0cyclictest3154-21irqbalance09:49:021
158349943,0cyclictest3154-21irqbalance09:14:321
158349943,0cyclictest3154-21irqbalance09:06:321
158349943,0cyclictest3154-21irqbalance09:00:221
158349943,0cyclictest3154-21irqbalance07:03:421
158349943,0cyclictest29878-21awk09:38:421
158349943,0cyclictest29141-21awk10:18:451
158349943,0cyclictest22512-21awk10:53:441
158349943,0cyclictest21834-21awk08:53:421
158349943,0cyclictest13373-21awk10:28:441
158349943,0cyclictest12912-21awk10:08:431
158339944,0cyclictest0-21swapper/011:19:000
158339944,0cyclictest0-21swapper/010:15:090
158339943,0cyclictest4839-21awk10:03:420
158339943,0cyclictest3469-21lldpd10:29:400
158339943,0cyclictest3469-21lldpd08:37:410
158339943,0cyclictest3469-21lldpd07:18:010
158339943,0cyclictest3427-21snmpd10:08:270
158339943,0cyclictest3154-21irqbalance11:56:510
158339943,0cyclictest3154-21irqbalance11:43:110
158339943,0cyclictest3154-21irqbalance11:33:410
158339943,0cyclictest3154-21irqbalance11:25:110
158339943,0cyclictest3154-21irqbalance10:23:310
158339943,0cyclictest3154-21irqbalance09:53:120
158339943,0cyclictest3154-21irqbalance09:34:020
158339943,0cyclictest3154-21irqbalance08:43:320
158339943,0cyclictest3154-21irqbalance07:48:420
158339943,0cyclictest3154-21irqbalance07:33:420
158339943,0cyclictest3154-21irqbalance07:01:320
158339943,0cyclictest30932-21awk10:58:450
158339943,0cyclictest30807-21awk07:23:480
158339943,0cyclictest25587-21awk08:08:390
158339943,0cyclictest24282-21awk07:18:440
158339943,0cyclictest22587-21awk10:53:440
158339943,0cyclictest10997-21awk07:08:390
158339943,0cyclictest10913-21awk12:03:440
8225030,0irq/24-0000:00:0-21swapper/207:03:232
339930,0migration/328670-21kthreadcore08:08:413
28855030,0irq/27-eth0-tx-6039-21avahi-daemon08:28:450
28845030,0irq/26-eth0-rx-0-21swapper/308:43:503
28845030,0irq/26-eth0-rx-0-21swapper/308:39:013
28845030,0irq/26-eth0-rx-0-21swapper/308:33:393
28845030,0irq/26-eth0-rx-0-21swapper/308:14:513
28845030,0irq/26-eth0-rx-0-21swapper/308:03:483
28845030,0irq/26-eth0-rx-0-21swapper/307:58:463
28845030,0irq/26-eth0-rx-0-21swapper/307:54:393
28845030,0irq/26-eth0-rx-0-21swapper/307:48:473
28845030,0irq/26-eth0-rx-0-21swapper/307:10:533
28845030,0irq/26-eth0-rx-0-21swapper/307:00:233
249930,0watchdog/20-21swapper/207:00:542
179930,0migration/129928-21fschecks_time06:58:431
158429932,0cyclictest23512-21if_err_eth1.306:53:452
158429932,0cyclictest23392-21basename07:18:372
158429932,0cyclictest10952-21awk07:08:392
158429930,0cyclictest0-21swapper/206:48:232
158349933,0cyclictest6039-21avahi-daemon11:53:261
158349933,0cyclictest345-21kworker/1:108:18:261
158349933,0cyclictest345-21kworker/1:108:04:051
158349933,0cyclictest345-21kworker/1:107:58:331
158349933,0cyclictest345-21kworker/1:107:23:331
158349933,0cyclictest345-21kworker/1:106:53:431
158349933,0cyclictest345-21kworker/1:106:48:491
158349933,0cyclictest21724-21kworker/1:011:58:231
158349933,0cyclictest21724-21kworker/1:011:48:301
158349933,0cyclictest21724-21kworker/1:010:38:261
158349933,0cyclictest21724-21kworker/1:009:43:441
158349932,0cyclictest345-21kworker/1:108:43:421
158349932,0cyclictest23377-21head07:38:501
158349932,0cyclictest19685-21awk08:28:431
158339931,0cyclictest0-21swapper/010:18:480
158339930,0cyclictest31375-21munin-node08:13:390
28845020,0irq/26-eth0-rx-0-21swapper/307:43:483
158429920,0cyclictest0-21swapper/208:43:222
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional