You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-08-09 - 16:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.4.229-rt200 (Profile)

Latency plot of shadow in rack #1, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -n -a -t3 -p99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rack1slot2s.osadl.org (updated Tue Aug 09, 2022 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
396324433,0sleep00-21swapper/006:53:400
382324031,0sleep20-21swapper/206:51:582
379323724,0sleep10-21swapper/106:51:411
383422824,0sleep30-21swapper/306:52:073
423199130,0cyclictest0-21swapper/207:10:542
42319970,0cyclictest0-21swapper/208:56:142
42289977,0cyclictest0-21swapper/007:14:340
42299960,0cyclictest1095-21diskmemload09:59:091
42299960,0cyclictest0-21swapper/108:41:161
42289965,0cyclictest6641-21kworker/0:110:04:450
339960,0migration/37666-21munin-node08:36:073
339960,0migration/318440-21sh11:40:533
339960,0migration/317289-21sh09:22:363
339960,0migration/310659-21sh11:54:553
28715060,0irq/26-eth0-rx-7048-21open_inodes08:56:183
28715060,0irq/26-eth0-rx-5502-21if_eth009:56:273
28715060,0irq/26-eth0-rx-3652-21cut10:31:143
28715060,0irq/26-eth0-rx-26795-21ntp_states10:06:173
28715060,0irq/26-eth0-rx-25880-21python09:46:223
28715060,0irq/26-eth0-rx-14901-21kthreadcore10:01:153
28715060,0irq/26-eth0-rx-13099-21perl12:16:123
28715060,0irq/26-eth0-rx-12359-21munin-node10:36:173
28715060,0irq/26-eth0-rx-11166-21ssh12:13:213
179960,0migration/126057-21sh11:44:371
149960,0migration/025266-21sh11:25:300
42299955,0cyclictest0-21swapper/110:56:321
42299955,0cyclictest0-21swapper/109:16:061
42299954,0cyclictest32220-21kthreadcore09:31:141
42299950,0cyclictest0-21swapper/106:56:061
42289955,0cyclictest6641-21kworker/0:112:21:340
42289955,0cyclictest6641-21kworker/0:112:16:080
42289955,0cyclictest6641-21kworker/0:112:11:100
42289955,0cyclictest6641-21kworker/0:112:05:500
42289955,0cyclictest6641-21kworker/0:112:01:020
42289955,0cyclictest6641-21kworker/0:111:55:500
42289955,0cyclictest6641-21kworker/0:111:50:560
42289955,0cyclictest6641-21kworker/0:111:46:000
42289955,0cyclictest6641-21kworker/0:111:40:580
42289955,0cyclictest6641-21kworker/0:111:36:380
42289955,0cyclictest6641-21kworker/0:111:31:280
42289955,0cyclictest6641-21kworker/0:111:26:420
42289955,0cyclictest6641-21kworker/0:111:16:190
42289955,0cyclictest6641-21kworker/0:111:11:170
42289955,0cyclictest6641-21kworker/0:111:06:130
42289955,0cyclictest6641-21kworker/0:111:01:110
42289955,0cyclictest6641-21kworker/0:110:56:350
42289955,0cyclictest6641-21kworker/0:110:51:130
42289955,0cyclictest6641-21kworker/0:110:47:050
42289955,0cyclictest6641-21kworker/0:110:41:410
42289955,0cyclictest6641-21kworker/0:110:36:030
42289955,0cyclictest6641-21kworker/0:110:30:530
42289955,0cyclictest6641-21kworker/0:110:26:130
42289955,0cyclictest6641-21kworker/0:110:21:190
42289955,0cyclictest6641-21kworker/0:110:15:510
42289955,0cyclictest6641-21kworker/0:110:06:090
42289955,0cyclictest6641-21kworker/0:109:56:290
42289955,0cyclictest6641-21kworker/0:109:51:050
42289955,0cyclictest6641-21kworker/0:109:46:070
42289955,0cyclictest6641-21kworker/0:109:35:590
42289955,0cyclictest6641-21kworker/0:109:35:590
42289955,0cyclictest6641-21kworker/0:109:30:590
42289955,0cyclictest6641-21kworker/0:109:26:390
42289955,0cyclictest6641-21kworker/0:109:22:010
42289955,0cyclictest6641-21kworker/0:109:15:530
42289955,0cyclictest6641-21kworker/0:109:10:530
42289955,0cyclictest6641-21kworker/0:109:05:550
42289955,0cyclictest6641-21kworker/0:109:00:530
42289955,0cyclictest6641-21kworker/0:108:56:130
42289955,0cyclictest6641-21kworker/0:108:51:290
42289955,0cyclictest6641-21kworker/0:108:45:550
42289955,0cyclictest6641-21kworker/0:108:40:570
42289955,0cyclictest6641-21kworker/0:108:36:330
42289955,0cyclictest6641-21kworker/0:108:30:570
42289955,0cyclictest6641-21kworker/0:108:25:530
42289955,0cyclictest6641-21kworker/0:108:20:530
42289955,0cyclictest6641-21kworker/0:108:16:010
42289955,0cyclictest6641-21kworker/0:108:10:530
42289955,0cyclictest6641-21kworker/0:108:05:530
42289955,0cyclictest6641-21kworker/0:108:01:010
42289955,0cyclictest6641-21kworker/0:107:55:530
42289955,0cyclictest6641-21kworker/0:107:50:550
42289955,0cyclictest5237-21kworker/0:207:45:590
42289955,0cyclictest5237-21kworker/0:207:40:530
42289955,0cyclictest5237-21kworker/0:207:35:570
42289955,0cyclictest5237-21kworker/0:207:30:570
42289955,0cyclictest5237-21kworker/0:207:25:530
42289955,0cyclictest5237-21kworker/0:207:21:010
42289955,0cyclictest5237-21kworker/0:207:15:550
42289955,0cyclictest5237-21kworker/0:207:05:530
42289955,0cyclictest5237-21kworker/0:207:00:550
42289955,0cyclictest5237-21kworker/0:206:55:530
339950,0migration/3938-21taskset09:13:473
339950,0migration/36433-21taskset10:16:143
339950,0migration/33751-21munin-node07:16:163
339950,0migration/331857-21taskset10:51:023
339950,0migration/327466-21kthreadcore07:11:153
339950,0migration/322864-21kthreadcore09:26:153
339950,0migration/322086-21kthreadcore12:01:133
339950,0migration/322021-21kthreadcore08:46:113
339950,0migration/320896-21kthreadcore11:21:153
339950,0migration/31704-21irqrtprio11:11:143
339950,0migration/315382-21cpuspeed_turbos10:41:063
339950,0migration/314691-21gltestperf10:21:123
339950,0migration/311604-21kthreadcore11:16:143
339950,0migration/311207-21mdstat07:46:143
329950,0watchdog/325755-21sh11:26:063
329950,0watchdog/32535-21munin-node11:31:103
329950,0watchdog/32177-21ssh09:16:073
329950,0watchdog/318393-21kthreadcore08:41:113
329950,0watchdog/317719-21munin-node07:06:113
329950,0watchdog/310814-21kthreadcore08:11:113
329950,0watchdog/30-21swapper/311:56:103
329950,0watchdog/30-21swapper/310:11:113
329950,0watchdog/30-21swapper/307:56:073
28725050,0irq/27-eth0-tx-515-21aten2_r1power_c11:11:072
28715050,0irq/26-eth0-rx-367-21cat11:46:213
28715050,0irq/26-eth0-rx-3190-21cat09:35:513
28715050,0irq/26-eth0-rx-3190-21cat09:35:513
28715050,0irq/26-eth0-rx-25589-21interrupts11:06:113
28715050,0irq/26-eth0-rx-23251-21kthreadcore12:21:143
28715050,0irq/26-eth0-rx-14418-21ssh10:57:173
28715050,0irq/26-eth0-rx-1335-21kthreadcore09:51:183
28715050,0irq/26-eth0-rx-0-21swapper/312:06:203
28715050,0irq/26-eth0-rx-0-21swapper/311:36:193
28715050,0irq/26-eth0-rx-0-21swapper/311:01:523
28715050,0irq/26-eth0-rx-0-21swapper/310:45:543
28715050,0irq/26-eth0-rx-0-21swapper/310:28:113
28715050,0irq/26-eth0-rx-0-21swapper/309:01:163
249950,0watchdog/225596-21latency_hist11:25:502
249950,0watchdog/225139-21cut10:06:152
249950,0watchdog/219470-21idleruntime-cro12:00:502
249950,0watchdog/217263-21latency_hist11:20:502
249950,0watchdog/20-21swapper/212:16:182
179950,0migration/18798-21munin-node11:51:231
179950,0migration/128251-21kthreadcore08:01:121
179950,0migration/126968-21latency_hist08:50:511
179950,0migration/123954-21kthreadcore10:26:141
179950,0migration/119681-21latency_hist08:20:511
179950,0migration/118243-21sh09:06:111
169950,0watchdog/1437-21grep08:06:071
149950,0migration/030295-21gltestperf10:11:110
149950,0migration/017877-21sh09:41:270
42319943,0cyclictest5027-21awk12:11:122
42319943,0cyclictest3457-21lldpd10:23:042
42319943,0cyclictest3457-21lldpd08:43:052
42319943,0cyclictest3457-21lldpd08:31:052
42319943,0cyclictest3457-21lldpd08:19:352
42319943,0cyclictest3415-21snmpd12:21:092
42319943,0cyclictest3415-21snmpd12:00:542
42319943,0cyclictest3415-21snmpd10:15:532
42319943,0cyclictest3415-21snmpd08:53:232
42319943,0cyclictest3415-21snmpd07:46:082
42319943,0cyclictest20928-21awk09:46:132
42319943,0cyclictest19945-21awk09:26:132
42319943,0cyclictest16073-21awk10:41:102
42319943,0cyclictest12990-21sadc08:15:512
42319943,0cyclictest12703-21awk07:26:112
42319943,0cyclictest11930-21awk07:01:082
42299944,0cyclictest1095-21diskmemload11:17:301
42299944,0cyclictest1095-21diskmemload11:06:091
42299944,0cyclictest0-21swapper/109:47:101
42299944,0cyclictest0-21swapper/109:26:591
42299944,0cyclictest0-21swapper/107:18:481
42299943,0cyclictest4163-21awk09:36:131
42299943,0cyclictest4163-21awk09:36:131
42299943,0cyclictest3457-21lldpd11:37:051
42299943,0cyclictest3457-21lldpd10:03:051
42299943,0cyclictest3457-21lldpd07:43:051
42299943,0cyclictest3415-21snmpd07:50:231
42299943,0cyclictest32691-21awk10:51:101
42299943,0cyclictest31551-21awk10:31:091
42299943,0cyclictest27785-21awk11:46:121
42299943,0cyclictest27353-21awk08:26:101
42299943,0cyclictest25921-21awk07:36:101
42299943,0cyclictest25186-21awk07:11:131
42299943,0cyclictest2249-21awk08:56:111
42299943,0cyclictest21840-21sort10:06:091
42299943,0cyclictest19204-21awk07:31:081
339940,0migration/36631-21kthreadcore07:21:123
339940,0migration/332716-21taskset08:30:103
339940,0migration/32999-21kthreadcore07:41:113
339940,0migration/326210-21idleruntime-cro08:25:513
339940,0migration/319167-21uptime08:16:183
339940,0migration/318237-21switchtime07:51:193
329940,0watchdog/30-21swapper/309:40:593
329940,0watchdog/30-21swapper/308:06:073
329940,0watchdog/30-21swapper/307:01:113
329940,0watchdog/30-21swapper/306:56:113
28725040,0irq/27-eth0-tx-5292-21wget07:21:072
28725040,0irq/27-eth0-tx-2624-21ssh12:08:072
28725040,0irq/27-eth0-tx-24265-21ssh11:41:192
28725040,0irq/27-eth0-tx-14444-21kthreadcore09:21:152
28725040,0irq/27-eth0-tx-1095-21diskmemload11:06:402
28725040,0irq/27-eth0-tx-1082-21ssh09:31:312
28725040,0irq/27-eth0-tx-0-21swapper/211:51:082
28725040,0irq/27-eth0-tx-0-21swapper/211:32:572
28725040,0irq/27-eth0-tx-0-21swapper/211:16:152
28725040,0irq/27-eth0-tx-0-21swapper/211:01:112
28725040,0irq/27-eth0-tx-0-21swapper/210:56:142
28725040,0irq/27-eth0-tx-0-21swapper/210:46:142
28725040,0irq/27-eth0-tx-0-21swapper/210:26:082
28725040,0irq/27-eth0-tx-0-21swapper/210:01:092
28725040,0irq/27-eth0-tx-0-21swapper/209:02:332
28725040,0irq/27-eth0-tx-0-21swapper/207:31:072
28715040,0irq/26-eth0-rx-0-21swapper/309:31:113
28715040,0irq/26-eth0-rx-0-21swapper/309:05:563
28715040,0irq/26-eth0-rx-0-21swapper/307:30:523
259940,0migration/27945-21kthreadcore09:36:162
259940,0migration/27945-21kthreadcore09:36:162
259940,0migration/23211-21munin-node09:16:142
259940,0migration/227737-21munin-node11:46:112
259940,0migration/221381-21kthreadcore09:06:172
249940,0watchdog/28350-21kthreadcore09:56:312
249940,0watchdog/28185-21kthreadcore06:56:112
249940,0watchdog/25824-21ssh10:51:462
249940,0watchdog/2436-21tune2fs08:06:072
249940,0watchdog/230496-21port_sshpriv07:36:152
249940,0watchdog/230032-21fschecks_count10:11:112
249940,0watchdog/2116942sleep209:40:592
249940,0watchdog/211192-21kthreadcore10:36:142
249940,0watchdog/21095-21diskmemload09:10:552
249940,0watchdog/20-21swapper/211:36:142
249940,0watchdog/20-21swapper/210:32:022
249940,0watchdog/20-21swapper/209:51:232
249940,0watchdog/20-21swapper/208:46:112
249940,0watchdog/20-21swapper/208:36:072
249940,0watchdog/20-21swapper/207:56:072
179940,0migration/15168-21runrttasks07:20:531
179940,0migration/13218-21kthreadcore11:11:151
179940,0migration/131548-21load09:11:151
179940,0migration/13057-21sh12:09:071
179940,0migration/130134-21kthreadcore08:51:141
179940,0migration/125911-21kthreadcore10:46:151
179940,0migration/124991-21latency08:21:111
179940,0migration/119454-21kthreadcore10:41:141
179940,0migration/117961-21kthreadcore11:01:121
179940,0migration/117615-21latency_hist07:05:521
179940,0migration/114466-21kthreadcore07:51:131
179940,0migration/113552-21kthreadcore09:01:181
179940,0migration/112828-21taskset08:14:081
179940,0migration/112676-21if_eth1.209:41:121
179940,0migration/112641-21kthreadcore09:21:141
169940,0watchdog/18063-21rm11:31:421
169940,0watchdog/125595-21latency_hist11:25:501
169940,0watchdog/119205-21wget07:56:071
169940,0watchdog/111618-21cpuspeed_turbos11:56:101
169940,0watchdog/11095-21diskmemload09:52:591
169940,0watchdog/110343-21users12:11:221
169940,0watchdog/10-21swapper/112:16:181
169940,0watchdog/10-21swapper/110:36:141
169940,0watchdog/10-21swapper/110:11:111
169940,0watchdog/10-21swapper/108:36:071
169940,0watchdog/10-21swapper/107:01:111
42319933,0cyclictest3457-21lldpd07:17:352
42319932,0cyclictest3415-21snmpd08:26:532
42319932,0cyclictest18152-21sensors07:51:182
42319930,0cyclictest19823-21munin-node08:21:042
42299933,0cyclictest1095-21diskmemload12:00:511
42299932,0cyclictest6810-21ksmtuned08:32:481
42299932,0cyclictest12615-21awk07:26:101
42299930,0cyclictest21397-21grep12:21:091
329930,0watchdog/30-21swapper/307:36:273
28715030,0irq/26-eth0-rx-28387-21kthreadcore08:51:133
28715030,0irq/26-eth0-rx-0-21swapper/308:33:303
28715030,0irq/26-eth0-rx-0-21swapper/308:01:153
28715030,0irq/26-eth0-rx-0-21swapper/307:25:573
249930,0watchdog/225750-21munin-node08:01:072
249930,0watchdog/20-21swapper/207:41:112
169930,0watchdog/10-21swapper/111:20:501
169930,0watchdog/10-21swapper/110:22:311
169930,0watchdog/10-21swapper/110:16:031
42319921,0cyclictest17618-21cat07:05:522
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional