You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-16 - 15:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Tue Jul 16, 2024 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1220023929,0sleep10-21swapper/106:55:051
1231423727,0sleep30-21swapper/306:56:273
1210623627,0sleep20-21swapper/206:53:582
1241623423,0sleep00-21swapper/006:57:320
12503991313,0cyclictest6445-21snmpd11:51:001
125039988,0cyclictest6445-21snmpd08:50:291
339960,0migration/3761-21missed_timers10:43:103
339960,0migration/37127-21sh12:11:583
339960,0migration/34098-21sh10:05:013
339960,0migration/329768-21munin-node12:03:123
339960,0migration/312677-21sh10:12:153
28415060,0irq/27-eth0-tx-0-21swapper/212:18:162
28415060,0irq/27-eth0-tx-0-21swapper/212:12:162
28415060,0irq/27-eth0-tx-0-21swapper/212:05:422
28415060,0irq/27-eth0-tx-0-21swapper/212:01:562
28415060,0irq/27-eth0-tx-0-21swapper/211:51:022
28415060,0irq/27-eth0-tx-0-21swapper/211:43:482
28415060,0irq/27-eth0-tx-0-21swapper/211:35:482
28415060,0irq/27-eth0-tx-0-21swapper/211:22:102
28415060,0irq/27-eth0-tx-0-21swapper/211:00:322
28415060,0irq/27-eth0-tx-0-21swapper/210:32:542
28415060,0irq/27-eth0-tx-0-21swapper/210:32:422
28415060,0irq/27-eth0-tx-0-21swapper/210:18:032
28415060,0irq/27-eth0-tx-0-21swapper/210:15:502
28415060,0irq/27-eth0-tx-0-21swapper/210:11:122
28415060,0irq/27-eth0-tx-0-21swapper/209:59:402
28415060,0irq/27-eth0-tx-0-21swapper/209:38:082
28415060,0irq/27-eth0-tx-0-21swapper/209:19:332
28415060,0irq/27-eth0-tx-0-21swapper/209:04:442
28415060,0irq/27-eth0-tx-0-21swapper/007:58:260
28415060,0irq/27-eth0-tx-0-21swapper/007:16:520
28415060,0irq/27-eth0-tx-0-21swapper/006:58:210
28415060,0irq/27-eth0-tx-0-21swapper/006:58:210
28405060,0irq/26-eth0-rx-6124-21diskstats09:22:543
28405060,0irq/26-eth0-rx-27348-21perl11:03:053
28405060,0irq/26-eth0-rx-20404-21diskstats10:38:013
28405060,0irq/26-eth0-rx-15190-21df_inode09:07:563
28405060,0irq/26-eth0-rx-14628-21users11:53:273
249960,0watchdog/20-21swapper/210:53:262
249960,0watchdog/20-21swapper/209:27:302
179960,0migration/14704-21munin-node08:58:151
149960,0migration/08688-21sh11:32:090
149960,0migration/032046-21aten_r1power_en12:07:530
149960,0migration/017866-21rt-features10:33:150
149960,0migration/01394-21sh11:06:230
35150,0ktimersoftd/30-21swapper/309:28:023
339950,0migration/38089-21kthreadcore08:13:103
339950,0migration/34480-21irqrtprio10:48:073
339950,0migration/33625-21kthreadcore08:58:133
339950,0migration/328943-21taskset08:03:573
339950,0migration/326756-21latency_hist07:12:443
339950,0migration/325880-21kthreadcore12:23:143
339950,0migration/323641-21ntp_offset08:48:143
339950,0migration/323401-21munin-node09:13:063
339950,0migration/322845-21sh09:37:313
339950,0migration/320489-21kthreadcore11:58:083
339950,0migration/318676-21ntp_kernel_pll_10:13:193
339950,0migration/316960-21taskset11:37:533
339950,0migration/316759-21taskset08:19:503
339950,0migration/315307-21kthreadcore10:33:083
339950,0migration/313464-21gltestperf06:58:083
339950,0migration/313464-21gltestperf06:58:083
339950,0migration/310233-21missed_timers08:38:103
28415050,0irq/27-eth0-tx-32486-21if_eth108:58:080
28415050,0irq/27-eth0-tx-15279-21fschecks_count09:07:572
28415050,0irq/27-eth0-tx-0-21swapper/212:23:522
28415050,0irq/27-eth0-tx-0-21swapper/211:55:182
28415050,0irq/27-eth0-tx-0-21swapper/211:38:282
28415050,0irq/27-eth0-tx-0-21swapper/211:13:152
28415050,0irq/27-eth0-tx-0-21swapper/210:03:162
28415050,0irq/27-eth0-tx-0-21swapper/209:54:122
28415050,0irq/27-eth0-tx-0-21swapper/209:29:002
28415050,0irq/27-eth0-tx-0-21swapper/208:59:022
28415050,0irq/27-eth0-tx-0-21swapper/008:41:270
28415050,0irq/27-eth0-tx-0-21swapper/008:20:230
28415050,0irq/27-eth0-tx-0-21swapper/007:28:190
28405050,0irq/26-eth0-rx-5089-21tr11:28:063
28405050,0irq/26-eth0-rx-31355-21irqcore09:18:053
28405050,0irq/26-eth0-rx-22567-21memory08:23:173
28405050,0irq/26-eth0-rx-15889-21ssh11:35:093
28405050,0irq/26-eth0-rx-13360-21port_sshpriv12:13:173
28405050,0irq/26-eth0-rx-0-21swapper/312:18:253
28405050,0irq/26-eth0-rx-0-21swapper/311:46:173
28405050,0irq/26-eth0-rx-0-21swapper/310:53:043
28405050,0irq/26-eth0-rx-0-21swapper/310:27:593
28405050,0irq/26-eth0-rx-0-21swapper/310:22:463
28405050,0irq/26-eth0-rx-0-21swapper/310:21:503
28405050,0irq/26-eth0-rx-0-21swapper/309:58:123
28405050,0irq/26-eth0-rx-0-21swapper/309:48:063
28405050,0irq/26-eth0-rx-0-21swapper/309:45:223
249950,0watchdog/223897-21cut10:38:142
249950,0watchdog/222684-21grep08:48:102
249950,0watchdog/20-21swapper/212:13:422
249950,0watchdog/20-21swapper/211:28:222
249950,0watchdog/20-21swapper/211:24:222
249950,0watchdog/20-21swapper/211:07:542
249950,0watchdog/20-21swapper/211:02:462
249950,0watchdog/20-21swapper/210:43:102
249950,0watchdog/20-21swapper/209:48:062
249950,0watchdog/20-21swapper/209:33:222
249950,0watchdog/20-21swapper/209:13:382
249950,0watchdog/20-21swapper/208:52:542
249950,0watchdog/20-21swapper/208:41:422
249950,0watchdog/20-21swapper/208:32:462
249950,0watchdog/20-21swapper/208:30:022
249950,0watchdog/20-21swapper/208:23:022
249950,0watchdog/20-21swapper/208:07:582
249950,0watchdog/20-21swapper/207:31:302
249950,0watchdog/20-21swapper/207:13:062
249950,0watchdog/20-21swapper/206:58:022
249950,0watchdog/20-21swapper/206:58:022
179950,0migration/1784-21munin-node07:38:201
179950,0migration/17128-21sh11:28:151
179950,0migration/128205-21missed_timers08:03:091
179950,0migration/121845-21kthreadcore08:23:151
179950,0migration/121703-21hwlatdetect10:18:051
179950,0migration/120287-21timerwakeupswit07:28:251
179950,0migration/115722-21kthreadcore10:33:081
179950,0migration/110583-21kthreadcore10:08:111
169950,0watchdog/130614-21ps11:43:221
149950,0migration/09279-21taskset07:52:370
149950,0migration/08792-21taskset11:09:390
149950,0migration/07351-21kthreadcore10:48:110
149950,0migration/07273-21taskset07:20:070
149950,0migration/05475-21irqrtprio10:28:110
149950,0migration/03897-21switchtime09:18:190
149950,0migration/031715-21sh09:43:020
149950,0migration/028745-21kthreadcore08:28:080
149950,0migration/027437-21kthreadcore09:13:120
149950,0migration/025459-21kthreadcore10:18:120
149950,0migration/022105-21sh09:12:190
149950,0migration/021445-21aten_r1power_cu09:32:530
149950,0migration/020729-21sh09:31:250
149950,0migration/019553-21kthreadcore11:38:050
149950,0migration/018929-21munin-node10:13:260
149950,0migration/018814-21kthreadcore11:58:070
149950,0migration/017887-21taskset11:17:490
149950,0migration/016265-21taskset11:13:290
149950,0migration/01310-21timerandwakeup10:43:180
149950,0migration/011572-21kthreadcore11:33:120
149950,0migration/010750-21kthreadcore12:13:100
149950,0migration/010501-21kthreadcore09:03:080
125049954,0cyclictest3425-21lldpd07:35:182
125039955,0cyclictest0-21swapper/109:52:581
125039953,0cyclictest27453-21kworker/1:111:56:511
339940,0migration/332256-21kthreadcore07:38:123
339940,0migration/331212-21kthreadcore08:08:163
339940,0migration/314775-21kthreadcore07:53:153
339940,0migration/312368-21idleruntime08:43:063
28415040,0irq/27-eth0-tx-31237-21ssh09:42:582
28405040,0irq/26-eth0-rx-20134-21switchtime07:28:243
28405040,0irq/26-eth0-rx-0-21swapper/311:48:333
28405040,0irq/26-eth0-rx-0-21swapper/311:23:463
28405040,0irq/26-eth0-rx-0-21swapper/311:17:573
28405040,0irq/26-eth0-rx-0-21swapper/311:13:323
28405040,0irq/26-eth0-rx-0-21swapper/311:10:123
28405040,0irq/26-eth0-rx-0-21swapper/310:57:563
28405040,0irq/26-eth0-rx-0-21swapper/309:52:563
28405040,0irq/26-eth0-rx-0-21swapper/309:38:493
28405040,0irq/26-eth0-rx-0-21swapper/309:04:013
28405040,0irq/26-eth0-rx-0-21swapper/307:58:163
259940,0migration/25331-21kthreadcore07:48:132
249940,0watchdog/24843-21if_eth1.308:13:062
249940,0watchdog/20-21swapper/210:23:022
249940,0watchdog/20-21swapper/208:18:022
249940,0watchdog/20-21swapper/208:03:022
249940,0watchdog/20-21swapper/207:58:102
249940,0watchdog/20-21swapper/207:53:142
249940,0watchdog/20-21swapper/207:44:302
249940,0watchdog/20-21swapper/207:23:102
249940,0watchdog/20-21swapper/207:18:262
249940,0watchdog/20-21swapper/207:10:422
249940,0watchdog/20-21swapper/207:04:182
179940,0migration/19089-21kthreadcore09:23:031
179940,0migration/17800-21kthreadcore08:13:101
179940,0migration/15046-21kthreadcore07:48:131
179940,0migration/13991-21kthreadcore11:08:081
179940,0migration/132651-21runrttasks08:08:161
179940,0migration/131778-21kthreadcore10:23:091
179940,0migration/126900-21cpu11:03:031
179940,0migration/126398-21missed_timers07:33:081
179940,0migration/12540-21kthreadcore08:33:111
179940,0migration/124333-21missed_timers10:58:131
179940,0migration/12077-21kthreadcore12:08:071
179940,0migration/118010-21kthreadcore12:18:061
169940,0watchdog/17374-21munin-node09:48:021
169940,0watchdog/132474-21kthreadcore09:18:061
169940,0watchdog/131382-21grep07:13:181
169940,0watchdog/130983-21ssh12:27:221
169940,0watchdog/126213-21grep08:28:061
169940,0watchdog/120154-21ssh11:38:061
169940,0watchdog/116047-21sed09:08:061
169940,0watchdog/10-21swapper/110:43:101
169940,0watchdog/10-21swapper/108:18:181
149940,0migration/031403-21kthreadcore10:03:110
149940,0migration/030264-21sh09:41:260
149940,0migration/023186-21df08:02:560
149940,0migration/022267-21df09:57:570
149940,0migration/019979-21kthreadcore08:48:090
149940,0migration/01377-21kthreadcore10:23:100
149940,0migration/011782-21kthreadcore07:23:070
125049943,0cyclictest3110-21irqbalance10:47:582
125049943,0cyclictest28564-21awk07:38:042
125049943,0cyclictest12396-21awk08:43:062
125039944,0cyclictest0-21swapper/111:14:571
125039944,0cyclictest0-21swapper/109:13:141
125039943,0cyclictest6445-21snmpd07:01:061
125039943,0cyclictest6445-21snmpd07:01:061
125039943,0cyclictest4219-21awk10:48:041
125039943,0cyclictest3425-21lldpd11:17:441
125039943,0cyclictest3425-21lldpd10:28:501
125039943,0cyclictest3425-21lldpd09:27:481
125039943,0cyclictest3425-21lldpd07:57:561
125039943,0cyclictest3425-21lldpd07:43:181
125039943,0cyclictest3425-21lldpd07:08:191
125039943,0cyclictest27453-21kworker/1:112:13:271
125039943,0cyclictest27453-21kworker/1:112:03:591
125039943,0cyclictest27453-21kworker/1:112:00:291
125039943,0cyclictest27453-21kworker/1:111:22:471
125039943,0cyclictest27453-21kworker/1:110:54:331
125039943,0cyclictest27453-21kworker/1:110:15:511
125039943,0cyclictest27453-21kworker/1:110:04:071
125039943,0cyclictest27453-21kworker/1:109:43:311
125039943,0cyclictest27453-21kworker/1:109:40:351
125039943,0cyclictest27453-21kworker/1:109:35:191
125039943,0cyclictest27453-21kworker/1:109:03:031
125039943,0cyclictest27453-21kworker/1:108:43:251
125039943,0cyclictest27453-21kworker/1:107:52:591
125039943,0cyclictest20313-21awk07:03:071
125039943,0cyclictest10078-21awk11:33:051
125029944,0cyclictest0-21swapper/012:25:340
125029944,0cyclictest0-21swapper/012:04:260
125029944,0cyclictest0-21swapper/011:25:580
125029944,0cyclictest0-21swapper/009:56:250
125029944,0cyclictest0-21swapper/008:23:050
125029943,0cyclictest6445-21snmpd10:09:220
125029943,0cyclictest4771-21awk08:13:050
125029943,0cyclictest31469-21awk08:33:040
125029943,0cyclictest20956-21awk10:38:090
125029943,0cyclictest19710-21awk10:58:050
125029943,0cyclictest16066-21sadc11:57:430
125029943,0cyclictest11776-21head10:52:580
4130,0ktimersoftd/02362-21runrttasks07:46:130
28415030,0irq/27-eth0-tx-29535-21aten2.4-expect08:07:540
28415030,0irq/27-eth0-tx-0-21swapper/007:03:140
28405030,0irq/26-eth0-rx-0-21swapper/308:55:303
28405030,0irq/26-eth0-rx-0-21swapper/308:27:543
28405030,0irq/26-eth0-rx-0-21swapper/307:48:093
28405030,0irq/26-eth0-rx-0-21swapper/307:43:033
28405030,0irq/26-eth0-rx-0-21swapper/307:32:553
28405030,0irq/26-eth0-rx-0-21swapper/307:22:553
28405030,0irq/26-eth0-rx-0-21swapper/307:21:283
28405030,0irq/26-eth0-rx-0-21swapper/307:04:043
149930,0migration/012805-21sh09:49:070
125039933,0cyclictest27453-21kworker/1:110:37:471
125039933,0cyclictest27453-21kworker/1:109:58:031
125039933,0cyclictest27453-21kworker/1:108:37:591
125039933,0cyclictest27453-21kworker/1:107:17:531
125039932,0cyclictest8509-21hddtemp_smartct07:22:591
125039932,0cyclictest25589-21grep08:53:001
125029933,0cyclictest0-21swapper/007:08:310
125029932,0cyclictest6646-21awk09:22:590
125029932,0cyclictest6445-21snmpd11:47:220
125029932,0cyclictest11666-21basename08:42:590
125029932,0cyclictest10374-21awk07:53:050
125029930,0cyclictest24849-21munin-node08:53:000
125029930,0cyclictest0-21swapper/012:21:220
125029930,0cyclictest0-21swapper/007:38:150
28405020,0irq/26-eth0-rx-0-21swapper/308:32:543
28405020,0irq/26-eth0-rx-0-21swapper/307:08:033
125029922,0cyclictest3425-21lldpd11:47:490
125029920,0cyclictest20982-21latency_hist07:32:440
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional