You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-27 - 04:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Sat Jul 27, 2024 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
291350540,0irq/25-eth10-21swapper/318:54:263
2677623430,0sleep20-21swapper/218:57:002
2652423323,0sleep10-21swapper/118:54:031
2042923221,0sleep00-21swapper/018:52:260
268759972,0cyclictest9104-21munin-node22:47:322
268759972,0cyclictest17612-21munin-node23:32:392
268759971,0cyclictest271ktimersoftd/200:05:332
4160,0ktimersoftd/00-21swapper/022:43:220
28415060,0irq/27-eth0-tx-0-21swapper/020:57:420
28405060,0irq/26-eth0-rx-9909-21diskstats00:07:313
28405060,0irq/26-eth0-rx-26841-21tr21:37:403
28405060,0irq/26-eth0-rx-1234-21munin-run23:42:193
28405060,0irq/26-eth0-rx-10731-21iostat_ios23:47:373
268759962,0cyclictest9464-21cut21:07:312
268759962,0cyclictest9352-21grep19:52:442
268759962,0cyclictest8581-21cut19:07:332
268759962,0cyclictest7023-21which23:02:452
268759962,0cyclictest5168-21kthreadcore22:02:382
268759962,0cyclictest30255-21cut22:57:382
268759962,0cyclictest26335-21perl23:17:362
268759962,0cyclictest2577-21grep23:42:382
268759962,0cyclictest25759-21munin-node23:37:432
268759962,0cyclictest25007-21munin-run20:57:202
268759962,0cyclictest24753-21perf22:17:192
268759962,0cyclictest22737-21kthreadcore19:17:362
268759962,0cyclictest19215-21kthreadcore21:52:422
268759962,0cyclictest17548-21munin-node00:12:382
268759962,0cyclictest17451-21pmu-power20:47:452
268759962,0cyclictest14911-21sed22:07:482
268759962,0cyclictest11204-21cut19:32:352
268759962,0cyclictest10500-21sed23:47:342
268759961,0cyclictest0-21swapper/222:32:412
268759961,0cyclictest0-21swapper/222:27:372
268759961,0cyclictest0-21swapper/222:17:402
268759961,0cyclictest0-21swapper/221:37:402
268759960,0cyclictest0-21swapper/220:58:252
268749965,0cyclictest15801-21tr19:12:381
268749960,0cyclictest0-21swapper/123:47:281
268739966,0cyclictest0-21swapper/020:12:390
149960,0migration/024360-21sh23:53:590
149960,0migration/02066-21if_eth023:02:360
149960,0migration/017727-21fschecks_time22:32:330
35150,0ktimersoftd/30-21swapper/321:48:113
339950,0migration/35086-21kthreadcore22:22:443
339950,0migration/34895-21kthreadcore00:02:373
339950,0migration/34117-21kthreadcore19:27:393
339950,0migration/33886-21fschecks_count19:52:333
339950,0migration/331218-21smart_sdb22:57:433
339950,0migration/331117-21sh21:57:563
339950,0migration/330247-21irqrtprio19:47:393
339950,0migration/328528-21munin-node21:17:573
339950,0migration/32835-21kthreadcore23:02:403
339950,0migration/328197-21kthreadcore22:17:393
339950,0migration/324154-21kthreadcore19:42:403
339950,0migration/324012-21munin-node20:52:423
339950,0migration/318087-21idleruntime22:52:343
339950,0migration/317883-21idleruntime22:32:343
339950,0migration/31637-21fschecks_count00:22:323
339950,0migration/315900-21taskset21:31:143
339950,0migration/315210-21fschecks_count19:12:343
339950,0migration/310605-21taskset20:20:283
339950,0migration/310306-21munin-node23:27:313
28415050,0irq/27-eth0-tx-18601-21munin-node20:52:330
28415050,0irq/27-eth0-tx-0-21swapper/020:20:010
28415050,0irq/27-eth0-tx-0-21swapper/019:20:520
28405050,0irq/26-eth0-rx-4670-21kthreadcore23:42:403
28405050,0irq/26-eth0-rx-30520-21kthreadcore23:17:433
28405050,0irq/26-eth0-rx-27398-21kthreadcore00:17:353
28405050,0irq/26-eth0-rx-26002-21gltestperf20:57:333
28405050,0irq/26-eth0-rx-20197-21kthreadcore23:52:383
28405050,0irq/26-eth0-rx-18346-21hostname00:12:323
28405050,0irq/26-eth0-rx-16258-21ssh22:30:573
28405050,0irq/26-eth0-rx-0-21swapper/323:34:073
28405050,0irq/26-eth0-rx-0-21swapper/323:07:403
28405050,0irq/26-eth0-rx-0-21swapper/322:47:193
28405050,0irq/26-eth0-rx-0-21swapper/322:37:363
28405050,0irq/26-eth0-rx-0-21swapper/322:12:293
28405050,0irq/26-eth0-rx-0-21swapper/322:07:473
28405050,0irq/26-eth0-rx-0-21swapper/321:52:423
28405050,0irq/26-eth0-rx-0-21swapper/321:32:463
28405050,0irq/26-eth0-rx-0-21swapper/321:15:313
28405050,0irq/26-eth0-rx-0-21swapper/321:07:393
28405050,0irq/26-eth0-rx-0-21swapper/321:02:323
268759952,0cyclictest9423-21cat23:27:192
268759952,0cyclictest9405-21cat00:07:192
268759952,0cyclictest93812sleep223:07:312
268759952,0cyclictest8941-21expr21:47:312
268759952,0cyclictest8675-21expr21:27:312
268759952,0cyclictest573-21sort21:42:312
268759952,0cyclictest553-21df_abs21:22:312
268759952,0cyclictest5207-21sed20:42:202
268759952,0cyclictest4513-21cpuspeed_turbos20:17:322
268759952,0cyclictest30863-21ls20:37:202
268759952,0cyclictest29664-21tune2fs19:47:342
268759952,0cyclictest28305-21cpuspeed_turbos19:22:322
268759952,0cyclictest27823-21expr18:57:332
268759952,0cyclictest25914-21tail22:37:312
268759952,0cyclictest25869-21aten_r1power_en23:57:322
268759952,0cyclictest25735-21fschecks_count21:17:392
268759952,0cyclictest23428-21aten_r1power_po20:07:312
268759952,0cyclictest2328-21aten2.4-expect19:27:312
268759952,0cyclictest22497-21aten2.4-expect19:42:312
268759952,0cyclictest17729-21expr23:12:302
268759952,0cyclictest17493-21cpuspeed_turbos22:52:312
268759952,0cyclictest17018-21df20:02:332
268759952,0cyclictest15721-21cat19:37:202
268759952,0cyclictest1505-21aten_r1power_vo19:02:312
268759952,0cyclictest1478-21cpu23:22:302
268759952,0cyclictest1366-21expr00:22:302
268759952,0cyclictest11488-21fschecks_count20:22:322
268759952,0cyclictest1127-21awk22:42:332
268759952,0cyclictest10340-21expr19:57:312
268759951,0cyclictest82250irq/24-0000:00:20:27:202
268759951,0cyclictest25313-21sh00:17:202
268759951,0cyclictest24250-21cat20:32:202
268759951,0cyclictest16494-21latency_hist21:32:202
268759951,0cyclictest1068-21awk21:02:312
268759951,0cyclictest0-21swapper/223:52:292
268759951,0cyclictest0-21swapper/222:22:212
268759951,0cyclictest0-21swapper/221:57:272
268759951,0cyclictest0-21swapper/221:12:412
268759951,0cyclictest0-21swapper/219:12:352
268759950,0cyclictest0-21swapper/220:12:322
268749955,0cyclictest0-21swapper/123:34:131
268749955,0cyclictest0-21swapper/122:26:071
268749950,0cyclictest0-21swapper/122:59:231
268739955,0cyclictest0-21swapper/022:26:200
268739955,0cyclictest0-21swapper/019:47:440
19150,0ktimersoftd/10-21swapper/121:48:111
19150,0ktimersoftd/10-21swapper/121:38:021
179950,0migration/19025-21sh00:06:451
179950,0migration/129620-21sh23:37:381
179950,0migration/125077-21latency_hist20:57:201
179950,0migration/119383-21kthreadcore20:27:401
179950,0migration/115270-21munin-node00:07:401
169950,0watchdog/127493-21grep22:17:381
169950,0watchdog/118826-21ps19:37:391
149950,0migration/030869-21sh21:57:430
149950,0migration/030269-21kthreadcore19:22:390
149950,0migration/029657-21needreboot20:32:390
149950,0migration/028428-21taskset19:43:020
149950,0migration/026586-21irqrtprio22:57:370
149950,0migration/026412-21kthreadcore22:17:380
149950,0migration/025295-21aten_r1power_en21:17:320
149950,0migration/024074-21sh23:13:560
149950,0migration/020789-21kthreadcore23:32:360
149950,0migration/019426-21kthreadcore22:12:420
149950,0migration/016208-21runrttasks23:48:350
149950,0migration/015553-21pmu-power19:57:430
149950,0migration/015211-21sh21:29:220
149950,0migration/015005-21munin-node23:07:440
149950,0migration/014253-21kthreadcore22:27:380
149950,0migration/011823-21kthreadcore00:07:360
149950,0migration/011666-21fschecks_time20:22:320
339940,0migration/36098-21kthreadcore19:02:383
339940,0migration/327426-21kthreadcore23:57:403
339940,0migration/31175-21sh21:42:333
28415040,0irq/27-eth0-tx-0-21swapper/020:51:120
28415040,0irq/27-eth0-tx-0-21swapper/019:52:300
28415040,0irq/27-eth0-tx-0-21swapper/019:32:320
28415040,0irq/27-eth0-tx-0-21swapper/019:27:320
28415040,0irq/27-eth0-tx-0-21swapper/018:57:310
28405040,0irq/26-eth0-rx-3600-21kthreadcore20:37:383
28405040,0irq/26-eth0-rx-0-21swapper/323:22:353
28405040,0irq/26-eth0-rx-0-21swapper/323:12:203
28405040,0irq/26-eth0-rx-0-21swapper/322:42:493
28405040,0irq/26-eth0-rx-0-21swapper/322:02:583
28405040,0irq/26-eth0-rx-0-21swapper/321:22:293
28405040,0irq/26-eth0-rx-0-21swapper/320:42:303
28405040,0irq/26-eth0-rx-0-21swapper/320:32:373
28405040,0irq/26-eth0-rx-0-21swapper/319:32:433
28405040,0irq/26-eth0-rx-0-21swapper/319:07:313
268749944,0cyclictest0-21swapper/122:27:321
268749944,0cyclictest0-21swapper/100:01:011
268749943,0cyclictest9928-21awk19:32:331
268749943,0cyclictest9038-21awk19:07:371
268749943,0cyclictest6445-21snmpd21:52:341
268749943,0cyclictest6445-21snmpd21:35:041
268749943,0cyclictest6445-21snmpd20:17:031
268749943,0cyclictest6445-21snmpd19:42:481
268749943,0cyclictest6445-21snmpd19:30:031
268749943,0cyclictest6445-21snmpd19:19:031
268749943,0cyclictest3425-21lldpd23:04:331
268749943,0cyclictest3425-21lldpd22:47:331
268749943,0cyclictest3425-21lldpd20:57:331
268749943,0cyclictest24244-21awk20:07:371
268749943,0cyclictest23655-21kworker/1:021:03:481
268749943,0cyclictest2206-21awk23:22:341
268749943,0cyclictest18804-21awk23:52:361
268749943,0cyclictest18558-21awk00:12:351
268749943,0cyclictest17624-21awk20:02:361
268749943,0cyclictest1708-21kworker/1:123:47:191
268749943,0cyclictest1708-21kworker/1:123:17:371
268749943,0cyclictest1567-21awk22:02:351
268749943,0cyclictest10897-21awk19:57:351
268749943,0cyclictest10349-21awk23:07:371
268739944,0cyclictest0-21swapper/021:52:270
268739944,0cyclictest0-21swapper/021:16:580
268739943,0cyclictest9604-21awk21:47:360
268739943,0cyclictest6445-21snmpd21:35:340
268739943,0cyclictest6445-21snmpd00:17:190
268739943,0cyclictest26699-21awk23:37:350
268739943,0cyclictest25572-21awk21:37:370
268739943,0cyclictest2175-21awk23:22:330
268739943,0cyclictest1978-21awk00:22:330
268739943,0cyclictest1884-21awk21:02:350
268739943,0cyclictest1025-21ls23:22:190
268739941,0cyclictest0-21swapper/019:07:310
179940,0migration/14319-21kthreadcore22:42:411
179940,0migration/131468-21ksmtuned21:19:571
179940,0migration/130993-21kthreadcore18:57:381
179940,0migration/129816-21kthreadcore19:22:391
179940,0migration/12795-21kthreadcore21:42:381
179940,0migration/125406-21fschecks_count21:57:301
179940,0migration/122995-21munin-node22:52:411
179940,0migration/120405-21kthreadcore23:12:361
179940,0migration/120342-21kthreadcore22:12:431
179940,0migration/120111-21kthreadcore22:32:401
179940,0migration/117940-21gltestperf21:12:411
179940,0migration/115877-21kthreadcore20:47:401
179940,0migration/110473-21munin-node23:27:331
179940,0migration/110152-21kernelversion22:07:391
169940,0watchdog/15549-21kernelversion20:17:391
169940,0watchdog/125956-21sed00:17:301
169940,0watchdog/110085-21ps21:07:341
169940,0watchdog/10-21swapper/121:22:381
169940,0watchdog/10-21swapper/120:22:391
169940,0watchdog/10-21swapper/100:22:381
149940,0migration/06043-21kthreadcore23:42:400
149940,0migration/032117-21irqrtprio20:37:360
149940,0migration/018809-21kthreadcore19:37:390
149940,0migration/017157-21fschecks_count20:02:340
149940,0migration/010477-21kthreadcore20:42:390
339930,0migration/326734-21kthreadcore20:07:393
28415030,0irq/27-eth0-tx-1341-21munin-node19:02:310
28405030,0irq/26-eth0-rx-0-21swapper/320:27:313
28405030,0irq/26-eth0-rx-0-21swapper/320:12:293
28405030,0irq/26-eth0-rx-0-21swapper/319:57:303
28405030,0irq/26-eth0-rx-0-21swapper/319:37:303
28405030,0irq/26-eth0-rx-0-21swapper/319:23:193
28405030,0irq/26-eth0-rx-0-21swapper/319:17:303
28405030,0irq/26-eth0-rx-0-21swapper/318:57:403
268749933,0cyclictest9826-21kworker/1:121:27:481
268749933,0cyclictest23655-21kworker/1:019:52:241
268749932,0cyclictest6445-21snmpd22:37:341
268749932,0cyclictest31851-21awk20:37:341
268749932,0cyclictest30018-21awk19:47:371
268739933,0cyclictest3425-21lldpd22:08:030
268739933,0cyclictest0-21swapper/021:10:490
268739932,0cyclictest6797-21grep22:02:460
268739932,0cyclictest6445-21snmpd23:59:040
268739932,0cyclictest3425-21lldpd21:42:320
268739932,0cyclictest31153-21grep00:17:380
268739932,0cyclictest30968-21sensors22:37:420
268739932,0cyclictest25005-21diskmemload22:49:530
268739932,0cyclictest24220-21awk20:07:370
268739932,0cyclictest23258-21grep22:52:440
268739932,0cyclictest1202-21awk21:22:360
268739930,0cyclictest14775-21munin-node19:12:440
268739930,0cyclictest12837-21tr23:27:350
179930,0migration/11731-21df_abs19:02:311
169930,0watchdog/10-21swapper/120:42:431
169930,0watchdog/10-21swapper/120:32:311
329920,0watchdog/30-21swapper/320:22:303
329920,0watchdog/30-21swapper/320:02:383
28405020,0irq/26-eth0-rx-0-21swapper/320:49:403
268739921,0cyclictest17572-21latency_hist20:27:200
268739920,0cyclictest0-21swapper/000:02:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional