You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-26 - 16:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack1slot2s.osadl.org (updated Fri Apr 26, 2024 12:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
271610,0ktimersoftd/21381-21perf07:00:342
788923724,0sleep00-21swapper/006:58:510
766023726,0sleep30-21swapper/306:56:143
778822824,0sleep10-21swapper/106:57:441
8225080,0irq/24-0000:00:26149-21munin-run11:20:432
28415070,0irq/27-eth0-tx-0-21swapper/209:39:332
8225060,0irq/24-0000:00:0-21swapper/207:45:442
81199960,0cyclictest0-21swapper/211:51:482
81199960,0cyclictest0-21swapper/210:16:252
81189960,0cyclictest0-21swapper/109:46:041
339960,0migration/38917-21sh12:27:133
339960,0migration/38635-21munin-node11:06:133
339960,0migration/327420-21sh10:16:283
339960,0migration/31769-21timerwakeupswit07:16:123
339960,0migration/317228-21sh11:54:013
339960,0migration/310928-21sh10:49:523
339960,0migration/310738-21sh10:26:163
28415060,0irq/27-eth0-tx-9288-21ssh12:09:002
28415060,0irq/27-eth0-tx-4506-21munin-node10:26:082
28415060,0irq/27-eth0-tx-26293-21ssh10:36:252
28415060,0irq/27-eth0-tx-19986-21ssh10:12:582
28415060,0irq/27-eth0-tx-0-21swapper/212:24:572
28415060,0irq/27-eth0-tx-0-21swapper/212:12:322
28415060,0irq/27-eth0-tx-0-21swapper/211:47:532
28415060,0irq/27-eth0-tx-0-21swapper/211:43:432
28415060,0irq/27-eth0-tx-0-21swapper/211:40:222
28415060,0irq/27-eth0-tx-0-21swapper/211:33:382
28415060,0irq/27-eth0-tx-0-21swapper/211:06:112
28415060,0irq/27-eth0-tx-0-21swapper/210:44:272
28415060,0irq/27-eth0-tx-0-21swapper/210:01:492
28415060,0irq/27-eth0-tx-0-21swapper/209:57:062
28415060,0irq/27-eth0-tx-0-21swapper/209:51:082
28415060,0irq/27-eth0-tx-0-21swapper/209:27:262
28415060,0irq/27-eth0-tx-0-21swapper/209:23:592
28405060,0irq/26-eth0-rx-2775-21perl11:25:583
28405060,0irq/26-eth0-rx-26246-21munin-node12:00:573
28405060,0irq/26-eth0-rx-19163-21iostat_ios11:55:593
28405060,0irq/26-eth0-rx-17129-21sh11:12:433
249960,0watchdog/229404-40yum-updatesd-he11:21:062
249960,0watchdog/20-21swapper/211:25:462
249960,0watchdog/20-21swapper/210:09:062
249960,0watchdog/20-21swapper/209:40:542
149960,0migration/09068-21sh12:08:240
149960,0migration/015133-21idleruntime-cro09:30:430
149960,0migration/013163-21kthreadcore11:11:010
8225050,0irq/24-0000:00:22503-21cat07:35:442
81199955,0cyclictest0-21swapper/212:25:512
81189954,0cyclictest6445-21snmpd09:16:491
81189954,0cyclictest10222-21dump-pmu-power11:10:431
81189950,0cyclictest0-21swapper/111:00:011
81189950,0cyclictest0-21swapper/110:20:361
81189950,0cyclictest0-21swapper/109:31:341
81169954,0cyclictest0-21swapper/009:50:560
81169950,0cyclictest0-21swapper/007:51:450
4150,0ktimersoftd/00-21swapper/010:58:060
4150,0ktimersoftd/00-21swapper/010:45:290
4150,0ktimersoftd/00-21swapper/010:06:330
339950,0migration/35140-21munin-run10:05:433
339950,0migration/34486-21kthreadcore08:36:023
339950,0migration/332060-21munin-node08:56:093
339950,0migration/331753-21taskset08:34:453
339950,0migration/331753-21taskset08:34:453
339950,0migration/331729-21turbostat.cron09:20:383
339950,0migration/329137-21idleruntime-cro07:40:443
339950,0migration/325216-21munin-node08:51:053
339950,0migration/321970-21cpuspeed_turbos07:10:563
339950,0migration/321071-21munin-node09:31:063
339950,0migration/31912-21aten_r1power_po11:45:533
339950,0migration/316960-21sh12:12:413
339950,0migration/316148-21kthreadcore09:51:063
339950,0migration/313643-21kthreadcore10:30:593
339950,0migration/313454-21munin-node08:45:593
339950,0migration/311465-21kthreadcore08:41:003
339950,0migration/310811-21kthreadcore07:01:053
339950,0migration/310609-21timerandwakeup08:11:123
28415050,0irq/27-eth0-tx-6372-21iceccd08:40:480
28415050,0irq/27-eth0-tx-3293-21ssh10:22:512
28415050,0irq/27-eth0-tx-26246-21munin-node12:01:102
28415050,0irq/27-eth0-tx-11986-21fschecks_count10:50:572
28415050,0irq/27-eth0-tx-0-21swapper/212:15:472
28415050,0irq/27-eth0-tx-0-21swapper/211:01:012
28415050,0irq/27-eth0-tx-0-21swapper/210:56:362
28415050,0irq/27-eth0-tx-0-21swapper/210:49:382
28415050,0irq/27-eth0-tx-0-21swapper/209:15:462
28415050,0irq/27-eth0-tx-0-21swapper/209:11:492
28415050,0irq/27-eth0-tx-0-21swapper/008:46:050
28415050,0irq/27-eth0-tx-0-21swapper/008:26:060
28415050,0irq/27-eth0-tx-0-21swapper/008:05:530
28415050,0irq/27-eth0-tx-0-21swapper/007:20:330
28405050,0irq/26-eth0-rx-7505-21sh09:25:483
28405050,0irq/26-eth0-rx-32404-21df_abs09:20:583
28405050,0irq/26-eth0-rx-32013-21cat11:41:143
28405050,0irq/26-eth0-rx-31352sleep310:22:233
28405050,0irq/26-eth0-rx-24231-21if_eth109:36:003
28405050,0irq/26-eth0-rx-22155-21awk09:55:553
28405050,0irq/26-eth0-rx-21027-21kthreadcore10:56:033
28405050,0irq/26-eth0-rx-10507-21kthreadcore09:46:013
28405050,0irq/26-eth0-rx-0-21swapper/312:17:003
28405050,0irq/26-eth0-rx-0-21swapper/311:31:053
28405050,0irq/26-eth0-rx-0-21swapper/311:20:433
28405050,0irq/26-eth0-rx-0-21swapper/311:19:363
28405050,0irq/26-eth0-rx-0-21swapper/310:54:053
28405050,0irq/26-eth0-rx-0-21swapper/309:11:013
28405050,0irq/26-eth0-rx-0-21swapper/308:19:513
259950,0migration/2769-21aten_r1power_po09:00:542
259950,0migration/215789-21df_abs09:30:562
259950,0migration/213816-21ntp_kernel_err07:01:082
249950,0watchdog/217880-21tr08:46:062
249950,0watchdog/20-21swapper/211:56:022
249950,0watchdog/20-21swapper/211:10:542
249950,0watchdog/20-21swapper/208:36:062
249950,0watchdog/20-21swapper/208:12:502
249950,0watchdog/20-21swapper/208:00:542
249950,0watchdog/20-21swapper/207:21:182
249950,0watchdog/20-21swapper/207:15:462
249950,0watchdog/20-21swapper/207:12:182
19150,0ktimersoftd/10-21swapper/112:04:111
19150,0ktimersoftd/10-21swapper/107:26:141
149950,0migration/09738-21sh12:29:280
149950,0migration/08962-21sh11:48:360
149950,0migration/08773-21aten_r1power_vo09:05:560
149950,0migration/07900-21kthreadcore08:11:050
149950,0migration/04856-21taskset09:41:530
149950,0migration/032734-21sh11:42:260
149950,0migration/028285-21sh10:19:240
149950,0migration/028205-21latency_hist07:15:440
149950,0migration/026830-21kthreadcore09:56:040
149950,0migration/025230-21kthreadcore08:01:110
149950,0migration/022522-21latency_hist07:35:440
149950,0migration/022155-21kthreadcore10:36:040
149950,0migration/021534-21latency_hist07:10:440
149950,0migration/018706-21sh10:32:220
149950,0migration/017309-21sh11:34:100
149950,0migration/016713-21sh12:12:000
149950,0migration/016545-21cpuspeed_turbos09:10:570
149950,0migration/016048-21kthreadcore08:20:580
149950,0migration/014329-21irqcore10:11:030
149950,0migration/012717-21kthreadcore11:51:040
81199943,0cyclictest3425-21lldpd08:19:472
81199943,0cyclictest3110-21irqbalance09:45:582
81199943,0cyclictest3110-21irqbalance08:50:582
81199943,0cyclictest3110-21irqbalance08:26:082
81199943,0cyclictest26317-21awk08:31:002
81199943,0cyclictest26317-21awk08:31:002
81189944,0cyclictest0-21swapper/112:24:311
81189944,0cyclictest0-21swapper/111:00:571
81189943,0cyclictest6445-21snmpd09:38:341
81189943,0cyclictest3425-21lldpd11:13:171
81189943,0cyclictest3425-21lldpd09:51:061
81189943,0cyclictest3425-21lldpd09:00:171
81189943,0cyclictest3425-21lldpd08:31:441
81189943,0cyclictest3425-21lldpd08:31:441
81189943,0cyclictest3425-21lldpd08:22:251
81189943,0cyclictest30785-21awk10:00:581
81189943,0cyclictest2813-21awk11:46:001
81189943,0cyclictest27468-21awk11:21:041
81189943,0cyclictest26933-21kworker/1:110:40:571
81189943,0cyclictest26933-21kworker/1:109:27:391
81189943,0cyclictest23513-21sadc08:00:441
81189943,0cyclictest19683-21awk08:25:581
81189943,0cyclictest18984-21awk11:55:561
81189943,0cyclictest13048-21awk10:30:561
81189943,0cyclictest11092-21awk12:11:001
81169944,0cyclictest0-21swapper/012:23:330
81169943,0cyclictest6445-21snmpd11:58:500
81169943,0cyclictest3110-21irqbalance07:56:580
81169943,0cyclictest29895-21awk10:20:580
81169943,0cyclictest27268-21awk12:01:010
81169943,0cyclictest23599-21awk07:36:050
81169943,0cyclictest19324-21awk11:16:000
81169943,0cyclictest19002-21awk11:35:580
81169943,0cyclictest1683-21awk09:01:010
81169943,0cyclictest16384-21awk09:31:000
81169943,0cyclictest11885-21sadc08:20:440
81169940,0cyclictest31777-21munin-run09:20:430
81169940,0cyclictest30466-21perl10:00:550
339940,0migration/39249-21kthreadcore10:06:043
339940,0migration/331649-21irqrtprio08:06:003
339940,0migration/331594-21kthreadcore11:01:053
339940,0migration/328593-21kthreadcore12:21:033
339940,0migration/328209-21kthreadcore08:01:123
339940,0migration/323440-21taskset07:59:553
339940,0migration/320858-21kthreadcore07:31:033
339940,0migration/320027-21diskmemload10:35:453
339940,0migration/31655-21kthreadcore07:41:043
339940,0migration/316208-21timerandwakeup07:51:083
28415040,0irq/27-eth0-tx-0-21swapper/209:05:542
28415040,0irq/27-eth0-tx-0-21swapper/008:56:010
28415040,0irq/27-eth0-tx-0-21swapper/008:35:540
28415040,0irq/27-eth0-tx-0-21swapper/008:31:070
28415040,0irq/27-eth0-tx-0-21swapper/008:31:070
28405040,0irq/26-eth0-rx-0-21swapper/312:05:513
28405040,0irq/26-eth0-rx-0-21swapper/311:35:533
28405040,0irq/26-eth0-rx-0-21swapper/310:40:583
28405040,0irq/26-eth0-rx-0-21swapper/309:44:063
28405040,0irq/26-eth0-rx-0-21swapper/309:06:023
28405040,0irq/26-eth0-rx-0-21swapper/309:00:513
28405040,0irq/26-eth0-rx-0-21swapper/308:27:193
28405040,0irq/26-eth0-rx-0-21swapper/307:25:593
28405040,0irq/26-eth0-rx-0-21swapper/307:20:543
259940,0migration/23730-21aten_r1power_en07:45:552
259940,0migration/232062-21munin-node08:56:092
259940,0migration/213185-21kthreadcore07:51:002
259940,0migration/210740-21kthreadcore07:26:092
249940,0watchdog/20-21swapper/210:32:142
249940,0watchdog/20-21swapper/208:40:542
249940,0watchdog/20-21swapper/208:21:262
249940,0watchdog/20-21swapper/208:07:062
249940,0watchdog/20-21swapper/207:56:022
249940,0watchdog/20-21swapper/207:36:502
249940,0watchdog/20-21swapper/207:05:502
179940,0migration/18688-21kthreadcore10:26:041
179940,0migration/18578-21taskset09:05:461
179940,0migration/130832-21kthreadcore07:16:041
179940,0migration/129126-21munin-node08:01:131
179940,0migration/127485-21kernelversion11:41:051
179940,0migration/123086-21munin-node07:36:011
179940,0migration/121323-21kthreadcore08:51:011
179940,0migration/11653-21kthreadcore10:21:031
179940,0migration/112765-21kthreadcore07:01:051
179940,0migration/111320-21switchtime10:06:121
169940,0watchdog/1735-21basename09:00:541
169940,0watchdog/15701-21cut11:26:061
169940,0watchdog/12843-21tr09:41:061
169940,0watchdog/123474-21ssh09:14:221
169940,0watchdog/121409-21tr11:36:021
169940,0watchdog/116274-21cpuspeed_turbos07:30:541
169940,0watchdog/10-21swapper/112:30:261
169940,0watchdog/10-21swapper/111:31:141
169940,0watchdog/10-21swapper/110:36:101
169940,0watchdog/10-21swapper/108:36:021
169940,0watchdog/10-21swapper/108:06:021
169940,0watchdog/10-21swapper/107:46:061
149940,0migration/07961-21kthreadcore09:45:590
149940,0migration/07257-21kthreadcore07:46:070
149940,0migration/03062-21kthreadcore09:21:050
149940,0migration/027746-21fschecks_time11:00:590
149940,0migration/012487-21iostat10:51:010
81189933,0cyclictest3425-21lldpd08:15:471
81189933,0cyclictest26933-21kworker/1:107:50:571
81189932,0cyclictest99122sleep110:46:481
81189932,0cyclictest6445-21snmpd08:43:001
81189932,0cyclictest5233-21grep12:06:031
81189932,0cyclictest22515-21awk07:11:011
81189932,0cyclictest19292-21awk11:16:001
81189932,0cyclictest12367-21awk10:51:001
81189932,0cyclictest11724-21grep11:51:031
81189931,0cyclictest0-21swapper/110:11:041
81189930,0cyclictest0-21swapper/107:21:041
81169933,0cyclictest3425-21lldpd07:24:510
81169933,0cyclictest3006-21awk11:26:020
81169932,0cyclictest9192-21awk07:01:020
81169932,0cyclictest5493-21awk10:26:010
81169932,0cyclictest27352-21awk11:21:030
81169932,0cyclictest20886-21irqrtprio08:51:010
81169930,0cyclictest9787-21diskstats07:26:010
81169930,0cyclictest4898-21cat11:06:050
81169930,0cyclictest0-21swapper/009:35:550
28415030,0irq/27-eth0-tx-29385-21aten2.4-expect07:40:540
28405030,0irq/26-eth0-rx-0-21swapper/310:11:003
28405030,0irq/26-eth0-rx-0-21swapper/307:06:113
179930,0migration/19151-21kthreadcore08:11:051
169930,0watchdog/1355-21diskmemload09:56:021
169930,0watchdog/10-21swapper/112:15:461
169930,0watchdog/10-21swapper/109:21:141
169930,0watchdog/10-21swapper/108:46:061
169930,0watchdog/10-21swapper/107:41:021
169930,0watchdog/10-21swapper/107:06:021
81169921,0cyclictest18261-21latency_hist12:15:430
81169920,0cyclictest0-21swapper/010:45:440
339920,0migration/34066-21diskstats07:46:003
339920,0migration/312518-21diskstats08:20:533
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional