You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-24 - 09:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #1, slot #2

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack1slot2s.osadl.org (updated Wed Apr 24, 2024 00:43:23)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
291350580,0irq/25-eth10-21swapper/318:57:073
543324536,0sleep20-21swapper/218:58:172
555123624,0sleep00-21swapper/018:59:430
3165123425,0sleep10-21swapper/118:55:591
57159976,0cyclictest3425-21lldpd21:15:100
8225060,0irq/24-0000:00:16772-21latency_hist20:50:492
339960,0migration/319833-21sh22:33:013
339960,0migration/311017-21sh22:46:333
28415060,0irq/27-eth0-tx-5613-21ssh21:05:312
28415060,0irq/27-eth0-tx-19690-21ssh22:32:362
28415060,0irq/27-eth0-tx-14043-21munin-node21:11:112
28415060,0irq/27-eth0-tx-0-21swapper/223:56:122
28415060,0irq/27-eth0-tx-0-21swapper/223:46:012
28415060,0irq/27-eth0-tx-0-21swapper/223:29:072
28415060,0irq/27-eth0-tx-0-21swapper/223:12:192
28415060,0irq/27-eth0-tx-0-21swapper/223:10:212
28415060,0irq/27-eth0-tx-0-21swapper/222:51:552
28415060,0irq/27-eth0-tx-0-21swapper/222:47:042
28415060,0irq/27-eth0-tx-0-21swapper/222:41:162
28415060,0irq/27-eth0-tx-0-21swapper/222:24:582
28415060,0irq/27-eth0-tx-0-21swapper/222:20:332
28415060,0irq/27-eth0-tx-0-21swapper/222:11:362
28415060,0irq/27-eth0-tx-0-21swapper/221:59:242
28415060,0irq/27-eth0-tx-0-21swapper/221:54:352
28415060,0irq/27-eth0-tx-0-21swapper/221:50:352
28415060,0irq/27-eth0-tx-0-21swapper/221:44:032
28415060,0irq/27-eth0-tx-0-21swapper/221:35:502
28415060,0irq/27-eth0-tx-0-21swapper/221:28:312
28415060,0irq/27-eth0-tx-0-21swapper/221:23:582
28415060,0irq/27-eth0-tx-0-21swapper/200:28:212
28415060,0irq/27-eth0-tx-0-21swapper/200:16:342
28415060,0irq/27-eth0-tx-0-21swapper/200:13:262
28415060,0irq/27-eth0-tx-0-21swapper/200:03:172
28415060,0irq/27-eth0-tx-0-21swapper/020:56:010
28415060,0irq/27-eth0-tx-0-21swapper/020:20:320
28415060,0irq/27-eth0-tx-0-21swapper/019:41:010
28415060,0irq/27-eth0-tx-0-21swapper/019:10:200
28405060,0irq/26-eth0-rx-12617-21cat23:31:023
249960,0watchdog/23665-21ssh22:02:032
249960,0watchdog/20-21swapper/223:02:512
249960,0watchdog/20-21swapper/222:57:472
249960,0watchdog/20-21swapper/200:23:512
249960,0watchdog/20-21swapper/200:05:592
149960,0migration/020941-21runrttasks20:26:080
149960,0migration/020200-21sh21:52:440
149960,0migration/019736-21latency_hist23:55:480
149960,0migration/018566-21kthreadcore20:51:090
149960,0migration/016084-21kthreadcore00:11:110
149960,0migration/013344-21hddtemp_smartct23:11:020
57169954,0cyclictest6445-21snmpd20:52:401
57169954,0cyclictest14249-21awk22:11:061
57169950,0cyclictest0-21swapper/123:09:131
57159955,0cyclictest0-21swapper/000:09:370
57159950,0cyclictest0-21swapper/022:51:440
339950,0migration/38931-21kthreadcore21:06:073
339950,0migration/38541-21kthreadcore19:01:083
339950,0migration/38505-21switchtime20:16:133
339950,0migration/35417-21kthreadcore00:06:063
339950,0migration/34400-21taskset22:24:563
339950,0migration/331672-21kthreadcore22:41:083
339950,0migration/329047-21cpuspeed_turbos23:01:023
339950,0migration/328792-21munin-node00:01:043
339950,0migration/327103-21ksmtuned23:38:143
339950,0migration/327101-21gltestperf00:21:053
339950,0migration/323703-21kthreadcore21:36:073
339950,0migration/322670-21sendmail_mailqu20:51:173
339950,0migration/321713-21munin-node22:16:043
339950,0migration/320985-21irqcore23:56:103
339950,0migration/319454-21fschecks_time00:16:023
339950,0migration/318974-21munin-node21:51:063
339950,0migration/317694-21kthreadcore21:31:113
339950,0migration/315917-21munin-node20:46:133
339950,0migration/315863-21kthreadcore22:11:083
339950,0migration/314542-21munin-node19:36:013
339950,0migration/312909-21cpuspeed_turbos19:06:023
339950,0migration/311758-21kthreadcore19:56:083
339950,0migration/31119-21taskset19:46:223
339950,0migration/310823-21diskmemload23:06:343
28415050,0irq/27-eth0-tx-27281-21ssh22:36:332
28415050,0irq/27-eth0-tx-21984-21ssh21:15:582
28415050,0irq/27-eth0-tx-0-21swapper/223:42:192
28415050,0irq/27-eth0-tx-0-21swapper/223:30:542
28415050,0irq/27-eth0-tx-0-21swapper/222:26:162
28415050,0irq/27-eth0-tx-0-21swapper/221:07:072
28415050,0irq/27-eth0-tx-0-21swapper/021:00:560
28415050,0irq/27-eth0-tx-0-21swapper/020:21:030
28415050,0irq/27-eth0-tx-0-21swapper/020:06:010
28415050,0irq/27-eth0-tx-0-21swapper/019:30:230
28405050,0irq/26-eth0-rx-29267-21grep23:21:033
28405050,0irq/26-eth0-rx-2813-21ssh23:43:293
28405050,0irq/26-eth0-rx-25574-21swap19:16:073
28405050,0irq/26-eth0-rx-1208-21df_inode19:26:063
28405050,0irq/26-eth0-rx-0-21swapper/323:51:093
28405050,0irq/26-eth0-rx-0-21swapper/323:46:163
28405050,0irq/26-eth0-rx-0-21swapper/323:27:563
28405050,0irq/26-eth0-rx-0-21swapper/323:16:023
28405050,0irq/26-eth0-rx-0-21swapper/322:56:043
28405050,0irq/26-eth0-rx-0-21swapper/322:29:273
28405050,0irq/26-eth0-rx-0-21swapper/322:06:123
28405050,0irq/26-eth0-rx-0-21swapper/321:56:023
28405050,0irq/26-eth0-rx-0-21swapper/321:45:093
28405050,0irq/26-eth0-rx-0-21swapper/321:29:373
28405050,0irq/26-eth0-rx-0-21swapper/321:24:103
28405050,0irq/26-eth0-rx-0-21swapper/321:16:133
28405050,0irq/26-eth0-rx-0-21swapper/300:26:053
259950,0migration/227561-21munin-node19:21:062
259950,0migration/218488-21sh23:52:092
259950,0migration/211706-21sh22:07:002
249950,0watchdog/230552-21ps20:36:072
249950,0watchdog/222326-21memory20:51:112
249950,0watchdog/20-21swapper/223:38:192
249950,0watchdog/20-21swapper/223:21:232
249950,0watchdog/20-21swapper/223:18:152
249950,0watchdog/20-21swapper/221:30:552
249950,0watchdog/20-21swapper/220:25:352
249950,0watchdog/20-21swapper/219:58:472
249950,0watchdog/20-21swapper/219:15:512
249950,0watchdog/20-21swapper/219:05:472
179950,0migration/14368-21taskset22:24:501
169950,0watchdog/19296-21hwlatdetect19:56:031
169950,0watchdog/131051-21kthreadcore20:11:071
149950,0migration/06474-21kthreadcore22:46:090
149950,0migration/05760-21fschecks_count22:06:010
149950,0migration/0461-21kthreadcore21:41:080
149950,0migration/031232-21irqrtprio21:21:090
149950,0migration/031110-21kthreadcore22:01:050
149950,0migration/029146-21idleruntime23:41:090
149950,0migration/027950-21kthreadcore00:21:090
149950,0migration/027902-21sh23:40:330
149950,0migration/027673-21aten_r1power_vo19:46:020
149950,0migration/027411-21switchtime21:56:160
149950,0migration/025641-21kthreadcore21:16:090
149950,0migration/021164-21aten_r1power_po22:35:590
149950,0migration/020896-21munin-node19:16:000
149950,0migration/01959-21cpuspeed_turbos19:51:000
149950,0migration/019430-21proc_pri21:31:160
149950,0migration/01624-21sh00:01:290
149950,0migration/015966-21kthreadcore23:31:090
149950,0migration/014676-21kthreadcore20:46:090
149950,0migration/013253-21sh22:10:500
149950,0migration/011076-21kthreadcore19:56:070
8225040,0irq/24-0000:00:0-21swapper/219:31:092
57179943,0cyclictest3110-21irqbalance20:15:532
57179943,0cyclictest3110-21irqbalance20:06:232
57179943,0cyclictest23661-21awk20:31:062
57179943,0cyclictest15977-21awk20:01:042
57169943,0cyclictest6893-21awk21:06:041
57169943,0cyclictest6445-21snmpd19:13:101
57169943,0cyclictest5616-21awk22:46:051
57169943,0cyclictest3425-21lldpd23:56:431
57169943,0cyclictest3425-21lldpd22:56:121
57169943,0cyclictest3425-21lldpd22:52:411
57169943,0cyclictest3425-21lldpd22:31:111
57169943,0cyclictest3425-21lldpd22:04:421
57169943,0cyclictest3425-21lldpd21:36:111
57169943,0cyclictest3425-21lldpd21:27:111
57169943,0cyclictest3425-21lldpd20:42:091
57169943,0cyclictest3425-21lldpd20:42:091
57169943,0cyclictest3110-21irqbalance23:10:531
57169943,0cyclictest3110-21irqbalance22:16:331
57169943,0cyclictest3110-21irqbalance21:47:431
57169943,0cyclictest3110-21irqbalance21:24:031
57169943,0cyclictest3110-21irqbalance20:01:231
57169943,0cyclictest3110-21irqbalance19:52:131
57169943,0cyclictest3110-21irqbalance19:46:031
57169943,0cyclictest3110-21irqbalance19:43:131
57169943,0cyclictest3110-21irqbalance19:31:031
57169943,0cyclictest30232-21awk20:36:041
57169943,0cyclictest28702-21awk00:01:041
57169943,0cyclictest27259-21awk00:21:061
57169943,0cyclictest27185-21grep21:56:131
57169943,0cyclictest22726-21awk20:06:071
57169943,0cyclictest22003-21awk22:36:061
57169943,0cyclictest20751-21awk19:15:591
57169943,0cyclictest19598-21awk00:16:031
57169943,0cyclictest14897-21awk21:11:061
57169943,0cyclictest14481-21awk21:51:031
57169943,0cyclictest12022-21awk00:11:071
57159943,0cyclictest6652-21ps21:46:040
57159943,0cyclictest6445-21snmpd00:18:120
57159943,0cyclictest29562-21awk23:21:070
57159943,0cyclictest23542-21awk20:31:050
57159943,0cyclictest21465-21awk23:16:040
57159943,0cyclictest11919-21sadc23:30:480
57159940,0cyclictest13432-21munin-run21:30:480
57159940,0cyclictest0-21swapper/022:28:240
339940,0migration/332061-21kthreadcore20:36:093
339940,0migration/329136-21runrttasks20:35:473
339940,0migration/326106-21kthreadcore20:56:103
339940,0migration/321041-21runrttasks22:35:543
339940,0migration/319113-21diskmemload22:51:473
339940,0migration/311740-21kthreadcore20:21:043
339940,0migration/311191-21kthreadcore19:31:093
28405040,0irq/26-eth0-rx-0-21swapper/323:14:383
28405040,0irq/26-eth0-rx-0-21swapper/322:01:143
28405040,0irq/26-eth0-rx-0-21swapper/321:47:103
28405040,0irq/26-eth0-rx-0-21swapper/321:11:033
28405040,0irq/26-eth0-rx-0-21swapper/321:00:583
28405040,0irq/26-eth0-rx-0-21swapper/320:01:493
28405040,0irq/26-eth0-rx-0-21swapper/300:11:503
259940,0migration/26360-21kthreadcore20:41:082
259940,0migration/26360-21kthreadcore20:41:082
249940,0watchdog/20-21swapper/220:58:152
249940,0watchdog/20-21swapper/220:28:272
249940,0watchdog/20-21swapper/220:11:032
249940,0watchdog/20-21swapper/219:53:112
249940,0watchdog/20-21swapper/219:46:032
249940,0watchdog/20-21swapper/219:42:072
249940,0watchdog/20-21swapper/219:37:152
249940,0watchdog/20-21swapper/219:25:512
249940,0watchdog/20-21swapper/219:11:392
249940,0watchdog/20-21swapper/219:06:112
179940,0migration/18110-21kthreadcore23:26:111
179940,0migration/15598-21kthreadcore20:16:061
179940,0migration/14893-21latency_hist22:25:481
179940,0migration/13410-21aten_r1power_fr00:05:581
179940,0migration/131532-21irqcore21:01:061
179940,0migration/130326-21kthreadcore23:41:131
179940,0migration/129217-21munin-node23:01:041
179940,0migration/12661-21irqcore00:26:091
179940,0migration/12463-21kthreadcore21:41:091
179940,0migration/124105-21munin-node20:56:031
179940,0migration/122781-21aten_r1power_fr20:31:011
179940,0migration/114886-21kthreadcore23:31:091
179940,0migration/114029-21kthreadcore23:51:091
179940,0migration/112382-21kthreadcore20:21:041
179940,0migration/110908-21fschecks_count20:46:041
169940,0watchdog/16626-21perl19:01:031
169940,0watchdog/13926-21ssh23:45:551
169940,0watchdog/131401-21kthreadcore19:21:071
169940,0watchdog/122733-21gltestperf21:16:031
169940,0watchdog/118381-21kthreadcore19:36:071
169940,0watchdog/116888-21tr21:31:111
169940,0watchdog/114172-21cut19:06:071
169940,0watchdog/10-21swapper/123:16:071
149940,0migration/07745-21kthreadcore19:01:080
149940,0migration/04096-21sh23:46:020
149940,0migration/028921-21fschecks_count20:11:020
57169932,0cyclictest3110-21irqbalance23:36:031
57169932,0cyclictest17051-21awk20:26:051
57169930,0cyclictest693-21sed19:25:491
57159932,0cyclictest8455-21awk19:31:060
57159932,0cyclictest4020-21sh22:44:360
57159932,0cyclictest3425-21lldpd22:16:110
57159932,0cyclictest30262-21awk20:36:040
57159932,0cyclictest2451-21awk00:26:070
57159932,0cyclictest21787-21awk22:56:100
57159932,0cyclictest19447-21ssh22:31:540
57159930,0cyclictest6658-21perl21:06:030
339930,0migration/326613-21aten2.4-expect19:20:593
339930,0migration/325727-21kthreadcore19:41:103
28415030,0irq/27-eth0-tx-0-21swapper/020:41:010
28415030,0irq/27-eth0-tx-0-21swapper/020:41:010
28415030,0irq/27-eth0-tx-0-21swapper/020:01:000
28405030,0irq/26-eth0-rx-0-21swapper/320:25:593
28405030,0irq/26-eth0-rx-0-21swapper/320:11:003
28405030,0irq/26-eth0-rx-0-21swapper/320:06:013
28405030,0irq/26-eth0-rx-0-21swapper/319:50:593
19130,0ktimersoftd/129052-21sh22:40:581
169930,0watchdog/10-21swapper/123:21:511
169930,0watchdog/10-21swapper/122:05:591
149930,0migration/04600-21runrttasks22:25:310
149930,0migration/03090-21sh23:02:360
149930,0migration/028370-21runrttasks21:37:370
57159921,0cyclictest26461-21latency_hist19:20:490
57159921,0cyclictest19864-21latency_hist23:55:480
57159921,0cyclictest14176-21latency_hist19:35:490
57159920,0cyclictest19175-21cat19:10:490
57159920,0cyclictest0-21swapper/023:05:480
28405020,0irq/26-eth0-rx-0-21swapper/320:41:003
28405020,0irq/26-eth0-rx-0-21swapper/320:41:003
28405020,0irq/26-eth0-rx-0-21swapper/319:12:033
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional