You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-10-12 - 18:03

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Oct 12, 2024 12:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
10772995638,12cyclictest0-21swapper/207:40:142
1075599511,40cyclictest11361-21sendmail-msp11:20:130
1076499491,42cyclictest26320-21apt-get10:30:121
1077299471,28cyclictest26080-21ssh10:10:112
1076499441,39cyclictest0-21swapper/111:05:121
1075599449,26cyclictest13707-21sendmail-msp08:20:120
1076499431,35cyclictest15519-21sendmail-msp12:20:141
1076499430,39cyclictest1733-21apt-get10:15:131
1076499430,29cyclictest29425-21ssh12:07:531
1076499430,27cyclictest13465-21/usr/sbin/munin09:20:161
1076499421,35cyclictest31821-21apt-get08:05:121
1076499420,27cyclictest16010-21/usr/sbin/munin09:40:321
1075599421,32cyclictest4581-21ssh11:35:130
1076499417,27cyclictest24302-21ssh12:25:151
1076499412,34cyclictest3225-21apt-get11:15:121
1076499411,34cyclictest7388-21apt-get12:15:121
1076499411,22cyclictest15624-21sh10:02:021
1076499401,34cyclictest20768-21apt-get09:25:121
1076499401,27cyclictest0-21swapper/111:40:161
1076499401,22cyclictest16602-21ssh12:39:361
1076499401,21cyclictest12094-21ssh11:57:181
1076499400,23cyclictest6019-21ntp_192.168.11410:55:271
10755994025,9cyclictest13025-21sendmail-msp09:20:120
1077299392,26cyclictest261rcuc/211:30:132
1076499391,34cyclictest25464-21apt-get07:25:131
1076499391,34cyclictest18291-21apt-get09:45:131
1076499391,34cyclictest0-21swapper/108:55:111
1076499390,29cyclictest29686-21diskmemload09:57:041
1076499390,25cyclictest29686-21diskmemload12:33:231
1077299388,18cyclictest22146irq/40-dwc2_hso09:00:122
1077299382,28cyclictest3078-21cat09:55:202
1076499382,31cyclictest18120-21apt-get10:25:151
1076499381,33cyclictest0-21swapper/110:05:121
1076499381,25cyclictest17800-21ssh10:44:271
1076499381,24cyclictest15122-21irqstats12:00:191
1076499381,20cyclictest0-21swapper/111:48:091
1076499380,24cyclictest13654-21vmstat08:15:311
1076499380,24cyclictest13654-21vmstat08:15:301
1076499380,23cyclictest1948-21ssh10:54:541
1077299371,27cyclictest18535-21ssh10:45:142
1076499374,30cyclictest7306-21sendmail-msp07:40:141
1076499372,31cyclictest28400-21apt-get08:35:121
1076499371,32cyclictest0-21swapper/107:45:111
1076499371,24cyclictest29788-21ntp_states10:10:251
1076499370,25cyclictest29686-21diskmemload09:30:101
1076499370,22cyclictest4804-21ssh11:53:551
10772993624,8cyclictest261rcuc/212:35:132
1077299362,18cyclictest6356-21ps11:55:192
10772993615,14cyclictest0-21swapper/208:00:112
1076499362,20cyclictest9207-21/usr/sbin/munin09:15:271
1076499361,31cyclictest2077-21apt-get10:35:121
1076499361,30cyclictest0-21swapper/109:50:151
1076499361,26cyclictest0-21swapper/107:20:131
1076499361,21cyclictest0-21swapper/111:11:321
1076499360,24cyclictest27844-21ssh11:29:371
1076499360,24cyclictest15612-21ssh10:21:461
1076499360,23cyclictest4555-21find11:35:101
1076499360,22cyclictest29686-21diskmemload09:11:031
1076499360,20cyclictest0-21swapper/110:47:001
10772993512,18cyclictest2591-21ssh09:34:192
1076499351,25cyclictest8103-21/usr/sbin/munin08:10:291
1076499351,21cyclictest12963-21tr08:50:211
1076499350,23cyclictest6400-21ssh12:14:101
1076499350,22cyclictest29686-21diskmemload09:37:261
10772993422,5cyclictest22146irq/40-dwc2_hso10:20:132
10772993410,7cyclictest131rcu_preempt12:06:202
1076499341,30cyclictest22061-21ls08:25:231
1076499341,28cyclictest23577-21apt-get08:30:111
1076499340,20cyclictest2690-21ssh11:32:561
1078099334,19cyclictest25866irq/58-eth012:20:133
10780993319,9cyclictest25866irq/58-eth012:25:153
1077299339,9cyclictest10745-21cyclictest12:30:192
1077299333,23cyclictest3248-21ssh11:15:142
1077299331,15cyclictest22146irq/40-dwc2_hso10:40:122
1076499331,28cyclictest25073-21apt-get09:05:121
1076499331,27cyclictest0-21swapper/107:10:121
1076499330,30cyclictest20636-21irqcore09:00:181
1078099322,16cyclictest25866irq/58-eth011:25:133
1077299323,9cyclictest3360-21ssh09:35:132
1077299322,24cyclictest14078-21sendmail-msp12:00:132
1077299322,24cyclictest10830-21sendmail-msp09:40:132
1077299322,22cyclictest25161-21kthreadcore12:25:192
1077299322,16cyclictest0-21swapper/207:20:122
10772993217,9cyclictest0-21swapper/211:45:142
10772993212,13cyclictest27-21ksoftirqd/209:45:142
1076499323,26cyclictest15367-21ssh11:20:271
10764993214,15cyclictest20299-21cron07:20:011
1076499320,29cyclictest5617-21apt-get08:45:121
1075599325,19cyclictest12-21ksoftirqd/008:00:110
10755993223,4cyclictest18745-21apt-get11:05:130
10755993219,8cyclictest1632-21apt-get09:55:140
10755993217,9cyclictest10455-21apt-get08:50:130
1075599321,23cyclictest9607-21sendmail-msp10:20:150
1077299313,13cyclictest13120-21ssh11:40:132
10772993118,10cyclictest27160-21ssh11:10:122
1077299311,18cyclictest22146irq/40-dwc2_hso08:55:122
1076499311,16cyclictest0-21swapper/108:40:071
1077299304,21cyclictest27-21ksoftirqd/212:23:202
1076499302,25cyclictest13926-21apt-get08:20:131
1076499301,18cyclictest0-21swapper/107:55:031
1075599303,8cyclictest28144-21sh11:30:020
1075599302,22cyclictest10242-21cat09:15:310
1075599302,11cyclictest20087-21cut10:45:200
1075599301,22cyclictest18309-21ssh09:45:150
1078099297,11cyclictest25866irq/58-eth010:55:143
1077299299,15cyclictest0-21swapper/211:37:202
1077299294,12cyclictest27-21ksoftirqd/210:35:132
1076499291,26cyclictest6116-21ntp_192.168.11407:35:231
1076499291,25cyclictest26975-21apt-get08:00:131
1075599293,10cyclictest18984-21ssh11:41:550
1075599292,19cyclictest27187-21ssh11:10:140
10755992911,9cyclictest10193-21sendmail-msp11:00:140
10755992911,9cyclictest10193-21sendmail-msp11:00:130
1075599290,10cyclictest14796-21sh09:40:240
1078099283,16cyclictest0-21swapper/309:30:203
1077299285,9cyclictest29939-21ssh11:50:142
1077299282,20cyclictest0-21swapper/208:10:132
1077299282,13cyclictest18767-21ssh11:05:132
1076499281,24cyclictest10193-21sendmail-msp11:00:101
1076499281,24cyclictest10193-21sendmail-msp11:00:091
1075599282,7cyclictest29686-21diskmemload10:56:320
1075599282,20cyclictest11133-21ssh11:56:000
10755992818,7cyclictest9932-21sendmail-msp10:40:130
1075599281,20cyclictest2100-21ssh10:35:130
1078099276,12cyclictest25866irq/58-eth011:20:123
1078099272,18cyclictest0-21swapper/311:51:203
10780992710,11cyclictest0-21swapper/311:05:133
1077299272,11cyclictest0-21swapper/208:40:152
1077299272,11cyclictest0-21swapper/207:50:132
1077299271,22cyclictest18878-21grep08:25:152
1077299270,18cyclictest27-21ksoftirqd/209:05:122
1075599279,14cyclictest0-21swapper/011:15:130
1075599275,7cyclictest12-21ksoftirqd/008:40:150
1075599272,19cyclictest2274-21ssh11:32:200
1075599272,19cyclictest15519-21sendmail-msp12:20:130
10755992713,10cyclictest9477-21sendmail-msp10:00:130
1077299261,20cyclictest22146irq/40-dwc2_hso10:50:122
10772992610,3cyclictest131rcu_preempt10:25:212
1076499261,22cyclictest17450-21perl07:50:171
1075599267,9cyclictest0-21swapper/009:05:150
1075599265,6cyclictest29381-21grep10:50:230
1075599264,4cyclictest12-21ksoftirqd/007:20:240
1075599262,18cyclictest22882-21ssh12:05:140
10755992615,8cyclictest20772-21ssh09:25:130
1078099252,17cyclictest0-21swapper/311:57:183
1078099251,13cyclictest25866irq/58-eth009:20:123
1077299253,16cyclictest22146irq/40-dwc2_hso09:25:132
1077299252,9cyclictest0-21swapper/207:30:132
1077299251,9cyclictest27638-21ssh11:29:212
1077299251,19cyclictest31262-21sh12:10:042
10772992510,4cyclictest22244-21sshd07:55:172
1076499250,21cyclictest30241-21/usr/sbin/munin07:30:171
1075599257,8cyclictest0-21swapper/010:11:020
1075599254,5cyclictest12-21ksoftirqd/007:25:200
1075599254,11cyclictest0-21swapper/007:15:130
1075599253,4cyclictest21205-21rm12:03:120
10755992517,6cyclictest18023-21apt-get10:05:130
1075599251,18cyclictest9221-21sh09:37:200
1075599250,21cyclictest26322-21ssh10:30:120
1078099248,11cyclictest25866irq/58-eth009:10:143
1078099242,13cyclictest25866irq/58-eth010:45:123
1077299243,15cyclictest22146irq/40-dwc2_hso08:35:122
10772992411,11cyclictest3284-21kthreadcore10:55:212
1076499240,20cyclictest10822-21latency_hist07:10:011
1075599248,3cyclictest0-21swapper/010:15:130
1075599247,3cyclictest0-21swapper/009:03:190
1075599246,15cyclictest0-21swapper/012:15:120
1075599243,4cyclictest12-21ksoftirqd/007:30:140
1075599242,4cyclictest12-21ksoftirqd/011:48:020
1075599241,4cyclictest18883-21ntp_states08:55:260
1078099233,13cyclictest0-21swapper/311:32:203
1078099231,15cyclictest0-21swapper/309:37:203
1077299238,12cyclictest27-21ksoftirqd/210:10:022
1077299234,15cyclictest0-21swapper/208:45:122
1077299233,18cyclictest22146irq/40-dwc2_hso12:15:142
1077299233,15cyclictest22146irq/40-dwc2_hso07:25:142
1077299232,8cyclictest3105-21kthreadcore10:15:202
10772992314,4cyclictest11712-21ssh11:20:132
1077299231,15cyclictest17267-21ls09:20:292
10772992310,7cyclictest22146irq/40-dwc2_hso08:20:142
1075599236,11cyclictest0-21swapper/009:50:140
1075599235,4cyclictest12-21ksoftirqd/011:50:140
1075599234,7cyclictest5399-21rm12:12:420
1075599234,13cyclictest0-21swapper/008:45:130
1075599233,4cyclictest12-21ksoftirqd/008:35:240
1075599232,7cyclictest29686-21diskmemload10:25:200
10755992318,3cyclictest28324-21apt-get09:30:120
1075599231,7cyclictest29686-21diskmemload09:14:090
1078099222,14cyclictest0-21swapper/309:55:193
1078099222,12cyclictest25866irq/58-eth009:40:123
1077299229,6cyclictest17602-21cron10:05:002
10772992211,9cyclictest958-21ssh09:54:342
1077299221,19cyclictest7768-21kthreadcore09:15:212
1077299221,17cyclictest22146irq/40-dwc2_hso10:30:122
1077299221,15cyclictest22146irq/40-dwc2_hso07:10:122
1075599223,5cyclictest12-21ksoftirqd/008:25:300
1075599223,11cyclictest0-21swapper/007:55:150
1075599220,14cyclictest587-21/usr/sbin/munin12:30:170
1078099219,8cyclictest25866irq/58-eth012:05:133
10780992115,4cyclictest0-21swapper/310:41:093
10780992114,4cyclictest321rcuc/310:35:113
1077299219,9cyclictest13677-21sh08:20:012
1077299219,9cyclictest13677-21sh08:20:002
1077299211,10cyclictest6451-21grep07:35:282
1075599216,8cyclictest0-21swapper/008:30:110
1075599213,6cyclictest12-21ksoftirqd/008:15:160
1075599213,6cyclictest12-21ksoftirqd/008:15:150
1075599212,11cyclictest8772-21ssh12:35:130
1075599211,5cyclictest12-21ksoftirqd/012:25:240
10755992113,6cyclictest10885-21apt-get07:10:120
10755992113,5cyclictest17244-21apt-get07:50:130
10755992110,5cyclictest522-21usb-storage07:40:120
1078099205,10cyclictest25866irq/58-eth012:30:573
10780992014,4cyclictest0-21swapper/312:36:583
10780992014,4cyclictest0-21swapper/311:45:183
10780992014,4cyclictest0-21swapper/309:50:253
10780992013,5cyclictest0-21swapper/312:19:353
10780992013,5cyclictest0-21swapper/309:26:563
10780992012,5cyclictest0-21swapper/310:50:193
10780992010,7cyclictest0-21swapper/310:17:123
1077299209,8cyclictest0-21swapper/211:00:092
1077299209,8cyclictest0-21swapper/211:00:082
1077299206,6cyclictest15863-21fschecks_count07:15:152
1077299202,10cyclictest12622-21fschecks_time07:45:152
1077299201,9cyclictest28304-21latency_hist08:35:022
1077299201,10cyclictest30346-21if_err_eth009:10:182
1075599202,4cyclictest12-21ksoftirqd/007:35:280
1078099198,5cyclictest0-21swapper/310:10:133
1078099193,11cyclictest0-21swapper/311:35:203
10780991913,4cyclictest0-21swapper/312:13:443
10780991913,4cyclictest0-21swapper/311:40:063
10780991913,4cyclictest0-21swapper/311:10:133
10780991913,4cyclictest0-21swapper/310:30:433
10780991913,4cyclictest0-21swapper/310:00:153
10780991913,4cyclictest0-21swapper/309:15:133
10780991912,5cyclictest0-21swapper/311:16:523
1078099191,12cyclictest0-21swapper/309:45:323
10780991910,7cyclictest0-21swapper/312:00:213
1077299191,15cyclictest10562-21fschecks_count08:50:162
1077299191,10cyclictest431-21kthreadcore08:05:202
1075599194,4cyclictest12-21ksoftirqd/008:10:150
1075599193,4cyclictest12-21ksoftirqd/008:05:150
1078099188,3cyclictest0-21swapper/311:00:173
1078099188,3cyclictest0-21swapper/311:00:163
1078099182,10cyclictest0-21swapper/308:50:133
1078099182,10cyclictest0-21swapper/308:25:143
1078099182,10cyclictest0-21swapper/308:15:143
1078099182,10cyclictest0-21swapper/308:15:133
1078099182,10cyclictest0-21swapper/307:50:133
10780991812,4cyclictest0-21swapper/310:25:063
10780991812,4cyclictest0-21swapper/310:05:103
10780991812,4cyclictest0-21swapper/308:30:203
10780991811,5cyclictest0-21swapper/310:22:323
1078099181,12cyclictest0-21swapper/307:30:153
1077299181,13cyclictest10814-21latency_hist07:10:012
1075599187,9cyclictest10745-21cyclictest07:10:020
1075599184,2cyclictest131rcu_preempt07:45:210
1078099172,10cyclictest0-21swapper/308:40:143
1078099172,10cyclictest0-21swapper/308:35:143
1078099172,10cyclictest0-21swapper/308:20:133
1078099172,10cyclictest0-21swapper/308:10:133
1078099172,10cyclictest0-21swapper/308:05:143
1078099172,10cyclictest0-21swapper/308:00:123
1078099162,9cyclictest25866irq/58-eth007:20:123
1078099162,10cyclictest0-21swapper/307:55:153
1078099162,10cyclictest0-21swapper/307:40:143
10780991611,3cyclictest0-21swapper/308:45:293
1078099161,12cyclictest0-21swapper/308:55:233
1078099151,10cyclictest0-21swapper/307:10:133
1078099148,4cyclictest0-21swapper/309:00:283
1078099141,11cyclictest0-21swapper/307:15:143
1078099137,4cyclictest0-21swapper/309:05:173
1078099134,6cyclictest0-21swapper/307:35:123
1078099123,6cyclictest0-21swapper/307:25:143
1078099121,9cyclictest0-21swapper/307:45:123
107809995,2cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional