You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-07-29 - 18:23

Intel(R) Atom(TM) CPU E3845 @ 1.91GHz, Linux 4.19.37-rt20 (Profile)

Latency plot of system in rack #b, slot #6
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot6.osadl.org (updated Thu Jul 29, 2021 12:43:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31321710,5sleep21258699cyclictest09:47:102
87391950,3ptp4l401ktimersoftd/309:03:083
87391780,1ptp4l401ktimersoftd/313:41:303
87391730,14ptp4l401ktimersoftd/312:20:093
87391700,4ptp4l401ktimersoftd/309:06:183
186622660,3sleep018664-21gltestperf11:52:200
186622660,3sleep018664-21gltestperf11:52:200
198552650,1sleep30-21swapper/311:20:263
87391570,1ptp4l401ktimersoftd/311:56:413
1258399568,27cyclictest0-21swapper/011:12:090
8422550,1sleep20-21swapper/214:19:472
1258399538,18cyclictest26703-21rm12:32:480
1258399526,16cyclictest23128-21/usr/sbin/munin13:02:300
105232520,9sleep210528-21fschecks_time09:02:112
262452510,1sleep00-21swapper/013:39:210
254342510,2sleep30-21swapper/311:26:213
1258399516,16cyclictest10263-21/usr/sbin/munin12:17:120
1258399515,17cyclictest25515-21/usr/sbin/munin14:12:160
1258399509,11cyclictest15216-21cstates12:22:300
1258399506,16cyclictest5607-21diskmemload13:14:450
1258399496,18cyclictest4206-21cat11:02:230
1258399486,14cyclictest10277-21ssh13:56:590
1258399485,17cyclictest24482-21snmp_o5powerctl11:57:280
1258399484,17cyclictest5607-21diskmemload13:50:360
1258399482,19cyclictest19808-21sed14:06:250
1258399476,14cyclictest1067-21ssh11:33:390
1258399475,14cyclictest9880-21ssh14:28:220
1258399475,14cyclictest878-21ssh14:19:500
1258399475,14cyclictest32450-21ssh12:39:180
1258399475,14cyclictest30386-21ssh12:03:510
1258399475,14cyclictest30013-21ssh13:10:220
1258399475,14cyclictest28055-21ssh11:27:450
1258399475,14cyclictest24686-21ssh11:25:240
1258399475,14cyclictest15369-21ssh11:47:550
1258399474,16cyclictest1811-21/usr/sbin/munin12:07:320
1258399474,15cyclictest12451-21ssh13:25:570
1258399473,17cyclictest0-21swapper/011:16:400
1258399467,17cyclictest5521-21/usr/sbin/munin13:52:260
1258399467,11cyclictest10032-21rm11:10:400
1258399466,9cyclictest9677-21apt-get10:07:150
1258399466,13cyclictest20177-21ssh14:07:010
1258399465,15cyclictest16139-21ssh12:56:310
1258399465,14cyclictest7893-21ssh11:41:340
1258399465,14cyclictest7425-21ssh11:41:090
1258399465,14cyclictest6542-21ssh14:25:590
1258399465,14cyclictest28081-21ssh13:42:120
1258399465,14cyclictest13827-21ssh12:52:560
1258399465,13cyclictest21002-21ssh13:33:440
1258399464,14cyclictest12258-21ssh14:32:060
1258399456,8cyclictest25407-21snmp_o5powerctl10:37:320
1258399455,8cyclictest3887-21apt-get13:17:190
1258399455,11cyclictest4900-21apt-get09:57:100
1258399455,11cyclictest13034-21ls10:12:160
1258399454,9cyclictest8793-21users12:47:410
1258399445,9cyclictest27472-21ls10:42:170
1258399445,11cyclictest21938-21snmp_o5powerctl09:22:240
1258399444,9cyclictest338-21ls10:52:520
1258399444,18cyclictest29121-21snmp_o5powerctl09:37:300
1258399435,8cyclictest26403-21ls09:32:200
1258399435,8cyclictest14316-21ls09:07:290
1258399435,17cyclictest4640-21ssh12:11:280
1258399426,9cyclictest721-21interrupts09:47:140
1258399425,9cyclictest19163-21ls09:17:160
1258399425,15cyclictest29856-21ls10:47:170
1258399424,8cyclictest22686-21ls10:32:200
1258399424,8cyclictest19780-21apt-get12:27:140
1258399416,9cyclictest30503-21fschecks_time09:42:140
1258399414,9cyclictest4541-21munin-run09:56:190
1258399414,8cyclictest5607-21diskmemload12:42:270
87391400,1ptp4l401ktimersoftd/314:25:503
87391400,1ptp4l401ktimersoftd/312:50:133
1258399405,8cyclictest17847-21ls10:22:280
1258399405,8cyclictest17847-21ls10:22:280
1258399404,8cyclictest14470-21ls13:27:180
87391390,1ptp4l401ktimersoftd/314:27:553
87391390,1ptp4l401ktimersoftd/314:17:463
87391390,1ptp4l401ktimersoftd/313:36:403
87391390,1ptp4l401ktimersoftd/312:33:173
1258399394,8cyclictest24021-21ls09:27:210
87391380,1ptp4l401ktimersoftd/313:56:333
87391380,1ptp4l401ktimersoftd/313:49:223
87391380,1ptp4l401ktimersoftd/313:34:233
87391380,1ptp4l401ktimersoftd/313:19:443
87391380,1ptp4l401ktimersoftd/312:06:213
87391380,1ptp4l401ktimersoftd/311:51:233
87391380,1ptp4l401ktimersoftd/311:51:233
87391380,1ptp4l401ktimersoftd/311:11:403
1258399384,8cyclictest14947-21hddtemp_smartct10:17:130
87391370,1ptp4l401ktimersoftd/311:33:343
87391370,1ptp4l401ktimersoftd/311:28:523
87391370,0ptp4l401ktimersoftd/312:45:443
87391370,0ptp4l401ktimersoftd/312:28:233
87391370,0ptp4l401ktimersoftd/311:45:343
1258399375,17cyclictest7337-21grep10:02:100
87391360,1ptp4l401ktimersoftd/314:02:193
87391360,1ptp4l401ktimersoftd/313:15:463
87391360,1ptp4l401ktimersoftd/312:56:183
87391360,0ptp4l401ktimersoftd/313:55:383
87391350,2ptp4l12002-21kworker/3:013:22:053
87391350,1ptp4l401ktimersoftd/313:00:093
87391350,1ptp4l401ktimersoftd/312:22:033
87391350,1ptp4l401ktimersoftd/310:56:153
87391350,1ptp4l401ktimersoftd/309:51:193
87391350,0ptp4l401ktimersoftd/312:08:193
87391340,1ptp4l401ktimersoftd/314:15:113
87391340,1ptp4l401ktimersoftd/309:27:173
87391340,12ptp4l401ktimersoftd/313:29:173
87391340,0ptp4l401ktimersoftd/314:35:573
12584993421,11cyclictest25-21ksoftirqd/112:06:221
87391330,1ptp4l401ktimersoftd/313:08:263
87391330,11ptp4l401ktimersoftd/314:08:293
87391320,1ptp4l401ktimersoftd/312:40:563
87391320,1ptp4l401ktimersoftd/309:57:063
87391320,0ptp4l401ktimersoftd/312:12:013
87391310,1ptp4l401ktimersoftd/311:37:493
87391310,0ptp4l401ktimersoftd/309:50:443
87391310,0ptp4l401ktimersoftd/309:42:093
12584993120,9cyclictest25-21ksoftirqd/113:57:171
12584993118,3cyclictest25-21ksoftirqd/111:57:211
1258499311,27cyclictest6104-21ssh12:46:231
1258399316,9cyclictest14986-21snmp_o5powerctl09:11:190
87391300,0ptp4l401ktimersoftd/310:34:223
12584993024,3cyclictest25-21ksoftirqd/113:52:221
12584993024,3cyclictest25-21ksoftirqd/113:01:261
12584993024,3cyclictest25-21ksoftirqd/110:02:101
87391290,1ptp4l401ktimersoftd/310:19:143
87391290,0ptp4l401ktimersoftd/309:22:103
1258399295,9cyclictest1561-21perl10:57:120
101632296,10sleep10-21swapper/109:02:071
101282295,10sleep00-21swapper/009:02:060
87391270,1ptp4l401ktimersoftd/310:10:063
12584992722,3cyclictest25-21ksoftirqd/114:21:251
12584992721,3cyclictest25-21ksoftirqd/110:21:191
12584992721,3cyclictest231rcuc/110:37:121
12584992715,6cyclictest25-21ksoftirqd/112:27:151
1258499271,23cyclictest20707-21apt-get13:01:221
87391260,1ptp4l401ktimersoftd/310:30:483
87391260,1ptp4l401ktimersoftd/310:30:483
87391260,10ptp4l401ktimersoftd/309:39:373
87391260,0ptp4l401ktimersoftd/311:02:443
87391260,0ptp4l401ktimersoftd/311:00:083
305952260,2sleep20-21swapper/212:37:312
142512260,2sleep20-21swapper/214:00:472
1258699269,14cyclictest2953-21tr14:22:272
12584992621,3cyclictest25-21ksoftirqd/109:42:111
12584992620,4cyclictest25-21ksoftirqd/111:51:211
87391250,1ptp4l401ktimersoftd/311:10:083
87391250,0ptp4l401ktimersoftd/310:43:243
87391250,0ptp4l401ktimersoftd/310:37:173
87391250,0ptp4l401ktimersoftd/309:17:493
87391250,0ptp4l401ktimersoftd/309:12:023
269852250,2sleep10-21swapper/113:40:391
12584992520,3cyclictest25-21ksoftirqd/111:52:221
12584992520,3cyclictest25-21ksoftirqd/111:52:221
12584992520,3cyclictest25-21ksoftirqd/109:56:191
12584992520,3cyclictest25-21ksoftirqd/109:37:221
12584992519,4cyclictest25-21ksoftirqd/109:47:151
12584992519,3cyclictest25-21ksoftirqd/114:27:241
12584992519,3cyclictest25-21ksoftirqd/112:52:151
12584992519,3cyclictest25-21ksoftirqd/111:07:121
12584992519,3cyclictest25-21ksoftirqd/109:58:101
12584992519,3cyclictest25-21ksoftirqd/109:12:101
12584992518,4cyclictest25-21ksoftirqd/111:02:231
1258499251,21cyclictest15426-21ssh14:02:131
1258499251,21cyclictest11145-21ssh11:12:021
87391240,1ptp4l401ktimersoftd/313:02:153
87391240,1ptp4l401ktimersoftd/311:46:573
87391240,1ptp4l401ktimersoftd/310:47:253
87391240,1ptp4l401ktimersoftd/310:22:273
87391240,1ptp4l401ktimersoftd/310:22:273
87391240,1ptp4l401ktimersoftd/309:32:253
282562240,4sleep125-21ksoftirqd/113:42:151
1258699243,17cyclictest2500-21apt-get09:52:142
12584992419,3cyclictest25-21ksoftirqd/112:32:131
12584992419,3cyclictest25-21ksoftirqd/111:37:231
12584992419,3cyclictest25-21ksoftirqd/110:56:201
12584992418,3cyclictest25-21ksoftirqd/114:12:211
12584992418,3cyclictest25-21ksoftirqd/112:22:351
12584992418,3cyclictest25-21ksoftirqd/110:42:151
12584992418,3cyclictest25-21ksoftirqd/109:07:171
1258499241,4cyclictest31105-21snmp_o5powerctl12:37:371
12584992412,6cyclictest25-21ksoftirqd/114:22:261
1258499241,21cyclictest0-21swapper/113:29:171
1258499241,20cyclictest5607-21diskmemload13:17:131
1258499241,20cyclictest2370-21ssh13:15:131
1258499240,4cyclictest20886-21rm11:21:561
1258499240,21cyclictest12034-21ssh13:25:141
1258399242,13cyclictest1663-21sendmail-mta10:27:110
1258399242,13cyclictest1663-21sendmail-mta10:27:110
119192241,2sleep20-21swapper/214:31:322
87391230,1ptp4l401ktimersoftd/310:12:193
87391230,1ptp4l401ktimersoftd/310:02:103
1258699236,3cyclictest849-21runrttasks13:46:402
12584992318,3cyclictest25-21ksoftirqd/113:32:281
12584992318,3cyclictest25-21ksoftirqd/111:42:201
12584992318,3cyclictest25-21ksoftirqd/109:36:191
12584992317,3cyclictest25-21ksoftirqd/110:07:201
12584992313,8cyclictest25-21ksoftirqd/114:32:201
1258499231,20cyclictest0-21swapper/113:48:241
1258499231,20cyclictest0-21swapper/111:31:291
1258499231,19cyclictest7832-21ssh12:13:551
1258499231,19cyclictest23225-21df_inode09:27:131
12584992311,9cyclictest20633-21awk09:22:101
1258499231,19cyclictest0-21swapper/110:58:121
1258499230,2cyclictest0-21swapper/112:07:191
1258499230,20cyclictest28500-21ssh13:07:451
292522223,3sleep229255-21snmp_o5powerctl12:02:272
12586992215,4cyclictest33-21ksoftirqd/210:56:122
12586992215,4cyclictest33-21ksoftirqd/209:26:422
12586992215,4cyclictest33-21ksoftirqd/209:19:272
12586992215,4cyclictest28088-21/usr/sbin/munin13:42:212
12586992215,4cyclictest26963-21sh12:01:102
12586992215,4cyclictest20707-21apt-get13:01:222
12586992215,4cyclictest13568-21sh12:20:292
12586992214,5cyclictest7492-21/usr/sbin/munin14:27:372
12586992214,4cyclictest33-21ksoftirqd/209:06:182
12584992217,3cyclictest25-21ksoftirqd/111:27:291
12584992217,3cyclictest25-21ksoftirqd/110:47:271
12584992216,3cyclictest25-21ksoftirqd/110:27:171
12584992216,3cyclictest25-21ksoftirqd/110:27:171
12584992215,4cyclictest22707-21needreboot10:32:201
1258499220,20cyclictest0-21swapper/114:06:361
1258499220,20cyclictest0-21swapper/110:12:531
1258499220,20cyclictest0-21swapper/109:17:071
1258499220,19cyclictest5607-21diskmemload12:18:511
12586992115,3cyclictest23397-21sh12:30:442
12586992115,3cyclictest12200-21sh12:52:152
12586992114,4cyclictest6589-21sh12:12:252
12586992114,4cyclictest5379-21sh11:37:392
12586992114,4cyclictest3492-21/usr/sbin/munin11:02:352
12586992114,4cyclictest33-21ksoftirqd/211:00:142
12586992114,4cyclictest33-21ksoftirqd/210:04:002
12586992114,4cyclictest30063-21/usr/sbin/munin09:42:102
12586992114,4cyclictest28821-21needreboot09:37:252
12586992114,4cyclictest28132-21sh14:14:182
12586992114,4cyclictest20696-21sh11:21:372
12586992114,4cyclictest19664-21sed13:32:322
12586992114,4cyclictest18104-21sh14:03:472
1258699211,18cyclictest0-21swapper/211:32:212
1258699211,18cyclictest0-21swapper/210:27:252
1258699211,18cyclictest0-21swapper/210:27:252
1258699211,17cyclictest5607-21diskmemload13:37:292
1258699210,19cyclictest0-21swapper/211:50:292
12584992115,3cyclictest25-21ksoftirqd/110:22:261
12584992115,3cyclictest25-21ksoftirqd/110:22:261
12584992112,3cyclictest849-21runrttasks12:49:261
1258499210,19cyclictest0-21swapper/111:17:061
174832200,4sleep217482-21ssh11:17:202
12586992014,4cyclictest8225-21sh12:47:322
12586992014,4cyclictest5424-21sh13:52:032
12586992014,3cyclictest7852-21sh11:07:252
12586992014,3cyclictest32726-21sh12:07:142
12586992014,3cyclictest27152-21sh11:27:272
12586992014,3cyclictest26345-21sh13:07:052
12586992014,3cyclictest24484-21sh12:32:012
12586992014,3cyclictest16890-21sh13:30:172
12586992013,4cyclictest849-21runrttasks13:02:392
12586992013,4cyclictest28725-21/usr/sbin/munin10:47:082
12586992013,4cyclictest23962-21/usr/sbin/munin10:37:192
12586992013,4cyclictest22486-21sh14:07:572
12586992013,4cyclictest18570-21sed10:22:392
12586992013,4cyclictest18570-21sed10:22:392
12586992013,4cyclictest1504-21sh13:13:422
12586992013,4cyclictest14322-21sh12:21:252
12586992013,4cyclictest10692-21sh11:43:202
1258699201,17cyclictest0-21swapper/211:12:522
12586991914,3cyclictest15700-21/usr/sbin/munin09:12:152
12586991913,3cyclictest9600-21/usr/sbin/munin10:07:332
12586991913,3cyclictest849-21runrttasks13:21:362
12586991913,3cyclictest849-21runrttasks09:35:442
12586991913,3cyclictest5406-21sh12:45:112
12586991913,3cyclictest5290-21gltestperf09:57:142
12586991913,3cyclictest4504-21if_enp1s0.113:17:282
12586991913,3cyclictest33-21ksoftirqd/210:16:252
12586991912,4cyclictest26588-21cstates10:42:102
12586991912,4cyclictest21912-21fschecks_count10:32:142
12586991912,4cyclictest18981-21cut11:52:222
12586991912,4cyclictest18981-21cut11:52:222
12586991912,4cyclictest12536-21which10:12:132
12586991812,3cyclictest20503-21/usr/sbin/munin09:22:082
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional