You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2022-01-21 - 23:45

Intel(R) Xeon(R) CPU E3-1220L V2 @ 2.30GHz, Linux 4.18.7-rt5 (Profile)

Latency plot of system in rack #8, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the alll highest latencies:
System rack8slot0.osadl.org (updated Fri Jan 21, 2022 12:43:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1334921800,2sleep33222999cyclictest10:13:293
917721530,0sleep10-21swapper/111:28:501
470821510,8sleep03222699cyclictest12:16:290
206021210,1sleep02056-21systemd-cgroups12:27:400
3127121170,1sleep1857-21dbus-daemon11:54:451
2644721150,0sleep10-21swapper/109:27:151
2644721150,0sleep10-21swapper/109:27:151
1318621150,1sleep013181-21sshd12:01:410
848421130,0sleep3421rcuc/310:39:323
322121130,0sleep13190-21sshd09:22:141
606521120,1sleep26066-21grepconf.sh09:55:362
282572970,2sleep23222899cyclictest11:07:542
107762910,1sleep210720-21bash09:17:142
113912880,1sleep2857-21dbus-daemon11:38:542
136922840,2sleep213691-21sshd09:21:192
108782820,2sleep210875-21grepconf.sh11:18:412
264502790,2sleep226358-21sshd10:56:352
6652780,2sleep3592-21sshd11:43:363
301932770,1sleep30-21swapper/310:06:313
124502770,0sleep20-21swapper/212:06:372
77262760,0sleep20-21swapper/210:13:162
275022760,1sleep09-21ksoftirqd/012:38:430
264802740,1sleep226483-21sshd10:47:452
195812740,1sleep219580-21grepconf.sh10:37:262
324822730,1sleep31-21systemd11:04:183
252822730,2sleep33222999cyclictest10:51:273
252822730,2sleep33222999cyclictest10:51:273
243382730,1sleep124287-21bash11:36:521
9292720,1sleep10-21swapper/112:36:231
278432720,2sleep392150irq/27-p4p1-tx-09:15:183
205132720,0sleep10-21swapper/110:21:091
198152720,3sleep219817-21grep11:00:042
99982710,1sleep39983-21sshd12:29:193
61102710,1sleep16075-21bash12:00:101
31182710,1sleep33113-21bash09:53:003
170612710,1sleep117058-21grepconf.sh10:53:411
170612710,1sleep117058-21grepconf.sh10:53:411
15412710,1sleep11544-21grepconf.sh10:54:221
12552710,1sleep21248-21sshd12:20:042
323222700,1sleep332320-21sshd11:33:253
294222700,1sleep329418-21grepconf.sh09:59:053
265342700,1sleep0857-21dbus-daemon12:07:110
208902700,0sleep2857-21dbus-daemon11:44:232
208902700,0sleep2857-21dbus-daemon11:44:232
208682700,2sleep020821-21bash12:10:440
208682700,2sleep020821-21bash12:10:440
198452700,2sleep219832-21grepconf.sh12:30:552
180802700,3sleep218079-21grep11:12:392
326852690,1sleep132688-21grepconf.sh11:11:541
126582690,1sleep31-21systemd12:09:103
126582690,1sleep31-21systemd12:09:103
85412680,0sleep00-21swapper/010:12:020
59932680,0sleep30-21swapper/309:09:213
195552680,0sleep10-21swapper/110:09:541
169482680,0sleep10-21swapper/110:06:011
9492670,0sleep0933-21systemd-cgroups09:39:050
305432670,0sleep10-21swapper/112:16:141
274612670,1sleep31845-21nfsd11:24:233
240082670,1sleep124007-21bash09:52:341
125532670,2sleep012548-21grep11:44:050
183892660,1sleep018374-21sshd09:59:510
132932660,1sleep313292-21sshd12:02:573
41732650,0sleep30-21swapper/310:21:473
291492650,0sleep0141rcuc/011:07:560
188802650,0sleep10-21swapper/112:04:231
267012640,1sleep3857-21dbus-daemon10:33:583
251242640,1sleep225098-21sshd09:41:162
211042640,0sleep321103-21sshd11:20:183
17732640,1sleep01770-21sshd09:52:570
173062640,0sleep10-21swapper/109:30:481
10282630,0sleep126-21ksoftirqd/110:02:561
45692620,1sleep24533-21sshd10:01:492
190872620,0sleep00-21swapper/011:35:230
133592620,0sleep10-21swapper/111:41:291
67022610,1sleep16647-21sshd11:29:541
110992590,2sleep23222899cyclictest12:35:322
93312570,2sleep09327-21uname11:00:540
284032560,0sleep00-21swapper/010:54:080
140442560,0sleep20-21swapper/209:33:182
93272540,4sleep19316-21sshd11:46:231
93272540,4sleep19316-21sshd11:46:231
157302530,0sleep115731-21grep09:54:421
3183025226,8sleep20-21swapper/207:07:312
225222510,1sleep122525-21grep10:33:481
5012440,36sleep292050irq/26-p4p1-rx-11:27:112
3149024428,8sleep00-21swapper/007:04:400
3184224329,8sleep30-21swapper/307:07:363
2954424227,9sleep10-21swapper/107:04:091
38092330,1sleep13806-21id10:36:491
106842310,1sleep2857-21dbus-daemon09:12:092
32229993026,2cyclictest44-21ksoftirqd/311:02:183
54982290,1sleep15392-21sshd09:11:561
3222999292,1cyclictest121rcu_sched12:23:293
32228992927,1cyclictest35-21ksoftirqd/209:37:382
32228992915,1cyclictest35-21ksoftirqd/210:24:232
134862290,1sleep213487-21sshd10:32:132
32229992824,2cyclictest44-21ksoftirqd/311:47:333
32229992824,2cyclictest44-21ksoftirqd/311:47:333
289062280,1sleep228873-21sshd10:50:172
289062280,1sleep228873-21sshd10:50:172
297192270,1sleep129680-21bash10:45:231
3222999268,1cyclictest44-21ksoftirqd/311:34:173
32229992623,1cyclictest44-21ksoftirqd/311:55:533
32226992614,9cyclictest923-21polkitd09:17:490
326032250,1sleep132600-21sshd10:40:301
32229992523,1cyclictest44-21ksoftirqd/312:16:243
32229992512,1cyclictest44-21ksoftirqd/311:13:373
3222999251,1cyclictest121rcu_sched09:24:533
3222999251,1cyclictest121rcu_sched09:24:533
32229992422,1cyclictest44-21ksoftirqd/309:34:113
32229992421,1cyclictest44-21ksoftirqd/309:21:563
32229992412,2cyclictest44-21ksoftirqd/310:14:313
3222899242,2cyclictest35-21ksoftirqd/212:15:052
32228992420,3cyclictest35-21ksoftirqd/211:51:072
32228992420,1cyclictest35-21ksoftirqd/210:06:032
32228992414,9cyclictest35-21ksoftirqd/211:23:592
32228992414,1cyclictest35-21ksoftirqd/212:13:152
32228992414,1cyclictest35-21ksoftirqd/212:13:152
32228992411,1cyclictest35-21ksoftirqd/211:29:402
5332230,1sleep2531-21grep10:40:312
3222999238,1cyclictest44-21ksoftirqd/310:35:483
32229992319,2cyclictest44-21ksoftirqd/312:24:063
32229992319,2cyclictest44-21ksoftirqd/312:06:573
32229992310,1cyclictest44-21ksoftirqd/310:58:583
32228992321,1cyclictest35-21ksoftirqd/209:28:242
32228992321,1cyclictest35-21ksoftirqd/209:28:242
32228992313,1cyclictest35-21ksoftirqd/212:02:292
32228992312,1cyclictest35-21ksoftirqd/211:55:592
3222799237,15cyclictest2140-21sshd12:27:401
201352230,1sleep020126-21sshd09:21:370
173132230,0sleep00-21swapper/011:47:580
173132230,0sleep00-21swapper/011:47:580
13842230,0sleep20-21swapper/209:50:222
112302230,1sleep3879-21systemd-logind09:39:283
326082220,1sleep11-21systemd11:49:431
32229992212,1cyclictest44-21ksoftirqd/310:48:123
32229992211,1cyclictest44-21ksoftirqd/312:34:153
3222899229,2cyclictest35-21ksoftirqd/210:14:412
3222899228,1cyclictest35-21ksoftirqd/211:40:072
32228992218,1cyclictest35-21ksoftirqd/209:49:052
3222899221,1cyclictest35-21ksoftirqd/212:29:042
3222699225,15cyclictest857-21dbus-daemon12:22:430
213742220,2sleep321340-21bash11:49:213
202382220,1sleep120168-21bash09:36:021
107122220,1sleep01846-21nfsd11:33:500
32762210,3sleep03222699cyclictest11:51:070
32229992119,1cyclictest44-21ksoftirqd/310:27:053
32229992119,1cyclictest44-21ksoftirqd/309:30:383
32228992111,1cyclictest35-21ksoftirqd/210:19:342
3222699216,7cyclictest23275-21sshd11:15:220
3222699214,2cyclictest9986-21sshd11:26:150
3222699213,16cyclictest7324-21bash12:30:260
315732210,1sleep1251ktimersoftd/112:11:091
315732210,1sleep1251ktimersoftd/112:11:091
99202200,0sleep00-21swapper/010:53:240
99202200,0sleep00-21swapper/010:53:240
32229992018,1cyclictest44-21ksoftirqd/309:46:043
3222999201,1cyclictest121rcu_sched11:18:073
3222999200,1cyclictest121rcu_sched09:59:233
3222699204,2cyclictest10476-21uname09:55:470
3222699203,1cyclictest2168-21sshd11:22:070
3222699202,2cyclictest28188-21grepconf.sh09:45:090
3222699200,1cyclictest0-21swapper/010:39:020
3222699200,18cyclictest9063-21sshd10:39:330
3222699200,18cyclictest16298-21grep11:11:150
270652200,0sleep127024-21rm11:01:351
231082200,0sleep1251ktimersoftd/112:32:181
107422200,0sleep00-21swapper/010:27:100
3222699196,5cyclictest8140-21fw_forwarded_lo10:19:260
3222699193,15cyclictest32676-21sshd09:10:250
3222699191,2cyclictest0-21swapper/010:45:250
32226991910,7cyclictest17094-21grep10:06:010
3222699190,3cyclictest21705-21sshd10:14:580
3222699190,2cyclictest7680-21grepconf.sh10:31:590
3222699190,17cyclictest24398-21id09:24:260
3222699190,17cyclictest24398-21id09:24:260
3222699190,17cyclictest18373-21bash11:54:180
3222699190,12cyclictest757-21bash09:40:190
317692190,0sleep00-21swapper/008:09:050
3222799187,1cyclictest8586-21cut09:14:331
3222699180,16cyclictest22557-21systemd-cgroups09:31:020
32227991715,1cyclictest26-21ksoftirqd/111:14:581
32227991715,1cyclictest26-21ksoftirqd/110:25:181
32227991714,1cyclictest26-21ksoftirqd/112:23:251
3222799171,13cyclictest15956-21bash10:14:451
88822160,0sleep10-21swapper/108:24:321
32228991614,1cyclictest35-21ksoftirqd/209:09:052
32227991614,1cyclictest26-21ksoftirqd/111:22:441
3222799160,15cyclictest835-21kworker/1:709:47:241
32229991513,1cyclictest44-21ksoftirqd/309:09:063
32227991513,1cyclictest26-21ksoftirqd/111:05:291
32227991513,1cyclictest26-21ksoftirqd/109:40:271
3222699146,6cyclictest22830-21dump-pmu-power08:54:050
32229991311,1cyclictest2755-21hddtemp_smartct07:14:063
32226991210,1cyclictest0-21swapper/008:34:370
32226991210,1cyclictest0-21swapper/007:39:260
3222699120,3cyclictest31037-21bash09:09:060
3222699120,0cyclictest0-21swapper/008:26:150
3222899110,10cyclictest2480-21munin-node07:14:062
3222799110,10cyclictest0-21swapper/107:24:021
3222699115,1cyclictest28077-21expr07:59:310
3222699114,1cyclictest23559-21tune2fs08:54:240
3222699112,2cyclictest11390-21grep07:29:240
3222699110,10cyclictest3676-21ntp_states08:14:300
3222999108,1cyclictest21354-21hddtemp_smartct08:49:253
3222999108,1cyclictest1037-21ntp_states08:09:293
3222999108,1cyclictest0-21swapper/308:54:253
3222999100,9cyclictest0-21swapper/307:52:233
3222899108,1cyclictest0-21swapper/208:14:242
3222899100,9cyclictest0-21swapper/208:49:252
3222899100,9cyclictest0-21swapper/208:09:292
3222899100,1cyclictest23813-21fschecks_time08:54:242
3222899100,1cyclictest0-21swapper/208:59:262
3222899100,1cyclictest0-21swapper/208:39:242
3222799109,0cyclictest0-21swapper/108:39:181
3222799109,0cyclictest0-21swapper/107:46:061
3222799108,1cyclictest430-21smartctl08:09:241
3222799108,1cyclictest23821-21fschecks_time08:54:241
3222799108,1cyclictest0-21swapper/108:59:261
3222799100,9cyclictest10950-21smartctl08:29:271
3222799100,9cyclictest0-21swapper/108:47:471
3222799100,9cyclictest0-21swapper/108:34:541
3222799100,9cyclictest0-21swapper/108:19:251
3222799100,9cyclictest0-21swapper/108:14:301
3222799100,9cyclictest0-21swapper/107:57:571
3222799100,9cyclictest0-21swapper/107:35:531
3222699108,1cyclictest10441-21df08:29:240
3222699108,1cyclictest0-21swapper/008:59:260
3222699108,1cyclictest0-21swapper/008:09:230
3222699103,1cyclictest15639-21expr08:39:220
3222699102,2cyclictest6095-21nfsd408:19:300
3222699101,2cyclictest22396-21tail07:49:280
3222699100,9cyclictest0-21swapper/007:22:220
3222699100,1cyclictest0-21swapper/007:14:060
322289998,0cyclictest0-21swapper/207:44:302
322289998,0cyclictest0-21swapper/207:32:122
322289990,8cyclictest0-21swapper/208:37:432
322289990,8cyclictest0-21swapper/207:41:082
322289990,0cyclictest0-21swapper/207:35:552
322279990,0cyclictest0-21swapper/108:04:261
322269992,1cyclictest19170-21cpuspeed_turbos07:44:220
322269992,1cyclictest18354-21grep08:44:240
322269990,5cyclictest0-21swapper/007:35:180
322269990,0cyclictest0-21swapper/007:27:170
322269990,0cyclictest0-21swapper/007:09:140
322299987,0cyclictest0-21swapper/309:02:123
322299986,1cyclictest6310-21sh07:19:233
322299986,1cyclictest0-21swapper/308:39:243
322299983,1cyclictest0-21swapper/308:35:363
322299980,0cyclictest0-21swapper/307:36:563
322289987,0cyclictest0-21swapper/208:46:122
322289987,0cyclictest0-21swapper/208:32:012
322289987,0cyclictest0-21swapper/208:28:042
322289987,0cyclictest0-21swapper/207:59:372
322289987,0cyclictest0-21swapper/207:54:302
322289987,0cyclictest0-21swapper/207:09:532
322289980,0cyclictest0-21swapper/207:26:582
322279987,0cyclictest0-21swapper/108:53:361
322279987,0cyclictest0-21swapper/107:39:081
322279987,0cyclictest0-21swapper/107:25:391
322279983,2cyclictest0-21swapper/107:14:571
322269980,0cyclictest0-21swapper/007:54:090
322299975,1cyclictest8546-21cat08:24:303
322299975,1cyclictest32738-21cut07:09:263
322299975,1cyclictest19051-21expr08:44:293
322299970,1cyclictest18311-21vmstat07:39:393
322299970,1cyclictest10770-21perl08:29:253
322289970,1cyclictest6620-21fschecks_time07:19:252
322289970,1cyclictest10743-21kworker/2:108:04:282
322289970,1cyclictest0-21swapper/208:19:262
322279976,0cyclictest0-21swapper/109:08:001
322279975,1cyclictest11122-21/usr/sbin/munin07:29:241
322279970,1cyclictest32254-21/usr/sbin/munin07:09:291
322299965,0cyclictest0-21swapper/307:59:073
322299964,1cyclictest2425-21taskset08:14:213
322299964,1cyclictest19871-21if_em107:44:263
322299963,2cyclictest11953-21gltestperf07:29:273
322299960,5cyclictest0-21swapper/308:19:093
322299960,5cyclictest0-21swapper/308:04:383
322299960,5cyclictest0-21swapper/307:54:223
322289960,0cyclictest0-21swapper/207:49:202
322279960,1cyclictest0-21swapper/107:49:291
322299950,5cyclictest0-21swapper/307:24:093
322279950,0cyclictest0-21swapper/107:59:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional