You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-09-28 - 16:14

Intel(R) Xeon(R) CPU E3-1220L V2 @ 2.30GHz, Linux 4.18.7-rt5 (Profile)

Latency plot of system in rack #9, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack9slot0.osadl.org (updated Tue Sep 28, 2021 12:43:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3057221160,0sleep10-21swapper/110:10:281
15552750,1sleep01553-21sshd10:58:310
214312740,0sleep10-21swapper/109:52:281
214312740,0sleep10-21swapper/109:52:281
14832740,0sleep30-21swapper/312:16:213
14832740,0sleep30-21swapper/312:16:213
73352730,2sleep00-21swapper/009:50:390
73352730,2sleep00-21swapper/009:50:390
42312670,1sleep0842-21dbus-daemon11:16:030
301762670,1sleep20-21swapper/209:32:022
246452670,0sleep10-21swapper/111:14:391
269212650,0sleep126919-21systemd-cgroups09:35:011
179222640,1sleep117911-21bash12:11:061
200332630,0sleep1251ktimersoftd/110:36:521
200332630,0sleep1251ktimersoftd/110:36:521
271622620,0sleep2341ktimersoftd/211:35:442
16722620,0sleep00-21swapper/010:51:460
295042610,0sleep0141rcuc/009:12:240
261012610,0sleep30-21swapper/311:54:533
261012610,0sleep30-21swapper/311:54:533
69642590,0sleep30-21swapper/309:46:573
69642590,0sleep30-21swapper/309:46:573
264752580,0sleep20-21swapper/211:54:562
264752580,0sleep20-21swapper/211:54:562
11572560,0sleep20-21swapper/210:14:172
1937325339,8sleep30-21swapper/307:05:203
1946625124,9sleep00-21swapper/007:06:300
1734324935,8sleep20-21swapper/207:04:232
283392480,2sleep01987199cyclictest09:14:400
283392480,2sleep01987199cyclictest09:14:400
193272480,0sleep10-21swapper/111:07:261
1733724530,9sleep10-21swapper/107:04:181
6522270,0sleep30-21swapper/309:15:033
6522270,0sleep30-21swapper/309:15:033
277442240,0sleep1241rcuc/110:47:541
186222230,0sleep20-21swapper/212:03:542
301042210,0sleep20-21swapper/211:01:262
301042210,0sleep20-21swapper/211:01:262
69702200,1sleep26721-21sshd11:09:392
1987799207,1cyclictest26-21ksoftirqd/110:01:011
14622200,0sleep30-21swapper/309:58:073
313592190,1sleep331358-21sshd10:38:073
313592190,1sleep331358-21sshd10:38:073
1987799197,2cyclictest26-21ksoftirqd/109:24:411
1987199199,2cyclictest9-21ksoftirqd/010:21:000
1987199197,1cyclictest9-21ksoftirqd/012:11:220
165612190,0sleep10-21swapper/109:11:291
28242180,1sleep01-21systemd11:56:060
28242180,1sleep01-21systemd11:56:060
257522180,0sleep30-21swapper/312:04:313
257522180,0sleep30-21swapper/312:04:313
19890991814,1cyclictest8413-21bash11:46:263
147782180,0sleep2331rcuc/209:37:022
96982170,0sleep10-21swapper/110:15:081
67872170,1sleep06785-21systemd-cgroups12:16:580
67872170,1sleep06785-21systemd-cgroups12:16:580
36532170,0sleep00-21swapper/011:33:380
260522170,0sleep00-21swapper/010:47:450
1989099176,2cyclictest44-21ksoftirqd/311:10:393
19890991713,3cyclictest12319-21bash12:31:273
1988599179,3cyclictest35-21ksoftirqd/209:43:102
1988599177,1cyclictest35-21ksoftirqd/212:14:142
1988599177,1cyclictest35-21ksoftirqd/212:14:142
1987199177,3cyclictest9-21ksoftirqd/012:33:280
1987199177,2cyclictest9-21ksoftirqd/011:23:200
1987199176,1cyclictest9-21ksoftirqd/010:11:040
19871991711,2cyclictest9-21ksoftirqd/009:34:290
317722160,2sleep231777-21bash10:54:412
19890991614,1cyclictest44-21ksoftirqd/310:15:313
1989099160,15cyclictest15249-21grepconf.sh09:19:333
1988599164,3cyclictest35-21ksoftirqd/209:25:342
1987799165,3cyclictest26-21ksoftirqd/112:07:421
1987799165,3cyclictest26-21ksoftirqd/112:07:421
1987799160,3cyclictest121rcu_sched10:29:181
1987199167,1cyclictest9-21ksoftirqd/010:25:100
1987199166,1cyclictest9-21ksoftirqd/010:07:140
1987199162,1cyclictest9-21ksoftirqd/012:09:090
1987199162,1cyclictest9-21ksoftirqd/012:09:090
1987199160,0cyclictest4448-21sshd12:02:280
317022150,0sleep30-21swapper/310:45:123
1989099150,1cyclictest18958-21kworker/3:610:07:413
1988599156,1cyclictest35-21ksoftirqd/212:22:172
19885991513,1cyclictest35-21ksoftirqd/211:31:392
1988599150,1cyclictest121rcu_sched11:17:272
1988599150,14cyclictest27825-21kworker/2:211:22:402
1987799158,3cyclictest26-21ksoftirqd/109:14:381
1987799158,3cyclictest26-21ksoftirqd/109:14:381
1987799155,1cyclictest26-21ksoftirqd/112:18:361
1987799155,1cyclictest26-21ksoftirqd/112:18:361
1987799154,4cyclictest26-21ksoftirqd/111:47:441
1987799154,2cyclictest26-21ksoftirqd/112:03:321
19877991513,1cyclictest26-21ksoftirqd/109:40:241
19871991513,1cyclictest0-21swapper/011:35:320
19871991511,1cyclictest9-21ksoftirqd/011:24:280
19871991511,1cyclictest9-21ksoftirqd/011:24:280
1989099149,4cyclictest44-21ksoftirqd/309:39:123
19890991412,1cyclictest44-21ksoftirqd/311:52:353
19890991412,1cyclictest44-21ksoftirqd/311:07:563
19890991412,0cyclictest1524-21nfsd11:34:393
1989099141,1cyclictest121rcu_sched09:12:513
19890991411,2cyclictest44-21ksoftirqd/309:25:433
19890991411,1cyclictest44-21ksoftirqd/309:34:063
19890991411,1cyclictest0-21swapper/308:54:273
1988599149,1cyclictest35-21ksoftirqd/210:07:282
1988599143,1cyclictest35-21ksoftirqd/210:50:052
1988599141,2cyclictest121rcu_sched12:27:052
19885991412,1cyclictest35-21ksoftirqd/212:07:292
19885991412,1cyclictest35-21ksoftirqd/212:07:292
19885991412,1cyclictest14938-21systemd-cgroups09:59:242
19885991412,1cyclictest0-21swapper/207:41:382
1988599141,1cyclictest121rcu_sched09:11:112
1988599140,1cyclictest111rcu_preempt10:21:172
1988599140,13cyclictest0-21swapper/209:04:282
1988599140,13cyclictest0-21swapper/207:49:452
1987799149,1cyclictest26-21ksoftirqd/111:39:211
1987799149,1cyclictest26-21ksoftirqd/111:20:211
1987799147,1cyclictest26-21ksoftirqd/110:05:501
1987799146,3cyclictest26-21ksoftirqd/110:49:561
1987799146,1cyclictest26-21ksoftirqd/109:21:011
1987799144,3cyclictest26-21ksoftirqd/111:09:391
19877991412,1cyclictest26-21ksoftirqd/111:03:391
19877991412,1cyclictest26-21ksoftirqd/111:03:391
19877991411,1cyclictest32492-21cut07:29:351
1987799140,5cyclictest30346-21kworker/1:512:28:371
1987799140,1cyclictest111rcu_preempt10:27:291
1987199146,3cyclictest9-21ksoftirqd/009:59:240
1987199146,3cyclictest9-21ksoftirqd/009:21:030
1987199145,1cyclictest9-21ksoftirqd/012:36:090
19871991412,1cyclictest9-21ksoftirqd/012:24:400
1987199140,2cyclictest121rcu_sched11:41:140
1989099138,4cyclictest44-21ksoftirqd/310:26:073
1989099136,4cyclictest44-21ksoftirqd/311:01:533
1989099136,4cyclictest44-21ksoftirqd/311:01:533
1989099131,1cyclictest44-21ksoftirqd/311:24:183
1989099131,1cyclictest44-21ksoftirqd/311:24:183
19890991311,1cyclictest44-21ksoftirqd/312:37:373
19890991311,1cyclictest44-21ksoftirqd/310:21:473
19890991310,1cyclictest44-21ksoftirqd/310:01:063
19890991310,1cyclictest19201-21bash10:50:123
1989099130,1cyclictest876-21gdbus10:10:323
1989099130,12cyclictest15566-21systemctl10:33:113
1988599138,1cyclictest35-21ksoftirqd/212:32:032
1988599131,1cyclictest121rcu_sched10:25:502
1988599131,1cyclictest111rcu_preempt10:40:102
19885991311,1cyclictest35-21ksoftirqd/212:13:332
19885991311,1cyclictest35-21ksoftirqd/210:31:052
19885991311,1cyclictest35-21ksoftirqd/209:49:572
19885991311,1cyclictest35-21ksoftirqd/209:49:572
19885991311,1cyclictest35-21ksoftirqd/209:19:272
19885991311,1cyclictest35-21ksoftirqd/209:17:562
19885991311,1cyclictest35-21ksoftirqd/209:17:562
1988599130,1cyclictest121rcu_sched09:56:022
1987799139,1cyclictest26-21ksoftirqd/111:24:511
1987799139,1cyclictest26-21ksoftirqd/111:24:511
1987799138,1cyclictest26-21ksoftirqd/111:56:401
1987799138,1cyclictest26-21ksoftirqd/111:56:401
1987799138,1cyclictest26-21ksoftirqd/109:54:171
1987799137,3cyclictest26-21ksoftirqd/112:20:011
1987799131,2cyclictest111rcu_preempt10:22:161
19877991311,1cyclictest3097-21sshd10:55:021
19877991311,1cyclictest0-21swapper/112:37:581
1987799130,1cyclictest121rcu_sched09:29:131
1987199138,1cyclictest9-21ksoftirqd/011:49:390
1987199138,1cyclictest9-21ksoftirqd/010:30:310
1987199137,3cyclictest9-21ksoftirqd/011:01:220
1987199137,3cyclictest9-21ksoftirqd/011:01:220
1987199137,2cyclictest9-21ksoftirqd/010:35:260
1987199137,2cyclictest9-21ksoftirqd/010:35:260
1987199137,1cyclictest9-21ksoftirqd/011:44:570
1987199132,1cyclictest9-21ksoftirqd/010:39:390
19871991311,1cyclictest9-21ksoftirqd/011:12:350
19871991311,1cyclictest9-21ksoftirqd/009:55:510
19871991311,1cyclictest819-21systemd-logind09:29:030
1987199130,1cyclictest121rcu_sched11:07:540
1987199130,1cyclictest10979-21kworker/u8:110:15:160
1989099129,1cyclictest44-21ksoftirqd/311:15:293
1989099128,1cyclictest44-21ksoftirqd/311:39:183
19890991210,1cyclictest819-21systemd-logind10:55:073
19890991210,1cyclictest44-21ksoftirqd/312:25:583
19890991210,1cyclictest44-21ksoftirqd/312:19:183
19890991210,1cyclictest44-21ksoftirqd/311:19:403
1989099120,0cyclictest121rcu_sched11:30:153
1988599129,2cyclictest35-21ksoftirqd/211:24:442
1988599129,2cyclictest35-21ksoftirqd/211:24:442
1988599128,3cyclictest35-21ksoftirqd/210:38:072
1988599128,3cyclictest35-21ksoftirqd/210:38:072
1988599128,1cyclictest35-21ksoftirqd/209:48:252
1988599128,1cyclictest35-21ksoftirqd/209:48:252
1988599124,1cyclictest35-21ksoftirqd/211:42:122
1988599122,1cyclictest35-21ksoftirqd/211:44:342
19885991210,1cyclictest35-21ksoftirqd/211:05:132
1988599120,3cyclictest35-21ksoftirqd/212:38:302
1988599120,2cyclictest121rcu_sched10:46:022
1988599120,1cyclictest35-21ksoftirqd/211:51:152
1988599120,1cyclictest121rcu_sched10:09:162
1987799127,4cyclictest26-21ksoftirqd/112:31:591
1987799126,3cyclictest26-21ksoftirqd/111:49:481
1987799126,3cyclictest26-21ksoftirqd/109:44:501
1987799126,3cyclictest26-21ksoftirqd/109:44:501
1987799124,3cyclictest26-21ksoftirqd/111:34:241
19877991210,1cyclictest26-21ksoftirqd/111:30:131
1987799120,1cyclictest9634-21users08:49:371
1987799120,11cyclictest13186-21bash10:39:371
1987199121,1cyclictest121rcu_sched09:39:270
19871991210,1cyclictest9-21ksoftirqd/009:45:350
19871991210,1cyclictest9-21ksoftirqd/009:45:350
1987199120,1cyclictest121rcu_sched09:31:160
1989099119,1cyclictest44-21ksoftirqd/309:51:053
1989099119,1cyclictest44-21ksoftirqd/309:51:053
1989099119,1cyclictest44-21ksoftirqd/309:39:203
1989099116,4cyclictest44-21ksoftirqd/310:41:213
19890991110,0cyclictest0-21swapper/312:10:313
19890991110,0cyclictest0-21swapper/308:59:273
1989099110,1cyclictest121rcu_sched11:59:523
1989099110,10cyclictest0-21swapper/308:19:363
1989099110,10cyclictest0-21swapper/308:04:373
19885991110,0cyclictest0-21swapper/207:09:352
1987799119,1cyclictest0-21swapper/107:54:261
1987799110,10cyclictest0-21swapper/108:09:461
1987799110,10cyclictest0-21swapper/107:28:321
1987199117,1cyclictest9-21ksoftirqd/012:21:290
1987199110,0cyclictest0-21swapper/008:36:530
1989099100,9cyclictest0-21swapper/309:08:583
1989099100,9cyclictest0-21swapper/308:42:433
1989099100,0cyclictest0-21swapper/307:50:553
1988599109,0cyclictest0-21swapper/208:12:392
1987799109,0cyclictest0-21swapper/109:03:471
1987799100,9cyclictest0-21swapper/107:14:391
19871991010,0cyclictest678-40kipmi009:04:280
1987199100,9cyclictest0-21swapper/007:55:530
1987199100,0cyclictest0-21swapper/008:10:310
1987199100,0cyclictest0-21swapper/007:14:460
198909998,0cyclictest0-21swapper/308:31:483
198909990,2cyclictest0-21swapper/307:27:103
198859998,0cyclictest0-21swapper/208:29:582
198859991,4cyclictest15067-21cut07:59:292
198859990,0cyclictest0-21swapper/208:42:512
198779998,0cyclictest0-21swapper/108:37:181
198779990,0cyclictest0-21swapper/108:26:231
198719998,0cyclictest0-21swapper/009:00:350
198719998,0cyclictest0-21swapper/008:48:090
198909984,3cyclictest0-21swapper/308:44:303
198909980,4cyclictest22873-21hddtemp_smartct08:14:283
198859988,0cyclictest0-21swapper/207:29:332
198859987,0cyclictest0-21swapper/208:56:202
198859987,0cyclictest0-21swapper/208:04:352
198859984,3cyclictest0-21swapper/209:00:222
198859984,3cyclictest0-21swapper/208:49:222
198859984,3cyclictest0-21swapper/208:25:102
198859984,3cyclictest0-21swapper/208:17:582
198859984,3cyclictest0-21swapper/207:48:302
198859984,3cyclictest0-21swapper/207:24:142
198859984,3cyclictest0-21swapper/207:21:542
198859984,3cyclictest0-21swapper/207:15:422
198779985,2cyclictest7443-21idleruntime-cro08:49:111
198719987,0cyclictest0-21swapper/008:26:050
198909975,1cyclictest583-21fschecks_time08:34:253
198909971,4cyclictest0-21swapper/308:24:323
198909970,6cyclictest6948-21fschecks_count07:44:253
198909970,4cyclictest0-21swapper/308:51:023
198909970,4cyclictest0-21swapper/308:02:503
198909970,4cyclictest0-21swapper/307:29:403
198909970,3cyclictest19402-21/usr/sbin/munin08:09:283
198909970,3cyclictest0-21swapper/307:56:383
198909970,1cyclictest20691-21grep07:09:253
198909970,0cyclictest0-21swapper/307:39:203
198909970,0cyclictest0-21swapper/307:35:303
198909970,0cyclictest0-21swapper/307:20:223
198909970,0cyclictest0-21swapper/307:15:423
198859974,0cyclictest0-21swapper/208:47:322
198859973,3cyclictest0-21swapper/208:38:142
198859973,3cyclictest0-21swapper/207:36:022
198859970,4cyclictest25445-21hddtemp_smartct08:19:272
198859970,0cyclictest0-21swapper/207:54:182
198779971,3cyclictest121rcu_sched07:44:341
198779971,2cyclictest0-21swapper/108:39:331
198779971,2cyclictest0-21swapper/107:19:291
198779970,2cyclictest0-21swapper/108:29:301
198779970,2cyclictest0-21swapper/107:49:271
198779970,1cyclictest3959-21cut07:39:231
198779970,1cyclictest16005-21cut09:04:251
198779970,1cyclictest11082-21tail08:54:271
198779970,1cyclictest0-21swapper/108:04:551
198779970,1cyclictest0-21swapper/107:34:321
198779964,1cyclictest4641-21kworker/1:108:19:321
198779964,1cyclictest4641-21kworker/1:108:14:461
198779961,5cyclictest678-40kipmi007:09:281
198779960,0cyclictest15688-21/usr/sbin/munin07:59:351
198719964,1cyclictest14024-21kworker/0:008:55:280
198719964,1cyclictest14024-21kworker/0:008:49:140
198719964,1cyclictest14024-21kworker/0:008:40:480
198719964,1cyclictest14024-21kworker/0:008:29:220
198719964,1cyclictest14024-21kworker/0:008:20:080
198719964,1cyclictest14024-21kworker/0:008:00:260
198719964,1cyclictest14024-21kworker/0:007:50:080
198719964,1cyclictest14024-21kworker/0:007:45:020
198719964,1cyclictest14024-21kworker/0:007:41:160
198719964,1cyclictest14024-21kworker/0:007:35:220
198719964,1cyclictest14024-21kworker/0:007:30:080
198719964,1cyclictest14024-21kworker/0:007:24:360
198719964,1cyclictest14024-21kworker/0:007:20:220
198719962,2cyclictest0-21swapper/008:04:350
198719962,2cyclictest0-21swapper/007:09:280
198719961,1cyclictest14024-21kworker/0:008:14:250
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional