You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-09-17 - 20:50

Intel(R) Xeon(R) CPU E3-1220L V2 @ 2.30GHz, Linux 4.18.7-rt5 (Profile)

Latency plot of system in rack #9, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the alll highest latencies:
System rack9slot0.osadl.org (updated Fri Sep 17, 2021 12:43:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
975521350,0sleep3890-21gdbus10:00:173
975521350,0sleep3890-21gdbus10:00:173
3273521060,1sleep132663-21sshd09:32:421
274482710,0sleep10-21swapper/111:57:161
142692700,1sleep014268-21sshd12:37:310
30432660,1sleep13046-21grepconf.sh11:08:271
106532640,0sleep20-21swapper/212:16:172
321852620,8sleep288550irq/26-p4p1-rx-09:10:152
321852620,8sleep288550irq/26-p4p1-rx-09:10:152
55402610,0sleep10-21swapper/108:30:081
100552610,0sleep00-21swapper/011:03:250
100552610,0sleep00-21swapper/011:03:250
102532600,2sleep188650irq/27-p4p1-tx-10:00:191
102532600,2sleep188650irq/27-p4p1-tx-10:00:191
306542590,0sleep00-21swapper/010:05:190
2530525923,29sleep20-21swapper/207:05:062
2530525923,29sleep20-21swapper/207:05:062
235192590,1sleep2842-21dbus-daemon11:13:052
208212590,1sleep020823-21consoletype10:01:220
208212590,1sleep020823-21consoletype10:01:220
120562590,0sleep212053-21rt-features08:40:192
25922580,0sleep30-21swapper/310:27:223
123682580,1sleep012367-21grepconf.sh09:26:360
179522570,0sleep01992-21runrttasks12:16:550
293362550,0sleep30-21swapper/310:38:323
198252530,2sleep12790999cyclictest07:55:071
190102530,3sleep12790999cyclictest12:17:001
2749925238,8sleep00-21swapper/007:08:040
2749925238,8sleep00-21swapper/007:08:040
2743825124,9sleep30-21swapper/307:07:193
2743825124,9sleep30-21swapper/307:07:193
2735624923,8sleep10-21swapper/107:06:171
2735624923,8sleep10-21swapper/107:06:171
221162420,1sleep30-21swapper/310:51:583
221162420,1sleep30-21swapper/310:51:583
309022280,3sleep188650irq/27-p4p1-tx-12:21:051
37812230,1sleep20-21swapper/212:07:352
138912220,0sleep10-21swapper/112:25:461
310712200,0sleep20-21swapper/212:03:432
310712200,0sleep20-21swapper/212:03:432
255902200,0sleep30-21swapper/311:07:373
238232200,0sleep00-21swapper/009:58:410
187012200,0sleep00-21swapper/012:05:450
11652200,0sleep10-21swapper/111:33:401
37882190,1sleep332177-1kworker/3:1H09:25:263
259282190,1sleep3431ktimersoftd/311:54:023
243822190,0sleep20-21swapper/210:23:362
92482180,0sleep01527-21nfsd11:31:310
85752180,1sleep1241rcuc/111:14:411
2791399187,4cyclictest35-21ksoftirqd/210:27:052
2791399187,4cyclictest35-21ksoftirqd/209:22:182
2791399187,4cyclictest35-21ksoftirqd/209:22:182
132892180,0sleep30-21swapper/312:34:333
11382180,1sleep11-21systemd09:13:341
11382180,1sleep11-21systemd09:13:341
37272170,0sleep03726-21grep11:08:300
326362170,0sleep10-21swapper/110:05:321
2791399177,3cyclictest35-21ksoftirqd/210:14:212
2791399176,5cyclictest35-21ksoftirqd/209:43:242
2791399174,3cyclictest35-21ksoftirqd/210:05:182
2791399172,4cyclictest35-21ksoftirqd/210:31:452
2791399172,3cyclictest26769-21systemd09:49:322
2791399172,1cyclictest341ktimersoftd/211:04:042
2791399172,1cyclictest341ktimersoftd/211:04:042
2791399171,1cyclictest0-21swapper/209:52:412
2791399171,1cyclictest0-21swapper/209:52:412
27502170,1sleep21-21systemd10:53:142
27502170,1sleep21-21systemd10:53:142
217722170,0sleep20-21swapper/210:04:332
217722170,0sleep20-21swapper/210:04:332
128162170,1sleep112819-21bash12:31:431
53212160,1sleep30-21swapper/312:01:113
53212160,1sleep30-21swapper/312:01:113
2791399167,3cyclictest35-21ksoftirqd/212:21:062
2791399167,3cyclictest35-21ksoftirqd/209:17:162
2791399167,3cyclictest35-21ksoftirqd/209:17:162
2791399167,1cyclictest35-21ksoftirqd/212:34:112
2791399167,1cyclictest35-21ksoftirqd/211:26:162
2791399167,1cyclictest35-21ksoftirqd/211:15:212
2791399167,1cyclictest35-21ksoftirqd/211:15:212
2791399167,1cyclictest35-21ksoftirqd/210:58:312
2791399167,1cyclictest35-21ksoftirqd/210:58:312
2791399166,3cyclictest35-21ksoftirqd/209:58:032
2791399166,2cyclictest35-21ksoftirqd/211:50:282
2791399166,1cyclictest35-21ksoftirqd/209:33:162
2791399165,3cyclictest35-21ksoftirqd/212:09:562
2791399165,3cyclictest35-21ksoftirqd/211:55:152
2791399164,3cyclictest35-21ksoftirqd/211:44:542
2791399162,1cyclictest341ktimersoftd/212:35:352
2791399161,2cyclictest341ktimersoftd/210:17:222
2791399161,1cyclictest35-21ksoftirqd/211:24:252
2791399160,1cyclictest35-21ksoftirqd/211:38:292
215962160,1sleep321599-21grepconf.sh09:15:533
215962160,1sleep321599-21grepconf.sh09:15:533
107592160,0sleep00-21swapper/010:19:370
323482150,1sleep0890-21gdbus11:23:050
2791399156,3cyclictest35-21ksoftirqd/212:27:272
2791399156,3cyclictest35-21ksoftirqd/209:35:562
2791399156,2cyclictest35-21ksoftirqd/210:41:322
2791399156,1cyclictest35-21ksoftirqd/211:30:362
2791399155,2cyclictest35-21ksoftirqd/211:44:362
2791399152,2cyclictest341ktimersoftd/211:08:422
2791399151,3cyclictest35-21ksoftirqd/210:35:022
2791399151,1cyclictest35-21ksoftirqd/210:44:572
224212150,0sleep30-21swapper/311:41:563
158942150,0sleep00-21swapper/009:22:510
158942150,0sleep00-21swapper/009:22:510
2791499140,13cyclictest1-21systemd10:43:063
2791399145,1cyclictest35-21ksoftirqd/209:25:502
2790999148,5cyclictest532-21kworker/1:610:55:091
2790999148,5cyclictest532-21kworker/1:610:55:091
2790999142,1cyclictest121rcu_sched10:28:151
27909991412,1cyclictest26-21ksoftirqd/110:48:331
27909991412,1cyclictest0-21swapper/110:41:261
2790999140,1cyclictest17628-21sshd10:54:481
2790999140,1cyclictest17628-21sshd10:54:481
27904991412,1cyclictest9-21ksoftirqd/010:30:220
2790499140,13cyclictest0-21swapper/011:38:360
27914991311,1cyclictest44-21ksoftirqd/310:32:133
2791499130,1cyclictest593-21systemd-journal12:19:583
2791499130,1cyclictest27545-21kworker/u8:311:22:363
2791499130,12cyclictest842-21dbus-daemon12:19:453
27909991311,1cyclictest4707-21sshd09:43:151
27909991311,1cyclictest19217-21systemd-cgroups10:10:461
2790999130,12cyclictest842-21dbus-daemon11:22:361
27904991311,1cyclictest9-21ksoftirqd/010:39:540
27904991311,1cyclictest1524-21nfsd09:40:250
27904991311,1cyclictest0-21swapper/010:51:160
27904991311,1cyclictest0-21swapper/010:51:160
2791499121,1cyclictest121rcu_sched11:03:493
2791499121,1cyclictest121rcu_sched11:03:493
27914991210,1cyclictest44-21ksoftirqd/312:37:093
27914991210,1cyclictest44-21ksoftirqd/312:27:583
27914991210,1cyclictest44-21ksoftirqd/312:06:063
27914991210,1cyclictest44-21ksoftirqd/311:34:463
27914991210,1cyclictest32381-21sshd10:08:503
2791499120,11cyclictest12008-21sshd09:33:523
27909991210,1cyclictest0-21swapper/109:20:291
27909991210,1cyclictest0-21swapper/109:20:291
2790499127,1cyclictest9-21ksoftirqd/010:13:280
2790499126,1cyclictest9065-21basename07:34:540
27904991210,1cyclictest9-21ksoftirqd/012:32:460
27904991210,1cyclictest9-21ksoftirqd/011:29:160
27904991210,1cyclictest9-21ksoftirqd/010:21:510
27904991210,1cyclictest9-21ksoftirqd/009:32:390
180892120,0sleep10-21swapper/110:31:491
28852110,1sleep32868-21sshd09:36:123
2791499119,1cyclictest44-21ksoftirqd/312:14:453
2791499119,1cyclictest44-21ksoftirqd/309:41:363
2791499118,1cyclictest0-21swapper/309:13:273
2791499118,1cyclictest0-21swapper/309:13:273
2791499117,1cyclictest44-21ksoftirqd/311:48:093
2791499117,1cyclictest44-21ksoftirqd/310:23:263
2791499112,1cyclictest121rcu_sched10:17:493
27914991110,0cyclictest0-21swapper/309:23:393
27914991110,0cyclictest0-21swapper/309:23:393
27914991110,0cyclictest0-21swapper/308:55:053
2791499110,1cyclictest121rcu_sched11:15:503
2791499110,1cyclictest121rcu_sched11:15:503
27913991110,0cyclictest0-21swapper/207:50:212
2790999119,1cyclictest26-21ksoftirqd/112:13:011
2790999119,1cyclictest26-21ksoftirqd/111:43:401
2790999119,1cyclictest26-21ksoftirqd/109:36:261
2790999119,1cyclictest26079-21sshd11:19:251
2790999119,1cyclictest26079-21sshd11:19:251
2790999117,1cyclictest26-21ksoftirqd/111:46:131
2790999111,1cyclictest121rcu_sched12:38:311
27909991110,0cyclictest1527-21nfsd07:45:491
27909991110,0cyclictest0-21swapper/107:23:581
2790999110,10cyclictest0-21swapper/108:59:131
2790499119,1cyclictest9-21ksoftirqd/012:27:090
2790499119,1cyclictest9-21ksoftirqd/012:20:090
2790499119,1cyclictest9-21ksoftirqd/012:13:350
2790499119,1cyclictest9-21ksoftirqd/011:54:080
2790499119,1cyclictest9-21ksoftirqd/011:11:230
2790499119,1cyclictest9-21ksoftirqd/010:47:030
2790499119,1cyclictest9-21ksoftirqd/010:26:490
2790499119,1cyclictest9-21ksoftirqd/009:50:150
2790499119,1cyclictest9-21ksoftirqd/009:50:150
2790499119,1cyclictest9-21ksoftirqd/009:13:250
2790499119,1cyclictest9-21ksoftirqd/009:13:250
2790499118,2cyclictest9-21ksoftirqd/009:46:050
2790499118,1cyclictest9-21ksoftirqd/009:35:070
2790499117,1cyclictest9-21ksoftirqd/011:56:120
2790499117,1cyclictest9-21ksoftirqd/010:54:560
2790499117,1cyclictest9-21ksoftirqd/010:54:560
2790499116,1cyclictest9-21ksoftirqd/009:17:500
2790499116,1cyclictest9-21ksoftirqd/009:17:500
2790499111,1cyclictest121rcu_sched10:37:180
2791499109,0cyclictest0-21swapper/309:04:433
2791499108,1cyclictest44-21ksoftirqd/311:55:113
2791499108,1cyclictest44-21ksoftirqd/311:38:583
2791499108,1cyclictest44-21ksoftirqd/311:24:593
2791499108,1cyclictest44-21ksoftirqd/310:55:163
2791499108,1cyclictest44-21ksoftirqd/310:55:163
2791499108,1cyclictest44-21ksoftirqd/310:46:073
2791499108,1cyclictest44-21ksoftirqd/310:10:243
2791499106,1cyclictest44-21ksoftirqd/309:56:373
2791499106,1cyclictest44-21ksoftirqd/309:52:343
2791499106,1cyclictest44-21ksoftirqd/309:52:343
2791499100,9cyclictest0-21swapper/308:36:433
2791499100,9cyclictest0-21swapper/308:22:333
2791499100,0cyclictest0-21swapper/308:46:433
2791499100,0cyclictest0-21swapper/308:28:333
2791499100,0cyclictest0-21swapper/308:09:403
2791399104,3cyclictest18256-21kworker/2:108:29:542
2791399101,2cyclictest35-21ksoftirqd/207:16:092
2790999109,0cyclictest0-21swapper/109:09:261
2790999109,0cyclictest0-21swapper/108:50:171
2790999108,1cyclictest26-21ksoftirqd/112:07:151
2790999108,1cyclictest26-21ksoftirqd/111:53:131
2790999108,1cyclictest26-21ksoftirqd/111:01:041
2790999108,1cyclictest26-21ksoftirqd/111:01:041
2790999108,1cyclictest26-21ksoftirqd/110:37:211
2790999108,1cyclictest26-21ksoftirqd/110:23:541
2790999108,1cyclictest15-21kswork09:50:361
2790999108,1cyclictest15-21kswork09:50:361
2790999100,8cyclictest5879-21kworker/1:512:03:411
2790999100,8cyclictest5879-21kworker/1:512:03:411
2790999100,1cyclictest121rcu_sched11:25:191
2790999100,0cyclictest0-21swapper/108:42:091
2790999100,0cyclictest0-21swapper/108:14:581
2790499109,0cyclictest0-21swapper/008:38:580
2790499109,0cyclictest0-21swapper/008:22:350
2790499109,0cyclictest0-21swapper/008:07:360
2790499108,1cyclictest9-21ksoftirqd/011:40:170
2790499106,1cyclictest9-21ksoftirqd/011:45:430
2790499100,9cyclictest0-21swapper/008:48:500
2790499100,9cyclictest0-21swapper/007:21:290
2790499100,1cyclictest819-21systemd-logind11:15:220
2790499100,1cyclictest819-21systemd-logind11:15:220
2790499100,1cyclictest121rcu_sched12:02:260
2790499100,1cyclictest121rcu_sched12:02:260
279149997,1cyclictest44-21ksoftirqd/311:10:063
279149997,1cyclictest44-21ksoftirqd/309:46:023
279139992,4cyclictest35-21ksoftirqd/208:50:102
279139991,3cyclictest35-21ksoftirqd/207:35:092
279139990,2cyclictest35-21ksoftirqd/207:10:012
279139990,2cyclictest18256-21kworker/2:107:21:272
279139990,2cyclictest121rcu_sched08:15:112
279139990,2cyclictest0-21swapper/207:32:442
279099998,0cyclictest0-21swapper/108:28:191
279099998,0cyclictest0-21swapper/108:14:441
279099997,1cyclictest26-21ksoftirqd/109:55:571
279099997,1cyclictest26-21ksoftirqd/109:46:201
279099997,1cyclictest26-21ksoftirqd/109:29:221
279099994,2cyclictest21690-21systemd11:35:241
279099992,3cyclictest121rcu_sched10:15:111
279099992,2cyclictest17906-21ls09:15:221
279099992,2cyclictest17906-21ls09:15:221
279099990,0cyclictest0-21swapper/109:04:501
279049998,0cyclictest0-21swapper/009:09:380
279139982,3cyclictest35-21ksoftirqd/208:25:072
279139982,3cyclictest35-21ksoftirqd/208:10:092
279139982,3cyclictest35-21ksoftirqd/207:45:232
279139981,3cyclictest0-21swapper/208:35:132
279139981,1cyclictest121rcu_sched09:00:092
279139980,8cyclictest18256-21kworker/2:108:08:062
279139980,5cyclictest18256-21kworker/2:108:58:522
279049987,0cyclictest0-21swapper/008:58:010
279049985,2cyclictest685-21sed07:15:230
7491270,0sleep30-21swapper/308:32:513
6514270,0sleep3421rcuc/307:29:543
279149970,6cyclictest25017-21smart_sda09:05:193
279149970,1cyclictest29225-21grep07:10:143
279149970,1cyclictest22634-21df_inode08:00:073
279149970,1cyclictest0-21swapper/308:15:203
279139974,2cyclictest18256-21kworker/2:107:55:012
279139972,3cyclictest35-21ksoftirqd/208:45:092
279139971,2cyclictest0-21swapper/207:25:022
279139970,1cyclictest88550irq/26-p4p1-rx-07:59:532
279139970,1cyclictest0-21swapper/209:04:552
279139970,1cyclictest0-21swapper/208:20:092
279139970,1cyclictest0-21swapper/207:40:122
279099970,1cyclictest13846-21perl08:45:121
279049975,1cyclictest6622-21expr08:30:160
279049975,1cyclictest30976-21irqstats08:15:150
279049975,1cyclictest301312chrt07:13:120
279049975,1cyclictest20700-21latency07:55:130
279049975,1cyclictest16437-21seq07:45:410
279049970,1cyclictest916-21snmpd09:02:180
279049970,1cyclictest1527-21nfsd08:53:000
279049970,1cyclictest1527-21nfsd07:40:420
279149964,1cyclictest16186-21fschecks_count08:50:103
279149963,2cyclictest14071-21sshd07:43:233
279149960,5cyclictest0-21swapper/307:55:223
279149960,5cyclictest0-21swapper/307:35:263
279099965,0cyclictest0-21swapper/107:44:321
279099965,0cyclictest0-21swapper/107:24:541
279099964,1cyclictest24033-21kworker/1:107:15:091
279099964,1cyclictest18897-21sed07:50:201
279099960,0cyclictest0-21swapper/107:35:261
279049962,2cyclictest0-21swapper/008:10:140
279049960,5cyclictest0-21swapper/007:59:530
279049960,5cyclictest0-21swapper/007:50:210
279049960,5cyclictest0-21swapper/007:35:130
279049960,1cyclictest916-21snmpd08:43:100
279049960,0cyclictest0-21swapper/007:25:370
1336260,4sleep11404-21nfsd408:20:121
279149954,0cyclictest0-21swapper/307:30:513
279149953,1cyclictest18052-21kernelversion07:50:143
279149950,4cyclictest0-21swapper/308:40:083
279149950,1cyclictest14977-21fschecks_count07:45:123
279149950,0cyclictest0-21swapper/308:10:123
279149950,0cyclictest0-21swapper/307:19:593
279149950,0cyclictest0-21swapper/307:16:373
279099953,1cyclictest916-21snmpd07:59:541
279099953,1cyclictest8962-21grep08:35:141
279099953,1cyclictest7091-21df07:30:091
279099953,1cyclictest28385-21entropy07:10:081
279099953,1cyclictest24865-21cut08:05:051
279049950,0cyclictest0-21swapper/008:25:130
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional