You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-10-17 - 05:35

Intel(R) Xeon(R) CPU E31220L @ 2.20GHz, Linux 3.18.43-rt46-fulldebug (Profile)

Latency plot of system in rack #b, slot #0
Note that this system runs a non-optimized debug kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Characteristics of the alll highest latencies:
System rackbslot0.osadl.org (updated Sun Oct 17, 2021 00:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1565626834,11sleep20-21swapper/200:36:432
1555526853,10sleep30-21swapper/300:35:353
1544726736,10sleep00-21swapper/000:34:280
1547826449,10sleep10-21swapper/100:34:491
175472521,47sleep11505-21nfsd02:50:321
16006994932,8cyclictest31439-21turbostat01:58:360
1600699482,40cyclictest30462-21sshd05:03:490
16024994732,7cyclictest30811-21kworker/u8:203:09:072
16024994731,8cyclictest16473-21cp02:50:272
207842430,1sleep1201ktimersoftd/104:34:341
16024994331,6cyclictest32743-21sshd05:44:462
242799420,3rtkit-daemon2426-21rtkit-daemon03:10:160
1601699400,30cyclictest24521-21kworker/1:302:43:391
16024993820,16cyclictest967-21kworker/u8:104:05:082
242799370,5rtkit-daemon0-21swapper/202:00:462
16006993719,9cyclictest25193-21modprobe05:39:050
242799360,3rtkit-daemon2426-21rtkit-daemon03:42:020
242799360,10rtkit-daemon2426-21rtkit-daemon03:01:322
1603599360,33cyclictest26216-21kworker/3:302:43:393
16024993614,19cyclictest8-21rcu_preempt03:30:152
242799350,10rtkit-daemon2426-21rtkit-daemon03:35:160
242799350,1rtkit-daemon0-21swapper/003:04:140
1600699351,32cyclictest122750irq/25-em105:01:380
1601699341,30cyclictest0-21swapper/102:15:381
16006993417,8cyclictest22183-21modprobe03:48:590
242799330,4rtkit-daemon2426-21rtkit-daemon03:56:542
242799330,3rtkit-daemon2426-21rtkit-daemon03:08:012
1602499334,26cyclictest0-21swapper/205:08:572
16006993327,3cyclictest0-21swapper/005:49:220
1600699330,30cyclictest122750irq/25-em103:46:070
242799320,3rtkit-daemon2426-21rtkit-daemon03:57:540
242799320,2rtkit-daemon2426-21rtkit-daemon04:55:490
1602499324,18cyclictest0-21swapper/201:28:592
16024993215,9cyclictest12604-21sshd05:50:562
1601699321,28cyclictest0-21swapper/100:44:311
16006993223,7cyclictest20195-21perf03:18:320
1600699321,28cyclictest0-21swapper/003:30:540
325552310,1sleep1201ktimersoftd/103:16:561
242799310,3rtkit-daemon2426-21rtkit-daemon05:54:280
242799310,3rtkit-daemon2426-21rtkit-daemon05:29:312
242799310,3rtkit-daemon2426-21rtkit-daemon05:29:312
242799310,3rtkit-daemon2426-21rtkit-daemon04:35:540
242799310,3rtkit-daemon2426-21rtkit-daemon03:59:450
242799310,3rtkit-daemon2426-21rtkit-daemon02:29:102
242799310,3rtkit-daemon2426-21rtkit-daemon01:05:132
242799310,3rtkit-daemon2426-21rtkit-daemon00:54:170
242799310,2rtkit-daemon2426-21rtkit-daemon03:35:012
242799310,2rtkit-daemon2426-21rtkit-daemon00:44:313
16024993114,6cyclictest31616-21modprobe06:07:472
16006993122,6cyclictest0-21swapper/004:09:570
1600699311,27cyclictest0-21swapper/006:03:400
1600699311,27cyclictest0-21swapper/005:10:440
1600699311,26cyclictest0-21swapper/002:03:530
242799300,3rtkit-daemon2426-21rtkit-daemon05:22:370
242799300,3rtkit-daemon2426-21rtkit-daemon02:41:160
242799300,3rtkit-daemon2426-21rtkit-daemon00:59:592
242799300,2rtkit-daemon2426-21rtkit-daemon04:46:270
242799300,2rtkit-daemon2426-21rtkit-daemon04:10:332
1602499305,20cyclictest0-21swapper/201:39:222
1602499302,16cyclictest0-21swapper/205:19:362
1601699301,26cyclictest0-21swapper/104:39:171
1601699301,26cyclictest0-21swapper/102:25:371
1601699301,26cyclictest0-21swapper/101:35:201
1601699301,26cyclictest0-21swapper/100:39:231
1601699301,25cyclictest0-21swapper/105:16:231
1600699307,20cyclictest0-21swapper/002:34:350
1600699302,25cyclictest0-21swapper/002:45:210
1600699301,6cyclictest0-21swapper/005:25:150
101982302,7sleep20-21swapper/204:03:252
9942291,6sleep30-21swapper/301:05:023
73892291,6sleep20-21swapper/201:17:502
312752291,6sleep20-21swapper/201:57:342
310342291,6sleep30-21swapper/301:54:543
306782291,6sleep30-21swapper/301:02:093
283292291,6sleep30-21swapper/301:52:573
283292291,6sleep30-21swapper/301:52:573
276472291,6sleep20-21swapper/200:56:422
242799290,3rtkit-daemon2426-21rtkit-daemon05:15:440
242799290,3rtkit-daemon2426-21rtkit-daemon04:52:290
242799290,2rtkit-daemon2426-21rtkit-daemon04:56:182
242799290,2rtkit-daemon2426-21rtkit-daemon04:33:102
242799290,2rtkit-daemon2426-21rtkit-daemon02:42:232
222452291,6sleep30-21swapper/301:41:323
191402291,6sleep20-21swapper/201:35:252
1602499295,21cyclictest0-21swapper/203:50:092
16024992922,4cyclictest22876-21modprobe04:19:292
1602499292,23cyclictest0-21swapper/200:48:232
1602499292,16cyclictest0-21swapper/201:52:122
1602499292,16cyclictest0-21swapper/201:52:122
1602499291,24cyclictest0-21swapper/202:11:372
1601699292,23cyclictest0-21swapper/102:03:261
1601699292,23cyclictest0-21swapper/101:05:411
1601699291,25cyclictest0-21swapper/101:52:391
1601699291,25cyclictest0-21swapper/101:52:391
1601699291,25cyclictest0-21swapper/101:20:011
1601699291,25cyclictest0-21swapper/101:00:301
1600699296,20cyclictest23338-21kworker/0:405:35:130
1600699295,21cyclictest21371-21sshd04:16:530
16006992922,4cyclictest0-21swapper/005:28:240
16006992922,4cyclictest0-21swapper/005:28:240
1600699291,25cyclictest122750irq/25-em104:23:490
139382291,6sleep20-21swapper/202:23:002
104392291,6sleep30-21swapper/301:23:333
103612291,6sleep20-21swapper/201:22:402
72162281,6sleep30-21swapper/301:15:503
70602281,6sleep30-21swapper/305:55:353
42912281,6sleep30-21swapper/301:11:253
253422281,6sleep30-21swapper/301:47:593
242799280,3rtkit-daemon2426-21rtkit-daemon04:34:022
242799280,2rtkit-daemon2426-21rtkit-daemon04:59:582
242799280,2rtkit-daemon2426-21rtkit-daemon04:44:512
242799280,2rtkit-daemon2426-21rtkit-daemon02:48:530
22472281,6sleep30-21swapper/302:03:453
188982280,2sleep121-21ksoftirqd/104:09:011
187382281,6sleep20-21swapper/200:42:202
16232281,6sleep30-21swapper/301:59:563
1602499286,19cyclictest0-21swapper/205:17:092
1602499285,18cyclictest0-21swapper/202:45:132
1602499284,18cyclictest0-21swapper/201:43:542
1602499282,22cyclictest0-21swapper/200:52:142
1602499282,22cyclictest0-21swapper/200:52:142
1602499281,24cyclictest20834-21sshd05:38:492
1601699285,19cyclictest0-21swapper/101:45:281
1601699282,23cyclictest0-21swapper/100:53:431
1601699282,22cyclictest0-21swapper/104:24:121
1601699282,22cyclictest0-21swapper/102:59:461
1601699281,24cyclictest0-21swapper/105:46:221
1601699281,24cyclictest0-21swapper/105:12:241
1601699281,24cyclictest0-21swapper/104:46:491
1601699281,24cyclictest0-21swapper/102:11:181
1601699281,24cyclictest0-21swapper/101:24:211
1601699281,24cyclictest0-21swapper/101:12:411
1601699281,23cyclictest0-21swapper/104:22:511
1601699281,23cyclictest0-21swapper/101:55:271
1601699281,23cyclictest0-21swapper/100:51:031
1601699281,23cyclictest0-21swapper/100:51:031
1601699280,24cyclictest16162-21kworker/1:004:51:341
1600699289,16cyclictest0-21swapper/001:59:080
1600699286,19cyclictest0-21swapper/001:39:030
1600699282,23cyclictest0-21swapper/004:20:360
1600699282,23cyclictest0-21swapper/003:25:050
16006992813,12cyclictest0-21swapper/001:06:540
1600699281,24cyclictest0-21swapper/003:18:550
87992271,6sleep10-21swapper/103:09:591
242799270,2rtkit-daemon2426-21rtkit-daemon04:38:430
242799270,2rtkit-daemon2426-21rtkit-daemon04:30:210
242799270,2rtkit-daemon2426-21rtkit-daemon04:25:122
1603599270,23cyclictest31219-21kworker/3:003:06:413
1603599270,23cyclictest12162-21kworker/3:304:59:483
1602499275,16cyclictest0-21swapper/201:23:442
1602499274,15cyclictest0-21swapper/204:49:012
1602499274,15cyclictest0-21swapper/204:13:392
1602499272,21cyclictest0-21swapper/205:23:502
1602499271,22cyclictest0-21swapper/205:57:252
1602499271,22cyclictest0-21swapper/203:15:222
1602499271,22cyclictest0-21swapper/201:10:172
1602499271,15cyclictest0-21swapper/203:25:392
1601699272,22cyclictest0-21swapper/103:51:541
1601699272,21cyclictest0-21swapper/105:24:151
1601699272,21cyclictest0-21swapper/104:30:181
1601699271,23cyclictest9407-21kworker/1:404:55:081
1601699271,23cyclictest0-21swapper/102:29:371
1601699271,23cyclictest0-21swapper/102:19:251
1601699271,23cyclictest0-21swapper/102:04:181
1601699271,23cyclictest0-21swapper/101:32:201
1601699271,23cyclictest0-21swapper/101:14:511
1600699274,20cyclictest0-21swapper/001:46:500
16006992713,11cyclictest0-21swapper/001:15:220
16006992711,14cyclictest0-21swapper/006:00:030
1600699271,23cyclictest0-21swapper/000:40:380
1600699271,22cyclictest0-21swapper/001:48:590
1600699271,22cyclictest0-21swapper/001:48:590
275902261,4sleep30-21swapper/305:16:263
242799260,2rtkit-daemon2426-21rtkit-daemon05:34:052
242799260,2rtkit-daemon2426-21rtkit-daemon05:05:082
198622261,5sleep120079-21sshd06:04:121
160992261,6sleep10-21swapper/103:20:381
1603599261,22cyclictest0-21swapper/302:42:553
1603599261,21cyclictest0-21swapper/303:24:303
1602499267,5cyclictest22690-21sshd06:01:562
1602499265,12cyclictest0-21swapper/202:33:552
1602499264,15cyclictest0-21swapper/202:04:032
1602499262,20cyclictest26193-21sshd03:46:502
16024992612,8cyclictest32284-21sshd03:19:192
1602499261,21cyclictest0-21swapper/202:55:072
1602499261,21cyclictest0-21swapper/202:28:272
1601699261,22cyclictest0-21swapper/101:40:281
1601699261,21cyclictest0-21swapper/104:03:061
1601699261,21cyclictest0-21swapper/102:33:371
1601699260,23cyclictest8838-21kworker/1:005:54:151
1600699267,11cyclictest0-21swapper/000:46:200
33752252,18sleep10-21swapper/106:00:221
226072251,5sleep30-21swapper/303:56:403
213112251,6sleep30-21swapper/303:03:313
169102251,6sleep00-21swapper/002:27:510
1603599251,21cyclictest2529-21kworker/3:103:11:083
1603599251,21cyclictest12800-21kworker/3:305:38:153
1603599251,20cyclictest30749-21kworker/3:102:53:463
1603599251,20cyclictest22393-21sshd03:46:323
1603599251,14cyclictest0-21swapper/301:25:403
1603599250,22cyclictest9130-21kworker/3:303:43:183
1603599250,22cyclictest17240-21kworker/3:104:30:393
1603599250,22cyclictest10478-21kworker/3:304:12:503
1603599250,21cyclictest7449-21kworker/3:203:16:123
1603599250,21cyclictest2529-21kworker/3:103:20:153
1603599250,21cyclictest21869-21kworker/3:105:43:153
1603599250,21cyclictest12800-21kworker/3:305:23:353
1603599250,21cyclictest12725-21kworker/3:105:19:183
1603599250,21cyclictest12162-21kworker/3:304:51:333
1602499252,20cyclictest0-21swapper/202:14:362
16024992517,5cyclictest6990-21sshd04:38:392
1602499251,21cyclictest0-21swapper/203:40:162
1601699251,21cyclictest0-21swapper/104:15:561
1601699251,21cyclictest0-21swapper/102:54:561
1601699251,20cyclictest0-21swapper/105:35:271
1601699250,21cyclictest17606-21kworker/1:105:49:021
1600699257,14cyclictest0-21swapper/002:19:030
1600699254,15cyclictest0-21swapper/004:06:350
16006992511,11cyclictest14002-21kworker/0:102:58:540
16006992511,11cyclictest0-21swapper/001:18:540
16006992510,12cyclictest0-21swapper/001:24:530
1600699251,21cyclictest0-21swapper/005:45:090
1600699251,21cyclictest0-21swapper/002:28:470
1600699251,21cyclictest0-21swapper/000:53:340
1600699251,21cyclictest0-21swapper/000:53:340
1600699251,16cyclictest0-21swapper/002:16:200
1600699251,16cyclictest0-21swapper/001:33:530
78892241,18sleep18131-21sshd02:39:451
78522241,4sleep30-21swapper/302:11:323
308052241,3sleep10-21swapper/103:04:051
1603599241,20cyclictest7929-21kworker/3:002:22:033
1603599241,20cyclictest7620-21kworker/3:003:50:043
1603599241,19cyclictest12394-21kworker/3:305:08:443
1603599240,21cyclictest9130-21kworker/3:303:59:093
1603599240,21cyclictest29427-21kworker/3:404:35:343
1603599240,21cyclictest22612-21kworker/3:005:58:453
1603599240,21cyclictest10478-21kworker/3:304:19:273
1603599240,21cyclictest10478-21kworker/3:304:14:373
1603599240,20cyclictest12001-21kworker/3:004:24:313
1601699242,19cyclictest0-21swapper/103:25:501
1601699242,18cyclictest15563-21kworker/1:203:32:321
1601699242,18cyclictest0-21swapper/103:42:331
1601699241,20cyclictest9070-21kworker/1:305:05:281
1601699240,21cyclictest28296-21kworker/1:103:46:461
1600699244,15cyclictest23001-21sshd02:56:010
16006992410,11cyclictest0-21swapper/000:59:240
1600699241,20cyclictest0-21swapper/002:10:320
1600699241,20cyclictest0-21swapper/001:30:100
1600699241,20cyclictest0-21swapper/001:09:380
1603599233,16cyclictest0-21swapper/301:28:563
1603599231,19cyclictest10611-21kworker/3:004:53:453
1603599231,18cyclictest9285-21kworker/3:103:33:473
1603599231,18cyclictest16754-21munin-node01:34:133
1603599230,20cyclictest9285-21kworker/3:103:31:143
1603599230,20cyclictest22970-21kworker/3:106:03:533
1603599230,20cyclictest21869-21kworker/3:105:48:473
1603599230,20cyclictest17240-21kworker/3:104:44:493
1603599230,20cyclictest14293-21kworker/3:202:33:543
1603599230,20cyclictest11737-21kworker/3:205:05:313
1603599230,20cyclictest11180-21kworker/3:204:40:023
1603599230,19cyclictest26216-21kworker/3:302:48:353
1603599230,19cyclictest18547-21kworker/3:405:43:343
1603599230,19cyclictest11803-21kworker/3:005:29:053
1603599230,19cyclictest11803-21kworker/3:005:29:053
1601699231,19cyclictest20838-21kworker/1:105:21:351
1601699231,19cyclictest0-21swapper/105:30:371
1601699231,19cyclictest0-21swapper/105:30:371
1601699230,20cyclictest6592-21kworker/1:003:55:361
1601699230,20cyclictest19568-21kworker/1:105:00:041
1603599223,16cyclictest0-21swapper/300:54:153
1603599221,18cyclictest0-21swapper/302:30:233
1603599220,19cyclictest15594-21kworker/3:200:39:023
1603599220,18cyclictest10378-21kworker/3:004:03:533
1601699220,19cyclictest29016-21kworker/1:303:33:551
1601699220,19cyclictest15563-21kworker/1:204:03:531
1601699220,17cyclictest20940-21kworker/1:205:38:241
1603599215,13cyclictest0-21swapper/302:13:533
1603599212,16cyclictest0-21swapper/300:49:113
1603599212,16cyclictest0-21swapper/300:49:113
1603599211,17cyclictest0-21swapper/302:23:533
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional