You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-08-04 - 21:22

Intel(R) Xeon(R) CPU E31220L @ 2.20GHz, Linux 3.18.43-rt46-fulldebug (Profile)

Latency plot of system in rack #b, slot #0
Note that this system runs a non-optimized debug kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Characteristics of the alll highest latencies:
System rackbslot0.osadl.org (updated Wed Aug 04, 2021 12:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1930927357,11sleep20-21swapper/214:21:212
1921726550,10sleep30-21swapper/314:20:303
1914026238,19sleep10-21swapper/114:19:351
1946326136,20sleep00-21swapper/014:23:110
228042590,2sleep322806-21modprobe18:23:313
260362550,1sleep00-21swapper/019:46:520
242799530,2rtkit-daemon2426-21rtkit-daemon14:37:003
242799480,6rtkit-daemon0-21swapper/319:04:313
242799470,4rtkit-daemon0-21swapper/116:00:541
222412470,2sleep122236-21bash16:50:151
242799460,5rtkit-daemon0-21swapper/317:38:453
1968699461,42cyclictest4344-21modprobe19:21:001
242799450,5rtkit-daemon0-21swapper/318:40:493
242799430,5rtkit-daemon0-21swapper/318:15:113
1968699431,36cyclictest12762-21bash19:35:131
242799410,4rtkit-daemon0-21swapper/114:31:101
19704994126,11cyclictest14263-21sshd17:44:553
242799400,5rtkit-daemon0-21swapper/319:33:153
242799400,5rtkit-daemon0-21swapper/316:37:133
242799400,5rtkit-daemon0-21swapper/316:19:063
242799400,5rtkit-daemon0-21swapper/316:19:063
242799400,5rtkit-daemon0-21swapper/315:09:453
242799400,4rtkit-daemon0-21swapper/314:27:593
242799400,4rtkit-daemon0-21swapper/117:04:591
242799400,4rtkit-daemon0-21swapper/116:22:031
242799400,4rtkit-daemon0-21swapper/116:22:031
242799400,4rtkit-daemon0-21swapper/115:47:121
242799400,4rtkit-daemon0-21swapper/115:39:221
242799400,4rtkit-daemon0-21swapper/115:32:151
242799400,4rtkit-daemon0-21swapper/115:25:401
242799400,4rtkit-daemon0-21swapper/115:05:161
242799400,4rtkit-daemon0-21swapper/114:51:231
242799400,3rtkit-daemon0-21swapper/115:37:251
242799400,3rtkit-daemon0-21swapper/115:21:511
242799400,3rtkit-daemon0-21swapper/114:35:501
19704994019,17cyclictest31917-21systemd-cgroups19:31:123
19704994010,25cyclictest11796-21sshd19:50:413
242799390,5rtkit-daemon0-21swapper/316:54:563
242799390,5rtkit-daemon0-21swapper/315:55:193
242799390,4rtkit-daemon0-21swapper/319:39:533
242799390,4rtkit-daemon0-21swapper/318:08:073
242799390,4rtkit-daemon0-21swapper/317:13:483
242799390,4rtkit-daemon0-21swapper/316:07:393
242799390,4rtkit-daemon0-21swapper/315:40:543
242799390,4rtkit-daemon0-21swapper/315:38:133
242799390,4rtkit-daemon0-21swapper/315:03:373
242799390,4rtkit-daemon0-21swapper/314:57:413
242799390,4rtkit-daemon0-21swapper/314:50:183
242799390,4rtkit-daemon0-21swapper/314:29:313
242799390,4rtkit-daemon0-21swapper/116:16:291
242799390,4rtkit-daemon0-21swapper/116:10:341
242799390,4rtkit-daemon0-21swapper/116:05:191
242799390,4rtkit-daemon0-21swapper/114:53:421
242799390,3rtkit-daemon0-21swapper/115:53:361
242799390,3rtkit-daemon0-21swapper/115:48:501
242799390,3rtkit-daemon0-21swapper/114:43:371
242799390,3rtkit-daemon0-21swapper/114:38:401
19704993928,8cyclictest0-21swapper/319:13:263
1970499392,32cyclictest0-21swapper/318:22:133
19704993910,24cyclictest31113-21sshd19:12:373
242799380,5rtkit-daemon0-21swapper/317:55:583
242799380,5rtkit-daemon0-21swapper/316:50:553
242799380,5rtkit-daemon0-21swapper/316:27:313
242799380,5rtkit-daemon0-21swapper/316:14:213
242799380,5rtkit-daemon0-21swapper/315:28:173
242799380,5rtkit-daemon0-21swapper/315:23:023
242799380,5rtkit-daemon0-21swapper/315:02:273
242799380,4rtkit-daemon0-21swapper/318:32:223
242799380,4rtkit-daemon0-21swapper/315:29:213
242799380,4rtkit-daemon0-21swapper/315:17:533
242799380,4rtkit-daemon0-21swapper/119:14:031
242799380,3rtkit-daemon0-21swapper/315:53:073
242799380,3rtkit-daemon0-21swapper/115:13:521
242799380,3rtkit-daemon0-21swapper/115:12:121
242799380,3rtkit-daemon0-21swapper/115:01:131
242799380,3rtkit-daemon0-21swapper/114:26:331
242799380,2rtkit-daemon2426-21rtkit-daemon18:29:241
242799380,2rtkit-daemon0-21swapper/118:37:161
1970499382,8cyclictest0-21swapper/318:48:423
1970499382,5cyclictest0-21swapper/319:20:283
1970499382,31cyclictest0-21swapper/317:51:283
242799370,4rtkit-daemon7727-21sshd18:50:091
242799370,4rtkit-daemon0-21swapper/315:44:023
242799370,3rtkit-daemon0-21swapper/119:08:131
1970499377,4cyclictest122850irq/26-p4p1-rx-17:18:453
19704993710,23cyclictest19860-21sshd19:43:483
242799360,3rtkit-daemon2426-21rtkit-daemon19:31:401
242799360,3rtkit-daemon2426-21rtkit-daemon16:24:211
242799360,2rtkit-daemon0-21swapper/119:43:491
1970499369,22cyclictest1333-21sshd17:23:293
1970499369,22cyclictest10797-21sshd16:44:033
19704993610,21cyclictest25483-21systemd-cgroups17:13:083
19704993610,21cyclictest25483-21systemd-cgroups17:13:083
19686993622,10cyclictest4057-21bash17:34:011
19686993618,16cyclictest24006-21modprobe18:15:571
19686993611,21cyclictest22713-21bash18:20:541
242799350,2rtkit-daemon0-21swapper/117:12:491
242799350,2rtkit-daemon0-21swapper/117:12:491
1970499357,5cyclictest0-21swapper/319:28:123
1970499353,9cyclictest0-21swapper/316:38:533
1970499353,28cyclictest0-21swapper/318:58:333
1970499352,8cyclictest0-21swapper/318:01:233
1970499352,8cyclictest0-21swapper/318:01:233
1970499352,7cyclictest0-21swapper/316:58:303
1970499352,7cyclictest0-21swapper/314:47:493
1970499352,6cyclictest0-21swapper/318:44:323
1970499352,29cyclictest0-21swapper/316:31:223
1970499352,28cyclictest0-21swapper/317:07:063
1970499352,10cyclictest0-21swapper/317:35:423
19704993514,17cyclictest28809-21munin-node14:38:593
242799340,3rtkit-daemon12114-21chrt16:39:031
1970499342,9cyclictest0-21swapper/318:09:033
1970499342,9cyclictest0-21swapper/317:28:523
1970499342,8cyclictest0-21swapper/318:54:213
1970499342,7cyclictest0-21swapper/316:08:533
1970499342,10cyclictest0-21swapper/318:33:273
1969499341,30cyclictest0-21swapper/215:38:452
1969499341,22cyclictest17908-21kworker/2:218:29:312
242799330,3rtkit-daemon6108-21bash19:42:501
1970499332,8cyclictest0-21swapper/315:58:353
1969499331,21cyclictest30779-21kworker/2:316:34:522
1967699336,22cyclictest7721-21sshd16:31:170
242799320,3rtkit-daemon3410-21bash18:24:241
242799320,2rtkit-daemon25228-21bash17:15:311
242799310,3rtkit-daemon2426-21rtkit-daemon17:18:271
19676993114,13cyclictest0-21swapper/015:11:490
1967699311,27cyclictest0-21swapper/019:02:050
242799300,2rtkit-daemon2426-21rtkit-daemon19:26:091
1969499304,20cyclictest0-21swapper/219:05:082
1969499301,26cyclictest17908-21kworker/2:218:36:252
1969499301,25cyclictest29-21ksoftirqd/219:01:222
1969499301,19cyclictest24020-21kworker/2:317:37:382
1967699301,26cyclictest0-21swapper/016:45:190
1967699301,26cyclictest0-21swapper/014:40:130
297872291,6sleep00-21swapper/016:43:080
1969499299,16cyclictest0-21swapper/219:19:522
1969499294,11cyclictest0-21swapper/217:49:442
19694992914,11cyclictest16424-21kworker/2:018:22:062
1969499291,25cyclictest30779-21kworker/2:317:08:022
1969499291,25cyclictest20640-21kworker/2:216:13:492
1969499291,25cyclictest14879-21kworker/2:318:12:462
1969499291,24cyclictest18401-21kworker/2:418:55:532
1969499291,24cyclictest15402-21kworker/2:319:24:132
1969499291,18cyclictest18401-21kworker/2:418:49:362
1969499291,17cyclictest16424-21kworker/2:018:23:262
1968699292,12cyclictest0-21swapper/116:28:381
1968699291,25cyclictest26982-21sshd17:38:231
1967699299,10cyclictest41ktimersoftd/014:54:000
1967699292,23cyclictest0-21swapper/017:57:220
19676992910,16cyclictest0-21swapper/014:51:040
1967699291,25cyclictest0-21swapper/015:59:080
1967699291,25cyclictest0-21swapper/015:54:480
1967699291,25cyclictest0-21swapper/015:23:440
1967699291,25cyclictest0-21swapper/015:22:300
1967699291,25cyclictest0-21swapper/014:30:130
1967699291,25cyclictest0-21swapper/014:24:140
281442281,6sleep20-21swapper/214:34:382
242799280,2rtkit-daemon2426-21rtkit-daemon19:13:131
235072282,4sleep20-21swapper/217:23:082
1969499289,16cyclictest0-21swapper/219:16:002
1969499289,16cyclictest0-21swapper/216:12:342
1969499289,16cyclictest0-21swapper/215:47:312
1969499289,16cyclictest0-21swapper/215:24:192
1969499289,16cyclictest0-21swapper/214:45:242
1969499289,15cyclictest0-21swapper/215:03:462
1969499288,16cyclictest0-21swapper/218:40:292
1969499288,16cyclictest0-21swapper/216:52:062
1969499288,16cyclictest0-21swapper/216:38:342
1969499285,19cyclictest0-21swapper/217:38:392
19694992811,14cyclictest0-21swapper/219:35:152
1969499281,24cyclictest6286-21kworker/2:317:26:442
1969499281,24cyclictest16303-21kworker/2:218:15:392
1969499281,23cyclictest0-21swapper/216:43:422
1969499281,23cyclictest0-21swapper/215:35:042
1969499281,23cyclictest0-21swapper/214:29:422
1969499281,16cyclictest20746-21kworker/2:019:40:422
1969499281,16cyclictest0-21swapper/218:03:562
1969499280,24cyclictest11684-21kworker/2:115:58:492
1968699283,20cyclictest0-21swapper/118:55:381
19686992821,4cyclictest0-21swapper/116:56:331
1968699282,22cyclictest0-21swapper/117:58:331
1968699282,22cyclictest0-21swapper/117:58:331
1967699282,23cyclictest0-21swapper/018:05:160
19676992810,14cyclictest41ktimersoftd/019:09:270
1967699281,24cyclictest0-21swapper/018:21:380
1967699281,24cyclictest0-21swapper/016:04:000
1967699281,24cyclictest0-21swapper/015:43:030
1967699281,24cyclictest0-21swapper/014:58:370
1969499279,8cyclictest0-21swapper/216:24:202
1969499279,15cyclictest0-21swapper/219:48:432
1969499279,15cyclictest0-21swapper/219:46:532
1969499279,15cyclictest0-21swapper/215:30:372
1969499278,8cyclictest0-21swapper/216:23:232
1969499278,8cyclictest0-21swapper/216:23:232
1969499278,16cyclictest0-21swapper/218:00:022
1969499278,16cyclictest0-21swapper/218:00:022
1969499278,16cyclictest0-21swapper/217:13:592
1969499278,16cyclictest0-21swapper/216:54:142
1969499272,22cyclictest0-21swapper/215:09:552
1969499272,21cyclictest0-21swapper/214:52:352
19694992713,4cyclictest25587-21kworker/2:114:40:152
1969499271,23cyclictest0-21swapper/216:06:272
1969499271,23cyclictest0-21swapper/214:56:122
1969499271,22cyclictest24020-21kworker/2:317:48:042
1969499271,22cyclictest0-21swapper/219:11:122
1969499271,22cyclictest0-21swapper/215:19:012
1969499271,16cyclictest30779-21kworker/2:317:08:522
1969499271,16cyclictest30779-21kworker/2:317:08:522
1969499271,15cyclictest0-21swapper/218:44:162
1969499270,16cyclictest19052-21kworker/2:017:00:422
1968699278,16cyclictest29239-21sshd17:43:371
1968699272,21cyclictest0-21swapper/117:01:431
1968699272,20cyclictest0-21swapper/118:39:081
1968699271,3cyclictest0-21swapper/118:06:561
1968699271,23cyclictest0-21swapper/116:46:281
1967699279,14cyclictest41ktimersoftd/017:27:520
1967699277,11cyclictest0-21swapper/015:48:370
1967699272,21cyclictest0-21swapper/017:18:220
19676992710,13cyclictest41ktimersoftd/016:57:360
1967699271,24cyclictest0-21swapper/014:35:530
1967699271,23cyclictest0-21swapper/015:35:060
1967699271,22cyclictest0-21swapper/018:15:010
1967699271,22cyclictest0-21swapper/016:58:300
1967699271,22cyclictest0-21swapper/016:16:270
1967699271,17cyclictest0-21swapper/019:04:410
1969499268,7cyclictest0-21swapper/219:30:192
1969499268,15cyclictest0-21swapper/216:31:392
1969499268,15cyclictest0-21swapper/215:14:262
1969499268,15cyclictest0-21swapper/214:27:212
1969499268,14cyclictest0-21swapper/217:54:002
1969499261,22cyclictest0-21swapper/215:58:232
1969499261,22cyclictest0-21swapper/215:50:102
1969499261,22cyclictest0-21swapper/215:00:372
1969499261,21cyclictest0-21swapper/217:28:442
1968699262,22cyclictest0-21swapper/116:36:081
1968699262,21cyclictest0-21swapper/118:08:491
1968699261,22cyclictest0-21swapper/117:48:211
1968699261,22cyclictest0-21swapper/117:24:211
1967699268,15cyclictest122750irq/25-em118:26:090
1967699267,15cyclictest41ktimersoftd/017:40:580
1967699267,15cyclictest0-21swapper/015:28:280
1967699266,17cyclictest0-21swapper/015:16:120
1967699265,18cyclictest0-21swapper/015:07:220
1967699262,21cyclictest0-21swapper/017:32:220
19676992613,10cyclictest122750irq/25-em118:38:570
1967699261,22cyclictest0-21swapper/017:11:390
1967699261,22cyclictest0-21swapper/017:11:390
1967699261,22cyclictest0-21swapper/016:21:070
1967699261,22cyclictest0-21swapper/016:21:070
1967699261,22cyclictest0-21swapper/014:45:200
1967699261,21cyclictest0-21swapper/016:08:570
1967699261,21cyclictest0-21swapper/015:43:290
13592261,5sleep11562-21sshd17:28:521
1968699252,19cyclictest0-21swapper/117:53:431
1968699251,21cyclictest0-21swapper/118:43:411
1967699255,9cyclictest41ktimersoftd/017:52:280
1967699252,20cyclictest0-21swapper/016:50:020
19676992512,10cyclictest122750irq/25-em117:37:340
19676992512,10cyclictest122750irq/25-em116:24:520
19676992510,11cyclictest122750irq/25-em119:23:310
19676992510,11cyclictest122750irq/25-em116:34:460
1967699251,21cyclictest0-21swapper/019:22:300
1967699251,21cyclictest0-21swapper/017:16:110
1967699251,16cyclictest0-21swapper/019:37:550
1967699250,3cyclictest8598-21sshd19:53:010
1967699250,3cyclictest26046-21sshd18:33:480
1967699250,3cyclictest22984-21sshd18:28:340
1967699250,3cyclictest15349-21sshd17:45:010
1968699242,18cyclictest0-21swapper/119:48:531
1968699242,18cyclictest0-21swapper/118:59:101
1967699245,15cyclictest41ktimersoftd/019:13:500
1967699242,19cyclictest0-21swapper/018:48:000
19676992411,9cyclictest41ktimersoftd/019:29:190
1967699241,20cyclictest0-21swapper/018:11:380
1967699241,20cyclictest0-21swapper/017:05:470
1967699241,13cyclictest0-21swapper/019:38:480
1967699241,13cyclictest0-21swapper/018:54:520
1967699237,13cyclictest0-21swapper/017:58:400
1967699237,13cyclictest0-21swapper/017:58:400
1967699232,11cyclictest0-21swapper/018:48:350
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional