You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-10-17 - 21:30

Intel(R) Xeon(R) CPU E31220L @ 2.20GHz, Linux 3.18.43-rt46-fulldebug (Profile)

Latency plot of system in rack #b, slot #0
Note that this system runs a non-optimized debug kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Characteristics of the highest latencies:
System rackbslot0.osadl.org (updated Sun Oct 17, 2021 12:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2995826338,11sleep30-21swapper/312:36:593
242799610,3rtkit-daemon0-21swapper/012:40:322
2983526036,19sleep10-21swapper/112:35:371
2973426043,11sleep20-21swapper/212:34:252
2983925842,11sleep00-21swapper/012:35:400
242799560,4rtkit-daemon0-21swapper/112:44:541
9232550,1sleep25469-21kworker/2:314:52:002
242799540,4rtkit-daemon0-21swapper/113:37:591
242799520,3rtkit-daemon0-21swapper/113:03:361
242799510,4rtkit-daemon0-21swapper/114:08:051
242799490,4rtkit-daemon0-21swapper/113:36:151
242799490,3rtkit-daemon0-21swapper/113:26:101
30255994830,9cyclictest11709-21sshd15:27:481
30255994830,9cyclictest11709-21sshd15:27:481
242799480,5rtkit-daemon0-21swapper/115:34:341
3027699473,36cyclictest21377-21modprobe15:59:023
3026599462,36cyclictest6422-21sshd15:42:382
30276994539,4cyclictest0-21swapper/316:28:193
30276994528,9cyclictest27715-21latency_hist13:27:553
30276994515,27cyclictest25630-21kworker/u8:015:15:223
30265994528,9cyclictest23701-21sshd15:21:022
30265994431,8cyclictest3789-21modprobe15:55:082
3025599443,33cyclictest31709-21dump-pmu-power14:57:501
3024699431,25cyclictest1841-21runrttasks15:14:500
242799430,4rtkit-daemon0-21swapper/116:21:131
242799430,2rtkit-daemon2426-21rtkit-daemon14:59:431
3027699421,35cyclictest11984-21kworker/3:416:53:463
3024699424,34cyclictest32387-21tr13:33:210
30265994124,11cyclictest13716-21sshd17:52:582
3024699418,29cyclictest14414-21sshd15:40:390
3024699411,36cyclictest10025-21sshd16:54:100
3024699411,36cyclictest0-21swapper/013:06:350
259612410,2sleep20-21swapper/216:02:032
242799410,2rtkit-daemon2426-21rtkit-daemon16:16:411
242799410,2rtkit-daemon2426-21rtkit-daemon16:16:411
242799410,2rtkit-daemon2426-21rtkit-daemon15:59:531
242799410,2rtkit-daemon2426-21rtkit-daemon15:12:131
30265994021,16cyclictest7612-21modprobe16:41:182
3024699408,27cyclictest8061-21cat13:48:100
3024699402,22cyclictest27908-21sshd17:18:270
3024699402,22cyclictest0-21swapper/018:04:360
3024699401,35cyclictest12787-21munin-node13:53:260
242799400,4rtkit-daemon0-21swapper/113:27:571
242799400,4rtkit-daemon0-21swapper/113:27:571
242799400,2rtkit-daemon2426-21rtkit-daemon12:59:041
30276993917,19cyclictest5167-21bash18:05:013
30265993921,16cyclictest29204-21kworker/u8:117:37:282
3024699394,30cyclictest122750irq/25-em114:08:230
3024699392,33cyclictest0-21swapper/017:16:050
3024699391,34cyclictest3102-21sshd16:48:310
3024699391,34cyclictest30810-21latency_hist13:32:550
3024699391,34cyclictest30810-21latency_hist13:32:550
3024699391,34cyclictest18324-21modprobe14:59:230
3024699391,34cyclictest13906-21atenall_rbpower17:42:530
3024699391,34cyclictest12513-21modprobe15:10:510
3024699391,34cyclictest0-21swapper/016:40:370
3024699391,34cyclictest0-21swapper/015:18:500
3024699391,33cyclictest4263-21runrttasks13:41:510
3024699391,33cyclictest29920-21sshd16:35:240
3024699391,33cyclictest20047-21modprobe17:30:330
3024699391,26cyclictest3036-21sshd16:58:390
242799390,3rtkit-daemon0-21swapper/318:02:363
242799390,3rtkit-daemon0-21swapper/114:52:131
242799390,2rtkit-daemon2426-21rtkit-daemon13:21:091
242799390,2rtkit-daemon2426-21rtkit-daemon12:53:571
3024699384,30cyclictest0-21swapper/012:50:060
3024699383,31cyclictest21578-21sshd16:09:130
3024699382,32cyclictest0-21swapper/017:34:130
3024699382,32cyclictest0-21swapper/016:24:120
3024699382,32cyclictest0-21swapper/015:51:330
3024699381,33cyclictest21306-21sshd17:23:040
3024699381,33cyclictest21206-21sshd16:22:050
3024699381,33cyclictest0-21swapper/016:15:310
3024699381,33cyclictest0-21swapper/016:15:310
3024699381,33cyclictest0-21swapper/015:44:340
3024699381,33cyclictest0-21swapper/014:39:270
3024699381,32cyclictest28290-21sshd17:54:040
3024699381,28cyclictest0-21swapper/016:42:520
3024699381,22cyclictest31039-21sshd14:51:440
242799380,3rtkit-daemon0-21swapper/114:23:081
242799380,3rtkit-daemon0-21swapper/113:10:291
3027699377,27cyclictest13125-21kworker/3:016:34:473
3024699379,24cyclictest122750irq/25-em117:58:160
3024699378,26cyclictest0-21swapper/014:28:060
3024699378,26cyclictest0-21swapper/014:28:060
3024699377,27cyclictest18234-21kworker/0:316:30:000
3024699376,27cyclictest0-21swapper/017:49:050
3024699375,27cyclictest8813-21sshd15:07:270
3024699373,29cyclictest2292-21Xorg12:41:560
3024699372,31cyclictest0-21swapper/017:08:270
3024699372,31cyclictest0-21swapper/015:53:250
3024699372,31cyclictest0-21swapper/015:33:290
3024699372,31cyclictest0-21swapper/013:21:010
30246993719,14cyclictest0-21swapper/013:27:410
3024699371,32cyclictest2134-21cut14:33:050
3024699371,32cyclictest17561-21modprobe15:58:440
3024699371,32cyclictest0-21swapper/017:37:540
3024699371,32cyclictest0-21swapper/013:13:130
3024699371,31cyclictest1443-21snmpd14:14:120
3024699371,31cyclictest12026-21sshd17:04:290
3024699371,26cyclictest1841-21runrttasks14:26:440
3024699371,21cyclictest1841-21runrttasks14:05:030
3024699361,31cyclictest15904-21sshd15:23:030
3024699361,31cyclictest15904-21sshd15:23:030
3024699361,31cyclictest1443-21snmpd12:53:400
3024699361,31cyclictest0-21swapper/015:28:540
3024699361,31cyclictest0-21swapper/014:47:230
3024699361,31cyclictest0-21swapper/012:58:230
3024699361,30cyclictest15725-21munin-node14:53:160
3024699361,25cyclictest2903-21sshd16:02:470
3024699361,18cyclictest15326-21open_files13:58:220
242799360,3rtkit-daemon1503-21nfsd17:01:063
242799360,2rtkit-daemon26621-21sshd16:30:041
3027699355,24cyclictest0-21swapper/316:18:023
30276993525,7cyclictest0-21swapper/315:17:523
3024699351,30cyclictest6179-21nfsd413:43:230
3024699351,29cyclictest27232-21munin-node14:18:190
252222350,2sleep119-21rcuc/114:17:471
242799350,3rtkit-daemon2426-21rtkit-daemon14:36:331
76362340,1sleep37638-21modprobe17:29:333
3027699341,7cyclictest0-21swapper/317:22:273
3027699341,30cyclictest0-21swapper/317:35:393
3026599343,24cyclictest0-21swapper/217:23:322
3024699341,29cyclictest122750irq/25-em112:43:200
3024699341,28cyclictest17902-21munin-node13:08:300
242799340,4rtkit-daemon0-21swapper/115:57:041
242799340,4rtkit-daemon0-21swapper/113:14:111
242799340,3rtkit-daemon2426-21rtkit-daemon17:42:461
242799340,3rtkit-daemon2426-21rtkit-daemon14:06:091
242799340,2rtkit-daemon0-21swapper/116:24:181
242799340,1rtkit-daemon0-21swapper/116:55:321
3027699331,21cyclictest5384-21kworker/3:117:03:143
242799330,6rtkit-daemon2426-21rtkit-daemon16:03:401
242799330,3rtkit-daemon2426-21rtkit-daemon17:24:261
242799330,3rtkit-daemon2426-21rtkit-daemon17:11:591
242799330,3rtkit-daemon2426-21rtkit-daemon16:11:101
242799330,3rtkit-daemon2426-21rtkit-daemon15:43:231
242799330,3rtkit-daemon2426-21rtkit-daemon15:42:391
242799330,3rtkit-daemon2426-21rtkit-daemon15:29:311
242799330,3rtkit-daemon2426-21rtkit-daemon15:18:541
242799330,3rtkit-daemon2426-21rtkit-daemon15:15:421
242799330,3rtkit-daemon2426-21rtkit-daemon15:02:531
242799330,3rtkit-daemon2426-21rtkit-daemon14:45:341
242799330,3rtkit-daemon2426-21rtkit-daemon14:29:561
242799330,3rtkit-daemon2426-21rtkit-daemon14:29:561
242799330,3rtkit-daemon2426-21rtkit-daemon14:20:161
242799330,3rtkit-daemon2426-21rtkit-daemon13:58:211
242799330,3rtkit-daemon2426-21rtkit-daemon13:53:041
242799330,3rtkit-daemon2426-21rtkit-daemon13:49:481
242799330,3rtkit-daemon2426-21rtkit-daemon13:44:121
242799330,3rtkit-daemon2426-21rtkit-daemon12:49:161
242799330,3rtkit-daemon2426-21rtkit-daemon12:42:041
242799330,2rtkit-daemon27687-21sshd18:01:441
242799330,2rtkit-daemon10627-21sshd17:42:391
242799330,2rtkit-daemon0-21swapper/117:47:421
3027699325,17cyclictest0-21swapper/316:26:463
30276993217,9cyclictest8-21rcu_preempt15:37:013
3027699321,28cyclictest0-21swapper/315:28:223
3027699321,27cyclictest13125-21kworker/3:016:44:233
242799320,4rtkit-daemon21-21ksoftirqd/117:07:001
242799320,4rtkit-daemon1507-21nfsd17:19:521
242799320,4rtkit-daemon0-21swapper/114:40:401
242799320,3rtkit-daemon2426-21rtkit-daemon16:38:161
3027699318,19cyclictest0-21swapper/314:08:143
3027699314,17cyclictest0-21swapper/312:40:493
3027699312,25cyclictest0-21swapper/315:53:433
3027699312,17cyclictest0-21swapper/316:03:213
3027699311,27cyclictest19274-21kworker/3:114:38:513
3027699311,26cyclictest24390-21kworker/3:317:38:053
3027699311,20cyclictest8901-21kworker/3:317:14:373
3027699311,19cyclictest30810-21kworker/3:115:10:143
3027699311,18cyclictest8901-21kworker/3:317:22:553
3025599311,27cyclictest0-21swapper/117:33:421
242799310,3rtkit-daemon2426-21rtkit-daemon16:58:251
242799310,3rtkit-daemon2426-21rtkit-daemon16:48:471
242799310,3rtkit-daemon2426-21rtkit-daemon16:45:231
242799310,3rtkit-daemon2426-21rtkit-daemon16:34:211
242799310,2rtkit-daemon2426-21rtkit-daemon18:04:191
242799310,2rtkit-daemon2426-21rtkit-daemon17:28:071
242799310,2rtkit-daemon2426-21rtkit-daemon17:13:061
189832311,6sleep20-21swapper/214:04:442
14642311,6sleep20-21swapper/214:28:572
14642311,6sleep20-21swapper/214:28:572
88942301,6sleep30-21swapper/313:48:193
3027699309,18cyclictest0-21swapper/313:03:113
3027699309,17cyclictest0-21swapper/314:47:563
3027699305,21cyclictest0-21swapper/313:38:413
3027699305,18cyclictest0-21swapper/314:35:143
3027699305,18cyclictest0-21swapper/312:51:403
3027699305,18cyclictest0-21swapper/312:46:403
3027699303,23cyclictest0-21swapper/313:12:063
3027699302,25cyclictest0-21swapper/317:07:503
3027699302,24cyclictest0-21swapper/315:05:223
30276993015,4cyclictest13125-21kworker/3:016:16:493
30276993015,4cyclictest13125-21kworker/3:016:16:493
30276993015,12cyclictest5212-21kworker/3:216:37:563
3027699301,26cyclictest22418-21kworker/3:215:26:233
3027699301,26cyclictest22418-21kworker/3:215:26:233
3027699301,26cyclictest0-21swapper/317:45:133
3027699301,18cyclictest5384-21kworker/3:116:48:583
3027699301,18cyclictest27852-21kworker/3:317:50:243
3027699301,18cyclictest11230-21kworker/3:017:57:283
3027699301,18cyclictest0-21swapper/315:50:323
3026599305,21cyclictest0-21swapper/213:34:172
3026599305,18cyclictest0-21swapper/214:24:512
3026599304,15cyclictest0-21swapper/215:32:062
30265993018,6cyclictest19321-21awk17:07:422
3026599301,26cyclictest0-21swapper/216:03:512
3026599301,26cyclictest0-21swapper/215:09:322
242799300,4rtkit-daemon0-21swapper/114:54:543
242799300,2rtkit-daemon2426-21rtkit-daemon15:49:101
67852291,6sleep20-21swapper/212:52:562
48252291,5sleep20-21swapper/214:36:562
39932291,6sleep20-21swapper/213:38:382
3027699299,16cyclictest0-21swapper/313:53:183
3027699295,21cyclictest0-21swapper/313:47:403
3027699295,20cyclictest0-21swapper/314:59:373
3027699295,17cyclictest0-21swapper/314:04:243
3027699295,15cyclictest0-21swapper/313:00:243
3027699294,18cyclictest0-21swapper/312:56:253
3027699292,24cyclictest0-21swapper/314:43:343
3027699292,17cyclictest0-21swapper/313:36:513
30276992910,9cyclictest0-21swapper/316:08:033
3027699291,25cyclictest5575-21kworker/3:015:45:503
3026599291,24cyclictest0-21swapper/214:00:162
242799290,2rtkit-daemon2426-21rtkit-daemon17:56:261
303732281,6sleep30-21swapper/313:29:273
303732281,6sleep30-21swapper/313:29:273
3027699285,17cyclictest0-21swapper/313:17:333
3027699283,15cyclictest0-21swapper/314:18:403
3027699281,24cyclictest8597-21kworker/3:115:38:063
3026599285,19cyclictest0-21swapper/217:45:392
3026599285,17cyclictest0-21swapper/213:45:072
3026599285,15cyclictest0-21swapper/216:17:402
3026599285,15cyclictest0-21swapper/216:17:402
3026599285,15cyclictest0-21swapper/215:04:312
3026599285,13cyclictest0-21swapper/214:45:082
3026599285,13cyclictest0-21swapper/212:57:192
3026599284,14cyclictest0-21swapper/213:05:512
3026599283,21cyclictest0-21swapper/213:13:002
3026599282,23cyclictest0-21swapper/213:55:562
30265992814,9cyclictest10025-21runrttasks13:49:342
43292271,6sleep20-21swapper/215:27:132
43292271,6sleep20-21swapper/215:27:132
3027699275,15cyclictest0-21swapper/313:20:143
30276992712,4cyclictest19274-21kworker/3:114:23:213
3027699271,23cyclictest0-21swapper/314:13:513
3027699271,22cyclictest0-21swapper/313:58:163
3026599274,18cyclictest0-21swapper/215:44:412
3026599274,17cyclictest0-21swapper/216:37:342
30265992710,15cyclictest0-21swapper/216:27:062
3026599271,23cyclictest0-21swapper/213:25:582
3026599271,23cyclictest0-21swapper/213:07:592
3026599271,18cyclictest0-21swapper/216:28:282
3026599271,12cyclictest0-21swapper/216:54:482
283802271,3sleep20-21swapper/213:28:102
283802271,3sleep20-21swapper/213:28:102
16642271,6sleep30-21swapper/314:31:113
16642271,6sleep30-21swapper/314:31:113
3026599269,14cyclictest0-21swapper/217:21:272
3026599264,12cyclictest0-21swapper/215:14:462
3026599264,12cyclictest0-21swapper/212:59:242
3026599263,21cyclictest0-21swapper/212:43:042
3026599262,21cyclictest0-21swapper/215:01:282
3026599262,21cyclictest0-21swapper/213:21:292
3026599262,20cyclictest0-21swapper/217:13:302
3026599262,20cyclictest0-21swapper/215:34:022
3026599262,20cyclictest0-21swapper/214:55:032
3026599262,18cyclictest0-21swapper/217:01:162
30265992612,4cyclictest122850irq/26-p4p1-rx-16:42:522
30265992611,11cyclictest24299-21sshd15:49:022
66752251,20sleep21507-21nfsd17:39:442
3026599255,18cyclictest0-21swapper/217:48:042
3026599255,15cyclictest0-21swapper/216:50:122
3026599254,18cyclictest0-21swapper/217:10:582
3026599253,18cyclictest0-21swapper/216:18:282
3026599253,17cyclictest0-21swapper/217:29:502
3026599252,17cyclictest0-21swapper/214:15:212
3026599252,17cyclictest0-21swapper/214:09:542
3026599251,21cyclictest0-21swapper/214:41:492
301602251,4sleep20-21swapper/218:07:052
3026599245,14cyclictest0-21swapper/214:19:522
3026599244,17cyclictest0-21swapper/216:09:562
3026599231,19cyclictest0-21swapper/218:00:172
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional