You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-05-12 - 13:55

Intel(R) Xeon(R) CPU E31220L @ 2.20GHz, Linux 3.18.43-rt46-fulldebug (Profile)

Latency plot of system in rack #b, slot #0
Note that this system runs a non-optimized debug kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: ondemand
Characteristics of the highest latencies:
System rackbslot0.osadl.org (updated Wed May 12, 2021 00:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
242799950,6rtkit-daemon2426-21rtkit-daemon07:52:520
242799950,6rtkit-daemon2426-21rtkit-daemon07:52:520
1755627459,10sleep30-21swapper/304:22:133
1764727155,11sleep10-21swapper/104:23:191
1767427036,11sleep00-21swapper/004:23:340
322142690,2sleep332209-21bash08:40:193
322142690,2sleep332209-21bash08:40:193
1765726953,11sleep20-21swapper/204:23:242
242799620,14rtkit-daemon26023-21modprobe07:43:520
248902610,5sleep01789099cyclictest08:11:030
1791499493,23cyclictest0-21swapper/204:31:542
242799470,11rtkit-daemon0-21swapper/208:34:162
17890994730,14cyclictest4271-21modprobe06:42:500
1789099473,40cyclictest3777-21modprobe09:29:410
1789099473,40cyclictest3777-21modprobe09:29:410
1789099471,43cyclictest813-21kworker/0:309:10:520
1791499464,20cyclictest0-21swapper/205:35:542
1791499464,20cyclictest0-21swapper/205:35:542
1789099462,36cyclictest24170-21modprobe08:30:500
1790499431,38cyclictest3528-21sshd09:35:371
242799420,7rtkit-daemon0-21swapper/005:36:260
242799420,7rtkit-daemon0-21swapper/005:36:260
17890994228,8cyclictest8120-21modprobe06:33:550
242799400,4rtkit-daemon0-21swapper/006:12:240
242799400,4rtkit-daemon0-21swapper/005:46:580
242799400,4rtkit-daemon0-21swapper/005:46:580
242799400,4rtkit-daemon0-21swapper/005:08:480
242799400,4rtkit-daemon0-21swapper/004:53:420
242799400,3rtkit-daemon0-21swapper/206:26:352
17914994025,9cyclictest10693-21sshd09:19:072
17914994023,9cyclictest7616-21modprobe08:58:562
17890994025,8cyclictest26575-21sshd09:25:280
17890994025,8cyclictest26575-21sshd09:25:280
1791499395,17cyclictest0-21swapper/207:17:162
1791499395,11cyclictest0-21swapper/207:13:372
251782380,3sleep225527-21modprobe06:34:462
242799380,3rtkit-daemon0-21swapper/005:51:040
242799380,3rtkit-daemon0-21swapper/004:55:590
17890993822,13cyclictest29407-21modprobe08:45:400
242799370,4rtkit-daemon0-21swapper/207:24:342
242799370,2rtkit-daemon2426-21rtkit-daemon09:00:403
1791499374,15cyclictest0-21swapper/206:58:502
17890993727,6cyclictest30276-21systemd-cgroups07:27:550
17890993714,20cyclictest26942-21kworker/0:007:08:470
242799360,3rtkit-daemon0-21swapper/005:28:210
242799360,3rtkit-daemon0-21swapper/005:28:210
17914993624,7cyclictest1333-21sshd08:40:432
17914993624,7cyclictest1333-21sshd08:40:432
17914993624,6cyclictest18178-21bash08:30:292
1789099367,19cyclictest0-21swapper/006:14:130
1789099366,27cyclictest0-21swapper/009:39:570
17890993628,6cyclictest760-21sshd06:24:180
17890993625,8cyclictest6850irq/24-0000:00:07:23:520
242799350,4rtkit-daemon0-21swapper/209:38:302
242799350,3rtkit-daemon0-21swapper/006:06:580
242799350,2rtkit-daemon11454-21modprobe07:46:422
1791499355,26cyclictest0-21swapper/206:01:342
17914993523,6cyclictest17204-21sshd09:32:322
17914993523,6cyclictest17204-21sshd09:32:322
280942340,0sleep10-21swapper/108:14:491
242799340,4rtkit-daemon0-21swapper/208:02:382
242799340,3rtkit-daemon0-21swapper/208:18:052
242799340,2rtkit-daemon2426-21rtkit-daemon08:59:181
242799340,2rtkit-daemon0-21swapper/008:15:370
242799340,1rtkit-daemon0-21swapper/208:44:022
1791499346,24cyclictest29367-21kworker/u8:208:05:512
17890993417,14cyclictest122750irq/25-em109:03:120
242799330,3rtkit-daemon2426-21rtkit-daemon09:26:272
242799330,3rtkit-daemon2426-21rtkit-daemon09:26:272
242799330,3rtkit-daemon2426-21rtkit-daemon09:11:172
242799330,3rtkit-daemon2426-21rtkit-daemon08:52:392
242799330,3rtkit-daemon12087-21sshd08:21:110
242799330,2rtkit-daemon8394-21sshd07:19:212
242799330,2rtkit-daemon4899-21sshd07:57:122
242799330,2rtkit-daemon2088-21sshd08:54:592
242799330,2rtkit-daemon19201-21sshd09:15:573
242799330,2rtkit-daemon0-21swapper/206:29:262
242799330,2rtkit-daemon0-21swapper/005:54:500
242799320,3rtkit-daemon2426-21rtkit-daemon08:22:252
242799320,3rtkit-daemon2426-21rtkit-daemon07:03:332
242799320,3rtkit-daemon2426-21rtkit-daemon05:14:490
242799320,2rtkit-daemon2426-21rtkit-daemon05:21:170
242799320,2rtkit-daemon0-21swapper/209:14:062
242799320,2rtkit-daemon0-21swapper/007:42:010
1791499326,17cyclictest0-21swapper/204:27:202
17914993218,8cyclictest19426-21rm08:10:432
1790499321,27cyclictest0-21swapper/106:14:381
1790499321,27cyclictest0-21swapper/105:58:191
1789099321,27cyclictest0-21swapper/004:37:560
242799310,3rtkit-daemon2426-21rtkit-daemon09:44:242
242799310,3rtkit-daemon2426-21rtkit-daemon09:42:272
242799310,3rtkit-daemon2426-21rtkit-daemon06:38:460
242799310,3rtkit-daemon2426-21rtkit-daemon06:21:050
242799310,3rtkit-daemon2426-21rtkit-daemon05:59:490
242799310,3rtkit-daemon2426-21rtkit-daemon05:39:270
242799310,3rtkit-daemon2426-21rtkit-daemon05:12:140
242799310,3rtkit-daemon2426-21rtkit-daemon05:00:090
242799310,3rtkit-daemon2426-21rtkit-daemon04:44:070
242799310,3rtkit-daemon2426-21rtkit-daemon04:40:480
242799310,3rtkit-daemon2426-21rtkit-daemon04:29:170
242799310,3rtkit-daemon2426-21rtkit-daemon04:28:550
242799310,1rtkit-daemon0-21swapper/007:00:360
1791499314,23cyclictest0-21swapper/206:21:272
1791499314,23cyclictest0-21swapper/205:53:472
1791499313,19cyclictest0-21swapper/205:27:302
1791499313,19cyclictest0-21swapper/205:27:302
17914993123,4cyclictest17955-21sshd06:49:022
1790499311,27cyclictest0-21swapper/106:04:391
17890993118,8cyclictest17294-21sshd09:14:030
1789099310,4cyclictest1454-21sshd09:37:340
1789099310,3cyclictest23749-21sshd06:47:330
1789099310,11cyclictest15117-21sshd07:37:550
155062310,1sleep115508-21systemd-cgroups07:39:371
149252314,5sleep20-21swapper/205:09:582
29752301,6sleep30-21swapper/304:49:313
242799300,3rtkit-daemon2426-21rtkit-daemon09:18:011
242799300,3rtkit-daemon2426-21rtkit-daemon07:14:100
242799300,2rtkit-daemon2426-21rtkit-daemon09:20:221
242799300,2rtkit-daemon2426-21rtkit-daemon06:46:462
242799300,10rtkit-daemon2426-21rtkit-daemon07:51:162
242799300,10rtkit-daemon2426-21rtkit-daemon07:51:162
17922993010,17cyclictest0-21swapper/309:52:003
1791499305,21cyclictest0-21swapper/205:20:332
1791499305,21cyclictest0-21swapper/205:14:012
1791499305,19cyclictest0-21swapper/205:57:102
1790499309,10cyclictest20118-21kworker/1:006:45:291
1790499302,24cyclictest0-21swapper/106:54:441
1790499301,26cyclictest0-21swapper/106:03:391
1790499301,26cyclictest0-21swapper/105:46:481
1790499301,26cyclictest0-21swapper/105:46:481
1790499301,26cyclictest0-21swapper/105:32:201
1790499301,26cyclictest0-21swapper/105:09:501
1790499301,26cyclictest0-21swapper/104:30:041
1789099309,18cyclictest0-21swapper/008:04:430
1789099308,19cyclictest9407-21kworker/0:007:32:210
1789099302,13cyclictest0-21swapper/005:29:260
1789099301,26cyclictest0-21swapper/009:45:080
1789099301,26cyclictest0-21swapper/009:03:490
96812291,6sleep30-21swapper/305:56:333
92972291,6sleep10-21swapper/105:03:431
5902291,6sleep30-21swapper/305:42:043
294202291,6sleep30-21swapper/304:40:563
272262291,6sleep30-21swapper/305:33:403
242799290,9rtkit-daemon2426-21rtkit-daemon06:35:573
242799290,3rtkit-daemon0-21swapper/207:36:292
242799290,2rtkit-daemon31794-21sshd09:11:093
242799290,2rtkit-daemon2426-21rtkit-daemon09:53:002
242799290,2rtkit-daemon2426-21rtkit-daemon09:47:511
242799290,2rtkit-daemon2426-21rtkit-daemon08:58:411
242799290,2rtkit-daemon2426-21rtkit-daemon08:50:260
242799290,2rtkit-daemon2426-21rtkit-daemon08:36:513
242799290,2rtkit-daemon2426-21rtkit-daemon08:27:222
242799290,1rtkit-daemon0-21swapper/207:08:302
240382291,6sleep30-21swapper/305:26:063
240382291,6sleep30-21swapper/305:26:063
215022291,6sleep30-21swapper/306:14:563
210102291,6sleep30-21swapper/305:20:393
202362291,6sleep30-21swapper/308:30:373
1791499297,10cyclictest0-21swapper/209:07:182
1791499295,15cyclictest0-21swapper/206:07:042
1791499294,21cyclictest0-21swapper/207:32:492
1791499293,22cyclictest0-21swapper/205:29:512
1791499292,23cyclictest0-21swapper/206:09:582
1790499292,24cyclictest0-21swapper/105:19:491
1790499292,23cyclictest0-21swapper/107:07:291
1790499292,17cyclictest0-21swapper/104:54:141
1790499292,16cyclictest0-21swapper/105:41:241
1790499291,25cyclictest0-21swapper/106:20:221
1790499291,25cyclictest0-21swapper/105:25:211
1790499291,25cyclictest0-21swapper/105:25:211
1790499291,24cyclictest0-21swapper/106:28:571
17890992912,6cyclictest2976-21sshd06:57:130
17890992910,16cyclictest0-21swapper/009:49:480
1789099291,4cyclictest11-21rcuc/008:01:390
1789099291,25cyclictest0-21swapper/007:11:500
156702291,6sleep30-21swapper/306:06:573
242799280,2rtkit-daemon24568-21sshd09:05:213
242799280,2rtkit-daemon2426-21rtkit-daemon09:04:441
236732281,6sleep30-21swapper/304:33:253
181452281,6sleep30-21swapper/305:17:023
1792299282,23cyclictest0-21swapper/307:00:293
17922992812,13cyclictest21161-21sshd08:25:183
1791499287,18cyclictest0-21swapper/205:02:432
1791499285,19cyclictest0-21swapper/205:40:142
1791499285,19cyclictest0-21swapper/205:08:192
1791499285,18cyclictest0-21swapper/205:44:312
1791499285,18cyclictest0-21swapper/205:44:312
1791499284,21cyclictest0-21swapper/204:37:542
1791499282,23cyclictest0-21swapper/206:16:242
1791499282,22cyclictest0-21swapper/204:54:212
1790499282,24cyclictest0-21swapper/109:52:071
1790499282,23cyclictest0-21swapper/105:06:521
1790499282,22cyclictest0-21swapper/106:41:391
1790499282,16cyclictest0-21swapper/109:24:451
1790499282,16cyclictest0-21swapper/109:24:451
1790499281,3cyclictest0-21swapper/106:24:431
1790499281,24cyclictest0-21swapper/107:35:171
1790499281,24cyclictest0-21swapper/106:11:381
1790499281,24cyclictest0-21swapper/105:37:401
1790499281,24cyclictest0-21swapper/105:37:401
1790499281,24cyclictest0-21swapper/105:16:211
1790499281,24cyclictest0-21swapper/104:43:511
1790499281,24cyclictest0-21swapper/104:36:091
1790499281,24cyclictest0-21swapper/104:25:441
1789099289,4cyclictest24307-21kworker/0:308:26:390
1789099288,9cyclictest31032-21kworker/0:408:43:240
1789099288,9cyclictest31032-21kworker/0:408:43:240
1789099284,22cyclictest0-21swapper/006:52:380
1789099281,24cyclictest0-21swapper/008:54:140
242799270,2rtkit-daemon2426-21rtkit-daemon09:32:351
242799270,2rtkit-daemon2426-21rtkit-daemon09:32:351
242799270,2rtkit-daemon2426-21rtkit-daemon08:37:501
242799270,2rtkit-daemon2426-21rtkit-daemon07:40:482
242799270,2rtkit-daemon0-21swapper/109:44:023
1792299277,17cyclictest31427-21kworker/3:008:22:373
1792299277,17cyclictest25780-21kworker/3:107:43:473
1791499272,21cyclictest0-21swapper/204:46:532
1790499272,22cyclictest0-21swapper/105:49:391
1790499271,24cyclictest0-21swapper/107:16:421
1790499271,23cyclictest0-21swapper/107:13:191
1790499271,23cyclictest0-21swapper/104:50:511
1790499271,23cyclictest0-21swapper/104:46:031
91202261,6sleep10-21swapper/108:39:061
91202261,6sleep10-21swapper/108:39:061
70922261,4sleep30-21swapper/306:48:283
60602261,6sleep30-21swapper/304:55:303
1792299269,15cyclictest0-21swapper/308:07:153
1792299267,16cyclictest0-21swapper/306:24:303
17922992610,12cyclictest19746-21kworker/3:009:32:443
17922992610,12cyclictest19746-21kworker/3:009:32:443
1791499266,13cyclictest0-21swapper/204:49:272
1791499262,20cyclictest0-21swapper/206:39:042
1790499266,9cyclictest0-21swapper/108:31:351
1790499262,22cyclictest0-21swapper/108:49:461
1790499262,20cyclictest0-21swapper/108:08:561
1790499262,20cyclictest0-21swapper/107:19:501
1790499262,14cyclictest0-21swapper/107:29:341
17904992619,4cyclictest23610-21sshd08:23:441
17904992612,11cyclictest0-21swapper/107:44:131
1790499261,22cyclictest0-21swapper/109:42:221
1790499261,22cyclictest0-21swapper/108:03:421
1790499261,22cyclictest0-21swapper/107:57:351
1790499261,21cyclictest0-21swapper/108:04:591
1789099261,4cyclictest19601-21modprobe09:19:340
1789099261,22cyclictest6850irq/24-0000:00:08:35:490
1789099261,22cyclictest11-21rcuc/007:55:040
294202251,6sleep30-21swapper/307:33:223
242799250,2rtkit-daemon2426-21rtkit-daemon09:09:471
242799250,2rtkit-daemon2426-21rtkit-daemon06:34:081
1792299259,5cyclictest1359-21sshd08:11:333
1792299258,15cyclictest0-21swapper/308:03:423
1792299257,15cyclictest0-21swapper/307:36:023
1792299252,20cyclictest0-21swapper/307:53:533
17922992516,5cyclictest20127-21sshd06:29:093
17922992512,10cyclictest0-21swapper/307:52:043
17922992512,10cyclictest0-21swapper/307:52:043
1792299251,13cyclictest21578-21kworker/3:209:39:423
1791499255,15cyclictest0-21swapper/204:39:472
1790499251,22cyclictest0-21swapper/108:45:001
113102251,5sleep30-21swapper/309:24:343
113102251,5sleep30-21swapper/309:24:343
301302241,6sleep30-21swapper/305:37:423
301302241,6sleep30-21swapper/305:37:423
1792299249,4cyclictest31427-21kworker/3:008:14:593
1792299249,12cyclictest0-21swapper/307:21:113
1792299248,13cyclictest29585-21sshd06:53:183
1792299248,13cyclictest0-21swapper/307:07:533
1792299248,12cyclictest28668-21kworker/3:208:48:513
1792299246,15cyclictest0-21swapper/307:45:103
1792299246,15cyclictest0-21swapper/306:19:033
1792299246,15cyclictest0-21swapper/305:59:243
1792299242,18cyclictest0-21swapper/304:59:423
1790499242,19cyclictest0-21swapper/107:49:351
1790499242,19cyclictest0-21swapper/107:49:351
1790499242,18cyclictest0-21swapper/108:28:371
1790499242,18cyclictest0-21swapper/107:00:121
1792299239,4cyclictest6720-21sshd08:55:163
1792299238,4cyclictest22501-21kworker/3:207:14:003
1792299237,13cyclictest28152-21kworker/3:108:43:573
1792299236,14cyclictest0-21swapper/307:25:243
1792299232,18cyclictest0-21swapper/306:42:363
1792299232,17cyclictest0-21swapper/306:09:183
17922992314,6cyclictest0-21swapper/309:37:093
17922992314,5cyclictest12837-21sshd06:55:563
1792299231,18cyclictest0-21swapper/305:51:593
1792299231,13cyclictest0-21swapper/309:19:143
1790499232,17cyclictest0-21swapper/106:50:081
1792299228,11cyclictest25711-21ntp_kernel_err04:34:233
1792299222,17cyclictest0-21swapper/305:45:573
1792299222,17cyclictest0-21swapper/305:45:573
1792299222,17cyclictest0-21swapper/305:13:373
17922992210,9cyclictest0-21swapper/307:12:003
1792299221,17cyclictest0-21swapper/304:44:023
1792299221,17cyclictest0-21swapper/304:24:223
1790499222,17cyclictest0-21swapper/107:24:071
1792299219,9cyclictest0-21swapper/305:04:103
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional