You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2022-01-23 - 20:25

Intel(R) Xeon(R) CPU E3-1220L V2 @ 2.30GHz, Linux 4.18.7-rt5 (Profile)

Latency plot of system in rack #8, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of all highest latencies:
System rack8slot0.osadl.org (updated Sun Jan 23, 2022 12:43:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3261021500,1sleep232601-21grepconf.sh10:00:402
1536421310,1sleep115358-21systemd-cgroups11:30:061
899921260,1sleep18928-21bash11:03:351
1463921260,2sleep014634-21grep12:07:340
2687921220,2sleep33181899cyclictest10:36:383
3074421210,0sleep130745-21sshd11:53:151
650521200,1sleep36490-21sshd09:17:153
828421190,0sleep1241rcuc/109:48:321
2860921120,0sleep1241rcuc/109:55:261
3240021090,3sleep23181799cyclictest11:43:182
3062921070,1sleep1251ktimersoftd/112:10:361
1504321060,2sleep03181599cyclictest12:01:200
14602860,0sleep20-21swapper/211:44:342
56092840,4sleep05525-21sshd11:44:440
277562830,1sleep127684-21sshd12:15:291
249582820,1sleep124964-21grepconf.sh09:21:381
231862820,2sleep023184-21bash11:29:120
214392820,1sleep221437-21grep12:22:522
34452810,1sleep3606-21systemd-journal12:30:553
325062800,2sleep2857-21dbus-daemon11:37:042
65292790,1sleep16526-21sshd11:37:191
266252790,1sleep0857-21dbus-daemon09:12:470
250072770,2sleep324915-21sshd11:57:583
159512770,1sleep115944-21bash11:15:061
322832760,1sleep032284-21bash10:44:210
226982760,0sleep3857-21dbus-daemon09:19:073
118492760,1sleep111846-21sshd10:07:211
66082750,1sleep10-21swapper/109:09:191
76892740,3sleep31-21systemd11:18:373
73252740,0sleep0141rcuc/010:54:310
301602740,1sleep130158-21sshd11:40:401
143872740,1sleep314372-21grepconf.sh12:26:173
122352740,1sleep119138-21httpd11:48:501
110412740,1sleep2331rcuc/211:11:062
54992730,1sleep15500-21systemd-cgroups10:43:181
287842730,1sleep328770-21cpuspeed_turbos09:49:183
185672730,0sleep018565-21seq09:17:440
300552720,2sleep03181599cyclictest12:26:550
103192720,1sleep210320-21bash11:04:522
103192720,1sleep210320-21bash11:04:522
89422710,1sleep38941-21systemd-cgroups10:27:083
61482710,1sleep16147-21systemd-cgroups09:37:101
145432710,0sleep0101ktimersoftd/012:21:210
13862710,1sleep1857-21dbus-daemon10:55:331
71872700,1sleep07186-21systemd-cgroups09:44:380
42212700,3sleep23181799cyclictest10:49:322
280602700,0sleep10-21swapper/111:59:211
94192690,1sleep31-21systemd12:19:523
62642680,1sleep01844-21nfsd12:12:110
57262670,0sleep3431ktimersoftd/309:48:263
32542670,0sleep10-21swapper/111:28:251
136382670,1sleep213644-21bash09:24:552
30462660,1sleep32982-21sshd10:03:223
92912650,0sleep00-21swapper/010:52:110
304512650,1sleep1241rcuc/112:33:131
264172640,0sleep30-21swapper/311:30:323
111772630,0sleep00-21swapper/011:52:300
67102620,0sleep1241rcuc/109:50:531
100122620,1sleep29974-21bash10:29:392
64222610,1sleep26420-21bash09:42:082
49902610,0sleep34991-21sshd11:25:543
323912610,1sleep132388-21sshd12:27:001
260212610,1sleep026017-21bash09:41:380
96142600,0sleep00-21swapper/010:20:590
70552600,0sleep30-21swapper/309:40:523
312082600,0sleep30-21swapper/312:37:043
28992600,2sleep03181599cyclictest12:37:130
138392590,0sleep20-21swapper/212:05:032
157342580,1sleep015631-21sshd12:17:360
155482560,0sleep00-21swapper/012:33:560
3127525426,7sleep20-21swapper/207:05:382
3127525426,7sleep20-21swapper/207:05:382
3125125326,9sleep30-21swapper/307:05:193
3125125326,9sleep30-21swapper/307:05:193
3151724329,8sleep00-21swapper/007:08:330
3151724329,8sleep00-21swapper/007:08:330
3122524228,8sleep10-21swapper/107:05:011
3122524228,8sleep10-21swapper/107:05:011
107952320,0sleep10-21swapper/110:22:161
260712310,2sleep226026-21bash12:00:302
3181899290,1cyclictest1846-21nfsd11:44:523
97602280,0sleep2331rcuc/209:10:442
31815992724,1cyclictest9-21ksoftirqd/009:35:530
31817992623,1cyclictest35-21ksoftirqd/211:51:522
31816992622,2cyclictest26-21ksoftirqd/111:05:431
31816992622,2cyclictest26-21ksoftirqd/111:05:431
31815992522,1cyclictest9-21ksoftirqd/011:58:480
263532250,1sleep126355-21sshd09:40:221
31818992422,1cyclictest44-21ksoftirqd/311:37:373
3181899241,1cyclictest121rcu_sched12:17:403
313502240,1sleep331346-21bash09:54:203
274302240,0sleep20-21swapper/211:55:332
223482240,0sleep20-21swapper/209:59:002
127792240,3sleep312768-21id12:01:153
97322230,1sleep2857-21dbus-daemon11:26:052
67362230,1sleep1251ktimersoftd/109:32:161
320912230,2sleep132074-21bash11:11:571
31817992319,3cyclictest35-21ksoftirqd/210:27:572
31816992320,1cyclictest26-21ksoftirqd/112:07:501
3181699230,1cyclictest121rcu_sched11:55:031
304712230,0sleep20-21swapper/209:16:542
290352230,1sleep329034-21sshd10:46:453
31817992220,1cyclictest35-21ksoftirqd/210:12:112
31817992220,1cyclictest35-21ksoftirqd/210:08:582
31817992217,1cyclictest35-21ksoftirqd/209:37:562
31816992216,2cyclictest26-21ksoftirqd/110:48:331
304052220,1sleep030399-21sshd10:43:010
300222220,2sleep030001-21sshd10:00:340
227242220,3sleep33181899cyclictest10:31:293
173762220,2sleep317377-21grep11:07:383
173762220,2sleep317377-21grep11:07:383
126482220,1sleep012637-21sshd10:31:060
57762210,0sleep35671-21sshd09:24:363
31818992116,3cyclictest44-21ksoftirqd/311:12:053
3181799217,2cyclictest35-21ksoftirqd/212:33:112
31817992119,1cyclictest35-21ksoftirqd/210:54:412
31817992119,1cyclictest35-21ksoftirqd/210:47:322
31817992119,1cyclictest35-21ksoftirqd/209:51:442
31817992119,1cyclictest35-21ksoftirqd/209:46:142
31817992118,1cyclictest35-21ksoftirqd/212:34:322
31817992118,1cyclictest35-21ksoftirqd/210:35:532
31817992117,2cyclictest35-21ksoftirqd/211:21:182
3181699212,1cyclictest121rcu_sched09:14:441
31816992119,1cyclictest26-21ksoftirqd/110:27:211
31816992119,1cyclictest26-21ksoftirqd/109:59:241
31815992118,1cyclictest9-21ksoftirqd/011:00:570
31815992117,1cyclictest9-21ksoftirqd/011:11:390
31815992116,1cyclictest9-21ksoftirqd/010:38:240
216332210,1sleep121639-21bash11:19:111
175362210,0sleep1241rcuc/112:22:431
156252210,2sleep11-21systemd10:51:121
70292200,2sleep37025-21systemd10:08:233
31818992016,1cyclictest44-21ksoftirqd/310:41:353
31818992014,3cyclictest44-21ksoftirqd/311:51:203
31817992018,1cyclictest35-21ksoftirqd/211:00:022
31817992017,2cyclictest35-21ksoftirqd/211:14:152
31817992017,1cyclictest35-21ksoftirqd/210:40:572
31817992016,1cyclictest35-21ksoftirqd/212:18:462
3181699208,1cyclictest121rcu_sched12:36:431
31816992017,1cyclictest26-21ksoftirqd/110:37:591
31816992017,1cyclictest26-21ksoftirqd/110:31:121
31816992017,1cyclictest26-21ksoftirqd/110:09:321
31816992014,1cyclictest26-21ksoftirqd/109:24:411
31815992018,1cyclictest9-21ksoftirqd/011:34:260
31815992018,1cyclictest9-21ksoftirqd/011:24:350
238002200,0sleep00-21swapper/009:30:230
12132200,0sleep3421rcuc/312:08:173
37662190,0sleep34265-21bash09:13:113
31818991917,1cyclictest44-21ksoftirqd/311:21:463
31818991917,1cyclictest44-21ksoftirqd/309:35:513
31818991916,2cyclictest44-21ksoftirqd/311:43:403
31818991915,1cyclictest44-21ksoftirqd/310:14:113
3181799196,1cyclictest35-21ksoftirqd/211:30:152
31817991917,1cyclictest35-21ksoftirqd/210:19:092
31817991917,1cyclictest35-21ksoftirqd/209:31:192
31817991916,2cyclictest35-21ksoftirqd/209:19:422
3181699199,9cyclictest26-21ksoftirqd/110:15:211
3181599198,1cyclictest9-21ksoftirqd/010:26:310
31815991917,1cyclictest9-21ksoftirqd/010:07:510
31815991917,1cyclictest9-21ksoftirqd/009:57:410
31815991916,1cyclictest9-21ksoftirqd/010:15:010
31815991915,3cyclictest31601-21kworker/u8:211:39:510
31815991915,2cyclictest9-21ksoftirqd/009:50:500
31818991816,1cyclictest44-21ksoftirqd/310:19:433
31817991816,1cyclictest35-21ksoftirqd/212:24:332
31817991816,1cyclictest35-21ksoftirqd/212:11:532
31817991816,1cyclictest35-21ksoftirqd/210:15:452
31815991816,1cyclictest9-21ksoftirqd/011:14:270
31815991816,1cyclictest9-21ksoftirqd/011:04:580
31815991816,1cyclictest9-21ksoftirqd/011:04:580
31815991816,1cyclictest9-21ksoftirqd/010:09:300
31815991816,1cyclictest9-21ksoftirqd/009:19:400
31815991815,1cyclictest9-21ksoftirqd/011:19:380
31818991715,1cyclictest44-21ksoftirqd/312:13:283
31818991715,1cyclictest44-21ksoftirqd/310:55:413
31818991715,1cyclictest44-21ksoftirqd/310:51:563
31818991714,1cyclictest44-21ksoftirqd/311:02:203
3181899170,16cyclictest5154-21kworker/3:510:09:253
31815991715,1cyclictest9-21ksoftirqd/009:26:480
31818991614,1cyclictest44-21ksoftirqd/309:29:253
3181799120,1cyclictest0-21swapper/207:19:212
3181799118,2cyclictest18220-21perl08:44:242
3181799118,2cyclictest18220-21perl08:44:242
3181699110,1cyclictest0-21swapper/107:14:031
3181699110,10cyclictest0-21swapper/108:13:141
3181899108,1cyclictest11500-21fw_conntrack07:29:213
3181899108,1cyclictest0-21swapper/308:49:263
3181899100,1cyclictest29771-21perl08:04:213
3181899100,1cyclictest21948-21fschecks_time07:49:153
3181899100,0cyclictest0-21swapper/307:57:203
3181799109,0cyclictest0-21swapper/207:54:262
3181799108,1cyclictest0-21swapper/208:59:282
3181799108,1cyclictest0-21swapper/207:49:202
3181799108,1cyclictest0-21swapper/207:44:172
3181799108,1cyclictest0-21swapper/207:14:022
3181799100,9cyclictest32561-21perl07:09:202
3181799100,9cyclictest0-21swapper/208:49:222
3181799100,1cyclictest0-21swapper/207:39:142
3181699109,0cyclictest0-21swapper/107:53:361
3181699108,1cyclictest23487-21fw_conntrack08:54:211
3181699100,9cyclictest0-21swapper/108:59:281
3181699100,9cyclictest0-21swapper/108:47:371
3181699100,9cyclictest0-21swapper/108:47:371
3181699100,1cyclictest20892-21cat08:49:221
3181699100,1cyclictest16700-21perl07:39:141
3181699100,0cyclictest0-21swapper/108:03:011
3181599108,1cyclictest17351-21ntp_states07:39:200
3181599108,1cyclictest14079-21perl07:34:170
3181599100,9cyclictest26741-21ntp_states08:59:280
3181599100,9cyclictest0-21swapper/008:54:220
3181599100,9cyclictest0-21swapper/007:49:150
3181599100,1cyclictest27811-21ntp_states07:59:260
3181599100,1cyclictest0-21swapper/009:09:000
318189998,0cyclictest0-21swapper/307:48:103
318189997,1cyclictest0-21swapper/308:59:283
318189990,9cyclictest0-21swapper/308:44:243
318189990,9cyclictest0-21swapper/308:44:243
318189990,8cyclictest0-21swapper/308:40:473
318189990,8cyclictest0-21swapper/307:23:533
318189990,0cyclictest0-21swapper/308:19:223
318189990,0cyclictest0-21swapper/307:34:183
318189990,0cyclictest0-21swapper/307:14:143
318189990,0cyclictest0-21swapper/307:09:203
318179997,1cyclictest30450-21latency_hist09:09:002
318169998,0cyclictest0-21swapper/109:09:001
318169998,0cyclictest0-21swapper/107:34:191
318169998,0cyclictest0-21swapper/107:29:331
318169998,0cyclictest0-21swapper/107:19:201
318169990,0cyclictest0-21swapper/108:40:111
318169990,0cyclictest0-21swapper/107:55:071
318159998,0cyclictest0-21swapper/007:54:130
318159990,8cyclictest0-21swapper/008:15:560
318159990,8cyclictest0-21swapper/007:11:350
318159990,0cyclictest16514-21kworker/0:008:04:210
318189987,0cyclictest0-21swapper/308:54:583
318189987,0cyclictest0-21swapper/307:25:513
318179986,1cyclictest0-21swapper/207:59:262
318179982,0cyclictest12723-21kworker/2:008:20:182
318179982,0cyclictest11652-21kworker/2:208:36:192
318179982,0cyclictest11652-21kworker/2:208:29:582
318179980,2cyclictest0-21swapper/208:09:282
318169988,0cyclictest0-21swapper/107:44:161
318169987,0cyclictest0-21swapper/108:35:391
318169987,0cyclictest0-21swapper/108:31:221
318169987,0cyclictest0-21swapper/108:26:361
318169987,0cyclictest0-21swapper/108:19:381
318169987,0cyclictest0-21swapper/108:07:471
318159987,0cyclictest0-21swapper/008:44:240
318159987,0cyclictest0-21swapper/008:44:240
318159980,7cyclictest0-21swapper/008:38:570
318159980,7cyclictest0-21swapper/007:19:010
318189976,0cyclictest0-21swapper/308:29:243
318189975,1cyclictest7884-21expr08:24:223
318189975,1cyclictest3462-21swap08:14:313
318189975,1cyclictest26823-21dnf07:59:193
318189975,1cyclictest158982sleep307:39:093
318189970,1cyclictest28303-21df09:04:183
318179975,1cyclictest15673-21idleruntime-cro07:39:002
318179972,0cyclictest12723-21kworker/2:008:17:152
318179970,1cyclictest9732-21smartctl07:24:282
318179970,1cyclictest23288-21fschecks_count08:54:202
318179970,1cyclictest16306-21awk08:39:282
318179970,1cyclictest0-21swapper/208:24:282
318179970,1cyclictest0-21swapper/208:08:172
318169976,0cyclictest0-21swapper/107:28:241
318169970,1cyclictest2608-21fw_conntrack08:14:211
318169970,1cyclictest0-21swapper/107:09:251
318159975,1cyclictest9790-21cpu08:29:190
318159975,1cyclictest7028-21/usr/sbin/munin08:24:260
318159975,1cyclictest20386-21cstates08:49:190
318159970,1cyclictest6291-21perl07:19:200
318159970,1cyclictest571-21ntp_states08:09:280
318159970,1cyclictest18510-21/usr/sbin/munin07:44:150
318159970,1cyclictest15234-21df08:39:180
318159970,1cyclictest11723-21irqstats07:29:230
318159970,1cyclictest0-21swapper/007:24:260
318189964,1cyclictest435-21expr08:09:273
318189964,1cyclictest13620-21nfsd408:34:283
318179960,0cyclictest0-21swapper/207:30:062
318159964,1cyclictest6704-21sshd08:23:100
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional