You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-10-22 - 02:05

Intel(R) Xeon(R) CPU E3-1220L V2 @ 2.30GHz, Linux 4.18.7-rt5 (Profile)

Latency plot of system in rack #9, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack9slot0.osadl.org (updated Thu Oct 21, 2021 12:43:25)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
67272770,0sleep11527-21nfsd12:35:201
150432770,0sleep30-21swapper/310:50:523
179092700,1sleep217906-21sshd11:59:422
85382690,0sleep10-21swapper/109:48:581
283272690,0sleep20-21swapper/210:19:572
20192690,1sleep12022-21kworker/u8:309:30:221
9692680,0sleep20-21swapper/209:12:072
312792680,0sleep20-21swapper/209:25:382
177132680,0sleep10-21swapper/107:28:061
139902670,1sleep213989-21systemd10:53:082
191292660,1sleep319127-21sshd09:45:113
126922650,0sleep00-21swapper/011:02:000
12612650,0sleep00-21swapper/011:51:500
12612650,0sleep00-21swapper/011:51:500
300022640,0sleep20-21swapper/212:34:412
108522640,0sleep3421rcuc/312:08:243
97192630,0sleep10-21swapper/111:56:521
302112600,0sleep30-21swapper/309:52:533
282772600,0sleep30-21swapper/311:21:193
207922590,0sleep020791-21seq11:34:380
230142580,0sleep00-21swapper/009:20:350
10702580,0sleep20-21swapper/211:49:352
10702580,0sleep20-21swapper/211:49:352
275192570,2sleep027518-21bash12:11:500
198462570,1sleep319848-21uname10:19:223
482925529,8sleep10-21swapper/107:05:081
472925338,9sleep00-21swapper/007:03:500
38032530,2sleep1530799cyclictest09:14:351
488525225,9sleep30-21swapper/307:05:503
474124430,8sleep20-21swapper/207:04:002
229112270,7sleep3840-21avahi-daemon10:15:093
5306992624,1cyclictest9-21ksoftirqd/009:58:170
67292240,1sleep01526-21nfsd09:37:420
113742240,0sleep00-21swapper/009:56:100
5308992315,1cyclictest23302-21systemd-cgroups09:47:462
530899229,1cyclictest35-21ksoftirqd/210:05:002
5308992212,1cyclictest35-21ksoftirqd/212:22:392
530899221,1cyclictest35-21ksoftirqd/211:13:012
530899217,1cyclictest35-21ksoftirqd/211:12:412
530899217,1cyclictest35-21ksoftirqd/211:12:412
5308992118,1cyclictest35-21ksoftirqd/209:57:282
5308992112,2cyclictest35-21ksoftirqd/212:07:482
5308992111,1cyclictest35-21ksoftirqd/212:23:022
72582200,0sleep20-21swapper/211:26:512
530899209,1cyclictest35-21ksoftirqd/209:37:462
530899208,3cyclictest35-21ksoftirqd/209:19:382
530899207,2cyclictest35-21ksoftirqd/211:01:452
530899205,3cyclictest35-21ksoftirqd/209:41:472
5308992018,1cyclictest35-21ksoftirqd/211:56:192
5308992018,1cyclictest35-21ksoftirqd/210:16:532
224132200,0sleep11-21systemd10:15:071
5309991916,2cyclictest23051-21bash09:11:223
530899199,1cyclictest35-21ksoftirqd/211:32:462
530899199,1cyclictest35-21ksoftirqd/210:40:362
530899198,1cyclictest35-21ksoftirqd/210:24:042
530899196,2cyclictest111rcu_preempt11:34:082
5308991916,2cyclictest35-21ksoftirqd/209:59:072
5306991915,2cyclictest9-21ksoftirqd/011:43:100
5306991915,2cyclictest9-21ksoftirqd/011:43:100
43082190,1sleep0141rcuc/009:46:240
286012190,0sleep00-21swapper/009:14:020
179222190,0sleep10-21swapper/112:04:181
530899188,2cyclictest35-21ksoftirqd/210:30:242
530899188,1cyclictest35-21ksoftirqd/210:42:492
530899188,1cyclictest35-21ksoftirqd/210:09:462
530899188,1cyclictest35-21ksoftirqd/209:13:042
530899187,1cyclictest35-21ksoftirqd/212:05:362
530899187,1cyclictest35-21ksoftirqd/209:52:352
530899186,1cyclictest35-21ksoftirqd/210:36:562
530899184,4cyclictest35-21ksoftirqd/211:30:382
5308991814,1cyclictest35-21ksoftirqd/212:14:122
530799180,3cyclictest32728-21kill10:34:051
5306991813,2cyclictest9-21ksoftirqd/010:19:420
183262180,1sleep018315-21sshd12:17:550
530899178,6cyclictest35-21ksoftirqd/212:28:362
530899178,2cyclictest35-21ksoftirqd/211:43:582
530899178,2cyclictest35-21ksoftirqd/211:43:582
530899178,1cyclictest35-21ksoftirqd/210:48:052
530899176,1cyclictest35-21ksoftirqd/209:27:572
5308991712,1cyclictest35-21ksoftirqd/211:02:592
5307991714,1cyclictest26-21ksoftirqd/112:10:071
5306991715,1cyclictest9-21ksoftirqd/010:16:240
5306991714,1cyclictest9-21ksoftirqd/011:13:110
240082170,1sleep11527-21nfsd11:21:011
234862170,0sleep10-21swapper/111:23:131
530899165,1cyclictest35-21ksoftirqd/211:38:012
530899165,1cyclictest35-21ksoftirqd/211:18:022
5306991614,1cyclictest9-21ksoftirqd/010:53:050
5306991611,1cyclictest9-21ksoftirqd/010:40:550
306362160,1sleep31527-21nfsd11:46:563
306362160,1sleep31527-21nfsd11:46:563
254172160,0sleep10-21swapper/109:18:271
5309991512,2cyclictest1-21systemd10:45:373
530999150,14cyclictest31638-21taskset09:27:553
5307991513,1cyclictest0-21swapper/110:25:021
5307991511,1cyclictest26-21ksoftirqd/110:30:541
530799150,14cyclictest12604-21systemd-cgroups09:44:431
530799150,13cyclictest21632-21systemd-cgroups09:56:551
5306991513,1cyclictest9-21ksoftirqd/011:25:050
530699151,1cyclictest121rcu_sched10:27:290
530699151,1cyclictest111rcu_preempt11:10:360
530699151,1cyclictest111rcu_preempt11:10:360
530699150,1cyclictest5898-21systemd-cgroups10:45:400
50402150,0sleep14981-21sshd10:00:121
530999140,1cyclictest0-21swapper/309:24:463
530799148,2cyclictest26-21ksoftirqd/111:47:261
530799148,2cyclictest26-21ksoftirqd/111:47:261
5307991412,1cyclictest26-21ksoftirqd/110:51:521
5307991411,1cyclictest26-21ksoftirqd/110:43:231
5307991411,1cyclictest26-21ksoftirqd/110:40:441
5307991410,1cyclictest19615-21diskstats12:17:591
530799140,2cyclictest121rcu_sched12:23:091
530799140,1cyclictest1-21systemd11:29:371
530799140,1cyclictest1-21systemd11:29:371
530799140,1cyclictest0-21swapper/110:09:281
530699142,2cyclictest111rcu_preempt11:04:550
530699142,1cyclictest9-21ksoftirqd/010:08:420
5306991412,1cyclictest9-21ksoftirqd/012:24:160
5306991412,1cyclictest9-21ksoftirqd/009:28:030
530699141,1cyclictest121rcu_sched10:32:270
530699140,1cyclictest3227-21systemd-cgroups09:07:460
242082140,0sleep30-21swapper/311:37:103
105882140,2sleep310545-21sshd11:01:513
530999136,3cyclictest819-21systemd-logind11:03:303
530999135,7cyclictest890-21gdbus11:27:323
5309991311,1cyclictest0-21swapper/312:19:003
5309991311,1cyclictest0-21swapper/309:33:133
5309991310,2cyclictest876-21gdbus10:55:583
5309991310,2cyclictest32096-21systemd12:28:073
5309991310,2cyclictest31822-21kworker/3:711:51:493
5309991310,2cyclictest31822-21kworker/3:711:51:493
5309991310,1cyclictest0-21swapper/307:18:033
530999130,1cyclictest13444-21ystemctl12:01:423
5307991311,1cyclictest26-21ksoftirqd/112:27:491
5307991311,1cyclictest26-21ksoftirqd/111:38:101
5307991311,1cyclictest26-21ksoftirqd/109:10:311
5307991311,1cyclictest0-21swapper/112:15:021
5307991310,0cyclictest26-21ksoftirqd/111:06:521
530699139,1cyclictest9-21ksoftirqd/012:06:360
530699138,1cyclictest9-21ksoftirqd/011:58:260
5306991311,1cyclictest9-21ksoftirqd/012:28:020
5306991311,1cyclictest9-21ksoftirqd/010:05:090
5306991311,1cyclictest9-21ksoftirqd/009:42:040
5306991311,1cyclictest9-21ksoftirqd/009:26:030
5306991310,1cyclictest9-21ksoftirqd/011:31:480
5306991310,1cyclictest9-21ksoftirqd/011:31:480
530699130,1cyclictest32563-21sshd12:34:520
530999123,4cyclictest888-21polkitd12:37:273
530999122,6cyclictest0-21swapper/310:31:083
5309991210,1cyclictest6130-21sshd09:42:053
530999120,8cyclictest10649-21systemd10:34:513
530999120,1cyclictest0-21swapper/309:49:283
530799128,1cyclictest26-21ksoftirqd/110:59:591
530799122,5cyclictest12833-21sshd11:50:241
530799122,5cyclictest12833-21sshd11:50:241
530799122,1cyclictest111rcu_preempt09:33:001
5307991210,1cyclictest26-21ksoftirqd/110:07:371
530799120,11cyclictest27337-21sshd11:12:181
530799120,11cyclictest27337-21sshd11:12:181
530799120,11cyclictest0-21swapper/109:26:081
530799120,0cyclictest0-21swapper/111:33:201
530699129,1cyclictest9-21ksoftirqd/011:56:050
530699129,1cyclictest9-21ksoftirqd/010:37:060
530699129,1cyclictest9-21ksoftirqd/009:08:000
5306991210,1cyclictest9-21ksoftirqd/012:17:290
5306991210,1cyclictest9-21ksoftirqd/011:38:030
5306991210,1cyclictest9-21ksoftirqd/009:48:210
530699120,1cyclictest1-21systemd11:18:190
530999119,1cyclictest44-21ksoftirqd/310:00:463
530999119,1cyclictest44-21ksoftirqd/309:15:233
530999119,1cyclictest1527-21nfsd08:58:313
530999116,1cyclictest0-21swapper/311:28:383
530999116,1cyclictest0-21swapper/311:28:383
530999115,3cyclictest32499-21sshd12:25:543
530999114,3cyclictest819-21systemd-logind12:04:093
530999113,3cyclictest842-21dbus-daemon10:38:123
530999111,1cyclictest121rcu_sched10:25:463
530999110,8cyclictest819-21systemd-logind11:54:073
530999110,10cyclictest16037-21ntp_states12:13:113
530999110,10cyclictest0-21swapper/310:09:363
530999110,10cyclictest0-21swapper/308:50:023
530999110,10cyclictest0-21swapper/308:20:043
530899115,0cyclictest0-21swapper/208:16:012
530899113,7cyclictest0-21swapper/208:25:492
530899110,1cyclictest111rcu_preempt09:07:462
530799119,1cyclictest26-21ksoftirqd/111:58:091
530799119,1cyclictest26-21ksoftirqd/111:13:161
530799118,1cyclictest26-21ksoftirqd/110:19:211
530799116,1cyclictest26-21ksoftirqd/109:40:401
530799110,10cyclictest0-21swapper/109:04:041
530799110,10cyclictest0-21swapper/107:43:341
530699119,1cyclictest9-21ksoftirqd/010:47:540
530999109,0cyclictest0-21swapper/308:34:213
530999109,0cyclictest0-21swapper/308:27:023
530999108,1cyclictest44-21ksoftirqd/311:07:533
530999108,1cyclictest44-21ksoftirqd/311:07:533
530999108,1cyclictest19149-21systemctl09:18:023
530999106,2cyclictest44-21ksoftirqd/311:13:013
530999100,9cyclictest0-21swapper/311:38:123
530999100,9cyclictest0-21swapper/307:35:143
530999100,9cyclictest0-21swapper/307:15:573
530999100,0cyclictest0-21swapper/307:55:273
530899109,0cyclictest0-21swapper/207:28:172
530899104,3cyclictest35-21ksoftirqd/208:28:032
530799109,0cyclictest0-21swapper/108:46:321
530799109,0cyclictest0-21swapper/108:38:381
530799109,0cyclictest0-21swapper/108:10:171
530799105,4cyclictest888-21polkitd10:53:201
530799100,9cyclictest0-21swapper/108:04:011
530799100,0cyclictest0-21swapper/108:17:011
530699109,0cyclictest0-21swapper/008:57:320
530699109,0cyclictest0-21swapper/008:33:230
530699109,0cyclictest0-21swapper/008:12:350
530699109,0cyclictest0-21swapper/008:00:020
530699100,9cyclictest0-21swapper/008:19:120
530699100,9cyclictest0-21swapper/007:52:020
530699100,0cyclictest0-21swapper/008:40:560
530699100,0cyclictest0-21swapper/008:26:110
530699100,0cyclictest0-21swapper/008:06:120
53099998,0cyclictest0-21swapper/308:41:543
53099998,0cyclictest0-21swapper/308:13:273
53099992,6cyclictest13110-21sshd10:03:013
53099990,6cyclictest0-21swapper/308:42:593
53099990,0cyclictest0-21swapper/308:07:183
53089992,3cyclictest0-21swapper/207:41:272
53089990,2cyclictest121rcu_sched07:12:472
53089990,0cyclictest0-21swapper/208:08:562
53079998,0cyclictest0-21swapper/108:31:191
53079998,0cyclictest0-21swapper/108:23:461
53079998,0cyclictest0-21swapper/107:57:581
53079998,0cyclictest0-21swapper/107:26:091
53079991,5cyclictest0-21swapper/108:18:051
53069998,0cyclictest0-21swapper/008:48:570
53069998,0cyclictest0-21swapper/007:32:240
53099987,0cyclictest0-21swapper/308:01:083
53089987,0cyclictest0-21swapper/208:02:052
53089985,3cyclictest0-21swapper/207:26:052
53089982,5cyclictest0-21swapper/207:11:182
53089981,3cyclictest0-21swapper/208:53:232
53089981,3cyclictest0-21swapper/208:37:272
53089980,4cyclictest0-21swapper/208:42:072
53089980,1cyclictest121rcu_sched07:53:102
53089980,1cyclictest0-21swapper/207:50:362
53079987,0cyclictest0-21swapper/108:54:461
53079987,0cyclictest0-21swapper/107:18:081
53079981,2cyclictest0-21swapper/108:48:111
53099970,1cyclictest4761-21/usr/sbin/munin08:08:093
53099970,1cyclictest0-21swapper/307:48:063
53089970,5cyclictest0-21swapper/207:18:062
53089970,3cyclictest0-21swapper/208:18:042
53089970,1cyclictest0-21swapper/208:58:032
53089970,1cyclictest0-21swapper/208:03:052
53089970,1cyclictest0-21swapper/207:43:142
53089970,0cyclictest0-21swapper/208:48:192
53089970,0cyclictest0-21swapper/208:42:592
53089970,0cyclictest0-21swapper/207:33:012
53079971,2cyclictest0-21swapper/108:33:091
53079970,1cyclictest9066-21cstates07:12:591
53079970,1cyclictest32644-21runrttasks08:59:071
53079970,1cyclictest30406-21kernelversion07:53:061
53079970,1cyclictest28621-21ls07:48:121
53079970,1cyclictest22194-21cut07:38:041
53069970,1cyclictest9846-21memory07:13:050
53069970,1cyclictest30261-21hddtemp_smartct07:53:040
53069970,1cyclictest22417-21fw_forwarded_lo07:38:050
53069970,1cyclictest1992-21runrttasks07:43:100
53069970,1cyclictest1527-21nfsd08:58:110
53069970,1cyclictest1527-21nfsd07:23:340
53099964,1cyclictest7674-21turbostat.cron07:12:453
53099963,3cyclictest0-21swapper/307:42:593
53099963,2cyclictest28332-21expr08:53:003
53099962,2cyclictest0-21swapper/309:03:093
53099962,2cyclictest0-21swapper/307:28:043
53099962,2cyclictest0-21swapper/307:23:093
53079962,2cyclictest0-21swapper/107:08:091
53079960,0cyclictest0-21swapper/107:33:121
53069964,1cyclictest28777-21kworker/0:108:12:590
53069964,1cyclictest21945-21kworker/0:207:07:570
53069964,1cyclictest19946-21sed07:33:040
53069964,1cyclictest12616-21awk07:18:080
53069963,2cyclictest15385-21cstates08:27:590
53069960,0cyclictest0-21swapper/008:43:020
53099953,1cyclictest15531-21sed08:28:003
53099953,1cyclictest0-21swapper/307:38:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional