You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-05-27 - 01:39

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.18.7-rt5 (Profile)

Latency plot of system in rack #9, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the alll highest latencies:
System rack9slot0.osadl.org (updated Fri May 20, 2022 00:43:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3132821230,0sleep10-21swapper/100:32:121
1232021160,2sleep03132399cyclictest23:21:250
281682880,0sleep20-21swapper/222:55:452
289452810,0sleep00-21swapper/019:00:090
262362740,1sleep00-21swapper/022:04:490
118072740,0sleep10-21swapper/100:26:111
268142720,0sleep10-21swapper/100:16:011
85192710,1sleep2331rcuc/223:16:382
46552710,0sleep14656-21bash23:43:061
281282710,0sleep30-21swapper/300:22:483
114202710,0sleep20-21swapper/221:14:452
3342700,1sleep2598-21systemd-journal22:14:012
183872700,0sleep20-21swapper/222:04:152
130542700,0sleep2331rcuc/221:32:362
65032690,0sleep10-21swapper/122:49:511
294672680,0sleep3838-21systemd-logind00:18:273
287752670,1sleep21717-21nfsd23:13:332
250252670,0sleep30-21swapper/322:09:093
84202660,1sleep38342-21sshd22:56:393
80872660,1sleep18086-21sshd22:12:191
164232660,1sleep126-21ksoftirqd/121:37:121
28012640,0sleep00-21swapper/022:05:210
208142640,1sleep1806-21dbus-daemon21:53:161
169502630,1sleep11718-21nfsd00:08:311
102652630,0sleep30-21swapper/322:54:343
180692620,1sleep018044-21rm23:32:550
261372610,0sleep30-21swapper/323:22:243
233092610,0sleep20-21swapper/222:53:132
209462610,0sleep20-21swapper/222:39:502
195532610,1sleep219552-21id22:24:192
137502610,0sleep30-21swapper/321:23:483
163572590,1sleep016353-21systemd22:15:070
212312580,1sleep321233-21uname00:08:493
209982580,0sleep10-21swapper/122:24:261
3962560,0sleep20-21swapper/221:07:232
3107225641,9sleep10-21swapper/119:04:321
246452560,0sleep30-21swapper/322:35:303
220602560,3sleep33132799cyclictest00:04:253
180612560,0sleep0141rcuc/022:52:510
76282550,0sleep00-21swapper/022:56:360
3090624834,8sleep30-21swapper/319:02:263
3090024621,8sleep20-21swapper/219:02:212
282462310,0sleep0141rcuc/022:29:140
53502250,1sleep00-21swapper/022:43:170
31325992311,1cyclictest35-21ksoftirqd/200:34:572
3132599228,2cyclictest35-21ksoftirqd/222:42:062
170042220,1sleep217002-21systemd-cgroups00:10:452
170042220,1sleep217002-21systemd-cgroups00:10:452
90602210,1sleep09062-21grep23:16:400
74892210,0sleep30-21swapper/323:56:473
52532210,1sleep25252-21bash22:05:312
3132599219,1cyclictest35-21ksoftirqd/223:00:522
31325992111,1cyclictest35-21ksoftirqd/221:40:212
31325992110,2cyclictest35-21ksoftirqd/223:06:122
31325992110,1cyclictest35-21ksoftirqd/221:49:502
261492210,1sleep2331rcuc/221:15:412
3132599209,3cyclictest35-21ksoftirqd/223:25:182
3132599209,1cyclictest35-21ksoftirqd/223:36:012
3132599209,1cyclictest35-21ksoftirqd/221:28:512
3132599209,1cyclictest35-21ksoftirqd/221:28:512
3132599208,1cyclictest35-21ksoftirqd/223:42:282
3132599208,1cyclictest35-21ksoftirqd/222:28:202
3132599204,3cyclictest35-21ksoftirqd/223:56:512
31325992016,3cyclictest35-21ksoftirqd/219:47:092
31325992013,1cyclictest35-21ksoftirqd/200:03:382
31325992010,1cyclictest35-21ksoftirqd/221:56:072
215362200,1sleep021482-21sshd23:57:440
75642190,0sleep00-21swapper/021:59:000
42502190,0sleep10-21swapper/100:03:101
3132599199,1cyclictest35-21ksoftirqd/222:48:532
3132599199,1cyclictest35-21ksoftirqd/221:36:592
3132599199,1cyclictest35-21ksoftirqd/200:06:102
3132599198,3cyclictest35-21ksoftirqd/222:19:062
3132599198,2cyclictest35-21ksoftirqd/223:50:482
3132599198,1cyclictest35-21ksoftirqd/200:20:252
3132599198,0cyclictest35-21ksoftirqd/200:16:312
31325991910,2cyclictest35-21ksoftirqd/223:30:402
25972190,0sleep30-21swapper/322:03:083
169822190,0sleep016977-21sshd23:08:180
77552180,0sleep30-21swapper/300:30:263
35632180,0sleep20-21swapper/223:45:122
35632180,0sleep20-21swapper/223:45:122
3132599188,1cyclictest35-21ksoftirqd/221:54:112
3132599187,1cyclictest35-21ksoftirqd/221:20:502
3132599186,1cyclictest35-21ksoftirqd/223:23:442
31325991816,1cyclictest35-21ksoftirqd/200:25:102
31325991814,3cyclictest30554-21kworker/2:020:01:112
31325991812,3cyclictest30554-21kworker/2:019:19:452
3132499180,16cyclictest4060-21kworker/1:022:07:521
253012180,0sleep10-21swapper/122:42:261
203552180,0sleep00-21swapper/021:13:100
168422180,2sleep13132499cyclictest23:23:581
133062180,0sleep3421rcuc/322:23:523
3132799178,4cyclictest806-21dbus-daemon21:30:053
3132799173,6cyclictest30533-21kworker/3:700:13:163
3132799173,6cyclictest30533-21kworker/3:700:13:163
3132799171,1cyclictest838-21systemd-logind23:29:083
31325991713,3cyclictest30554-21kworker/2:020:51:472
31325991711,3cyclictest35-21ksoftirqd/222:31:022
291382170,8sleep388150irq/26-p4p1-rx-21:49:253
123932170,0sleep30-21swapper/321:43:443
4512160,0sleep0141rcuc/023:29:300
319702160,0sleep10-21swapper/123:51:401
3132799160,8cyclictest27308-21systemctl00:27:183
31325991612,3cyclictest30554-21kworker/2:021:00:272
31325991612,3cyclictest30554-21kworker/2:019:54:052
31324991613,1cyclictest26-21ksoftirqd/122:30:141
31324991610,4cyclictest12203-21sshd23:07:591
3132399169,2cyclictest9-21ksoftirqd/023:50:130
160602160,0sleep10-21swapper/121:46:141
90412150,0sleep00-21swapper/000:28:180
3132799153,11cyclictest14773-21bash22:10:333
3132599159,3cyclictest30554-21kworker/2:020:32:332
31325991512,2cyclictest0-21swapper/219:32:032
31325991511,3cyclictest30554-21kworker/2:019:14:132
31325991511,3cyclictest0-21swapper/220:37:252
3132599150,14cyclictest30554-21kworker/2:019:59:272
31324991513,1cyclictest26-21ksoftirqd/123:11:041
31324991513,1cyclictest26-21ksoftirqd/122:29:121
31324991512,2cyclictest26-21ksoftirqd/122:57:491
31324991512,1cyclictest26-21ksoftirqd/121:05:111
31324991510,1cyclictest26-21ksoftirqd/121:43:151
3132799148,5cyclictest838-21systemd-logind23:33:363
3132799146,4cyclictest838-21systemd-logind21:08:113
3132799143,8cyclictest0-21swapper/322:16:053
3132799141,1cyclictest0-21swapper/323:07:103
31327991410,2cyclictest886-21gdbus21:52:283
3132799140,9cyclictest19208-21sshd23:46:163
3132799140,9cyclictest19208-21sshd23:46:163
3132799140,5cyclictest0-21swapper/319:32:033
3132799140,13cyclictest20154-21kworker/3:121:56:493
3132599149,4cyclictest35-21ksoftirqd/219:38:392
31325991411,2cyclictest0-21swapper/220:59:572
31325991410,3cyclictest30554-21kworker/2:020:20:352
31325991410,3cyclictest30554-21kworker/2:019:40:432
3132599140,13cyclictest30554-21kworker/2:020:13:292
3132499148,4cyclictest1718-21nfsd22:50:171
3132499142,1cyclictest121rcu_sched21:15:111
31324991411,1cyclictest26-21ksoftirqd/123:25:081
31324991411,1cyclictest26-21ksoftirqd/121:55:361
3132499140,13cyclictest14853-21sshd21:32:441
31323991412,1cyclictest0-21swapper/021:28:550
31323991412,1cyclictest0-21swapper/021:28:550
31323991411,2cyclictest23912-21uname22:46:380
3132399140,13cyclictest1-21systemd22:38:570
3132799134,8cyclictest19575-21grep21:26:253
3132799134,8cyclictest19575-21grep21:26:253
31327991311,1cyclictest838-21systemd-logind22:26:523
31327991311,1cyclictest0-21swapper/323:35:243
3132799130,7cyclictest18686-21uname23:10:343
3132799130,4cyclictest18917-21/usr/sbin/munin21:15:133
3132799130,12cyclictest0-21swapper/323:50:233
3132599139,3cyclictest30554-21kworker/2:020:41:232
3132599139,3cyclictest30554-21kworker/2:019:05:432
3132599139,3cyclictest0-21swapper/220:29:052
3132599130,10cyclictest30554-21kworker/2:019:23:232
3132499139,2cyclictest21461-21bash23:02:031
3132499132,5cyclictest884-21polkitd23:56:041
31324991311,1cyclictest26-21ksoftirqd/123:17:211
31324991310,2cyclictest17984-21sshd22:17:241
3132399139,2cyclictest9-21ksoftirqd/023:10:390
31323991311,1cyclictest9-21ksoftirqd/022:22:290
31323991311,1cyclictest9-21ksoftirqd/000:18:590
3132399130,12cyclictest7068-21systemd00:05:300
3132399130,12cyclictest0-21swapper/000:12:490
3132399130,12cyclictest0-21swapper/000:12:490
3132799124,8cyclictest0-21swapper/320:41:033
3132799123,8cyclictest21393-21grep23:44:143
3132799123,8cyclictest18813-21kworker/3:221:38:533
3132799120,9cyclictest0-21swapper/322:39:583
3132799120,1cyclictest11759-21sshd22:48:003
3132799120,11cyclictest598-21systemd-journal21:10:313
3132799120,11cyclictest0-21swapper/322:31:093
3132599128,3cyclictest30554-21kworker/2:020:06:232
31325991211,0cyclictest0-21swapper/220:15:232
3132599120,11cyclictest30554-21kworker/2:020:46:052
3132499129,1cyclictest26-21ksoftirqd/122:36:211
3132499122,3cyclictest0-21swapper/121:13:061
3132499121,1cyclictest121rcu_sched00:23:181
3132499121,1cyclictest121rcu_sched00:10:391
3132499121,1cyclictest121rcu_sched00:10:391
31324991210,1cyclictest26-21ksoftirqd/123:35:081
3132499120,2cyclictest598-21systemd-journal21:25:231
3132499120,2cyclictest598-21systemd-journal21:25:231
3132499120,2cyclictest26097-21sshd21:20:101
3132499120,1cyclictest0-21swapper/123:30:091
3132499120,11cyclictest28270-21ls22:00:201
3132399128,2cyclictest9-21ksoftirqd/023:49:010
3132399128,2cyclictest9-21ksoftirqd/023:49:010
3132399126,3cyclictest806-21dbus-daemon21:53:400
3132399121,10cyclictest24219-21systemd21:35:290
31323991211,0cyclictest0-21swapper/000:34:540
31323991210,1cyclictest0-21swapper/021:18:490
3132399120,1cyclictest30574-21sshd23:02:400
3132399120,1cyclictest19902-21sshd00:20:010
3132399120,1cyclictest0-21swapper/000:02:500
3132399120,11cyclictest24106-21sshd21:42:160
3132399120,11cyclictest11997-21sshd22:30:150
3132399120,10cyclictest1718-21nfsd21:30:190
218702120,0sleep3421rcuc/323:17:353
3132799115,3cyclictest0-21swapper/320:35:513
31327991110,0cyclictest0-21swapper/320:25:213
3132799110,7cyclictest0-21swapper/323:00:403
3132799110,7cyclictest0-21swapper/320:10:113
3132599117,3cyclictest30554-21kworker/2:019:28:152
3132499119,1cyclictest26-21ksoftirqd/123:45:121
3132499119,1cyclictest26-21ksoftirqd/123:45:121
3132399119,1cyclictest9-21ksoftirqd/023:35:180
3132399119,1cyclictest26845-21/usr/sbin/munin21:00:200
3132399118,2cyclictest12479-21systemd22:10:230
3132399110,1cyclictest121rcu_sched21:08:280
3132399110,0cyclictest0-21swapper/021:47:090
3132799109,0cyclictest0-21swapper/320:08:523
3132799100,9cyclictest0-21swapper/320:20:153
3132799100,6cyclictest0-21swapper/320:45:093
3132799100,0cyclictest0-21swapper/320:50:133
3132799100,0cyclictest0-21swapper/319:40:103
3132399109,0cyclictest0-21swapper/019:41:000
3132399109,0cyclictest0-21swapper/019:22:110
3132399108,1cyclictest28300-21nfsd423:40:170
3132399106,2cyclictest806-21dbus-daemon21:20:030
3132399100,9cyclictest0-21swapper/019:19:170
313279994,4cyclictest0-21swapper/319:10:173
313279990,9cyclictest0-21swapper/319:47:093
313279990,5cyclictest0-21swapper/320:04:393
313279990,0cyclictest0-21swapper/319:53:393
313249997,1cyclictest15310-21/usr/sbin/munin19:35:101
313239998,0cyclictest0-21swapper/020:05:240
313239998,0cyclictest0-21swapper/020:03:250
313239990,0cyclictest0-21swapper/020:49:530
313239990,0cyclictest0-21swapper/020:42:550
313239990,0cyclictest0-21swapper/019:38:060
313279982,2cyclictest0-21swapper/319:25:143
313279980,4cyclictest6226-21nfsd419:15:173
313279980,4cyclictest0-21swapper/319:59:173
313279980,4cyclictest0-21swapper/319:05:533
313249980,7cyclictest0-21swapper/120:23:151
313249980,7cyclictest0-21swapper/119:05:261
313239987,0cyclictest0-21swapper/019:34:440
313239987,0cyclictest0-21swapper/019:11:440
313239980,7cyclictest0-21swapper/020:56:510
313239980,7cyclictest0-21swapper/020:32:560
313239980,7cyclictest0-21swapper/019:59:280
31116280,0sleep10-21swapper/120:05:081
313279976,0cyclictest0-21swapper/320:30:213
313279972,4cyclictest0-21swapper/320:14:593
313279971,2cyclictest0-21swapper/319:20:143
313279970,5cyclictest0-21swapper/319:35:133
313279970,3cyclictest0-21swapper/321:00:273
313279970,2cyclictest0-21swapper/320:55:133
313249975,1cyclictest12327-21kworker/1:019:30:191
313249972,3cyclictest24499-21cpuspeed_turbos20:55:081
313249972,2cyclictest0-21swapper/120:45:141
313249970,2cyclictest0-21swapper/120:00:161
313249970,1cyclictest7041-21sshd19:16:271
313249970,1cyclictest3995-21df20:15:091
313249970,1cyclictest31862sleep119:10:121
313249970,1cyclictest290702sleep121:03:251
313249970,1cyclictest22392-21fschecks_time20:50:101
313249970,1cyclictest22086-21smart_sda19:45:191
313249970,1cyclictest17534-21irqstats20:40:151
313249970,1cyclictest15299-21awk20:35:161
313239976,0cyclictest0-21swapper/020:39:260
313239976,0cyclictest0-21swapper/019:50:110
313239972,2cyclictest0-21swapper/020:20:140
313239970,6cyclictest11731-21find19:25:180
313239970,1cyclictest9765-21/usr/sbin/munin20:25:140
313239970,1cyclictest31709-21expr19:05:090
313239970,1cyclictest22550-21smartctl20:50:110
313239970,1cyclictest0-21swapper/020:15:020
313239970,1cyclictest0-21swapper/019:45:170
313249964,1cyclictest2927-21kworker/1:120:25:471
313249964,1cyclictest251ktimersoftd/120:10:101
313249964,1cyclictest1584-21kworker/1:119:22:191
313249964,1cyclictest12327-21kworker/1:019:50:111
313249964,1cyclictest12327-21kworker/1:019:42:451
313249960,4cyclictest0-21swapper/119:55:131
313249960,2cyclictest0-21swapper/120:30:141
313249960,2cyclictest0-21swapper/119:25:141
313239960,5cyclictest0-21swapper/020:09:590
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional