You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-05-28 - 02:47

x86 Intel Xeon-E31220L @2200 MHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #5, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rack5slot0.osadl.org (updated Sun May 28, 2023 00:44:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
275121350,3sleep10-21swapper/123:38:471
8019210133,55sleep10-21swapper/119:07:431
220242970,2sleep12121-21runrttasks23:48:241
37362930,2sleep041ktimersoftd/023:12:410
125822930,4sleep00-21swapper/021:19:030
232082890,1sleep0837899cyclictest23:03:480
21982890,1sleep02199-21bash23:08:480
58642860,2sleep20-21swapper/221:33:002
246632860,2sleep10-21swapper/121:35:111
281532840,0sleep00-21swapper/000:34:410
820528333,46sleep30-21swapper/319:09:583
100002830,2sleep30-21swapper/322:17:423
100002830,2sleep30-21swapper/322:17:413
197532820,0sleep019750-21sshd22:52:160
131352820,3sleep30-21swapper/322:36:413
13052820,5sleep3838299cyclictest23:16:053
86952810,2sleep28687-21hddtemp_smartct21:55:252
86952810,2sleep28687-21hddtemp_smartct21:55:252
176672810,2sleep20-21swapper/223:06:492
138272810,1sleep30-21swapper/323:25:073
61532800,0sleep041ktimersoftd/022:35:490
152632790,2sleep30-21swapper/300:21:373
112332780,2sleep30-21swapper/300:25:013
272112770,4sleep3838299cyclictest00:15:283
72272760,2sleep20-21swapper/222:06:152
72272760,2sleep20-21swapper/222:06:152
85262740,0sleep00-21swapper/023:16:580
208052740,1sleep30-21swapper/322:45:023
44162730,4sleep3838299cyclictest00:39:283
806227232,9sleep20-21swapper/219:08:162
59142700,1sleep041ktimersoftd/022:43:200
310282680,3sleep2838199cyclictest22:38:482
235832670,4sleep3838299cyclictest21:42:223
302632650,2sleep30-21swapper/323:41:543
270682640,4sleep3838299cyclictest21:24:243
23972630,2sleep30-21swapper/322:13:083
109172590,1sleep20-21swapper/223:54:392
142402480,1sleep20-21swapper/222:55:232
813324632,11sleep00-21swapper/019:09:070
222062460,1sleep222203-21sshd23:22:172
312812372,33sleep1138550irq/27-eth021:32:121
34432360,1sleep10-21swapper/122:46:401
70372340,4sleep20-21swapper/200:13:072
151772340,2sleep10-21swapper/122:40:351
837899334,8cyclictest6993-21sshd00:13:050
312282280,1sleep20-21swapper/222:12:432
121662270,3sleep20-21swapper/223:17:242
312782260,24sleep3138550irq/27-eth022:57:183
838199253,18cyclictest0-21swapper/220:55:292
198032250,3sleep20-21swapper/221:38:172
307332240,2sleep230727-21sshd22:49:552
103532240,22sleep2138550irq/27-eth022:02:582
103532240,22sleep2138550irq/27-eth022:02:582
838199230,10cyclictest14051-21bash22:21:542
838199230,10cyclictest14051-21bash22:21:532
837899230,12cyclictest0-21swapper/023:57:220
308822230,0sleep230885-21bash21:17:252
140552231,19sleep1138550irq/27-eth021:48:441
111672232,18sleep3138550irq/27-eth022:06:453
111672232,18sleep3138550irq/27-eth022:06:443
837899220,14cyclictest0-21swapper/021:20:200
29862220,2sleep32987-21id23:38:483
224382220,20sleep1138550irq/27-eth000:26:171
116062220,2sleep111601-21rm21:44:451
838299216,3cyclictest445-21jbd2/dm-0-822:27:293
838199212,14cyclictest0-21swapper/222:42:492
837899218,5cyclictest0-21swapper/023:50:110
837899217,5cyclictest0-21swapper/022:05:230
837899217,5cyclictest0-21swapper/022:05:230
837899210,9cyclictest11807-21sshd22:29:090
28372210,1sleep01876-21nfsd23:27:320
256862210,19sleep3138550irq/27-eth021:53:423
107232210,1sleep00-21swapper/022:17:470
107232210,1sleep00-21swapper/022:17:460
838199201,7cyclictest138550irq/27-eth021:52:022
238122200,18sleep1138550irq/27-eth000:07:281
838299190,16cyclictest0-21swapper/323:13:163
838199193,7cyclictest138550irq/27-eth023:59:452
838199193,7cyclictest138550irq/27-eth023:01:422
838199192,14cyclictest0-21swapper/200:27:322
171042191,16sleep1138550irq/27-eth022:03:461
171042191,16sleep1138550irq/27-eth022:03:451
96612180,16sleep1138550irq/27-eth022:14:001
838199183,11cyclictest30247-21sshd00:08:142
838199183,10cyclictest19470-21sshd00:22:082
838199182,5cyclictest138550irq/27-eth000:33:062
838199182,10cyclictest12915-21id23:32:262
838199181,9cyclictest0-21swapper/220:45:442
838099180,16cyclictest22-21ksoftirqd/123:17:151
837899180,16cyclictest25751-21sshd23:22:410
151512180,3sleep10-21swapper/121:15:321
8382991710,6cyclictest24079-21sshd23:22:293
838299170,4cyclictest3222-21sshd21:25:213
838199174,8cyclictest28984-21df19:55:182
8381991713,3cyclictest14402-21bash21:22:552
838199171,11cyclictest138550irq/27-eth021:14:112
838199171,10cyclictest138550irq/27-eth020:07:182
838199170,14cyclictest12948-21sshd23:36:082
838099178,4cyclictest1807-21sshd21:10:251
838099177,4cyclictest5202-21sshd23:20:211
838099170,16cyclictest0-21swapper/122:52:451
837899178,7cyclictest0-21swapper/022:45:240
837899171,15cyclictest0-21swapper/021:58:070
837899171,15cyclictest0-21swapper/021:58:060
299042170,15sleep1138550irq/27-eth022:35:001
298402170,2sleep129841-21sshd22:16:111
298402170,2sleep129841-21sshd22:16:111
245412170,2sleep224539-21bash23:41:122
11262171,14sleep2138550irq/27-eth022:27:562
838299168,3cyclictest0-21swapper/321:17:213
838299162,12cyclictest0-21swapper/321:35:083
838299160,14cyclictest0-21swapper/323:45:163
838199163,8cyclictest8660-21munin-run00:40:002
838199163,8cyclictest23973-21kworker/u8:019:12:062
838199163,8cyclictest138550irq/27-eth023:45:022
838199163,8cyclictest138550irq/27-eth019:16:582
838199163,7cyclictest1987-21sshd21:42:212
838199163,7cyclictest138550irq/27-eth022:32:462
838199163,10cyclictest21870-21bash23:10:572
838199163,10cyclictest1877-21nfsd21:46:382
838199163,10cyclictest15223-21sshd23:25:152
838199162,8cyclictest1876-21nfsd19:36:502
838199162,11cyclictest0-21swapper/220:29:292
838199162,10cyclictest138550irq/27-eth022:50:012
838199161,10cyclictest3596-21sshd21:29:052
838199160,4cyclictest21908-21fschecks_count22:15:222
838199160,4cyclictest21908-21fschecks_count22:15:212
838199160,10cyclictest138550irq/27-eth020:35:302
838099167,5cyclictest0-21swapper/122:21:291
838099167,5cyclictest0-21swapper/122:21:291
838099167,3cyclictest0-21swapper/100:00:331
8380991613,2cyclictest6103-21sshd00:20:271
8380991612,3cyclictest0-21swapper/123:55:311
8380991611,4cyclictest20061-21sshd23:44:301
838099160,4cyclictest2499-21sshd23:53:401
838099160,14cyclictest0-21swapper/119:40:161
838099160,11cyclictest445-21jbd2/dm-0-800:35:171
838099160,11cyclictest19409-21sshd23:03:201
8378991613,2cyclictest1759-21bash21:54:410
837899160,1cyclictest0-21swapper/000:04:200
292452160,1sleep30-21swapper/323:08:103
291622160,14sleep2138550irq/27-eth000:15:392
22232160,14sleep1138550irq/27-eth021:21:291
219232160,14sleep3138550irq/27-eth000:10:593
838299158,6cyclictest3240-21sshd22:43:013
838299158,6cyclictest27384-21cp23:34:093
838299155,8cyclictest0-21swapper/320:30:253
838299154,6cyclictest21772-21idleruntime20:50:213
838299152,3cyclictest5744-21sshd21:10:473
8382991511,3cyclictest8158-21sshd23:02:013
838299150,6cyclictest30870-21sshd22:20:093
838299150,6cyclictest30870-21sshd22:20:083
838299150,6cyclictest0-21swapper/321:03:183
838299150,3cyclictest0-21swapper/323:57:283
838299150,13cyclictest0-21swapper/319:48:133
838299150,11cyclictest5364-21sshd21:32:563
838299150,11cyclictest21387-21runrttasks19:35:583
838299150,11cyclictest138550irq/27-eth020:55:293
838199156,4cyclictest0-21swapper/219:23:362
838199156,2cyclictest23313-21ls19:40:242
838199153,10cyclictest47-21rcuop/319:45:152
838199152,10cyclictest14028-21taskset20:34:072
8381991510,4cyclictest0-21swapper/219:29:062
8381991510,3cyclictest28383-21bash00:00:232
838099150,3cyclictest28335-21sshd21:57:391
838099150,3cyclictest28335-21sshd21:57:381
838099150,14cyclictest22-21ksoftirqd/122:56:501
838099150,13cyclictest4598-21hddtemp_smartct23:05:211
838099150,12cyclictest7384-21basename20:20:001
838099150,11cyclictest537-21sshd00:16:051
837899152,12cyclictest0-21swapper/000:35:240
837899150,14cyclictest0-21swapper/022:58:340
47852150,13sleep1138550irq/27-eth022:05:591
47852150,13sleep1138550irq/27-eth022:05:581
192602151,12sleep2138550irq/27-eth019:32:212
157512150,13sleep128750irq/26-ahci[00000:10:221
838299148,5cyclictest15058-21if_eth020:35:213
838299148,5cyclictest0-21swapper/320:06:353
838299148,4cyclictest4926-21munin-node00:05:323
838299147,5cyclictest24363-21fschecks_time00:30:213
838299147,3cyclictest23953-21sshd21:49:493
8382991412,1cyclictest0-21swapper/321:56:103
8382991412,1cyclictest0-21swapper/321:56:093
838299140,3cyclictest138550irq/27-eth020:16:023
838299140,12cyclictest9759-21swap19:10:233
838299140,12cyclictest5420-21sshd22:50:343
838299140,12cyclictest4863-21sshd23:53:553
838299140,12cyclictest28714-21munin-node19:55:183
838299140,12cyclictest11728-21sshd22:32:503
838199143,1cyclictest0-21swapper/220:23:222
838199141,7cyclictest138550irq/27-eth020:44:462
8381991410,1cyclictest0-21swapper/219:50:482
838099148,4cyclictest13389-21munin-node19:20:181
838099147,3cyclictest8750-21sshd21:25:521
838099143,10cyclictest22-21ksoftirqd/121:50:471
838099142,11cyclictest138550irq/27-eth020:56:201
838099140,3cyclictest138550irq/27-eth020:27:311
838099140,13cyclictest8997-21sshd22:25:071
838099140,12cyclictest17844-21bash23:10:311
838099140,11cyclictest0-21swapper/119:55:241
837899149,1cyclictest0-21swapper/000:20:270
837899148,3cyclictest0-21swapper/019:31:050
837899145,4cyclictest0-21swapper/000:29:100
837899140,4cyclictest0-21swapper/019:40:350
837899140,3cyclictest0-21swapper/023:45:050
837899140,1cyclictest0-21swapper/021:47:540
837899140,1cyclictest0-21swapper/000:06:140
837899140,13cyclictest13732-21sshd21:33:540
837899140,13cyclictest0-21swapper/020:35:180
837899140,12cyclictest0-21swapper/021:35:230
837899140,11cyclictest0-21swapper/022:32:540
837899140,10cyclictest0-21swapper/023:35:250
837899140,10cyclictest0-21swapper/022:00:140
837899140,10cyclictest0-21swapper/022:00:140
202422140,2sleep10-21swapper/123:33:181
838299137,4cyclictest9851-21munin-node20:25:293
838299131,4cyclictest26028-21sshd00:00:083
838299130,5cyclictest0-21swapper/319:33:403
838299130,4cyclictest30966-21munin-node20:00:203
838299130,4cyclictest17202-21sshd22:00:073
838299130,4cyclictest17202-21sshd22:00:063
838299130,12cyclictest28122-21runrttasks19:50:483
838299130,12cyclictest0-21swapper/320:48:543
838299130,12cyclictest0-21swapper/320:41:453
838299130,12cyclictest0-21swapper/320:21:573
838299130,12cyclictest0-21swapper/320:11:003
838299130,12cyclictest0-21swapper/319:42:303
838199139,3cyclictest138550irq/27-eth020:10:222
838199130,12cyclictest32153-21proc_pri20:00:272
838199130,12cyclictest0-21swapper/221:06:552
838199130,12cyclictest0-21swapper/221:00:152
838199130,12cyclictest0-21swapper/220:54:462
838099137,5cyclictest0-21swapper/120:22:051
838099136,6cyclictest0-21swapper/120:10:201
838099134,8cyclictest138550irq/27-eth022:35:501
838099132,10cyclictest138550irq/27-eth019:25:491
838099130,8cyclictest13532-21sshd23:25:041
838099130,5cyclictest1206-21fschecks_count20:05:181
838099130,5cyclictest0-21swapper/119:51:051
838099130,4cyclictest31447-21sshd00:31:091
838099130,4cyclictest0-21swapper/119:16:181
838099130,12cyclictest138550irq/27-eth020:40:061
838099130,12cyclictest0-21swapper/121:00:361
838099130,12cyclictest0-21swapper/120:02:161
838099130,12cyclictest0-21swapper/119:48:271
838099130,12cyclictest0-21swapper/119:30:081
838099130,11cyclictest19696-21memory20:45:231
838099130,10cyclictest138550irq/27-eth020:30:321
837899138,4cyclictest18171-21sshd23:40:280
837899138,4cyclictest15763-21sshd21:11:580
837899137,3cyclictest0-21swapper/021:41:490
837899132,6cyclictest25366-21awk21:00:000
8378991311,1cyclictest16931-21sshd21:26:510
8378991311,1cyclictest0-21swapper/023:30:090
8378991311,1cyclictest0-21swapper/019:55:350
8378991311,1cyclictest0-21swapper/019:35:100
837899131,10cyclictest0-21swapper/019:45:330
837899130,3cyclictest0-21swapper/022:20:160
837899130,3cyclictest0-21swapper/022:20:150
837899130,1cyclictest0-21swapper/000:17:450
837899130,10cyclictest0-21swapper/019:17:320
838299127,4cyclictest0-21swapper/321:05:023
838299120,11cyclictest15939-21if_err_eth019:25:223
838299120,10cyclictest13307-21diskstats19:20:193
838299120,10cyclictest11065-21diskstats19:15:243
838199120,11cyclictest0-21swapper/220:16:102
838099120,3cyclictest22208-21ntp_states20:50:251
838099120,3cyclictest14611-21df20:35:191
838099120,11cyclictest0-21swapper/121:05:321
838099120,11cyclictest0-21swapper/119:10:111
838099120,10cyclictest0-21swapper/119:35:221
837899129,2cyclictest101432sleep019:10:500
837899128,3cyclictest0-21swapper/020:26:030
837899128,3cyclictest0-21swapper/019:51:040
837899123,1cyclictest6612-21smart_sda20:15:240
8378991210,1cyclictest0-21swapper/020:54:470
837899120,4cyclictest0-21swapper/020:01:360
837899120,2cyclictest0-21swapper/020:05:510
837899120,2cyclictest0-21swapper/019:25:040
837899120,1cyclictest27822-21bash22:12:200
837899120,11cyclictest26440-21memory21:00:210
837899120,11cyclictest0-21swapper/021:05:270
837899120,11cyclictest0-21swapper/020:40:460
837899120,11cyclictest0-21swapper/020:31:110
837899120,11cyclictest0-21swapper/020:13:030
837899120,11cyclictest0-21swapper/019:21:190
837899110,3cyclictest0-21swapper/020:45:310
837899110,10cyclictest0-21swapper/020:20:060
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional