You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-01-18 - 11:19
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack5slot0.osadl.org (updated Sat Jan 18, 2025 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
200921310,2sleep11999-21sshd22:46:351
174021290,2sleep30-21swapper/323:59:443
3237421240,2sleep10-21swapper/122:15:421
2559421240,1sleep025593-21bash22:58:040
1564021190,2sleep20-21swapper/221:43:042
3887210834,47sleep30-21swapper/319:08:133
2195321070,2sleep30-21swapper/322:24:063
2660210333,44sleep10-21swapper/119:05:141
950521010,1sleep00-21swapper/022:01:030
77332990,0sleep00-21swapper/021:36:050
41382970,0sleep01847-21nfsd21:48:150
227432960,2sleep322739-21sshd21:34:323
48732930,2sleep14871-21sshd22:40:431
241912930,2sleep10-21swapper/123:10:161
20792930,1sleep00-21swapper/021:44:550
155272920,1sleep02405-21runrttasks23:45:430
70912910,1sleep00-21swapper/022:53:160
313742890,2sleep20-21swapper/223:35:112
280862880,4sleep1429299cyclictest21:35:031
12952870,1sleep2429399cyclictest22:40:232
56652860,1sleep20-21swapper/221:39:032
141672850,1sleep014166-21bash00:10:220
40892840,2sleep20-21swapper/223:56:512
153252840,3sleep10-21swapper/121:30:381
52512830,2sleep30-21swapper/322:40:443
318042830,1sleep20-21swapper/222:55:312
27582830,1sleep30-21swapper/321:51:073
243872830,1sleep024374-21sshd23:22:210
11262830,0sleep041ktimersoftd/000:22:340
7952820,0sleep00-21swapper/022:28:240
288442800,3sleep20-21swapper/222:36:582
168012790,1sleep10-21swapper/123:15:371
76042780,1sleep17600-21bash00:06:271
232782780,2sleep223273-21sshd23:13:182
137532770,1sleep113752-21sshd00:17:021
127792770,2sleep20-21swapper/200:07:012
87542760,2sleep30-21swapper/323:33:033
223062750,1sleep022303-21sshd23:55:270
100002750,2sleep39999-21bash00:20:123
299882740,4sleep3429499cyclictest00:05:253
398827233,11sleep20-21swapper/219:09:242
25142720,4sleep2429399cyclictest23:47:322
175462720,1sleep30-21swapper/321:12:073
310952710,3sleep0429199cyclictest22:06:190
323442700,4sleep1429299cyclictest00:39:421
283602700,3sleep0429199cyclictest00:08:390
321102690,2sleep132087-21sshd22:03:151
315802690,2sleep21848-21nfsd23:17:032
73572670,1sleep20-21swapper/200:23:112
209752660,4sleep3429499cyclictest00:01:253
209752660,4sleep3429499cyclictest00:01:243
148092660,0sleep21846-21nfsd21:30:352
212932650,4sleep3429499cyclictest00:31:413
118752610,3sleep1429299cyclictest21:27:121
384924632,11sleep00-21swapper/019:07:420
94272370,3sleep20-21swapper/200:03:242
94272370,3sleep20-21swapper/200:03:232
25392350,1sleep00-21swapper/022:22:110
189422310,2sleep21848-21nfsd23:09:532
202562300,0sleep00-21swapper/022:33:120
229282270,2sleep322922-21sshd22:02:223
83682260,1sleep30-21swapper/322:56:233
70842260,1sleep10-21swapper/121:51:341
157662260,2sleep20-21swapper/222:20:192
322282250,4sleep3429499cyclictest23:29:143
210112250,2sleep3429499cyclictest22:27:113
122972250,3sleep10-21swapper/100:00:341
122972250,3sleep10-21swapper/100:00:341
88182240,2sleep28812-21sshd23:02:352
324662240,2sleep30-21swapper/323:50:183
102972240,1sleep20-21swapper/200:16:392
67292230,3sleep20-21swapper/222:53:152
93552220,2sleep39352-21bash00:10:023
429499220,19cyclictest3724-21awk00:30:013
232482220,2sleep123241-21sshd00:14:361
194402220,1sleep21848-21nfsd22:14:352
164582220,20sleep1135650irq/27-eth022:11:061
81502210,19sleep226950irq/26-ahci[00023:26:572
69432210,19sleep2135650irq/27-eth000:12:592
429499210,7cyclictest22666-21sshd21:18:463
4292992115,4cyclictest1229-21sshd22:34:281
47232200,2sleep14722-21sshd22:37:471
429299207,5cyclictest12077-21sshd22:56:451
4291992013,6cyclictest25129-21sshd23:01:020
204902200,2sleep120486-21bash23:37:051
180932200,18sleep3135650irq/27-eth022:48:123
429299198,3cyclictest0-21swapper/123:51:561
429299197,5cyclictest135650irq/27-eth021:22:451
429299197,4cyclictest18720-21sshd00:27:581
288532190,4sleep10-21swapper/122:21:341
4294991815,2cyclictest16644-21sshd22:51:023
429499181,12cyclictest26057-21sshd00:39:003
429399182,10cyclictest33-21ksoftirqd/221:57:472
429399180,12cyclictest33-21ksoftirqd/222:28:202
429299188,4cyclictest11777-21sshd22:50:321
429299187,5cyclictest23158-21bash00:24:501
4292991815,2cyclictest0-21swapper/123:23:521
429299180,11cyclictest23276-21sshd22:08:431
242872180,1sleep344-21ksoftirqd/321:21:583
115172180,16sleep2135650irq/27-eth021:45:512
429499175,1cyclictest16732-21sshd23:15:363
429499175,10cyclictest10702-21sshd21:45:443
4294991710,5cyclictest31153-21munin-node00:15:193
429499170,6cyclictest30886-21sshd23:01:363
4293991713,2cyclictest12879-21sshd21:17:492
429399171,11cyclictest9053-21sshd21:14:282
429399170,13cyclictest0-21swapper/223:51:302
429299179,2cyclictest10771-21sshd23:57:281
429299177,3cyclictest0-21swapper/123:31:261
429299170,12cyclictest0-21swapper/122:25:171
429199174,9cyclictest0-21swapper/023:43:270
429199171,12cyclictest10772-21sshd22:35:170
429199170,6cyclictest0-21swapper/022:17:010
296782170,15sleep3135650irq/27-eth022:37:043
270812170,15sleep3135650irq/27-eth023:37:443
4294991612,3cyclictest29397-21sshd23:07:453
429499160,5cyclictest0-21swapper/322:08:373
429499160,3cyclictest135650irq/27-eth023:41:583
429499160,14cyclictest20861-21sshd23:10:043
429499160,14cyclictest0-21swapper/321:40:353
429499160,14cyclictest0-21swapper/321:27:053
429499160,11cyclictest3130-21sshd21:35:383
429399168,6cyclictest1285-21sshd21:50:582
429399166,2cyclictest10271-21sshd22:19:542
429399165,3cyclictest4771-21sshd23:20:282
429399163,8cyclictest6615-21sshd22:47:022
4293991613,2cyclictest27799-21sshd22:06:002
4293991611,3cyclictest0-21swapper/223:41:312
429399160,11cyclictest25045-21sshd22:02:332
429299163,4cyclictest135650irq/27-eth021:11:581
429299160,9cyclictest0-21swapper/123:40:541
429299160,7cyclictest0-21swapper/121:45:531
429299160,14cyclictest0-21swapper/121:55:121
429299160,14cyclictest0-21swapper/119:10:111
429299160,12cyclictest0-21swapper/121:44:401
429299160,11cyclictest0-21swapper/121:16:491
429199168,2cyclictest0-21swapper/000:17:240
429199162,3cyclictest17896-21sshd22:41:580
429199161,1cyclictest0-21swapper/023:33:570
429199160,3cyclictest0-21swapper/023:15:160
429199160,14cyclictest27644-21cp21:13:060
429199160,12cyclictest0-21swapper/000:04:040
429199160,12cyclictest0-21swapper/000:04:030
429199160,11cyclictest7878-21sshd21:17:200
221842160,14sleep1135650irq/27-eth000:31:481
429499157,6cyclictest12635-21sshd23:45:263
429499151,13cyclictest135650irq/27-eth021:57:513
4294991510,3cyclictest1437-21cp22:31:243
429499150,6cyclictest0-21swapper/322:10:443
429499150,3cyclictest30909-21sshd22:15:323
429399158,5cyclictest28080-21sshd00:25:192
429399153,8cyclictest4960-21sshd21:26:312
429399153,8cyclictest0-21swapper/219:10:392
429399153,11cyclictest2933-21sshd22:31:332
429399151,12cyclictest12541-21sshd21:20:462
429399150,13cyclictest15373-21sshd23:30:322
429299155,2cyclictest23296-21sshd23:46:271
429299153,5cyclictest23122-21sshd23:25:171
4292991511,3cyclictest15048-21cpuspeed_turbos23:00:121
429199158,3cyclictest24110-21sshd21:25:140
429199155,9cyclictest3-21ksoftirqd/023:27:430
4291991513,1cyclictest1848-21nfsd00:39:230
429199150,13cyclictest0-21swapper/023:35:490
429199150,12cyclictest1358-21kworker/0:019:43:070
429199150,12cyclictest0-21swapper/021:56:550
429199150,12cyclictest0-21swapper/021:21:510
429499142,9cyclictest0-21swapper/319:35:443
429499142,1cyclictest0-21swapper/319:18:073
429499141,12cyclictest0-21swapper/319:31:233
429499140,12cyclictest18804-21sshd23:21:503
429399149,4cyclictest135650irq/27-eth019:41:092
429399148,4cyclictest21921-21bash00:35:122
429399147,6cyclictest0-21swapper/220:55:502
429399143,4cyclictest17612-21kworker/u8:219:45:592
429399140,12cyclictest9381-21sshd00:30:262
429299141,1cyclictest3618-21sshd23:05:141
429299141,11cyclictest0-21swapper/120:50:171
429299140,13cyclictest0-21swapper/119:25:111
429199148,5cyclictest0-21swapper/022:50:010
429199148,2cyclictest0-21swapper/000:31:520
429199144,5cyclictest0-21swapper/019:10:140
429199142,8cyclictest0-21swapper/023:11:490
429199142,2cyclictest0-21swapper/020:54:160
429199141,1cyclictest135650irq/27-eth023:53:410
4291991410,3cyclictest0-21swapper/000:27:440
429199140,1cyclictest2749-21sshd21:32:340
429199140,13cyclictest0-21swapper/023:06:030
429199140,13cyclictest0-21swapper/021:01:540
429199140,11cyclictest0-21swapper/022:12:440
429199140,11cyclictest0-21swapper/021:53:010
429499139,3cyclictest135650irq/27-eth020:50:453
429499139,3cyclictest135650irq/27-eth020:00:053
429499139,3cyclictest0-21swapper/320:56:523
429499137,4cyclictest24899-21atenall_r5power19:55:123
429499134,1cyclictest0-21swapper/319:10:173
429499132,6cyclictest0-21swapper/319:25:173
4294991311,1cyclictest0-21swapper/319:47:053
429499130,3cyclictest5452-21users20:20:203
429499130,12cyclictest0-21swapper/321:05:483
429499130,12cyclictest0-21swapper/321:00:383
429499130,12cyclictest0-21swapper/320:30:003
429499130,12cyclictest0-21swapper/320:27:423
429499130,12cyclictest0-21swapper/320:13:273
429499130,12cyclictest0-21swapper/319:40:023
429499130,12cyclictest0-21swapper/319:20:013
429499130,11cyclictest24216-21runrttasks19:51:003
429499130,11cyclictest15970-21irqstats20:45:173
429499130,11cyclictest15970-21irqstats20:45:163
429399138,3cyclictest30649-21munin-node20:05:202
429399138,3cyclictest23170-21ntp_states21:00:212
429399138,3cyclictest17609-21munin-node20:50:162
429399137,5cyclictest0-21swapper/219:25:042
429399131,3cyclictest13582-21fw_packets20:40:132
429399131,3cyclictest12281-21users20:35:212
4293991311,1cyclictest0-21swapper/220:23:242
429399131,10cyclictest9246-21memory20:30:172
429399131,10cyclictest14008-21munin-node19:30:172
429399130,4cyclictest2405-21runrttasks19:51:032
429399130,12cyclictest1847-21nfsd19:36:332
429399130,12cyclictest0-21swapper/220:26:242
429399130,12cyclictest0-21swapper/220:15:372
429399130,12cyclictest0-21swapper/219:22:542
429399130,12cyclictest0-21swapper/219:19:272
429399130,11cyclictest31905-21df20:10:162
429399130,11cyclictest27942-21munin-node20:00:182
429299137,5cyclictest0-21swapper/119:20:101
429299132,3cyclictest135650irq/27-eth020:40:251
429299132,0cyclictest0-21swapper/120:25:431
429299131,3cyclictest0-21swapper/119:35:211
429299131,3cyclictest0-21swapper/119:30:171
429299131,11cyclictest0-21swapper/120:22:261
429299131,10cyclictest0-21swapper/120:45:181
429299131,10cyclictest0-21swapper/120:45:181
429299130,8cyclictest26849-21awk20:00:001
429299130,5cyclictest0-21swapper/119:40:271
429299130,5cyclictest0-21swapper/119:15:091
429299130,12cyclictest135650irq/27-eth021:00:001
429299130,12cyclictest0-21swapper/120:31:271
429299130,12cyclictest0-21swapper/120:15:011
429299130,12cyclictest0-21swapper/120:10:311
429299130,12cyclictest0-21swapper/120:06:381
429299130,12cyclictest0-21swapper/120:00:111
429299130,12cyclictest0-21swapper/119:50:531
429299130,10cyclictest22297-21munin-run19:50:001
4291991311,1cyclictest0-21swapper/020:49:030
4291991311,1cyclictest0-21swapper/020:49:020
429199130,8cyclictest0-21swapper/019:55:140
429199130,5cyclictest0-21swapper/019:20:190
429199130,3cyclictest30993-21users20:05:230
429199130,3cyclictest0-21swapper/019:47:140
429199130,12cyclictest0-21swapper/020:00:120
429499127,3cyclictest29444-21munin-node20:05:103
429499120,3cyclictest11752-21ntp_kernel_err20:35:173
429499120,11cyclictest135650irq/27-eth020:15:023
429499120,10cyclictest0-21swapper/320:40:133
429399128,3cyclictest25466-21sh19:55:152
429399127,4cyclictest15181-21runrttasks20:45:042
429399127,4cyclictest15181-21runrttasks20:45:032
429399121,10cyclictest25451-21ntp_states21:05:182
429299128,3cyclictest0-21swapper/120:35:111
429299127,4cyclictest19834-21atenall_r5power20:55:101
429299120,11cyclictest135650irq/27-eth021:05:011
429199127,1cyclictest0-21swapper/020:20:180
4291991210,1cyclictest0-21swapper/020:40:260
4291991210,1cyclictest0-21swapper/019:39:090
429199120,8cyclictest0-21swapper/020:15:210
429199120,1cyclictest0-21swapper/021:06:200
429199120,1cyclictest0-21swapper/020:28:580
429199120,11cyclictest0-21swapper/020:36:210
429199120,11cyclictest0-21swapper/020:10:290
429199120,11cyclictest0-21swapper/019:50:330
429199120,11cyclictest0-21swapper/019:30:120
429199120,11cyclictest0-21swapper/019:28:000
429199114,3cyclictest0-21swapper/020:30:120
429199110,3cyclictest0-21swapper/019:15:210
429199110,10cyclictest20179-21fschecks_count20:55:110
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional