You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-10-06 - 06:50

x86 Intel Core i3-2100T @2500 MHz, Linux 4.14.28-rt23 (Profile)

Latency plot of system in rack #0, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack0slot0.osadl.org (updated Thu Oct 06, 2022 00:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1533721650,3sleep2484499cyclictest23:37:052
296321540,0sleep70-21swapper/700:05:567
296321540,0sleep70-21swapper/700:05:557
2175621500,3sleep3485599cyclictest21:36:163
108521480,4sleep5486899cyclictest23:52:225
1549621380,1sleep41-21systemd00:29:434
2074321290,3sleep1484099cyclictest22:57:201
1576721270,0sleep70-21swapper/723:55:217
3064221240,1sleep430641-21sshd21:19:434
1235121220,0sleep00-21swapper/022:59:260
1754221190,1sleep20-21swapper/200:39:342
1528221190,0sleep40-21swapper/400:22:214
2482321170,0sleep60-21swapper/600:11:416
2482321170,0sleep60-21swapper/600:11:416
2061321170,1sleep30-21swapper/300:17:393
1645921150,1sleep342-21ksoftirqd/300:27:173
2921821090,0sleep40-21swapper/422:25:004
1620621040,0sleep316202-21bash00:37:033
299521020,0sleep60-21swapper/600:27:596
1245721000,2sleep078950irq/26-eno1-tx-00:33:150
81162980,0sleep40-21swapper/400:31:504
308642960,2sleep6487299cyclictest00:21:436
203002920,0sleep10-21swapper/123:27:341
53622910,0sleep45355-21sshd00:03:374
135852880,1sleep313524-21sshd23:30:563
143842870,1sleep4501ktimersoftd/422:29:164
136102870,0sleep30-21swapper/323:55:173
3842860,0sleep60-21swapper/622:34:486
222962860,1sleep60-21swapper/621:41:086
158732860,4sleep6487299cyclictest21:11:516
141582860,1sleep40-21swapper/423:55:194
98732850,0sleep10-21swapper/100:07:271
98732850,0sleep10-21swapper/100:07:261
275532850,0sleep3401rcuc/321:17:093
136622850,0sleep10-21swapper/100:00:131
75912840,2sleep4751-21NetworkManager22:36:144
247442840,1sleep50-21swapper/521:25:295
235932840,1sleep6671rcuc/622:56:136
197422840,0sleep10-21swapper/100:35:561
184242840,4sleep6487299cyclictest00:17:336
172942840,0sleep20-21swapper/222:32:592
9782830,0sleep70-21swapper/722:15:257
69722830,1sleep21-21systemd23:25:512
38342830,0sleep60-21swapper/623:31:486
308292830,1sleep70-21swapper/722:05:387
298662820,0sleep0121rcuc/022:45:310
86342810,0sleep30-21swapper/300:12:183
86342810,0sleep30-21swapper/300:12:183
278592810,0sleep60-21swapper/600:00:416
210732810,1sleep00-21swapper/000:38:280
300762800,0sleep10-21swapper/122:15:191
296202800,5sleep5486899cyclictest21:12:225
279792800,1sleep727978-21bash21:30:287
226572800,1sleep70-21swapper/722:34:247
206092800,1sleep30-21swapper/323:28:483
146472800,1sleep314612-21sshd23:22:313
27682790,1sleep60-21swapper/621:47:436
261822790,4sleep5486899cyclictest21:36:275
260282790,0sleep50-21swapper/522:57:315
200262790,1sleep620030-21gltestperf23:20:186
144112790,0sleep50-21swapper/523:56:325
11602790,1sleep2311rcuc/200:12:012
11602790,1sleep2311rcuc/200:12:002
98092780,1sleep69810-21sshd21:20:096
91882780,1sleep59190-21bash22:48:255
315402780,0sleep50-21swapper/522:43:135
307552780,0sleep20-21swapper/222:45:342
192182780,1sleep00-21swapper/022:40:190
80002770,1sleep00-21swapper/022:13:170
157432770,1sleep715742-21sshd23:43:117
118282770,0sleep4501ktimersoftd/421:28:414
79832760,1sleep30-21swapper/323:49:013
72792760,0sleep50-21swapper/522:08:265
71732760,0sleep20-21swapper/222:24:092
145912760,0sleep30-21swapper/323:00:423
139662760,0sleep50-21swapper/523:35:475
110482760,0sleep10-21swapper/122:44:531
86832750,4sleep0483499cyclictest22:08:290
5472750,1sleep3545-21grepconf.sh21:31:513
203282750,0sleep020331-21grepconf.sh21:12:010
138062750,1sleep713773-21sshd21:27:307
67322740,3sleep3485599cyclictest21:59:573
30442740,0sleep30-21swapper/323:06:203
30442740,0sleep30-21swapper/323:06:193
298672740,0sleep70-21swapper/721:41:267
265942740,0sleep40-21swapper/421:30:244
240732740,0sleep50-21swapper/523:33:475
228612740,1sleep01-21systemd21:38:470
173062730,1sleep20-21swapper/222:54:482
173062730,1sleep20-21swapper/222:54:472
89522720,1sleep68950-21grepconf.sh22:24:136
82512720,4sleep5486899cyclictest23:25:535
300722720,0sleep30-21swapper/322:48:003
165772720,0sleep50-21swapper/523:14:075
249572710,0sleep50-21swapper/523:07:105
249572710,0sleep50-21swapper/523:07:095
235022710,0sleep70-21swapper/722:24:467
232532710,1sleep7771ktimersoftd/722:55:017
232532710,1sleep7771ktimersoftd/722:55:007
225052710,0sleep20-21swapper/221:55:402
9182690,0sleep00-21swapper/000:18:070
231942690,3sleep6487299cyclictest22:35:356
205822690,0sleep50-21swapper/523:16:395
111272690,0sleep30-21swapper/323:19:593
79872680,0sleep10-21swapper/121:44:161
56462670,1sleep05643-21sshd00:23:130
241252670,0sleep30-21swapper/323:44:433
152642670,1sleep015260-21sshd23:35:500
324442640,1sleep51-21systemd21:41:315
261772620,0sleep526179-21sshd00:14:125
261772620,0sleep526179-21sshd00:14:125
143942620,0sleep70-21swapper/721:50:317
265182610,1sleep026519-21id00:26:250
76562600,0sleep10-21swapper/121:51:291
286472590,1sleep1221rcuc/121:11:051
449125848,6sleep40-21swapper/419:08:444
155662570,1sleep6671rcuc/600:07:406
155662570,1sleep6671rcuc/600:07:396
133322570,0sleep40-21swapper/421:43:154
256125646,6sleep60-21swapper/619:05:086
145032560,10sleep278850irq/25-eno1-rx-21:54:122
445025343,6sleep10-21swapper/119:08:071
270022530,0sleep20-21swapper/222:12:482
447425241,7sleep50-21swapper/519:08:275
177892510,0sleep60-21swapper/600:34:426
457024939,6sleep00-21swapper/019:09:430
251332490,0sleep00-21swapper/023:50:500
159342490,3sleep5486899cyclictest00:29:445
455424826,7sleep20-21swapper/219:09:292
449424736,7sleep70-21swapper/719:08:467
291072470,0sleep20-21swapper/200:21:392
198272460,1sleep451-21ksoftirqd/423:37:154
111502460,0sleep30-21swapper/322:53:213
111502460,0sleep30-21swapper/322:53:203
322552450,3sleep41-21systemd00:08:184
322552450,3sleep41-21systemd00:08:184
305682450,1sleep230524-21sshd00:18:022
84232430,0sleep70-21swapper/700:17:117
441624231,7sleep30-21swapper/319:07:453
125642420,0sleep20-21swapper/222:17:042
322852390,0sleep50-21swapper/500:34:015
4834993614,2cyclictest19399-21gdbus22:51:550
4834993614,2cyclictest19399-21gdbus22:51:550
116112340,1sleep60-21swapper/623:03:026
119352330,0sleep40-21swapper/400:19:484
35792320,0sleep53581-21id22:11:545
281162300,15sleep278850irq/25-eno1-rx-22:06:452
187862280,0sleep30-21swapper/322:56:023
161442270,4sleep616147-21sshd22:49:546
306202260,1sleep50-21swapper/521:24:335
307002250,4sleep7487899cyclictest00:02:037
242922250,1sleep30-21swapper/300:23:553
199192250,1sleep70-21swapper/721:58:027
149232250,1sleep6684-21dbus-daemon00:35:466
486199240,12cyclictest7377-21sshd23:30:434
199202240,0sleep50-21swapper/523:21:295
170522240,1sleep617056-21sshd22:51:086
170522240,1sleep617056-21sshd22:51:076
76882230,0sleep20-21swapper/222:55:372
35212230,1sleep43515-21sshd23:02:444
326092230,1sleep332612-21sshd00:07:053
326092230,1sleep332612-21sshd00:07:043
293932230,1sleep40-21swapper/423:49:504
67912220,0sleep50-21swapper/522:16:525
486199223,8cyclictest683-21systemd-logind22:42:374
486199220,8cyclictest7393-21sshd22:03:364
486199220,8cyclictest7393-21sshd22:03:354
486199220,21cyclictest0-21swapper/419:54:324
486199220,17cyclictest24613-21sshd23:10:464
486199220,10cyclictest0-21swapper/421:57:434
240052220,1sleep3411ktimersoftd/322:12:413
22942220,4sleep0483499cyclictest21:33:090
486199213,6cyclictest463-21systemd-journal22:18:314
486199212,8cyclictest684-21dbus-daemon21:10:064
486199210,8cyclictest684-21dbus-daemon22:56:434
486199210,14cyclictest0-21swapper/422:10:404
486199210,11cyclictest24518-21grepconf.sh23:20:264
486199210,10cyclictest29605-21bash00:38:484
323852210,1sleep00-21swapper/022:20:140
281272210,0sleep50-21swapper/521:18:225
258572210,1sleep4491rcuc/422:05:284
135032210,1sleep4501ktimersoftd/423:07:574
135032210,1sleep4501ktimersoftd/423:07:564
78862200,2sleep6671rcuc/623:06:316
78862200,2sleep6671rcuc/623:06:306
68022200,1sleep10-21swapper/100:31:481
486199203,7cyclictest21530-21nscd21:45:234
486199203,6cyclictest684-21dbus-daemon22:45:474
486199203,4cyclictest0-21swapper/400:12:024
486199203,4cyclictest0-21swapper/400:12:014
486199203,1cyclictest0-21swapper/423:25:354
486199201,18cyclictest0-21swapper/421:08:314
486199200,19cyclictest0-21swapper/420:43:154
486199200,11cyclictest9426-21sshd21:52:474
486199200,11cyclictest26106-21sshd22:50:174
486199200,11cyclictest26106-21sshd22:50:164
483499200,19cyclictest0-21swapper/019:59:170
325002200,1sleep3401rcuc/322:05:423
262902200,0sleep30-21swapper/321:24:223
214432200,1sleep121369-21sshd21:56:511
97832190,1sleep0121rcuc/023:00:320
486199193,15cyclictest0-21swapper/419:27:254
486199190,4cyclictest29291-21sshd23:52:134
486199190,18cyclictest0-21swapper/419:13:374
486199190,17cyclictest0-21swapper/423:41:274
486199190,15cyclictest32121-21bash21:36:414
486199190,10cyclictest0-21swapper/422:30:204
486199190,10cyclictest0-21swapper/421:20:584
484499192,15cyclictest0-21swapper/223:06:152
484499192,15cyclictest0-21swapper/223:06:152
239622190,1sleep1221rcuc/123:02:171
18192190,1sleep7684-21dbus-daemon23:26:527
16952190,0sleep00-21swapper/023:28:040
14282190,5sleep4486199cyclictest23:18:224
486199183,14cyclictest0-21swapper/420:10:214
486199181,15cyclictest0-21swapper/420:36:134
486199180,17cyclictest0-21swapper/420:59:134
483499183,3cyclictest7-21ksoftirqd/021:49:070
483499180,3cyclictest29621-21sshd23:14:370
483499180,1cyclictest678-21irqbalance21:22:380
483499180,1cyclictest0-21swapper/023:40:470
316822180,1sleep631685-21sshd23:13:276
309632180,0sleep30-21swapper/321:25:433
264102180,0sleep60-21swapper/623:42:226
183622180,1sleep60-21swapper/623:56:416
16822180,4sleep2484499cyclictest23:57:162
487299170,15cyclictest1-21systemd23:25:566
486899170,15cyclictest0-21swapper/521:46:225
486199170,16cyclictest0-21swapper/419:35:414
4855991714,2cyclictest20547-21sshd23:13:033
483499170,2cyclictest21034-21perf20:50:000
483499170,2cyclictest0-21swapper/021:25:160
483499170,1cyclictest684-21dbus-daemon00:01:460
327442170,1sleep6681ktimersoftd/622:15:246
320602170,2sleep1221rcuc/100:15:361
309712170,1sleep2311rcuc/223:31:352
262252170,1sleep626228-21kworker/u16:321:35:156
70182160,1sleep7761rcuc/700:30:347
66552160,1sleep76654-21grepconf.sh23:20:587
487299160,14cyclictest0-21swapper/621:25:066
484499160,2cyclictest1-21systemd23:01:072
484099160,14cyclictest0-21swapper/122:29:221
483499168,4cyclictest7-21ksoftirqd/022:37:400
483499163,11cyclictest0-21swapper/019:15:280
483499160,1cyclictest5421-21sshd00:06:020
483499160,1cyclictest5421-21sshd00:06:010
483499160,15cyclictest11646-21sshd23:05:260
483499160,15cyclictest11646-21sshd23:05:250
483499160,14cyclictest81ktimersoftd/021:19:040
271172160,0sleep527062-21sshd21:30:255
258562160,0sleep10-21swapper/123:20:281
487899150,2cyclictest12110-21sshd00:35:397
487899150,2cyclictest10114-21sshd23:52:447
487899150,1cyclictest0-21swapper/723:08:177
487899150,1cyclictest0-21swapper/723:08:177
487899150,13cyclictest1-21systemd21:17:237
487299150,13cyclictest24425-21sshd22:27:136
4855991512,2cyclictest27236-21sshd23:36:183
483499159,1cyclictest7-21ksoftirqd/022:18:360
483499154,3cyclictest7-21ksoftirqd/021:40:270
483499150,1cyclictest18304-21sshd21:54:200
325302150,0sleep70-21swapper/721:46:237
77382140,0sleep30-21swapper/322:01:103
77382140,0sleep30-21swapper/322:01:103
4878991412,1cyclictest78-21ksoftirqd/722:27:337
4878991412,1cyclictest0-21swapper/722:04:597
4878991412,1cyclictest0-21swapper/722:04:587
487899140,2cyclictest15318-21sshd22:43:507
487899140,1cyclictest19399-21gdbus22:38:277
487899140,13cyclictest698-21polkitd00:24:227
487899140,13cyclictest28464-21sshd00:14:177
487899140,13cyclictest28464-21sshd00:14:167
487899140,12cyclictest0-21swapper/722:46:307
4872991412,1cyclictest0-21swapper/623:51:306
487299141,12cyclictest751-21NetworkManager21:15:106
487299140,2cyclictest0-21swapper/621:57:316
487299140,13cyclictest10492-21sshd21:32:156
4868991412,1cyclictest0-21swapper/521:50:315
4868991410,1cyclictest13804-21bash22:23:115
485599140,1cyclictest768-21gdbus22:32:513
485599140,13cyclictest3942-21bash21:10:113
484499140,1cyclictest8048-21sshd21:22:282
484499140,12cyclictest29888-21sshd21:36:352
4840991412,1cyclictest1011-21sshd23:46:461
4840991412,1cyclictest0-21swapper/123:14:471
4840991412,1cyclictest0-21swapper/122:49:151
4840991412,1cyclictest0-21swapper/100:11:471
4840991412,1cyclictest0-21swapper/100:11:461
483499149,3cyclictest7-21ksoftirqd/023:20:200
483499148,3cyclictest7-21ksoftirqd/000:13:230
483499148,3cyclictest7-21ksoftirqd/000:13:230
483499147,3cyclictest7-21ksoftirqd/023:17:560
483499143,8cyclictest7-21ksoftirqd/022:33:140
483499141,3cyclictest101rcu_sched22:03:050
483499141,3cyclictest101rcu_sched22:03:040
483499141,1cyclictest91rcu_preempt23:30:590
483499141,1cyclictest12289-21sshd23:55:150
483499140,1cyclictest101rcu_sched21:56:430
483499140,1cyclictest0-21swapper/023:45:160
281282140,1sleep7761rcuc/723:35:107
180802140,1sleep318082-21grepconf.sh22:28:113
152762140,0sleep60-21swapper/622:07:316
4878991311,1cyclictest78-21ksoftirqd/723:03:087
4878991311,1cyclictest0-21swapper/722:59:177
487899130,12cyclictest10065-21kworker/7:023:16:087
487899130,12cyclictest0-21swapper/719:45:207
487299130,1cyclictest8028-21bash23:46:306
487299130,1cyclictest24334-21sshd21:53:206
487299130,12cyclictest932-21bash23:38:596
486899139,3cyclictest60-21ksoftirqd/523:44:445
4868991310,2cyclictest2634-21bash23:03:555
486899130,1cyclictest15464-21grepconf.sh00:06:255
486899130,1cyclictest15464-21grepconf.sh00:06:255
486899130,12cyclictest0-21swapper/519:40:015
486199130,12cyclictest0-21swapper/420:17:004
485599139,3cyclictest42-21ksoftirqd/322:16:373
485599139,1cyclictest42-21ksoftirqd/323:53:533
485599137,2cyclictest42-21ksoftirqd/322:21:493
4855991311,1cyclictest2468-21sshd21:44:043
485599130,1cyclictest1011-21sshd21:46:533
485599130,12cyclictest5723-21kworker/3:400:33:163
485599130,12cyclictest28253-21sshd21:51:043
4844991311,1cyclictest78850irq/25-eno1-rx-23:53:252
4844991311,1cyclictest751-21NetworkManager00:26:052
4844991311,1cyclictest0-21swapper/221:40:462
484499130,1cyclictest23335-21sshd00:31:122
484499130,1cyclictest10895-21bash23:13:532
484499130,1cyclictest0-21swapper/223:43:292
484499130,1cyclictest0-21swapper/222:35:042
484499130,1cyclictest0-21swapper/200:04:432
484499130,12cyclictest25351-21sshd22:04:172
484499130,12cyclictest25351-21sshd22:04:162
484499130,12cyclictest0-21swapper/223:48:262
484499130,12cyclictest0-21swapper/221:47:442
4840991311,1cyclictest683-21systemd-logind23:07:081
4840991311,1cyclictest683-21systemd-logind23:07:071
4840991311,1cyclictest5846-21sshd00:25:371
4840991311,1cyclictest24-21ksoftirqd/122:07:421
4840991311,1cyclictest0-21swapper/122:13:291
4840991311,1cyclictest0-21swapper/122:02:371
4840991311,1cyclictest0-21swapper/122:02:361
484099130,1cyclictest2912-21sshd22:51:491
484099130,1cyclictest2912-21sshd22:51:481
484099130,1cyclictest0-21swapper/123:34:091
484099130,12cyclictest10796-21sshd21:16:301
484099130,12cyclictest0-21swapper/123:40:361
483499132,4cyclictest7-21ksoftirqd/022:25:190
143602130,0sleep20-21swapper/223:20:052
487899129,1cyclictest78-21ksoftirqd/723:14:407
487899128,1cyclictest78-21ksoftirqd/722:12:577
4878991210,1cyclictest78-21ksoftirqd/723:32:137
4878991210,1cyclictest22959-21unixbench-2d20:50:237
4878991210,1cyclictest0-21swapper/721:35:487
4878991210,1cyclictest0-21swapper/721:23:037
487899120,11cyclictest24214-21kworker/7:200:27:357
487899120,11cyclictest13514-21sshd23:46:437
487299120,11cyclictest0-21swapper/623:15:366
486899128,1cyclictest60-21ksoftirqd/500:22:135
486899122,1cyclictest101rcu_sched22:04:175
486899122,1cyclictest101rcu_sched22:04:165
4868991210,1cyclictest777-21gdbus00:01:215
4868991210,1cyclictest0-21swapper/500:17:415
486899120,1cyclictest101rcu_sched22:34:005
486199124,7cyclictest8461-21/usr/sbin/munin19:15:234
486199123,7cyclictest653-21avahi-daemon19:44:554
486199122,8cyclictest17303-21snmpd19:20:524
486199120,11cyclictest0-21swapper/420:01:264
4855991210,1cyclictest42-21ksoftirqd/322:35:593
4855991210,1cyclictest0-21swapper/319:10:173
4844991210,1cyclictest33-21ksoftirqd/223:17:582
4844991210,1cyclictest0-21swapper/221:15:212
4844991210,1cyclictest0-21swapper/200:09:412
4844991210,1cyclictest0-21swapper/200:09:412
484499120,1cyclictest1-21systemd21:30:572
484099129,2cyclictest24-21ksoftirqd/123:51:271
484099128,3cyclictest24-21ksoftirqd/100:21:111
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional