You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-02-05 - 09:03

x86 Intel Core i3-2100T @2500 MHz, Linux 4.14.28-rt23 (Profile)

Latency plot of system in rack #0, slot #0
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack0slot0.osadl.org (updated Sun Feb 05, 2023 00:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1173121930,3sleep5621399cyclictest23:59:385
898321870,5sleep1618799cyclictest23:58:211
2526421870,0sleep7622699cyclictest23:29:577
3257821760,4sleep7622699cyclictest21:23:467
52521620,4sleep7622699cyclictest00:36:197
2813121600,2sleep4620599cyclictest23:43:564
2163921600,1sleep61013-21runrttasks00:09:176
2163921600,1sleep61013-21runrttasks00:09:166
1940021510,1sleep519398-21grep00:31:125
961121430,4sleep3619899cyclictest23:03:543
3193221400,4sleep0618199cyclictest00:31:400
1248721340,4sleep4620599cyclictest23:56:084
801421290,1sleep6671rcuc/621:48:066
778121290,0sleep30-21swapper/323:10:443
778121290,0sleep30-21swapper/323:10:443
2253121290,0sleep1221rcuc/122:16:061
317621270,1sleep03089-21sshd22:45:150
664121190,1sleep46640-21sshd21:15:574
664121190,1sleep46640-21sshd21:15:574
3116721190,1sleep531158-21bash23:09:185
2775621120,0sleep50-21swapper/521:36:105
2732921100,1sleep233-21ksoftirqd/200:01:192
1293021070,0sleep30-21swapper/323:57:183
2562421050,1sleep51131-21nfsd00:24:325
1595021050,1sleep015918-21sshd23:44:420
454321020,1sleep44489-21sshd23:10:374
454321020,1sleep44489-21sshd23:10:374
35882990,0sleep10-21swapper/123:15:161
35882990,0sleep10-21swapper/123:15:151
78172980,1sleep70-21swapper/723:13:057
78172980,1sleep70-21swapper/723:13:057
62342970,0sleep60-21swapper/622:55:406
221962970,4sleep6621999cyclictest23:13:376
221962970,4sleep6621999cyclictest23:13:366
267002960,4sleep3619899cyclictest23:39:153
169882960,0sleep70-21swapper/700:18:247
282282950,0sleep60-21swapper/623:32:206
62372940,1sleep669-21ksoftirqd/600:35:226
256502940,0sleep70-21swapper/721:29:157
38962930,1sleep20-21swapper/223:38:252
262572930,1sleep626254-21sshd00:02:276
252602920,4sleep0618199cyclictest21:33:490
16622910,1sleep00-21swapper/022:28:010
160222900,1sleep30-21swapper/322:31:573
60682890,0sleep40-21swapper/423:25:444
281532890,2sleep028155-21sshd21:12:080
20452890,0sleep40-21swapper/421:20:224
240402850,0sleep70-21swapper/723:31:007
33262830,1sleep50-21swapper/500:36:265
320712830,1sleep2321ktimersoftd/200:28:132
202342830,1sleep40-21swapper/400:19:414
167422830,1sleep616744-21sshd22:50:206
100702830,0sleep7761rcuc/700:04:137
184002820,0sleep60-21swapper/623:06:306
176402820,1sleep417639-21systemd-cgroups21:14:064
310262810,1sleep131004-21sshd22:57:431
29512810,1sleep51-21systemd22:50:585
253022810,1sleep50-21swapper/523:10:155
253022810,1sleep50-21swapper/523:10:145
233402810,0sleep00-21swapper/021:26:510
156492810,0sleep30-21swapper/321:35:443
125902810,1sleep012495-21sshd22:22:390
70722800,0sleep60-21swapper/621:20:326
42732800,1sleep10-21swapper/121:46:481
323072800,0sleep30-21swapper/322:17:373
282372800,0sleep70-21swapper/721:44:147
248072800,1sleep3401rcuc/321:26:543
206932800,1sleep220697-21kworker/u16:222:11:262
206932800,1sleep220697-21kworker/u16:222:11:262
172502800,1sleep6751-21NetworkManager21:12:546
114612800,1sleep5581rcuc/500:17:015
172712790,0sleep30-21swapper/300:24:133
147782790,0sleep30-21swapper/300:31:023
142672790,0sleep60-21swapper/621:34:356
98242780,1sleep30-21swapper/322:08:473
75142780,0sleep10-21swapper/122:07:331
220812780,1sleep322060-21sshd21:50:513
158692780,0sleep50-21swapper/521:27:445
99982760,1sleep1221rcuc/123:37:281
202702760,1sleep0121rcuc/021:19:540
202702760,1sleep0121rcuc/021:19:540
315662750,0sleep10-21swapper/122:45:081
100262750,1sleep410018-21bash22:51:134
281332740,0sleep50-21swapper/521:57:595
257362740,0sleep00-21swapper/021:57:540
226012740,0sleep30-21swapper/321:44:023
130942740,0sleep3401rcuc/300:26:223
38142730,0sleep10-21swapper/122:31:301
309062730,0sleep20-21swapper/222:22:102
302002730,0sleep20-21swapper/221:52:192
295272730,3sleep5621399cyclictest22:41:365
272752730,0sleep30-21swapper/323:30:003
238712730,2sleep20-21swapper/222:03:342
213002730,0sleep30-21swapper/322:44:463
188162730,0sleep60-21swapper/621:42:456
297752720,5sleep278850irq/25-eno1-rx-23:09:152
239532720,0sleep10-21swapper/122:35:391
169972720,1sleep516999-21sshd22:31:585
56872710,0sleep30-21swapper/300:40:003
302562710,0sleep3401rcuc/322:24:263
269362710,3sleep3619899cyclictest22:50:403
275682700,1sleep07-21ksoftirqd/023:01:030
91812690,0sleep10-21swapper/119:15:121
308042690,1sleep50-21swapper/521:05:165
254512690,0sleep30-21swapper/321:32:403
234892670,0sleep623481-21bash21:50:546
102002660,5sleep7622699cyclictest00:07:417
102002660,5sleep7622699cyclictest00:07:417
289922650,1sleep628990-21sshd21:29:226
19682650,0sleep3401rcuc/322:14:173
19682650,0sleep3401rcuc/322:14:163
150672650,1sleep315068-21sshd00:06:423
150672650,1sleep315068-21sshd00:06:413
139852650,1sleep513978-21sshd23:26:015
152622630,1sleep7761rcuc/723:55:067
285752620,1sleep328574-21sshd23:17:183
285752620,1sleep328574-21sshd23:17:173
82272610,1sleep58220-21sshd00:02:585
267342610,1sleep426733-21grepconf.sh23:53:104
178602610,1sleep669-21ksoftirqd/623:15:456
178602610,1sleep669-21ksoftirqd/623:15:446
117712610,3sleep7622699cyclictest22:08:517
594425949,6sleep30-21swapper/319:09:543
584725847,7sleep70-21swapper/719:08:387
596025747,6sleep10-21swapper/119:10:001
183342560,0sleep70-21swapper/723:15:467
183342560,0sleep70-21swapper/723:15:457
123342560,0sleep60-21swapper/622:06:336
589025544,7sleep50-21swapper/519:09:165
298892550,0sleep00-21swapper/023:35:510
260542540,5sleep4620599cyclictest23:20:404
202082540,4sleep6621999cyclictest23:52:576
94062520,2sleep59405-21sshd23:42:055
60852500,0sleep40-21swapper/400:12:124
60852500,0sleep40-21swapper/400:12:114
592324939,6sleep00-21swapper/019:09:350
582624636,6sleep60-21swapper/619:08:296
265222460,0sleep30-21swapper/300:15:163
212412460,1sleep121238-21sshd23:30:541
151432460,2sleep5684-21dbus-daemon23:34:125
148342460,1sleep7684-21dbus-daemon00:12:317
148342460,1sleep7684-21dbus-daemon00:12:307
209542450,1sleep420951-21sshd23:47:104
35382440,0sleep40-21swapper/423:33:464
250392440,0sleep10-21swapper/100:07:031
250392440,0sleep10-21swapper/100:07:031
194892440,0sleep30-21swapper/323:32:013
16232440,0sleep40-21swapper/423:02:274
113442440,0sleep30-21swapper/323:52:373
109732440,0sleep00-21swapper/022:55:500
100992440,0sleep00-21swapper/000:06:310
100992440,0sleep00-21swapper/000:06:300
88022430,0sleep20-21swapper/223:43:142
578624231,7sleep20-21swapper/219:07:532
311652420,0sleep60-21swapper/623:41:436
231032410,1sleep3683-21systemd-logind00:04:403
67212400,0sleep10-21swapper/123:01:291
559024028,7sleep40-21swapper/419:05:294
265302380,1sleep526525-21sshd23:20:405
63512350,2sleep078950irq/26-eno1-tx-23:10:420
63512350,2sleep078950irq/26-eno1-tx-23:10:410
128572340,0sleep212856-21bash23:13:172
128572340,0sleep212856-21bash23:13:162
258312320,1sleep6671rcuc/600:16:226
9792290,0sleep20-21swapper/223:22:062
38582280,1sleep73851-21sshd00:33:007
4722270,0sleep7761rcuc/723:23:157
620599261,22cyclictest0-21swapper/419:32:424
33302260,1sleep03322-21sshd22:07:250
149462260,0sleep70-21swapper/722:03:147
140042260,1sleep5591ktimersoftd/521:31:055
621999252,18cyclictest0-21swapper/620:01:406
621399252,22cyclictest0-21swapper/520:33:235
621399252,22cyclictest0-21swapper/520:33:235
289762250,1sleep00-21swapper/000:22:180
65852240,1sleep46578-21sshd00:36:334
622699240,21cyclictest0-21swapper/720:59:447
266552240,1sleep20-21swapper/222:08:142
1-21240,1systemd23405-21sshd22:11:331
1-21240,1systemd23405-21sshd22:11:321
621999235,15cyclictest21240-21sshd21:18:476
621999235,15cyclictest21240-21sshd21:18:476
621999233,1cyclictest0-21swapper/600:31:316
621399233,1cyclictest0-21swapper/522:03:105
621399231,20cyclictest25606-21cp23:50:475
619299230,7cyclictest0-21swapper/223:04:002
40112230,3sleep1618799cyclictest23:11:461
40112230,3sleep1618799cyclictest23:11:461
302442230,0sleep10-21swapper/121:15:381
302442230,0sleep10-21swapper/121:15:381
291432230,1sleep40-21swapper/421:43:074
156572230,1sleep10-21swapper/123:43:301
12692230,4sleep5621399cyclictest00:06:115
12692230,4sleep5621399cyclictest00:06:105
73332220,0sleep00-21swapper/022:43:070
622699220,20cyclictest768-21gdbus22:29:237
622699220,20cyclictest683-21systemd-logind23:49:157
622699220,1cyclictest24927-21grep23:07:547
621999222,18cyclictest0-21swapper/619:54:466
621999221,19cyclictest11195-21sshd23:00:286
621999220,4cyclictest29857-21sshd22:33:356
621399222,1cyclictest0-21swapper/521:49:005
621399220,4cyclictest18255-21grep00:26:325
621399220,20cyclictest31483-21bash23:35:545
621399220,20cyclictest30949-21sshd23:48:425
621399220,19cyclictest0-21swapper/519:57:065
620599222,1cyclictest0-21swapper/400:24:564
620599221,19cyclictest7576-21sshd00:09:574
620599221,19cyclictest7576-21sshd00:09:564
620599220,20cyclictest5749-21sshd22:48:484
620599220,20cyclictest1-21systemd21:45:544
620599220,20cyclictest1014-21bash21:54:444
620599220,19cyclictest0-21swapper/420:05:104
619299223,18cyclictest0-21swapper/220:52:022
618199220,20cyclictest653-21avahi-daemon22:52:080
618199220,20cyclictest0-21swapper/019:18:520
618199220,19cyclictest0-21swapper/020:25:310
298142220,1sleep40-21swapper/423:38:114
622699210,19cyclictest8811-21sshd22:11:027
622699210,19cyclictest8811-21sshd22:11:017
622699210,19cyclictest653-21avahi-daemon21:35:237
622699210,19cyclictest28475-21sshd22:55:197
622699210,19cyclictest25699-21sshd22:41:277
622699210,19cyclictest21545-21sshd21:30:147
622699210,19cyclictest1-21systemd22:24:117
621999212,15cyclictest0-21swapper/620:24:396
621999211,4cyclictest0-21swapper/600:14:396
621999211,4cyclictest0-21swapper/600:14:386
621999210,19cyclictest9128-21sshd22:39:456
621999210,19cyclictest7948-21sshd23:59:296
621999210,19cyclictest653-21avahi-daemon21:59:216
621999210,19cyclictest30355-21sshd00:22:216
621999210,19cyclictest25445-21bash22:10:276
621999210,19cyclictest25445-21bash22:10:276
621999210,19cyclictest23776-21sshd22:21:536
621999210,19cyclictest21136-21bash22:17:136
621999210,19cyclictest17603-21sshd23:26:096
6213992117,2cyclictest7664-21sshd22:59:135
621399210,1cyclictest0-21swapper/521:10:155
621399210,19cyclictest768-21gdbus23:03:355
621399210,19cyclictest768-21gdbus22:13:265
621399210,19cyclictest768-21gdbus22:13:265
621399210,19cyclictest683-21systemd-logind22:35:175
621399210,19cyclictest683-21systemd-logind00:11:525
621399210,19cyclictest683-21systemd-logind00:11:515
621399210,19cyclictest653-21avahi-daemon21:22:225
620599211,18cyclictest27296-21sshd00:29:124
620599211,18cyclictest23364-21sshd22:08:074
620599210,19cyclictest768-21gdbus23:06:124
620599210,19cyclictest768-21gdbus22:41:324
620599210,19cyclictest768-21gdbus22:02:164
620599210,19cyclictest653-21avahi-daemon22:19:364
620599210,19cyclictest653-21avahi-daemon21:32:114
620599210,19cyclictest653-21avahi-daemon21:26:284
620599210,19cyclictest653-21avahi-daemon00:02:454
620599210,19cyclictest29380-21grep22:23:154
620599210,19cyclictest24666-21sshd21:38:234
619899211,18cyclictest18235-21sshd21:20:573
619899210,19cyclictest683-21systemd-logind23:20:433
619899210,19cyclictest27023-21sshd22:26:353
619899210,19cyclictest0-21swapper/319:12:053
619299210,19cyclictest768-21gdbus00:22:062
619299210,19cyclictest683-21systemd-logind21:21:382
619299210,19cyclictest653-21avahi-daemon22:25:442
619299210,19cyclictest20397-21sshd21:16:282
619299210,19cyclictest20397-21sshd21:16:272
619299210,19cyclictest0-21swapper/220:01:462
618799211,4cyclictest5892-21sshd00:17:591
618799211,18cyclictest27117-21sshd22:00:151
618799210,19cyclictest9313-21sshd23:08:301
618799210,19cyclictest653-21avahi-daemon21:10:221
618799210,19cyclictest29276-21bash22:20:561
618799210,19cyclictest24016-21bash21:20:021
618799210,19cyclictest21740-21sshd23:50:401
618199211,1cyclictest14300-21bash22:39:560
618199210,19cyclictest653-21avahi-daemon23:48:440
618199210,19cyclictest653-21avahi-daemon23:16:350
618199210,19cyclictest653-21avahi-daemon23:16:350
618199210,19cyclictest0-21swapper/019:22:000
618199210,18cyclictest0-21swapper/021:02:290
33142210,1sleep669-21ksoftirqd/600:26:006
318842210,1sleep231876-21bash22:33:392
244262210,1sleep224359-21sshd22:41:252
229972210,1sleep70-21swapper/721:16:327
229972210,1sleep70-21swapper/721:16:327
224392210,1sleep1683-21systemd-logind00:32:291
201482210,2sleep30-21swapper/321:47:243
622699201,18cyclictest0-21swapper/721:02:087
622699201,17cyclictest1-21systemd22:16:087
622699200,19cyclictest653-21avahi-daemon23:01:497
622699200,19cyclictest0-21swapper/720:38:087
622699200,18cyclictest9886-21sshd00:25:077
622699200,18cyclictest768-21gdbus22:45:237
622699200,18cyclictest6834-21bash21:14:517
622699200,18cyclictest653-21avahi-daemon21:57:397
622699200,18cyclictest0-21swapper/719:37:217
621999200,19cyclictest0-21swapper/621:06:326
621999200,19cyclictest0-21swapper/620:42:206
621999200,18cyclictest768-21gdbus22:04:356
621999200,18cyclictest683-21systemd-logind23:21:246
621999200,18cyclictest653-21avahi-daemon22:42:346
621999200,18cyclictest1-21systemd23:47:206
621999200,17cyclictest0-21swapper/620:47:506
621999200,17cyclictest0-21swapper/619:13:296
621399201,17cyclictest27281-21sshd21:44:135
621399200,19cyclictest768-21gdbus22:22:465
621399200,19cyclictest683-21systemd-logind22:20:005
621399200,19cyclictest0-21swapper/520:35:195
621399200,19cyclictest0-21swapper/520:27:375
621399200,19cyclictest0-21swapper/520:08:525
621399200,19cyclictest0-21swapper/519:51:395
621399200,19cyclictest0-21swapper/519:25:185
621399200,18cyclictest768-21gdbus21:16:205
621399200,18cyclictest768-21gdbus21:16:205
620599201,17cyclictest18411-21grep00:31:104
620599200,4cyclictest0-21swapper/422:37:294
620599200,4cyclictest0-21swapper/422:34:354
620599200,19cyclictest0-21swapper/420:17:314
620599200,18cyclictest768-21gdbus22:28:084
620599200,18cyclictest683-21systemd-logind21:58:374
620599200,18cyclictest13463-21sshd23:19:054
620599200,18cyclictest13463-21sshd23:19:054
620599200,17cyclictest0-21swapper/420:45:414
620599200,17cyclictest0-21swapper/420:30:384
620599200,17cyclictest0-21swapper/420:30:384
619899200,4cyclictest0-21swapper/323:40:523
619899200,1cyclictest5272-21sshd22:49:563
619899200,19cyclictest0-21swapper/320:52:593
619899200,19cyclictest0-21swapper/319:39:243
619899200,18cyclictest683-21systemd-logind00:13:303
619899200,18cyclictest683-21systemd-logind00:13:303
619899200,18cyclictest28159-21sshd21:55:393
619299202,2cyclictest0-21swapper/221:31:122
619299200,2cyclictest0-21swapper/223:32:312
619299200,18cyclictest9959-21grepconf.sh22:37:282
619299200,18cyclictest768-21gdbus21:36:132
619299200,18cyclictest768-21gdbus21:12:332
619299200,18cyclictest768-21gdbus00:14:242
619299200,18cyclictest768-21gdbus00:14:242
619299200,18cyclictest653-21avahi-daemon21:25:402
619299200,18cyclictest653-21avahi-daemon00:34:172
619299200,18cyclictest5498-21sshd00:37:402
619299200,18cyclictest463-21systemd-journal22:47:372
619299200,18cyclictest32640-21bash23:17:262
619299200,18cyclictest32640-21bash23:17:262
619299200,18cyclictest29533-21bash23:47:282
619299200,18cyclictest25926-21grepconf.sh21:41:512
619299200,18cyclictest13201-21sshd23:55:012
619299200,18cyclictest1-21systemd21:49:172
618799209,2cyclictest24-21ksoftirqd/123:25:451
618799200,19cyclictest0-21swapper/120:59:381
618799200,18cyclictest7106-21bash22:29:211
618799200,18cyclictest29558-21grepconf.sh22:42:451
618799200,18cyclictest2188-21bash21:32:591
618799200,18cyclictest1-21systemd00:24:511
618799200,18cyclictest10892-21sshd00:00:431
618199201,18cyclictest32276-21sshd23:33:390
618199200,19cyclictest0-21swapper/020:05:520
618199200,18cyclictest768-21gdbus00:00:400
618199200,18cyclictest683-21systemd-logind21:24:520
618199200,18cyclictest653-21avahi-daemon23:56:580
618199200,18cyclictest20819-21sshd00:35:530
269822200,0sleep70-21swapper/723:40:237
197322200,0sleep70-21swapper/721:48:317
187082200,1sleep1751-21NetworkManager23:45:541
15052200,1sleep5581rcuc/522:25:405
123152200,0sleep00-21swapper/022:32:570
622699190,17cyclictest653-21avahi-daemon23:38:327
622699190,17cyclictest1-21systemd22:38:207
621999190,17cyclictest768-21gdbus22:27:346
621999190,17cyclictest683-21systemd-logind21:36:516
621999190,17cyclictest4572-21systemd-cgroups22:45:186
621399192,15cyclictest0-21swapper/519:17:445
621399190,18cyclictest0-21swapper/520:44:385
621399190,18cyclictest0-21swapper/520:23:195
621399190,18cyclictest0-21swapper/519:49:065
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional