You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-12 - 11:16
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Sat Jul 12, 2025 00:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
143852810,0sleep3391rcuc/321:51:183
249582710,0sleep20-21swapper/222:32:102
67352550,0sleep40-21swapper/423:58:534
101232530,0sleep50-21swapper/523:50:385
152632510,0sleep00-21swapper/023:11:160
86222500,0sleep70-21swapper/722:12:127
85212490,1sleep6668-21systemd-logind23:59:036
325124825,9sleep40-21swapper/418:58:154
76092470,0sleep00-21swapper/021:02:460
63702470,1sleep5859-21polkitd21:24:055
310024725,8sleep70-21swapper/718:56:107
72732460,0sleep70-21swapper/721:37:297
72732460,0sleep70-21swapper/721:37:297
202882460,0sleep20-21swapper/200:11:572
202882460,0sleep20-21swapper/200:11:572
178742460,0sleep40-21swapper/421:27:404
267852450,1sleep773-21ksoftirqd/722:21:427
186372450,0sleep20-21swapper/223:06:142
203812440,0sleep50-21swapper/522:05:065
47532430,0sleep10-21swapper/121:55:491
209592430,1sleep665-21ksoftirqd/623:06:266
170382430,2sleep2349899cyclictest21:35:312
170382430,2sleep2349899cyclictest21:35:312
104022430,0sleep2321ktimersoftd/221:03:042
33192420,1sleep23324-21grep22:27:452
267112420,0sleep3391rcuc/322:19:053
267112420,0sleep3391rcuc/322:19:053
5402410,0sleep50-21swapper/521:50:085
312124129,9sleep00-21swapper/018:56:270
260252410,0sleep50-21swapper/523:30:355
175102410,0sleep60-21swapper/623:16:436
159382410,0sleep40-21swapper/400:17:364
150092410,0sleep60-21swapper/621:51:216
148632410,0sleep40-21swapper/423:29:384
133442410,1sleep013345-21grep22:25:510
324024028,9sleep30-21swapper/318:58:063
207172400,0sleep40-21swapper/422:02:324
121662400,0sleep00-21swapper/000:14:110
328923927,8sleep20-21swapper/218:58:452
311823928,8sleep60-21swapper/618:56:256
305223926,9sleep50-21swapper/518:55:365
300923926,9sleep10-21swapper/118:55:001
226612390,0sleep60-21swapper/621:25:186
167642390,1sleep616762-21sshd23:59:366
80762380,0sleep40-21swapper/422:12:094
256472380,2sleep0348399cyclictest00:00:260
3537993011,18cyclictest668-21systemd-logind00:04:187
3537992812,15cyclictest861-21gdbus22:54:507
353799258,11cyclictest0-21swapper/720:40:007
353799257,17cyclictest9549-21sshd22:36:127
353799212,18cyclictest1697-21sshd23:18:067
353799212,13cyclictest13044-21sshd23:50:527
353799201,18cyclictest8505-21sshd00:10:487
353799201,18cyclictest8505-21sshd00:10:487
353799200,19cyclictest13155-21sshd21:13:587
353799200,19cyclictest11726-21sshd23:05:387
353799198,5cyclictest0-21swapper/722:18:327
353799198,5cyclictest0-21swapper/722:18:327
353799198,5cyclictest0-21swapper/720:30:267
353799198,5cyclictest0-21swapper/719:59:147
353799198,0cyclictest0-21swapper/700:21:067
353799198,0cyclictest0-21swapper/700:01:447
353799192,5cyclictest0-21swapper/720:10:547
353799190,18cyclictest2187-21sshd21:02:087
321572190,1sleep09-21ksoftirqd/000:19:240
176282190,0sleep00-21swapper/023:32:390
353799187,5cyclictest0-21swapper/723:48:547
353799187,5cyclictest0-21swapper/723:20:167
353799187,5cyclictest0-21swapper/723:10:387
353799187,5cyclictest0-21swapper/722:03:527
353799187,5cyclictest0-21swapper/719:35:477
353799187,0cyclictest0-21swapper/723:43:307
353799187,0cyclictest0-21swapper/721:42:447
353799180,13cyclictest18864-21bash21:30:207
353799180,13cyclictest14348-21grepconf.sh23:34:567
84002170,1sleep68402-21sshd00:27:076
353799176,5cyclictest0-21swapper/722:39:467
353799176,5cyclictest0-21swapper/722:04:547
353799176,5cyclictest0-21swapper/719:21:517
353799176,5cyclictest0-21swapper/719:13:317
353799176,0cyclictest0-21swapper/720:58:507
353799175,5cyclictest0-21swapper/722:31:187
353799170,16cyclictest99-21kswapd020:34:437
353799170,11cyclictest0-21swapper/719:41:367
352199178,9cyclictest0-21swapper/522:33:385
352199178,9cyclictest0-21swapper/522:03:445
3506991711,5cyclictest29187-21bash21:36:353
3506991711,5cyclictest29187-21bash21:36:353
349099171,15cyclictest29483-21sshd00:03:581
353799166,4cyclictest0-21swapper/721:06:267
353799165,5cyclictest0-21swapper/720:27:067
353799165,5cyclictest0-21swapper/719:00:317
3537991611,1cyclictest859-21polkitd22:52:067
352999160,13cyclictest4736-21sshd23:50:106
352199167,9cyclictest0-21swapper/523:28:405
352199166,10cyclictest0-21swapper/523:45:325
352199165,6cyclictest0-21swapper/522:41:545
352199161,6cyclictest0-21swapper/522:36:425
352199160,15cyclictest23812-21sshd23:17:155
3513991614,1cyclictest668-21systemd-logind21:06:144
3490991612,1cyclictest25-21ksoftirqd/121:54:101
349099160,14cyclictest18605-21bash23:11:331
353799154,5cyclictest0-21swapper/719:57:147
353799152,12cyclictest711-21dbus-daemon23:55:027
353799152,12cyclictest711-21dbus-daemon21:26:527
353799150,9cyclictest0-21swapper/721:15:507
353799150,9cyclictest0-21swapper/719:18:597
3529991512,2cyclictest29362-21bash22:29:466
352199156,9cyclictest0-21swapper/523:07:465
352199156,9cyclictest0-21swapper/520:39:105
3513991510,2cyclictest49-21ksoftirqd/422:41:424
350699150,10cyclictest16361-21bash22:23:343
350699150,1cyclictest21416-21sshd22:37:143
3498991513,2cyclictest33-21ksoftirqd/221:24:272
21802150,1sleep32138-21sshd22:32:593
353799148,5cyclictest30500-21sshd23:30:587
353799143,5cyclictest0-21swapper/720:45:027
353799143,5cyclictest0-21swapper/720:05:107
353799143,0cyclictest0-21swapper/723:24:247
353799141,12cyclictest1378-21sshd21:23:407
3529991411,2cyclictest65-21ksoftirqd/623:01:376
352199146,0cyclictest0-21swapper/520:52:485
352199140,5cyclictest0-21swapper/523:12:465
3498991413,1cyclictest33-21ksoftirqd/223:44:012
3498991412,1cyclictest33-21ksoftirqd/223:01:202
3498991412,1cyclictest33-21ksoftirqd/221:51:502
3498991412,1cyclictest33-21ksoftirqd/200:18:372
3498991411,2cyclictest7136-21sshd21:21:252
3498991410,1cyclictest33-21ksoftirqd/222:13:242
32352140,0sleep40-21swapper/423:04:554
283682140,0sleep60-21swapper/600:19:066
164002140,0sleep20-21swapper/222:44:432
353799138,5cyclictest0-21swapper/721:58:227
353799138,5cyclictest0-21swapper/720:23:047
353799138,5cyclictest0-21swapper/720:14:447
353799138,1cyclictest19363-21sshd22:44:587
353799138,0cyclictest0-21swapper/723:01:507
353799138,0cyclictest0-21swapper/719:28:357
353799137,5cyclictest0-21swapper/722:26:107
353799135,5cyclictest0-21swapper/721:49:447
353799132,5cyclictest22662-21sshd00:25:077
353799132,5cyclictest0-21swapper/719:52:207
353799132,5cyclictest0-21swapper/719:52:207
353799131,6cyclictest0-21swapper/719:47:567
352999133,9cyclictest3710-21bash22:38:286
352999131,11cyclictest6010-21sshd22:41:176
352199138,5cyclictest758-21sshd21:54:485
352199138,5cyclictest0-21swapper/523:54:545
352199138,5cyclictest0-21swapper/523:34:325
352199138,5cyclictest0-21swapper/523:19:365
352199138,5cyclictest0-21swapper/522:51:585
352199138,5cyclictest0-21swapper/522:46:345
352199138,5cyclictest0-21swapper/521:45:025
352199138,5cyclictest0-21swapper/521:30:285
352199138,5cyclictest0-21swapper/521:28:325
352199138,5cyclictest0-21swapper/519:59:285
352199138,5cyclictest0-21swapper/519:54:245
352199138,5cyclictest0-21swapper/500:29:025
352199138,5cyclictest0-21swapper/500:15:085
352199138,5cyclictest0-21swapper/500:09:365
352199138,5cyclictest0-21swapper/500:09:365
352199138,5cyclictest0-21swapper/500:07:025
352199137,5cyclictest0-21swapper/522:19:225
352199132,5cyclictest0-21swapper/522:28:505
349899139,1cyclictest668-21systemd-logind23:48:082
3498991312,1cyclictest33-21ksoftirqd/221:39:212
3498991311,1cyclictest33-21ksoftirqd/222:37:222
3498991311,1cyclictest33-21ksoftirqd/200:07:432
3490991311,1cyclictest25-21ksoftirqd/100:06:421
3490991310,2cyclictest1204-21bash23:23:211
3483991311,1cyclictest9-21ksoftirqd/023:08:070
27142130,0sleep30-21swapper/323:31:213
77932120,0sleep30-21swapper/323:34:263
353799127,5cyclictest0-21swapper/721:44:227
353799127,5cyclictest0-21swapper/719:32:457
353799127,5cyclictest0-21swapper/719:06:157
353799121,5cyclictest31991-21sshd00:15:527
353799120,6cyclictest0-21swapper/720:50:107
352999122,4cyclictest9589-21sshd00:05:036
352999121,5cyclictest1827-21bash22:56:586
352199128,4cyclictest0-21swapper/522:15:045
352199128,4cyclictest0-21swapper/522:15:045
352199128,4cyclictest0-21swapper/521:35:565
352199128,4cyclictest0-21swapper/521:35:565
352199128,4cyclictest0-21swapper/521:14:465
352199128,4cyclictest0-21swapper/520:04:265
352199128,4cyclictest0-21swapper/519:22:195
352199128,4cyclictest0-21swapper/500:22:385
352199127,5cyclictest0-21swapper/523:39:165
352199127,5cyclictest0-21swapper/522:58:005
352199127,5cyclictest0-21swapper/522:09:565
352199127,5cyclictest0-21swapper/521:40:245
352199127,5cyclictest0-21swapper/520:54:305
352199127,5cyclictest0-21swapper/519:39:265
352199127,5cyclictest0-21swapper/500:00:465
352199127,0cyclictest0-21swapper/521:11:065
352199123,6cyclictest10524-21bash20:59:245
352199120,6cyclictest0-21swapper/523:01:465
352199120,5cyclictest0-21swapper/521:04:505
351399122,4cyclictest5518-21sshd22:30:304
351399122,4cyclictest3583-21bash23:23:334
351399122,4cyclictest13921-21sshd22:57:594
351399122,4cyclictest0-21swapper/423:48:304
3506991211,1cyclictest0-21swapper/323:03:093
3498991211,1cyclictest33-21ksoftirqd/222:54:582
3498991211,1cyclictest33-21ksoftirqd/222:14:562
3498991211,1cyclictest33-21ksoftirqd/222:14:562
3498991211,1cyclictest33-21ksoftirqd/221:29:322
3498991211,1cyclictest33-21ksoftirqd/221:06:522
3498991210,2cyclictest33-21ksoftirqd/200:01:122
3498991210,1cyclictest33-21ksoftirqd/223:50:532
349099126,4cyclictest21475-21cp21:09:161
3490991211,1cyclictest25-21ksoftirqd/121:23:021
349099120,11cyclictest0-21swapper/121:48:111
348399128,2cyclictest9-21ksoftirqd/022:06:110
3483991211,1cyclictest0-21swapper/023:03:560
3483991210,1cyclictest9-21ksoftirqd/023:14:300
348399120,2cyclictest8870-21grepconf.sh21:05:270
259842120,0sleep40-21swapper/423:09:284
232122120,0sleep30-21swapper/300:00:133
171542120,0sleep40-21swapper/421:46:154
144822120,0sleep20-21swapper/221:48:452
62452110,0sleep30-21swapper/322:46:363
4442110,0sleep00-21swapper/021:28:570
352999119,1cyclictest1-21systemd21:34:346
352999119,1cyclictest1-21systemd21:34:346
352999112,4cyclictest9225-21sshd22:14:486
352999112,4cyclictest9225-21sshd22:14:486
3529991110,1cyclictest85350irq/25-em1-rx-019:34:086
352999111,4cyclictest32754-21grep21:39:316
352999111,4cyclictest29125-21bash23:44:166
352999111,4cyclictest26345-21sshd23:30:376
352999110,10cyclictest668-21systemd-logind21:21:006
352999110,10cyclictest14875-21systemd-cgroups23:24:276
352999110,1cyclictest1-21systemd21:49:046
352199117,4cyclictest0-21swapper/520:34:265
352199117,4cyclictest0-21swapper/520:26:365
352199117,4cyclictest0-21swapper/520:14:105
352199117,4cyclictest0-21swapper/519:49:285
352199117,4cyclictest0-21swapper/519:49:285
352199117,4cyclictest0-21swapper/519:29:255
352199116,5cyclictest0-21swapper/520:09:305
352199116,0cyclictest0-21swapper/519:19:015
351399112,4cyclictest0-21swapper/421:16:584
351399111,4cyclictest7703-21sshd21:21:284
351399111,4cyclictest668-21systemd-logind21:10:544
351399110,5cyclictest19530-21bash22:44:594
351399110,10cyclictest8812-21sendmail_mailst23:34:294
350699112,4cyclictest16355-21sshd22:02:103
350699110,1cyclictest1-21systemd00:06:193
349899119,2cyclictest33-21ksoftirqd/223:10:012
349899119,1cyclictest33-21ksoftirqd/223:34:222
349899119,1cyclictest33-21ksoftirqd/223:15:482
349899119,1cyclictest33-21ksoftirqd/222:49:272
349899119,1cyclictest33-21ksoftirqd/222:20:392
349899119,1cyclictest33-21ksoftirqd/200:26:442
3498991110,1cyclictest33-21ksoftirqd/223:21:442
3498991110,1cyclictest33-21ksoftirqd/221:54:312
349099119,1cyclictest25-21ksoftirqd/123:32:451
349099119,1cyclictest25-21ksoftirqd/123:04:351
349099119,1cyclictest25-21ksoftirqd/121:25:081
349099118,2cyclictest859-21polkitd23:03:261
349099117,2cyclictest31284-21kworker/1:222:56:541
349099115,3cyclictest32411-21httpd22:10:181
349099114,5cyclictest25-21ksoftirqd/123:44:341
3490991110,1cyclictest25-21ksoftirqd/123:39:311
3490991110,1cyclictest25-21ksoftirqd/123:14:231
3490991110,1cyclictest25-21ksoftirqd/121:59:181
3490991110,1cyclictest25-21ksoftirqd/100:23:511
348399119,1cyclictest9-21ksoftirqd/022:30:160
348399110,2cyclictest1116-21sshd21:52:560
348399110,10cyclictest7858-21sshd23:42:290
301282110,0sleep60-21swapper/622:48:406
352999109,1cyclictest0-21swapper/600:11:446
352999109,1cyclictest0-21swapper/600:11:446
352999108,1cyclictest65-21ksoftirqd/623:34:206
352999108,1cyclictest65-21ksoftirqd/600:22:266
352999104,5cyclictest3275-21sshd21:31:466
352999103,6cyclictest10496-21sshd21:05:356
352999103,4cyclictest9285-21timerandwakeup22:09:316
352999102,7cyclictest732-21NetworkManager22:21:296
352999102,3cyclictest20539-21sshd22:50:266
352999102,3cyclictest0-21swapper/620:59:366
352999100,4cyclictest11786-21sshd21:11:046
352999100,4cyclictest11053-21rm23:21:276
352999100,3cyclictest0-21swapper/623:13:156
352199103,5cyclictest0-21swapper/518:59:275
352199100,0cyclictest0-21swapper/520:21:145
351399109,1cyclictest0-21swapper/421:54:314
351399108,1cyclictest49-21ksoftirqd/423:40:194
351399105,5cyclictest0-21swapper/421:39:184
351399103,6cyclictest29920-21bash00:00:494
351399103,6cyclictest27116-21sshd21:52:244
351399103,1cyclictest1878-21grep23:49:554
351399101,4cyclictest0-21swapper/422:49:354
350699102,3cyclictest5960-21sshd22:49:203
350699101,4cyclictest0-21swapper/323:07:103
350699101,4cyclictest0-21swapper/321:12:083
350699100,1cyclictest32243-21grepconf.sh22:27:283
349899109,1cyclictest33-21ksoftirqd/223:26:072
349899109,1cyclictest33-21ksoftirqd/222:07:292
349899109,1cyclictest33-21ksoftirqd/222:02:472
349899109,1cyclictest33-21ksoftirqd/221:14:322
349899109,1cyclictest33-21ksoftirqd/200:20:512
349899108,2cyclictest33-21ksoftirqd/221:14:072
349899108,1cyclictest33-21ksoftirqd/223:57:332
349899108,1cyclictest33-21ksoftirqd/223:29:552
349099109,1cyclictest25-21ksoftirqd/123:51:021
349099109,1cyclictest25-21ksoftirqd/121:33:101
349099108,1cyclictest25-21ksoftirqd/122:50:351
349099108,1cyclictest25-21ksoftirqd/122:45:451
349099100,10cyclictest0-21swapper/122:35:101
348399109,1cyclictest9-21ksoftirqd/023:19:120
348399109,1cyclictest9-21ksoftirqd/022:42:460
348399109,1cyclictest9-21ksoftirqd/022:03:480
348399108,1cyclictest9-21ksoftirqd/023:36:100
348399108,1cyclictest9-21ksoftirqd/022:49:350
348399108,1cyclictest9-21ksoftirqd/022:36:030
348399103,4cyclictest23482-21sshd23:54:270
348399100,10cyclictest0-21swapper/021:11:360
348399100,1cyclictest0-21swapper/023:27:020
312452100,0sleep40-21swapper/421:01:464
35299997,1cyclictest65-21ksoftirqd/621:54:196
35299991,4cyclictest3441-21sshd23:39:256
35299990,9cyclictest0-21swapper/620:19:126
35299990,4cyclictest26074-21grep22:24:226
35299990,4cyclictest17799-21bash21:16:596
35299990,4cyclictest13690-21sshd22:04:326
35299990,4cyclictest0-21swapper/622:02:516
35219998,1cyclictest0-21swapper/519:09:115
35139993,6cyclictest21035-21sshd00:06:124
35139992,6cyclictest19801-21sshd21:38:334
35139992,6cyclictest19801-21sshd21:38:334
35139991,4cyclictest7057-21sshd23:26:244
35139991,4cyclictest261202sleep400:12:314
35139991,4cyclictest261202sleep400:12:314
35139991,3cyclictest32000-21sshd22:16:464
35139991,3cyclictest32000-21sshd22:16:464
35139990,9cyclictest0-21swapper/423:00:354
35139990,9cyclictest0-21swapper/422:19:174
35139990,4cyclictest5823-21sshd23:15:444
35069998,1cyclictest0-21swapper/323:25:103
35069997,1cyclictest41-21ksoftirqd/322:39:193
35069996,3cyclictest21215-21kworker/3:023:16:563
35069992,4cyclictest18814-21systemd-cgroups21:27:453
35069991,4cyclictest13961-21grep21:19:223
35069991,2cyclictest0-21swapper/300:09:243
35069991,2cyclictest0-21swapper/300:09:243
35069990,9cyclictest0-21swapper/300:22:143
35069990,4cyclictest20362-21bash23:54:173
35069990,1cyclictest4256-21grep21:29:143
34989998,1cyclictest33-21ksoftirqd/222:39:402
34909998,1cyclictest30192-21kworker/1:000:11:281
34909998,1cyclictest30192-21kworker/1:000:11:281
34909998,1cyclictest25-21ksoftirqd/122:22:091
34909998,1cyclictest25-21ksoftirqd/122:15:341
34909998,1cyclictest25-21ksoftirqd/122:15:341
34909998,1cyclictest25-21ksoftirqd/121:43:361
34909998,1cyclictest0-21swapper/123:54:281
34909997,1cyclictest30097-21kworker/1:300:18:041
34909997,1cyclictest25-21ksoftirqd/122:31:571
34909997,1cyclictest25-21ksoftirqd/121:16:241
34909993,4cyclictest17472-21grepconf.sh22:26:121
34909992,4cyclictest711-21dbus-daemon22:40:531
34839999,0cyclictest0-21swapper/021:41:150
34839998,1cyclictest9-21ksoftirqd/023:53:290
34839998,1cyclictest9-21ksoftirqd/022:54:360
34839998,1cyclictest9-21ksoftirqd/022:46:530
34839998,1cyclictest9-21ksoftirqd/022:20:330
34839997,1cyclictest9-21ksoftirqd/021:44:210
34839997,1cyclictest9-21ksoftirqd/021:35:560
34839997,1cyclictest9-21ksoftirqd/021:35:560
34839997,1cyclictest9-21ksoftirqd/021:15:580
34839997,1cyclictest9-21ksoftirqd/000:09:510
34839997,1cyclictest9-21ksoftirqd/000:09:510
34839993,3cyclictest29236-21cp23:44:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional