You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-28 - 06:35
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Fri Nov 28, 2025 00:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
234142810,0sleep50-21swapper/522:23:035
27462700,2sleep62172499cyclictest00:09:486
158442670,1sleep449-21ksoftirqd/422:17:344
203932530,1sleep720386-21sshd21:42:457
2136525129,8sleep10-21swapper/118:51:591
32032490,0sleep50-21swapper/523:04:525
246842480,1sleep341-21ksoftirqd/323:16:513
2135024826,8sleep40-21swapper/418:51:464
228182470,0sleep10-21swapper/121:16:281
33752460,0sleep30-21swapper/300:00:163
2136224432,9sleep70-21swapper/718:51:577
325652430,0sleep40-21swapper/423:45:354
2150424129,8sleep60-21swapper/618:53:576
2137824128,9sleep50-21swapper/518:52:125
187262410,0sleep00-21swapper/000:12:300
128872410,0sleep20-21swapper/223:15:062
83532400,0sleep20-21swapper/222:07:392
249512400,0sleep00-21swapper/021:52:150
2131024027,9sleep30-21swapper/318:51:213
106232400,0sleep40-21swapper/420:56:374
215362390,0sleep50-21swapper/522:36:095
2124023928,8sleep00-21swapper/018:50:220
2120523928,8sleep20-21swapper/218:49:522
152052390,0sleep60-21swapper/622:13:036
317562380,0sleep00-21swapper/023:26:570
199352380,0sleep70-21swapper/700:02:527
127682380,0sleep20-21swapper/223:24:192
224782370,0sleep40-21swapper/422:58:504
184012370,0sleep20-21swapper/222:13:312
2171799269,0cyclictest0-21swapper/500:13:175
21717992514,5cyclictest0-21swapper/500:18:075
2173399237,15cyclictest14737-21sshd22:44:197
21733992317,5cyclictest711-21dbus-daemon23:17:477
2173399225,0cyclictest0-21swapper/719:44:357
2173399210,9cyclictest0-21swapper/720:49:517
2171799217,0cyclictest0-21swapper/521:49:115
21717992110,5cyclictest0-21swapper/519:38:015
2173399208,0cyclictest0-21swapper/721:24:577
2173399200,19cyclictest25951-21kworker/7:100:04:297
43362190,0sleep10-21swapper/123:41:321
2173399198,5cyclictest0-21swapper/723:28:097
2173399198,5cyclictest0-21swapper/721:56:417
2173399198,5cyclictest0-21swapper/721:56:417
2173399198,5cyclictest0-21swapper/721:15:317
2173399198,5cyclictest0-21swapper/720:10:317
2173399198,5cyclictest0-21swapper/720:07:557
2173399198,5cyclictest0-21swapper/719:50:317
2173399198,5cyclictest0-21swapper/719:01:537
2173399198,0cyclictest0-21swapper/723:13:237
2173399198,0cyclictest0-21swapper/721:52:197
2173399198,0cyclictest0-21swapper/719:37:117
2173399198,0cyclictest0-21swapper/719:11:077
2173399197,11cyclictest3733-21bash22:15:497
2173399197,0cyclictest0-21swapper/719:20:177
21733991914,0cyclictest0-21swapper/718:56:397
21733991914,0cyclictest0-21swapper/718:56:397
2173399191,0cyclictest0-21swapper/720:25:457
2171799198,5cyclictest0-21swapper/522:10:175
2171799198,5cyclictest0-21swapper/521:40:315
2171799198,5cyclictest0-21swapper/521:10:415
2171799198,5cyclictest0-21swapper/520:14:355
2171799198,5cyclictest0-21swapper/520:06:575
2171799198,5cyclictest0-21swapper/520:01:175
2171799198,5cyclictest0-21swapper/519:57:595
2171799198,5cyclictest0-21swapper/519:49:555
2171799198,5cyclictest0-21swapper/519:10:535
2171799198,11cyclictest0-21swapper/519:44:175
2171799198,0cyclictest0-21swapper/522:17:275
2171799198,0cyclictest0-21swapper/521:16:195
2171799198,0cyclictest0-21swapper/521:05:395
2173399187,5cyclictest0-21swapper/723:58:557
2173399187,5cyclictest0-21swapper/723:50:517
2173399187,5cyclictest0-21swapper/723:04:537
2173399187,5cyclictest0-21swapper/722:57:217
2173399187,5cyclictest0-21swapper/719:28:197
2173399187,0cyclictest0-21swapper/720:34:577
2173399187,0cyclictest0-21swapper/720:31:137
2173399187,0cyclictest0-21swapper/719:59:337
2173399187,0cyclictest0-21swapper/719:43:597
2173399187,0cyclictest0-21swapper/719:33:097
2173399187,0cyclictest0-21swapper/719:14:197
2173399187,0cyclictest0-21swapper/700:22:257
2173399180,6cyclictest1-21systemd22:49:397
2173399180,6cyclictest0-21swapper/720:47:197
2173399180,0cyclictest0-21swapper/719:56:397
21724991810,1cyclictest65-21ksoftirqd/621:10:326
2171799187,5cyclictest0-21swapper/522:57:075
2171799187,5cyclictest0-21swapper/522:39:375
2171799187,5cyclictest0-21swapper/522:30:315
2171799187,5cyclictest0-21swapper/521:35:375
2171799187,5cyclictest0-21swapper/520:24:495
2171799187,5cyclictest0-21swapper/519:39:475
2171799187,0cyclictest0-21swapper/523:50:495
2171799187,0cyclictest0-21swapper/523:09:375
2171799187,0cyclictest0-21swapper/521:48:495
2171799187,0cyclictest0-21swapper/520:09:355
2171799187,0cyclictest0-21swapper/500:20:375
21687991816,2cyclictest30429-21kworker/1:122:14:511
2173399179,0cyclictest0-21swapper/721:21:597
2173399176,5cyclictest0-21swapper/723:47:357
2173399176,5cyclictest0-21swapper/723:43:417
2173399176,5cyclictest0-21swapper/723:37:217
2173399176,5cyclictest0-21swapper/722:32:377
2173399176,5cyclictest0-21swapper/722:27:137
2173399176,5cyclictest0-21swapper/720:23:277
2173399176,5cyclictest0-21swapper/720:23:277
2173399176,0cyclictest0-21swapper/722:43:057
2173399170,5cyclictest0-21swapper/722:04:477
2173399170,12cyclictest12367-21sshd22:12:397
2172499173,3cyclictest65-21ksoftirqd/621:48:286
2171799177,0cyclictest0-21swapper/522:04:195
2171799176,5cyclictest0-21swapper/523:34:235
2171799176,5cyclictest0-21swapper/520:41:215
2171799176,5cyclictest0-21swapper/520:21:175
2171799176,5cyclictest0-21swapper/520:21:175
2171799176,0cyclictest0-21swapper/520:29:135
36492160,2sleep42170999cyclictest00:05:064
2173399165,0cyclictest0-21swapper/720:17:037
2173399165,0cyclictest0-21swapper/719:04:177
2173399160,4cyclictest0-21swapper/722:59:117
2172499166,1cyclictest65-21ksoftirqd/623:33:276
21724991612,1cyclictest65-21ksoftirqd/621:19:236
2172499160,3cyclictest0-21swapper/600:18:316
2172499160,1cyclictest0-21swapper/621:42:516
2171799165,5cyclictest0-21swapper/523:14:355
2171799165,5cyclictest0-21swapper/519:04:035
2171799165,5cyclictest0-21swapper/500:03:355
2171799165,0cyclictest0-21swapper/523:20:035
2171799165,0cyclictest0-21swapper/520:51:435
21717991611,0cyclictest0-21swapper/519:32:375
21709991612,1cyclictest4923-21bash21:49:284
2170999160,15cyclictest30550-21sshd22:01:474
21687991614,1cyclictest25-21ksoftirqd/121:49:101
21675991614,2cyclictest9-21ksoftirqd/022:15:460
21675991614,1cyclictest9-21ksoftirqd/023:44:280
92032150,0sleep20-21swapper/223:56:232
2173399154,0cyclictest0-21swapper/722:20:157
2173399154,0cyclictest0-21swapper/720:41:197
2173399153,0cyclictest0-21swapper/721:44:137
2173399152,12cyclictest23898-21sshd00:18:297
2173399152,12cyclictest1-21systemd21:01:317
21733991510,1cyclictest21613-21bash00:12:597
2172499156,3cyclictest65-21ksoftirqd/600:03:316
2172499151,1cyclictest65-21ksoftirqd/622:16:576
2172499151,1cyclictest641ktimersoftd/622:54:286
2172499151,1cyclictest0-21swapper/623:16:596
2171799154,5cyclictest0-21swapper/521:28:255
2171799154,5cyclictest0-21swapper/518:58:315
2171799154,5cyclictest0-21swapper/518:58:315
2171799154,0cyclictest28105-21nfsd20:47:455
2171799154,0cyclictest0-21swapper/522:45:315
2171799154,0cyclictest0-21swapper/519:18:255
2171799150,9cyclictest0-21swapper/523:47:075
2171799150,9cyclictest0-21swapper/522:28:035
2171799150,0cyclictest0-21swapper/521:59:255
21701991513,1cyclictest41-21ksoftirqd/323:34:333
2168799154,1cyclictest25-21ksoftirqd/122:14:101
21687991514,1cyclictest25-21ksoftirqd/123:02:411
21687991514,1cyclictest25-21ksoftirqd/121:49:231
21687991513,1cyclictest25-21ksoftirqd/121:57:111
21687991513,1cyclictest25-21ksoftirqd/121:57:111
21687991511,4cyclictest25-21ksoftirqd/123:12:371
21687991510,3cyclictest25-21ksoftirqd/121:13:331
21675991514,1cyclictest9-21ksoftirqd/023:40:310
21675991513,1cyclictest9-21ksoftirqd/021:09:300
149682150,0sleep10-21swapper/123:29:151
9452140,0sleep20-21swapper/222:02:102
2173399149,5cyclictest4509-21sshd21:31:437
2173399148,5cyclictest17097-21sshd21:11:157
2173399143,5cyclictest0-21swapper/721:04:517
2173399140,9cyclictest10181-21sshd23:19:217
2172499146,1cyclictest65-21ksoftirqd/622:22:066
2172499145,3cyclictest65-21ksoftirqd/623:44:116
2172499145,2cyclictest65-21ksoftirqd/623:24:176
2172499142,4cyclictest65-21ksoftirqd/620:24:106
2172499142,4cyclictest65-21ksoftirqd/620:24:106
21724991413,1cyclictest65-21ksoftirqd/623:04:516
2172499141,1cyclictest65-21ksoftirqd/623:49:306
2172499140,1cyclictest0-21swapper/621:16:176
2171799148,5cyclictest26016-21sshd23:54:115
2171799148,1cyclictest30804-21systemd-cgroups22:50:535
2171799146,8cyclictest0-21swapper/523:02:235
2171799143,5cyclictest0-21swapper/523:31:015
2171799143,0cyclictest0-21swapper/521:31:555
2171799142,12cyclictest0-21swapper/521:00:275
2171799142,0cyclictest0-21swapper/500:06:255
2171799140,9cyclictest0-21swapper/523:28:535
2168799149,3cyclictest25-21ksoftirqd/100:09:251
2168799142,1cyclictest25-21ksoftirqd/122:35:071
21687991413,1cyclictest25-21ksoftirqd/123:35:061
21687991413,1cyclictest25-21ksoftirqd/122:41:021
21687991413,1cyclictest25-21ksoftirqd/122:26:021
21687991413,1cyclictest25-21ksoftirqd/122:00:431
21687991412,2cyclictest25-21ksoftirqd/121:39:261
21687991412,2cyclictest25-21ksoftirqd/100:14:411
21687991412,1cyclictest25-21ksoftirqd/123:59:521
21687991412,1cyclictest25-21ksoftirqd/122:49:301
21687991412,1cyclictest25-21ksoftirqd/122:30:111
21687991412,1cyclictest25-21ksoftirqd/100:04:301
21687991411,3cyclictest25-21ksoftirqd/100:23:341
21675991412,1cyclictest9-21ksoftirqd/023:54:400
21675991412,1cyclictest9-21ksoftirqd/021:44:580
21675991412,1cyclictest9-21ksoftirqd/021:14:420
21675991411,2cyclictest9-21ksoftirqd/023:01:140
21675991411,1cyclictest9-21ksoftirqd/021:30:080
21675991410,1cyclictest9-21ksoftirqd/022:22:460
214622140,0sleep00-21swapper/022:54:140
2173399138,5cyclictest0-21swapper/722:35:557
2173399138,5cyclictest0-21swapper/720:55:077
2173399138,0cyclictest0-21swapper/721:59:497
2173399138,0cyclictest0-21swapper/721:34:197
2173399132,5cyclictest0-21swapper/723:30:537
2172499135,3cyclictest65-21ksoftirqd/623:19:526
2172499135,1cyclictest65-21ksoftirqd/622:36:436
21724991312,1cyclictest65-21ksoftirqd/623:54:216
21724991312,1cyclictest65-21ksoftirqd/621:39:096
21724991312,1cyclictest65-21ksoftirqd/621:30:556
21724991311,1cyclictest65-21ksoftirqd/620:56:316
2172499131,2cyclictest0-21swapper/620:59:216
2172499130,1cyclictest0-21swapper/623:12:486
2172499130,1cyclictest0-21swapper/622:59:216
2172499130,1cyclictest0-21swapper/622:30:286
2172499130,1cyclictest0-21swapper/622:24:416
2172499130,0cyclictest0-21swapper/621:06:326
2171799138,5cyclictest0-21swapper/521:54:315
2171799138,5cyclictest0-21swapper/521:54:315
2171799138,5cyclictest0-21swapper/521:20:475
2171799138,5cyclictest0-21swapper/520:58:275
2171799138,5cyclictest0-21swapper/520:35:595
2171799138,5cyclictest0-21swapper/519:24:195
2171799138,5cyclictest0-21swapper/519:20:495
2171799138,0cyclictest0-21swapper/523:40:555
2171799138,0cyclictest0-21swapper/519:08:075
21701991312,1cyclictest41-21ksoftirqd/322:39:533
21701991312,1cyclictest41-21ksoftirqd/322:39:073
21701991312,1cyclictest41-21ksoftirqd/322:09:583
21701991312,1cyclictest41-21ksoftirqd/321:29:213
21701991311,1cyclictest41-21ksoftirqd/322:31:553
21693991312,1cyclictest33-21ksoftirqd/200:18:442
21687991312,1cyclictest25-21ksoftirqd/123:57:481
21687991312,1cyclictest25-21ksoftirqd/123:49:301
21687991312,1cyclictest25-21ksoftirqd/123:22:161
21687991312,1cyclictest25-21ksoftirqd/123:14:581
21687991311,1cyclictest25-21ksoftirqd/123:27:391
21687991311,1cyclictest25-21ksoftirqd/123:04:221
21687991311,1cyclictest25-21ksoftirqd/122:54:471
21687991311,1cyclictest25-21ksoftirqd/122:04:221
21687991311,1cyclictest25-21ksoftirqd/121:31:111
21687991311,1cyclictest25-21ksoftirqd/121:25:501
21687991311,1cyclictest25-21ksoftirqd/120:57:581
21687991311,1cyclictest25-21ksoftirqd/120:14:321
21687991310,1cyclictest25-21ksoftirqd/122:44:341
2168799130,1cyclictest0-21swapper/123:45:231
2167599139,4cyclictest9-21ksoftirqd/021:35:120
2167599139,1cyclictest9-21ksoftirqd/022:03:070
2167599138,3cyclictest9-21ksoftirqd/000:02:330
2167599138,2cyclictest9-21ksoftirqd/023:06:540
2167599138,2cyclictest131rcu_sched23:18:080
21675991312,1cyclictest9-21ksoftirqd/021:28:320
21675991311,1cyclictest9-21ksoftirqd/022:50:250
21675991311,1cyclictest9-21ksoftirqd/022:06:090
21675991311,1cyclictest9-21ksoftirqd/020:59:230
21675991311,1cyclictest9-21ksoftirqd/020:54:140
21675991310,3cyclictest9-21ksoftirqd/020:54:090
2172499126,3cyclictest65-21ksoftirqd/623:40:476
2172499125,1cyclictest65-21ksoftirqd/600:19:126
2172499124,2cyclictest65-21ksoftirqd/622:45:016
21724991211,1cyclictest65-21ksoftirqd/622:49:236
21724991211,1cyclictest65-21ksoftirqd/621:59:226
21724991210,1cyclictest65-21ksoftirqd/620:34:106
2172499121,1cyclictest65-21ksoftirqd/622:04:466
2172499120,2cyclictest0-21swapper/600:07:376
2172499120,1cyclictest0-21swapper/621:53:436
2170199128,3cyclictest41-21ksoftirqd/322:08:003
2170199127,4cyclictest861-21gdbus00:21:033
21701991211,1cyclictest41-21ksoftirqd/323:29:153
21701991211,1cyclictest41-21ksoftirqd/323:04:103
21701991211,1cyclictest41-21ksoftirqd/321:16:193
21701991210,1cyclictest41-21ksoftirqd/323:04:243
21701991210,1cyclictest41-21ksoftirqd/321:54:303
21701991210,1cyclictest41-21ksoftirqd/321:54:303
2169399129,2cyclictest8045-21bash23:05:372
21693991211,1cyclictest33-21ksoftirqd/223:38:242
21693991210,1cyclictest33-21ksoftirqd/222:44:212
21687991211,1cyclictest25-21ksoftirqd/122:20:561
21687991211,1cyclictest25-21ksoftirqd/121:35:171
21687991210,1cyclictest25-21ksoftirqd/121:19:271
21687991210,1cyclictest25-21ksoftirqd/121:04:241
21687991210,1cyclictest25-21ksoftirqd/120:59:251
2167599129,3cyclictest9-21ksoftirqd/022:33:380
2167599128,3cyclictest9-21ksoftirqd/021:55:290
2167599128,3cyclictest9-21ksoftirqd/021:55:290
21675991211,1cyclictest9-21ksoftirqd/023:37:310
21675991211,1cyclictest9-21ksoftirqd/023:33:250
21675991211,1cyclictest9-21ksoftirqd/022:42:320
21675991211,1cyclictest9-21ksoftirqd/020:19:100
21675991210,1cyclictest9-21ksoftirqd/021:23:540
21675991210,1cyclictest9-21ksoftirqd/000:20:570
21675991210,1cyclictest9-21ksoftirqd/000:14:160
2167599120,11cyclictest1-21systemd21:08:320
2172499119,1cyclictest65-21ksoftirqd/622:44:086
21724991110,1cyclictest85350irq/25-em1-rx-019:40:046
21724991110,1cyclictest65-21ksoftirqd/623:35:266
2172499111,3cyclictest0-21swapper/621:26:426
2172499111,2cyclictest8864-21systemd-cgroups21:54:266
2172499111,2cyclictest8864-21systemd-cgroups21:54:266
2170999119,1cyclictest49-21ksoftirqd/423:04:274
21709991110,1cyclictest49-21ksoftirqd/423:51:174
2170999110,10cyclictest6511-21sshd00:15:284
2170999110,10cyclictest18892-21bash21:15:544
2170999110,1cyclictest31161-21taskset21:57:324
2170999110,1cyclictest31161-21taskset21:57:324
2170199119,1cyclictest41-21ksoftirqd/322:19:333
2170199119,1cyclictest41-21ksoftirqd/322:01:163
2170199118,3cyclictest41-21ksoftirqd/321:07:343
2170199116,3cyclictest5498-21sshd22:51:553
21701991110,1cyclictest41-21ksoftirqd/323:58:543
21701991110,1cyclictest41-21ksoftirqd/321:27:013
21701991110,1cyclictest41-21ksoftirqd/321:22:113
21701991110,1cyclictest41-21ksoftirqd/300:13:133
2169399119,2cyclictest33-21ksoftirqd/222:24:312
2169399119,1cyclictest33-21ksoftirqd/223:10:552
2169399119,1cyclictest33-21ksoftirqd/221:39:322
2169399119,1cyclictest0-21swapper/223:45:022
2169399110,10cyclictest1-21systemd23:30:322
21687991110,1cyclictest25-21ksoftirqd/119:49:251
2167599119,1cyclictest9-21ksoftirqd/023:52:380
2167599119,1cyclictest9-21ksoftirqd/022:27:260
2167599119,1cyclictest9-21ksoftirqd/022:11:550
2167599118,3cyclictest9-21ksoftirqd/022:44:320
2167599110,10cyclictest16006-21sshd23:11:120
218502100,0sleep40-21swapper/421:38:344
2172499104,3cyclictest65-21ksoftirqd/620:44:366
2172499103,3cyclictest65-21ksoftirqd/619:18:546
2170999109,1cyclictest0-21swapper/423:21:524
2170999109,1cyclictest0-21swapper/422:35:314
2170999108,1cyclictest49-21ksoftirqd/421:48:554
2170999100,10cyclictest0-21swapper/423:58:174
2170199109,1cyclictest41-21ksoftirqd/323:25:563
2170199109,1cyclictest41-21ksoftirqd/323:11:353
2170199109,1cyclictest41-21ksoftirqd/321:49:273
2170199109,1cyclictest41-21ksoftirqd/321:40:593
2170199109,1cyclictest41-21ksoftirqd/321:13:553
2170199109,1cyclictest0-21swapper/300:18:213
2170199108,1cyclictest41-21ksoftirqd/323:49:253
2170199100,10cyclictest0-21swapper/323:43:093
2170199100,1cyclictest0-21swapper/322:27:283
2169399109,1cyclictest33-21ksoftirqd/222:54:202
2169399108,1cyclictest33-21ksoftirqd/221:44:582
2169399100,10cyclictest27136-21kworker/2:022:59:352
2169399100,10cyclictest21335-21sshd21:25:032
2169399100,1cyclictest0-21swapper/222:49:232
2168799109,1cyclictest25-21ksoftirqd/119:24:251
2168799108,1cyclictest25-21ksoftirqd/119:39:091
2167599109,1cyclictest9-21ksoftirqd/020:09:100
2167599109,1cyclictest9-21ksoftirqd/000:04:200
2167599108,1cyclictest9-21ksoftirqd/022:34:180
2167599108,1cyclictest9-21ksoftirqd/021:40:070
2167599100,5cyclictest16691-21kworker/0:023:22:400
217249997,1cyclictest65-21ksoftirqd/618:59:246
217249991,2cyclictest0-21swapper/620:42:536
217249991,1cyclictest65-21ksoftirqd/619:59:066
217249990,9cyclictest0-21swapper/619:44:336
217099999,0cyclictest0-21swapper/422:30:194
217099999,0cyclictest0-21swapper/421:08:494
217099998,1cyclictest49-21ksoftirqd/423:35:204
217099997,1cyclictest49-21ksoftirqd/422:07:214
217099997,1cyclictest49-21ksoftirqd/421:40:564
217099996,2cyclictest22539-21sshd23:03:184
217099990,0cyclictest0-21swapper/421:27:534
217019998,1cyclictest41-21ksoftirqd/323:46:113
217019998,1cyclictest0-21swapper/321:35:083
217019997,1cyclictest41-21ksoftirqd/323:20:123
217019997,1cyclictest14990-21kworker/3:220:52:293
217019994,3cyclictest41-21ksoftirqd/321:02:253
217019993,4cyclictest12094-21kworker/3:022:44:193
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional