You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-27 - 11:18
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Thu Nov 27, 2025 00:43:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1933699118116,1cyclictest28104-21nfsd22:19:160
1939299990,98cyclictest16546-21kworker/7:322:19:167
176462800,4sleep60-21swapper/618:49:266
135272780,1sleep21935299cyclictest21:07:272
105472730,0sleep10-21swapper/100:16:271
210232510,0sleep40-21swapper/422:29:064
80942500,0sleep40-21swapper/421:51:264
189362500,0sleep20-21swapper/221:48:362
115332500,0sleep00-21swapper/000:06:250
161702470,0sleep50-21swapper/520:58:415
174242460,0sleep00-21swapper/023:48:310
231652440,1sleep449-21ksoftirqd/422:56:054
19392994442,1cyclictest3954-21cp22:49:127
1884924228,9sleep50-21swapper/518:49:345
292332410,0sleep30-21swapper/322:07:513
1905924129,8sleep70-21swapper/718:52:097
221222400,0sleep30-21swapper/323:26:563
202182400,0sleep70-21swapper/723:58:237
202182400,0sleep70-21swapper/723:58:237
1934499400,39cyclictest12707-21kworker/1:222:49:121
1896024027,9sleep40-21swapper/418:50:454
1895724029,8sleep10-21swapper/118:50:421
1891524027,9sleep30-21swapper/318:50:123
1895623927,8sleep00-21swapper/018:50:410
1893823928,8sleep20-21swapper/218:50:272
102342390,0sleep20-21swapper/200:21:252
152092360,0sleep71939299cyclictest00:17:137
1936599236,3cyclictest49-21ksoftirqd/423:40:064
150602230,0sleep70-21swapper/722:14:367
19392992211,0cyclictest0-21swapper/700:19:197
1939299212,18cyclictest22517-21grep22:51:377
1939299212,18cyclictest22517-21grep22:51:377
1937799210,9cyclictest0-21swapper/522:24:335
1939299202,15cyclictest8596-21smart_sda21:24:317
1937799203,5cyclictest0-21swapper/520:16:215
1937799200,8cyclictest0-21swapper/521:40:035
1939299198,5cyclictest0-21swapper/722:27:317
1937799198,5cyclictest0-21swapper/523:57:075
1937799198,5cyclictest0-21swapper/523:57:075
1937799198,5cyclictest0-21swapper/523:44:415
1937799198,5cyclictest0-21swapper/523:38:535
1937799198,5cyclictest0-21swapper/523:16:395
1937799198,5cyclictest0-21swapper/522:55:455
1937799198,5cyclictest0-21swapper/522:20:035
1937799198,5cyclictest0-21swapper/522:04:255
1937799198,5cyclictest0-21swapper/522:00:235
1937799198,5cyclictest0-21swapper/521:54:155
1937799198,5cyclictest0-21swapper/521:36:275
1937799198,5cyclictest0-21swapper/521:17:495
1937799198,5cyclictest0-21swapper/520:49:055
1937799198,5cyclictest0-21swapper/520:37:395
1937799198,5cyclictest0-21swapper/520:12:375
1937799198,5cyclictest0-21swapper/519:40:155
1937799198,5cyclictest0-21swapper/519:34:575
1937799198,5cyclictest0-21swapper/519:22:135
1937799198,5cyclictest0-21swapper/519:14:475
1937799198,5cyclictest0-21swapper/519:08:455
1937799198,5cyclictest0-21swapper/519:02:155
1937799198,5cyclictest0-21swapper/519:02:155
1937799198,5cyclictest0-21swapper/500:16:395
1937799198,5cyclictest0-21swapper/500:10:535
1937799198,5cyclictest0-21swapper/500:06:475
1937799198,11cyclictest0-21swapper/523:24:135
1937799198,11cyclictest0-21swapper/522:45:535
1937799198,11cyclictest0-21swapper/521:50:475
1937799198,0cyclictest0-21swapper/523:42:455
1937799198,0cyclictest0-21swapper/523:32:595
1937799198,0cyclictest0-21swapper/523:19:555
1937799198,0cyclictest0-21swapper/523:07:075
1937799198,0cyclictest0-21swapper/522:42:475
1937799198,0cyclictest0-21swapper/522:15:195
1937799198,0cyclictest0-21swapper/521:31:195
1937799198,0cyclictest0-21swapper/521:21:175
1937799198,0cyclictest0-21swapper/520:52:495
1937799198,0cyclictest0-21swapper/519:13:555
1937799198,0cyclictest0-21swapper/500:23:255
1937799198,0cyclictest0-21swapper/500:02:015
1937799197,6cyclictest0-21swapper/523:13:395
1937799197,12cyclictest0-21swapper/523:49:475
1937799192,5cyclictest0-21swapper/519:33:395
43882180,0sleep20-21swapper/221:42:032
1939299180,17cyclictest19735-21sshd23:22:157
1939299180,17cyclictest15438-21sshd23:04:077
1937799188,0cyclictest0-21swapper/522:36:075
1937799188,0cyclictest0-21swapper/518:56:535
1937799188,0cyclictest0-21swapper/518:56:535
1937799187,5cyclictest0-21swapper/523:00:415
1937799187,5cyclictest0-21swapper/522:50:515
1937799187,5cyclictest0-21swapper/522:50:515
1937799187,5cyclictest0-21swapper/522:30:195
1937799187,5cyclictest0-21swapper/521:46:135
1937799187,5cyclictest0-21swapper/521:10:295
1937799187,5cyclictest0-21swapper/521:10:295
1937799187,5cyclictest0-21swapper/520:41:395
1937799187,5cyclictest0-21swapper/520:26:255
1937799187,5cyclictest0-21swapper/519:51:315
1937799187,5cyclictest0-21swapper/519:24:455
1937799187,0cyclictest0-21swapper/522:10:595
1937799187,0cyclictest0-21swapper/520:29:255
1937799187,0cyclictest0-21swapper/520:08:495
1937799187,0cyclictest0-21swapper/519:59:475
1937799187,0cyclictest0-21swapper/519:57:575
1937799187,0cyclictest0-21swapper/519:44:175
1937799176,5cyclictest0-21swapper/521:04:295
1937799176,5cyclictest0-21swapper/521:00:015
1937799176,0cyclictest0-21swapper/521:26:415
1937799176,0cyclictest0-21swapper/520:19:135
19365991713,3cyclictest49-21ksoftirqd/423:32:514
19365991712,3cyclictest49-21ksoftirqd/423:59:304
19365991712,3cyclictest49-21ksoftirqd/421:35:564
19392991611,1cyclictest32743-21bash21:32:337
1939299160,13cyclictest22852-21sshd21:49:117
1936599165,1cyclictest49-21ksoftirqd/422:39:274
19365991614,2cyclictest49-21ksoftirqd/423:50:164
19365991613,0cyclictest49-21ksoftirqd/422:44:314
19365991611,3cyclictest49-21ksoftirqd/421:39:254
19365991611,3cyclictest49-21ksoftirqd/421:27:544
19365991611,3cyclictest49-21ksoftirqd/421:11:374
19365991611,3cyclictest49-21ksoftirqd/421:11:374
1936599160,15cyclictest99-21kswapd020:48:214
1933699160,15cyclictest13189-21sshd21:07:250
162952160,0sleep60-21swapper/622:55:036
239282150,1sleep1763-21runrttasks21:44:341
1939299158,7cyclictest0-21swapper/722:07:197
1939299157,2cyclictest0-21swapper/723:37:137
1939299150,14cyclictest9524-21sshd20:57:377
1936599158,4cyclictest49-21ksoftirqd/422:08:184
1936599154,3cyclictest49-21ksoftirqd/421:04:554
19365991513,1cyclictest49-21ksoftirqd/423:19:274
19365991513,1cyclictest49-21ksoftirqd/422:19:244
19365991513,1cyclictest49-21ksoftirqd/421:24:074
19365991513,1cyclictest49-21ksoftirqd/400:22:454
19365991512,1cyclictest49-21ksoftirqd/423:09:544
19365991510,3cyclictest49-21ksoftirqd/423:14:284
19365991510,3cyclictest49-21ksoftirqd/423:05:414
230582140,0sleep00-21swapper/023:18:200
1939299149,0cyclictest0-21swapper/720:48:157
1939299146,2cyclictest0-21swapper/719:50:237
1939299146,1cyclictest28822-21grepconf.sh21:00:237
1939299146,0cyclictest0-21swapper/719:03:357
1939299146,0cyclictest0-21swapper/719:03:357
19392991413,1cyclictest73-21ksoftirqd/723:06:037
1936599149,3cyclictest49-21ksoftirqd/423:38:444
1936599149,3cyclictest49-21ksoftirqd/400:12:144
1936599144,3cyclictest49-21ksoftirqd/423:24:574
1936599144,1cyclictest49-21ksoftirqd/422:49:184
1936599144,1cyclictest49-21ksoftirqd/422:49:184
1936599143,2cyclictest49-21ksoftirqd/423:48:444
19365991413,1cyclictest49-21ksoftirqd/400:07:164
19365991412,2cyclictest49-21ksoftirqd/421:03:284
19365991412,1cyclictest49-21ksoftirqd/421:54:254
19365991412,1cyclictest49-21ksoftirqd/421:14:384
1936599141,2cyclictest49-21ksoftirqd/422:16:374
314112130,0sleep20-21swapper/221:14:242
240452137,0sleep70-21swapper/721:40:097
225392130,0sleep10-21swapper/121:08:461
1939299135,2cyclictest0-21swapper/723:25:057
1939299135,2cyclictest0-21swapper/719:08:577
1939299135,0cyclictest0-21swapper/723:41:237
1936599139,3cyclictest49-21ksoftirqd/421:48:324
1936599139,1cyclictest49-21ksoftirqd/400:14:254
1936599135,3cyclictest49-21ksoftirqd/421:31:004
19365991311,2cyclictest49-21ksoftirqd/422:10:484
19365991311,1cyclictest49-21ksoftirqd/423:54:284
19365991311,1cyclictest49-21ksoftirqd/423:54:284
19365991311,1cyclictest49-21ksoftirqd/422:35:384
19365991311,1cyclictest49-21ksoftirqd/422:29:374
19365991311,1cyclictest49-21ksoftirqd/420:54:304
19365991310,2cyclictest49-21ksoftirqd/421:59:244
78272120,0sleep30-21swapper/322:00:143
70482120,0sleep00-21swapper/021:29:040
314872120,1sleep125-21ksoftirqd/121:27:531
255012120,0sleep10-21swapper/121:22:321
19392991210,1cyclictest9945-21bash21:56:097
19392991210,1cyclictest73-21ksoftirqd/722:37:577
1939299120,11cyclictest711-21dbus-daemon21:52:337
1938499129,2cyclictest30728-21iostat00:04:276
1938499128,1cyclictest65-21ksoftirqd/622:34:576
1938499120,11cyclictest15098-21bash22:01:196
19365991211,1cyclictest49-21ksoftirqd/419:44:124
19365991211,1cyclictest49-21ksoftirqd/419:09:124
19365991210,1cyclictest49-21ksoftirqd/422:59:134
1935999129,2cyclictest5408-21grep22:53:503
1935999129,2cyclictest5408-21grep22:53:503
19336991211,1cyclictest9-21ksoftirqd/021:34:120
1933699120,7cyclictest31966-21sshd23:32:440
1933699120,1cyclictest0-21swapper/022:16:330
320852110,1sleep233-21ksoftirqd/223:01:482
1939299117,3cyclictest31845-21bash21:36:537
1939299117,1cyclictest0-21swapper/722:01:597
1939299116,5cyclictest0-21swapper/720:08:037
1939299116,0cyclictest0-21swapper/720:27:497
1939299116,0cyclictest0-21swapper/719:55:037
1939299113,8cyclictest0-21swapper/721:04:357
1939299110,2cyclictest0-21swapper/720:32:597
1939299110,2cyclictest0-21swapper/720:12:017
1939299110,2cyclictest0-21swapper/719:38:597
1939299110,11cyclictest0-21swapper/723:09:337
1939299110,10cyclictest867-21sshd23:46:037
1939299110,10cyclictest0-21swapper/723:16:357
1939299110,1cyclictest9281-21sshd22:58:467
1938499119,1cyclictest711-21dbus-daemon22:44:126
1938499113,5cyclictest18253-21sshd00:22:486
1938499111,10cyclictest85350irq/25-em1-rx-021:18:556
1938499110,1cyclictest711-21dbus-daemon23:11:476
19365991110,1cyclictest49-21ksoftirqd/420:24:124
1935999119,1cyclictest0-21swapper/300:09:113
19359991110,1cyclictest0-21swapper/322:15:163
1935999110,10cyclictest13369-21sshd21:43:223
1934499119,1cyclictest25-21ksoftirqd/122:07:531
1934499116,5cyclictest19081-21kworker/1:219:06:471
1934499110,10cyclictest668-21systemd-logind21:52:331
1934499110,10cyclictest18395-21sshd23:26:241
1934499110,1cyclictest0-21swapper/122:28:351
1933699119,1cyclictest9-21ksoftirqd/021:44:050
1933699119,1cyclictest16074-21cpuspeed_turbos19:44:250
1933699110,2cyclictest12399-21bash22:50:070
1933699110,2cyclictest12399-21bash22:50:070
1939299108,2cyclictest0-21swapper/723:49:297
1939299108,2cyclictest0-21swapper/721:14:597
1939299108,2cyclictest0-21swapper/720:17:177
1939299108,2cyclictest0-21swapper/719:20:237
1939299108,2cyclictest0-21swapper/719:11:317
1939299108,2cyclictest0-21swapper/700:12:157
1939299108,1cyclictest3026-21sshd22:30:557
1939299108,0cyclictest0-21swapper/700:00:157
1939299105,5cyclictest0-21swapper/723:30:557
1939299105,5cyclictest0-21swapper/720:49:437
1939299105,5cyclictest0-21swapper/719:31:197
1939299105,0cyclictest0-21swapper/719:17:197
1939299105,0cyclictest0-21swapper/700:04:497
1939299104,5cyclictest0-21swapper/720:38:057
1939299102,6cyclictest28105-21nfsd21:20:477
1939299100,8cyclictest0-21swapper/722:10:337
1939299100,10cyclictest0-21swapper/722:42:127
1938499109,1cyclictest65-21ksoftirqd/623:31:396
1938499109,1cyclictest65-21ksoftirqd/622:13:186
1938499109,1cyclictest0-21swapper/623:21:066
1938499105,0cyclictest0-21swapper/622:18:116
1938499102,3cyclictest711-21dbus-daemon21:45:436
1938499100,5cyclictest4416-21kworker/6:021:40:256
1938499100,10cyclictest0-21swapper/621:35:286
1938499100,1cyclictest85350irq/25-em1-rx-020:55:066
1938499100,0cyclictest0-21swapper/623:05:016
1936599108,1cyclictest49-21ksoftirqd/419:29:124
1935999102,3cyclictest26385-21sshd21:58:333
1935999102,3cyclictest26310-21sshd22:47:453
1935999101,4cyclictest26436-21sshd23:49:363
1935999100,4cyclictest31517-21sshd23:41:253
1935999100,4cyclictest30117-21sshd23:45:353
1935999100,4cyclictest0-21swapper/322:41:563
1935999100,10cyclictest0-21swapper/323:34:403
1935999100,10cyclictest0-21swapper/322:38:363
1935299109,1cyclictest0-21swapper/200:17:442
1935299109,1cyclictest0-21swapper/200:03:422
1935299105,1cyclictest859-21polkitd21:58:592
1935299100,10cyclictest0-21swapper/222:42:122
1935299100,10cyclictest0-21swapper/222:10:112
1934499109,1cyclictest25-21ksoftirqd/123:07:541
1934499109,1cyclictest25-21ksoftirqd/122:59:041
1934499109,1cyclictest25-21ksoftirqd/121:41:351
1934499108,1cyclictest16670-21kworker/1:100:10:331
1934499106,4cyclictest17554-21kworker/1:321:57:431
1934499104,3cyclictest13606-21sshd23:39:081
1934499100,10cyclictest0-21swapper/100:06:261
1933699108,1cyclictest9-21ksoftirqd/023:29:110
1933699100,9cyclictest27392-21sshd23:54:310
1933699100,9cyclictest27392-21sshd23:54:310
1933699100,10cyclictest0-21swapper/000:10:510
1933699100,1cyclictest0-21swapper/022:42:440
28026290,0sleep60-21swapper/619:09:126
193929994,5cyclictest0-21swapper/719:25:117
193929994,1cyclictest32198-21sshd21:09:517
193929994,1cyclictest32198-21sshd21:09:517
193929993,5cyclictest0-21swapper/719:39:277
193929991,5cyclictest0-21swapper/719:44:297
193929990,9cyclictest0-21swapper/720:40:437
193929990,9cyclictest0-21swapper/720:21:277
193929990,9cyclictest0-21swapper/720:00:057
193929990,9cyclictest0-21swapper/718:55:497
193929990,9cyclictest0-21swapper/718:55:497
193849999,0cyclictest0-21swapper/623:00:456
193849999,0cyclictest0-21swapper/621:20:396
193849997,1cyclictest859-21polkitd21:32:596
193849993,5cyclictest1-21systemd21:56:216
193849993,3cyclictest32344-21sshd22:08:186
193849993,3cyclictest19288-21sshd23:44:196
193849992,5cyclictest30898-21sshd23:28:136
193849991,5cyclictest16707-21grep23:53:116
193849990,8cyclictest10343-21kworker/6:022:40:046
193849990,5cyclictest24633-21bash22:34:076
193849990,0cyclictest0-21swapper/622:49:436
193849990,0cyclictest0-21swapper/622:49:436
193849990,0cyclictest0-21swapper/620:39:306
193659998,1cyclictest49-21ksoftirqd/420:54:124
193659998,1cyclictest49-21ksoftirqd/419:34:114
193659997,1cyclictest49-21ksoftirqd/418:59:294
193659997,1cyclictest49-21ksoftirqd/418:59:294
193599999,0cyclictest0-21swapper/320:56:133
193599998,1cyclictest41-21ksoftirqd/323:29:173
193599997,1cyclictest41-21ksoftirqd/320:24:123
193599997,1cyclictest41-21ksoftirqd/319:24:123
193599996,3cyclictest18128-21kworker/3:221:38:493
193599994,4cyclictest861-21gdbus00:18:153
193599993,6cyclictest0-21swapper/322:27:263
193599993,3cyclictest25002-21bash21:22:283
193599991,4cyclictest0-21swapper/321:17:183
193599991,3cyclictest0-21swapper/321:25:253
193599990,9cyclictest0-21swapper/323:23:573
193599990,9cyclictest0-21swapper/323:17:103
193599990,9cyclictest0-21swapper/321:11:213
193599990,9cyclictest0-21swapper/321:11:213
193599990,4cyclictest4709-21sshd21:01:463
193599990,4cyclictest26014-21sshd23:09:403
193599990,0cyclictest0-21swapper/323:56:063
193599990,0cyclictest0-21swapper/323:56:063
193529999,0cyclictest0-21swapper/223:44:192
193529999,0cyclictest0-21swapper/223:27:362
193529999,0cyclictest0-21swapper/222:08:562
193529990,9cyclictest0-21swapper/223:41:192
193529990,9cyclictest0-21swapper/222:54:292
193529990,9cyclictest0-21swapper/222:48:582
193529990,9cyclictest0-21swapper/221:35:592
193529990,9cyclictest0-21swapper/220:57:212
193529990,9cyclictest0-21swapper/200:09:512
193529990,0cyclictest0-21swapper/221:12:292
193529990,0cyclictest0-21swapper/221:12:292
193449999,0cyclictest0-21swapper/123:41:511
193449998,1cyclictest25-21ksoftirqd/123:55:391
193449998,1cyclictest25-21ksoftirqd/123:55:391
193449998,1cyclictest25-21ksoftirqd/122:31:151
193449998,1cyclictest25-21ksoftirqd/122:01:291
193449998,1cyclictest25-21ksoftirqd/121:00:231
193449997,1cyclictest25633-21sshd22:34:131
193449997,1cyclictest25-21ksoftirqd/123:19:191
193449997,1cyclictest25-21ksoftirqd/123:01:491
193449997,1cyclictest25-21ksoftirqd/122:41:521
193449995,1cyclictest14746-21sshd22:14:351
193449994,3cyclictest0-21swapper/100:03:041
193449992,6cyclictest8499-21sshd23:51:531
193449990,0cyclictest0-21swapper/123:44:501
193449990,0cyclictest0-21swapper/123:14:061
193449990,0cyclictest0-21swapper/121:10:201
193449990,0cyclictest0-21swapper/121:10:201
193369998,1cyclictest9-21ksoftirqd/021:51:000
193369998,1cyclictest9-21ksoftirqd/021:22:120
193369997,1cyclictest9-21ksoftirqd/023:42:240
193369997,1cyclictest9-21ksoftirqd/023:09:100
193369997,1cyclictest9-21ksoftirqd/023:01:460
193369997,1cyclictest9-21ksoftirqd/022:10:470
193369997,1cyclictest9-21ksoftirqd/022:02:510
193369997,1cyclictest9-21ksoftirqd/021:57:380
193369997,1cyclictest9-21ksoftirqd/021:48:080
193369997,1cyclictest1-21systemd23:34:120
193369994,3cyclictest711-21dbus-daemon21:10:450
193369994,3cyclictest711-21dbus-daemon21:10:450
193369994,3cyclictest26936-21sshd22:07:310
193369993,3cyclictest6587-21sshd21:02:030
193369990,9cyclictest0-21swapper/023:10:190
193369990,5cyclictest0-21swapper/023:54:120
3285280,6sleep00-21swapper/023:19:490
193849988,0cyclictest0-21swapper/600:14:326
193849986,1cyclictest65-21ksoftirqd/623:18:536
193849986,1cyclictest65-21ksoftirqd/622:27:226
193849986,1cyclictest0-21swapper/621:52:296
193849984,1cyclictest85350irq/25-em1-rx-023:43:136
193849984,1cyclictest22021-21sshd21:04:136
193849984,1cyclictest16227-21sshd21:25:366
193849983,4cyclictest10983-21bash00:01:366
193849983,1cyclictest12922-21sshd00:11:416
193849982,3cyclictest26882-21sshd21:09:226
193849982,3cyclictest26882-21sshd21:09:226
193849981,3cyclictest21682-21sshd23:35:376
193849980,4cyclictest29997-21bash22:21:136
193659988,0cyclictest481ktimersoftd/420:04:094
193659987,1cyclictest49-21ksoftirqd/419:54:284
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional