You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-25 - 11:59
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Tue Nov 25, 2025 00:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
72002820,0sleep70-21swapper/722:26:037
137852570,2sleep0517699cyclictest21:14:200
485525025,9sleep60-21swapper/618:51:346
472325026,8sleep70-21swapper/718:49:427
15832490,1sleep01578-21sshd23:03:440
296082480,1sleep3401ktimersoftd/320:29:393
9392460,0sleep40-21swapper/422:42:104
64782460,0sleep30-21swapper/320:49:163
295752460,0sleep20-21swapper/221:10:142
273552460,0sleep60-21swapper/621:47:186
153812460,0sleep60-21swapper/621:38:216
124622450,0sleep00-21swapper/023:55:580
124622450,0sleep00-21swapper/023:55:580
64872440,0sleep60-21swapper/621:00:256
153492440,0sleep30-21swapper/323:07:323
141532440,0sleep00-21swapper/021:32:540
141532440,0sleep00-21swapper/021:32:540
70012430,0sleep40-21swapper/423:28:154
33682430,0sleep10-21swapper/122:25:411
17862430,2sleep4519299cyclictest23:14:304
217882420,0sleep10-21swapper/100:08:411
129582420,0sleep00-21swapper/000:24:090
122902420,0sleep70-21swapper/723:12:407
81612410,0sleep10-21swapper/100:23:381
502624128,9sleep50-21swapper/518:53:495
274032410,0sleep40-21swapper/422:22:164
26182410,1sleep62620-21sshd21:39:496
213422410,0sleep20-21swapper/222:32:552
194082410,0sleep70-21swapper/700:18:007
18012410,1sleep71800-21bash23:52:137
105272410,0sleep50-21swapper/523:36:335
316572400,0sleep30-21swapper/322:30:563
492523928,8sleep20-21swapper/218:52:262
489923928,8sleep40-21swapper/418:52:044
479423926,9sleep10-21swapper/118:50:411
258323927,9sleep30-21swapper/318:49:223
234882390,0sleep30-21swapper/322:54:423
270742380,0sleep70-21swapper/720:59:307
258923826,8sleep00-21swapper/018:49:270
119892380,0sleep10-21swapper/121:06:121
519999213,17cyclictest23981-21grepconf.sh21:12:325
263962210,1sleep1241ktimersoftd/123:51:271
183842210,0sleep10-21swapper/122:13:241
519999208,3cyclictest57-21ksoftirqd/521:07:335
519999207,3cyclictest57-21ksoftirqd/523:58:185
519999207,3cyclictest57-21ksoftirqd/523:58:185
5199992011,3cyclictest57-21ksoftirqd/522:09:505
517799202,15cyclictest711-21dbus-daemon23:17:321
220682200,0sleep60-21swapper/622:10:486
5203991910,1cyclictest65-21ksoftirqd/621:08:366
519999191,3cyclictest57-21ksoftirqd/521:16:075
519999191,3cyclictest0-21swapper/523:16:185
519299194,2cyclictest49-21ksoftirqd/422:11:044
519299189,1cyclictest49-21ksoftirqd/400:23:224
5192991815,3cyclictest49-21ksoftirqd/421:34:354
5192991813,4cyclictest49-21ksoftirqd/421:07:274
520399177,2cyclictest65-21ksoftirqd/623:37:316
520399175,3cyclictest65-21ksoftirqd/600:03:206
5203991716,1cyclictest65-21ksoftirqd/621:59:386
520399171,15cyclictest5234-21sshd00:19:436
519999179,1cyclictest57-21ksoftirqd/521:42:095
519999171,3cyclictest57-21ksoftirqd/522:42:125
519999171,3cyclictest57-21ksoftirqd/521:52:045
519999171,2cyclictest0-21swapper/521:30:455
519999171,2cyclictest0-21swapper/521:30:455
519999170,1cyclictest0-21swapper/522:50:475
519299175,4cyclictest49-21ksoftirqd/421:02:484
5192991713,3cyclictest49-21ksoftirqd/423:42:354
5192991712,3cyclictest49-21ksoftirqd/422:07:204
5192991711,4cyclictest49-21ksoftirqd/423:56:194
5192991711,4cyclictest49-21ksoftirqd/423:56:194
519299170,3cyclictest12111-21grepconf.sh22:59:164
517799170,16cyclictest13028-21bash22:59:201
170262170,0sleep70-21swapper/721:22:287
520499160,10cyclictest0-21swapper/719:20:207
520399169,1cyclictest65-21ksoftirqd/623:00:166
520399168,1cyclictest65-21ksoftirqd/600:12:116
520399167,3cyclictest65-21ksoftirqd/622:47:546
520399166,3cyclictest65-21ksoftirqd/623:55:516
520399166,3cyclictest65-21ksoftirqd/623:55:516
520399166,3cyclictest65-21ksoftirqd/623:25:146
5203991614,2cyclictest65-21ksoftirqd/622:51:136
5203991613,2cyclictest631rcuc/623:29:436
519999167,3cyclictest57-21ksoftirqd/521:27:535
519999166,3cyclictest57-21ksoftirqd/521:02:485
519999164,3cyclictest57-21ksoftirqd/523:07:205
5199991611,5cyclictest57-21ksoftirqd/519:43:165
519999161,3cyclictest57-21ksoftirqd/522:06:095
519999161,3cyclictest57-21ksoftirqd/521:45:265
519999161,3cyclictest57-21ksoftirqd/500:04:385
519999161,1cyclictest57-21ksoftirqd/522:30:555
519999161,1cyclictest57-21ksoftirqd/521:35:195
519299164,4cyclictest49-21ksoftirqd/423:05:014
519299162,2cyclictest49-21ksoftirqd/423:11:284
5192991614,1cyclictest49-21ksoftirqd/423:49:294
5192991614,1cyclictest49-21ksoftirqd/423:38:504
5192991613,1cyclictest49-21ksoftirqd/423:45:574
5192991612,3cyclictest49-21ksoftirqd/421:16:294
518399160,15cyclictest31978-21sshd00:00:432
518399160,15cyclictest14985-21grepconf.sh22:04:482
520499154,10cyclictest0-21swapper/720:12:527
520499150,10cyclictest0-21swapper/719:05:067
520399156,3cyclictest65-21ksoftirqd/622:25:506
520399156,3cyclictest65-21ksoftirqd/600:17:496
520399155,3cyclictest65-21ksoftirqd/621:09:376
5203991510,1cyclictest65-21ksoftirqd/623:04:416
520399151,2cyclictest2254-21grepconf.sh22:34:096
520399151,2cyclictest0-21swapper/622:57:576
520399151,2cyclictest0-21swapper/600:06:366
520399151,1cyclictest0-21swapper/621:24:496
520399150,1cyclictest0-21swapper/623:21:546
520399150,1cyclictest0-21swapper/622:22:216
520399150,1cyclictest0-21swapper/622:04:326
520399150,1cyclictest0-21swapper/621:49:316
519999156,3cyclictest57-21ksoftirqd/522:03:575
519999155,3cyclictest57-21ksoftirqd/523:49:245
519999155,3cyclictest57-21ksoftirqd/523:20:265
519999154,6cyclictest57-21ksoftirqd/519:47:565
5199991514,1cyclictest57-21ksoftirqd/523:27:025
5199991514,1cyclictest57-21ksoftirqd/500:09:345
519999151,3cyclictest57-21ksoftirqd/523:59:525
519999151,3cyclictest57-21ksoftirqd/522:59:485
519999151,3cyclictest57-21ksoftirqd/522:27:575
519999151,3cyclictest20576-21sshd23:45:285
519999151,3cyclictest10448-21sshd23:39:195
519999151,1cyclictest57-21ksoftirqd/522:16:575
519999151,1cyclictest57-21ksoftirqd/500:18:155
519999151,1cyclictest0-21swapper/521:20:525
519999150,4cyclictest57-21ksoftirqd/522:56:105
519999150,3cyclictest0-21swapper/523:09:435
519999150,1cyclictest0-21swapper/521:56:255
519299154,1cyclictest49-21ksoftirqd/420:57:084
5192991514,1cyclictest49-21ksoftirqd/400:03:494
5192991513,1cyclictest49-21ksoftirqd/422:28:104
5192991513,1cyclictest49-21ksoftirqd/422:18:044
5192991513,1cyclictest49-21ksoftirqd/421:54:414
5192991513,1cyclictest49-21ksoftirqd/421:46:184
5192991513,1cyclictest49-21ksoftirqd/421:30:274
5192991513,1cyclictest49-21ksoftirqd/421:30:274
5192991513,1cyclictest49-21ksoftirqd/400:05:144
5192991512,3cyclictest49-21ksoftirqd/421:26:004
5192991511,2cyclictest49-21ksoftirqd/421:19:364
519299150,15cyclictest21541-21kworker/4:421:39:414
518399150,12cyclictest7357-21sshd23:20:082
518399150,12cyclictest18710-21systemd-cgroups21:30:292
518399150,12cyclictest18710-21systemd-cgroups21:30:292
5177991512,2cyclictest21609-21sshd23:21:211
520499140,4cyclictest0-21swapper/718:55:147
520399146,3cyclictest65-21ksoftirqd/623:16:496
5203991413,1cyclictest65-21ksoftirqd/622:35:226
5203991413,1cyclictest65-21ksoftirqd/622:18:266
5203991413,1cyclictest65-21ksoftirqd/621:56:016
5203991412,2cyclictest65-21ksoftirqd/621:14:206
520399141,3cyclictest65-21ksoftirqd/622:39:436
520399141,3cyclictest0-21swapper/623:45:396
520399140,1cyclictest0-21swapper/621:32:176
520399140,1cyclictest0-21swapper/621:32:176
519999147,3cyclictest57-21ksoftirqd/523:30:565
519999145,2cyclictest57-21ksoftirqd/500:21:185
519999143,6cyclictest57-21ksoftirqd/519:24:565
5199991413,1cyclictest57-21ksoftirqd/522:46:465
519999141,3cyclictest57-21ksoftirqd/522:21:535
519999141,13cyclictest57-21ksoftirqd/520:30:185
519999140,3cyclictest0-21swapper/522:34:275
519999140,2cyclictest0-21swapper/520:54:305
5192991413,1cyclictest49-21ksoftirqd/400:09:314
5192991412,2cyclictest49-21ksoftirqd/423:33:304
5192991412,2cyclictest0-21swapper/419:23:594
5192991412,1cyclictest49-21ksoftirqd/423:24:164
5192991412,1cyclictest49-21ksoftirqd/423:03:594
5192991412,1cyclictest49-21ksoftirqd/422:46:424
5192991412,1cyclictest49-21ksoftirqd/422:35:044
5192991412,1cyclictest49-21ksoftirqd/422:29:304
5192991412,1cyclictest49-21ksoftirqd/421:59:344
5192991412,1cyclictest49-21ksoftirqd/421:50:084
5192991412,1cyclictest49-21ksoftirqd/400:14:414
519299140,1cyclictest49-21ksoftirqd/421:09:274
44042140,0sleep00-21swapper/023:46:530
259502140,0sleep10-21swapper/122:38:551
140992140,0sleep20-21swapper/222:18:292
125792140,0sleep70-21swapper/722:34:527
5742130,0sleep20-21swapper/223:38:302
520499130,4cyclictest0-21swapper/719:35:407
520399135,3cyclictest65-21ksoftirqd/619:39:376
5203991313,0cyclictest631rcuc/621:22:566
520399130,6cyclictest5665-21bash23:09:286
520399130,1cyclictest0-21swapper/623:52:306
520399130,1cyclictest0-21swapper/623:39:576
520399130,1cyclictest0-21swapper/620:54:386
519999138,5cyclictest57-21ksoftirqd/520:50:505
519999130,8cyclictest0-21swapper/520:38:365
519999130,2cyclictest0-21swapper/519:59:545
519999130,13cyclictest0-21swapper/520:21:205
5192991310,3cyclictest49-21ksoftirqd/422:50:454
518399130,2cyclictest5906-21sshd22:20:222
517799132,5cyclictest5787-21bash23:25:201
5176991311,1cyclictest9-21ksoftirqd/000:14:390
5176991310,2cyclictest11374-21bash21:03:360
90302120,0sleep20-21swapper/200:07:292
83772120,0sleep70-21swapper/723:06:567
5204991211,1cyclictest12744-21kworker/u16:021:57:127
5204991210,1cyclictest73-21ksoftirqd/723:23:137
520499120,12cyclictest0-21swapper/721:49:567
520399121,1cyclictest0-21swapper/618:54:316
519999127,5cyclictest57-21ksoftirqd/519:32:285
519999127,0cyclictest57-21ksoftirqd/520:05:465
519999125,2cyclictest57-21ksoftirqd/520:25:065
519999124,3cyclictest57-21ksoftirqd/518:54:295
519999124,1cyclictest57-21ksoftirqd/519:34:355
519999123,9cyclictest57-21ksoftirqd/520:15:405
519999120,6cyclictest13079-21kworker/5:119:50:185
519999120,6cyclictest13079-21kworker/5:119:50:185
519999120,6cyclictest0-21swapper/520:47:325
519999120,12cyclictest0-21swapper/520:40:585
519999120,0cyclictest13079-21kworker/5:119:19:005
518899120,11cyclictest0-21swapper/323:15:093
518399129,2cyclictest31920-21bash23:11:362
518399123,4cyclictest29720-21sshd00:22:212
5183991210,1cyclictest861-21gdbus23:14:362
5183991210,1cyclictest33-21ksoftirqd/220:04:172
518399120,11cyclictest12616-21id21:16:502
518399120,11cyclictest1-21systemd22:40:232
517799129,2cyclictest4558-21grep22:58:381
517799122,4cyclictest15978-21sshd22:43:271
5177991210,1cyclictest26577-21systemd-cgroups21:33:591
5177991210,1cyclictest26577-21systemd-cgroups21:33:591
517799121,4cyclictest26029-21sshd22:05:471
5176991211,1cyclictest9-21ksoftirqd/023:30:020
517699120,11cyclictest19499-21kworker/0:322:45:380
248312120,0sleep70-21swapper/721:33:497
248312120,0sleep70-21swapper/721:33:497
181822120,0sleep00-21swapper/021:46:320
151342120,0sleep30-21swapper/321:27:373
520499119,1cyclictest73-21ksoftirqd/700:24:167
520499119,1cyclictest594-21sshd00:00:487
520499110,2cyclictest23825-21bash20:55:437
520499110,11cyclictest0-21swapper/720:41:307
520499110,11cyclictest0-21swapper/720:27:567
520499110,11cyclictest0-21swapper/720:02:587
520499110,11cyclictest0-21swapper/719:27:047
520499110,10cyclictest0-21swapper/720:21:487
520499110,10cyclictest0-21swapper/719:00:527
520499110,1cyclictest12777-21bash21:38:087
520499110,0cyclictest0-21swapper/723:43:387
520399114,3cyclictest65-21ksoftirqd/620:09:596
5203991110,1cyclictest65-21ksoftirqd/620:54:176
519999114,3cyclictest57-21ksoftirqd/518:59:335
519999114,2cyclictest57-21ksoftirqd/519:22:345
519999114,1cyclictest57-21ksoftirqd/519:54:345
519999114,1cyclictest57-21ksoftirqd/519:54:345
519999110,6cyclictest13079-21kworker/5:120:13:485
518899119,1cyclictest41-21ksoftirqd/323:59:223
518899119,1cyclictest0-21swapper/322:49:273
518899112,4cyclictest0-21swapper/323:56:293
518899112,4cyclictest0-21swapper/323:56:293
518899111,4cyclictest27974-21sshd21:34:063
518899111,4cyclictest27974-21sshd21:34:063
518899110,5cyclictest11061-21id21:19:243
518899110,1cyclictest20268-21sshd22:10:403
518399112,4cyclictest3421-21sshd21:24:042
518399112,4cyclictest0-21swapper/222:12:142
518399111,4cyclictest0-21swapper/223:45:372
518399110,2cyclictest10265-21bash21:40:292
518399110,10cyclictest732-21NetworkManager21:59:332
518399110,10cyclictest0-21swapper/219:04:312
518399110,1cyclictest0-21swapper/222:59:372
517799118,1cyclictest22859-21kworker/1:223:07:541
517799112,4cyclictest4031-21sshd21:00:121
517799112,4cyclictest16867-21sshd22:29:371
517799111,4cyclictest3465-21cp22:04:011
517799110,5cyclictest17766-21sshd21:49:141
517799110,11cyclictest0-21swapper/122:16:191
517699119,1cyclictest9-21ksoftirqd/022:33:100
517699119,1cyclictest9-21ksoftirqd/022:02:290
517699110,1cyclictest668-21systemd-logind22:58:030
517699110,1cyclictest22122-21sshd22:21:480
153262110,0sleep50-21swapper/519:09:345
125922110,0sleep30-21swapper/322:37:443
520499109,1cyclictest20663-21sshd22:02:437
520499109,1cyclictest0-21swapper/721:46:107
520499109,1cyclictest0-21swapper/721:09:547
520499108,1cyclictest73-21ksoftirqd/722:30:267
520499103,6cyclictest10626-21sshd22:45:387
520499100,5cyclictest0-21swapper/719:29:307
520499100,4cyclictest0-21swapper/723:38:407
520499100,4cyclictest0-21swapper/722:53:407
520499100,4cyclictest0-21swapper/722:23:287
520499100,4cyclictest0-21swapper/722:04:437
520499100,4cyclictest0-21swapper/700:11:067
520499100,10cyclictest0-21swapper/723:57:327
520499100,10cyclictest0-21swapper/723:57:327
520499100,10cyclictest0-21swapper/722:16:457
520499100,10cyclictest0-21swapper/720:51:087
520499100,10cyclictest0-21swapper/720:46:507
520499100,10cyclictest0-21swapper/720:35:007
520499100,10cyclictest0-21swapper/720:32:367
520499100,10cyclictest0-21swapper/720:15:067
520499100,10cyclictest0-21swapper/720:05:407
520499100,10cyclictest0-21swapper/719:54:407
520499100,10cyclictest0-21swapper/719:54:407
520499100,10cyclictest0-21swapper/719:51:527
520499100,10cyclictest0-21swapper/719:51:527
520499100,10cyclictest0-21swapper/719:43:087
520499100,1cyclictest0-21swapper/721:39:387
520499100,0cyclictest0-21swapper/723:01:307
519299109,1cyclictest49-21ksoftirqd/420:49:294
519299109,1cyclictest49-21ksoftirqd/419:59:324
519299109,1cyclictest49-21ksoftirqd/419:49:164
519299108,1cyclictest49-21ksoftirqd/420:44:364
518899109,1cyclictest0-21swapper/300:09:423
518899107,2cyclictest6399-21bash00:04:253
518899107,2cyclictest423-21bash22:06:243
518899101,4cyclictest0-21swapper/323:00:143
518899100,4cyclictest740-21gdbus23:34:443
518899100,10cyclictest0-21swapper/323:30:073
518899100,10cyclictest0-21swapper/322:28:573
518399109,1cyclictest0-21swapper/223:29:342
518399102,2cyclictest16134-21bash21:06:342
518399100,4cyclictest7206-21expr22:34:292
518399100,4cyclictest21922-21rm23:42:582
518399100,4cyclictest13702-21sshd22:45:542
518399100,10cyclictest0-21swapper/223:05:372
518399100,10cyclictest0-21swapper/220:24:292
517799109,1cyclictest0-21swapper/123:30:201
517799102,2cyclictest24633-21bash21:20:221
517799101,4cyclictest28684-21sshd22:44:301
517799101,4cyclictest21309-21sshd23:40:051
517799101,4cyclictest0-21swapper/122:19:401
517799101,4cyclictest0-21swapper/121:55:531
517799100,4cyclictest31923-21sshd23:38:241
517799100,4cyclictest18707-21sshd22:51:431
517799100,4cyclictest0-21swapper/123:11:151
517799100,4cyclictest0-21swapper/121:49:371
517799100,4cyclictest0-21swapper/121:25:061
517799100,10cyclictest0-21swapper/100:00:261
517699109,1cyclictest9-21ksoftirqd/023:37:070
517699109,1cyclictest9-21ksoftirqd/023:17:230
517699109,1cyclictest9-21ksoftirqd/022:15:280
517699109,1cyclictest9-21ksoftirqd/021:41:460
517699109,1cyclictest9-21ksoftirqd/000:10:480
517699108,1cyclictest9-21ksoftirqd/023:20:590
517699108,1cyclictest9-21ksoftirqd/021:19:270
517699108,1cyclictest9-21ksoftirqd/020:49:160
517699108,1cyclictest9-21ksoftirqd/000:00:580
517699104,3cyclictest18057-21bash23:42:380
517699100,10cyclictest0-21swapper/021:25:500
52049999,0cyclictest0-21swapper/723:30:137
52049990,4cyclictest1-21systemd22:42:407
52049990,4cyclictest0-21swapper/721:27:197
52049990,4cyclictest0-21swapper/719:47:307
52049990,4cyclictest0-21swapper/719:15:467
52049990,0cyclictest0-21swapper/723:44:247
52049990,0cyclictest0-21swapper/723:18:207
52039991,3cyclictest65-21ksoftirqd/619:21:286
52039991,2cyclictest0-21swapper/620:38:586
52039991,1cyclictest65-21ksoftirqd/620:18:376
52039991,1cyclictest65-21ksoftirqd/619:14:346
52039991,1cyclictest65-21ksoftirqd/619:00:296
52039990,0cyclictest0-21swapper/619:09:336
51999991,3cyclictest57-21ksoftirqd/519:04:285
51929997,1cyclictest49-21ksoftirqd/420:34:164
51929997,1cyclictest49-21ksoftirqd/420:29:164
51929997,1cyclictest49-21ksoftirqd/419:54:384
51929997,1cyclictest49-21ksoftirqd/419:54:384
51929997,1cyclictest49-21ksoftirqd/419:54:164
51929997,1cyclictest49-21ksoftirqd/419:54:164
51929997,1cyclictest49-21ksoftirqd/419:24:364
51889998,1cyclictest41-21ksoftirqd/322:01:253
51889998,1cyclictest31157-21sshd20:59:433
51889998,1cyclictest0-21swapper/319:44:413
51889997,1cyclictest9810-21cp21:51:073
51889997,1cyclictest41-21ksoftirqd/322:20:573
51889996,2cyclictest6136-21kworker/3:120:54:343
51889993,6cyclictest6508-21sshd21:11:023
51889992,6cyclictest5783-21id23:47:003
51889990,9cyclictest0-21swapper/322:16:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional