You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-11 - 12:52
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Fri Jul 11, 2025 00:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
191642960,0sleep60-21swapper/621:46:506
224842910,0sleep30-21swapper/321:33:343
219482770,0sleep2321ktimersoftd/223:57:332
763425625,18sleep50-21swapper/518:54:175
763425625,18sleep50-21swapper/518:54:175
259732530,0sleep70-21swapper/719:24:307
94572520,0sleep20-21swapper/223:50:362
168422510,0sleep70-21swapper/722:54:357
984225026,9sleep30-21swapper/318:55:363
984225026,9sleep30-21swapper/318:55:363
265122500,0sleep10-21swapper/122:06:321
181752480,0sleep00-21swapper/022:35:380
181752480,0sleep00-21swapper/022:35:380
178462480,0sleep10-21swapper/100:05:491
26202460,1sleep665-21ksoftirqd/600:27:246
811624423,16sleep00-21swapper/018:54:280
811624423,16sleep00-21swapper/018:54:280
120652440,0sleep6641ktimersoftd/621:57:086
87892430,0sleep50-21swapper/521:56:505
84182430,0sleep00-21swapper/023:18:150
101262430,0sleep40-21swapper/400:24:584
73082420,0sleep40-21swapper/400:18:014
190742420,0sleep60-21swapper/623:37:596
190742420,0sleep60-21swapper/623:37:596
984524128,9sleep60-21swapper/618:55:396
984524128,9sleep60-21swapper/618:55:396
979924130,8sleep70-21swapper/718:54:597
979924130,8sleep70-21swapper/718:54:597
84282410,0sleep70-21swapper/722:32:087
194072410,0sleep00-21swapper/021:16:570
133222410,0sleep10-21swapper/121:08:171
992624028,8sleep20-21swapper/218:56:392
992624028,8sleep20-21swapper/218:56:392
298542400,0sleep30-21swapper/321:25:573
249872400,0sleep70-21swapper/720:19:307
200162400,0sleep60-21swapper/600:15:556
166112400,0sleep70-21swapper/721:52:047
995523928,8sleep10-21swapper/118:57:021
995523928,8sleep10-21swapper/118:57:021
979623927,8sleep40-21swapper/418:54:564
979623927,8sleep40-21swapper/418:54:564
40422390,0sleep00-21swapper/021:34:350
320022390,0sleep10-21swapper/121:23:311
319362390,0sleep10-21swapper/123:14:391
281622390,0sleep50-21swapper/522:36:315
281622390,0sleep50-21swapper/522:36:315
276642390,0sleep10-21swapper/121:14:501
274872390,0sleep20-21swapper/221:23:082
174632380,0sleep60-21swapper/619:09:316
121852370,0sleep70-21swapper/722:08:067
119302360,0sleep10-21swapper/122:16:091
117102360,0sleep20-21swapper/223:45:242
165062320,0sleep30-21swapper/322:11:063
110182290,0sleep20-21swapper/223:29:142
10262992115,4cyclictest33-21ksoftirqd/223:43:052
271842190,2sleep01025399cyclictest00:09:400
24522190,1sleep72441-21rm22:45:147
181962190,7sleep557-21ksoftirqd/523:57:125
1027799190,13cyclictest6524-21sshd00:07:484
10262991912,4cyclictest33-21ksoftirqd/222:19:262
1026299190,15cyclictest468-21systemd-journal20:19:122
52332180,0sleep10-21swapper/121:29:231
141472180,1sleep514148-21bash22:16:215
10294991816,1cyclictest73-21ksoftirqd/700:07:267
10262991815,1cyclictest33-21ksoftirqd/221:57:292
138952170,0sleep7721ktimersoftd/723:54:027
1029499178,8cyclictest0-21swapper/721:46:277
1029099179,8cyclictest0-21swapper/622:51:316
1026299179,4cyclictest33-21ksoftirqd/222:16:272
10262991714,3cyclictest33-21ksoftirqd/221:49:362
10262991714,3cyclictest33-21ksoftirqd/200:14:052
10262991714,3cyclictest33-21ksoftirqd/200:02:172
1029499168,7cyclictest20000-21bash21:57:497
1029499165,11cyclictest0-21swapper/723:37:017
1029499165,11cyclictest0-21swapper/723:37:017
1028399168,8cyclictest0-21swapper/500:10:215
1028399168,2cyclictest0-21swapper/521:37:335
1028399168,1cyclictest9030-21id22:40:215
1028399168,1cyclictest5643-21sshd22:04:415
1028399161,8cyclictest16411-21kworker/5:322:48:575
1026599160,15cyclictest32166-21kworker/3:021:43:503
10262991615,1cyclictest33-21ksoftirqd/223:34:122
10262991615,1cyclictest33-21ksoftirqd/223:34:122
10262991615,1cyclictest33-21ksoftirqd/222:41:452
10262991614,1cyclictest33-21ksoftirqd/222:01:492
10262991614,1cyclictest33-21ksoftirqd/221:06:342
10262991612,3cyclictest33-21ksoftirqd/223:08:022
10262991612,1cyclictest33-21ksoftirqd/221:09:372
10262991611,3cyclictest13450irq/24-ahci[00000:04:332
1026299160,2cyclictest33-21ksoftirqd/222:56:392
273522150,0sleep60-21swapper/622:25:356
193172150,0sleep20-21swapper/221:41:262
1029499152,5cyclictest15320-21grepconf.sh23:16:017
10294991514,1cyclictest73-21ksoftirqd/723:13:077
10294991513,1cyclictest73-21ksoftirqd/700:24:357
1028399158,7cyclictest0-21swapper/521:45:295
1028399157,2cyclictest0-21swapper/523:07:015
10283991512,2cyclictest57-21ksoftirqd/523:29:515
1028399150,13cyclictest1072-21sshd23:22:595
1027799150,15cyclictest99-21kswapd020:50:094
1026299159,4cyclictest33-21ksoftirqd/223:01:152
1026299152,1cyclictest33-21ksoftirqd/222:29:262
10262991513,1cyclictest33-21ksoftirqd/222:51:062
10262991513,1cyclictest33-21ksoftirqd/221:44:342
10262991513,1cyclictest33-21ksoftirqd/221:35:512
10262991513,1cyclictest33-21ksoftirqd/221:14:312
10262991513,1cyclictest33-21ksoftirqd/221:03:552
10262991513,1cyclictest33-21ksoftirqd/200:28:562
10262991511,3cyclictest33-21ksoftirqd/223:26:362
10262991511,1cyclictest33-21ksoftirqd/200:15:382
10262991510,3cyclictest33-21ksoftirqd/222:24:422
1026299151,2cyclictest131rcu_sched21:26:462
1029499148,5cyclictest9249-21grep23:04:457
1029499148,5cyclictest28034-21sshd00:09:477
1029499148,5cyclictest0-21swapper/721:32:027
1029499146,8cyclictest0-21swapper/722:25:057
1029499146,8cyclictest0-21swapper/720:57:347
1029499146,8cyclictest0-21swapper/720:25:347
1029499146,0cyclictest0-21swapper/720:33:387
10294991412,1cyclictest73-21ksoftirqd/723:41:497
10294991412,1cyclictest73-21ksoftirqd/721:23:287
10283991411,2cyclictest22690-21grepconf.sh23:51:505
10277991413,1cyclictest49-21ksoftirqd/422:45:144
10265991413,1cyclictest41-21ksoftirqd/322:17:343
1026599140,14cyclictest17981-21kworker/3:123:30:233
1026299147,4cyclictest33-21ksoftirqd/222:36:542
1026299147,4cyclictest33-21ksoftirqd/222:36:542
10262991413,1cyclictest33-21ksoftirqd/223:16:552
10262991413,1cyclictest33-21ksoftirqd/221:30:432
10262991413,1cyclictest33-21ksoftirqd/220:53:062
10262991412,1cyclictest33-21ksoftirqd/223:23:382
10262991412,1cyclictest33-21ksoftirqd/223:12:362
10262991412,1cyclictest33-21ksoftirqd/222:09:302
10262991412,1cyclictest33-21ksoftirqd/222:04:382
10262991412,1cyclictest33-21ksoftirqd/200:23:392
327032130,0sleep60-21swapper/619:34:376
200642130,0sleep40-21swapper/421:14:214
170032130,0sleep60-21swapper/622:43:536
103532130,0sleep60-21swapper/623:07:396
1029499138,5cyclictest0-21swapper/723:33:457
1029499138,5cyclictest0-21swapper/722:14:417
1029499138,5cyclictest0-21swapper/722:11:417
1029499138,5cyclictest0-21swapper/720:13:387
1029499138,2cyclictest0-21swapper/719:15:567
1029499138,1cyclictest14895-21kworker/7:321:16:547
1029499138,0cyclictest0-21swapper/721:38:217
1029499138,0cyclictest0-21swapper/720:35:067
1029499138,0cyclictest0-21swapper/719:56:227
1029499137,5cyclictest16714-21systemd-cgroups21:44:017
1029499132,5cyclictest0-21swapper/723:55:177
10294991313,0cyclictest0-21swapper/721:06:287
10294991312,1cyclictest73-21ksoftirqd/722:40:047
1029499131,1cyclictest24512-21sshd22:03:417
1029099137,6cyclictest0-21swapper/623:45:096
1028399135,2cyclictest0-21swapper/519:14:505
1028399135,2cyclictest0-21swapper/500:07:135
1028399135,1cyclictest28102-21nfsd21:22:285
10283991312,1cyclictest57-21ksoftirqd/523:34:435
10283991312,1cyclictest57-21ksoftirqd/523:34:435
10283991311,2cyclictest5449-21kworker/u16:100:03:195
1028399130,12cyclictest16359-21kworker/5:321:04:345
10265991312,1cyclictest41-21ksoftirqd/323:26:123
10265991312,1cyclictest41-21ksoftirqd/322:54:583
10265991312,1cyclictest41-21ksoftirqd/321:21:433
10265991311,1cyclictest41-21ksoftirqd/323:35:503
10265991311,1cyclictest41-21ksoftirqd/323:35:503
10265991311,1cyclictest41-21ksoftirqd/323:03:193
10265991311,1cyclictest41-21ksoftirqd/322:49:323
10262991311,1cyclictest33-21ksoftirqd/222:45:542
325842120,0sleep50-21swapper/521:39:455
286102120,0sleep00-21swapper/000:26:460
262882120,0sleep40-21swapper/422:14:354
183192120,0sleep40-21swapper/422:11:154
140542120,2sleep61029099cyclictest23:56:496
1029499127,5cyclictest0-21swapper/720:41:407
1029499127,5cyclictest0-21swapper/720:16:427
1029499127,5cyclictest0-21swapper/720:04:527
1029499127,5cyclictest0-21swapper/719:20:467
1029499127,5cyclictest0-21swapper/719:09:247
1029499127,1cyclictest859-21polkitd23:20:117
1029499127,0cyclictest0-21swapper/723:27:257
1029499127,0cyclictest0-21swapper/719:35:307
1029499127,0cyclictest0-21swapper/719:30:027
1029499127,0cyclictest0-21swapper/719:05:147
1029499125,6cyclictest27736-21grep00:03:557
1029499124,7cyclictest684-21nscd20:00:067
1029499124,7cyclictest684-21nscd20:00:067
10294991211,1cyclictest73-21ksoftirqd/723:45:367
10294991211,1cyclictest73-21ksoftirqd/721:25:347
10294991210,1cyclictest73-21ksoftirqd/722:19:567
1029099122,4cyclictest1692-21sshd21:29:086
1029099121,5cyclictest10614-21rm21:32:326
1029099121,10cyclictest758-21sshd23:50:426
1028399129,3cyclictest57-21ksoftirqd/521:04:105
1028399129,1cyclictest57-21ksoftirqd/522:22:165
1028399128,0cyclictest0-21swapper/519:53:465
1028399127,5cyclictest0-21swapper/519:43:465
1028399126,1cyclictest57-21ksoftirqd/521:11:185
1028399124,8cyclictest14262-21kworker/5:122:27:075
1028399124,1cyclictest11647-21bash22:56:575
10283991211,1cyclictest57-21ksoftirqd/523:00:025
10283991211,1cyclictest57-21ksoftirqd/522:51:165
10283991211,1cyclictest57-21ksoftirqd/521:16:105
10283991210,1cyclictest57-21ksoftirqd/500:15:445
1028399120,6cyclictest0-21swapper/521:32:375
1027799122,4cyclictest861-21gdbus21:21:184
1027799122,4cyclictest10194-21sshd21:08:014
1027799122,4cyclictest1-21systemd00:11:474
1026599126,5cyclictest41-21ksoftirqd/322:35:233
1026599126,5cyclictest41-21ksoftirqd/322:35:233
10265991211,1cyclictest41-21ksoftirqd/323:21:433
10265991211,1cyclictest41-21ksoftirqd/322:21:203
10265991210,1cyclictest41-21ksoftirqd/323:45:153
10265991210,1cyclictest41-21ksoftirqd/322:05:473
1025399122,4cyclictest0-21swapper/023:44:450
1029499117,0cyclictest0-21swapper/719:50:527
1029499117,0cyclictest0-21swapper/719:03:307
1029499116,5cyclictest0-21swapper/721:09:147
1029499116,5cyclictest0-21swapper/720:53:187
1029499116,5cyclictest0-21swapper/720:44:427
1029499116,4cyclictest0-21swapper/721:01:067
1029499116,0cyclictest0-21swapper/700:21:297
1029499115,5cyclictest17724-21kworker/7:123:00:117
10294991110,1cyclictest13780-21kworker/7:500:15:137
1029499110,5cyclictest0-21swapper/719:47:347
1029099118,2cyclictest28787-21bash23:33:236
1029099112,4cyclictest0-21swapper/623:28:286
1029099112,3cyclictest19337-21bash00:12:356
1029099111,4cyclictest861-21gdbus22:13:176
1029099110,5cyclictest7977-21bash22:32:066
1029099110,10cyclictest17424-40dnf21:08:396
1029099110,10cyclictest0-21swapper/619:04:356
1029099110,1cyclictest0-21swapper/623:00:266
1028399119,1cyclictest57-21ksoftirqd/522:30:185
1028399119,1cyclictest57-21ksoftirqd/500:23:075
1028399116,5cyclictest0-21swapper/520:10:585
1028399116,0cyclictest0-21swapper/520:47:105
1028399110,11cyclictest0-21swapper/500:25:545
1028399110,10cyclictest668-21systemd-logind21:27:005
1027799119,1cyclictest49-21ksoftirqd/422:53:284
1027799112,4cyclictest16587-21sshd22:43:514
10277991110,1cyclictest49-21ksoftirqd/421:38:114
1027799110,10cyclictest28493-21sshd23:35:574
1027799110,10cyclictest28493-21sshd23:35:574
1027799110,1cyclictest0-21swapper/423:00:294
1026599119,1cyclictest859-21polkitd00:23:003
1026599119,1cyclictest41-21ksoftirqd/323:58:423
1026599119,1cyclictest41-21ksoftirqd/322:02:303
1026599119,1cyclictest41-21ksoftirqd/321:14:323
1026599119,1cyclictest41-21ksoftirqd/300:03:453
10265991110,1cyclictest41-21ksoftirqd/300:16:083
1026599110,11cyclictest26759-21grepconf.sh22:33:453
1026299119,2cyclictest33-21ksoftirqd/220:09:132
1025799119,1cyclictest25-21ksoftirqd/123:50:271
1025799118,2cyclictest25-21ksoftirqd/122:44:211
1025799112,4cyclictest14997-21sshd22:57:151
10257991110,1cyclictest25-21ksoftirqd/121:57:121
10257991110,1cyclictest25-21ksoftirqd/121:48:251
1025799110,10cyclictest0-21swapper/121:59:231
1025799110,10cyclictest0-21swapper/121:50:141
1025399119,1cyclictest9-21ksoftirqd/023:03:060
1025399119,1cyclictest9-21ksoftirqd/022:32:570
1025399111,4cyclictest31580-21cp23:58:270
1025399110,5cyclictest6375-21grepconf.sh23:26:040
1025399110,10cyclictest4588-21sshd21:48:230
1025399110,10cyclictest23147-21sshd00:16:160
43772100,0sleep40-21swapper/422:04:344
1029499108,1cyclictest73-21ksoftirqd/722:34:307
1029499108,1cyclictest73-21ksoftirqd/722:34:307
1029499100,9cyclictest15760-21sshd22:49:137
1029499100,9cyclictest0-21swapper/719:39:547
1029099109,1cyclictest0-21swapper/623:19:026
1029099108,1cyclictest85350irq/25-em1-rx-021:20:406
1029099107,2cyclictest2321-21grepconf.sh23:20:186
1029099103,1cyclictest24708-21sshd22:03:426
1029099102,2cyclictest30768-21bash00:24:056
1029099100,4cyclictest1111-21grep22:36:596
1029099100,4cyclictest1111-21grep22:36:596
1028399109,1cyclictest57-21ksoftirqd/523:41:185
1028399109,1cyclictest57-21ksoftirqd/523:24:405
1028399108,2cyclictest0-21swapper/523:17:255
1028399108,2cyclictest0-21swapper/522:09:455
1028399108,2cyclictest0-21swapper/520:38:025
1028399108,2cyclictest0-21swapper/520:24:465
1028399108,2cyclictest0-21swapper/520:07:465
1028399108,2cyclictest0-21swapper/519:54:505
1028399108,1cyclictest13870-21awk19:59:305
1028399108,1cyclictest13870-21awk19:59:305
1028399108,0cyclictest0-21swapper/520:29:505
1028399102,6cyclictest0-21swapper/521:59:455
1028399102,0cyclictest0-21swapper/519:37:585
1028399102,0cyclictest0-21swapper/519:32:285
1028399102,0cyclictest0-21swapper/519:12:145
1027799109,1cyclictest740-21gdbus23:12:054
1027799109,1cyclictest49-21ksoftirqd/422:37:154
1027799109,1cyclictest49-21ksoftirqd/422:37:154
1027799103,6cyclictest7685-21sshd22:26:414
1027799103,6cyclictest15223-21sshd21:51:564
1027799103,1cyclictest29873-21sshd00:04:074
1027799100,9cyclictest30284-21sshd23:25:204
1027799100,4cyclictest11054-21sshd23:39:544
1026599109,1cyclictest41-21ksoftirqd/323:17:023
1026599109,1cyclictest41-21ksoftirqd/322:25:383
1026599109,1cyclictest41-21ksoftirqd/321:58:183
1026599108,1cyclictest41-21ksoftirqd/322:42:313
1026599108,1cyclictest41-21ksoftirqd/321:08:263
1026599108,1cyclictest41-21ksoftirqd/300:07:503
1026599107,1cyclictest41-21ksoftirqd/323:08:213
1026599102,5cyclictest669-21avahi-daemon22:46:293
1026299109,1cyclictest33-21ksoftirqd/220:59:122
1025799109,1cyclictest25-21ksoftirqd/122:32:361
1025799108,2cyclictest0-21swapper/100:22:321
1025799108,1cyclictest25-21ksoftirqd/123:48:441
1025799108,1cyclictest25-21ksoftirqd/122:36:411
1025799108,1cyclictest25-21ksoftirqd/122:36:411
1025799108,1cyclictest25-21ksoftirqd/100:04:111
1025799107,2cyclictest25-21ksoftirqd/122:40:371
1025799106,2cyclictest25-21ksoftirqd/100:26:471
1025799105,2cyclictest1782-21grep21:12:471
1025799104,3cyclictest11869-21cp21:40:461
1025799100,10cyclictest0-21swapper/100:09:361
1025399108,1cyclictest859-21polkitd21:55:380
1025399108,1cyclictest22484-21sshd22:27:570
1025399103,6cyclictest17149-21bash00:08:500
1025399103,4cyclictest9304-21/usr/sbin/munin19:54:120
1025399103,4cyclictest26153-21bash22:44:310
1025399102,3cyclictest7969-21sshd23:36:590
1025399102,3cyclictest7969-21sshd23:36:590
1025399101,4cyclictest22063-21sshd23:19:230
1025399101,3cyclictest0-21swapper/022:19:450
1025399100,4cyclictest17855-21sshd21:05:510
1025399100,4cyclictest1778-21sshd23:12:170
1025399100,4cyclictest0-21swapper/021:20:080
1025399100,10cyclictest0-21swapper/000:21:190
102909997,1cyclictest65-21ksoftirqd/623:39:306
102909997,1cyclictest65-21ksoftirqd/622:15:496
102909997,1cyclictest65-21ksoftirqd/621:49:306
102909993,6cyclictest0-21swapper/622:44:506
102909993,5cyclictest7982-21sshd21:13:196
102909991,7cyclictest4641-21kworker/6:123:09:506
102909991,3cyclictest0-21swapper/621:40:066
102909990,9cyclictest0-21swapper/621:16:516
102909990,0cyclictest0-21swapper/622:05:156
102839998,0cyclictest0-21swapper/520:15:465
102839997,2cyclictest0-21swapper/519:07:505
102839997,2cyclictest0-21swapper/519:03:565
102839997,0cyclictest0-21swapper/520:50:185
102839997,0cyclictest0-21swapper/520:40:105
102839997,0cyclictest0-21swapper/519:46:105
102839996,2cyclictest14608-21grepconf.sh23:45:395
102839996,1cyclictest57-21ksoftirqd/521:50:315
102839993,5cyclictest668-21systemd-logind23:10:215
102839990,9cyclictest0-21swapper/520:55:545
102779998,1cyclictest49-21ksoftirqd/423:17:214
102779998,1cyclictest481ktimersoftd/421:40:064
102779997,1cyclictest49-21ksoftirqd/421:25:154
102779993,4cyclictest20893-21grep21:57:544
102779992,6cyclictest29709-21sshd00:23:594
102779992,6cyclictest13379-21sshd22:32:344
102779990,9cyclictest0-21swapper/422:03:154
102779990,4cyclictest19170-21bash23:21:454
102659998,1cyclictest41-21ksoftirqd/321:49:313
102659998,1cyclictest41-21ksoftirqd/300:24:433
102659997,1cyclictest41-21ksoftirqd/323:51:503
102659997,1cyclictest41-21ksoftirqd/321:34:253
102659996,1cyclictest41-21ksoftirqd/323:09:253
102629997,2cyclictest33-21ksoftirqd/219:29:332
102629997,1cyclictest33-21ksoftirqd/219:54:122
102629997,1cyclictest33-21ksoftirqd/219:44:342
102579998,1cyclictest25-21ksoftirqd/123:40:531
102579998,1cyclictest25-21ksoftirqd/122:24:551
102579997,1cyclictest25-21ksoftirqd/123:56:561
102579997,1cyclictest25-21ksoftirqd/123:29:521
102579997,1cyclictest25-21ksoftirqd/122:10:521
102579997,1cyclictest25-21ksoftirqd/100:15:071
102579993,4cyclictest18044-21bash22:52:051
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional