You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-28 - 18:10
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Sat Jun 28, 2025 12:43:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
325321020,0sleep10-21swapper/111:17:281
300772580,0sleep70-21swapper/711:23:037
139952510,0sleep20-21swapper/211:03:442
1932324924,8sleep40-21swapper/406:58:424
122482490,0sleep00-21swapper/012:22:060
1930724725,4sleep60-21swapper/606:58:286
295052460,0sleep00-21swapper/010:43:230
1931724621,9sleep70-21swapper/706:58:377
87322450,0sleep30-21swapper/311:44:533
56062450,0sleep50-21swapper/510:50:255
1933024534,8sleep20-21swapper/206:58:482
1935124433,8sleep50-21swapper/506:59:075
142962430,0sleep20-21swapper/212:07:272
104862430,1sleep610485-21bash09:17:366
40762420,0sleep70-21swapper/709:26:027
1927924129,8sleep00-21swapper/006:58:060
301962400,0sleep60-21swapper/611:51:216
203332400,1sleep5763-21runrttasks11:55:285
1905624028,9sleep30-21swapper/306:55:043
66432390,0sleep60-21swapper/612:14:206
44512390,0sleep40-21swapper/410:44:084
31932390,0sleep60-21swapper/610:53:236
307862390,0sleep00-21swapper/010:21:210
249102390,0sleep10-21swapper/109:09:541
227902390,0sleep60-21swapper/611:19:256
1907423927,9sleep10-21swapper/106:55:181
60882380,0sleep40-21swapper/409:20:064
212392370,0sleep30-21swapper/310:58:083
19531992620,6cyclictest73-21ksoftirqd/710:35:447
229902240,0sleep70-21swapper/710:07:437
1953199226,2cyclictest73-21ksoftirqd/709:07:237
1950699222,19cyclictest17364-21bash11:00:411
1952299210,9cyclictest0-21swapper/511:05:325
19513992110,1cyclictest41-21ksoftirqd/312:22:543
19506992118,1cyclictest28104-21nfsd12:24:261
19531992015,3cyclictest73-21ksoftirqd/710:03:387
1953199192,3cyclictest73-21ksoftirqd/712:18:457
19531991916,3cyclictest73-21ksoftirqd/709:15:147
19531991915,1cyclictest73-21ksoftirqd/710:56:197
19531991914,1cyclictest73-21ksoftirqd/709:54:587
1952299198,0cyclictest0-21swapper/512:17:245
19513991916,3cyclictest41-21ksoftirqd/310:07:393
19513991916,1cyclictest41-21ksoftirqd/310:50:303
19513991915,3cyclictest41-21ksoftirqd/311:21:443
19513991915,1cyclictest41-21ksoftirqd/312:26:563
19513991914,3cyclictest41-21ksoftirqd/312:14:133
19513991914,3cyclictest41-21ksoftirqd/309:13:023
19513991913,4cyclictest41-21ksoftirqd/309:29:393
1950699191,17cyclictest711-21dbus-daemon10:10:211
19531991816,1cyclictest73-21ksoftirqd/710:41:377
19531991814,3cyclictest73-21ksoftirqd/710:10:337
19531991812,1cyclictest73-21ksoftirqd/712:07:487
19531991812,1cyclictest73-21ksoftirqd/711:53:507
1952299187,5cyclictest0-21swapper/509:25:085
19514991814,3cyclictest49-21ksoftirqd/410:30:164
19514991814,2cyclictest49-21ksoftirqd/409:04:374
19513991817,1cyclictest41-21ksoftirqd/309:04:593
19513991816,1cyclictest41-21ksoftirqd/311:55:043
19513991816,1cyclictest41-21ksoftirqd/310:32:253
19513991816,1cyclictest41-21ksoftirqd/309:03:503
19513991812,4cyclictest41-21ksoftirqd/311:12:523
259482170,0sleep00-21swapper/009:28:030
19531991715,1cyclictest73-21ksoftirqd/711:49:347
19531991715,1cyclictest73-21ksoftirqd/711:36:517
19531991715,1cyclictest73-21ksoftirqd/710:20:467
19531991714,3cyclictest73-21ksoftirqd/709:29:467
19531991714,1cyclictest73-21ksoftirqd/709:49:077
19531991713,1cyclictest73-21ksoftirqd/711:04:577
19531991712,1cyclictest73-21ksoftirqd/710:14:427
19531991712,0cyclictest73-21ksoftirqd/711:54:407
19525991716,1cyclictest85350irq/25-em1-rx-010:29:346
19525991716,1cyclictest85350irq/25-em1-rx-010:29:346
1951399175,3cyclictest41-21ksoftirqd/310:36:433
19513991715,1cyclictest41-21ksoftirqd/312:04:043
19513991715,1cyclictest41-21ksoftirqd/311:08:593
19513991715,1cyclictest41-21ksoftirqd/310:22:183
19513991715,1cyclictest41-21ksoftirqd/309:59:413
19513991715,1cyclictest41-21ksoftirqd/309:49:433
19513991715,1cyclictest41-21ksoftirqd/309:23:343
19513991714,3cyclictest41-21ksoftirqd/309:15:243
19513991713,3cyclictest41-21ksoftirqd/312:07:273
19513991713,2cyclictest41-21ksoftirqd/310:45:163
19513991712,2cyclictest41-21ksoftirqd/311:00:023
19513991711,4cyclictest41-21ksoftirqd/310:16:133
19513991711,4cyclictest41-21ksoftirqd/310:12:483
73252160,0sleep10-21swapper/109:56:551
1953199168,1cyclictest73-21ksoftirqd/712:20:357
19531991614,1cyclictest73-21ksoftirqd/712:25:307
19531991614,1cyclictest73-21ksoftirqd/711:31:047
19531991614,1cyclictest73-21ksoftirqd/711:10:097
19531991614,1cyclictest73-21ksoftirqd/709:34:537
19531991614,1cyclictest73-21ksoftirqd/709:14:177
19531991613,3cyclictest73-21ksoftirqd/709:24:047
19531991612,4cyclictest73-21ksoftirqd/709:42:067
19531991612,3cyclictest73-21ksoftirqd/711:42:097
19531991612,1cyclictest73-21ksoftirqd/711:27:457
19531991612,1cyclictest73-21ksoftirqd/711:27:457
19531991612,1cyclictest73-21ksoftirqd/709:52:497
19531991611,3cyclictest73-21ksoftirqd/711:15:157
1953199160,1cyclictest121rcu_preempt10:30:367
1952299166,0cyclictest0-21swapper/507:44:595
1952299165,5cyclictest0-21swapper/507:25:575
19514991614,1cyclictest49-21ksoftirqd/412:10:114
19514991613,1cyclictest49-21ksoftirqd/411:59:394
1951399168,1cyclictest41-21ksoftirqd/311:14:543
1951399166,1cyclictest41-21ksoftirqd/309:40:383
19513991614,1cyclictest41-21ksoftirqd/312:16:123
19513991614,1cyclictest41-21ksoftirqd/311:44:393
19513991614,1cyclictest41-21ksoftirqd/311:29:563
19513991614,1cyclictest41-21ksoftirqd/310:40:583
19513991614,1cyclictest41-21ksoftirqd/309:56:013
19513991614,1cyclictest41-21ksoftirqd/309:46:333
19513991614,1cyclictest41-21ksoftirqd/309:35:023
19513991614,1cyclictest41-21ksoftirqd/309:30:173
19513991612,3cyclictest41-21ksoftirqd/311:29:223
19513991612,3cyclictest41-21ksoftirqd/311:29:223
1951099160,15cyclictest15558-21sshd09:21:022
275502150,1sleep41951499cyclictest09:46:094
19531991513,1cyclictest73-21ksoftirqd/712:10:007
19531991513,1cyclictest73-21ksoftirqd/712:00:077
19531991513,1cyclictest73-21ksoftirqd/711:04:197
19531991513,1cyclictest73-21ksoftirqd/710:51:167
19531991513,1cyclictest73-21ksoftirqd/710:25:017
19531991513,1cyclictest73-21ksoftirqd/710:25:017
1953199150,1cyclictest121rcu_preempt10:44:547
1952299158,6cyclictest32224-21sshd10:34:325
1952299155,5cyclictest2691-21kworker/u16:112:00:205
1952299154,5cyclictest0-21swapper/510:38:465
1952299150,9cyclictest0-21swapper/511:53:165
1952299150,0cyclictest0-21swapper/507:09:535
19514991514,1cyclictest49-21ksoftirqd/410:56:244
19514991513,1cyclictest49-21ksoftirqd/410:08:014
19514991513,1cyclictest49-21ksoftirqd/409:34:264
19514991512,2cyclictest49-21ksoftirqd/411:52:214
19513991514,1cyclictest41-21ksoftirqd/310:24:483
19513991514,1cyclictest41-21ksoftirqd/310:24:483
19513991511,2cyclictest41-21ksoftirqd/311:52:323
19513991510,1cyclictest41-21ksoftirqd/311:34:523
1950599150,13cyclictest7355-21sshd10:19:180
215922140,0sleep60-21swapper/610:39:376
1953199140,1cyclictest121rcu_preempt09:02:287
1952299148,6cyclictest0-21swapper/509:13:225
1952299148,5cyclictest0-21swapper/507:00:035
1952299148,0cyclictest0-21swapper/511:19:385
1952299144,4cyclictest0-21swapper/509:45:085
1952299142,11cyclictest1-21systemd12:24:265
1952299142,0cyclictest0-21swapper/512:11:505
1952299141,11cyclictest18678-21sshd09:42:185
1952299140,2cyclictest0-21swapper/510:17:185
1952299140,14cyclictest0-21swapper/511:27:065
1952299140,14cyclictest0-21swapper/511:27:065
1951499149,3cyclictest49-21ksoftirqd/412:29:004
1951499146,5cyclictest49-21ksoftirqd/410:19:084
19514991413,1cyclictest49-21ksoftirqd/409:44:234
19514991412,2cyclictest49-21ksoftirqd/411:25:494
19514991412,2cyclictest49-21ksoftirqd/411:25:494
19514991412,1cyclictest49-21ksoftirqd/411:46:434
19514991412,1cyclictest49-21ksoftirqd/411:19:394
19514991412,1cyclictest49-21ksoftirqd/411:05:374
19514991412,1cyclictest49-21ksoftirqd/410:27:004
19514991412,1cyclictest49-21ksoftirqd/410:27:004
19514991412,1cyclictest49-21ksoftirqd/409:55:004
19513991412,1cyclictest41-21ksoftirqd/308:24:393
176192140,0sleep20-21swapper/210:00:492
19531991310,0cyclictest73-21ksoftirqd/707:38:077
19531991310,0cyclictest73-21ksoftirqd/707:38:077
19525991312,1cyclictest85350irq/25-em1-rx-008:24:596
1952599130,7cyclictest0-21swapper/609:30:366
1952299138,5cyclictest0-21swapper/512:07:205
1952299138,5cyclictest0-21swapper/511:36:165
1952299138,5cyclictest0-21swapper/511:22:305
1952299138,5cyclictest0-21swapper/510:46:085
1952299138,5cyclictest0-21swapper/510:21:565
1952299138,5cyclictest0-21swapper/510:03:005
1952299138,5cyclictest0-21swapper/509:34:245
1952299138,5cyclictest0-21swapper/509:15:485
1952299138,5cyclictest0-21swapper/509:06:355
1952299138,5cyclictest0-21swapper/508:52:145
1952299138,5cyclictest0-21swapper/508:52:145
1952299138,5cyclictest0-21swapper/508:44:555
1952299138,5cyclictest0-21swapper/508:37:195
1952299138,5cyclictest0-21swapper/508:37:195
1952299138,5cyclictest0-21swapper/508:16:015
1952299138,5cyclictest0-21swapper/508:04:495
1952299138,5cyclictest0-21swapper/508:01:155
1952299138,5cyclictest0-21swapper/507:32:095
1952299138,5cyclictest0-21swapper/507:04:495
1952299138,4cyclictest20434-21cp10:07:165
1952299138,0cyclictest0-21swapper/511:30:485
1952299138,0cyclictest0-21swapper/507:53:195
1952299138,0cyclictest0-21swapper/507:24:355
1952299137,6cyclictest0-21swapper/507:36:095
1952299137,6cyclictest0-21swapper/507:36:095
1952299137,5cyclictest19010-21grep11:12:545
1952299135,5cyclictest0-21swapper/510:41:265
1952299133,5cyclictest711-21dbus-daemon10:10:505
1952299131,7cyclictest0-21swapper/509:21:005
1952299130,6cyclictest0-21swapper/511:44:505
1951499138,3cyclictest49-21ksoftirqd/410:00:004
19514991312,1cyclictest49-21ksoftirqd/411:14:234
19514991312,1cyclictest49-21ksoftirqd/410:50:164
19514991312,1cyclictest49-21ksoftirqd/409:34:454
19514991312,1cyclictest49-21ksoftirqd/409:28:104
19514991311,1cyclictest49-21ksoftirqd/412:23:004
19514991311,1cyclictest49-21ksoftirqd/412:07:554
19514991311,1cyclictest49-21ksoftirqd/412:01:054
19514991311,1cyclictest49-21ksoftirqd/411:40:434
19514991311,1cyclictest49-21ksoftirqd/411:34:454
19514991311,1cyclictest49-21ksoftirqd/411:33:454
19514991311,1cyclictest49-21ksoftirqd/411:04:324
19514991311,1cyclictest49-21ksoftirqd/409:19:164
1951499130,2cyclictest131rcu_sched12:19:084
1951499130,2cyclictest131rcu_sched11:24:114
19513991312,1cyclictest41-21ksoftirqd/307:34:553
19513991312,1cyclictest41-21ksoftirqd/307:34:553
19513991311,1cyclictest41-21ksoftirqd/308:39:393
19513991311,1cyclictest41-21ksoftirqd/308:39:393
19513991311,1cyclictest41-21ksoftirqd/307:49:563
19506991311,2cyclictest17564-21bash10:20:031
107482130,0sleep00-21swapper/011:24:240
60132120,0sleep50-21swapper/510:25:195
60132120,0sleep50-21swapper/510:25:195
54672120,0sleep60-21swapper/610:05:496
1953199128,4cyclictest73-21ksoftirqd/707:57:077
1953199127,0cyclictest73-21ksoftirqd/707:15:157
19531991210,1cyclictest73-21ksoftirqd/707:34:397
1952599120,12cyclictest0-21swapper/612:28:246
1952299127,5cyclictest0-21swapper/509:56:585
1952299127,5cyclictest0-21swapper/509:38:405
1952299127,5cyclictest0-21swapper/508:56:535
1952299127,5cyclictest0-21swapper/508:26:435
1952299127,1cyclictest15326-21sshd11:42:505
1952299127,0cyclictest0-21swapper/507:55:235
1952299126,5cyclictest20796-21bash11:04:225
1952299121,5cyclictest0-21swapper/509:02:435
1952299120,12cyclictest10565-21kworker/5:112:27:085
1952299120,12cyclictest0-21swapper/507:40:555
1952299120,0cyclictest0-21swapper/507:17:095
19514991211,1cyclictest49-21ksoftirqd/409:09:434
19514991210,1cyclictest49-21ksoftirqd/410:49:064
19514991210,1cyclictest49-21ksoftirqd/410:19:584
19514991210,1cyclictest49-21ksoftirqd/409:08:204
19513991211,1cyclictest41-21ksoftirqd/307:20:003
19513991210,1cyclictest41-21ksoftirqd/308:44:543
19513991210,1cyclictest41-21ksoftirqd/307:09:393
1951099128,4cyclictest33-21ksoftirqd/210:43:152
1951099128,3cyclictest33-21ksoftirqd/210:04:402
19510991211,1cyclictest33-21ksoftirqd/210:14:242
19510991211,1cyclictest33-21ksoftirqd/209:16:362
19510991210,1cyclictest33-21ksoftirqd/210:19:152
1951099120,11cyclictest25451-21bash11:58:272
195072120,0sleep10-21swapper/111:55:241
1950699122,4cyclictest26392-21sshd11:53:361
1950599127,4cyclictest9-21ksoftirqd/010:51:020
183002120,0sleep10-21swapper/111:09:591
308042110,2sleep11950699cyclictest09:25:321
228112110,1sleep622812-21grep11:13:176
1953199117,4cyclictest73-21ksoftirqd/708:45:417
1953199117,4cyclictest73-21ksoftirqd/708:26:217
1953199117,4cyclictest73-21ksoftirqd/707:49:437
1953199110,11cyclictest0-21swapper/707:11:317
1952599119,1cyclictest65-21ksoftirqd/612:06:586
1952599119,1cyclictest65-21ksoftirqd/611:06:156
1952599119,1cyclictest65-21ksoftirqd/609:34:426
19525991110,1cyclictest65-21ksoftirqd/611:48:156
19525991110,0cyclictest85350irq/25-em1-rx-008:15:036
1952299117,0cyclictest0-21swapper/510:59:325
1952299116,5cyclictest0-21swapper/509:51:105
1952299116,0cyclictest0-21swapper/508:40:355
1952299116,0cyclictest0-21swapper/508:31:035
1952299116,0cyclictest0-21swapper/508:10:455
1951499119,2cyclictest49-21ksoftirqd/410:10:344
1951499119,1cyclictest49-21ksoftirqd/410:35:184
1951499119,1cyclictest49-21ksoftirqd/409:52:264
1951399119,1cyclictest41-21ksoftirqd/308:59:403
1951399119,1cyclictest41-21ksoftirqd/308:09:593
1951399119,1cyclictest41-21ksoftirqd/307:40:033
19513991110,1cyclictest41-21ksoftirqd/308:52:513
19513991110,1cyclictest41-21ksoftirqd/308:52:513
19513991110,1cyclictest41-21ksoftirqd/308:29:393
19513991110,1cyclictest41-21ksoftirqd/308:05:013
19513991110,1cyclictest41-21ksoftirqd/307:59:573
19513991110,1cyclictest41-21ksoftirqd/307:24:593
1951099119,1cyclictest33-21ksoftirqd/209:46:382
1951099119,1cyclictest33-21ksoftirqd/209:38:382
1951099112,4cyclictest25741-21bash10:46:102
19510991110,1cyclictest33-21ksoftirqd/212:27:242
19510991110,1cyclictest33-21ksoftirqd/212:11:092
19510991110,1cyclictest33-21ksoftirqd/212:00:572
1951099111,4cyclictest19892-21sshd12:17:462
1951099110,11cyclictest0-21swapper/210:56:012
1951099110,11cyclictest0-21swapper/209:34:212
1950699119,1cyclictest25-21ksoftirqd/111:30:451
1950699117,1cyclictest22963-21sshd11:25:201
1950699117,1cyclictest22963-21sshd11:25:201
1950699113,7cyclictest22378-21cp11:37:211
1950699112,4cyclictest8544-21systemd-cgroups12:26:461
1950699112,4cyclictest25215-21id10:36:401
1950699112,4cyclictest0-21swapper/112:15:531
1950699111,4cyclictest31138-21sshd10:09:041
1950699111,4cyclictest0-21swapper/109:39:491
1950599119,1cyclictest711-21dbus-daemon10:09:150
1950599118,2cyclictest17856-21bash11:27:550
1950599118,2cyclictest17856-21bash11:27:550
1950599117,3cyclictest9-21ksoftirqd/011:46:390
1950599112,5cyclictest0-21swapper/011:59:390
19505991110,1cyclictest0-21swapper/010:35:060
1950599110,10cyclictest711-21dbus-daemon11:08:030
1950599110,1cyclictest0-21swapper/012:08:170
225132100,0sleep10-21swapper/110:29:551
1952599108,1cyclictest65-21ksoftirqd/611:33:596
1952599103,5cyclictest1-21systemd11:35:516
1952599102,5cyclictest12676-21sshd09:41:436
1952599102,3cyclictest21139-21bash09:51:536
1952599100,5cyclictest24126-21sshd10:46:006
1951399109,1cyclictest41-21ksoftirqd/308:40:023
1951399109,1cyclictest41-21ksoftirqd/307:09:573
1951399108,1cyclictest41-21ksoftirqd/308:30:033
1951399107,1cyclictest41-21ksoftirqd/308:14:563
1951099109,1cyclictest0-21swapper/210:28:252
1951099109,1cyclictest0-21swapper/210:28:252
1951099108,1cyclictest33-21ksoftirqd/211:37:192
1951099108,1cyclictest33-21ksoftirqd/210:20:012
1951099108,1cyclictest28079-21systemd-cgroups12:23:212
1951099104,5cyclictest668-21systemd-logind11:14:012
1951099102,5cyclictest17454-21bash10:33:062
1951099101,8cyclictest566-21grepconf.sh09:56:062
1951099100,10cyclictest0-21swapper/211:44:442
1951099100,10cyclictest0-21swapper/211:21:142
1950699106,1cyclictest8583-21kworker/1:309:52:141
1950699103,6cyclictest3695-21grepconf.sh12:09:101
1950699103,6cyclictest12671-21fschecks_time10:44:551
1950699103,2cyclictest28696-21rm10:18:001
1950699102,7cyclictest17978-21sshd11:40:111
1950699101,4cyclictest8065-21sshd12:14:261
1950699101,4cyclictest23092-21sshd09:45:401
1950699100,4cyclictest6520-21sshd09:59:521
1950699100,4cyclictest4845-21bash11:23:501
1950699100,10cyclictest0-21swapper/111:45:131
1950699100,10cyclictest0-21swapper/109:34:511
1950599109,1cyclictest9-21ksoftirqd/012:00:150
1950599109,1cyclictest9-21ksoftirqd/011:50:120
1950599109,1cyclictest0-21swapper/007:24:390
1950599108,1cyclictest31508-21bash10:12:050
1950599104,3cyclictest711-21dbus-daemon09:05:070
1950599103,4cyclictest740-21gdbus09:12:200
1950599103,4cyclictest23445-21sshd09:45:420
1950599103,4cyclictest2158-21bash11:38:540
1950599103,4cyclictest20404-21sshd09:55:000
1950599100,10cyclictest20335-21kworker/0:212:15:120
1950599100,10cyclictest0-21swapper/010:58:170
31238296,1sleep09-21ksoftirqd/009:22:360
26870290,0sleep10-21swapper/110:43:081
195319998,1cyclictest73-21ksoftirqd/707:59:577
195319998,1cyclictest73-21ksoftirqd/707:00:047
195319997,1cyclictest73-21ksoftirqd/708:59:407
195319997,1cyclictest73-21ksoftirqd/708:34:557
195319997,1cyclictest73-21ksoftirqd/708:34:557
195319995,1cyclictest73-21ksoftirqd/708:29:497
195319991,0cyclictest73-21ksoftirqd/708:14:397
195259998,1cyclictest65-21ksoftirqd/611:59:136
195259997,1cyclictest65-21ksoftirqd/611:43:096
195259997,1cyclictest65-21ksoftirqd/609:28:256
195259994,4cyclictest711-21dbus-daemon10:41:316
195259994,1cyclictest1-21systemd09:04:596
195259993,3cyclictest23624-21bash10:20:386
195259992,6cyclictest19768-21grepconf.sh11:00:566
195259992,4cyclictest861-21gdbus11:23:406
195259992,4cyclictest4481-21sshd10:56:296
195259991,4cyclictest22324-21bash12:00:336
195259990,9cyclictest23888-21kworker/6:209:22:196
195259990,8cyclictest31164-21kworker/6:409:10:386
195229990,9cyclictest0-21swapper/508:19:455
195139998,1cyclictest41-21ksoftirqd/307:54:553
195139998,1cyclictest41-21ksoftirqd/307:44:593
195139998,1cyclictest41-21ksoftirqd/307:14:523
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional