You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-08-27 - 15:06
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Wed Aug 27, 2025 12:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
173412820,1sleep42768499cyclictest09:48:014
119462720,0sleep00-21swapper/010:22:080
43672660,0sleep70-21swapper/710:24:247
27472620,0sleep70-21swapper/711:12:057
142332550,1sleep714232-21grep09:22:417
92222520,2sleep02765399cyclictest10:05:580
137402520,0sleep10-21swapper/111:23:521
2729525139,8sleep40-21swapper/406:55:134
2746624926,8sleep30-21swapper/306:57:283
67622480,0sleep70-21swapper/709:43:477
65382480,0sleep60-21swapper/611:34:066
2665324728,13sleep60-21swapper/606:52:506
2730724624,18sleep70-21swapper/706:55:247
188142460,0sleep00-21swapper/009:20:010
281212450,0sleep50-21swapper/510:58:495
267222450,0sleep60-21swapper/612:18:116
147402450,0sleep40-21swapper/410:09:394
49192430,0sleep20-21swapper/211:36:352
173612430,0sleep10-21swapper/109:38:281
170742430,0sleep50-21swapper/511:13:075
170742430,0sleep50-21swapper/511:13:075
140482430,0sleep40-21swapper/411:18:184
10062430,0sleep50-21swapper/512:13:465
321272420,2sleep42768499cyclictest10:23:564
135072420,0sleep50-21swapper/511:49:435
8972410,0sleep30-21swapper/309:58:573
2746424128,9sleep10-21swapper/106:57:261
2499424129,8sleep00-21swapper/006:52:360
24352410,0sleep20-21swapper/211:51:242
212642410,0sleep50-21swapper/511:10:515
2727124029,8sleep20-21swapper/206:54:552
301352390,0sleep30-21swapper/309:04:433
2740823928,8sleep50-21swapper/506:56:425
260992390,0sleep30-21swapper/308:47:293
259752390,0sleep20-21swapper/209:36:212
259752390,0sleep20-21swapper/209:36:212
250642390,0sleep30-21swapper/310:32:443
250642390,0sleep30-21swapper/310:32:443
12262390,0sleep10-21swapper/112:08:451
324732380,0sleep20-21swapper/210:05:042
318932380,0sleep0763-21runrttasks11:58:350
318932380,0sleep0763-21runrttasks11:58:350
186022380,0sleep60-21swapper/612:02:416
127742380,0sleep40-21swapper/410:15:404
204942370,0sleep40-21swapper/409:26:144
170662360,2sleep52769299cyclictest11:39:585
118792360,0sleep20-21swapper/211:06:552
314342240,0sleep2321ktimersoftd/211:43:312
27692992419,1cyclictest57-21ksoftirqd/510:10:515
27692992419,0cyclictest0-21swapper/507:15:305
2769299235,13cyclictest57-21ksoftirqd/509:01:335
27712992211,6cyclictest721ktimersoftd/707:09:507
27712992211,11cyclictest2899-21kworker/7:110:06:597
27692992217,1cyclictest57-21ksoftirqd/511:43:535
27692992217,1cyclictest57-21ksoftirqd/507:45:185
2771299214,0cyclictest0-21swapper/708:02:227
27712992110,0cyclictest0-21swapper/712:04:277
2769299214,11cyclictest28105-21nfsd10:13:475
27692992116,5cyclictest0-21swapper/507:07:105
27676992116,4cyclictest861-21gdbus10:42:443
2769299208,7cyclictest57-21ksoftirqd/511:55:015
27692992016,4cyclictest57-21ksoftirqd/508:07:305
27692992015,5cyclictest57-21ksoftirqd/510:52:195
27692992010,10cyclictest57-21ksoftirqd/509:15:155
2771299198,5cyclictest0-21swapper/708:52:517
2771299198,5cyclictest0-21swapper/707:27:067
2771299198,0cyclictest0-21swapper/710:14:437
2771299198,0cyclictest0-21swapper/709:34:517
2771299198,0cyclictest0-21swapper/709:34:517
2771299198,0cyclictest0-21swapper/709:00:297
2771299198,0cyclictest0-21swapper/708:40:107
2771299198,0cyclictest0-21swapper/708:40:107
2771299198,0cyclictest0-21swapper/708:32:287
2771299198,0cyclictest0-21swapper/708:14:187
2771299198,0cyclictest0-21swapper/708:05:347
2771299198,0cyclictest0-21swapper/707:27:547
2771299193,15cyclictest16500-21sshd11:39:557
2771299191,12cyclictest30211-21kworker/7:211:50:337
2769299199,9cyclictest57-21ksoftirqd/511:30:055
2769299198,5cyclictest57-21ksoftirqd/511:58:215
2769299198,5cyclictest57-21ksoftirqd/511:58:215
2769299198,5cyclictest0-21swapper/508:55:075
2769299198,0cyclictest0-21swapper/507:11:465
27692991918,1cyclictest57-21ksoftirqd/510:32:095
27692991913,3cyclictest57-21ksoftirqd/510:54:225
2769299190,1cyclictest21256-21kworker/u16:111:20:135
2771299188,5cyclictest0-21swapper/712:24:257
2771299188,0cyclictest0-21swapper/707:44:467
2771299187,5cyclictest0-21swapper/710:53:297
2771299187,0cyclictest0-21swapper/708:49:547
2771299187,0cyclictest0-21swapper/708:49:547
2771299187,0cyclictest0-21swapper/708:27:107
2771299187,0cyclictest0-21swapper/708:11:007
27712991814,0cyclictest0-21swapper/709:20:217
27712991813,0cyclictest0-21swapper/707:39:107
2769299188,4cyclictest57-21ksoftirqd/509:58:575
2769299188,4cyclictest57-21ksoftirqd/509:21:295
27692991815,2cyclictest57-21ksoftirqd/512:10:115
27692991815,1cyclictest57-21ksoftirqd/512:23:385
27692991813,1cyclictest57-21ksoftirqd/510:26:495
27692991813,1cyclictest57-21ksoftirqd/509:32:135
27692991813,0cyclictest0-21swapper/507:34:445
27692991813,0cyclictest0-21swapper/507:34:445
27692991811,6cyclictest57-21ksoftirqd/510:39:555
2771299177,5cyclictest0-21swapper/708:17:467
2771299176,5cyclictest0-21swapper/711:34:297
2771299176,5cyclictest0-21swapper/710:09:157
2771299176,5cyclictest0-21swapper/707:32:527
2771299176,5cyclictest0-21swapper/707:32:527
2771299176,0cyclictest0-21swapper/710:41:257
2771299176,0cyclictest0-21swapper/710:36:037
2771299176,0cyclictest0-21swapper/710:36:037
2771299176,0cyclictest0-21swapper/710:20:177
27704991715,1cyclictest85350irq/25-em1-rx-008:27:296
2769299179,8cyclictest57-21ksoftirqd/511:33:315
2769299176,1cyclictest17148-21kworker/5:111:27:095
27692991714,2cyclictest57-21ksoftirqd/512:04:005
27692991713,4cyclictest57-21ksoftirqd/509:48:335
27692991712,5cyclictest57-21ksoftirqd/509:09:075
27692991712,0cyclictest0-21swapper/508:38:425
27692991712,0cyclictest0-21swapper/508:38:425
27692991712,0cyclictest0-21swapper/508:19:385
27692991712,0cyclictest0-21swapper/507:53:385
27692991712,0cyclictest0-21swapper/507:41:305
27692991711,5cyclictest57-21ksoftirqd/510:36:355
27692991711,5cyclictest57-21ksoftirqd/510:36:355
27692991711,5cyclictest57-21ksoftirqd/510:03:455
27692991711,5cyclictest57-21ksoftirqd/509:06:595
27692991711,5cyclictest20071-21kworker/5:009:23:555
27692991711,1cyclictest8257-21kworker/5:108:14:265
27692991710,2cyclictest57-21ksoftirqd/509:36:055
27692991710,2cyclictest57-21ksoftirqd/509:36:055
2769299171,16cyclictest0-21swapper/508:23:065
2769299171,1cyclictest121rcu_preempt10:46:505
27684991715,1cyclictest49-21ksoftirqd/410:03:154
2771299165,5cyclictest0-21swapper/707:52:547
2771299165,0cyclictest0-21swapper/710:45:417
27712991612,0cyclictest0-21swapper/709:59:497
2771299160,4cyclictest0-21swapper/711:22:597
2771299160,10cyclictest0-21swapper/711:47:157
2769299166,5cyclictest57-21ksoftirqd/507:30:545
2769299165,6cyclictest57-21ksoftirqd/511:06:175
2769299164,1cyclictest57-21ksoftirqd/509:54:425
27692991614,1cyclictest57-21ksoftirqd/512:18:125
27692991612,3cyclictest57-21ksoftirqd/509:45:435
27692991611,5cyclictest57-21ksoftirqd/509:41:055
27692991611,5cyclictest57-21ksoftirqd/508:05:565
27692991611,5cyclictest0-21swapper/508:49:065
27692991611,5cyclictest0-21swapper/508:49:065
2768499163,12cyclictest16543-21sshd09:03:134
27676991614,1cyclictest41-21ksoftirqd/312:00:203
27676991614,1cyclictest41-21ksoftirqd/312:00:203
27372160,0sleep10-21swapper/110:14:401
222802160,0sleep40-21swapper/409:38:564
2771299158,6cyclictest13160-21sshd10:31:417
2771299154,5cyclictest0-21swapper/712:16:457
2771299154,0cyclictest0-21swapper/709:29:217
2771299154,0cyclictest0-21swapper/708:36:527
2771299154,0cyclictest0-21swapper/707:05:027
2771299154,0cyclictest0-21swapper/706:59:087
2771299153,11cyclictest11166-21grep11:18:037
2771299150,9cyclictest0-21swapper/711:01:537
2771299150,9cyclictest0-21swapper/708:45:477
2771299150,9cyclictest0-21swapper/707:48:267
27704991513,1cyclictest85350irq/25-em1-rx-011:59:236
27704991513,1cyclictest85350irq/25-em1-rx-011:59:236
2769299154,7cyclictest57-21ksoftirqd/507:00:065
27692991512,3cyclictest57-21ksoftirqd/510:18:085
27692991510,5cyclictest0-21swapper/507:59:105
27692991510,5cyclictest0-21swapper/507:48:265
27661991512,2cyclictest29554-21bash09:33:101
27661991512,2cyclictest29554-21bash09:33:101
27653991513,1cyclictest9-21ksoftirqd/009:09:540
2765399150,1cyclictest131rcu_sched09:38:380
111662150,0sleep60-21swapper/609:25:206
34822140,0sleep40-21swapper/410:17:564
2771299148,5cyclictest711-21dbus-daemon09:10:277
2771299148,5cyclictest0-21swapper/710:48:417
2771299148,5cyclictest0-21swapper/709:52:217
2771299143,11cyclictest0-21swapper/711:05:097
2771299142,6cyclictest32341-21bash09:14:257
2771299140,8cyclictest0-21swapper/707:18:127
2771299140,11cyclictest0-21swapper/707:15:367
2770499147,6cyclictest0-21swapper/609:42:096
27676991412,1cyclictest41-21ksoftirqd/312:11:503
27669991411,2cyclictest33-21ksoftirqd/212:25:082
27669991411,1cyclictest33-21ksoftirqd/209:30:102
27653991413,1cyclictest9-21ksoftirqd/009:42:430
27653991412,1cyclictest9-21ksoftirqd/010:13:170
207582140,0sleep30-21swapper/311:27:233
166602140,0sleep30-21swapper/311:13:053
166602140,0sleep30-21swapper/311:13:053
2771299138,5cyclictest0-21swapper/712:07:557
2771299138,5cyclictest0-21swapper/711:53:457
2771299138,5cyclictest0-21swapper/711:29:237
2771299138,5cyclictest0-21swapper/709:52:417
2771299138,5cyclictest0-21swapper/709:40:537
2771299138,5cyclictest0-21swapper/709:02:417
2771299132,5cyclictest0-21swapper/712:17:577
2771299131,0cyclictest0-21swapper/711:14:217
2771299131,0cyclictest0-21swapper/711:14:217
2769299138,5cyclictest0-21swapper/508:28:065
2769299138,0cyclictest0-21swapper/508:46:185
2769299138,0cyclictest0-21swapper/507:24:045
2769299138,0cyclictest0-21swapper/507:17:345
2768499138,3cyclictest49-21ksoftirqd/411:26:594
27684991311,2cyclictest12940-21grepconf.sh08:59:094
27684991311,1cyclictest49-21ksoftirqd/411:45:294
27676991312,1cyclictest41-21ksoftirqd/311:48:313
27676991312,1cyclictest41-21ksoftirqd/309:38:143
27676991311,1cyclictest41-21ksoftirqd/311:36:163
27676991311,1cyclictest41-21ksoftirqd/310:05:093
27661991311,2cyclictest30983-21bash10:56:001
27653991312,1cyclictest9-21ksoftirqd/010:54:290
9332120,0sleep70-21swapper/711:58:427
9332120,0sleep70-21swapper/711:58:427
2769299120,0cyclictest0-21swapper/508:36:135
2768499126,1cyclictest49-21ksoftirqd/411:00:554
2768499120,11cyclictest24590-21bash12:10:384
2767699129,1cyclictest0-21swapper/310:15:083
27676991211,1cyclictest41-21ksoftirqd/311:01:113
27676991211,1cyclictest25673-21kworker/3:210:50:253
27676991210,1cyclictest41-21ksoftirqd/312:20:343
27676991210,1cyclictest41-21ksoftirqd/311:39:293
27676991210,1cyclictest41-21ksoftirqd/310:38:563
27669991211,1cyclictest33-21ksoftirqd/212:09:442
27669991211,1cyclictest33-21ksoftirqd/211:59:542
27669991211,1cyclictest33-21ksoftirqd/211:59:542
27669991211,1cyclictest33-21ksoftirqd/210:09:562
2766999120,9cyclictest711-21dbus-daemon10:01:352
2766999120,10cyclictest1-21systemd10:36:572
2766999120,10cyclictest1-21systemd10:36:572
2766199129,2cyclictest25542-21id09:04:021
27653991211,1cyclictest9-21ksoftirqd/010:31:160
27653991211,1cyclictest9-21ksoftirqd/009:28:470
27653991210,2cyclictest9-21ksoftirqd/011:30:260
27653991210,1cyclictest9-21ksoftirqd/011:55:460
27653991210,1cyclictest9-21ksoftirqd/011:44:120
27653991210,1cyclictest9-21ksoftirqd/011:08:270
27653991210,1cyclictest9-21ksoftirqd/010:52:020
27653991210,1cyclictest9-21ksoftirqd/010:08:350
27653991210,1cyclictest9-21ksoftirqd/009:34:180
27653991210,1cyclictest9-21ksoftirqd/009:34:180
2765399121,1cyclictest0-21swapper/011:02:120
220412120,2sleep62770499cyclictest12:12:516
212922120,0sleep30-21swapper/312:25:213
119082120,0sleep011901-21sshd11:34:320
2770499119,1cyclictest65-21ksoftirqd/611:38:576
2770499119,1cyclictest65-21ksoftirqd/610:42:476
2770499119,1cyclictest0-21swapper/611:02:436
2770499115,5cyclictest24764-21hddtemp_smartct10:32:436
2770499115,5cyclictest24764-21hddtemp_smartct10:32:436
2770499110,10cyclictest25822-21grepconf.sh09:29:516
2768499118,2cyclictest8036-21sshd11:36:504
2768499113,0cyclictest0-21swapper/411:02:544
2768499110,11cyclictest6796-21kworker/4:410:32:474
2768499110,11cyclictest6796-21kworker/4:410:32:474
2768499110,1cyclictest711-21dbus-daemon12:26:114
2767699119,1cyclictest7791-21kworker/3:011:04:273
2767699119,1cyclictest7609-21kworker/3:011:44:293
2767699119,1cyclictest41-21ksoftirqd/311:19:063
2767699119,1cyclictest41-21ksoftirqd/310:22:523
2767699119,1cyclictest41-21ksoftirqd/310:09:423
2767699119,1cyclictest41-21ksoftirqd/309:54:033
2767699119,1cyclictest41-21ksoftirqd/309:46:453
2767699119,1cyclictest41-21ksoftirqd/309:27:423
27676991110,1cyclictest41-21ksoftirqd/310:53:383
27676991110,1cyclictest17054-21kworker/3:412:03:333
2767699110,10cyclictest23622-21kworker/3:212:15:113
2767699110,1cyclictest0-21swapper/311:54:223
2766999119,1cyclictest668-21systemd-logind09:07:412
2766999119,1cyclictest33-21ksoftirqd/210:45:212
2766999119,1cyclictest33-21ksoftirqd/209:06:432
2766999119,1cyclictest0-21swapper/210:27:512
27669991110,1cyclictest33-21ksoftirqd/212:16:252
27669991110,1cyclictest33-21ksoftirqd/211:27:022
27669991110,1cyclictest33-21ksoftirqd/211:13:362
27669991110,1cyclictest33-21ksoftirqd/211:13:362
27669991110,1cyclictest33-21ksoftirqd/210:17:142
2766999110,1cyclictest7145-21sshd11:28:512
2766999110,1cyclictest0-21swapper/212:06:322
2766999110,1cyclictest0-21swapper/209:47:292
2766199119,1cyclictest25-21ksoftirqd/111:45:221
2766199113,7cyclictest14781-21grepconf.sh11:34:461
2766199113,2cyclictest25715-21sshd10:58:361
2766199112,4cyclictest32219-21sshd09:24:151
2766199112,4cyclictest23400-21sshd11:40:281
2766199112,4cyclictest21263-21bash09:20:151
2766199112,4cyclictest18699-21sshd12:20:081
2766199111,5cyclictest32659-21cp11:11:511
2766199111,5cyclictest10531-21bash09:53:331
2766199111,4cyclictest3659-21systemd-cgroups09:11:301
2766199110,11cyclictest0-21swapper/110:37:061
2766199110,11cyclictest0-21swapper/110:37:061
2766199110,10cyclictest5742-21bash10:41:031
2766199110,10cyclictest17378-21sshd11:47:351
2766199110,10cyclictest0-21swapper/109:43:021
2766199110,1cyclictest28365-21sshd09:52:201
2765399119,1cyclictest9-21ksoftirqd/012:21:380
2765399119,1cyclictest9-21ksoftirqd/011:23:270
2765399119,1cyclictest9-21ksoftirqd/010:43:390
2765399119,1cyclictest9-21ksoftirqd/010:01:560
2765399119,1cyclictest711-21dbus-daemon08:59:040
2765399118,1cyclictest9-21ksoftirqd/012:12:030
2765399118,1cyclictest9-21ksoftirqd/011:17:010
2765399118,1cyclictest9-21ksoftirqd/011:17:010
27653991110,1cyclictest9-21ksoftirqd/011:06:590
27653991110,1cyclictest9-21ksoftirqd/010:32:440
27653991110,1cyclictest9-21ksoftirqd/010:32:440
27653991110,1cyclictest9-21ksoftirqd/009:26:490
132952110,0sleep013296-21fschecks_count07:27:430
2770499109,1cyclictest65-21ksoftirqd/612:24:096
2770499109,1cyclictest65-21ksoftirqd/609:01:586
2770499108,1cyclictest85350irq/25-em1-rx-009:09:286
2770499107,2cyclictest8282-21sshd10:05:536
2770499104,6cyclictest0-21swapper/609:54:256
2770499103,6cyclictest18627-21grepconf.sh11:47:416
2770499103,4cyclictest7667-21rm11:00:006
2770499103,4cyclictest30393-21systemd-cgroups09:58:386
2770499102,4cyclictest711-21dbus-daemon11:53:156
2770499101,8cyclictest5776-21grep09:43:416
2770499101,4cyclictest711-21dbus-daemon10:30:186
2770499100,9cyclictest22011-21kworker/6:010:40:386
2770499100,10cyclictest5947-21kworker/6:111:46:186
2768499108,1cyclictest49-21ksoftirqd/412:03:094
2768499108,1cyclictest49-21ksoftirqd/409:21:074
2768499108,1cyclictest49-21ksoftirqd/409:13:164
2768499108,1cyclictest0-21swapper/411:47:334
2768499100,10cyclictest0-21swapper/410:51:074
2767699109,1cyclictest41-21ksoftirqd/309:49:113
2767699109,1cyclictest41-21ksoftirqd/309:37:083
2767699109,1cyclictest41-21ksoftirqd/309:37:083
2767699108,1cyclictest41-21ksoftirqd/311:31:483
2767699108,1cyclictest41-21ksoftirqd/309:22:343
2767699108,1cyclictest41-21ksoftirqd/309:21:093
2767699106,4cyclictest10573-21kworker/3:210:19:053
2767699104,3cyclictest16019-21sshd09:09:323
2767699100,10cyclictest0-21swapper/311:11:293
2766999109,1cyclictest33-21ksoftirqd/209:56:382
2766999108,1cyclictest33-21ksoftirqd/211:38:122
2766999108,1cyclictest33-21ksoftirqd/211:00:362
2766999108,1cyclictest33-21ksoftirqd/210:24:102
2766999108,1cyclictest33-21ksoftirqd/210:18:332
2766199109,1cyclictest0-21swapper/111:27:431
2766199103,6cyclictest20984-21bash10:32:261
2766199103,6cyclictest14102-21sshd10:06:371
2766199102,4cyclictest687-21nscd12:22:561
2766199101,3cyclictest955-21sshd10:02:091
2766199100,4cyclictest19237-21sh11:57:411
2766199100,4cyclictest19237-21sh11:57:411
2766199100,4cyclictest10730-21sshd11:06:491
2766199100,10cyclictest0-21swapper/112:14:041
2766199100,10cyclictest0-21swapper/110:46:151
2766199100,10cyclictest0-21swapper/109:29:191
2765399109,1cyclictest9-21ksoftirqd/012:14:580
2765399108,1cyclictest9-21ksoftirqd/011:41:400
2765399108,1cyclictest9-21ksoftirqd/010:41:090
2765399108,1cyclictest9-21ksoftirqd/009:53:130
2765399108,1cyclictest9-21ksoftirqd/009:50:400
2765399108,1cyclictest9-21ksoftirqd/009:13:360
2765399104,3cyclictest668-21systemd-logind12:06:330
2765399104,3cyclictest2650-21sshd11:51:250
2765399100,10cyclictest0-21swapper/009:06:000
277049998,1cyclictest65-21ksoftirqd/611:08:116
277049997,2cyclictest85350irq/25-em1-rx-008:07:506
277049995,2cyclictest17628-21grepconf.sh10:25:466
277049993,5cyclictest0-21swapper/611:13:286
277049993,5cyclictest0-21swapper/611:13:286
277049993,4cyclictest5959-21nscd11:29:126
277049992,5cyclictest22197-21grep10:13:266
277049992,4cyclictest711-21dbus-daemon10:54:056
277049992,4cyclictest5544-21grepconf.sh10:11:536
277049992,4cyclictest11867-21bash09:06:026
277049992,1cyclictest24912-21systemd-cgroups12:10:406
277049990,9cyclictest10221-21kworker/6:410:20:486
277049990,8cyclictest18766-21kworker/6:009:47:366
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional