You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-06 - 22:27
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Fri Mar 06, 2026 12:43:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
114712980,0sleep411469-21systemd-cgroups10:23:554
306162910,1sleep330617-21grep10:02:483
53142810,0sleep233-21ksoftirqd/209:42:522
262752780,0sleep20-21swapper/209:27:312
317252770,2sleep73114399cyclictest11:39:187
141192710,0sleep70-21swapper/709:39:457
302522530,0sleep00-21swapper/010:08:210
3067925125,9sleep40-21swapper/407:02:564
277452510,2sleep73114399cyclictest10:08:127
80322500,1sleep6686-21dbus-daemon12:00:166
316122500,2sleep73114399cyclictest12:23:557
79412490,1sleep67939-21bash10:12:366
179302490,0sleep50-21swapper/512:15:445
147652490,2sleep114714-21sshd11:29:031
38352480,0sleep50-21swapper/510:08:425
37182480,0sleep40-21swapper/411:43:164
229082480,0sleep20-21swapper/211:05:262
224512480,0sleep10-21swapper/109:34:421
218202480,0sleep121811-21sshd11:14:381
194102480,1sleep519411-21grep12:17:375
194102480,1sleep519411-21grep12:17:375
157042480,0sleep20-21swapper/212:19:182
157042480,0sleep20-21swapper/212:19:182
99732470,1sleep39972-21bash12:05:573
56852470,0sleep70-21swapper/710:14:217
52282470,1sleep09-21ksoftirqd/011:08:040
273812470,0sleep60-21swapper/612:05:046
57862460,0sleep75783-21sshd11:47:117
322462460,1sleep632248-21ntpq10:32:306
207362460,0sleep00-21swapper/010:50:260
258032450,0sleep30-21swapper/311:11:113
220402450,0sleep00-21swapper/012:25:170
138242450,1sleep513819-21sshd10:33:175
209772430,0sleep10-21swapper/110:35:371
118112430,0sleep30-21swapper/310:22:103
74662420,0sleep10-21swapper/109:30:101
64742420,0sleep40-21swapper/409:28:134
3088724229,9sleep70-21swapper/707:05:477
214052420,0sleep10-21swapper/112:04:431
98032410,0sleep00-21swapper/010:25:430
82692410,0sleep08270-21sshd12:13:150
312902410,0sleep7721ktimersoftd/710:58:267
3080624128,9sleep10-21swapper/107:04:451
106592410,0sleep30-21swapper/309:39:333
309892400,0sleep50-21swapper/509:44:245
3068724029,8sleep30-21swapper/307:03:033
3065924029,8sleep60-21swapper/607:02:426
2851824027,9sleep50-21swapper/507:02:175
2832924027,9sleep00-21swapper/007:02:120
218762400,0sleep50-21swapper/509:10:365
216442400,0sleep20-21swapper/211:49:552
53732390,1sleep09-21ksoftirqd/009:52:110
3068623929,7sleep20-21swapper/207:03:022
26522390,0sleep00-21swapper/009:52:030
26522390,0sleep00-21swapper/009:52:030
252942380,0sleep40-21swapper/412:27:214
148982380,0sleep30-21swapper/311:16:083
54672370,0sleep00-21swapper/012:20:360
54672370,0sleep00-21swapper/012:20:360
176062360,0sleep2311rcuc/212:26:552
80972320,0sleep50-21swapper/510:29:175
292392320,0sleep60-21swapper/610:47:126
31139992316,4cyclictest1-21systemd11:32:493
31139992316,4cyclictest1-21systemd11:32:493
3113499225,2cyclictest9-21ksoftirqd/011:26:040
3113499225,1cyclictest9-21ksoftirqd/011:35:220
3113499225,1cyclictest9-21ksoftirqd/011:35:220
284982220,4sleep4481ktimersoftd/409:33:094
3113999219,2cyclictest41-21ksoftirqd/311:20:213
31139992116,3cyclictest41-21ksoftirqd/309:12:453
3113499216,3cyclictest9-21ksoftirqd/009:39:240
209552210,0sleep60-21swapper/609:19:506
101912210,1sleep610189-21grepconf.sh11:27:006
3113999209,1cyclictest41-21ksoftirqd/309:32:463
3113999207,1cyclictest41-21ksoftirqd/312:14:303
3113999205,3cyclictest41-21ksoftirqd/309:17:213
3113499207,1cyclictest9-21ksoftirqd/012:31:080
31134992017,1cyclictest9-21ksoftirqd/011:00:360
3114099190,18cyclictest15584-21kworker/4:112:25:124
3113999198,1cyclictest41-21ksoftirqd/311:02:343
31139991915,3cyclictest41-21ksoftirqd/312:31:033
31139991914,4cyclictest41-21ksoftirqd/310:53:473
31139991914,4cyclictest41-21ksoftirqd/310:40:133
31139991914,3cyclictest41-21ksoftirqd/312:33:483
31139991914,3cyclictest41-21ksoftirqd/311:37:443
31139991914,2cyclictest25347-21cp10:13:373
3113499194,2cyclictest9-21ksoftirqd/011:06:280
3113499193,3cyclictest9-21ksoftirqd/011:51:360
31134991918,1cyclictest9-21ksoftirqd/012:32:190
31134991915,3cyclictest9-21ksoftirqd/009:33:460
31134991915,3cyclictest9-21ksoftirqd/009:25:160
31134991915,1cyclictest9-21ksoftirqd/009:45:510
31134991914,3cyclictest9-21ksoftirqd/010:54:300
31134991910,1cyclictest121rcu_preempt10:37:050
3113499191,3cyclictest131rcu_sched11:46:250
22272190,0sleep10-21swapper/111:45:051
31139991816,1cyclictest41-21ksoftirqd/311:47:403
31139991816,1cyclictest41-21ksoftirqd/311:28:223
31139991816,1cyclictest41-21ksoftirqd/310:23:183
31139991816,1cyclictest41-21ksoftirqd/309:59:003
31139991815,3cyclictest41-21ksoftirqd/310:35:003
31139991814,3cyclictest41-21ksoftirqd/309:44:083
31139991812,4cyclictest41-21ksoftirqd/310:29:333
3113999181,1cyclictest41-21ksoftirqd/309:23:493
3113999180,2cyclictest121rcu_preempt11:58:133
3113499186,3cyclictest9-21ksoftirqd/011:13:480
31134991817,1cyclictest9-21ksoftirqd/012:06:330
31134991816,2cyclictest26178-21kworker/0:107:12:200
31134991814,3cyclictest9-21ksoftirqd/009:13:410
31134991813,2cyclictest9-21ksoftirqd/010:43:540
31134991813,2cyclictest9-21ksoftirqd/010:43:540
31134991811,2cyclictest9-21ksoftirqd/010:40:410
3113499180,1cyclictest131rcu_sched10:14:480
31140991713,2cyclictest1-21systemd09:58:044
31139991716,1cyclictest41-21ksoftirqd/311:22:453
31139991715,1cyclictest492-21systemd-journal11:00:033
31139991715,1cyclictest41-21ksoftirqd/312:18:243
31139991715,1cyclictest41-21ksoftirqd/312:18:243
31139991715,1cyclictest41-21ksoftirqd/312:08:203
31139991715,1cyclictest41-21ksoftirqd/311:56:053
31139991715,1cyclictest41-21ksoftirqd/310:50:433
31139991715,1cyclictest41-21ksoftirqd/310:08:353
31139991715,1cyclictest41-21ksoftirqd/309:55:093
31139991715,1cyclictest41-21ksoftirqd/309:47:283
31139991715,1cyclictest41-21ksoftirqd/309:47:283
31139991715,1cyclictest41-21ksoftirqd/309:09:213
31139991714,3cyclictest41-21ksoftirqd/311:42:113
3113499176,1cyclictest9-21ksoftirqd/009:10:590
31134991715,2cyclictest9-21ksoftirqd/011:18:360
31134991715,2cyclictest9-21ksoftirqd/009:17:190
31134991715,2cyclictest26178-21kworker/0:109:03:360
31134991715,2cyclictest26178-21kworker/0:108:16:120
31134991715,2cyclictest26178-21kworker/0:108:01:180
31134991715,1cyclictest9-21ksoftirqd/012:08:100
31134991715,1cyclictest9-21ksoftirqd/011:28:370
31134991714,1cyclictest9-21ksoftirqd/010:02:180
31134991714,1cyclictest9-21ksoftirqd/009:58:300
31134991713,3cyclictest9-21ksoftirqd/010:28:150
31134991712,1cyclictest9-21ksoftirqd/009:30:000
31134991711,3cyclictest26178-21kworker/0:107:38:480
31134991711,3cyclictest26178-21kworker/0:107:38:480
3113499171,3cyclictest131rcu_sched11:38:570
3113499171,1cyclictest131rcu_sched11:57:140
25392170,2sleep53114199cyclictest09:48:165
25392170,2sleep53114199cyclictest09:48:165
235692170,0sleep523571-21grep10:39:265
325172160,0sleep70-21swapper/711:52:247
31139991614,1cyclictest41-21ksoftirqd/310:42:543
31139991614,1cyclictest41-21ksoftirqd/310:42:543
31139991614,1cyclictest41-21ksoftirqd/309:28:413
31139991613,3cyclictest41-21ksoftirqd/312:22:523
31134991614,2cyclictest26178-21kworker/0:108:33:400
31134991614,2cyclictest26178-21kworker/0:107:23:280
31134991614,1cyclictest9-21ksoftirqd/011:53:290
31134991614,1cyclictest26178-21kworker/0:108:37:500
31134991612,1cyclictest9-21ksoftirqd/010:18:440
31134991611,0cyclictest26178-21kworker/0:107:20:400
175822160,0sleep517580-21sshd12:10:085
159052160,2sleep23113899cyclictest11:53:152
121012160,0sleep10-21swapper/110:44:181
121012160,0sleep10-21swapper/110:44:181
47522150,0sleep40-21swapper/410:05:054
31140991513,1cyclictest49-21ksoftirqd/412:09:134
31140991513,1cyclictest49-21ksoftirqd/410:30:224
31139991513,1cyclictest41-21ksoftirqd/309:07:113
31134991513,2cyclictest26178-21kworker/0:108:08:080
31134991513,0cyclictest26178-21kworker/0:107:28:060
31134991510,3cyclictest26178-21kworker/0:108:56:240
31134991510,2cyclictest26178-21kworker/0:108:25:100
238312150,0sleep1231rcuc/109:51:241
238312150,0sleep1231rcuc/109:51:241
185672150,0sleep50-21swapper/511:47:495
145832150,0sleep50-21swapper/509:25:015
31140991412,1cyclictest49-21ksoftirqd/410:34:204
3114099140,12cyclictest0-21swapper/411:06:034
31138991412,1cyclictest33-21ksoftirqd/210:34:472
31135991413,1cyclictest25-21ksoftirqd/111:49:281
31135991412,1cyclictest25-21ksoftirqd/111:20:201
3113499149,3cyclictest26178-21kworker/0:108:59:500
3113499149,0cyclictest26178-21kworker/0:108:03:080
31134991412,2cyclictest26178-21kworker/0:108:19:320
31134991412,2cyclictest26178-21kworker/0:107:55:420
31134991412,2cyclictest26178-21kworker/0:107:08:200
31134991411,3cyclictest26178-21kworker/0:107:33:000
273312140,0sleep10-21swapper/111:22:221
224852140,0sleep10-21swapper/112:30:541
204712140,0sleep50-21swapper/510:00:255
203012140,1sleep125-21ksoftirqd/110:22:321
66912130,1sleep56675-21sshd12:33:385
63682130,0sleep10-21swapper/110:38:261
3114399132,5cyclictest11880-21grepconf.sh09:56:157
31143991312,1cyclictest73-21ksoftirqd/710:22:367
3114399130,12cyclictest686-21dbus-daemon11:47:077
31142991312,1cyclictest65-21ksoftirqd/609:44:376
3114299130,12cyclictest1876-21bash11:35:466
3114299130,12cyclictest1876-21bash11:35:466
3114099139,1cyclictest49-21ksoftirqd/410:12:474
31140991312,1cyclictest49-21ksoftirqd/411:29:374
31140991312,1cyclictest49-21ksoftirqd/410:53:274
31140991311,1cyclictest49-21ksoftirqd/412:35:194
31140991311,1cyclictest49-21ksoftirqd/411:00:464
31140991311,1cyclictest49-21ksoftirqd/410:08:114
3113899134,3cyclictest0-21swapper/211:30:032
3113899133,4cyclictest29617-21id09:33:132
3113899133,4cyclictest24078-21sd-pam12:10:312
31138991311,1cyclictest33-21ksoftirqd/210:46:122
31138991311,1cyclictest33-21ksoftirqd/210:46:122
31138991311,1cyclictest1-21systemd10:24:462
3113899130,12cyclictest1-21systemd11:26:172
31135991312,1cyclictest25-21ksoftirqd/112:13:561
31135991311,1cyclictest25-21ksoftirqd/112:20:141
31135991311,1cyclictest25-21ksoftirqd/112:20:141
31135991311,1cyclictest25-21ksoftirqd/112:10:331
31135991311,1cyclictest25-21ksoftirqd/109:44:291
31134991312,0cyclictest26178-21kworker/0:107:45:360
305632130,1sleep230547-21bash11:17:032
3114399122,4cyclictest32397-21sshd11:13:207
3114399122,4cyclictest28256-21sshd11:03:507
3114399122,4cyclictest22593-21sshd09:45:477
3114399122,4cyclictest1782-21sshd11:09:467
3114399122,4cyclictest1775-21sshd09:27:577
3114399122,4cyclictest14663-21sshd09:17:347
31143991210,1cyclictest73-21ksoftirqd/711:57:447
3114399121,5cyclictest6249-21sshd10:42:117
3114399121,5cyclictest6249-21sshd10:42:117
3114399121,5cyclictest14158-21cp10:40:467
3114399121,5cyclictest1-21systemd11:22:247
3114399120,2cyclictest1-21systemd09:23:407
3114399120,1cyclictest0-21swapper/710:54:477
3114299129,2cyclictest11411-21systemd12:35:506
3114299124,5cyclictest456-21sshd11:11:366
31142991210,1cyclictest65-21ksoftirqd/610:38:396
3114199128,3cyclictest57-21ksoftirqd/510:05:495
3114199124,4cyclictest4229-21bash11:39:365
3114099128,1cyclictest49-21ksoftirqd/411:52:474
3114099124,8cyclictest26155-21kworker/4:109:23:534
31140991211,1cyclictest49-21ksoftirqd/412:19:404
31140991211,1cyclictest49-21ksoftirqd/412:19:404
31140991211,1cyclictest49-21ksoftirqd/410:50:194
31140991210,1cyclictest49-21ksoftirqd/412:16:384
31140991210,1cyclictest49-21ksoftirqd/411:32:144
31140991210,1cyclictest49-21ksoftirqd/411:32:144
31140991210,1cyclictest49-21ksoftirqd/411:17:194
31140991210,1cyclictest49-21ksoftirqd/409:47:274
31140991210,1cyclictest49-21ksoftirqd/409:47:274
31140991210,1cyclictest49-21ksoftirqd/409:37:484
3114099120,11cyclictest31954-21grep09:11:174
31139991211,1cyclictest41-21ksoftirqd/307:47:273
31139991210,1cyclictest41-21ksoftirqd/307:42:103
31139991210,1cyclictest41-21ksoftirqd/307:42:103
3113899122,4cyclictest686-21dbus-daemon10:40:432
3113899122,4cyclictest0-21swapper/209:51:192
3113899122,4cyclictest0-21swapper/209:51:192
31138991211,1cyclictest33-21ksoftirqd/209:23:252
31138991210,1cyclictest33-21ksoftirqd/210:51:062
31138991210,1cyclictest33-21ksoftirqd/210:02:352
31138991210,1cyclictest33-21ksoftirqd/209:37:102
3113899120,11cyclictest32666-21grepconf.sh10:19:362
3113899120,11cyclictest10725-21kworker/2:011:10:052
3113899120,1cyclictest0-21swapper/210:55:332
3113599129,2cyclictest25-21ksoftirqd/112:23:041
3113599120,11cyclictest31698-21kworker/1:011:02:251
3113599120,11cyclictest26965-21sshd11:57:341
3113599120,1cyclictest0-21swapper/110:55:271
31134991212,0cyclictest26178-21kworker/0:107:51:200
31134991211,1cyclictest26178-21kworker/0:108:43:200
31134991211,0cyclictest26178-21kworker/0:108:50:400
31134991211,0cyclictest26178-21kworker/0:108:50:400
31134991210,0cyclictest26178-21kworker/0:108:29:360
251422120,0sleep50-21swapper/511:37:085
251422120,0sleep50-21swapper/511:37:085
103252120,0sleep710316-21sshd10:31:197
49142110,0sleep40-21swapper/409:46:394
3114399119,1cyclictest9107-21bash12:33:487
3114399119,1cyclictest73-21ksoftirqd/709:32:107
3114399119,1cyclictest11647-21sshd10:18:227
3114399113,7cyclictest828-21bash09:16:547
3114399113,7cyclictest10910-21kill12:06:007
3114399112,4cyclictest6902-21sshd11:30:307
3114399111,4cyclictest29067-21sshd10:02:427
3114399111,4cyclictest13499-21systemd12:13:347
3114399111,4cyclictest11206-21systemd-cgroups10:47:557
3114399110,5cyclictest2238-21sshd11:33:537
3114399110,5cyclictest2238-21sshd11:33:537
3114299119,1cyclictest65-21ksoftirqd/610:53:476
3114299119,1cyclictest65-21ksoftirqd/609:56:366
3114299117,1cyclictest87950irq/25-em1-rx-009:12:566
3114299116,1cyclictest26455-21grepconf.sh10:19:146
3114299113,4cyclictest887-21gdbus10:22:226
3114299112,6cyclictest21341-21id10:57:516
3114299112,5cyclictest1647-21systemd-cgroups10:30:486
3114299112,4cyclictest3592-21sshd12:12:586
3114299112,4cyclictest28467-21sshd11:03:506
3114299112,4cyclictest28299-21sshd10:47:106
3114299112,4cyclictest28299-21sshd10:47:106
3114299112,4cyclictest16148-21sshd12:24:566
31142991110,1cyclictest0-21swapper/609:26:206
3114299111,4cyclictest368-21id11:54:156
3114299110,7cyclictest13037-21sshd11:30:516
3114299110,11cyclictest0-21swapper/609:36:266
3114199119,1cyclictest57-21ksoftirqd/512:06:065
3114199119,1cyclictest57-21ksoftirqd/511:53:535
3114199119,1cyclictest57-21ksoftirqd/510:48:315
3114199119,1cyclictest24706-21bash12:29:085
31141991110,1cyclictest57-21ksoftirqd/512:00:065
31141991110,1cyclictest57-21ksoftirqd/511:26:315
31141991110,1cyclictest0-21swapper/509:39:395
3114199110,10cyclictest25978-21systemd-cgroups09:16:295
3114199110,1cyclictest0-21swapper/511:46:535
3114199110,1cyclictest0-21swapper/510:58:005
3114099119,1cyclictest49-21ksoftirqd/409:20:144
3114099118,1cyclictest49-21ksoftirqd/411:49:374
3114099118,1cyclictest49-21ksoftirqd/411:16:204
3114099117,2cyclictest32386-21ps11:37:274
3114099117,1cyclictest49-21ksoftirqd/409:16:134
3114099112,4cyclictest10010-21sshd10:22:034
31140991110,1cyclictest0-21swapper/411:59:394
3113999119,1cyclictest41-21ksoftirqd/309:02:103
3113899114,6cyclictest33-21ksoftirqd/209:17:392
3113899112,4cyclictest32304-21sshd11:18:562
3113899112,4cyclictest28422-21systemd-cgroups11:41:012
31138991110,1cyclictest33-21ksoftirqd/212:15:392
3113899111,4cyclictest1-21systemd11:34:402
3113899111,4cyclictest1-21systemd11:34:402
3113899110,11cyclictest4061-21kworker/2:611:46:492
3113899110,11cyclictest22074-21kworker/2:112:00:332
3113899110,1cyclictest686-21dbus-daemon12:35:112
3113599119,1cyclictest25-21ksoftirqd/111:36:071
3113599119,1cyclictest25-21ksoftirqd/111:36:071
3113599119,1cyclictest25-21ksoftirqd/110:07:231
3113599119,1cyclictest0-21swapper/110:19:311
3113599114,1cyclictest21456-21systemd-cgroups10:13:231
31135991110,1cyclictest25-21ksoftirqd/112:34:531
3113599110,1cyclictest16003-21sshd09:54:361
241662110,0sleep50-21swapper/509:55:055
3114399107,2cyclictest31306-21ystemctl09:48:047
3114399107,2cyclictest31306-21ystemctl09:48:047
3114399102,7cyclictest11059-21grepconf.sh12:19:017
3114399102,7cyclictest11059-21grepconf.sh12:19:017
3114399102,2cyclictest10358-21bash10:33:057
3114399101,4cyclictest32423-21sshd12:27:397
3114399100,4cyclictest11140-21sshd12:07:517
3114399100,4cyclictest0-21swapper/711:20:297
3114399100,4cyclictest0-21swapper/709:58:227
3114399100,10cyclictest10250-21kworker/7:409:11:427
3114299108,1cyclictest65-21ksoftirqd/612:28:246
3114299108,1cyclictest65-21ksoftirqd/611:12:266
3114299107,1cyclictest65-21ksoftirqd/611:38:336
3114299103,4cyclictest1-21systemd09:59:476
3114299103,3cyclictest691-21systemd-logind11:20:016
3114299103,2cyclictest15918-21kill09:39:526
3114299102,5cyclictest13371-21sshd11:45:456
3114299102,4cyclictest18963-21sshd10:02:136
3114299102,3cyclictest0-21swapper/612:17:516
3114299102,3cyclictest0-21swapper/612:17:516
3114299101,4cyclictest27075-21id09:47:496
3114299101,4cyclictest27075-21id09:47:496
3114299100,6cyclictest0-21swapper/609:27:156
3114299100,1cyclictest0-21swapper/611:49:016
3114199109,1cyclictest57-21ksoftirqd/511:19:295
3114199109,1cyclictest57-21ksoftirqd/511:08:495
3114199109,1cyclictest57-21ksoftirqd/509:18:465
3114199109,1cyclictest0-21swapper/512:24:055
3114199108,1cyclictest57-21ksoftirqd/510:17:295
3114199102,5cyclictest24715-21systemd10:43:095
3114199102,5cyclictest24715-21systemd10:43:095
3114199101,4cyclictest18824-21bash11:29:185
3114199100,10cyclictest0-21swapper/511:12:215
3114099108,1cyclictest49-21ksoftirqd/409:53:224
3114099108,1cyclictest49-21ksoftirqd/409:07:104
3114099108,1cyclictest21668-21bash11:23:524
3114099106,4cyclictest49-21ksoftirqd/412:02:134
3114099104,3cyclictest17095-21sshd10:37:174
3114099101,5cyclictest23770-21bash11:07:214
3113999109,1cyclictest41-21ksoftirqd/308:47:103
3113999108,1cyclictest41-21ksoftirqd/307:42:293
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional