You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-29 - 02:22
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Sat Aug 29, 2026 00:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
190432950,1sleep125-21ksoftirqd/122:38:151
83312570,1sleep6758-21runrttasks23:05:196
12762520,0sleep00-21swapper/022:20:440
336425026,8sleep10-21swapper/119:02:501
194622490,0sleep219459-21systemd-cgroups21:43:132
194622490,0sleep219459-21systemd-cgroups21:43:132
14782490,0sleep40-21swapper/400:25:584
142302490,0sleep60-21swapper/621:26:276
270962480,11sleep685150irq/25-em1-rx-000:27:286
65192470,0sleep60-21swapper/600:33:376
65192470,0sleep60-21swapper/600:33:376
341024725,8sleep20-21swapper/219:03:312
221532460,1sleep522152-21sshd00:05:095
221072460,0sleep00-21swapper/023:15:140
275782450,1sleep125-21ksoftirqd/122:15:001
241772450,0sleep00-21swapper/021:27:010
138632450,0sleep20-21swapper/222:03:162
62972440,0sleep40-21swapper/400:04:114
62372440,0sleep60-21swapper/621:59:116
280732440,0sleep30-21swapper/321:38:103
236082440,0sleep123609-21sshd23:26:201
14602440,0sleep70-21swapper/722:02:327
6212430,0sleep20-21swapper/200:27:482
295072430,0sleep20-21swapper/222:29:432
259822430,0sleep00-21swapper/000:01:350
224092430,0sleep70-21swapper/723:20:417
156312430,0sleep70-21swapper/722:12:267
230692420,0sleep00-21swapper/022:45:420
38192410,0sleep70-21swapper/721:49:437
38192410,0sleep70-21swapper/721:49:437
344624129,8sleep70-21swapper/719:04:007
325924129,8sleep30-21swapper/319:01:243
267342410,0sleep40-21swapper/422:57:004
206372410,0sleep420632-21sshd22:34:414
161422410,0sleep20-21swapper/222:08:522
343624028,8sleep60-21swapper/619:03:516
320924027,9sleep40-21swapper/419:00:444
175962400,0sleep20-21swapper/223:40:472
325223927,9sleep50-21swapper/519:01:185
322023926,9sleep00-21swapper/019:00:480
251822390,0sleep30-21swapper/322:51:223
191872390,0sleep60-21swapper/600:21:286
191872390,0sleep60-21swapper/600:21:286
95822380,0sleep00-21swapper/023:49:420
122482380,0sleep50-21swapper/523:42:265
112892370,0sleep20-21swapper/222:50:332
266912330,0sleep6631rcuc/621:41:416
266912330,0sleep6631rcuc/621:41:416
157572270,0sleep40-21swapper/423:48:104
189092260,0sleep70-21swapper/723:48:217
373499240,10cyclictest16489-21bash22:45:251
50862230,0sleep40-21swapper/422:08:144
373699225,1cyclictest41-21ksoftirqd/321:55:253
373499220,7cyclictest4810-21grep22:26:251
373499220,7cyclictest0-21swapper/122:42:231
373499220,7cyclictest0-21swapper/122:42:231
172302220,1sleep617228-21sshd21:48:386
172302220,1sleep617228-21sshd21:48:386
373699215,3cyclictest41-21ksoftirqd/323:56:103
373699212,2cyclictest41-21ksoftirqd/322:40:123
373699212,2cyclictest41-21ksoftirqd/322:40:123
3736992111,1cyclictest41-21ksoftirqd/300:15:473
3736992110,3cyclictest41-21ksoftirqd/322:35:113
373499210,9cyclictest31512-21id00:13:011
373499210,9cyclictest31512-21id00:13:011
373499210,9cyclictest13183-21grepconf.sh23:40:311
373499210,8cyclictest15377-21grep23:12:591
373499210,7cyclictest1-21systemd21:23:041
373499210,7cyclictest0-21swapper/123:54:091
373499210,7cyclictest0-21swapper/122:54:291
373499210,10cyclictest594-21sd-pam21:36:351
373499210,10cyclictest4610-21bash21:53:251
373499210,10cyclictest31637-21bash23:56:251
373699206,1cyclictest41-21ksoftirqd/321:46:523
373699206,1cyclictest41-21ksoftirqd/321:46:523
3736992017,1cyclictest41-21ksoftirqd/321:24:213
373499203,7cyclictest721-21dbus-daemon00:20:551
373499203,7cyclictest721-21dbus-daemon00:20:551
373499200,9cyclictest4825-21sshd00:02:141
373499200,9cyclictest18010-21sshd23:37:121
373499200,9cyclictest18010-21sshd23:37:121
373499200,9cyclictest14510-21grepconf.sh21:31:561
373499200,8cyclictest859-21gdbus22:23:571
373499200,8cyclictest20059-21id00:08:391
373499200,12cyclictest30533-21kill22:02:211
373499200,10cyclictest4085-21sshd21:47:521
373499200,10cyclictest4085-21sshd21:47:521
373499200,10cyclictest22752-21bash21:41:281
373499200,10cyclictest22752-21bash21:41:281
373499200,10cyclictest13260-21grep00:34:011
373499200,10cyclictest13260-21grep00:34:011
3745991914,3cyclictest73-21ksoftirqd/722:07:587
373699194,2cyclictest41-21ksoftirqd/321:10:493
3736991914,3cyclictest41-21ksoftirqd/322:28:393
373499193,9cyclictest25222-21grep22:56:541
373499191,7cyclictest1-21systemd22:09:301
373499190,9cyclictest721-21dbus-daemon21:56:121
373499190,9cyclictest721-21dbus-daemon21:06:141
373499190,9cyclictest15981-21grepconf.sh00:17:401
373499190,8cyclictest24629-21grep21:28:561
373499190,8cyclictest19285-21bash23:48:221
373499190,8cyclictest13059-21systemd21:15:171
373499190,8cyclictest11801-21bash23:31:171
373499190,8cyclictest11801-21bash23:31:171
373499190,7cyclictest721-21dbus-daemon21:10:191
373499190,5cyclictest0-21swapper/100:26:121
373499190,18cyclictest11650-21systemd22:17:401
373499190,12cyclictest0-21swapper/122:30:281
373499190,11cyclictest27357-21grepconf.sh23:24:461
373499190,11cyclictest22941-21cp23:06:021
373499190,10cyclictest3579-21sshd23:15:551
373499190,10cyclictest14115-21grepconf.sh23:01:551
3745991816,1cyclictest73-21ksoftirqd/723:31:477
3745991816,1cyclictest73-21ksoftirqd/723:31:477
373699189,1cyclictest41-21ksoftirqd/322:13:013
373699186,3cyclictest41-21ksoftirqd/321:16:053
373699185,1cyclictest41-21ksoftirqd/323:09:543
373699183,3cyclictest41-21ksoftirqd/323:45:163
373699182,1cyclictest41-21ksoftirqd/323:32:203
373699182,1cyclictest41-21ksoftirqd/323:32:203
3736991816,1cyclictest41-21ksoftirqd/322:20:333
3736991816,1cyclictest41-21ksoftirqd/322:09:523
3736991810,4cyclictest41-21ksoftirqd/300:07:223
373699181,3cyclictest41-21ksoftirqd/321:41:053
373699181,3cyclictest41-21ksoftirqd/321:41:053
373699175,1cyclictest41-21ksoftirqd/323:55:073
3736991715,1cyclictest41-21ksoftirqd/323:24:033
3736991715,1cyclictest41-21ksoftirqd/322:15:183
3736991714,1cyclictest41-21ksoftirqd/323:40:343
3736991714,1cyclictest41-21ksoftirqd/323:36:393
3736991714,1cyclictest41-21ksoftirqd/323:36:393
3736991713,3cyclictest41-21ksoftirqd/300:27:413
3736991713,1cyclictest41-21ksoftirqd/322:56:493
3736991713,1cyclictest41-21ksoftirqd/300:31:193
3736991713,1cyclictest41-21ksoftirqd/300:31:193
373699170,1cyclictest0-21swapper/323:17:543
168682170,0sleep0101ktimersoftd/023:55:340
3745991614,1cyclictest73-21ksoftirqd/721:27:307
3745991614,1cyclictest73-21ksoftirqd/721:15:307
3745991612,1cyclictest73-21ksoftirqd/700:06:487
3745991610,3cyclictest73-21ksoftirqd/721:10:467
373699165,1cyclictest41-21ksoftirqd/323:00:373
373699163,1cyclictest41-21ksoftirqd/322:01:263
3736991614,1cyclictest41-21ksoftirqd/322:36:463
3736991614,1cyclictest41-21ksoftirqd/321:26:083
3736991613,2cyclictest41-21ksoftirqd/300:24:483
3736991613,2cyclictest41-21ksoftirqd/300:24:483
3736991613,1cyclictest41-21ksoftirqd/321:09:503
3736991612,1cyclictest41-21ksoftirqd/323:25:293
3736991612,1cyclictest41-21ksoftirqd/321:50:233
373699161,3cyclictest41-21ksoftirqd/322:47:553
373699160,2cyclictest41-21ksoftirqd/300:10:523
373699160,2cyclictest41-21ksoftirqd/300:10:523
373699160,2cyclictest131rcu_sched23:10:283
373699160,2cyclictest131rcu_sched21:33:323
3735991614,2cyclictest33-21ksoftirqd/223:33:282
3735991614,2cyclictest33-21ksoftirqd/223:33:282
71822150,1sleep47180-21grepconf.sh22:19:194
3745991513,1cyclictest73-21ksoftirqd/723:25:447
3745991513,1cyclictest73-21ksoftirqd/721:50:367
3745991513,1cyclictest73-21ksoftirqd/721:38:017
3745991513,1cyclictest73-21ksoftirqd/721:20:237
3745991513,1cyclictest73-21ksoftirqd/700:12:247
3745991513,1cyclictest73-21ksoftirqd/700:12:247
3745991513,1cyclictest73-21ksoftirqd/700:01:257
3745991512,2cyclictest73-21ksoftirqd/723:16:047
3745991512,2cyclictest73-21ksoftirqd/721:32:507
3745991512,1cyclictest73-21ksoftirqd/723:02:097
3745991510,3cyclictest73-21ksoftirqd/723:09:557
374599150,2cyclictest121rcu_preempt21:58:207
3736991513,1cyclictest41-21ksoftirqd/300:00:373
3735991513,1cyclictest33-21ksoftirqd/221:29:312
325382150,0sleep20-21swapper/221:10:452
272222150,0sleep20-21swapper/223:13:402
221762150,0sleep20-21swapper/221:30:322
160142150,1sleep233-21ksoftirqd/223:03:542
100632150,1sleep410050-21bash22:13:594
58922140,0sleep60-21swapper/622:04:406
47432140,0sleep20-21swapper/221:38:412
374599149,3cyclictest73-21ksoftirqd/722:45:387
374599149,3cyclictest73-21ksoftirqd/722:36:387
3745991412,1cyclictest73-21ksoftirqd/723:45:127
3745991412,1cyclictest73-21ksoftirqd/723:35:337
3745991412,1cyclictest73-21ksoftirqd/723:35:337
3745991412,1cyclictest73-21ksoftirqd/722:21:177
3745991412,1cyclictest73-21ksoftirqd/722:16:137
3745991412,1cyclictest73-21ksoftirqd/721:40:277
3745991412,1cyclictest73-21ksoftirqd/721:40:277
3745991412,1cyclictest73-21ksoftirqd/721:06:597
3745991412,1cyclictest73-21ksoftirqd/700:34:047
3745991412,1cyclictest73-21ksoftirqd/700:34:047
3745991412,1cyclictest73-21ksoftirqd/700:16:297
3745991411,2cyclictest73-21ksoftirqd/723:54:217
3745991410,3cyclictest73-21ksoftirqd/700:21:057
3745991410,3cyclictest73-21ksoftirqd/700:21:057
374599140,13cyclictest6587-21kworker/7:023:14:227
374599140,13cyclictest15299-21kworker/7:522:56:497
373999148,1cyclictest65-21ksoftirqd/600:13:166
373999148,1cyclictest65-21ksoftirqd/600:13:166
373899140,13cyclictest3415-21sshd23:51:065
373799144,4cyclictest0-21swapper/423:52:144
373599149,1cyclictest33-21ksoftirqd/223:45:502
3735991413,1cyclictest33-21ksoftirqd/221:59:102
3735991412,1cyclictest33-21ksoftirqd/222:35:382
3735991412,1cyclictest33-21ksoftirqd/200:20:462
3735991412,1cyclictest33-21ksoftirqd/200:20:462
254062140,0sleep20-21swapper/223:08:032
152712140,0sleep60-21swapper/621:50:226
3745991312,1cyclictest73-21ksoftirqd/723:58:547
3745991312,1cyclictest73-21ksoftirqd/722:55:127
3745991312,1cyclictest73-21ksoftirqd/722:28:487
3745991311,1cyclictest73-21ksoftirqd/722:41:237
3745991311,1cyclictest73-21ksoftirqd/722:41:237
3745991311,1cyclictest73-21ksoftirqd/722:34:177
3745991311,1cyclictest73-21ksoftirqd/700:26:477
373999132,5cyclictest8052-21bash23:33:026
373999132,5cyclictest8052-21bash23:33:026
373999130,12cyclictest31760-21kworker/6:323:25:496
373899139,3cyclictest721-21dbus-daemon23:57:065
373899134,3cyclictest0-21swapper/522:17:015
373899133,4cyclictest22987-21sshd21:36:005
373899132,5cyclictest32658-21sshd22:48:115
373899132,5cyclictest22717-21bash21:41:285
373899132,5cyclictest22717-21bash21:41:285
373899132,1cyclictest0-21swapper/522:57:055
3738991311,1cyclictest57-21ksoftirqd/521:45:265
3738991311,1cyclictest57-21ksoftirqd/521:45:265
3738991311,1cyclictest57-21ksoftirqd/500:22:485
3738991311,1cyclictest57-21ksoftirqd/500:22:485
3737991311,1cyclictest49-21ksoftirqd/423:15:224
373799130,12cyclictest27027-21grepconf.sh23:45:114
373699134,1cyclictest41-21ksoftirqd/321:05:123
373599139,3cyclictest33-21ksoftirqd/200:10:272
373599139,3cyclictest33-21ksoftirqd/200:10:272
373599139,1cyclictest33-21ksoftirqd/223:26:332
3735991311,1cyclictest33-21ksoftirqd/221:49:432
3735991311,1cyclictest33-21ksoftirqd/221:49:432
373599130,10cyclictest29045-21kworker/2:423:51:502
373399132,5cyclictest995-21systemd21:42:050
373399132,5cyclictest995-21systemd21:42:050
263202130,0sleep426300-21sshd21:58:294
217072130,0sleep20-21swapper/222:40:162
217072130,0sleep20-21swapper/222:40:162
108712130,0sleep40-21swapper/423:58:594
373999129,2cyclictest3683-21rm23:01:186
373999122,4cyclictest6989-21sshd23:12:306
373999122,4cyclictest23797-21sshd22:58:516
373999122,4cyclictest22060-21sshd21:17:366
373999122,4cyclictest1-21systemd21:05:336
3739991211,1cyclictest65-21ksoftirqd/622:42:496
3739991211,1cyclictest65-21ksoftirqd/622:42:496
3739991210,1cyclictest857-21polkitd23:18:576
3739991210,1cyclictest65-21ksoftirqd/623:48:006
3739991210,1cyclictest65-21ksoftirqd/622:26:166
3739991210,1cyclictest12786-21systemd22:37:486
373999121,5cyclictest923-21grep00:02:006
373999120,11cyclictest14367-21kworker/6:422:20:286
373899129,2cyclictest57-21ksoftirqd/522:40:125
373899129,2cyclictest21882-21/usr/sbin/munin21:30:315
373899122,4cyclictest620-21sshd00:09:245
373899122,4cyclictest21440-21sshd22:05:275
373899122,4cyclictest11309-21sshd23:05:265
3738991210,1cyclictest57-21ksoftirqd/522:51:165
373899120,11cyclictest3492-21systemd22:24:355
373899120,11cyclictest32509-21kworker/5:100:26:135
373899120,1cyclictest22712-21grepconf.sh23:26:175
373799129,2cyclictest4072-21grep00:05:504
373799125,6cyclictest0-21swapper/421:07:084
373799123,4cyclictest8675-21sshd21:42:314
373799123,4cyclictest8675-21sshd21:42:314
373799123,4cyclictest20807-21sshd23:26:094
373799122,4cyclictest32070-21sshd22:37:044
373799122,4cyclictest2924-21sshd23:30:454
373799122,4cyclictest2924-21sshd23:30:454
3737991210,2cyclictest0-21swapper/422:48:264
3737991210,1cyclictest30942-21sshd21:29:174
373799121,5cyclictest10862-21kill21:15:104
373799120,11cyclictest20435-21bash21:52:284
373599129,3cyclictest33-21ksoftirqd/223:21:072
373599128,3cyclictest33-21ksoftirqd/222:10:252
373599128,3cyclictest33-21ksoftirqd/200:05:502
373599123,6cyclictest1-21systemd23:59:242
3735991211,1cyclictest33-21ksoftirqd/200:00:592
3735991210,1cyclictest33-21ksoftirqd/200:30:512
3735991210,1cyclictest33-21ksoftirqd/200:30:512
373599120,12cyclictest2520-21kworker/2:022:24:462
373599120,11cyclictest24688-21bash22:58:542
373499120,9cyclictest16177-21awk20:20:321
373399125,3cyclictest5961-21bash21:47:590
373399125,3cyclictest5961-21bash21:47:590
373399122,4cyclictest721-21dbus-daemon00:20:090
373399122,4cyclictest27636-21bash23:02:430
373399122,4cyclictest13931-21grep00:28:340
373399122,4cyclictest1130-21sshd22:28:040
3733991210,1cyclictest9-21ksoftirqd/023:44:420
3733991210,1cyclictest9-21ksoftirqd/022:38:500
373399121,5cyclictest5026-21sshd00:05:540
373399120,11cyclictest721-21dbus-daemon21:20:240
283952120,0sleep40-21swapper/423:21:024
197112120,0sleep60-21swapper/621:13:466
47912110,0sleep6631rcuc/623:40:116
47912110,0sleep6631rcuc/623:40:116
373999118,2cyclictest19626-21bash22:30:526
373999112,4cyclictest12314-21sshd21:20:416
373999112,4cyclictest0-21swapper/622:10:476
373999111,4cyclictest7357-21grep00:06:026
373999111,4cyclictest0-21swapper/623:21:566
373999111,4cyclictest0-21swapper/622:15:286
373999110,1cyclictest0-21swapper/621:34:546
373899113,7cyclictest15311-21bash23:18:315
373899112,4cyclictest0-21swapper/521:11:595
373899111,5cyclictest31884-21bash21:21:505
373899111,4cyclictest17167-21iostat_ios23:20:275
373899110,5cyclictest15107-21sshd00:13:565
373899110,5cyclictest15107-21sshd00:13:565
373899110,5cyclictest14635-21systemd22:28:515
373899110,4cyclictest16033-21kworker/5:022:44:455
373899110,4cyclictest16033-21kworker/5:022:44:455
373899110,1cyclictest27993-21systemd-cgroups23:02:445
373899110,1cyclictest0-21swapper/523:11:275
373899110,0cyclictest0-21swapper/521:58:175
373799119,1cyclictest49-21ksoftirqd/422:52:574
373799119,1cyclictest49-21ksoftirqd/421:30:264
373799112,4cyclictest30073-21sshd21:45:364
373799112,4cyclictest30073-21sshd21:45:364
373799112,4cyclictest20866-21sshd23:37:224
373799112,4cyclictest20866-21sshd23:37:224
3737991110,1cyclictest0-21swapper/421:22:404
3737991110,1cyclictest0-21swapper/400:23:584
3737991110,1cyclictest0-21swapper/400:23:584
373799111,4cyclictest27771-21sshd00:34:544
373799111,4cyclictest27771-21sshd00:34:544
373799110,10cyclictest721-21dbus-daemon23:09:334
373699119,1cyclictest41-21ksoftirqd/320:30:123
373699119,1cyclictest41-21ksoftirqd/320:30:123
373699110,1cyclictest41-21ksoftirqd/320:40:233
373599119,1cyclictest33-21ksoftirqd/223:38:372
373599119,1cyclictest33-21ksoftirqd/223:38:372
373599118,3cyclictest33-21ksoftirqd/222:45:492
373599117,3cyclictest33-21ksoftirqd/222:17:292
373599112,3cyclictest29182-21systemd21:06:282
3735991110,1cyclictest33-21ksoftirqd/221:22:412
3735991110,1cyclictest33-21ksoftirqd/221:15:512
373599110,4cyclictest6176-21grepconf.sh00:17:052
373599110,1cyclictest0-21swapper/222:31:092
373499110,9cyclictest13782-21vmstat20:15:331
373499110,10cyclictest6066-21tr20:05:231
373499110,1cyclictest0-21swapper/121:02:501
373499110,0cyclictest0-21swapper/120:04:441
373399119,1cyclictest9-21ksoftirqd/022:16:380
373399118,2cyclictest31979-21grepconf.sh22:13:220
373399112,4cyclictest26551-21sshd21:52:490
373399111,4cyclictest25640-21sshd22:44:070
373399111,4cyclictest25640-21sshd22:44:070
373399111,4cyclictest20926-21sshd00:30:420
373399111,4cyclictest20926-21sshd00:30:420
373399111,4cyclictest18148-21sshd21:35:430
373399110,11cyclictest6806-21kworker/0:223:30:520
373399110,11cyclictest6806-21kworker/0:223:30:520
285932110,1sleep61-21systemd22:49:496
374599108,1cyclictest73-21ksoftirqd/720:10:117
373999104,1cyclictest1258-21sshd21:36:376
373999103,1cyclictest0-21swapper/622:05:366
373999102,2cyclictest721-21dbus-daemon23:43:126
373999101,4cyclictest11841-21sshd22:50:356
373999100,4cyclictest6965-21sshd00:17:086
373999100,4cyclictest16639-21sshd23:55:346
373999100,4cyclictest0-21swapper/623:50:526
373899108,1cyclictest57-21ksoftirqd/523:34:225
373899108,1cyclictest57-21ksoftirqd/523:34:225
373899105,4cyclictest709-21systemd-logind22:34:005
373899105,4cyclictest24049-21sshd21:25:165
373899103,4cyclictest24411-21bash22:03:525
373899102,4cyclictest30126-21bash00:33:075
373899102,4cyclictest30126-21bash00:33:075
373899101,6cyclictest721-21dbus-daemon00:17:185
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional