You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-10 - 07:38
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Fri Jul 10, 2026 00:43:26)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
16131999120,36cyclictest5842-21systemd21:15:394
16139998335,14cyclictest5986-21systemctl21:15:405
16147998147,0cyclictest0-21swapper/621:15:396
327252540,2sleep51613999cyclictest23:14:065
127872530,1sleep7711rcuc/722:49:577
267042500,0sleep60-21swapper/621:52:136
267042500,0sleep60-21swapper/621:52:136
165012490,2sleep41613199cyclictest21:48:034
132832490,1sleep213256-21bash23:04:102
52442480,1sleep557-21ksoftirqd/500:18:295
42092480,1sleep125-21ksoftirqd/122:42:221
303552470,0sleep30-21swapper/322:10:133
130992470,0sleep00-21swapper/022:58:470
92462460,0sleep50-21swapper/522:01:585
41842460,1sleep04183-21grepconf.sh21:29:410
200052460,1sleep09-21ksoftirqd/000:03:190
197012460,1sleep319650-21sshd22:16:453
165442460,0sleep0101ktimersoftd/023:50:450
277042450,1sleep727705-21sshd21:27:247
240282450,0sleep40-21swapper/400:19:334
211712450,1sleep321173-21bash22:59:143
191772450,2sleep71615899cyclictest22:18:267
75242440,1sleep07522-21systemd-cgroups23:02:080
206262440,1sleep3721-21dbus-daemon21:51:543
166732440,0sleep30-21swapper/322:20:063
14252440,0sleep10-21swapper/121:38:231
99082430,1sleep69904-21systemd23:14:396
263712430,0sleep60-21swapper/621:29:076
245212430,1sleep3401ktimersoftd/323:11:563
1571424330,9sleep40-21swapper/419:03:244
108692430,0sleep30-21swapper/322:44:333
324372420,0sleep00-21swapper/023:42:360
205042420,0sleep40-21swapper/423:29:274
171142420,0sleep60-21swapper/623:57:486
125772420,0sleep20-21swapper/222:25:092
190832410,1sleep419072-21sshd22:14:544
1588924129,8sleep50-21swapper/519:05:415
1577824130,8sleep10-21swapper/119:04:091
139362410,1sleep557-21ksoftirqd/500:11:575
102002410,0sleep20-21swapper/223:43:102
221272400,0sleep30-21swapper/322:06:123
125922400,0sleep10-21swapper/100:27:471
113282400,0sleep50-21swapper/522:39:145
10812400,0sleep40-21swapper/423:33:434
83132390,0sleep00-21swapper/022:08:570
1611599390,1cyclictest5908-21sshd21:15:402
1579323928,8sleep70-21swapper/719:04:237
1573523927,8sleep20-21swapper/219:03:382
1394723925,9sleep00-21swapper/019:02:160
1350523927,8sleep60-21swapper/619:02:066
52962380,0sleep20-21swapper/222:51:202
1577123827,8sleep30-21swapper/319:04:033
16099993332,1cyclictest25552-21kworker/0:521:15:400
16107993130,1cyclictest21575-21kworker/1:121:15:401
1612399247,16cyclictest0-21swapper/321:15:403
16107992410,3cyclictest25-21ksoftirqd/122:38:211
1610799239,1cyclictest25-21ksoftirqd/121:11:571
1610799235,2cyclictest25-21ksoftirqd/100:05:361
16107992310,2cyclictest25-21ksoftirqd/123:26:481
198752220,0sleep20-21swapper/221:46:332
1615899224,6cyclictest30099-21sshd00:03:547
1615899223,5cyclictest12800-21sshd23:52:187
1610799229,1cyclictest25-21ksoftirqd/121:58:151
1610799227,2cyclictest25-21ksoftirqd/123:38:451
1615899214,6cyclictest721-21dbus-daemon00:30:107
1615899214,4cyclictest1-21systemd21:56:567
1615899214,4cyclictest1-21systemd21:56:567
1615899213,2cyclictest1-21systemd22:46:257
1615899212,7cyclictest0-21swapper/721:38:047
1610799219,1cyclictest25-21ksoftirqd/122:23:421
1610799212,1cyclictest25-21ksoftirqd/121:18:511
16107992111,1cyclictest25-21ksoftirqd/123:27:361
1615899203,5cyclictest7337-21grepconf.sh22:54:587
1615899203,5cyclictest7337-21grepconf.sh22:54:587
1615899203,5cyclictest28895-21sshd22:24:167
1615899202,5cyclictest0-21swapper/700:12:167
1615899202,4cyclictest19922-21bash23:29:257
1610799208,2cyclictest25-21ksoftirqd/123:12:161
1610799207,2cyclictest25-21ksoftirqd/123:10:451
1610799204,3cyclictest25-21ksoftirqd/100:07:051
16107992014,4cyclictest25-21ksoftirqd/122:51:541
16107992012,4cyclictest25-21ksoftirqd/123:42:221
222202190,0sleep70-21swapper/723:58:057
194662190,0sleep00-21swapper/022:18:270
1615899194,5cyclictest25915-21bash23:26:147
1615899193,6cyclictest8542-21systemd21:10:237
1615899193,6cyclictest25657-21rm23:11:597
1615899193,6cyclictest21959-21grep22:38:017
1615899193,6cyclictest1-21systemd00:10:107
1615899193,5cyclictest29107-21kill23:01:307
1615899192,5cyclictest27153-21cp23:03:077
1610799199,1cyclictest25-21ksoftirqd/121:30:231
1610799196,3cyclictest25-21ksoftirqd/100:22:241
1610799195,5cyclictest25-21ksoftirqd/122:03:421
16107991917,1cyclictest25-21ksoftirqd/123:59:051
247392180,0sleep00-21swapper/000:08:550
1615899184,3cyclictest19861-21grepconf.sh23:17:027
1615899184,2cyclictest26903-21grepconf.sh23:17:207
1615899183,5cyclictest9105-21bash21:33:297
1615899183,5cyclictest31051-21grep22:27:537
1615899183,5cyclictest20739-21grepconf.sh22:07:517
1615899183,5cyclictest19710-21grepconf.sh23:38:207
1615899183,5cyclictest18277-21grepconf.sh21:49:587
1615899183,4cyclictest24205-21bash00:19:337
1615899183,2cyclictest721-21dbus-daemon23:48:297
1615899183,2cyclictest13990-21systemd-cgroups22:02:127
1615899183,14cyclictest464-21systemd-journal21:16:027
1615899182,3cyclictest0-21swapper/721:44:037
1615899181,3cyclictest0-21swapper/721:24:477
1610799187,3cyclictest25-21ksoftirqd/122:33:531
1610799187,1cyclictest25-21ksoftirqd/121:33:551
1610799186,2cyclictest25-21ksoftirqd/121:23:081
1610799186,1cyclictest25-21ksoftirqd/100:19:461
1610799185,2cyclictest25-21ksoftirqd/121:53:171
1610799185,2cyclictest25-21ksoftirqd/121:53:171
1610799185,1cyclictest121rcu_preempt22:10:301
1610799184,4cyclictest25-21ksoftirqd/123:47:421
1610799184,1cyclictest25-21ksoftirqd/100:32:501
16107991815,1cyclictest25-21ksoftirqd/122:57:031
16107991815,1cyclictest25-21ksoftirqd/122:57:031
16107991814,3cyclictest25-21ksoftirqd/121:43:491
1610799181,3cyclictest25-21ksoftirqd/123:17:221
160412180,0sleep20-21swapper/222:32:222
1615899173,5cyclictest7778-21systemd23:43:037
1615899173,5cyclictest23544-21grepconf.sh22:32:457
1615899172,5cyclictest9775-21grepconf.sh21:17:347
1615899172,5cyclictest31313-21grepconf.sh00:23:287
1615899172,5cyclictest22576-21grepconf.sh22:13:177
1615899172,4cyclictest19356-21bash23:34:457
1615899172,3cyclictest26190-21sshd00:35:417
16147991714,2cyclictest14885-21sshd21:49:466
16123991713,1cyclictest41-21ksoftirqd/323:39:313
16123991712,3cyclictest41-21ksoftirqd/323:13:043
1610799175,3cyclictest25-21ksoftirqd/100:14:141
1610799174,3cyclictest25-21ksoftirqd/123:53:221
1610799174,1cyclictest25-21ksoftirqd/122:59:361
1610799172,1cyclictest25-21ksoftirqd/122:18:591
16107991716,1cyclictest25-21ksoftirqd/123:04:541
16107991713,1cyclictest25-21ksoftirqd/122:15:371
16107991711,3cyclictest25-21ksoftirqd/122:27:221
93862160,0sleep0111rcuc/023:21:460
201152160,1sleep01-21systemd23:36:360
1615899163,6cyclictest30862-21grepconf.sh21:59:327
16147991612,1cyclictest65-21ksoftirqd/622:14:176
16107991613,3cyclictest25-21ksoftirqd/123:32:061
16107991513,1cyclictest25-21ksoftirqd/121:47:251
1609999150,11cyclictest7021-21sshd23:37:360
194212140,0sleep30-21swapper/321:21:433
1613199149,5cyclictest7027-21sshd23:57:184
1612399148,4cyclictest41-21ksoftirqd/321:27:073
16123991412,1cyclictest41-21ksoftirqd/300:21:233
16123991410,4cyclictest41-21ksoftirqd/300:27:383
16123991410,1cyclictest41-21ksoftirqd/322:38:213
1611599142,6cyclictest1728-21sshd22:38:422
16107991413,1cyclictest27226-21kworker/1:020:28:001
16099991412,1cyclictest9-21ksoftirqd/000:30:220
108102140,0sleep40-21swapper/422:21:354
64852130,0sleep6631rcuc/622:24:496
46442130,6sleep54635-21sshd21:47:245
276002130,0sleep60-21swapper/619:27:036
191002130,0sleep20-21swapper/221:30:312
16147991311,1cyclictest65-21ksoftirqd/621:22:016
1613999132,5cyclictest26890-21sshd22:29:275
1613999132,5cyclictest14733-21sshd00:33:135
1613999132,5cyclictest13962-21systemd-cgroups21:32:025
1613999130,5cyclictest23666-21sshd21:25:275
1613999130,12cyclictest721-21dbus-daemon22:07:185
16131991311,1cyclictest49-21ksoftirqd/423:45:374
16131991311,1cyclictest49-21ksoftirqd/400:03:444
16131991310,2cyclictest49-21ksoftirqd/422:22:484
1612399139,1cyclictest41-21ksoftirqd/323:02:073
1612399139,1cyclictest41-21ksoftirqd/321:53:103
1612399139,1cyclictest41-21ksoftirqd/321:53:103
1612399138,3cyclictest41-21ksoftirqd/323:24:323
16123991312,1cyclictest41-21ksoftirqd/300:13:433
16123991311,1cyclictest41-21ksoftirqd/322:33:193
1612399130,1cyclictest131rcu_sched23:50:373
1611599139,3cyclictest6618-21sshd22:44:182
1611599132,5cyclictest16141-21sshd22:11:132
16115991311,1cyclictest33-21ksoftirqd/223:35:182
16107991313,0cyclictest27226-21kworker/1:021:06:081
16107991313,0cyclictest27226-21kworker/1:021:06:081
16107991313,0cyclictest27226-21kworker/1:019:56:181
277032120,0sleep50-21swapper/521:55:495
277032120,0sleep50-21swapper/521:55:495
1614799122,4cyclictest2016-21bash22:05:056
16147991211,1cyclictest65-21ksoftirqd/623:51:436
16147991210,1cyclictest65-21ksoftirqd/623:32:156
16147991210,1cyclictest65-21ksoftirqd/600:16:036
1614799120,11cyclictest11776-21sshd22:57:026
1614799120,11cyclictest11776-21sshd22:57:026
1613999124,5cyclictest7962-21systemd22:12:285
1613999123,7cyclictest857-21polkitd23:10:595
1613999122,4cyclictest4362-21systemd-cgroups00:27:215
1613999122,4cyclictest24148-21sshd23:22:285
1613999122,4cyclictest11545-21sshd23:02:195
1613999122,4cyclictest1-21systemd22:53:455
1613999122,4cyclictest1-21systemd22:53:455
1613999122,4cyclictest0-21swapper/523:54:015
1613999121,4cyclictest0-21swapper/523:33:405
1613999120,5cyclictest3431-21bash23:37:235
1613999120,11cyclictest21438-21kworker/5:023:48:055
1613999120,11cyclictest17670-21kworker/5:321:45:355
16131991210,1cyclictest49-21ksoftirqd/421:34:094
16131991210,1cyclictest49-21ksoftirqd/400:12:044
1613199120,8cyclictest2058-21systemd22:59:594
1612399128,3cyclictest41-21ksoftirqd/323:42:423
16123991210,1cyclictest41-21ksoftirqd/323:32:423
16123991210,1cyclictest41-21ksoftirqd/321:59:553
16123991210,1cyclictest41-21ksoftirqd/321:44:413
16123991210,1cyclictest41-21ksoftirqd/300:09:253
16123991210,1cyclictest41-21ksoftirqd/300:01:223
1612399120,12cyclictest21368-21kworker/3:522:49:103
1612399120,1cyclictest0-21swapper/300:05:213
1611599124,7cyclictest27828-21cp23:13:502
1611599122,4cyclictest8062-21sshd00:13:192
1611599122,4cyclictest23521-21systemd-cgroups23:29:372
1611599122,4cyclictest16644-21bash00:31:382
16115991211,1cyclictest33-21ksoftirqd/221:58:582
16115991211,1cyclictest33-21ksoftirqd/200:08:412
16115991210,1cyclictest27917-21sshd00:17:572
1611599120,11cyclictest12303-21grepconf.sh00:33:052
16107991211,1cyclictest27226-21kworker/1:020:55:441
1609999122,4cyclictest20443-21sshd00:17:320
1609999122,4cyclictest0-21swapper/021:56:500
1609999122,4cyclictest0-21swapper/021:56:500
1609999120,11cyclictest19449-21kworker/0:322:12:270
67122111,1sleep665-21ksoftirqd/621:08:096
181192110,0sleep60-21swapper/622:28:586
1614799119,1cyclictest65-21ksoftirqd/623:20:356
1614799119,1cyclictest65-21ksoftirqd/622:48:106
1614799119,1cyclictest65-21ksoftirqd/622:41:516
1614799119,1cyclictest65-21ksoftirqd/621:45:056
1614799119,1cyclictest65-21ksoftirqd/600:29:436
1614799119,1cyclictest65-21ksoftirqd/600:04:446
1614799117,3cyclictest65-21ksoftirqd/600:21:416
1614799112,6cyclictest6802-21grepconf.sh23:05:376
1614799111,4cyclictest24816-21sshd22:36:306
1614799110,1cyclictest0-21swapper/623:56:186
1613999119,1cyclictest57-21ksoftirqd/523:58:055
1613999119,1cyclictest1-21systemd22:17:395
1613999119,1cyclictest0-21swapper/500:12:165
1613999113,7cyclictest13052-21sshd22:42:515
1613999112,4cyclictest30502-21grepconf.sh00:02:115
1613999112,4cyclictest0-21swapper/521:17:545
1613999111,4cyclictest11461-21grepconf.sh00:24:115
1613999110,5cyclictest1-21systemd22:50:015
1613999110,11cyclictest29207-21kworker/5:221:10:055
1613999110,11cyclictest13584-21kworker/5:222:57:365
1613999110,11cyclictest13293-21kworker/5:122:22:415
1613999110,11cyclictest0-21swapper/522:05:475
1613999110,1cyclictest0-21swapper/523:28:265
1613199119,1cyclictest49-21ksoftirqd/422:33:264
1613199119,1cyclictest49-21ksoftirqd/421:29:214
1613199119,1cyclictest49-21ksoftirqd/421:19:504
1613199119,1cyclictest49-21ksoftirqd/400:35:364
1613199114,4cyclictest721-21dbus-daemon23:07:234
1613199114,4cyclictest1423-21sshd23:21:194
1612399119,1cyclictest41-21ksoftirqd/323:28:033
1612399119,1cyclictest41-21ksoftirqd/323:19:133
1612399119,1cyclictest41-21ksoftirqd/300:32:233
1612399117,4cyclictest41-21ksoftirqd/321:37:203
1612399117,1cyclictest41-21ksoftirqd/321:25:393
16123991110,1cyclictest41-21ksoftirqd/323:52:213
1612399110,11cyclictest6159-21kworker/3:421:09:263
1611599119,1cyclictest709-21systemd-logind00:24:342
1611599119,1cyclictest33-21ksoftirqd/223:47:162
1611599119,1cyclictest33-21ksoftirqd/200:05:152
1611599119,1cyclictest2986-21sshd22:17:302
1611599113,2cyclictest12862-21sshd21:17:442
1611599112,3cyclictest33-21ksoftirqd/221:50:402
1611599111,4cyclictest20554-21sshd23:54:292
1611599111,4cyclictest17509-21systemd21:08:512
1611599110,5cyclictest22670-21cp23:17:112
1611599110,5cyclictest21008-21systemd-cgroups22:04:192
1611599110,5cyclictest16374-21sshd22:57:152
1611599110,11cyclictest10480-21kworker/2:421:37:342
1611599110,1cyclictest28519-21sshd22:27:452
1611599110,1cyclictest0-21swapper/221:52:172
1611599110,1cyclictest0-21swapper/221:52:172
16107991111,0cyclictest27226-21kworker/1:019:30:481
16107991111,0cyclictest27226-21kworker/1:019:30:481
16107991110,1cyclictest27226-21kworker/1:020:47:041
16107991110,1cyclictest27226-21kworker/1:020:03:441
16107991110,1cyclictest25-21ksoftirqd/119:52:031
1609999119,1cyclictest8544-21systemd-cgroups23:52:060
1609999113,2cyclictest10403-21sshd22:41:000
1609999112,4cyclictest0-21swapper/023:27:270
1609999112,4cyclictest0-21swapper/000:15:010
16099991110,1cyclictest0-21swapper/021:17:170
1609999110,11cyclictest31823-21kworker/0:221:24:190
1609999110,11cyclictest26459-21kworker/0:123:23:260
1615899102,5cyclictest6084-21/usr/sbin/munin20:42:187
1615899102,5cyclictest28851-21iostat_ios19:27:187
1615899102,5cyclictest28851-21iostat_ios19:27:187
1615899102,5cyclictest27962-21iostat20:22:187
1615899102,5cyclictest17038-21ntp_states20:02:207
1614799108,1cyclictest65-21ksoftirqd/623:24:076
1614799108,1cyclictest65-21ksoftirqd/623:07:266
1614799108,1cyclictest65-21ksoftirqd/621:23:286
1614799107,2cyclictest4218-21grepconf.sh00:34:286
1614799106,1cyclictest65-21ksoftirqd/623:27:216
1614799103,4cyclictest709-21systemd-logind22:18:236
1614799103,3cyclictest24534-21sshd21:59:106
1614799102,5cyclictest32554-21sshd23:39:056
1614799102,5cyclictest26156-21sshd21:37:576
1614799101,5cyclictest24806-21bash23:42:166
1614799100,6cyclictest12247-21grep00:08:136
1614799100,10cyclictest0-21swapper/622:59:246
1614799100,10cyclictest0-21swapper/600:25:096
1613999109,1cyclictest57-21ksoftirqd/521:40:085
1613999103,6cyclictest859-21gdbus23:17:335
1613999102,7cyclictest14670-21bash22:34:065
1613999100,10cyclictest25009-21kworker/5:423:44:115
1613999100,10cyclictest21563-21kworker/5:421:32:095
1613199109,1cyclictest0-21swapper/423:12:394
1613199108,1cyclictest49-21ksoftirqd/423:55:324
1613199108,1cyclictest49-21ksoftirqd/421:59:304
1613199106,3cyclictest21327-21sshd22:04:204
1613199105,1cyclictest20666-21memory23:22:194
1613199104,3cyclictest721-21dbus-daemon22:52:134
1613199104,3cyclictest721-21dbus-daemon22:52:134
1613199104,2cyclictest27497-21sshd21:45:104
1613199103,4cyclictest29438-21systemd00:07:234
1613199103,4cyclictest1-21systemd22:47:154
1613199103,2cyclictest23050-21sshd23:47:264
1613199102,4cyclictest0-21swapper/422:07:394
1613199100,9cyclictest26945-21kworker/4:023:03:174
1613199100,9cyclictest26576-21kworker/4:522:42:334
1613199100,9cyclictest1992-21kworker/4:121:37:234
1613199100,6cyclictest4940-21grepconf.sh23:39:214
1613199100,10cyclictest0-21swapper/422:28:514
1613199100,1cyclictest0-21swapper/400:31:384
1612399108,1cyclictest41-21ksoftirqd/322:56:193
1612399108,1cyclictest41-21ksoftirqd/322:56:193
1612399108,1cyclictest41-21ksoftirqd/321:34:413
1612399100,9cyclictest19117-21kworker/3:522:24:433
1612399100,9cyclictest18990-21kworker/3:600:25:163
1611599103,6cyclictest0-21swapper/223:25:262
1611599103,1cyclictest0-21swapper/223:09:412
1611599102,4cyclictest730-21NetworkManager21:25:102
1611599101,1cyclictest131rcu_sched21:32:052
1611599100,10cyclictest12631-21kworker/2:423:59:262
1610799109,1cyclictest27226-21kworker/1:020:58:421
1610799109,1cyclictest27226-21kworker/1:020:41:521
1610799109,1cyclictest27226-21kworker/1:020:08:561
1610799109,1cyclictest27226-21kworker/1:019:42:261
1610799109,1cyclictest25-21ksoftirqd/119:57:251
1610799108,1cyclictest25-21ksoftirqd/119:23:041
16107991010,0cyclictest27226-21kworker/1:019:35:501
1609999108,1cyclictest9-21ksoftirqd/022:27:530
1609999108,1cyclictest9-21ksoftirqd/022:02:140
1609999106,3cyclictest9-21ksoftirqd/000:00:430
1609999103,6cyclictest7846-21sshd00:34:400
1609999103,6cyclictest5432-21sshd23:10:510
1609999100,4cyclictest6292-21sshd21:35:090
1609999100,4cyclictest464-21systemd-journal00:26:220
1609999100,4cyclictest3419-21sshd21:57:570
1609999100,4cyclictest15065-21sshd22:55:240
1609999100,4cyclictest15065-21sshd22:55:240
1609999100,4cyclictest1-21systemd22:23:070
161589992,6cyclictest0-21swapper/719:47:177
161589992,4cyclictest5049-21/usr/sbin/munin19:42:187
161589992,4cyclictest30362-21diskstats20:27:157
161589992,4cyclictest25997-21cat20:17:237
161589992,4cyclictest20198-21systemd21:07:037
161589992,4cyclictest20198-21systemd21:07:037
161479997,1cyclictest65-21ksoftirqd/622:08:266
161479991,5cyclictest19370-21systemd21:32:176
161479990,3cyclictest328-21sshd22:42:156
161319997,1cyclictest49-21ksoftirqd/421:53:264
161319997,1cyclictest49-21ksoftirqd/421:53:264
161319993,3cyclictest31015-21sshd21:07:374
161319992,4cyclictest32576-21missed_timers21:22:224
161319990,8cyclictest31047-21kworker/4:100:23:144
161319990,8cyclictest16965-21kworker/4:422:37:114
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional