You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-23 - 09:17
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Sun Nov 23, 2025 00:43:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
253112980,2sleep72699899cyclictest21:20:547
219712870,2sleep22695899cyclictest23:17:182
169232810,0sleep50-21swapper/522:49:345
2570325026,9sleep30-21swapper/318:49:413
2570325026,9sleep30-21swapper/318:49:413
2655824825,8sleep10-21swapper/118:52:111
2655824825,8sleep10-21swapper/118:52:111
74952470,0sleep60-21swapper/623:10:416
10342440,0sleep70-21swapper/700:09:377
44142430,1sleep22695899cyclictest20:54:542
75152420,0sleep00-21swapper/000:05:450
52962420,0sleep30-21swapper/323:33:053
2669824129,8sleep70-21swapper/718:54:097
2669824129,8sleep70-21swapper/718:54:097
2651624128,9sleep00-21swapper/018:51:380
2651624128,9sleep00-21swapper/018:51:380
229122410,0sleep10-21swapper/122:50:181
151342410,0sleep70-21swapper/722:08:437
131182410,0sleep00-21swapper/023:24:530
103142410,0sleep70-21swapper/723:06:427
312392400,0sleep40-21swapper/419:54:434
312392400,0sleep40-21swapper/419:54:434
2659524028,9sleep20-21swapper/218:52:442
2659524028,9sleep20-21swapper/218:52:442
2637624027,9sleep40-21swapper/418:49:504
2637624027,9sleep40-21swapper/418:49:504
259362400,2sleep62699099cyclictest21:16:376
196092400,0sleep20-21swapper/222:49:472
178862400,2sleep52698299cyclictest22:40:295
2640923927,9sleep60-21swapper/618:50:166
2640923927,9sleep60-21swapper/618:50:166
2423223927,8sleep50-21swapper/518:49:275
2423223927,8sleep50-21swapper/518:49:275
56162380,0sleep40-21swapper/421:09:224
10082380,0sleep20-21swapper/223:27:592
325812370,0sleep30-21swapper/323:59:563
181092360,0sleep40-21swapper/421:29:004
65722340,2sleep72699899cyclictest00:19:467
216642260,0sleep40-21swapper/422:04:564
2696699234,18cyclictest27509-21bash23:04:343
26998992212,3cyclictest73-21ksoftirqd/723:31:347
26998992117,4cyclictest73-21ksoftirqd/723:23:337
2699899208,1cyclictest73-21ksoftirqd/722:48:537
2699899200,20cyclictest0-21swapper/720:45:457
2698299202,6cyclictest0-21swapper/520:47:595
100782200,0sleep70-21swapper/721:45:377
2699899197,12cyclictest73-21ksoftirqd/722:24:437
2699899195,2cyclictest73-21ksoftirqd/721:49:347
2699899194,15cyclictest0-21swapper/721:27:217
26998991916,1cyclictest73-21ksoftirqd/722:31:227
26998991913,6cyclictest73-21ksoftirqd/723:40:037
231282190,2sleep12695099cyclictest00:03:301
307962180,1sleep01-21systemd21:17:190
26998991812,1cyclictest73-21ksoftirqd/721:06:437
2698299187,0cyclictest0-21swapper/522:20:415
2698299181,5cyclictest0-21swapper/523:18:115
2698299180,6cyclictest0-21swapper/519:24:005
2698299180,6cyclictest0-21swapper/519:12:085
235252180,0sleep40-21swapper/422:59:344
2699899178,2cyclictest73-21ksoftirqd/722:59:507
2699899175,3cyclictest73-21ksoftirqd/722:57:377
2699899175,0cyclictest73-21ksoftirqd/723:56:257
2699899174,3cyclictest73-21ksoftirqd/723:09:427
2699899174,3cyclictest73-21ksoftirqd/721:37:147
2699899174,0cyclictest0-21swapper/721:32:337
26998991712,0cyclictest73-21ksoftirqd/720:40:437
26998991712,0cyclictest73-21ksoftirqd/720:40:437
2698299176,0cyclictest0-21swapper/521:03:475
26982991712,5cyclictest0-21swapper/522:15:435
2698299170,5cyclictest0-21swapper/522:29:575
2698299170,5cyclictest0-21swapper/521:47:355
2698299170,5cyclictest0-21swapper/521:38:235
2695099172,1cyclictest121rcu_preempt23:13:081
26950991716,1cyclictest14192-21kworker/1:423:51:331
26950991711,4cyclictest25-21ksoftirqd/121:51:291
2699899165,2cyclictest73-21ksoftirqd/700:01:107
2699899164,12cyclictest73-21ksoftirqd/721:01:417
2699899164,0cyclictest0-21swapper/722:02:117
26998991614,2cyclictest0-21swapper/719:46:407
26998991614,1cyclictest73-21ksoftirqd/723:34:317
26998991614,1cyclictest73-21ksoftirqd/721:39:337
26998991614,0cyclictest0-21swapper/720:09:327
26998991611,1cyclictest73-21ksoftirqd/723:25:157
2698299165,5cyclictest0-21swapper/523:07:175
2698299165,5cyclictest0-21swapper/520:03:405
2698299165,0cyclictest0-21swapper/521:53:295
26950991615,1cyclictest2813-21kworker/1:421:13:191
26950991614,1cyclictest25-21ksoftirqd/123:18:521
26950991614,1cyclictest25-21ksoftirqd/122:59:391
26950991614,1cyclictest25-21ksoftirqd/100:05:331
26950991611,3cyclictest25-21ksoftirqd/100:17:071
2699899157,1cyclictest73-21ksoftirqd/700:18:167
2699899157,0cyclictest0-21swapper/723:18:277
2699899156,1cyclictest73-21ksoftirqd/723:46:307
2699899155,3cyclictest73-21ksoftirqd/700:08:337
2699899153,12cyclictest0-21swapper/720:54:457
26998991513,2cyclictest0-21swapper/718:56:347
26998991513,1cyclictest73-21ksoftirqd/722:19:207
26998991513,1cyclictest73-21ksoftirqd/721:10:497
26998991513,0cyclictest0-21swapper/720:01:017
26998991511,0cyclictest0-21swapper/719:07:187
26998991510,5cyclictest25466-21kworker/7:119:38:307
26998991510,5cyclictest0-21swapper/719:39:547
26998991510,0cyclictest73-21ksoftirqd/722:41:157
2699899151,2cyclictest73-21ksoftirqd/720:23:137
2699899150,15cyclictest0-21swapper/719:20:507
2699899150,1cyclictest121rcu_preempt22:09:407
2698299155,9cyclictest5556-21sshd21:31:355
2698299154,5cyclictest0-21swapper/521:39:515
2698299150,9cyclictest0-21swapper/523:38:195
2698299150,9cyclictest0-21swapper/522:44:235
2698299150,9cyclictest0-21swapper/522:00:035
2698299150,9cyclictest0-21swapper/520:30:055
2698299150,9cyclictest0-21swapper/520:10:055
2698299150,9cyclictest0-21swapper/518:58:565
2698299150,3cyclictest0-21swapper/522:25:315
26958991513,1cyclictest33-21ksoftirqd/221:30:062
26958991513,1cyclictest33-21ksoftirqd/221:04:182
26950991514,1cyclictest29224-21kworker/1:122:05:391
26950991513,2cyclictest25-21ksoftirqd/100:19:491
26950991513,1cyclictest25-21ksoftirqd/123:24:241
26950991513,1cyclictest25-21ksoftirqd/123:04:361
26950991513,1cyclictest25-21ksoftirqd/122:02:261
26950991513,1cyclictest25-21ksoftirqd/121:47:291
26950991513,1cyclictest25-21ksoftirqd/121:34:321
26950991513,1cyclictest25-21ksoftirqd/121:26:341
26950991513,1cyclictest13816-21kworker/1:321:41:131
26950991512,2cyclictest25-21ksoftirqd/123:19:211
26950991511,1cyclictest25-21ksoftirqd/122:59:021
26950991511,1cyclictest25-21ksoftirqd/122:49:181
2695099150,14cyclictest3505-21/usr/sbin/munin22:29:311
75862140,0sleep40-21swapper/422:39:164
2699899149,5cyclictest73-21ksoftirqd/719:26:467
2699899149,3cyclictest73-21ksoftirqd/722:21:077
2699899145,1cyclictest73-21ksoftirqd/723:49:297
2699899143,2cyclictest0-21swapper/720:35:257
2699899143,0cyclictest0-21swapper/720:49:337
26998991412,1cyclictest711-21dbus-daemon22:35:037
26998991412,0cyclictest0-21swapper/719:50:387
26998991410,3cyclictest73-21ksoftirqd/721:55:197
26998991410,0cyclictest0-21swapper/720:14:437
26998991410,0cyclictest0-21swapper/719:03:207
2699099148,6cyclictest0-21swapper/600:16:536
2698299149,5cyclictest0-21swapper/523:46:015
2698299148,1cyclictest19262-21sshd23:21:215
2698299147,6cyclictest23219-21bash21:24:595
2698299142,6cyclictest0-21swapper/523:59:275
2698299142,6cyclictest0-21swapper/523:54:575
2698299142,6cyclictest0-21swapper/523:32:535
2698299142,6cyclictest0-21swapper/519:35:005
2698299142,6cyclictest0-21swapper/519:25:245
2698299142,6cyclictest0-21swapper/519:07:005
2698299142,0cyclictest0-21swapper/520:14:245
2698299142,0cyclictest0-21swapper/520:08:135
2698299140,9cyclictest0-21swapper/523:51:255
2698299140,9cyclictest0-21swapper/523:27:135
2698299140,9cyclictest0-21swapper/520:36:475
2698299140,8cyclictest0-21swapper/523:12:175
2698299140,8cyclictest0-21swapper/521:19:355
2698299140,8cyclictest0-21swapper/521:14:535
2698299140,8cyclictest0-21swapper/520:56:235
2698299140,8cyclictest0-21swapper/519:42:465
2696699140,14cyclictest99-21kswapd019:20:093
26958991413,1cyclictest33-21ksoftirqd/221:46:302
26958991412,1cyclictest33-21ksoftirqd/223:01:072
26958991412,1cyclictest33-21ksoftirqd/222:14:372
26958991411,2cyclictest5032-21sshd23:56:042
26950991413,1cyclictest31459-21kworker/1:223:48:051
26950991413,1cyclictest25-21ksoftirqd/122:40:521
26950991413,1cyclictest25-21ksoftirqd/121:29:261
26950991413,1cyclictest25-21ksoftirqd/121:15:491
26950991413,1cyclictest1973-21kworker/1:121:05:091
26950991412,1cyclictest25-21ksoftirqd/123:54:391
26950991412,1cyclictest25-21ksoftirqd/123:42:481
26950991412,1cyclictest25-21ksoftirqd/122:25:421
26950991412,1cyclictest25-21ksoftirqd/122:19:201
26950991412,1cyclictest25-21ksoftirqd/100:09:471
26950991412,1cyclictest1973-21kworker/1:120:19:361
26950991410,1cyclictest25-21ksoftirqd/122:37:461
2699899138,3cyclictest73-21ksoftirqd/721:14:577
2699899137,5cyclictest73-21ksoftirqd/719:09:407
2699899133,1cyclictest73-21ksoftirqd/722:49:297
26998991311,1cyclictest73-21ksoftirqd/720:09:207
2699899130,13cyclictest73-21ksoftirqd/719:19:067
26990991311,1cyclictest85350irq/25-em1-rx-022:37:056
2698299138,5cyclictest0-21swapper/522:04:475
2698299137,3cyclictest57-21ksoftirqd/522:12:365
2698299132,6cyclictest0-21swapper/523:41:315
2698299132,6cyclictest0-21swapper/523:00:195
2698299132,6cyclictest0-21swapper/500:23:555
2698299132,5cyclictest0-21swapper/519:51:105
2698299132,5cyclictest0-21swapper/519:45:225
2698299132,5cyclictest0-21swapper/519:33:245
2698299132,11cyclictest0-21swapper/522:57:075
2698299132,0cyclictest0-21swapper/519:18:265
2698299132,0cyclictest0-21swapper/500:17:395
2698299131,0cyclictest0-21swapper/522:35:115
2698299130,8cyclictest0-21swapper/521:10:215
2696699130,2cyclictest29027-21bash23:18:203
26958991312,1cyclictest33-21ksoftirqd/222:54:362
26958991312,1cyclictest33-21ksoftirqd/221:29:052
26958991311,2cyclictest33-21ksoftirqd/221:57:132
26958991311,1cyclictest33-21ksoftirqd/223:22:132
26958991311,1cyclictest33-21ksoftirqd/222:29:322
26958991311,1cyclictest33-21ksoftirqd/221:39:332
2695899130,1cyclictest121rcu_preempt22:48:172
2695099138,2cyclictest4875-21bash20:54:591
26950991313,0cyclictest28706-21kworker/1:422:10:311
26950991313,0cyclictest1973-21kworker/1:120:45:451
26950991312,1cyclictest30573-21kworker/1:221:01:211
26950991312,1cyclictest25-21ksoftirqd/122:19:381
26950991312,1cyclictest25-21ksoftirqd/121:19:231
26950991312,1cyclictest14192-21kworker/1:423:39:051
26950991312,1cyclictest13816-21kworker/1:321:55:151
26950991311,1cyclictest25-21ksoftirqd/123:29:331
134982130,1sleep413499-21grep23:34:184
2699899127,5cyclictest73-21ksoftirqd/719:32:187
2699899124,6cyclictest73-21ksoftirqd/720:29:457
26998991210,2cyclictest73-21ksoftirqd/719:54:407
26998991210,2cyclictest73-21ksoftirqd/719:54:407
2699899120,0cyclictest0-21swapper/720:28:357
2699099126,6cyclictest0-21swapper/623:27:416
2698299121,6cyclictest0-21swapper/520:49:355
2698299121,5cyclictest0-21swapper/521:05:535
2698299120,6cyclictest0-21swapper/521:58:535
2698299120,6cyclictest0-21swapper/520:21:335
2698299120,6cyclictest0-21swapper/500:06:535
2698299120,0cyclictest0-21swapper/520:41:435
2698299120,0cyclictest0-21swapper/520:41:435
2698299120,0cyclictest0-21swapper/520:25:235
2698299120,0cyclictest0-21swapper/519:56:565
2698299120,0cyclictest0-21swapper/519:56:565
2698299120,0cyclictest0-21swapper/518:59:505
2697499122,4cyclictest7606-21sshd23:37:564
2697499120,1cyclictest0-21swapper/422:22:004
26966991211,1cyclictest0-21swapper/323:20:313
2696699120,11cyclictest9658-21bash00:06:063
2696699120,1cyclictest0-21swapper/322:01:243
2695899128,1cyclictest33-21ksoftirqd/223:38:452
26958991211,1cyclictest33-21ksoftirqd/222:12:182
26958991211,1cyclictest33-21ksoftirqd/221:34:542
26958991211,1cyclictest33-21ksoftirqd/221:10:322
26958991211,1cyclictest33-21ksoftirqd/200:22:542
26958991211,1cyclictest33-21ksoftirqd/200:18:032
26958991211,1cyclictest33-21ksoftirqd/200:06:272
26958991210,2cyclictest33-21ksoftirqd/222:23:452
26958991210,1cyclictest33-21ksoftirqd/223:45:542
26958991210,1cyclictest33-21ksoftirqd/222:05:222
26958991210,1cyclictest33-21ksoftirqd/221:49:372
26958991210,1cyclictest33-21ksoftirqd/221:22:292
26958991210,1cyclictest33-21ksoftirqd/221:06:222
26958991210,1cyclictest33-21ksoftirqd/200:03:252
26950991211,1cyclictest8650-21kworker/1:219:39:441
26950991211,1cyclictest8650-21kworker/1:219:29:401
2695099120,9cyclictest15581-21processes20:24:411
2694399124,4cyclictest0-21swapper/020:04:210
26943991210,2cyclictest779-21rm21:44:310
228152120,0sleep40-21swapper/419:41:304
110992120,0sleep00-21swapper/021:32:240
104512120,0sleep30-21swapper/323:29:203
2699099112,9cyclictest0-21swapper/622:46:156
26990991110,1cyclictest0-21swapper/621:49:356
2699099111,4cyclictest2733-21sshd23:23:426
2699099111,1cyclictest544-21sshd21:26:266
2698299110,0cyclictest0-21swapper/500:10:055
2697499110,10cyclictest31835-21sshd22:15:174
2697499110,10cyclictest11269-21systemd-cgroups23:11:154
2696699115,4cyclictest531-21sshd22:06:353
2696699110,10cyclictest711-21dbus-daemon22:17:003
2695899119,1cyclictest33-21ksoftirqd/223:13:082
2695899118,3cyclictest33-21ksoftirqd/223:31:132
26958991110,1cyclictest33-21ksoftirqd/222:35:442
26958991110,1cyclictest33-21ksoftirqd/222:27:022
26958991110,1cyclictest33-21ksoftirqd/221:14:362
2695899110,3cyclictest0-21swapper/223:49:412
2695099119,1cyclictest25-21ksoftirqd/120:09:351
2695099119,1cyclictest25-21ksoftirqd/119:59:431
26950991111,0cyclictest32356-21kworker/1:119:10:261
26950991111,0cyclictest32356-21kworker/1:119:07:081
199422110,0sleep40-21swapper/400:17:114
2699099108,1cyclictest65-21ksoftirqd/621:54:486
2699099105,2cyclictest4698-21bash23:32:596
2699099105,1cyclictest0-21swapper/622:05:376
2699099103,6cyclictest15929-21sshd23:03:046
2699099102,1cyclictest31887-21sshd23:59:496
2699099101,4cyclictest4380-21sshd21:04:366
2699099100,1cyclictest711-21dbus-daemon20:58:026
2699099100,1cyclictest0-21swapper/600:21:556
2697499103,6cyclictest27307-21sshd22:32:484
2697499100,10cyclictest0-21swapper/421:54:004
2697499100,1cyclictest0-21swapper/422:41:304
2696699108,1cyclictest22423-21sshd23:54:073
2695899109,1cyclictest33-21ksoftirqd/223:39:422
2695899109,1cyclictest33-21ksoftirqd/221:59:372
2695899108,1cyclictest33-21ksoftirqd/223:04:312
2695899108,1cyclictest33-21ksoftirqd/222:40:002
2695899100,1cyclictest1-21systemd00:10:322
2695099109,1cyclictest8650-21kworker/1:219:38:301
2695099109,1cyclictest8650-21kworker/1:219:26:021
2695099109,1cyclictest25-21ksoftirqd/120:04:391
2695099109,1cyclictest0-21swapper/118:54:351
2695099108,1cyclictest25-21ksoftirqd/119:59:211
2695099108,1cyclictest25-21ksoftirqd/119:59:211
2694399100,10cyclictest0-21swapper/022:43:380
269909999,0cyclictest0-21swapper/623:05:166
269909997,1cyclictest711-21dbus-daemon22:26:576
269909997,1cyclictest65-21ksoftirqd/623:50:146
269909997,1cyclictest65-21ksoftirqd/621:42:166
269909997,1cyclictest65-21ksoftirqd/621:01:186
269909993,6cyclictest0-21swapper/621:29:436
269909993,3cyclictest21880-21sshd21:47:236
269909993,1cyclictest8564-21sshd21:36:296
269909992,6cyclictest25534-21sshd23:35:466
269909990,9cyclictest0-21swapper/622:40:496
269909990,9cyclictest0-21swapper/621:09:426
269909990,9cyclictest0-21swapper/600:06:546
269749999,0cyclictest0-21swapper/422:48:434
269749999,0cyclictest0-21swapper/420:56:184
269749999,0cyclictest0-21swapper/400:03:264
269749994,1cyclictest0-21swapper/422:02:494
269749993,6cyclictest0-21swapper/423:54:414
269749993,5cyclictest13698-21sshd21:14:464
269749991,3cyclictest0-21swapper/420:54:214
269749990,9cyclictest0-21swapper/400:22:314
269749990,0cyclictest0-21swapper/422:50:554
269749990,0cyclictest0-21swapper/421:44:424
269749990,0cyclictest0-21swapper/421:42:304
269749990,0cyclictest0-21swapper/421:04:024
269669999,0cyclictest0-21swapper/322:34:493
269669999,0cyclictest0-21swapper/300:20:173
269669997,1cyclictest41-21ksoftirqd/323:47:183
269669997,1cyclictest41-21ksoftirqd/321:52:243
269669997,1cyclictest2246-21kworker/3:222:23:113
269669996,2cyclictest9978-21rm21:27:493
269669994,2cyclictest669-21avahi-daemon23:37:123
269669990,9cyclictest0-21swapper/323:13:263
269669990,9cyclictest0-21swapper/321:55:143
269669990,0cyclictest0-21swapper/323:04:083
269509999,0cyclictest8650-21kworker/1:219:51:521
269439999,0cyclictest0-21swapper/023:49:320
269439999,0cyclictest0-21swapper/021:42:180
269439999,0cyclictest0-21swapper/021:24:390
269439999,0cyclictest0-21swapper/000:15:140
269439993,1cyclictest27904-21systemd-cgroups21:21:170
269439992,6cyclictest28105-21nfsd22:05:580
269439990,9cyclictest0-21swapper/023:15:570
269439990,9cyclictest0-21swapper/023:08:510
269439990,9cyclictest0-21swapper/022:30:140
269439990,9cyclictest0-21swapper/021:10:560
269439990,1cyclictest1-21systemd23:36:450
23310290,1sleep60-21swapper/622:09:406
20344290,2sleep32696699cyclictest21:02:243
6312280,0sleep40-21swapper/421:22:514
269909988,0cyclictest0-21swapper/622:29:386
269909986,1cyclictest65-21ksoftirqd/622:54:066
269909986,1cyclictest65-21ksoftirqd/622:19:216
269909986,1cyclictest65-21ksoftirqd/620:44:216
269909986,1cyclictest65-21ksoftirqd/620:44:216
269909986,1cyclictest65-21ksoftirqd/600:13:476
269909984,3cyclictest19311-21sshd23:48:506
269909983,5cyclictest0-21swapper/622:14:246
269909983,2cyclictest859-21polkitd23:16:576
269909982,3cyclictest19708-21diskstats22:54:336
269909981,6cyclictest31809-21sshd22:02:036
269749988,0cyclictest0-21swapper/423:23:144
269749988,0cyclictest0-21swapper/422:55:204
269749988,0cyclictest0-21swapper/422:10:244
269749988,0cyclictest0-21swapper/421:56:134
269749988,0cyclictest0-21swapper/400:10:324
269749986,1cyclictest49-21ksoftirqd/423:43:194
269749986,1cyclictest49-21ksoftirqd/421:06:044
269749986,1cyclictest49-21ksoftirqd/400:06:044
269749985,2cyclictest4672-21grep23:51:184
269749985,2cyclictest29049-21grep21:34:434
269749983,1cyclictest0-21swapper/423:04:234
269749981,6cyclictest2888-21sshd21:31:124
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional