You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 21:37
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Fri Nov 21, 2025 12:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
80422970,2sleep7764499cyclictest10:04:387
724225625,18sleep00-21swapper/006:51:330
278702540,0sleep70-21swapper/710:26:327
717625026,8sleep10-21swapper/106:50:381
737824926,9sleep30-21swapper/306:53:203
712724825,8sleep60-21swapper/606:49:556
165732470,1sleep716578-21cpuspeed_turbos07:09:337
271862460,0sleep60-21swapper/611:42:156
67122440,1sleep46707-21bash09:54:504
138042440,0sleep30-21swapper/307:59:413
80722420,0sleep40-21swapper/411:25:244
263422420,0sleep00-21swapper/012:04:120
253812420,1sleep025382-21latency_hist09:39:240
81022410,0sleep00-21swapper/009:45:570
305742410,0sleep50-21swapper/508:29:385
196902410,2sleep4762199cyclictest09:47:454
131812410,0sleep40-21swapper/410:29:094
725324028,9sleep20-21swapper/206:51:432
720424027,9sleep70-21swapper/706:51:007
265242400,0sleep40-21swapper/410:30:544
713523927,8sleep50-21swapper/506:50:025
497923928,8sleep40-21swapper/406:49:284
272412390,0sleep00-21swapper/009:01:520
262992380,0sleep70-21swapper/711:04:367
34072320,2sleep3761099cyclictest09:03:073
165992310,2sleep585450irq/26-em1-tx-010:43:525
764499255,15cyclictest73-21ksoftirqd/709:42:297
232822240,0sleep10-21swapper/111:04:141
7644992318,5cyclictest29222-21kworker/7:106:54:417
764499227,9cyclictest73-21ksoftirqd/711:00:047
7644992115,5cyclictest73-21ksoftirqd/711:30:207
7644992110,5cyclictest0-21swapper/709:00:537
764499207,0cyclictest0-21swapper/711:57:507
7644992015,4cyclictest73-21ksoftirqd/711:42:247
7644992015,0cyclictest0-21swapper/708:09:137
7644992014,1cyclictest73-21ksoftirqd/709:56:377
7644992011,4cyclictest73-21ksoftirqd/710:12:147
764499198,0cyclictest0-21swapper/712:04:267
764499198,0cyclictest0-21swapper/708:39:057
764499197,12cyclictest73-21ksoftirqd/712:24:007
7644991913,1cyclictest73-21ksoftirqd/711:16:447
764499190,1cyclictest4241-21kworker/7:309:51:097
7621991916,1cyclictest49-21ksoftirqd/410:51:244
764499186,6cyclictest73-21ksoftirqd/709:38:177
764499186,1cyclictest73-21ksoftirqd/711:48:147
764499184,8cyclictest0-21swapper/707:37:577
7644991814,4cyclictest73-21ksoftirqd/709:59:337
7644991813,0cyclictest0-21swapper/709:14:457
7644991812,1cyclictest73-21ksoftirqd/710:22:007
760299187,1cyclictest33-21ksoftirqd/209:42:572
760299185,1cyclictest33-21ksoftirqd/209:15:532
760299184,1cyclictest33-21ksoftirqd/212:15:392
760299184,1cyclictest33-21ksoftirqd/211:51:302
764499176,5cyclictest73-21ksoftirqd/708:56:397
764499175,0cyclictest0-21swapper/707:50:357
7644991712,5cyclictest73-21ksoftirqd/710:16:567
7644991711,1cyclictest73-21ksoftirqd/711:37:127
764499170,11cyclictest0-21swapper/712:02:427
762199172,3cyclictest49-21ksoftirqd/410:01:074
7621991717,0cyclictest49-21ksoftirqd/407:58:164
7621991714,3cyclictest49-21ksoftirqd/409:38:514
7621991714,2cyclictest49-21ksoftirqd/410:11:474
760299178,1cyclictest33-21ksoftirqd/212:06:452
760299178,1cyclictest33-21ksoftirqd/211:43:332
760299177,3cyclictest33-21ksoftirqd/210:28:492
760299177,1cyclictest33-21ksoftirqd/211:24:432
7602991713,1cyclictest33-21ksoftirqd/211:02:132
7602991713,1cyclictest33-21ksoftirqd/210:07:442
89272160,0sleep00-21swapper/011:57:230
764499164,6cyclictest73-21ksoftirqd/710:51:087
7644991611,5cyclictest73-21ksoftirqd/710:40:307
7644991611,5cyclictest73-21ksoftirqd/710:30:247
7644991611,5cyclictest721ktimersoftd/709:13:137
7644991611,5cyclictest0-21swapper/709:49:157
7644991611,5cyclictest0-21swapper/707:43:297
7644991611,5cyclictest0-21swapper/707:43:297
7644991611,0cyclictest0-21swapper/708:24:197
7644991611,0cyclictest0-21swapper/707:01:377
7644991611,0cyclictest0-21swapper/707:01:377
7644991610,0cyclictest0-21swapper/708:47:217
764499160,16cyclictest0-21swapper/709:08:097
762199167,2cyclictest49-21ksoftirqd/411:44:484
761099162,13cyclictest859-21polkitd11:10:453
760299169,2cyclictest33-21ksoftirqd/211:49:072
760299167,1cyclictest33-21ksoftirqd/210:22:272
760299166,1cyclictest33-21ksoftirqd/212:01:042
760299165,3cyclictest33-21ksoftirqd/209:07:412
7602991614,1cyclictest33-21ksoftirqd/211:35:012
7602991614,1cyclictest33-21ksoftirqd/211:34:162
7602991614,1cyclictest33-21ksoftirqd/209:22:582
7602991613,3cyclictest33-21ksoftirqd/209:53:182
7602991613,2cyclictest33-21ksoftirqd/212:20:222
7602991611,2cyclictest33-21ksoftirqd/212:09:382
760299161,1cyclictest33-21ksoftirqd/210:54:382
760299160,3cyclictest0-21swapper/210:17:162
149682160,3sleep10-21swapper/110:34:031
764499159,1cyclictest73-21ksoftirqd/709:27:117
764499158,1cyclictest73-21ksoftirqd/710:47:167
764499156,4cyclictest73-21ksoftirqd/711:10:047
764499154,5cyclictest0-21swapper/707:45:477
764499154,5cyclictest0-21swapper/707:24:537
764499154,0cyclictest0-21swapper/708:31:297
764499154,0cyclictest0-21swapper/708:01:537
7644991512,3cyclictest73-21ksoftirqd/709:19:297
7644991511,4cyclictest0-21swapper/707:58:397
7644991511,0cyclictest0-21swapper/708:42:477
7644991511,0cyclictest0-21swapper/708:42:477
7644991510,5cyclictest0-21swapper/708:53:477
7644991510,0cyclictest28105-21nfsd07:23:597
7644991510,0cyclictest0-21swapper/709:31:557
7644991510,0cyclictest0-21swapper/707:15:197
764499151,14cyclictest27384-21kworker/7:010:56:067
764499150,15cyclictest0-21swapper/711:26:487
762199153,1cyclictest49-21ksoftirqd/411:01:054
7621991513,1cyclictest49-21ksoftirqd/409:33:074
7621991512,3cyclictest49-21ksoftirqd/411:32:324
7621991510,3cyclictest49-21ksoftirqd/411:56:504
760299159,3cyclictest33-21ksoftirqd/211:56:432
760299156,1cyclictest33-21ksoftirqd/211:09:352
760299156,1cyclictest33-21ksoftirqd/210:34:462
760299156,1cyclictest33-21ksoftirqd/209:02:222
760299155,2cyclictest33-21ksoftirqd/209:10:212
760299155,1cyclictest33-21ksoftirqd/211:17:562
760299155,1cyclictest33-21ksoftirqd/208:57:192
7602991514,1cyclictest33-21ksoftirqd/210:33:132
7602991514,1cyclictest33-21ksoftirqd/209:29:432
7602991513,1cyclictest33-21ksoftirqd/210:50:402
7602991513,1cyclictest33-21ksoftirqd/210:09:262
7602991513,1cyclictest33-21ksoftirqd/210:03:472
7602991513,1cyclictest33-21ksoftirqd/209:28:192
7602991511,3cyclictest33-21ksoftirqd/209:58:572
758699150,13cyclictest26379-21sshd10:11:560
758699150,12cyclictest99-21kswapd008:51:410
758699150,12cyclictest1-21systemd08:55:140
83432140,0sleep50-21swapper/511:01:545
764499149,5cyclictest73-21ksoftirqd/712:11:427
764499148,6cyclictest0-21swapper/707:05:577
764499144,5cyclictest73-21ksoftirqd/711:21:127
764499142,0cyclictest0-21swapper/708:10:157
7644991412,1cyclictest73-21ksoftirqd/712:14:467
7644991412,1cyclictest73-21ksoftirqd/711:53:147
7621991413,1cyclictest49-21ksoftirqd/411:49:544
7621991413,1cyclictest49-21ksoftirqd/410:09:034
7621991412,1cyclictest49-21ksoftirqd/412:04:364
7621991412,1cyclictest49-21ksoftirqd/409:24:544
7621991412,1cyclictest49-21ksoftirqd/409:01:584
7621991411,2cyclictest8744-21bash12:01:414
7621991411,1cyclictest49-21ksoftirqd/411:12:114
7621991410,1cyclictest49-21ksoftirqd/412:22:174
762199140,2cyclictest131rcu_sched12:18:134
762199140,2cyclictest131rcu_sched11:43:584
762199140,1cyclictest121rcu_preempt11:07:464
760299146,1cyclictest33-21ksoftirqd/211:04:502
760299145,1cyclictest33-21ksoftirqd/209:49:032
7602991413,1cyclictest33-21ksoftirqd/210:48:342
7602991412,1cyclictest33-21ksoftirqd/211:19:372
764499138,5cyclictest0-21swapper/708:14:577
764499138,1cyclictest73-21ksoftirqd/708:27:597
764499138,1cyclictest29222-21kworker/7:107:30:457
7636991312,1cyclictest65-21ksoftirqd/611:25:516
762999132,6cyclictest0-21swapper/511:06:425
762199138,3cyclictest49-21ksoftirqd/411:37:064
762199133,3cyclictest49-21ksoftirqd/409:08:284
7621991312,1cyclictest49-21ksoftirqd/410:39:414
7621991312,1cyclictest49-21ksoftirqd/409:39:364
7621991312,1cyclictest49-21ksoftirqd/409:19:394
7621991311,1cyclictest49-21ksoftirqd/412:09:364
7621991311,1cyclictest49-21ksoftirqd/411:15:294
7621991311,1cyclictest49-21ksoftirqd/410:56:204
7621991311,1cyclictest49-21ksoftirqd/410:19:124
7621991311,1cyclictest49-21ksoftirqd/409:09:444
762199130,1cyclictest49-21ksoftirqd/410:48:534
7602991312,1cyclictest33-21ksoftirqd/210:40:392
7602991312,1cyclictest33-21ksoftirqd/209:34:442
759499130,12cyclictest11183-21bash11:21:161
264252130,0sleep10-21swapper/110:40:121
134932130,0sleep30-21swapper/309:14:203
764499127,1cyclictest73-21ksoftirqd/710:35:167
762999128,3cyclictest57-21ksoftirqd/509:49:415
762999120,11cyclictest668-21systemd-logind11:17:395
762199124,1cyclictest49-21ksoftirqd/410:20:214
7621991210,1cyclictest49-21ksoftirqd/411:19:414
7621991210,1cyclictest49-21ksoftirqd/410:36:364
7621991210,1cyclictest49-21ksoftirqd/409:15:134
7621991210,1cyclictest49-21ksoftirqd/408:54:494
762199121,1cyclictest121rcu_preempt09:49:254
761099120,11cyclictest668-21systemd-logind10:03:513
760299124,1cyclictest33-21ksoftirqd/207:59:392
758699124,4cyclictest28873-21/usr/sbin/munin11:19:230
7586991210,1cyclictest711-21dbus-daemon10:28:380
758699120,11cyclictest32543-21bash12:22:080
40382120,0sleep50-21swapper/509:12:545
265192120,0sleep10-21swapper/110:16:331
172972120,0sleep60-21swapper/610:05:576
763699119,1cyclictest65-21ksoftirqd/612:07:226
763699113,4cyclictest7134-21bash09:32:056
762999119,1cyclictest711-21dbus-daemon09:47:285
762999110,10cyclictest711-21dbus-daemon09:00:305
762199119,1cyclictest49-21ksoftirqd/407:24:474
7621991110,1cyclictest49-21ksoftirqd/408:09:444
7621991110,1cyclictest49-21ksoftirqd/407:49:354
761099112,4cyclictest26554-21sshd12:17:163
761099111,4cyclictest30604-21sshd12:04:383
760299111,2cyclictest33-21ksoftirqd/207:19:412
759499119,1cyclictest25-21ksoftirqd/110:01:421
759499112,6cyclictest15863-21sshd11:44:561
7594991110,1cyclictest25-21ksoftirqd/111:17:591
759499110,11cyclictest5180-21bash10:23:151
758699119,1cyclictest9-21ksoftirqd/010:54:060
758699113,4cyclictest7735-21sshd10:09:240
7586991110,1cyclictest9-21ksoftirqd/011:29:420
758699110,1cyclictest859-21polkitd12:13:190
763699103,5cyclictest29783-21sshd11:19:286
763699100,7cyclictest20210-21id09:10:146
763699100,10cyclictest0-21swapper/610:37:206
763699100,10cyclictest0-21swapper/609:39:596
762999109,1cyclictest57-21ksoftirqd/510:03:225
762999108,1cyclictest57-21ksoftirqd/511:52:255
762999108,1cyclictest57-21ksoftirqd/511:35:155
762999108,1cyclictest57-21ksoftirqd/510:34:495
762999108,1cyclictest57-21ksoftirqd/509:39:355
762199109,1cyclictest49-21ksoftirqd/408:29:384
762199109,1cyclictest49-21ksoftirqd/407:44:384
761099109,1cyclictest0-21swapper/309:20:583
761099103,4cyclictest4940-21awk09:54:403
761099102,3cyclictest0-21swapper/310:31:273
761099101,4cyclictest0-21swapper/311:55:283
761099100,4cyclictest21256-21sshd10:06:353
761099100,4cyclictest1287-21sshd10:22:403
761099100,10cyclictest0-21swapper/309:47:183
761099100,10cyclictest0-21swapper/309:39:063
760299108,2cyclictest33-21ksoftirqd/207:14:202
760299108,1cyclictest33-21ksoftirqd/208:34:452
760299102,1cyclictest33-21ksoftirqd/208:24:382
760299101,1cyclictest121rcu_preempt08:14:432
759499109,1cyclictest25-21ksoftirqd/110:46:211
759499109,1cyclictest25-21ksoftirqd/109:01:431
759499107,1cyclictest25-21ksoftirqd/111:11:441
758699108,1cyclictest9-21ksoftirqd/010:35:210
758699108,1cyclictest9-21ksoftirqd/009:59:350
758699100,1cyclictest0-21swapper/010:18:320
76369999,0cyclictest0-21swapper/610:39:256
76369999,0cyclictest0-21swapper/606:54:466
76369998,1cyclictest0-21swapper/609:35:156
76369997,1cyclictest65-21ksoftirqd/612:12:156
76369997,1cyclictest65-21ksoftirqd/610:13:356
76369993,5cyclictest0-21swapper/609:54:376
76369993,1cyclictest0-21swapper/607:19:426
76369992,4cyclictest19249-21grep11:17:516
76369992,3cyclictest7149-21nfsd409:59:406
76369992,3cyclictest6464-21sshd12:18:556
76369991,4cyclictest18967-21grep12:03:086
76369990,9cyclictest0-21swapper/611:50:136
76369990,9cyclictest0-21swapper/611:33:116
76369990,9cyclictest0-21swapper/610:32:396
76369990,9cyclictest0-21swapper/609:49:556
76369990,9cyclictest0-21swapper/609:48:116
76369990,9cyclictest0-21swapper/609:23:356
76369990,0cyclictest0-21swapper/611:38:426
76299999,0cyclictest0-21swapper/509:24:375
76299998,1cyclictest57-21ksoftirqd/512:16:215
76299998,1cyclictest57-21ksoftirqd/511:24:405
76299997,1cyclictest57-21ksoftirqd/511:34:245
76299997,1cyclictest57-21ksoftirqd/510:49:525
76299997,1cyclictest57-21ksoftirqd/509:14:505
76299997,1cyclictest57-21ksoftirqd/508:49:245
76299996,2cyclictest57-21ksoftirqd/510:06:595
76299994,1cyclictest57-21ksoftirqd/511:54:345
76299993,3cyclictest11516-21sshd12:06:265
76299990,9cyclictest0-21swapper/510:56:415
76299990,9cyclictest0-21swapper/509:36:175
76299990,0cyclictest0-21swapper/510:18:045
76299990,0cyclictest0-21swapper/508:57:025
76219998,1cyclictest49-21ksoftirqd/408:39:394
76219998,1cyclictest49-21ksoftirqd/408:39:394
76219998,1cyclictest49-21ksoftirqd/408:34:424
76219998,1cyclictest49-21ksoftirqd/408:24:424
76219997,1cyclictest49-21ksoftirqd/408:04:354
76219997,1cyclictest49-21ksoftirqd/407:44:244
76219997,1cyclictest49-21ksoftirqd/407:44:244
76219997,1cyclictest49-21ksoftirqd/407:14:344
76109997,1cyclictest41-21ksoftirqd/311:30:323
76109997,1cyclictest41-21ksoftirqd/309:07:033
76109997,1cyclictest17704-21kworker/3:412:22:543
76109996,3cyclictest17157-21kworker/3:411:27:003
76109993,4cyclictest0-21swapper/310:40:213
76109992,6cyclictest10115-21sshd10:37:543
76109992,2cyclictest30229-21sshd11:47:003
76109990,9cyclictest0-21swapper/312:09:583
76109990,9cyclictest0-21swapper/311:49:433
76109990,9cyclictest0-21swapper/311:19:123
76109990,4cyclictest29614-21sshd09:25:563
76109990,4cyclictest0-21swapper/309:29:353
76109990,0cyclictest22810-21sshd08:55:543
76109990,0cyclictest0-21swapper/310:55:433
76029998,1cyclictest33-21ksoftirqd/208:19:372
76029998,1cyclictest33-21ksoftirqd/208:09:352
76029997,2cyclictest33-21ksoftirqd/208:29:392
76029997,1cyclictest33-21ksoftirqd/208:54:242
76029997,1cyclictest33-21ksoftirqd/208:09:242
76029991,1cyclictest33-21ksoftirqd/208:44:362
75949999,0cyclictest0-21swapper/110:39:201
75949999,0cyclictest0-21swapper/110:05:551
75949998,1cyclictest25-21ksoftirqd/110:54:351
75949997,1cyclictest25-21ksoftirqd/111:52:421
75949997,1cyclictest25-21ksoftirqd/109:58:271
75949997,1cyclictest25-21ksoftirqd/109:36:541
75949996,1cyclictest25-21ksoftirqd/109:50:221
75949994,3cyclictest1-21systemd09:45:381
75949994,2cyclictest711-21dbus-daemon10:13:471
75949993,3cyclictest15547-21sshd09:28:491
75949990,9cyclictest0-21swapper/109:19:341
75949990,9cyclictest0-21swapper/109:06:101
75949990,9cyclictest0-21swapper/107:39:361
75949990,9cyclictest0-21swapper/107:39:361
75949990,0cyclictest0-21swapper/112:06:031
75949990,0cyclictest0-21swapper/109:42:251
75869999,0cyclictest0-21swapper/011:45:430
75869999,0cyclictest0-21swapper/009:32:020
75869996,2cyclictest9-21ksoftirqd/011:21:200
75869994,4cyclictest24742-21bash11:37:320
75869994,1cyclictest11752-21sshd11:11:570
75869993,3cyclictest24269-21perf11:04:240
75869992,3cyclictest7476-21sshd09:54:580
75869991,4cyclictest711-21dbus-daemon12:17:590
75869990,5cyclictest11260-21grepconf.sh09:41:500
75869990,0cyclictest0-21swapper/010:57:100
75869990,0cyclictest0-21swapper/009:07:260
16117290,0sleep10-21swapper/112:15:501
76369988,0cyclictest0-21swapper/611:05:176
76369986,1cyclictest65-21ksoftirqd/612:22:526
76369986,1cyclictest65-21ksoftirqd/611:00:126
76369986,1cyclictest65-21ksoftirqd/610:16:416
76369983,3cyclictest18035-21bash08:55:046
76369983,2cyclictest18247-21sshd10:24:546
76369983,2cyclictest0-21swapper/610:58:366
76369982,5cyclictest1777-21expr08:34:416
76369982,3cyclictest32240-21sshd10:46:216
76369982,3cyclictest28541-21sshd09:06:406
76369982,3cyclictest18902-21bash10:49:286
76369982,3cyclictest16654-21sshd09:00:026
76369982,3cyclictest13778-21df10:19:356
76369981,3cyclictest24440-21grepconf.sh09:15:476
76369980,8cyclictest0-21swapper/611:44:326
76369980,8cyclictest0-21swapper/608:09:496
76369980,7cyclictest28386-21kworker/6:311:09:356
76299988,0cyclictest0-21swapper/512:10:285
76299988,0cyclictest0-21swapper/510:28:095
76299988,0cyclictest0-21swapper/509:19:535
76299987,1cyclictest57-21ksoftirqd/512:03:045
76299987,1cyclictest57-21ksoftirqd/511:44:215
76299986,1cyclictest57-21ksoftirqd/512:22:525
76299986,1cyclictest57-21ksoftirqd/510:46:365
76299986,1cyclictest57-21ksoftirqd/509:32:315
76299986,1cyclictest1938-21grepconf.sh09:54:255
76299984,3cyclictest16071-21sshd11:44:585
76299984,3cyclictest10594-21sshd11:11:455
76299983,2cyclictest11388-21grepconf.sh10:33:315
76299982,3cyclictest24035-21sshd10:11:345
76219987,1cyclictest49-21ksoftirqd/407:29:434
76219987,1cyclictest49-21ksoftirqd/407:04:244
76219987,1cyclictest49-21ksoftirqd/407:04:244
76219986,1cyclictest49-21ksoftirqd/407:59:394
76219986,1cyclictest49-21ksoftirqd/407:19:464
76219986,1cyclictest49-21ksoftirqd/407:04:354
76219986,1cyclictest49-21ksoftirqd/406:54:414
76109988,0cyclictest0-21swapper/309:14:303
76109986,1cyclictest41-21ksoftirqd/310:49:393
76109986,1cyclictest20308-21kworker/3:410:26:343
76109982,4cyclictest26382-21sshd09:44:093
76109981,6cyclictest4064-21grep12:01:013
76109980,7cyclictest32455-21grepconf.sh09:49:363
76109980,3cyclictest0-21swapper/310:59:453
76109980,0cyclictest0-21swapper/307:04:363
76029986,1cyclictest33-21ksoftirqd/207:24:382
76029986,1cyclictest33-21ksoftirqd/207:04:332
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional