You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-25 - 20:26
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Tue Nov 25, 2025 12:43:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
182052920,1sleep21633899cyclictest08:49:282
111032730,2sleep71637899cyclictest09:58:367
1606627248,8sleep70-21swapper/706:52:357
111602720,0sleep30-21swapper/310:31:243
25932530,0sleep10-21swapper/109:29:231
25932530,0sleep10-21swapper/109:29:231
16323995250,1cyclictest28104-21nfsd12:04:090
1601525027,9sleep10-21swapper/106:51:491
182902490,0sleep50-21swapper/510:57:055
1371824927,8sleep30-21swapper/306:49:183
29322480,0sleep72931-21sshd11:10:527
76622470,0sleep20-21swapper/212:21:512
320822470,0sleep70-21swapper/710:52:187
228582470,0sleep10-21swapper/112:08:451
1616324725,8sleep60-21swapper/606:53:466
318452460,0sleep10-21swapper/109:16:091
165262460,0sleep10-21swapper/110:53:561
119272450,1sleep5551rcuc/509:11:095
44322440,0sleep20-21swapper/209:00:172
213282440,0sleep40-21swapper/409:53:214
211502440,0sleep0111rcuc/009:56:290
74322430,2sleep01632399cyclictest11:26:230
48142430,0sleep60-21swapper/609:39:136
235522430,0sleep21-21systemd10:05:282
115582430,0sleep4481ktimersoftd/412:02:074
276642420,0sleep30-21swapper/311:04:003
96522410,0sleep60-21swapper/609:52:106
47742410,0sleep00-21swapper/011:08:050
1591124129,8sleep40-21swapper/406:50:244
157752410,1sleep01632399cyclictest11:48:420
197962400,1sleep719798-21id09:40:267
1618024027,9sleep50-21swapper/506:54:015
1590624026,10sleep00-21swapper/006:50:200
215872390,0sleep50-21swapper/511:49:115
1362523927,8sleep20-21swapper/206:49:172
130152390,0sleep60-21swapper/611:59:296
77142380,0sleep60-21swapper/610:43:526
200972360,0sleep70-21swapper/709:59:247
164762350,1sleep585450irq/26-em1-tx-010:10:365
302042340,0sleep10-21swapper/109:09:451
16378992715,5cyclictest0-21swapper/710:46:307
1637899250,24cyclictest1531-21kworker/7:112:04:097
1637899247,0cyclictest0-21swapper/708:19:367
1637899247,0cyclictest0-21swapper/707:23:367
1637899247,0cyclictest0-21swapper/707:23:367
16378992110,0cyclictest0-21swapper/706:57:347
1637899210,15cyclictest0-21swapper/707:03:187
264992200,0sleep40-21swapper/409:31:344
264992200,0sleep40-21swapper/409:31:344
1637899208,6cyclictest0-21swapper/711:53:267
1637899208,6cyclictest0-21swapper/708:16:467
1637899208,6cyclictest0-21swapper/708:11:587
1637899208,6cyclictest0-21swapper/707:43:547
1637899208,6cyclictest0-21swapper/707:26:387
1637899208,6cyclictest0-21swapper/707:16:087
1637899208,12cyclictest0-21swapper/711:21:407
1637899208,0cyclictest0-21swapper/707:49:207
1637899202,0cyclictest0-21swapper/711:31:527
1637899198,5cyclictest0-21swapper/712:19:367
1637899198,5cyclictest0-21swapper/712:17:347
1637899198,5cyclictest0-21swapper/711:39:487
1637899198,5cyclictest0-21swapper/711:35:447
1637899198,5cyclictest0-21swapper/710:58:147
1637899198,5cyclictest0-21swapper/710:14:387
1637899198,5cyclictest0-21swapper/709:20:547
1637899198,5cyclictest0-21swapper/709:05:027
1637899198,5cyclictest0-21swapper/708:34:167
1637899198,5cyclictest0-21swapper/708:29:247
1637899198,5cyclictest0-21swapper/708:24:167
1637899198,5cyclictest0-21swapper/707:46:187
1637899198,5cyclictest0-21swapper/707:35:027
1637899198,5cyclictest0-21swapper/707:04:207
1637899198,0cyclictest0-21swapper/712:09:367
1637899198,0cyclictest0-21swapper/711:45:487
1637899198,0cyclictest0-21swapper/711:24:547
1637899198,0cyclictest0-21swapper/711:05:087
1637899198,0cyclictest0-21swapper/709:27:067
1637899198,0cyclictest0-21swapper/709:14:527
1637899198,0cyclictest0-21swapper/709:12:267
1637899198,0cyclictest0-21swapper/708:55:587
1637899198,0cyclictest0-21swapper/708:48:207
1637899198,0cyclictest0-21swapper/708:05:347
1637899198,0cyclictest0-21swapper/707:55:507
1637899197,6cyclictest0-21swapper/708:39:187
1637899197,0cyclictest0-21swapper/710:24:007
1637899197,0cyclictest0-21swapper/710:13:547
1637899197,0cyclictest0-21swapper/707:09:367
1637899191,6cyclictest0-21swapper/708:02:427
1637899191,12cyclictest0-21swapper/709:37:007
1637899191,0cyclictest0-21swapper/710:40:367
225532180,0sleep20-21swapper/209:15:142
1637899187,6cyclictest0-21swapper/707:29:427
1637899187,5cyclictest0-21swapper/710:38:087
1637899187,5cyclictest0-21swapper/709:50:007
1637899187,5cyclictest0-21swapper/708:50:327
1637899187,0cyclictest0-21swapper/711:17:407
1637899187,0cyclictest0-21swapper/710:32:567
1637899187,0cyclictest0-21swapper/710:25:127
1637899187,0cyclictest0-21swapper/709:03:287
1637199180,17cyclictest8421-21sshd11:05:126
1637899177,4cyclictest0-21swapper/711:00:427
1637899176,5cyclictest0-21swapper/712:08:387
1637899176,0cyclictest0-21swapper/711:59:067
1637899176,0cyclictest0-21swapper/710:08:107
1636399176,11cyclictest0-21swapper/508:13:505
294362160,0sleep1241ktimersoftd/110:11:521
1637899165,0cyclictest0-21swapper/709:31:467
1637899165,0cyclictest0-21swapper/709:31:467
1637199160,15cyclictest20187-21grepconf.sh12:11:386
1637199160,13cyclictest15724-21grep10:44:306
1636399165,5cyclictest0-21swapper/510:31:205
1636399160,15cyclictest20341-21bash10:38:355
1636399160,0cyclictest0-21swapper/509:49:385
1632399161,15cyclictest8360-21bash11:35:490
236752150,0sleep423674-21systemd-cgroups10:26:284
223542150,1sleep233-21ksoftirqd/209:21:232
1637899154,5cyclictest0-21swapper/709:45:167
1637199152,3cyclictest22600-21systemd-cgroups10:54:276
1636399152,5cyclictest668-21systemd-logind09:58:245
1636399150,9cyclictest8976-21sshd12:10:395
125192150,0sleep30-21swapper/311:56:373
9522140,0sleep40-21swapper/410:55:204
32032140,0sleep10-21swapper/109:51:331
226332140,0sleep40-21swapper/411:09:374
198272140,0sleep20-21swapper/209:27:482
185292140,0sleep30-21swapper/309:24:283
1636399144,4cyclictest12873-21bash09:17:305
1636399140,1cyclictest0-21swapper/507:44:285
1636399140,0cyclictest0-21swapper/509:25:265
1632399149,4cyclictest9-21ksoftirqd/010:57:450
161772140,0sleep316164-21sshd08:58:023
313892130,1sleep1758-21sshd10:36:261
173842130,0sleep60-21swapper/611:21:166
1637199132,5cyclictest20028-21bash12:23:006
1636399139,0cyclictest0-21swapper/511:13:255
1636399138,5cyclictest0-21swapper/508:42:585
1636399138,1cyclictest861-21gdbus09:59:305
1636399132,5cyclictest0-21swapper/511:15:045
1636399132,11cyclictest0-21swapper/509:01:065
1636399130,8cyclictest0-21swapper/508:57:105
1636399130,7cyclictest13385-21sshd10:07:325
1636399130,7cyclictest0-21swapper/507:10:405
1636399130,1cyclictest0-21swapper/508:36:005
1636399130,1cyclictest0-21swapper/508:27:405
1636399130,1cyclictest0-21swapper/507:38:205
1636399130,1cyclictest0-21swapper/507:19:525
1636399130,1cyclictest0-21swapper/507:19:525
1635399130,12cyclictest3947-21bash09:48:374
16345991311,1cyclictest41-21ksoftirqd/311:20:253
1632399130,12cyclictest25858-21bash12:14:540
133412130,0sleep10-21swapper/111:39:211
318772120,1sleep2311rcuc/212:03:572
193692120,0sleep20-21swapper/211:15:202
16371991210,1cyclictest65-21ksoftirqd/610:33:486
1637199121,5cyclictest18834-21systemd-cgroups09:24:296
1637199120,10cyclictest1215-21bash08:59:566
1636399129,2cyclictest13729-21grepconf.sh11:26:595
1636399121,5cyclictest0-21swapper/511:59:265
1636399121,5cyclictest0-21swapper/511:37:305
1636399121,5cyclictest0-21swapper/511:20:565
1636399121,5cyclictest0-21swapper/511:05:265
1636399121,5cyclictest0-21swapper/510:27:045
1636399120,9cyclictest0-21swapper/508:50:245
1636399120,6cyclictest0-21swapper/511:29:225
1636399120,12cyclictest0-21swapper/508:46:125
1636399120,12cyclictest0-21swapper/508:18:105
1636399120,12cyclictest0-21swapper/507:30:165
1636399120,12cyclictest0-21swapper/507:07:205
1636399120,11cyclictest0-21swapper/511:41:485
16353991210,1cyclictest49-21ksoftirqd/409:04:214
16345991211,1cyclictest41-21ksoftirqd/310:49:323
1634599120,11cyclictest22138-21sshd11:24:343
1633899129,2cyclictest33-21ksoftirqd/207:49:152
16338991211,1cyclictest33-21ksoftirqd/211:39:002
1632399120,1cyclictest0-21swapper/010:45:470
146072120,0sleep10-21swapper/111:27:041
103552120,0sleep30-21swapper/311:39:073
250022110,0sleep40-21swapper/410:29:334
240062110,0sleep00-21swapper/010:08:300
1637199110,10cyclictest4799-21sshd10:37:016
1636399117,0cyclictest0-21swapper/512:23:185
1636399116,5cyclictest0-21swapper/512:06:585
1636399116,5cyclictest0-21swapper/508:09:065
1636399115,1cyclictest0-21swapper/512:16:385
1636399112,6cyclictest711-21dbus-daemon11:53:045
1636399111,4cyclictest10425-21systemd-cgroups09:49:155
1636399110,5cyclictest0-21swapper/510:41:405
1636399110,5cyclictest0-21swapper/509:39:225
1636399110,5cyclictest0-21swapper/509:36:025
1636399110,11cyclictest0-21swapper/507:51:545
1636399110,0cyclictest0-21swapper/510:59:385
1636399110,0cyclictest0-21swapper/509:07:345
1636399110,0cyclictest0-21swapper/507:02:245
1635399119,1cyclictest49-21ksoftirqd/410:11:104
1635399115,3cyclictest49-21ksoftirqd/411:22:184
1635399113,4cyclictest18748-21sshd11:39:414
16353991110,1cyclictest0-21swapper/412:14:364
16353991110,1cyclictest0-21swapper/411:44:294
1635399110,1cyclictest29233-21sshd09:41:224
1634599119,1cyclictest41-21ksoftirqd/309:16:173
1634599115,3cyclictest41-21ksoftirqd/309:41:283
16345991110,1cyclictest41-21ksoftirqd/311:43:053
16345991110,1cyclictest41-21ksoftirqd/310:07:173
16345991110,1cyclictest41-21ksoftirqd/309:58:003
1633899119,1cyclictest33-21ksoftirqd/211:06:262
1633899118,3cyclictest33-21ksoftirqd/211:43:062
1633899116,1cyclictest29372-21grep11:19:302
16338991110,1cyclictest0-21swapper/209:57:062
1633899110,10cyclictest29513-21sshd10:54:572
1633099119,1cyclictest706-21sshd10:18:181
1633099112,5cyclictest1073-21bash11:32:031
1633099110,1cyclictest20084-21sshd09:40:281
1632399119,1cyclictest5517-21sshd09:42:170
1632399119,1cyclictest0-21swapper/011:49:490
1632399110,10cyclictest12675-21grep09:36:300
1632399110,1cyclictest0-21swapper/009:48:160
137532110,0sleep60-21swapper/610:16:216
1637199109,1cyclictest19298-21sshd11:15:196
1637199109,1cyclictest0-21swapper/612:07:596
1637199108,2cyclictest0-21swapper/611:44:026
1637199108,1cyclictest85350irq/25-em1-rx-012:16:306
1637199103,6cyclictest668-21systemd-logind11:11:056
1637199103,4cyclictest1357-21sshd11:35:046
1637199102,4cyclictest30318-21grep11:55:196
1637199100,4cyclictest6540-21sshd10:12:456
1637199100,10cyclictest0-21swapper/611:47:106
1636399107,2cyclictest16084-21grepconf.sh10:47:485
1636399106,0cyclictest0-21swapper/509:32:065
1636399106,0cyclictest0-21swapper/509:32:065
1636399105,5cyclictest0-21swapper/510:20:045
1636399105,5cyclictest0-21swapper/510:15:485
1636399100,7cyclictest0-21swapper/507:54:325
1635399108,1cyclictest49-21ksoftirqd/411:36:084
1635399108,1cyclictest49-21ksoftirqd/411:06:354
1635399108,1cyclictest49-21ksoftirqd/409:10:524
1635399106,1cyclictest18451-21grepconf.sh09:27:414
1635399105,3cyclictest9758-21sshd12:04:394
1635399105,3cyclictest680-21nscd10:42:094
1635399103,4cyclictest668-21systemd-logind11:24:434
1635399102,4cyclictest9357-21bash12:13:324
1635399102,2cyclictest29719-21sshd08:59:354
1635399100,10cyclictest0-21swapper/411:56:474
1635399100,1cyclictest0-21swapper/408:57:414
1634599109,1cyclictest41-21ksoftirqd/312:04:463
1634599109,1cyclictest41-21ksoftirqd/311:59:563
1634599108,1cyclictest41-21ksoftirqd/312:10:153
1634599108,1cyclictest41-21ksoftirqd/311:12:343
1634599108,1cyclictest41-21ksoftirqd/309:37:123
1634599108,1cyclictest0-21swapper/311:33:523
1634599107,1cyclictest20287-21kworker/3:110:12:503
1634599100,10cyclictest5172-21kworker/3:511:05:593
1634599100,10cyclictest0-21swapper/311:51:413
1634599100,10cyclictest0-21swapper/309:05:593
1633899109,1cyclictest33-21ksoftirqd/210:02:492
1633899109,1cyclictest33-21ksoftirqd/209:46:152
1633899108,1cyclictest33-21ksoftirqd/210:12:422
1633899104,3cyclictest912-21rm10:30:222
1633899104,3cyclictest2232-21grep11:53:012
1633899104,3cyclictest17124-21bash11:57:012
1633899103,1cyclictest0-21swapper/211:25:182
1633899101,5cyclictest0-21swapper/212:17:322
1633899100,9cyclictest23198-21sshd10:26:252
1633899100,1cyclictest0-21swapper/209:50:192
1633899100,0cyclictest0-21swapper/211:00:292
1633099109,1cyclictest25-21ksoftirqd/109:56:331
1633099108,1cyclictest25-21ksoftirqd/108:54:301
1633099108,1cyclictest0-21swapper/110:48:351
1633099106,1cyclictest14728-21sshd11:56:491
1633099104,3cyclictest15978-21sshd12:11:171
1633099104,2cyclictest2792-21grepconf.sh11:16:511
1633099103,4cyclictest27793-21sshd09:59:571
1633099103,4cyclictest0-21swapper/111:22:311
1633099100,1cyclictest0-21swapper/111:13:391
1632399108,1cyclictest9-21ksoftirqd/009:33:390
1632399108,1cyclictest9-21ksoftirqd/009:33:390
1632399103,6cyclictest31478-21sshd12:09:300
1632399103,4cyclictest24539-21cp10:17:300
1632399103,4cyclictest0-21swapper/008:29:160
1632399102,4cyclictest16596-21bash11:15:030
1632399102,3cyclictest18505-21sshd09:05:250
1632399101,4cyclictest16057-21sshd10:41:170
1632399100,1cyclictest0-21swapper/008:56:240
1632399100,0cyclictest0-21swapper/011:03:100
109672100,0sleep40-21swapper/412:22:094
27120290,1sleep11-21systemd09:06:161
163719997,1cyclictest65-21ksoftirqd/610:05:086
163719992,6cyclictest19704-21sshd09:59:236
163719992,4cyclictest0-21swapper/609:19:236
163719992,3cyclictest1-21systemd10:25:036
163719991,6cyclictest32363-21sshd10:52:206
163719990,4cyclictest17596-21sshd09:11:456
163719990,0cyclictest0-21swapper/609:54:276
163539998,1cyclictest49-21ksoftirqd/409:54:574
163539997,1cyclictest49-21ksoftirqd/411:49:354
163539994,4cyclictest26636-21bash10:14:304
163539993,3cyclictest9179-21bash11:32:494
163539993,3cyclictest4355-21bash11:01:394
163539993,3cyclictest20360-21bash08:54:154
163539993,3cyclictest13937-21bash10:04:364
163539992,4cyclictest4612-21sshd09:35:434
163539992,4cyclictest23319-21bash09:15:184
163539992,3cyclictest2171-21bash09:22:354
163539990,9cyclictest0-21swapper/410:19:594
163459998,1cyclictest41-21ksoftirqd/312:14:153
163459998,1cyclictest41-21ksoftirqd/311:15:313
163459998,1cyclictest41-21ksoftirqd/310:20:123
163459998,1cyclictest41-21ksoftirqd/309:31:413
163459998,1cyclictest41-21ksoftirqd/309:31:413
163459998,1cyclictest0-21swapper/310:44:123
163459997,1cyclictest41-21ksoftirqd/310:36:243
163459997,1cyclictest41-21ksoftirqd/309:59:243
163459997,1cyclictest41-21ksoftirqd/309:45:063
163459992,3cyclictest668-21systemd-logind10:27:443
163459990,9cyclictest0-21swapper/310:18:113
163389999,0cyclictest0-21swapper/208:55:272
163389998,1cyclictest0-21swapper/209:09:292
163389994,4cyclictest18556-21sshd10:19:542
163389993,3cyclictest9929-21systemd-cgroups09:29:562
163389993,3cyclictest9929-21systemd-cgroups09:29:562
163389993,3cyclictest3958-21grep11:44:302
163389992,7cyclictest25235-21kworker/2:411:10:022
163389992,3cyclictest1-21systemd11:29:272
163389991,5cyclictest31721-21bash09:41:372
163389990,9cyclictest0-21swapper/210:49:262
163389990,5cyclictest30189-21grepconf.sh10:42:442
163389990,5cyclictest25742-21grep10:45:212
163389990,0cyclictest0-21swapper/212:09:132
163389990,0cyclictest0-21swapper/209:08:142
163309999,0cyclictest0-21swapper/110:40:041
163309996,1cyclictest15394-21kworker/1:512:19:241
163309995,4cyclictest26256-21kworker/1:409:23:261
163309995,3cyclictest17275-21sshd11:48:491
163309993,5cyclictest711-21dbus-daemon10:30:211
163309993,5cyclictest30707-21grepconf.sh09:35:061
163309993,1cyclictest12092-21sshd09:01:031
163309991,5cyclictest758-21sshd09:45:381
163309990,9cyclictest0-21swapper/110:59:371
163309990,8cyclictest19723-21kworker/1:212:15:191
163309990,8cyclictest18460-21kworker/1:410:23:291
163309990,6cyclictest21717-21cp10:54:241
163239999,0cyclictest0-21swapper/012:20:250
163239997,1cyclictest9-21ksoftirqd/010:21:050
163239997,1cyclictest9-21ksoftirqd/010:00:130
163239997,1cyclictest18873-21cstates07:54:270
163239994,4cyclictest27489-21grepconf.sh10:29:430
163239993,2cyclictest668-21systemd-logind11:58:110
163239991,3cyclictest13241-21sshd09:11:170
163239990,9cyclictest0-21swapper/012:05:070
163239990,9cyclictest0-21swapper/009:51:160
163239990,9cyclictest0-21swapper/009:03:390
163239990,8cyclictest28464-21kworker/0:109:29:100
163239990,4cyclictest13983-21sshd10:13:250
163239990,4cyclictest10679-21sshd09:17:160
163239990,2cyclictest25920-21sshd11:13:060
163239990,0cyclictest0-21swapper/011:21:270
163719988,0cyclictest0-21swapper/611:52:416
163719988,0cyclictest0-21swapper/610:20:466
163719988,0cyclictest0-21swapper/609:15:406
163719987,1cyclictest641ktimersoftd/611:00:296
163719987,1cyclictest0-21swapper/609:30:576
163719987,1cyclictest0-21swapper/609:30:576
163719985,2cyclictest3833-21bash09:45:226
163719985,2cyclictest24404-21sshd08:58:566
163719984,3cyclictest4036-21grepconf.sh09:07:116
163719984,3cyclictest18212-21sshd09:40:176
163719984,2cyclictest18320-21sshd11:30:346
163639985,2cyclictest6258-21grep10:49:455
163639983,5cyclictest0-21swapper/509:19:375
163639983,5cyclictest0-21swapper/508:30:405
163639980,8cyclictest0-21swapper/511:57:325
163639980,8cyclictest0-21swapper/506:55:525
163539986,1cyclictest49-21ksoftirqd/409:59:394
163539985,2cyclictest24955-21grepconf.sh10:45:174
163539983,2cyclictest5664-21grepconf.sh10:49:424
163539982,3cyclictest11289-21fschecks_time10:34:284
163539980,7cyclictest3549-21kworker/4:011:15:044
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional