You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-24 - 00:42
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Sun Nov 23, 2025 12:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
82025228,9sleep00-21swapper/006:51:290
100125026,8sleep70-21swapper/706:53:367
234172490,0sleep30-21swapper/309:38:593
262132470,1sleep557-21ksoftirqd/510:32:595
262132470,1sleep557-21ksoftirqd/510:32:595
298082460,0sleep30-21swapper/308:39:333
77222420,1sleep47721-21sshd10:34:584
234882420,0sleep30-21swapper/310:37:373
81124130,8sleep20-21swapper/206:51:232
77924128,9sleep10-21swapper/106:50:581
72924128,9sleep30-21swapper/306:50:203
12142410,0sleep40-21swapper/409:19:384
93124027,9sleep40-21swapper/406:52:454
13712400,1sleep1241ktimersoftd/110:05:001
72323926,9sleep60-21swapper/606:50:156
3096823926,9sleep50-21swapper/506:49:255
282302390,0sleep665-21ksoftirqd/611:37:316
155502370,0sleep20-21swapper/209:16:562
124502340,1sleep557-21ksoftirqd/511:39:465
124502340,1sleep557-21ksoftirqd/511:39:465
125999258,11cyclictest0-21swapper/709:18:547
124499230,7cyclictest0-21swapper/510:37:095
125999225,5cyclictest0-21swapper/707:04:277
124499220,8cyclictest711-21dbus-daemon09:39:355
124499220,6cyclictest0-21swapper/509:25:305
125999210,8cyclictest0-21swapper/712:06:067
124499211,9cyclictest28821-21sshd11:18:015
124499210,8cyclictest668-21systemd-logind11:06:365
125999202,6cyclictest0-21swapper/708:52:507
124499200,9cyclictest12817-21sshd12:12:585
124499200,7cyclictest3876-21grepconf.sh10:59:325
124499200,7cyclictest0-21swapper/512:07:245
124499200,10cyclictest711-21dbus-daemon11:28:335
124499200,10cyclictest17529-21grepconf.sh09:06:355
125999198,5cyclictest0-21swapper/710:59:567
125999198,5cyclictest0-21swapper/710:03:127
125999198,5cyclictest0-21swapper/709:50:007
125999198,0cyclictest0-21swapper/711:40:147
125999198,0cyclictest0-21swapper/711:40:147
125999198,0cyclictest0-21swapper/708:19:547
125999198,0cyclictest0-21swapper/708:16:327
125999198,0cyclictest0-21swapper/707:57:067
125999196,12cyclictest17427-21sshd12:18:087
124499191,7cyclictest12340-21systemd-cgroups10:06:455
124499190,7cyclictest26588-21bash12:14:425
124499190,6cyclictest0-21swapper/510:40:475
124499190,11cyclictest0-21swapper/511:56:205
124499190,10cyclictest1-21systemd11:49:435
1208991918,1cyclictest23159-21kworker/1:310:25:041
293972180,0sleep30-21swapper/311:08:213
150162180,0sleep40-21swapper/410:07:114
125999188,5cyclictest0-21swapper/710:16:427
125999188,0cyclictest0-21swapper/709:04:087
125999187,5cyclictest0-21swapper/711:35:187
125999187,5cyclictest0-21swapper/709:38:027
125999187,5cyclictest0-21swapper/709:28:127
125999187,5cyclictest0-21swapper/708:09:347
125999187,5cyclictest0-21swapper/706:57:077
125999187,0cyclictest0-21swapper/710:05:027
125999181,5cyclictest0-21swapper/711:55:087
125999181,0cyclictest0-21swapper/710:44:147
125999180,6cyclictest0-21swapper/709:23:507
125999180,0cyclictest0-21swapper/711:49:267
124499180,9cyclictest23752-21sshd11:36:485
124499180,9cyclictest12880-21grep10:01:475
124499180,8cyclictest8391-21grep10:55:035
124499180,8cyclictest25219-21grepconf.sh12:19:215
124499180,8cyclictest1270-21cat09:04:195
124499180,7cyclictest25492-21sshd09:39:195
124499180,17cyclictest0-21swapper/511:44:345
124499180,10cyclictest32360-21grepconf.sh12:02:025
124499180,10cyclictest18784-21sshd09:22:395
124499180,10cyclictest1-21systemd10:16:095
124499180,10cyclictest0-21swapper/508:56:225
1208991817,1cyclictest25932-21kworker/1:011:15:101
125999176,5cyclictest0-21swapper/712:01:307
125999176,5cyclictest0-21swapper/709:58:047
125999176,5cyclictest0-21swapper/709:07:447
125999176,5cyclictest0-21swapper/708:29:367
125999176,5cyclictest0-21swapper/708:28:107
125999176,5cyclictest0-21swapper/707:53:597
125999176,5cyclictest0-21swapper/707:11:497
125999176,0cyclictest0-21swapper/711:45:267
125999176,0cyclictest0-21swapper/709:32:127
125999176,0cyclictest0-21swapper/707:19:077
125999170,5cyclictest0-21swapper/710:36:287
124499171,10cyclictest740-21gdbus09:55:005
124499170,9cyclictest0-21swapper/509:18:245
124499170,8cyclictest9165-21grepconf.sh11:09:545
124499170,8cyclictest22919-21grep10:52:335
124499170,7cyclictest25117-21sshd10:23:105
124499170,7cyclictest12251-21sshd09:51:585
124499170,6cyclictest0-21swapper/510:26:345
124499170,16cyclictest3871-21grepconf.sh10:10:065
124499170,16cyclictest15007-21bash09:47:345
124499170,11cyclictest8664-21sshd11:19:405
124499170,10cyclictest30131-21bash11:32:475
123699171,15cyclictest4980-21bash11:24:114
125999166,5cyclictest0-21swapper/711:33:227
125999166,0cyclictest0-21swapper/709:43:187
125999165,5cyclictest0-21swapper/707:31:317
125999165,0cyclictest0-21swapper/712:09:207
125999165,0cyclictest0-21swapper/710:47:087
125999165,0cyclictest0-21swapper/708:00:587
125999165,0cyclictest0-21swapper/707:25:597
125999165,0cyclictest0-21swapper/707:02:597
125999161,14cyclictest711-21dbus-daemon10:58:307
124499161,5cyclictest0-21swapper/509:11:275
124499160,9cyclictest15697-21sshd09:32:495
1208991614,1cyclictest25932-21kworker/1:011:24:201
125999155,0cyclictest0-21swapper/709:09:467
125999154,5cyclictest0-21swapper/710:19:407
125999154,5cyclictest0-21swapper/707:36:347
125999154,0cyclictest0-21swapper/711:16:027
125999154,0cyclictest0-21swapper/707:21:157
125999154,0cyclictest0-21swapper/707:21:157
1259991510,4cyclictest9383-21bash10:25:227
125999150,9cyclictest0-21swapper/708:44:207
125999150,3cyclictest0-21swapper/710:31:047
125999150,3cyclictest0-21swapper/710:31:047
124499150,7cyclictest22638-21grep10:47:205
1228991512,2cyclictest41-21ksoftirqd/310:46:283
1208991515,0cyclictest374-21kworker/1:009:43:481
1208991514,1cyclictest374-21kworker/1:009:48:301
1208991514,1cyclictest23159-21kworker/1:310:18:281
1208991513,1cyclictest30261-21kworker/1:109:15:441
1208991513,1cyclictest17681-21kworker/1:009:25:581
1208991510,3cyclictest25-21ksoftirqd/112:19:341
120299150,14cyclictest28183-21sshd10:18:450
125999148,5cyclictest4294-21sshd11:24:047
125999148,5cyclictest0-21swapper/710:54:167
125999143,5cyclictest0-21swapper/709:45:547
125999143,5cyclictest0-21swapper/708:06:447
125999142,0cyclictest0-21swapper/708:37:147
125999140,8cyclictest0-21swapper/707:40:477
124499140,6cyclictest0-21swapper/508:19:375
1228991411,1cyclictest401ktimersoftd/309:13:083
1208991413,1cyclictest9535-21kworker/1:308:57:001
1208991413,1cyclictest5340-21kworker/1:411:55:021
1208991413,1cyclictest4921-21kworker/1:209:00:281
1208991413,1cyclictest2917-21kworker/1:108:03:061
1208991413,1cyclictest26305-21kworker/1:010:09:381
1208991413,1cyclictest11506-21kworker/1:411:00:041
1208991412,1cyclictest25-21ksoftirqd/112:14:331
1208991412,1cyclictest25-21ksoftirqd/111:19:351
1208991411,3cyclictest15-21kswork10:45:331
120899140,14cyclictest23422-21kworker/1:410:20:401
79542130,1sleep09-21ksoftirqd/009:20:460
125999138,5cyclictest0-21swapper/712:23:247
125999138,5cyclictest0-21swapper/711:25:387
125999138,5cyclictest0-21swapper/711:13:267
125999138,5cyclictest0-21swapper/708:42:547
125999138,0cyclictest0-21swapper/707:46:397
125999133,5cyclictest0-21swapper/711:05:087
125999132,5cyclictest0-21swapper/710:09:487
124499130,5cyclictest0-21swapper/506:59:385
121999130,12cyclictest0-21swapper/209:49:562
120899139,1cyclictest25-21ksoftirqd/112:04:331
120899138,3cyclictest25-21ksoftirqd/112:12:351
120899138,3cyclictest25-21ksoftirqd/109:11:311
120899134,2cyclictest25-21ksoftirqd/111:49:461
1208991312,1cyclictest4921-21kworker/1:209:07:241
1208991312,1cyclictest3744-21kworker/1:210:00:581
1208991312,1cyclictest29758-21kworker/1:309:36:521
1208991312,1cyclictest2917-21kworker/1:108:44:321
1208991312,1cyclictest28909-21kworker/1:111:12:121
1208991312,1cyclictest25-21ksoftirqd/110:34:381
1208991312,1cyclictest17925-21kworker/1:509:33:241
1208991311,1cyclictest25-21ksoftirqd/110:39:211
1208991311,1cyclictest25-21ksoftirqd/109:57:331
125999127,5cyclictest0-21swapper/708:56:107
124499122,10cyclictest0-21swapper/508:08:225
123699120,10cyclictest99-21kswapd007:54:324
122899120,11cyclictest0-21swapper/310:33:573
122899120,11cyclictest0-21swapper/310:33:573
121999120,11cyclictest13524-21grep11:25:102
1208991211,1cyclictest25932-21kworker/1:011:34:241
1208991211,1cyclictest23159-21kworker/1:310:34:041
1208991211,1cyclictest23159-21kworker/1:310:34:041
1208991211,1cyclictest17681-21kworker/1:009:24:141
1208991211,1cyclictest11506-21kworker/1:410:54:221
1208991210,2cyclictest25-21ksoftirqd/111:04:471
1208991210,1cyclictest25-21ksoftirqd/111:59:371
1208991210,1cyclictest25-21ksoftirqd/111:42:291
1208991210,1cyclictest25-21ksoftirqd/111:42:291
1208991210,1cyclictest25-21ksoftirqd/111:29:401
1208991210,1cyclictest25-21ksoftirqd/110:49:321
1208991210,1cyclictest25-21ksoftirqd/109:49:401
1208991210,1cyclictest25-21ksoftirqd/108:34:391
111252120,0sleep20-21swapper/208:55:012
88842110,0sleep20-21swapper/209:20:562
125199116,0cyclictest0-21swapper/611:25:426
125199110,10cyclictest8777-21grepconf.sh09:51:276
124499117,4cyclictest0-21swapper/507:24:435
124499110,7cyclictest3541-21cpu07:54:325
124499110,7cyclictest31367-21nfsd407:44:365
124499110,6cyclictest0-21swapper/508:16:385
124499110,10cyclictest25813-21cut07:34:375
122899110,1cyclictest0-21swapper/310:06:343
121999119,1cyclictest33-21ksoftirqd/211:04:352
121999116,5cyclictest13450irq/24-ahci[00010:33:042
121999116,5cyclictest13450irq/24-ahci[00010:33:042
1219991110,1cyclictest33-21ksoftirqd/210:42:202
121999110,1cyclictest0-21swapper/210:00:302
120899119,1cyclictest25-21ksoftirqd/111:44:411
1208991111,0cyclictest6700-21kworker/1:007:21:411
1208991111,0cyclictest6700-21kworker/1:007:21:411
1208991110,1cyclictest2917-21kworker/1:108:16:481
120299110,1cyclictest28166-21sshd12:10:260
153842100,1sleep215383-21grepconf.sh09:57:242
125199109,1cyclictest65-21ksoftirqd/610:09:336
125199102,7cyclictest10818-21sshd11:10:096
125199101,4cyclictest11527-21sshd10:45:316
125199101,3cyclictest0-21swapper/611:30:336
125199100,4cyclictest24587-21sshd12:09:516
124499103,6cyclictest763-21runrttasks08:52:475
124499103,6cyclictest757-21snmpd08:29:275
124499103,6cyclictest757-21snmpd07:49:295
124499103,6cyclictest757-21snmpd07:10:345
124499103,6cyclictest757-21snmpd07:08:285
124499103,6cyclictest757-21snmpd06:58:165
124499103,6cyclictest21611-21cat08:24:345
124499101,5cyclictest0-21swapper/508:44:065
124499100,9cyclictest0-21swapper/507:19:365
124499100,9cyclictest0-21swapper/507:19:365
124499100,8cyclictest0-21swapper/508:34:305
124499100,1cyclictest0-21swapper/507:39:425
124499100,1cyclictest0-21swapper/507:16:145
123699109,1cyclictest0-21swapper/411:29:074
122899109,1cyclictest28290-21sshd10:58:233
122899108,1cyclictest711-21dbus-daemon09:23:573
122899108,1cyclictest41-21ksoftirqd/310:11:023
122899102,4cyclictest0-21swapper/309:57:293
122899100,10cyclictest0-21swapper/311:23:173
122899100,10cyclictest0-21swapper/309:42:143
122899100,1cyclictest0-21swapper/312:23:003
121999109,1cyclictest33-21ksoftirqd/212:09:372
121999109,1cyclictest33-21ksoftirqd/210:07:112
121999109,1cyclictest0-21swapper/211:00:152
121999109,1cyclictest0-21swapper/209:38:072
121999108,1cyclictest99-21kswapd007:20:462
121999108,1cyclictest99-21kswapd007:20:462
120899109,1cyclictest839-21kworker/1:207:02:171
120899109,1cyclictest6700-21kworker/1:007:30:011
120899109,1cyclictest6700-21kworker/1:007:18:131
120899109,1cyclictest2917-21kworker/1:108:06:441
120899109,1cyclictest25-21ksoftirqd/107:44:351
120899109,1cyclictest25-21ksoftirqd/107:39:381
120899109,1cyclictest2019-21kworker/1:008:53:121
120299109,1cyclictest0-21swapper/011:31:480
120299109,1cyclictest0-21swapper/010:08:430
120299108,1cyclictest9-21ksoftirqd/009:44:370
120299103,4cyclictest0-21swapper/012:04:190
120299101,9cyclictest0-21swapper/008:19:340
120299100,10cyclictest0-21swapper/011:09:530
24288290,0sleep00-21swapper/010:47:350
12519999,0cyclictest0-21swapper/610:37:396
12519999,0cyclictest0-21swapper/610:15:546
12519999,0cyclictest0-21swapper/609:22:366
12519998,1cyclictest65-21ksoftirqd/610:50:336
12519998,1cyclictest0-21swapper/612:16:086
12519996,2cyclictest65-21ksoftirqd/607:39:206
12519994,0cyclictest85350irq/25-em1-rx-011:54:586
12519993,6cyclictest14742-21sshd09:06:086
12519992,4cyclictest1-21systemd09:25:006
12519992,2cyclictest17598-21bash09:42:536
12519991,7cyclictest6649-21sshd10:29:486
12519991,7cyclictest6649-21sshd10:29:486
12519991,3cyclictest1413-21sshd11:18:496
12519990,9cyclictest0-21swapper/611:21:006
12519990,9cyclictest0-21swapper/609:45:186
12519990,4cyclictest896-21sshd11:48:166
12519990,4cyclictest0-21swapper/609:34:466
12519990,0cyclictest0-21swapper/611:08:336
12449990,8cyclictest7075-21gltestperf07:59:345
12449990,8cyclictest22292-21gltestperf07:29:345
12449990,8cyclictest13822-21switchtime08:09:415
12449990,7cyclictest32739-21tune2fs08:44:335
12369998,1cyclictest49-21ksoftirqd/412:19:174
12369997,1cyclictest49-21ksoftirqd/411:30:074
12369997,1cyclictest49-21ksoftirqd/409:34:464
12369997,1cyclictest49-21ksoftirqd/409:04:234
12369993,2cyclictest12638-21grepconf.sh10:16:154
12369992,3cyclictest859-21polkitd10:25:284
12369990,9cyclictest0-21swapper/411:06:354
12369990,9cyclictest0-21swapper/411:01:574
12369990,9cyclictest0-21swapper/410:56:104
12289999,0cyclictest0-21swapper/310:15:303
12289999,0cyclictest0-21swapper/309:46:113
12289998,1cyclictest41-21ksoftirqd/311:25:173
12289995,3cyclictest711-21dbus-daemon11:56:433
12289994,3cyclictest20322-21sshd10:42:013
12289993,5cyclictest26048-21bash11:02:563
12289993,3cyclictest10267-21bash10:25:313
12289992,4cyclictest26740-21sshd09:29:163
12289990,9cyclictest0-21swapper/311:29:453
12289990,9cyclictest0-21swapper/311:12:063
12289990,6cyclictest99-21kswapd007:38:023
12199999,0cyclictest0-21swapper/209:07:132
12199998,1cyclictest33-21ksoftirqd/211:33:082
12199997,1cyclictest33-21ksoftirqd/209:30:422
12199997,1cyclictest16359-21sshd10:26:312
12199993,3cyclictest26974-21bash10:13:442
12199991,5cyclictest659-21id12:15:382
12199990,9cyclictest0-21swapper/210:45:262
12199990,9cyclictest0-21swapper/210:16:352
12199990,9cyclictest0-21swapper/208:39:372
12089998,1cyclictest6700-21kworker/1:007:28:361
12089998,1cyclictest2917-21kworker/1:107:56:201
12029999,0cyclictest0-21swapper/011:56:240
12029999,0cyclictest0-21swapper/010:25:450
12029999,0cyclictest0-21swapper/008:44:320
12029998,1cyclictest32437-21sshd09:55:000
12029997,1cyclictest9-21ksoftirqd/011:04:350
12029994,1cyclictest0-21swapper/011:17:250
12029990,9cyclictest0-21swapper/010:35:120
12029990,9cyclictest0-21swapper/010:14:140
12029990,1cyclictest0-21swapper/011:34:490
12029990,0cyclictest0-21swapper/011:41:160
12029990,0cyclictest0-21swapper/011:41:160
12029990,0cyclictest0-21swapper/009:28:500
12029990,0cyclictest0-21swapper/009:01:050
8360285,1sleep665-21ksoftirqd/611:00:006
12519987,1cyclictest65-21ksoftirqd/610:42:536
12519986,1cyclictest65-21ksoftirqd/612:22:346
12519986,1cyclictest65-21ksoftirqd/612:08:566
12519986,1cyclictest65-21ksoftirqd/609:03:156
12519984,3cyclictest18827-21bash11:59:516
12519984,3cyclictest11803-21grep10:55:376
12519982,6cyclictest5212-21sshd10:00:336
12519982,6cyclictest1704-21sshd10:24:306
12519981,6cyclictest17730-21sshd09:33:106
12519981,1cyclictest5460-21sshd11:44:066
12519981,1cyclictest5460-21sshd11:44:066
12369988,0cyclictest0-21swapper/411:15:154
12369988,0cyclictest0-21swapper/411:11:474
12369986,1cyclictest49-21ksoftirqd/411:50:224
12369986,1cyclictest49-21ksoftirqd/410:54:194
12369986,1cyclictest49-21ksoftirqd/410:09:374
12369986,1cyclictest49-21ksoftirqd/409:40:394
12369985,2cyclictest711-21dbus-daemon09:45:264
12369985,2cyclictest5563-21bash09:09:494
12369985,2cyclictest20977-21grep08:56:524
12369983,3cyclictest9768-21sshd10:40:144
12369982,3cyclictest6049-21bash11:44:134
12369982,3cyclictest6049-21bash11:44:134
12369982,3cyclictest28515-21grepconf.sh09:59:244
12369982,3cyclictest20001-21bash09:58:074
12369982,3cyclictest0-21swapper/409:34:174
12369981,3cyclictest656-21sshd10:34:134
12369981,3cyclictest656-21sshd10:34:134
12369981,3cyclictest0-21swapper/410:46:294
12369980,8cyclictest0-21swapper/411:54:304
12369980,5cyclictest27098-21sshd12:19:334
12289988,0cyclictest0-21swapper/309:01:243
12289987,1cyclictest41-21ksoftirqd/309:17:453
12289986,1cyclictest4438-21kworker/3:412:04:543
12289986,1cyclictest41-21ksoftirqd/311:44:313
12289986,1cyclictest41-21ksoftirqd/309:04:353
12289986,1cyclictest41-21ksoftirqd/307:04:203
12289986,1cyclictest28212-21kworker/3:012:14:523
12289986,1cyclictest28212-21kworker/3:012:01:323
12289986,1cyclictest27881-21kworker/3:311:37:343
12289986,1cyclictest26375-21kworker/3:111:49:243
12289986,1cyclictest26072-21kworker/3:211:14:323
12289986,1cyclictest25500-21kworker/3:411:40:223
12289986,1cyclictest25500-21kworker/3:411:40:223
12289986,1cyclictest1490-21kworker/3:209:30:043
12289986,1cyclictest11985-21kworker/3:510:19:503
12289986,1cyclictest1183-21kworker/3:010:50:503
12289985,2cyclictest18716-21grep10:02:453
12289984,1cyclictest20497-21diskstats12:09:293
12289983,2cyclictest0-21swapper/307:56:113
12289982,3cyclictest7836-21sshd09:51:183
12199988,0cyclictest0-21swapper/212:01:252
12199988,0cyclictest0-21swapper/210:57:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional