You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 21:25
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack3slot0.osadl.org (updated Wed Nov 19, 2025 12:43:27)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2630925625,8sleep00-21swapper/006:50:490
253752560,0sleep10-21swapper/109:49:301
2654325428,9sleep40-21swapper/406:53:574
2505624825,18sleep30-21swapper/306:49:483
135572470,0sleep30-21swapper/310:22:323
208872450,0sleep00-21swapper/011:02:330
309132440,0sleep50-21swapper/511:58:205
60632430,0sleep10-21swapper/110:45:121
269152430,0sleep30-21swapper/309:54:453
224762420,0sleep30-21swapper/310:42:583
2643624129,8sleep70-21swapper/706:52:327
235252410,0sleep30-21swapper/309:24:173
61992400,0sleep20-21swapper/212:23:482
32672400,2sleep42674599cyclictest09:50:534
288492400,0sleep00-21swapper/010:43:590
2655124028,8sleep20-21swapper/206:54:032
2654524027,9sleep60-21swapper/606:53:596
2646423928,8sleep10-21swapper/106:52:501
2410723926,9sleep50-21swapper/506:49:345
193382390,0sleep30-21swapper/310:28:163
176242390,0sleep50-21swapper/511:22:005
89002380,1sleep32673599cyclictest09:26:503
2676999296,16cyclictest10506-21sshd09:42:077
2676999248,10cyclictest0-21swapper/707:00:337
2676999232,14cyclictest12548-21sshd12:14:457
26754992313,5cyclictest57-21ksoftirqd/510:17:055
2676999228,9cyclictest0-21swapper/707:05:157
26754992116,1cyclictest57-21ksoftirqd/511:51:475
189602210,1sleep518952-21bash11:37:065
2676999208,10cyclictest0-21swapper/707:11:017
26754992013,7cyclictest57-21ksoftirqd/510:24:335
26754992013,7cyclictest57-21ksoftirqd/510:00:175
2676999198,5cyclictest0-21swapper/711:47:597
2676999198,5cyclictest0-21swapper/707:38:057
2676999198,0cyclictest0-21swapper/707:56:597
2676999194,11cyclictest31198-21sshd09:15:237
2676999194,10cyclictest0-21swapper/708:22:037
2676999193,10cyclictest0-21swapper/706:55:257
2676999192,6cyclictest0-21swapper/710:40:177
26754991917,2cyclictest14646-21kworker/5:410:43:335
26754991913,6cyclictest57-21ksoftirqd/509:14:095
26754991913,6cyclictest57-21ksoftirqd/507:35:515
26754991912,7cyclictest57-21ksoftirqd/507:41:595
26754991912,7cyclictest57-21ksoftirqd/507:23:535
206982190,0sleep50-21swapper/509:43:505
2676999188,10cyclictest0-21swapper/708:25:137
2676999188,10cyclictest0-21swapper/708:12:397
2676999188,10cyclictest0-21swapper/707:25:177
2676999187,5cyclictest0-21swapper/710:26:157
2676999187,5cyclictest0-21swapper/710:04:197
2676999187,5cyclictest0-21swapper/709:02:117
2676999182,10cyclictest0-21swapper/708:30:117
2675499187,6cyclictest57-21ksoftirqd/511:04:555
26754991816,2cyclictest5112-21kworker/5:208:10:415
26754991813,5cyclictest22658-21kworker/5:209:55:015
26754991810,2cyclictest57-21ksoftirqd/511:42:155
186622180,0sleep30-21swapper/311:51:223
2676999178,9cyclictest0-21swapper/708:35:017
2676999178,9cyclictest0-21swapper/708:16:197
2676999178,9cyclictest0-21swapper/707:41:317
2676999177,0cyclictest0-21swapper/707:46:237
2676999176,5cyclictest0-21swapper/712:06:457
2676999176,5cyclictest0-21swapper/710:50:017
2676999176,5cyclictest0-21swapper/710:31:297
2676999176,5cyclictest0-21swapper/710:15:397
2676999176,5cyclictest0-21swapper/708:04:437
2676999176,11cyclictest0-21swapper/711:34:457
2676999176,0cyclictest0-21swapper/711:26:337
2676999176,0cyclictest0-21swapper/711:13:217
2676999176,0cyclictest0-21swapper/711:07:497
2676999176,0cyclictest0-21swapper/710:13:357
2676999176,0cyclictest0-21swapper/709:55:137
2676999176,0cyclictest0-21swapper/709:28:317
2676999176,0cyclictest0-21swapper/709:11:297
2676999176,0cyclictest0-21swapper/707:54:137
2676999176,0cyclictest0-21swapper/707:15:037
2676999171,12cyclictest5668-21sshd10:59:537
2676999170,15cyclictest0-21swapper/708:59:057
26754991716,1cyclictest57-21ksoftirqd/507:50:135
26754991716,1cyclictest21280-21kworker/5:009:04:355
26754991715,2cyclictest57-21ksoftirqd/510:22:235
26754991712,5cyclictest57-21ksoftirqd/507:56:595
26754991711,1cyclictest57-21ksoftirqd/509:23:275
2676999167,9cyclictest0-21swapper/707:30:337
2676999166,0cyclictest0-21swapper/712:11:057
2676999165,5cyclictest0-21swapper/712:00:077
2676999165,5cyclictest0-21swapper/711:31:197
2676999165,5cyclictest0-21swapper/709:07:537
2676999165,0cyclictest0-21swapper/711:53:137
2676999161,11cyclictest13518-21grepconf.sh09:52:417
2676999160,4cyclictest0-21swapper/708:41:117
2676999160,12cyclictest7700-21sshd10:55:117
2676999160,12cyclictest1673-21idleruntime10:44:457
2675499169,7cyclictest57-21ksoftirqd/506:59:315
2675499168,6cyclictest57-21ksoftirqd/508:41:555
2675499167,9cyclictest57-21ksoftirqd/512:20:495
26754991615,0cyclictest0-21swapper/508:50:335
26754991613,1cyclictest57-21ksoftirqd/510:59:145
26754991612,4cyclictest57-21ksoftirqd/512:18:225
26754991611,5cyclictest5112-21kworker/5:208:34:475
26754991611,5cyclictest0-21swapper/508:16:135
26754991611,0cyclictest0-21swapper/507:16:435
26754991610,5cyclictest0-21swapper/512:00:155
26754991610,2cyclictest57-21ksoftirqd/508:55:595
2675499160,11cyclictest5112-21kworker/5:207:25:375
26682160,0sleep30-21swapper/309:16:013
3672150,0sleep70-21swapper/709:30:307
290622150,1sleep3763-21runrttasks12:17:163
2676999154,5cyclictest0-21swapper/710:06:397
2676999154,5cyclictest0-21swapper/709:48:057
2676999154,5cyclictest0-21swapper/709:48:057
2676999154,5cyclictest0-21swapper/708:45:217
2676999154,5cyclictest0-21swapper/708:02:017
2676999154,0cyclictest0-21swapper/711:19:437
2676999154,0cyclictest0-21swapper/711:16:577
2676999150,9cyclictest0-21swapper/711:57:337
2676999150,9cyclictest0-21swapper/710:19:557
2676999150,9cyclictest0-21swapper/709:34:537
2676999150,9cyclictest0-21swapper/709:20:317
2675499159,5cyclictest57-21ksoftirqd/512:04:595
26754991514,1cyclictest57-21ksoftirqd/510:36:585
26754991514,1cyclictest57-21ksoftirqd/509:03:275
26754991514,1cyclictest57-21ksoftirqd/507:08:275
26754991514,0cyclictest5112-21kworker/5:207:30:495
26754991513,2cyclictest21867-21kworker/5:011:47:415
26754991513,2cyclictest21867-21kworker/5:011:10:475
26754991513,2cyclictest0-21swapper/508:28:015
26754991513,1cyclictest57-21ksoftirqd/510:46:375
26754991510,5cyclictest57-21ksoftirqd/511:27:055
26754991510,5cyclictest57-21ksoftirqd/509:44:535
26754991510,5cyclictest57-21ksoftirqd/509:44:535
26754991510,5cyclictest57-21ksoftirqd/509:14:375
26754991510,5cyclictest4062-21kworker/5:112:10:135
26754991510,3cyclictest57-21ksoftirqd/509:50:435
2675499150,9cyclictest0-21swapper/508:06:355
26735991512,1cyclictest41-21ksoftirqd/310:51:103
212732150,0sleep20-21swapper/210:18:542
136542150,0sleep00-21swapper/011:40:530
2676999143,5cyclictest0-21swapper/712:20:277
2676999143,5cyclictest0-21swapper/710:36:157
2676999142,6cyclictest0-21swapper/707:19:457
2676999140,8cyclictest0-21swapper/711:43:217
2675499149,5cyclictest57-21ksoftirqd/508:31:195
2675499149,5cyclictest5112-21kworker/5:207:09:515
2675499148,6cyclictest0-21swapper/508:01:475
2675499148,3cyclictest21867-21kworker/5:010:51:535
2675499144,5cyclictest57-21ksoftirqd/511:30:055
2675499143,6cyclictest57-21ksoftirqd/511:01:395
2675499143,5cyclictest121rcu_preempt08:44:395
26754991413,0cyclictest5112-21kworker/5:206:55:595
26754991412,1cyclictest57-21ksoftirqd/511:16:015
26754991412,1cyclictest57-21ksoftirqd/510:34:285
26754991411,3cyclictest57-21ksoftirqd/510:13:475
26754991411,3cyclictest57-21ksoftirqd/509:30:275
26754991410,4cyclictest57-21ksoftirqd/509:36:135
2675499140,2cyclictest57-21ksoftirqd/510:04:575
26735991413,1cyclictest41-21ksoftirqd/309:54:233
26735991413,1cyclictest41-21ksoftirqd/309:44:573
26735991413,1cyclictest41-21ksoftirqd/309:44:573
26735991413,1cyclictest41-21ksoftirqd/309:42:193
26735991413,1cyclictest41-21ksoftirqd/309:36:243
26735991413,1cyclictest41-21ksoftirqd/309:30:413
26735991412,1cyclictest41-21ksoftirqd/309:06:103
2673599140,1cyclictest131rcu_sched12:01:443
26728991411,3cyclictest33-21ksoftirqd/209:59:332
103292140,1sleep110321-21bash10:31:351
309482130,0sleep00-21swapper/012:22:340
2676999132,5cyclictest0-21swapper/708:53:177
2676199137,6cyclictest0-21swapper/612:18:176
2676199137,0cyclictest0-21swapper/611:24:156
2675499138,5cyclictest5112-21kworker/5:207:45:475
2675499138,0cyclictest0-21swapper/508:23:335
26735991312,1cyclictest41-21ksoftirqd/311:29:383
26735991312,1cyclictest41-21ksoftirqd/310:34:033
26735991312,1cyclictest41-21ksoftirqd/310:16:113
26735991311,1cyclictest41-21ksoftirqd/311:55:533
26735991311,1cyclictest41-21ksoftirqd/311:08:293
26735991311,1cyclictest41-21ksoftirqd/310:35:243
26735991311,1cyclictest41-21ksoftirqd/310:04:463
26728991311,1cyclictest33-21ksoftirqd/211:49:142
288892120,1sleep628890-21bash11:48:046
2676199126,5cyclictest30435-21grep09:39:566
26761991211,1cyclictest65-21ksoftirqd/610:42:496
2675499127,5cyclictest0-21swapper/509:24:535
26735991211,1cyclictest41-21ksoftirqd/312:14:173
26735991211,1cyclictest41-21ksoftirqd/312:06:563
26735991211,1cyclictest41-21ksoftirqd/311:25:563
26735991211,1cyclictest41-21ksoftirqd/311:13:473
26735991211,1cyclictest41-21ksoftirqd/308:57:223
26735991210,1cyclictest41-21ksoftirqd/312:19:433
26735991210,1cyclictest41-21ksoftirqd/311:41:313
26735991210,1cyclictest41-21ksoftirqd/311:04:083
26735991210,1cyclictest41-21ksoftirqd/309:00:093
2673599120,1cyclictest0-21swapper/311:14:343
26728991211,1cyclictest33-21ksoftirqd/210:53:132
26728991210,1cyclictest33-21ksoftirqd/211:55:512
2672899121,10cyclictest0-21swapper/212:08:542
26720991210,1cyclictest2949-21kworker/1:110:27:471
2672099120,2cyclictest3321-21sshd10:15:541
2672099120,11cyclictest0-21swapper/109:15:311
198852120,0sleep00-21swapper/010:33:020
2676199119,1cyclictest65-21ksoftirqd/610:04:016
2676199117,2cyclictest65-21ksoftirqd/610:31:096
2676199116,5cyclictest0-21swapper/609:00:236
2676199114,4cyclictest1-21systemd11:43:416
2676199110,1cyclictest1-21systemd10:51:396
2674599119,1cyclictest0-21swapper/409:34:454
2674599110,10cyclictest1-21systemd11:17:424
2674599110,1cyclictest0-21swapper/409:26:334
2673599119,1cyclictest41-21ksoftirqd/311:45:073
2673599119,1cyclictest41-21ksoftirqd/309:59:503
2673599118,1cyclictest41-21ksoftirqd/310:55:163
2673599117,4cyclictest41-21ksoftirqd/311:35:003
26735991110,1cyclictest41-21ksoftirqd/310:44:473
26735991110,1cyclictest41-21ksoftirqd/310:13:163
2672899119,2cyclictest33-21ksoftirqd/211:17:592
2672899119,1cyclictest33-21ksoftirqd/211:52:462
2672899119,1cyclictest33-21ksoftirqd/210:06:072
2672899119,1cyclictest33-21ksoftirqd/209:05:402
2672899118,2cyclictest32457-21grepconf.sh09:30:272
26728991110,1cyclictest33-21ksoftirqd/209:49:532
2672899110,1cyclictest0-21swapper/209:37:232
2672099113,5cyclictest0-21swapper/107:40:321
2672099110,10cyclictest0-21swapper/110:42:421
2671299113,4cyclictest26090-21/usr/sbin/munin10:24:280
265652110,0sleep60-21swapper/608:44:486
189212110,0sleep4471rcuc/411:12:144
185042110,5sleep40-21swapper/411:32:164
91542100,1sleep665-21ksoftirqd/611:10:406
2676199109,0cyclictest85350irq/25-em1-rx-007:24:406
2676199108,1cyclictest65-21ksoftirqd/607:54:286
2676199106,1cyclictest26462-21grep11:52:376
2676199104,3cyclictest4233-21bash09:21:146
2674599109,1cyclictest49-21ksoftirqd/411:42:094
2674599109,1cyclictest49-21ksoftirqd/409:16:204
2674599108,1cyclictest49-21ksoftirqd/411:35:434
2674599102,5cyclictest711-21dbus-daemon10:49:234
2674599100,10cyclictest0-21swapper/410:37:394
2674599100,10cyclictest0-21swapper/410:01:144
2673599109,1cyclictest41-21ksoftirqd/311:20:253
2673599109,1cyclictest41-21ksoftirqd/309:10:333
2672899109,1cyclictest33-21ksoftirqd/210:29:472
2672899109,1cyclictest33-21ksoftirqd/209:44:522
2672899109,1cyclictest33-21ksoftirqd/209:44:522
2672899109,1cyclictest33-21ksoftirqd/209:42:082
2672899109,1cyclictest33-21ksoftirqd/209:27:182
2672899109,1cyclictest33-21ksoftirqd/209:19:412
2672899109,1cyclictest0-21swapper/212:13:382
2672899105,4cyclictest33-21ksoftirqd/211:37:002
2672099109,1cyclictest30595-21kworker/1:311:33:391
2672099109,1cyclictest25-21ksoftirqd/110:09:221
2672099109,1cyclictest0-21swapper/112:22:581
2672099108,1cyclictest25-21ksoftirqd/110:12:191
2672099104,4cyclictest14589-21bash12:00:321
2672099100,10cyclictest0-21swapper/111:44:581
2672099100,10cyclictest0-21swapper/108:54:421
2672099100,1cyclictest0-21swapper/110:56:531
2671299108,1cyclictest9-21ksoftirqd/012:06:230
2671299108,1cyclictest9-21ksoftirqd/012:04:270
2671299108,1cyclictest9-21ksoftirqd/011:25:530
2671299108,1cyclictest9-21ksoftirqd/009:20:120
2671299104,3cyclictest0-21swapper/012:11:530
267619998,1cyclictest65-21ksoftirqd/610:12:386
267619992,4cyclictest7810-21grep10:26:256
267619992,4cyclictest0-21swapper/612:00:146
267619992,3cyclictest749-21sshd09:35:286
267619992,3cyclictest29205-21sshd08:59:166
267619992,3cyclictest0-21swapper/611:04:446
267619991,5cyclictest25815-21bash10:19:366
267619990,9cyclictest0-21swapper/610:55:046
267619990,3cyclictest29515-21sshd09:55:006
267619990,3cyclictest0-21swapper/610:05:456
267459999,0cyclictest0-21swapper/410:32:554
267459998,1cyclictest49-21ksoftirqd/410:49:394
267459998,1cyclictest49-21ksoftirqd/409:11:174
267459998,1cyclictest49-21ksoftirqd/409:08:534
267459998,1cyclictest49-21ksoftirqd/408:57:044
267459997,1cyclictest49-21ksoftirqd/412:22:254
267459997,1cyclictest49-21ksoftirqd/410:59:344
267459997,1cyclictest49-21ksoftirqd/410:39:474
267459997,1cyclictest49-21ksoftirqd/409:56:094
267459995,2cyclictest2999-21cp12:13:224
267459990,0cyclictest0-21swapper/412:04:504
267459990,0cyclictest0-21swapper/412:00:064
267459990,0cyclictest0-21swapper/410:19:594
267459990,0cyclictest0-21swapper/409:20:054
267359997,1cyclictest41-21ksoftirqd/308:54:283
267359997,1cyclictest41-21ksoftirqd/307:39:283
267359996,3cyclictest41-21ksoftirqd/307:54:433
267289999,0cyclictest13450irq/24-ahci[00010:43:032
267289998,1cyclictest33-21ksoftirqd/211:43:382
267289998,1cyclictest33-21ksoftirqd/211:09:432
267289998,1cyclictest33-21ksoftirqd/210:23:112
267289998,1cyclictest33-21ksoftirqd/210:13:232
267289998,1cyclictest33-21ksoftirqd/208:54:512
267289997,1cyclictest33-21ksoftirqd/212:18:252
267289997,1cyclictest33-21ksoftirqd/210:47:022
267289997,1cyclictest33-21ksoftirqd/210:24:292
267289996,2cyclictest12201-21bash09:17:322
267289990,9cyclictest0-21swapper/210:57:572
267289990,9cyclictest0-21swapper/209:55:122
267209998,1cyclictest25-21ksoftirqd/111:22:021
267209998,1cyclictest25-21ksoftirqd/109:02:071
267209997,1cyclictest25-21ksoftirqd/111:43:181
267209997,1cyclictest25-21ksoftirqd/111:26:181
267209997,1cyclictest25-21ksoftirqd/110:20:581
267209997,1cyclictest13013-21kworker/1:109:41:051
267209992,4cyclictest669-21avahi-daemon10:50:151
267209990,9cyclictest0-21swapper/112:14:461
267209990,9cyclictest0-21swapper/112:08:181
267209990,9cyclictest0-21swapper/110:36:141
267209990,9cyclictest0-21swapper/109:20:151
267129998,1cyclictest9-21ksoftirqd/009:18:280
267129998,1cyclictest0-21swapper/009:52:180
267129997,1cyclictest9-21ksoftirqd/011:47:440
267129997,1cyclictest9-21ksoftirqd/011:29:510
267129997,1cyclictest9-21ksoftirqd/010:50:290
267129997,1cyclictest9-21ksoftirqd/010:28:510
267129997,1cyclictest9-21ksoftirqd/009:39:290
267129997,1cyclictest9-21ksoftirqd/009:30:460
267129997,1cyclictest9-21ksoftirqd/009:13:050
267129994,3cyclictest23591-21sshd09:59:270
267129993,5cyclictest859-21polkitd10:16:140
267129990,9cyclictest0-21swapper/012:15:080
267129990,9cyclictest0-21swapper/011:35:530
267129990,9cyclictest0-21swapper/011:22:200
2130290,0sleep00-21swapper/009:45:450
2130290,0sleep00-21swapper/009:45:450
267619986,1cyclictest65-21ksoftirqd/612:10:106
267619986,1cyclictest65-21ksoftirqd/609:29:446
267619986,1cyclictest65-21ksoftirqd/609:07:596
267619985,2cyclictest615-21grep10:15:266
267619983,4cyclictest711-21dbus-daemon11:14:436
267619982,4cyclictest3627-21cp11:34:486
267619982,4cyclictest10745-21sshd12:19:346
267619982,3cyclictest11713-21bash09:47:196
267619982,3cyclictest11713-21bash09:47:196
267619982,3cyclictest11145-21systemd-cgroups09:52:166
267619982,3cyclictest0-21swapper/611:30:466
267619981,4cyclictest669-21avahi-daemon10:44:556
267619981,4cyclictest5583-21sshd10:59:526
267619981,4cyclictest11804-21sshd10:36:316
267619980,8cyclictest24163-21kworker/6:011:28:456
267619980,8cyclictest12899-21kworker/6:109:13:116
267619980,1cyclictest20908-21grepconf.sh12:06:136
267459988,0cyclictest0-21swapper/410:10:224
267459988,0cyclictest0-21swapper/409:03:414
267459987,1cyclictest49-21ksoftirqd/411:58:334
267459987,1cyclictest49-21ksoftirqd/411:07:014
267459987,1cyclictest49-21ksoftirqd/410:57:184
267459987,1cyclictest49-21ksoftirqd/409:44:074
267459986,1cyclictest49-21ksoftirqd/411:52:404
267459986,1cyclictest49-21ksoftirqd/409:48:044
267459986,1cyclictest49-21ksoftirqd/409:48:044
267459986,1cyclictest49-21ksoftirqd/409:31:024
267459983,3cyclictest711-21dbus-daemon12:15:224
267459980,8cyclictest0-21swapper/407:21:034
267459980,7cyclictest11860-21kworker/4:210:04:294
267459980,5cyclictest861-21gdbus11:45:544
267359987,1cyclictest10399-21kworker/3:008:34:473
267359982,3cyclictest468-21systemd-journal07:04:283
267289988,0cyclictest0-21swapper/207:04:472
267289987,1cyclictest33-21ksoftirqd/211:25:082
267289987,1cyclictest33-21ksoftirqd/211:04:482
267289987,1cyclictest33-21ksoftirqd/210:35:402
267289986,1cyclictest33-21ksoftirqd/211:02:232
267289986,1cyclictest33-21ksoftirqd/209:03:002
267289983,2cyclictest29162-21sshd11:33:592
267289980,5cyclictest27653-21httpd09:09:582
267209988,0cyclictest0-21swapper/109:26:431
267209987,1cyclictest25-21ksoftirqd/109:59:381
267209987,1cyclictest25-21ksoftirqd/109:56:581
267209986,1cyclictest31280-21kworker/1:112:11:531
267209986,1cyclictest25-21ksoftirqd/111:49:441
267209986,1cyclictest25-21ksoftirqd/111:34:301
267209986,1cyclictest25-21ksoftirqd/111:00:371
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional