You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-20 - 21:09
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Sat Jun 20, 2026 12:54:59)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
286719972747,592cyclictest0-21swapper/109:48:231
2867399705322,254cyclictest4274-21diskmemload11:11:093
28672996899,677cyclictest7194-21ssh11:13:572
286709966627,423cyclictest0-21swapper/011:13:260
286749966570,422cyclictest30918-21ssh10:49:064
286719965416,211cyclictest0-21swapper/110:56:471
2867699634123,508cyclictest0-21swapper/612:22:076
28671996341,630cyclictest0-21swapper/112:33:341
2867099632208,126cyclictest4274-21diskmemload12:32:290
286709962934,380cyclictest0-21swapper/010:48:400
286739961965,381cyclictest0-21swapper/309:49:113
286719961620,84cyclictest0-21swapper/111:51:321
2867199612232,175cyclictest31742-21nscd11:12:401
2867699609271,169cyclictest0-21swapper/611:11:526
286749960493,380cyclictest0-21swapper/412:31:284
28671995961,126cyclictest0-21swapper/111:00:221
28676995940,2cyclictest0-21swapper/610:59:136
286709959484,507cyclictest0-21swapper/012:22:070
28672995930,84cyclictest0-21swapper/212:32:172
28672995930,84cyclictest0-21swapper/209:48:092
28675995920,83cyclictest0-21swapper/511:14:295
28671995921,507cyclictest0-21swapper/110:47:401
2867799591121,254cyclictest0-21swapper/712:22:077
286749959082,126cyclictest0-21swapper/411:14:194
2867499590124,84cyclictest0-21swapper/410:34:054
2867299587165,257cyclictest4781-21kworker/u16:010:23:262
2867299587165,257cyclictest4781-21kworker/u16:010:23:262
286719958633,550cyclictest24877-21ssh11:25:081
286719957220,126cyclictest0-21swapper/112:12:101
2867499564225,337cyclictest64450irq/122-eth012:11:424
286719956416,295cyclictest0-21swapper/107:16:121
286759956313,84cyclictest0-21swapper/510:48:275
286739955888,296cyclictest0-21swapper/311:37:003
28671995532,84cyclictest0-21swapper/111:23:431
28671995532,84cyclictest0-21swapper/111:23:431
286739955240,381cyclictest0-21swapper/310:47:253
286769954980,211cyclictest641ktimersoftd/609:47:376
286729954981,296cyclictest0-21swapper/211:00:192
2867499546205,339cyclictest64450irq/122-eth012:23:334
286739954634,127cyclictest0-21swapper/311:22:133
286739954634,127cyclictest0-21swapper/311:22:133
286719954534,508cyclictest0-21swapper/112:01:111
286739954434,507cyclictest0-21swapper/312:22:073
286719954433,508cyclictest0-21swapper/111:36:341
2867499543204,337cyclictest64450irq/122-eth011:22:564
2867499543204,337cyclictest64450irq/122-eth011:22:564
286769954276,84cyclictest0-21swapper/610:46:286
286719954234,89cyclictest0-21swapper/110:50:041
286729953927,127cyclictest0-21swapper/207:18:212
286759953667,296cyclictest0-21swapper/511:25:095
2867599536110,258cyclictest121rcu_preempt12:23:095
286729952719,84cyclictest121rcu_preempt11:24:162
286729952719,84cyclictest121rcu_preempt11:24:162
286729952719,126cyclictest0-21swapper/210:57:522
286709952514,296cyclictest0-21swapper/011:35:490
286709952513,380cyclictest14293-21ssh09:46:080
286709952413,338cyclictest0-21swapper/007:16:380
286749952013,423cyclictest0-21swapper/411:15:154
286739951849,296cyclictest0-21swapper/312:32:173
286719951853,254cyclictest22918-21kworker/1:212:22:151
286749951710,259cyclictest0-21swapper/411:36:194
28675995168,126cyclictest0-21swapper/509:48:245
28672995121,342cyclictest0-21swapper/210:47:262
286779950683,211cyclictest0-21swapper/712:31:077
286769950679,211cyclictest65-21ksoftirqd/612:31:406
28671995060,2cyclictest0-21swapper/112:44:291
286709950581,84cyclictest121rcu_preempt11:51:010
286709950339,85cyclictest0-21swapper/012:12:060
286749950137,297cyclictest0-21swapper/412:41:074
2867799500285,84cyclictest0-21swapper/711:51:377
2867799498118,296cyclictest4274-21diskmemload11:21:397
2867799498118,296cyclictest4274-21diskmemload11:21:397
286739949829,211cyclictest0-21swapper/311:40:053
2867299498453,42cyclictest321ktimersoftd/212:12:092
286749949732,211cyclictest0-21swapper/410:56:324
286779949366,211cyclictest73-21ksoftirqd/711:11:137
2867699492237,91cyclictest0-21swapper/611:51:026
2867399490153,253cyclictest28259-1kworker/u17:012:12:533
286759948968,84cyclictest0-21swapper/512:41:185
286729948965,211cyclictest0-21swapper/212:21:342
286749948521,300cyclictest0-21swapper/409:47:424
2867199485313,84cyclictest0-21swapper/110:23:261
2867199485313,84cyclictest0-21swapper/110:23:261
286749948414,211cyclictest49-21ksoftirqd/411:00:074
286779948255,211cyclictest15126-1kworker/u17:211:36:337
2867099482227,253cyclictest101ktimersoftd/012:41:310
286779948117,2cyclictest0-21swapper/709:47:427
2867499480223,85cyclictest0-21swapper/411:40:044
286759947855,211cyclictest0-21swapper/507:16:185
28673994769,127cyclictest0-21swapper/310:56:353
28670994737,126cyclictest0-21swapper/011:23:440
28670994737,126cyclictest0-21swapper/011:23:440
28677994681,212cyclictest0-21swapper/712:11:427
286749946744,126cyclictest0-21swapper/407:18:174
28670994671,214cyclictest9-21ksoftirqd/010:56:330
286759946649,120cyclictest16999-21sh09:50:065
2867099466124,339cyclictest0-21swapper/012:25:050
286759946580,211cyclictest57-21ksoftirqd/510:20:525
286759946580,211cyclictest57-21ksoftirqd/510:20:525
2867699464421,41cyclictest631rcuc/612:12:096
286769946277,211cyclictest65-21ksoftirqd/607:19:416
286729946235,211cyclictest33-21ksoftirqd/211:36:042
286759946177,211cyclictest57-21ksoftirqd/512:31:445
2867399461122,84cyclictest16134-21cstates11:00:233
286779946076,381cyclictest0-21swapper/710:59:377
2867099460121,84cyclictest0-21swapper/007:20:080
2867299459118,338cyclictest0-21swapper/211:51:262
2867699458118,338cyclictest0-21swapper/611:37:576
286759945874,211cyclictest0-21swapper/512:12:105
286759945836,2cyclictest0-21swapper/511:51:405
2867499458117,338cyclictest0-21swapper/411:51:304
286739945735,256cyclictest0-21swapper/310:23:263
286739945735,256cyclictest0-21swapper/310:23:263
286759945530,85cyclictest0-21swapper/510:59:235
286769945431,257cyclictest65-21ksoftirqd/611:25:086
2867299454409,42cyclictest311rcuc/212:44:192
286719945432,338cyclictest0-21swapper/112:25:081
286779945326,211cyclictest73-21ksoftirqd/712:04:337
286759945374,87cyclictest0-21swapper/511:22:025
286759945374,87cyclictest0-21swapper/511:22:025
2867399453114,84cyclictest0-21swapper/312:41:313
2867099453113,337cyclictest0-21swapper/012:01:450
2867399452113,84cyclictest0-21swapper/311:25:113
286749945025,422cyclictest0-21swapper/412:01:284
286759944826,168cyclictest57-21ksoftirqd/511:36:065
2867399448109,84cyclictest0-21swapper/312:01:273
2867399448108,338cyclictest0-21swapper/312:25:053
286709944824,340cyclictest16151-21cstates11:00:230
286779944725,256cyclictest0-21swapper/711:25:087
286779944723,84cyclictest0-21swapper/710:46:557
286779944723,127cyclictest0-21swapper/710:50:097
286739944724,256cyclictest0-21swapper/307:19:393
286759944625,84cyclictest0-21swapper/511:15:085
286759944561,169cyclictest5045-21ssh10:34:195
286749944221,255cyclictest0-21swapper/407:20:064
286779944161,296cyclictest0-21swapper/712:41:187
286769944199,339cyclictest0-21swapper/610:20:476
286769944199,339cyclictest0-21swapper/610:20:476
286729944161,87cyclictest0-21swapper/212:01:042
286769944055,211cyclictest23804-21ssh11:24:206
286769944055,211cyclictest23804-21ssh11:24:206
286749944018,256cyclictest0-21swapper/411:25:074
286729943814,127cyclictest15739-21kworker/u16:312:25:062
286709943598,42cyclictest8053-21ssh11:15:090
286779943494,338cyclictest4274-21diskmemload11:40:047
286729943458,84cyclictest0-21swapper/207:20:082
28670994348,211cyclictest9-21ksoftirqd/010:50:060
286749943251,169cyclictest0-21swapper/412:15:024
286769942713,330cyclictest0-21swapper/612:02:166
2867399427128,126cyclictest41-21ksoftirqd/311:51:273
286779942483,338cyclictest0-21swapper/711:15:137
286779942483,338cyclictest0-21swapper/710:20:537
286779942483,338cyclictest0-21swapper/710:20:537
28674994242,256cyclictest0-21swapper/412:25:104
286779942382,338cyclictest1594-21needreboot07:15:197
286779942381,339cyclictest0-21swapper/711:00:187
286769942383,338cyclictest0-21swapper/611:00:146
286749942382,338cyclictest0-21swapper/410:22:154
286749942382,338cyclictest0-21swapper/410:22:144
286709942382,338cyclictest0-21swapper/009:50:060
286759942182,84cyclictest0-21swapper/511:00:215
286739942186,130cyclictest2120-21ssh10:31:353
286709942181,338cyclictest0-21swapper/011:40:050
286709942180,338cyclictest0-21swapper/010:21:110
286709942180,338cyclictest0-21swapper/010:21:110
2867299420376,41cyclictest311rcuc/211:15:152
286719941878,338cyclictest0-21swapper/111:15:101
286719941876,339cyclictest0-21swapper/112:15:061
286729941778,84cyclictest0-21swapper/211:40:042
286719941734,126cyclictest0-21swapper/110:34:041
286749941677,337cyclictest0-21swapper/410:50:234
286719941676,338cyclictest0-21swapper/107:20:111
286709941678,336cyclictest0-21swapper/011:25:110
286739941575,338cyclictest0-21swapper/307:20:073
286769941473,338cyclictest0-21swapper/612:44:196
286759941478,41cyclictest0-21swapper/510:50:055
286759941476,84cyclictest0-21swapper/512:02:175
286779941372,338cyclictest0-21swapper/709:50:067
286769941271,338cyclictest0-21swapper/611:40:036
2867399410111,127cyclictest41-21ksoftirqd/310:50:083
286769940968,338cyclictest0-21swapper/610:50:056
286739940924,296cyclictest0-21swapper/311:15:113
286719940766,338cyclictest0-21swapper/111:40:051
286769940567,211cyclictest353-21systemd-udevd10:31:366
286759940465,83cyclictest0-21swapper/512:05:075
286759940465,83cyclictest0-21swapper/512:05:075
286759940365,84cyclictest19208-21ssh11:40:055
286779940120,84cyclictest0-21swapper/707:20:117
286729940062,84cyclictest0-21swapper/211:25:082
286729939758,84cyclictest0-21swapper/210:50:052
286749939314,213cyclictest64450irq/122-eth009:50:094
286729939212,169cyclictest33-21ksoftirqd/210:31:342
286769938849,84cyclictest0-21swapper/611:15:106
286709938245,126cyclictest121rcu_preempt10:34:070
28676993811,85cyclictest0-21swapper/607:20:056
286769937880,126cyclictest121rcu_preempt09:50:126
286779937538,253cyclictest0-21swapper/710:31:347
286719937537,171cyclictest0-21swapper/109:50:091
286729937331,127cyclictest4274-21diskmemload09:50:092
286779937131,338cyclictest0-21swapper/712:25:087
286739937133,171cyclictest0-21swapper/309:50:093
286759936829,84cyclictest0-21swapper/512:15:065
286729936828,84cyclictest0-21swapper/212:15:062
286709936731,205cyclictest16406-21ssh12:15:060
286759935358,42cyclictest0-21swapper/512:25:105
286769934647,169cyclictest65-21ksoftirqd/612:25:066
286769933841,84cyclictest0-21swapper/612:15:066
286739933579,253cyclictest15746-21apt-get12:15:063
286719933237,217cyclictest0-21swapper/110:35:021
286769933174,254cyclictest0-21swapper/612:05:076
286769933174,254cyclictest0-21swapper/612:05:076
286709932829,218cyclictest0-21swapper/010:35:020
286779931861,254cyclictest0-21swapper/712:15:067
286749931862,253cyclictest0-21swapper/412:05:074
286749931862,253cyclictest0-21swapper/412:05:074
286759930448,253cyclictest0-21swapper/507:20:115
286729928941,84cyclictest33-21ksoftirqd/210:35:022
286769928835,2cyclictest6166-21ssh10:35:026
286779928225,254cyclictest0-21swapper/712:05:077
286779928225,254cyclictest0-21swapper/712:05:077
286709928230,41cyclictest0-21swapper/012:05:070
286709928230,41cyclictest0-21swapper/012:05:070
286729927016,84cyclictest0-21swapper/212:05:072
286729927016,84cyclictest0-21swapper/212:05:072
286759926857,44cyclictest0-21swapper/510:35:025
286749926215,2cyclictest0-21swapper/410:35:024
286739926213,3cyclictest0-21swapper/310:35:023
28673992561,253cyclictest0-21swapper/312:05:073
28673992561,253cyclictest0-21swapper/312:05:073
28671992553,84cyclictest25-21ksoftirqd/112:05:071
28671992553,84cyclictest25-21ksoftirqd/112:05:071
28677992513,36cyclictest0-21swapper/710:35:027
26860210196,2sleep20-21swapper/207:05:182
91322880,0sleep60-21swapper/611:33:346
2807728245,34sleep00-21swapper/007:07:400
2792128252,27sleep30-21swapper/307:05:273
2822128051,26sleep50-21swapper/507:09:455
2814028050,27sleep70-21swapper/707:08:357
2813727949,27sleep40-21swapper/407:08:324
2812127848,27sleep60-21swapper/607:08:186
2802927747,27sleep10-21swapper/107:07:011
278072710,2sleep22867299cyclictest10:10:432
208692690,1sleep00-21swapper/011:58:400
2867599563,44cyclictest57-21ksoftirqd/510:25:025
2867299564,44cyclictest33-21ksoftirqd/210:25:052
2867799501,46cyclictest0-21swapper/710:25:157
167752470,1sleep30-21swapper/311:55:303
2867699452,9cyclictest121rcu_preempt10:25:116
2867799392,33cyclictest0-21swapper/712:35:127
2867499392,4cyclictest0-21swapper/411:55:024
2867499391,5cyclictest21557-21/usr/sbin/munin10:25:164
2867399392,33cyclictest0-21swapper/310:25:233
2867299396,18cyclictest33-21ksoftirqd/211:56:002
2867299384,19cyclictest33-21ksoftirqd/212:35:132
2867199382,32cyclictest0-21swapper/111:55:041
2867099381,5cyclictest21531-21sed10:25:160
2867599372,30cyclictest0-21swapper/512:35:035
2867499371,33cyclictest0-21swapper/412:35:114
2867199372,31cyclictest0-21swapper/110:25:181
28670993728,5cyclictest17764-21ntpq12:35:240
28677993628,5cyclictest0-21swapper/711:55:017
2867299366,21cyclictest33-21ksoftirqd/209:55:042
2867699351,31cyclictest0-21swapper/612:35:036
2867599359,14cyclictest121rcu_preempt09:55:135
2867199342,28cyclictest0-21swapper/112:35:121
2867599338,13cyclictest16012-21netstat11:55:235
28677993122,5cyclictest27635-21cut09:55:167
28676993110,10cyclictest121rcu_preempt11:55:156
2867699307,10cyclictest121rcu_preempt09:55:126
28673993022,5cyclictest16054-21sh12:35:123
28671993023,4cyclictest26609-21ssh09:55:061
28670992922,4cyclictest27035-21expr09:55:130
28673992821,4cyclictest26632-21sh09:55:083
28677992727,0cyclictest0-21swapper/707:12:427
28676992727,0cyclictest0-21swapper/607:12:246
28675992727,0cyclictest0-21swapper/507:12:225
28674992710,17cyclictest0-21swapper/407:12:474
28673992727,0cyclictest0-21swapper/307:12:303
28672992710,17cyclictest0-21swapper/207:12:502
28671992710,17cyclictest0-21swapper/107:12:241
28674992620,4cyclictest28184-21cat09:55:204
28670992424,0cyclictest0-21swapper/007:12:500
71482230,2sleep02867099cyclictest09:10:220
174902210,1sleep50-21swapper/509:15:235
22262190,1sleep40-21swapper/409:05:134
201312190,0sleep40-21swapper/409:17:064
175252180,0sleep50-21swapper/508:35:185
28677991614,1cyclictest0-21swapper/707:40:187
2867499159,4cyclictest1531-21ssh11:47:174
2867499159,4cyclictest1531-21ssh11:47:174
2867299150,2cyclictest0-21swapper/211:47:572
2867299150,2cyclictest0-21swapper/211:47:572
2867099156,8cyclictest0-21swapper/008:40:000
28676991412,1cyclictest7207-21grep07:25:156
28675991410,2cyclictest6472-21cut07:25:135
2867499149,3cyclictest18581-21tr07:45:124
28674991411,2cyclictest481ktimersoftd/410:08:384
28674991411,2cyclictest481ktimersoftd/410:00:064
28673991412,1cyclictest12153-21sed07:35:003
28672991412,1cyclictest0-21swapper/210:09:342
2867199149,3cyclictest6302-21tr10:00:201
2867199149,3cyclictest20131-21tail08:40:151
28670991412,1cyclictest30395-21chrt10:12:250
2867799139,3cyclictest658-21date09:00:217
2867799139,3cyclictest658-21date09:00:217
2867799139,3cyclictest15776-21tr10:40:117
2867799138,3cyclictest15423-21head09:30:257
2867799138,3cyclictest15423-21head09:30:257
28677991311,1cyclictest7893-21rm10:00:597
28677991311,1cyclictest0-21swapper/711:32:427
28676991310,2cyclictest29763-21ls08:00:266
28676991310,2cyclictest22213-21tr08:45:126
2867599139,3cyclictest10318-21grep07:30:145
2867499139,3cyclictest9149-21sh09:11:314
2867499139,3cyclictest4766-21rm09:41:284
2867499139,3cyclictest27696-21sh09:36:524
2867499139,3cyclictest16299-21grep07:40:154
2867499139,2cyclictest25536-21cat09:20:114
2867499139,2cyclictest11262-21sort07:30:224
28674991311,1cyclictest96742sleep411:33:574
28674991311,1cyclictest14053-21smartctl09:30:164
28674991311,1cyclictest14053-21smartctl09:30:164
28674991311,1cyclictest0-21swapper/410:44:524
28674991311,1cyclictest0-21swapper/410:17:194
28674991310,2cyclictest481ktimersoftd/408:10:244
28673991311,1cyclictest15058-21cat10:05:133
2867299139,3cyclictest5351-21cat09:10:122
28672991311,1cyclictest5043-21cstates10:00:122
28672991311,1cyclictest182952sleep210:40:292
28672991311,1cyclictest1829-21ssh09:24:472
28672991311,1cyclictest0-21swapper/209:41:252
28672991310,2cyclictest376-21head08:05:232
28672991310,2cyclictest12225-21sh09:30:002
28671991311,1cyclictest32223-21ssh10:13:381
28671991311,1cyclictest24281-21ssh10:44:301
28671991311,1cyclictest15038-21sed08:30:211
28671991311,1cyclictest0-21swapper/109:10:111
28670991311,1cyclictest8810-21ssh10:01:350
28670991311,1cyclictest0-21swapper/007:40:020
28670991310,2cyclictest26731-21head08:50:190
277502130,11sleep627776-21ssh09:36:566
2867799129,2cyclictest5146-21sed09:25:237
2867799129,2cyclictest30622-21sshd08:59:167
2867799129,2cyclictest28650-21tr11:45:097
2867799129,2cyclictest28650-21tr11:45:097
2867799129,2cyclictest26216-21sed07:55:207
2867799129,2cyclictest17026-21ls10:05:237
2867799129,2cyclictest1169-21chrt08:10:057
2867799129,2cyclictest11199-21head08:25:167
28677991210,1cyclictest721ktimersoftd/711:05:217
28677991210,1cyclictest63042sleep707:25:127
28677991210,1cyclictest30590-21sh09:22:217
28677991210,1cyclictest28042-21cstates08:00:147
28677991210,1cyclictest0-21swapper/710:10:107
28677991210,1cyclictest0-21swapper/709:40:177
28677991210,1cyclictest0-21swapper/709:35:017
28677991210,1cyclictest0-21swapper/709:15:377
28677991210,1cyclictest0-21swapper/709:10:337
28677991210,1cyclictest0-21swapper/709:05:157
28677991210,1cyclictest0-21swapper/708:50:397
28677991210,1cyclictest0-21swapper/708:45:177
28677991210,1cyclictest0-21swapper/708:40:327
28677991210,1cyclictest0-21swapper/708:36:457
28677991210,1cyclictest0-21swapper/708:30:027
28677991210,1cyclictest0-21swapper/708:21:157
28677991210,1cyclictest0-21swapper/708:17:017
28677991210,1cyclictest0-21swapper/708:05:047
28677991210,1cyclictest0-21swapper/707:50:127
28677991210,1cyclictest0-21swapper/707:47:057
28677991210,1cyclictest0-21swapper/707:35:427
28677991210,1cyclictest0-21swapper/707:30:567
2867699129,2cyclictest641ktimersoftd/608:20:116
2867699129,2cyclictest6352-21tail08:15:226
2867699129,2cyclictest5435-21ssh09:25:276
2867699129,2cyclictest27606-21ssh10:10:346
2867699129,2cyclictest1721-21wc09:05:126
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional