You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-23 - 11:18
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Tue Jun 23, 2026 00:54:28)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
105729979169,719cyclictest0-21swapper/523:21:075
1057499717375,126cyclictest0-21swapper/700:06:227
105709968690,380cyclictest0-21swapper/300:26:183
1057299677124,381cyclictest0-21swapper/523:12:095
1057399662108,381cyclictest0-21swapper/600:08:536
105729965964,381cyclictest0-21swapper/523:04:505
1057299654100,381cyclictest0-21swapper/523:46:485
105679963380,84cyclictest9-21ksoftirqd/023:03:200
1056899632164,126cyclictest0-21swapper/123:24:481
1057299627116,295cyclictest14916-21ssh00:26:135
105729962272,126cyclictest0-21swapper/523:50:065
105689962128,127cyclictest0-21swapper/123:58:581
105699961522,126cyclictest0-21swapper/223:04:492
105729961317,507cyclictest0-21swapper/521:49:555
105709960650,381cyclictest0-21swapper/323:03:483
105679959946,126cyclictest0-21swapper/022:53:530
1056999592166,295cyclictest0-21swapper/222:53:592
105689958839,127cyclictest0-21swapper/121:48:311
105739958635,126cyclictest121rcu_preempt23:47:376
105729958431,423cyclictest0-21swapper/500:19:055
105729958431,423cyclictest0-21swapper/500:19:045
105709958274,126cyclictest0-21swapper/323:49:493
105729958129,168cyclictest0-21swapper/523:38:095
105729958129,168cyclictest0-21swapper/523:38:095
1057099580112,211cyclictest41-21ksoftirqd/323:39:443
1057099580112,211cyclictest41-21ksoftirqd/323:39:443
105719957727,126cyclictest121rcu_preempt23:57:214
105689957728,127cyclictest0-21swapper/123:15:081
1057299576111,296cyclictest121rcu_preempt23:58:325
105719957324,84cyclictest18634-21diskmemload00:06:224
1057499566186,256cyclictest10746-1kworker/u17:221:48:317
105719956553,380cyclictest0-21swapper/423:48:554
1057399562517,42cyclictest631rcuc/623:03:206
105679956011,85cyclictest0-21swapper/023:14:540
105689955951,127cyclictest0-21swapper/122:50:451
10567995554,84cyclictest0-21swapper/000:26:130
105729954841,165cyclictest0-21swapper/522:54:305
1056899547164,126cyclictest22577-21sh23:12:161
1056999545118,296cyclictest0-21swapper/223:58:382
1056999541115,257cyclictest0-21swapper/223:37:332
1056999541115,257cyclictest0-21swapper/223:37:332
105699954029,380cyclictest0-21swapper/200:16:402
105699954029,380cyclictest0-21swapper/200:16:402
105719953629,84cyclictest0-21swapper/421:47:454
105679953469,168cyclictest0-21swapper/023:46:490
105709953122,84cyclictest0-21swapper/323:12:103
105729953020,507cyclictest0-21swapper/500:10:105
105719952816,296cyclictest0-21swapper/422:52:354
105689952720,42cyclictest0-21swapper/100:30:031
105719952618,84cyclictest0-21swapper/400:26:174
1057199525187,83cyclictest64450irq/122-eth023:37:034
1057199525187,83cyclictest64450irq/122-eth023:37:034
105709952315,126cyclictest0-21swapper/321:48:283
1056999519176,254cyclictest0-21swapper/223:46:482
105699951810,84cyclictest0-21swapper/223:12:142
10567995143,84cyclictest0-21swapper/021:48:040
10567995121,169cyclictest0-21swapper/023:24:290
105709950943,128cyclictest0-21swapper/323:59:533
1057199507167,337cyclictest10565-21cyclictest00:19:494
1057199507167,337cyclictest10565-21cyclictest00:19:494
10571995060,295cyclictest0-21swapper/423:10:504
105689950641,344cyclictest0-21swapper/123:03:111
105679950582,169cyclictest0-21swapper/023:39:460
105679950582,169cyclictest0-21swapper/023:39:460
105709950481,84cyclictest229802sleep322:53:213
105699950178,126cyclictest0-21swapper/200:08:232
105689950137,380cyclictest0-21swapper/123:48:591
1056799500417,2cyclictest101ktimersoftd/000:06:110
105739949834,84cyclictest0-21swapper/623:23:356
105699949833,93cyclictest0-21swapper/200:26:152
105679949829,338cyclictest0-21swapper/000:18:560
105679949829,338cyclictest0-21swapper/000:18:560
105739949670,211cyclictest0-21swapper/623:37:396
105739949670,211cyclictest0-21swapper/623:37:386
105739949629,211cyclictest0-21swapper/622:55:096
105739949629,211cyclictest0-21swapper/622:55:086
105689949572,126cyclictest1651-21ntpd00:17:401
105689949572,126cyclictest1651-21ntpd00:17:391
105709949427,126cyclictest7352-21ssh23:23:293
105689949327,84cyclictest0-21swapper/123:39:471
105689949327,84cyclictest0-21swapper/123:39:471
105719949125,84cyclictest0-21swapper/423:03:484
1056799491447,42cyclictest111rcuc/023:58:310
105749949067,84cyclictest0-21swapper/723:05:007
105699949068,84cyclictest0-21swapper/221:48:172
105709948920,466cyclictest0-21swapper/323:15:213
105739948823,299cyclictest0-21swapper/600:26:466
105699948763,127cyclictest10565-21cyclictest23:50:042
105689948620,126cyclictest0-21swapper/100:06:101
1056799485187,126cyclictest121rcu_preempt23:15:250
105749948318,50cyclictest0-21swapper/723:49:327
105699948197,295cyclictest0-21swapper/223:21:192
1057499479100,295cyclictest0-21swapper/722:53:207
10572994756,211cyclictest0-21swapper/500:09:065
10571994728,295cyclictest49-21ksoftirqd/423:23:224
10568994717,42cyclictest0-21swapper/100:30:091
10574994671,84cyclictest0-21swapper/723:22:007
1057099467125,211cyclictest0-21swapper/323:25:063
105699946743,84cyclictest121rcu_preempt00:40:272
10573994662,89cyclictest0-21swapper/600:19:596
10573994662,89cyclictest0-21swapper/600:19:596
10568994651,41cyclictest0-21swapper/119:15:121
105709946481,380cyclictest16626-21ssh00:08:523
1057199463207,253cyclictest481ktimersoftd/422:55:084
1057199463207,253cyclictest481ktimersoftd/422:55:084
105689946278,168cyclictest0-21swapper/100:10:071
1057499461162,126cyclictest121rcu_preempt23:39:417
1057499461162,126cyclictest121rcu_preempt23:39:417
105729946138,303cyclictest0-21swapper/523:15:115
1057299459122,85cyclictest121rcu_preempt00:40:285
1057099459120,84cyclictest0-21swapper/300:16:223
1057099459120,84cyclictest0-21swapper/300:16:223
105749945629,211cyclictest0-21swapper/723:58:447
105729945631,422cyclictest0-21swapper/500:00:065
105739945570,211cyclictest0-21swapper/622:51:426
105749945427,212cyclictest21748-21ssh23:11:237
105729945028,256cyclictest31030-21latency21:50:255
105739944867,84cyclictest0-21swapper/600:40:216
1057399448149,126cyclictest121rcu_preempt23:14:446
105729944763,381cyclictest0-21swapper/519:15:105
105699944725,256cyclictest0-21swapper/223:15:082
105689944523,256cyclictest0-21swapper/100:00:041
105679944314,84cyclictest1180-21ssh00:00:070
105749944261,83cyclictest0-21swapper/719:15:107
105689944119,2cyclictest0-21swapper/123:05:031
1057199440144,125cyclictest121rcu_preempt21:50:154
105679943960,3cyclictest1651-21ntpd23:05:030
105729943614,343cyclictest0-21swapper/500:30:085
105679943552,211cyclictest0-21swapper/021:50:250
10573994253,2cyclictest0-21swapper/623:56:246
105739942384,84cyclictest0-21swapper/623:50:076
10570994231,255cyclictest0-21swapper/300:00:053
105749942281,338cyclictest0-21swapper/700:19:307
105749942281,338cyclictest0-21swapper/700:19:297
1057499422254,42cyclictest0-21swapper/700:26:187
105709942281,338cyclictest0-21swapper/321:50:163
105739942182,84cyclictest0-21swapper/621:48:246
105729942080,338cyclictest0-21swapper/523:40:085
105709941977,339cyclictest0-21swapper/300:30:063
105689941979,338cyclictest0-21swapper/100:40:271
105749941776,338cyclictest0-21swapper/723:15:067
105709941636,296cyclictest0-21swapper/323:05:033
105679941675,338cyclictest0-21swapper/000:10:050
105719941577,84cyclictest0-21swapper/423:15:054
105689941534,126cyclictest0-21swapper/123:25:071
105739941475,84cyclictest0-21swapper/600:10:046
105739941273,84cyclictest0-21swapper/623:15:066
105729941132,295cyclictest0-21swapper/522:55:085
105729941132,295cyclictest0-21swapper/522:55:085
105699941172,84cyclictest0-21swapper/219:15:112
105719941071,84cyclictest64450irq/122-eth000:10:044
105739940928,211cyclictest0-21swapper/623:05:036
105709940828,297cyclictest0-21swapper/300:40:213
105719940769,84cyclictest0-21swapper/400:20:094
105719940732,202cyclictest0-21swapper/400:00:084
105689940767,338cyclictest0-21swapper/123:40:091
105679940564,338cyclictest0-21swapper/019:15:100
105719940425,41cyclictest0-21swapper/419:15:114
105719940327,84cyclictest0-21swapper/400:30:084
105709940368,246cyclictest0-21swapper/323:50:043
105709940059,338cyclictest0-21swapper/319:15:113
105709940058,339cyclictest0-21swapper/323:40:093
105749939919,42cyclictest0-21swapper/700:20:087
105739939919,2cyclictest0-21swapper/621:50:176
105749939818,85cyclictest0-21swapper/700:10:127
105709939817,297cyclictest24871-21ssh22:55:083
105709939817,297cyclictest24871-21ssh22:55:083
105689939819,42cyclictest0-21swapper/100:20:081
1057299397101,84cyclictest0-21swapper/500:20:095
105679939514,84cyclictest0-21swapper/000:20:080
1057299394309,3cyclictest551rcuc/523:25:025
105699939313,7cyclictest0-21swapper/223:25:012
105689939150,126cyclictest0-21swapper/123:50:051
105679938546,336cyclictest0-21swapper/023:50:100
105749938243,83cyclictest0-21swapper/700:30:077
10574993821,256cyclictest0-21swapper/723:05:037
1056999382167,127cyclictest0-21swapper/221:50:172
105689938282,126cyclictest121rcu_preempt21:50:221
105719937882,84cyclictest7520-21ssh00:40:204
1057499377297,77cyclictest711rcuc/723:50:047
105709937634,339cyclictest0-21swapper/300:10:103
105719937576,126cyclictest49-21ksoftirqd/423:25:064
105739937378,211cyclictest0-21swapper/623:25:036
105679937029,338cyclictest0-21swapper/023:40:080
105749936930,84cyclictest0-21swapper/721:50:197
105699936930,84cyclictest0-21swapper/223:40:092
1056799369113,126cyclictest9-21ksoftirqd/000:30:070
1057399368110,85cyclictest121rcu_preempt00:00:086
105729936830,253cyclictest0-21swapper/523:05:035
105699936770,84cyclictest0-21swapper/200:20:082
105739936526,84cyclictest0-21swapper/600:20:086
105749936421,85cyclictest10565-21cyclictest23:25:087
1056799361147,126cyclictest121rcu_preempt23:25:070
105749936023,253cyclictest0-21swapper/700:00:077
105749936021,84cyclictest0-21swapper/723:40:097
105679936062,131cyclictest8048-21rm00:40:270
105699935917,127cyclictest0-21swapper/200:00:052
105719935761,83cyclictest892-21kworker/u16:223:50:104
105719935416,216cyclictest0-21swapper/423:05:044
105699935415,126cyclictest18041-21ssh00:10:062
105739935322,205cyclictest0-21swapper/623:40:086
10573993448,82cyclictest121rcu_preempt00:30:096
10574993423,84cyclictest0-21swapper/722:55:097
10574993423,84cyclictest0-21swapper/722:55:087
10574993401,84cyclictest0-21swapper/700:40:277
105709933334,296cyclictest0-21swapper/300:20:093
105739932832,42cyclictest0-21swapper/619:15:116
105699931822,126cyclictest33-21ksoftirqd/223:05:032
105699931620,171cyclictest18634-21diskmemload00:30:062
105689931419,42cyclictest0-21swapper/122:55:081
105689931419,42cyclictest0-21swapper/122:55:081
10567992982,84cyclictest0-21swapper/022:55:090
10567992982,84cyclictest0-21swapper/022:55:090
10569992961,211cyclictest0-21swapper/222:55:082
10569992961,211cyclictest0-21swapper/222:55:072
105719927622,84cyclictest0-21swapper/423:40:084
8744210398,2sleep50-21swapper/519:05:185
1010429264,24sleep40-21swapper/419:09:344
989428657,26sleep30-21swapper/319:06:353
991728253,26sleep60-21swapper/619:06:546
998728050,27sleep10-21swapper/119:07:531
988927950,26sleep70-21swapper/719:06:317
984527949,27sleep20-21swapper/219:05:542
301212780,0sleep70-21swapper/700:35:247
982527748,26sleep00-21swapper/019:05:370
219412720,0sleep20-21swapper/219:25:162
87242710,0sleep40-21swapper/422:12:124
311782690,0sleep30-21swapper/322:07:223
1057399599,37cyclictest0-21swapper/622:16:266
10573995717,26cyclictest3544-21ssh22:42:286
1057399556,35cyclictest0-21swapper/619:55:266
1057399544,33cyclictest0-21swapper/621:12:456
137512530,3sleep013757-21cpuspeed_turbos22:15:140
1057399537,31cyclictest0-21swapper/622:30:106
10573995316,22cyclictest11151-21ssh21:39:256
10573995314,27cyclictest20213-21ssh21:26:436
1057399526,32cyclictest0-21swapper/622:28:156
1057399526,32cyclictest0-21swapper/622:28:156
1057399521,44cyclictest0-21swapper/620:38:336
10573995114,24cyclictest22281-21ssh22:02:436
10570994821,24cyclictest11807-21date22:30:223
10570994711,31cyclictest10654-21ssh21:39:003
1057399466,33cyclictest0-21swapper/620:16:346
10570994617,25cyclictest22556-21sh22:02:583
10570994521,20cyclictest24347-21ssh21:12:183
10570994520,21cyclictest16298-21ssh22:15:383
10570994519,22cyclictest6423-21chrt19:55:143
10570994420,20cyclictest204502sleep320:16:523
10570994416,25cyclictest17069-21cut21:25:153
10570994412,27cyclictest13300-21sed22:15:003
10570994310,28cyclictest18634-21diskmemload22:43:553
1057399402,26cyclictest0-21swapper/620:58:466
1057399402,26cyclictest0-21swapper/620:58:456
10571993812,17cyclictest0-21swapper/421:39:284
1057399372,22cyclictest0-21swapper/622:15:006
1057199376,24cyclictest0-21swapper/421:26:384
10571993712,15cyclictest0-21swapper/422:16:364
10571993711,18cyclictest0-21swapper/422:44:054
1056999379,14cyclictest33-21ksoftirqd/222:30:402
1056999378,18cyclictest33-21ksoftirqd/222:15:212
1056999377,19cyclictest33-21ksoftirqd/221:26:472
1057199365,24cyclictest49-21ksoftirqd/421:12:144
10570993620,12cyclictest327482sleep320:38:083
1056999369,17cyclictest33-21ksoftirqd/221:39:192
1056999365,19cyclictest33-21ksoftirqd/222:30:022
1056999365,19cyclictest33-21ksoftirqd/222:30:022
1057199358,17cyclictest49-21ksoftirqd/422:03:414
10571993513,14cyclictest0-21swapper/420:39:174
1056999356,16cyclictest33-21ksoftirqd/222:03:172
1056999352,20cyclictest0-21swapper/220:16:302
1056999346,18cyclictest33-21ksoftirqd/222:42:332
1056999344,20cyclictest33-21ksoftirqd/221:11:292
10568993424,6cyclictest16036-21grep22:15:301
10573993315,14cyclictest0-21swapper/621:25:006
1057199338,18cyclictest49-21ksoftirqd/421:00:014
1057199338,18cyclictest49-21ksoftirqd/421:00:014
1056999336,15cyclictest33-21ksoftirqd/220:39:012
1056999335,18cyclictest0-21swapper/219:55:192
10568993323,6cyclictest6092-21grep19:55:121
10574993224,5cyclictest11083-21ssh21:39:227
1057199328,14cyclictest49-21ksoftirqd/422:28:524
1057199328,14cyclictest49-21ksoftirqd/422:28:514
1057099321,4cyclictest0-21swapper/322:29:163
1057099321,4cyclictest0-21swapper/322:29:163
1056999326,18cyclictest33-21ksoftirqd/221:00:012
1056999326,18cyclictest33-21ksoftirqd/221:00:012
1056999324,18cyclictest33-21ksoftirqd/222:15:002
10567993224,5cyclictest9193-21ssh21:37:410
10567993224,5cyclictest4115-21ssh22:42:570
10574993124,4cyclictest5369-21ssh22:44:057
10574993124,4cyclictest23834-21ssh21:11:527
1057499311,26cyclictest0-21swapper/721:25:237
10572993124,4cyclictest7711-21ssh22:28:195
10572993124,4cyclictest7711-21ssh22:28:195
1057199317,15cyclictest49-21ksoftirqd/419:55:154
1057199316,13cyclictest49-21ksoftirqd/422:30:194
10568993123,5cyclictest10045-21ssh21:38:271
10568993122,6cyclictest22330-21rm22:02:451
10567993124,4cyclictest8661-21ssh22:29:110
10567993124,4cyclictest8661-21ssh22:29:110
10567993124,4cyclictest20263-21ssh21:26:460
10574993023,4cyclictest7600-21ssh22:28:147
10574993023,4cyclictest7600-21ssh22:28:147
10574993022,5cyclictest15423-21grep22:15:267
10574993022,5cyclictest12434-21cron21:00:007
10574993022,5cyclictest12434-21cron21:00:007
10574993022,5cyclictest10506-21expr22:30:147
10572993024,4cyclictest20006-21ssh21:26:315
10572993023,4cyclictest25133-21ssh21:13:015
10572993023,4cyclictest15434-21head22:15:265
10572993023,4cyclictest12977-21pluginstate22:15:005
10572993022,5cyclictest5452-21sh22:44:105
10568993023,4cyclictest19847-21ssh21:26:231
10568993022,5cyclictest23249-21sh21:11:211
10568993021,6cyclictest8441-21sh22:29:001
10568993021,6cyclictest8441-21sh22:28:591
10568993021,5cyclictest10422-21grep22:30:141
10567993023,4cyclictest25086-21ssh21:12:590
10567993022,5cyclictest7264-21expr19:55:190
10567993022,5cyclictest21963-21rm22:02:270
10574992922,4cyclictest22208-21sh22:02:397
1057399299,17cyclictest0-21swapper/621:00:026
10572992922,4cyclictest12700-21grep21:00:025
10572992922,4cyclictest12700-21grep21:00:025
10572992922,4cyclictest10601-21cat22:30:135
10568992922,4cyclictest3856-21taskset22:42:441
10568992922,4cyclictest20443-21taskset20:16:461
10568992922,4cyclictest12337-21seq20:58:401
10568992922,4cyclictest12337-21seq20:58:401
10573992810,18cyclictest0-21swapper/619:12:216
10572992822,4cyclictest10926-21ssh21:39:135
10572992821,4cyclictest22845-21taskset22:03:145
1057299281,24cyclictest5910-21taskset19:55:035
10570992821,4cyclictest12605-21cut21:00:023
10570992821,4cyclictest12605-21cut21:00:013
10569992810,18cyclictest0-21swapper/219:12:282
10568992821,3cyclictest32766-21taskset20:38:221
10567992822,4cyclictest10689-21expr22:30:150
10567992822,3cyclictest13242-21sh22:15:010
10567992821,4cyclictest16006-21cut21:25:000
10567992821,4cyclictest12286-21taskset20:57:570
10567992821,4cyclictest12286-21taskset20:57:560
306852270,0sleep10-21swapper/122:23:471
10574992710,0cyclictest0-21swapper/719:12:197
10571992710,17cyclictest0-21swapper/419:12:294
10570992710,17cyclictest0-21swapper/319:12:233
10567992710,0cyclictest0-21swapper/019:12:230
10574992619,4cyclictest13147-21grep22:15:017
10572992626,0cyclictest0-21swapper/519:12:215
10572992620,3cyclictest15951-21sed21:25:005
10569992620,4cyclictest15957-21ssh21:25:012
10568992626,0cyclictest0-21swapper/119:12:321
10568992618,5cyclictest13244-21ssh22:15:011
10572992520,3cyclictest321-21chrt20:38:435
10572992519,4cyclictest20518-21taskset20:17:475
10570992518,4cyclictest15976-21cat21:25:013
10574992419,3cyclictest204562sleep720:16:577
10574992418,4cyclictest32743-21chrt20:38:047
10574992416,5cyclictest6930-21tail19:55:177
1057499241,1cyclictest0-21swapper/720:25:127
1057199241,16cyclictest49-21ksoftirqd/420:18:154
10569992410,4cyclictest121rcu_preempt21:00:032
10567992419,3cyclictest3362sleep020:38:530
10573992313,5cyclictest65-21ksoftirqd/600:35:246
10573992312,7cyclictest65-21ksoftirqd/619:25:116
10573992312,6cyclictest65-21ksoftirqd/622:20:216
10573992311,7cyclictest65-21ksoftirqd/622:35:126
10573992311,7cyclictest65-21ksoftirqd/622:05:236
62372220,1sleep70-21swapper/721:20:117
202362220,3sleep01056799cyclictest20:15:210
10573992213,5cyclictest65-21ksoftirqd/620:45:236
10573992212,6cyclictest65-21ksoftirqd/621:43:576
10573992212,6cyclictest65-21ksoftirqd/620:30:236
10573992212,5cyclictest65-21ksoftirqd/622:48:286
10573992212,5cyclictest65-21ksoftirqd/620:05:176
10573992212,5cyclictest121rcu_preempt21:55:216
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional