You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-29 - 13:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Wed Jul 29, 2026 00:57:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1120998071,803cyclictest0-21swapper/521:57:165
11209970568,634cyclictest0-21swapper/522:56:015
11209970568,634cyclictest0-21swapper/522:56:015
111999704194,337cyclictest5819-21kworker/u16:122:37:494
11209969516,676cyclictest0-21swapper/523:08:435
11209967739,507cyclictest0-21swapper/521:47:375
111799675165,338cyclictest121rcu_preempt00:28:292
11159967237,84cyclictest121rcu_preempt00:35:530
11209966974,84cyclictest0-21swapper/500:40:105
11209966931,634cyclictest18409-21fgrep23:40:355
111899669327,126cyclictest0-21swapper/322:36:103
11199966772,338cyclictest0-21swapper/423:40:274
11209966527,635cyclictest0-21swapper/520:18:225
11209966527,635cyclictest0-21swapper/520:18:215
11209966527,635cyclictest0-21swapper/520:18:215
11189965398,381cyclictest0-21swapper/321:50:223
111599652310,126cyclictest0-21swapper/023:08:430
11189965196,127cyclictest0-21swapper/321:49:373
11229964897,336cyclictest6460-21fschecks_count20:10:297
11189964147,127cyclictest8681-21diskmemload23:57:313
1120996402,635cyclictest0-21swapper/500:13:535
111899635208,254cyclictest0-21swapper/300:04:333
111899635208,254cyclictest0-21swapper/300:04:333
111799632290,126cyclictest0-21swapper/223:09:182
111999631123,83cyclictest0-21swapper/423:39:494
111999631123,83cyclictest0-21swapper/423:39:494
111899631290,126cyclictest0-21swapper/300:20:543
111999630120,380cyclictest0-21swapper/421:58:544
11209962934,592cyclictest26624-21ssh22:30:275
11169962731,85cyclictest0-21swapper/123:49:121
111799626115,126cyclictest33-21ksoftirqd/223:47:182
112299624113,295cyclictest0-21swapper/722:13:537
11219962272,296cyclictest0-21swapper/623:32:086
11189962271,127cyclictest0-21swapper/322:25:333
111899621110,380cyclictest0-21swapper/322:56:003
111899621110,380cyclictest0-21swapper/322:56:003
11209961924,592cyclictest0-21swapper/523:13:205
11199961865,296cyclictest0-21swapper/420:05:244
11159961868,127cyclictest0-21swapper/021:57:160
11159961663,84cyclictest131rcu_sched22:50:010
11169961364,338cyclictest1924-21rm21:57:161
111699613318,84cyclictest0-21swapper/120:28:411
11199960957,128cyclictest0-21swapper/422:10:224
11159960897,381cyclictest0-21swapper/022:56:570
11159960897,381cyclictest0-21swapper/022:56:570
11169960615,84cyclictest0-21swapper/123:08:181
11209960554,549cyclictest0-21swapper/521:32:225
11189960512,2cyclictest0-21swapper/322:10:433
11199960411,211cyclictest0-21swapper/422:26:404
1120996016,592cyclictest0-21swapper/523:34:025
11169960111,83cyclictest0-21swapper/123:57:181
111699599347,83cyclictest0-21swapper/100:32:511
112299594125,296cyclictest0-21swapper/720:05:177
1120995921,507cyclictest0-21swapper/500:04:385
1120995921,507cyclictest0-21swapper/500:04:385
11179959279,296cyclictest0-21swapper/220:10:152
111699592124,253cyclictest0-21swapper/122:27:011
11159959183,84cyclictest121rcu_preempt20:17:270
11159959183,84cyclictest121rcu_preempt20:17:270
11159959183,84cyclictest121rcu_preempt20:17:270
11229959081,126cyclictest0-21swapper/700:03:257
11229959081,126cyclictest0-21swapper/700:03:257
111999589120,380cyclictest0-21swapper/400:36:084
11209958834,127cyclictest0-21swapper/522:38:405
112299587121,84cyclictest0-21swapper/723:41:017
11209958633,550cyclictest0-21swapper/522:29:545
11189958534,126cyclictest0-21swapper/322:09:033
11179958577,126cyclictest0-21swapper/223:12:032
111699585285,85cyclictest0-21swapper/120:05:181
111599585116,296cyclictest0-21swapper/023:58:470
11229958473,380cyclictest0-21swapper/720:29:017
11229958434,211cyclictest0-21swapper/700:14:097
11209958478,126cyclictest0-21swapper/522:11:115
11199958334,465cyclictest0-21swapper/423:58:504
111899583119,253cyclictest0-21swapper/320:26:213
11169958274,126cyclictest0-21swapper/120:17:001
11169958274,126cyclictest0-21swapper/120:17:001
11169958274,126cyclictest0-21swapper/120:17:001
111699582325,126cyclictest0-21swapper/121:16:071
11189958132,84cyclictest0-21swapper/320:10:123
11169958130,126cyclictest0-21swapper/123:35:091
11169958130,126cyclictest0-21swapper/123:35:091
11219958071,126cyclictest0-21swapper/622:56:226
11219958071,126cyclictest0-21swapper/622:56:226
111899580114,129cyclictest27725-21/usr/sbin/munin22:50:123
111699580281,126cyclictest24883-21/usr/sbin/munin21:35:341
111599580323,126cyclictest0-21swapper/023:11:230
112199578240,84cyclictest725-21nscd21:34:406
11209957868,507cyclictest0-21swapper/500:38:105
11209957826,549cyclictest0-21swapper/523:35:165
11209957826,549cyclictest0-21swapper/523:35:165
11229957766,85cyclictest0-21swapper/720:15:447
11229957766,85cyclictest0-21swapper/720:15:447
11229957766,85cyclictest0-21swapper/720:15:447
11229957766,126cyclictest0-21swapper/723:23:247
11199957727,42cyclictest0-21swapper/423:47:234
11199957667,296cyclictest0-21swapper/400:05:544
11199957628,82cyclictest0-21swapper/421:34:484
111899576106,296cyclictest0-21swapper/300:28:263
111799576107,296cyclictest0-21swapper/222:12:142
11229957566,127cyclictest0-21swapper/722:10:037
111799575106,296cyclictest0-21swapper/200:20:572
11189957463,338cyclictest0-21swapper/320:39:413
111799574190,253cyclictest0-21swapper/223:57:312
11169957421,338cyclictest0-21swapper/100:27:011
11209957324,383cyclictest0-21swapper/523:48:495
11189957362,296cyclictest0-21swapper/321:33:303
11179957362,380cyclictest187-21md0_raid123:21:452
111599573274,126cyclictest29899-21diskmemload23:46:240
11199957059,254cyclictest0-21swapper/422:58:404
11199957059,254cyclictest0-21swapper/422:58:404
11179956918,126cyclictest0-21swapper/220:35:242
11159956819,5cyclictest8681-21diskmemload00:44:470
11209956715,549cyclictest6137-21taskset00:30:225
111899567102,345cyclictest32539-21kworker/u16:223:22:153
111799567141,295cyclictest1113-21cyclictest21:34:482
111699567101,126cyclictest0-21swapper/123:33:051
111599567141,253cyclictest0-21swapper/023:15:300
111799566138,169cyclictest121rcu_preempt23:31:402
11179956512,380cyclictest121rcu_preempt23:40:072
11169956557,84cyclictest121rcu_preempt22:56:231
11169956557,84cyclictest121rcu_preempt22:56:231
11229956411,380cyclictest0-21swapper/722:30:387
11169956415,84cyclictest121rcu_preempt21:49:011
11159956254,126cyclictest0-21swapper/022:12:010
11219956197,84cyclictest121rcu_preempt23:58:166
1115995608,83cyclictest0-21swapper/023:23:520
112299559263,50cyclictest0-21swapper/723:08:187
11169955911,7cyclictest0-21swapper/123:22:111
112299556257,126cyclictest0-21swapper/722:46:237
11199955690,85cyclictest121rcu_preempt00:10:194
111899554297,84cyclictest0-21swapper/323:41:283
111699554213,253cyclictest25860-21kworker/u16:321:34:511
1117995534,84cyclictest0-21swapper/222:09:102
1116995534,127cyclictest0-21swapper/120:36:201
11199955241,380cyclictest0-21swapper/421:48:244
111699552299,42cyclictest0-21swapper/100:37:341
1115995523,84cyclictest0-21swapper/022:35:540
11209955141,507cyclictest0-21swapper/522:48:425
1119995511,84cyclictest0-21swapper/400:28:264
11179955139,381cyclictest0-21swapper/223:15:082
11159955182,253cyclictest0-21swapper/022:25:350
1115995511,126cyclictest0-21swapper/000:20:280
11179955084,84cyclictest121rcu_preempt19:14:512
11179955037,338cyclictest0-21swapper/222:46:282
11209954939,507cyclictest0-21swapper/520:40:185
112199548124,126cyclictest0-21swapper/621:58:596
11169954884,253cyclictest187-21md0_raid120:40:101
111599548122,295cyclictest0-21swapper/023:33:220
11209954736,508cyclictest0-21swapper/500:28:045
1120995471,204cyclictest0-21swapper/521:35:255
112299546119,296cyclictest0-21swapper/722:50:127
11209954639,381cyclictest22977-21ssh00:20:545
11209954534,465cyclictest0-21swapper/520:10:335
11179954577,465cyclictest0-21swapper/221:48:122
11179954537,304cyclictest0-21swapper/200:35:532
11169954534,84cyclictest0-21swapper/100:44:541
11159954581,253cyclictest0-21swapper/020:10:350
11209954434,422cyclictest0-21swapper/500:09:565
11209954433,508cyclictest0-21swapper/522:09:035
11159954479,211cyclictest0-21swapper/000:28:280
111599544245,126cyclictest0-21swapper/000:04:180
111599544245,126cyclictest0-21swapper/000:04:180
11159954377,126cyclictest0-21swapper/000:14:540
11189954273,296cyclictest0-21swapper/300:10:193
111899542116,254cyclictest0-21swapper/323:08:343
11169954233,84cyclictest0-21swapper/123:13:521
111699542159,211cyclictest0-21swapper/122:11:301
11229954176,127cyclictest0-21swapper/722:39:217
11209954130,508cyclictest0-21swapper/523:24:395
11159954134,84cyclictest0-21swapper/019:58:400
11159954133,84cyclictest0-21swapper/023:40:320
11199954032,126cyclictest0-21swapper/400:40:044
111699540323,126cyclictest9808-21ssh22:20:271
111599540113,211cyclictest0-21swapper/021:49:180
11199953931,84cyclictest0-21swapper/422:00:044
111899539199,338cyclictest16521-1kworker/u17:022:47:323
11179953975,84cyclictest0-21swapper/220:28:532
11169953932,261cyclictest0-21swapper/100:05:081
11219953830,84cyclictest0-21swapper/600:26:006
11169953825,296cyclictest0-21swapper/122:00:101
112299537323,126cyclictest0-21swapper/721:31:507
112299537281,126cyclictest0-21swapper/721:56:027
111799537111,296cyclictest0-21swapper/200:14:532
11199953629,127cyclictest0-21swapper/421:05:174
111699536111,254cyclictest8681-21diskmemload00:01:521
111699536111,254cyclictest8681-21diskmemload00:01:521
11159953625,299cyclictest0-21swapper/022:00:220
111999535108,169cyclictest0-21swapper/400:02:474
111999535108,169cyclictest0-21swapper/400:02:474
11179953566,168cyclictest33-21ksoftirqd/222:35:462
11199953464,337cyclictest22867-21ssh00:20:424
111899534151,380cyclictest0-21swapper/300:39:243
111799534107,255cyclictest0-21swapper/221:55:412
111599534321,126cyclictest0-21swapper/022:20:070
112299533237,84cyclictest0-21swapper/723:34:227
11179953324,84cyclictest0-21swapper/221:05:122
11159953325,127cyclictest0-21swapper/022:08:270
11159953324,126cyclictest0-21swapper/021:33:200
11169953224,84cyclictest0-21swapper/122:46:231
112299529273,126cyclictest0-21swapper/700:29:367
11169952917,127cyclictest0-21swapper/122:09:591
11169952859,211cyclictest25-21ksoftirqd/100:20:541
11159952758,338cyclictest121rcu_preempt20:26:150
11189952619,84cyclictest0-21swapper/323:14:233
11169952620,84cyclictest0-21swapper/122:40:191
11169952618,126cyclictest0-21swapper/123:40:181
11159952517,126cyclictest0-21swapper/021:50:190
11229952397,295cyclictest121rcu_preempt22:28:027
111999523268,253cyclictest64450irq/122-eth023:13:254
11179952357,84cyclictest121rcu_preempt22:56:002
11179952357,84cyclictest121rcu_preempt22:56:002
11179952316,423cyclictest0-21swapper/200:31:032
11229952259,123cyclictest121rcu_preempt22:59:357
11229952259,123cyclictest121rcu_preempt22:59:357
11199952212,211cyclictest1113-21cyclictest20:19:184
11199952212,211cyclictest1113-21cyclictest20:19:184
11199952212,211cyclictest1113-21cyclictest20:19:184
11159952256,211cyclictest0-21swapper/000:30:220
11159952214,84cyclictest0-21swapper/021:16:080
112299521263,126cyclictest0-21swapper/700:39:027
11189951913,253cyclictest0-21swapper/323:25:163
11229951811,2cyclictest0-21swapper/700:31:037
11219951810,128cyclictest121rcu_preempt00:33:446
11189951852,296cyclictest0-21swapper/320:05:183
1117995187,380cyclictest0-21swapper/220:15:322
1117995187,380cyclictest0-21swapper/220:15:322
1117995187,380cyclictest0-21swapper/220:15:322
1122995168,126cyclictest0-21swapper/723:59:587
11199951648,253cyclictest24772-21diskmemload22:48:034
112199515344,83cyclictest0-21swapper/621:05:196
1117995157,126cyclictest0-21swapper/220:05:182
1115995157,126cyclictest0-21swapper/000:05:180
1118995149,168cyclictest0-21swapper/321:05:143
1115995131,380cyclictest0-21swapper/020:37:340
1122995121,253cyclictest0-21swapper/700:40:147
112199512212,126cyclictest0-21swapper/620:17:016
112199512212,126cyclictest0-21swapper/620:17:016
112199512212,126cyclictest0-21swapper/620:17:016
1120995121,296cyclictest0-21swapper/523:59:485
11219951185,253cyclictest0-21swapper/600:04:376
11219951185,253cyclictest0-21swapper/600:04:376
11229950981,211cyclictest73-21ksoftirqd/700:05:207
1119995091,127cyclictest0-21swapper/420:36:254
1116995091,126cyclictest0-21swapper/122:39:391
1115995091,84cyclictest0-21swapper/020:05:180
11199950840,296cyclictest1113-21cyclictest22:08:174
112299507123,211cyclictest0-21swapper/720:35:467
11219950781,126cyclictest0-21swapper/622:10:456
11209950681,295cyclictest1113-21cyclictest20:25:425
111599506124,126cyclictest121rcu_preempt23:35:260
111599506124,126cyclictest121rcu_preempt23:35:260
11189950532,84cyclictest0-21swapper/321:57:163
111799505249,253cyclictest321ktimersoftd/222:00:242
11199950377,254cyclictest121rcu_preempt23:33:074
111999503122,84cyclictest5840-21cpuspeed_turbos20:10:184
11209950235,464cyclictest0-21swapper/522:15:135
11189950232,296cyclictest0-21swapper/323:47:233
11219950178,126cyclictest0-21swapper/623:09:416
112199500328,85cyclictest0-21swapper/623:35:246
112199500328,85cyclictest0-21swapper/623:35:246
11199950076,294cyclictest0-21swapper/423:21:494
11179950035,84cyclictest0-21swapper/222:28:012
112199499158,338cyclictest18609-21ssh00:39:466
11199949976,84cyclictest0-21swapper/400:31:124
11169949935,84cyclictest0-21swapper/121:50:191
112199498157,338cyclictest30315-1kworker/u17:022:36:446
11199949834,42cyclictest0-21swapper/423:07:084
11169949833,84cyclictest0-21swapper/120:20:101
11189949730,127cyclictest0-21swapper/320:16:163
11189949730,127cyclictest0-21swapper/320:16:163
11189949730,127cyclictest0-21swapper/320:16:163
112299496197,211cyclictest0-21swapper/723:35:027
112299496197,211cyclictest0-21swapper/723:35:027
11199949632,168cyclictest0-21swapper/420:30:124
11189949670,168cyclictest27891-21turbostat00:45:003
11189949630,126cyclictest0-21swapper/322:30:173
11179949671,211cyclictest30433-21runrttasks22:15:052
11189949571,126cyclictest1113-21cyclictest23:34:313
111599495152,169cyclictest0-21swapper/023:25:040
11209949430,126cyclictest0-21swapper/520:38:415
11209949426,465cyclictest0-21swapper/521:50:075
11199949423,468cyclictest0-21swapper/420:28:044
11179949425,338cyclictest27727-21ssh22:50:142
11159949426,295cyclictest1113-21cyclictest22:30:180
11209949325,465cyclictest0-21swapper/520:05:215
111999493112,127cyclictest4732-21sh23:15:104
11179949328,211cyclictest0-21swapper/222:30:282
11229949223,84cyclictest121rcu_preempt23:48:487
11219949270,84cyclictest0-21swapper/600:09:476
11179949225,304cyclictest0-21swapper/223:35:162
11179949225,304cyclictest0-21swapper/223:35:162
11209949123,465cyclictest0-21swapper/519:34:165
11179949125,84cyclictest0-21swapper/200:09:562
11169949169,211cyclictest0-21swapper/122:15:051
11169949125,126cyclictest0-21swapper/100:14:521
11169949064,338cyclictest0-21swapper/120:10:191
11189948924,3cyclictest0-21swapper/300:33:553
11179948920,84cyclictest0-21swapper/200:03:152
11179948920,84cyclictest0-21swapper/200:03:152
11179948918,381cyclictest187-21md0_raid122:20:162
11199948823,87cyclictest0-21swapper/400:15:144
11169948866,126cyclictest0-21swapper/121:05:171
111699486102,253cyclictest0-21swapper/122:30:341
11209948517,465cyclictest0-21swapper/521:05:185
11199948563,169cyclictest0-21swapper/422:20:234
111999484103,126cyclictest18670-21rm23:03:024
11159948357,210cyclictest6392-21cat21:05:130
11209948117,383cyclictest0-21swapper/522:40:145
11189947998,84cyclictest721-21nscd22:00:203
11209947810,465cyclictest0-21swapper/522:00:235
11179947711,254cyclictest26616-21ssh21:35:242
11189947552,7cyclictest0-21swapper/300:05:383
11219947410,126cyclictest0-21swapper/623:23:156
1118994716,84cyclictest0-21swapper/319:58:203
112199467125,339cyclictest18182-21ssh23:40:316
112099467168,126cyclictest121rcu_preempt22:20:265
11179946743,216cyclictest0-21swapper/222:40:232
1121994661,84cyclictest0-21swapper/623:48:336
112299465250,84cyclictest0-21swapper/722:20:307
112199465122,127cyclictest65-21ksoftirqd/620:40:186
112199464123,338cyclictest0-21swapper/622:30:266
11199946484,296cyclictest10481-21fschecks_count22:40:174
11169946480,253cyclictest0-21swapper/123:15:151
11189946340,8cyclictest0-21swapper/323:15:063
11169946337,211cyclictest25-21ksoftirqd/100:15:161
11209946235,423cyclictest0-21swapper/523:00:035
111899462204,127cyclictest0-21swapper/300:15:163
112299461120,338cyclictest0-21swapper/723:12:597
112199461119,338cyclictest0-21swapper/622:47:526
11169946134,296cyclictest0-21swapper/123:00:031
112299460120,338cyclictest0-21swapper/722:00:127
112199460117,340cyclictest8681-21diskmemload22:27:436
111999460120,338cyclictest0-21swapper/421:35:234
11179946076,211cyclictest33-21ksoftirqd/223:25:152
112199459117,126cyclictest65-21ksoftirqd/620:35:446
11199945973,338cyclictest0-21swapper/422:15:064
11179945932,338cyclictest0-21swapper/220:30:122
112199457285,84cyclictest1553-21runrttasks23:11:446
112199456115,338cyclictest0-21swapper/620:27:406
11199945634,84cyclictest0-21swapper/423:25:054
111799456373,3cyclictest311rcuc/200:44:392
111799456241,126cyclictest0-21swapper/221:16:072
11229945574,85cyclictest121rcu_preempt00:20:457
11229945432,2cyclictest193162sleep721:49:357
112299454117,253cyclictest0-21swapper/720:40:217
11219945437,36cyclictest121rcu_preempt21:35:076
11219945432,84cyclictest0-21swapper/622:09:326
11209945428,338cyclictest0-21swapper/523:15:105
112199453112,126cyclictest3882-21date19:15:016
111999453112,338cyclictest1703-21sendmail-mta22:30:164
11179945230,256cyclictest0-21swapper/200:15:152
11219945171,84cyclictest0-21swapper/621:48:496
11209945125,423cyclictest0-21swapper/523:50:085
11179945171,295cyclictest0-21swapper/220:40:202
112199450106,296cyclictest0-21swapper/600:44:546
112299449108,338cyclictest0-21swapper/700:15:187
11219944771,121cyclictest26531-21cpu20:45:136
11189944724,219cyclictest0-21swapper/323:50:143
11199944621,422cyclictest0-21swapper/421:50:204
11189944619,84cyclictest0-21swapper/322:15:063
11229944564,84cyclictest0-21swapper/719:41:347
112299444232,42cyclictest0-21swapper/722:40:237
11199944326,287cyclictest0-21swapper/423:50:064
11209944260,127cyclictest57-21ksoftirqd/500:15:165
11209944226,163cyclictest22204-21ssh23:25:145
11209944216,423cyclictest0-21swapper/519:58:365
11189944220,84cyclictest0-21swapper/321:35:293
11179944216,253cyclictest11616-21runrttasks20:20:082
11219944161,84cyclictest0-21swapper/623:15:306
11219944115,126cyclictest0-21swapper/620:05:306
11209944119,2cyclictest0-21swapper/519:14:485
11189944119,84cyclictest0-21swapper/322:40:033
11189944059,84cyclictest9556-21missed_timers22:20:233
11219943955,211cyclictest65-21ksoftirqd/623:25:046
11199943917,253cyclictest0-21swapper/419:51:294
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional