You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-14 - 03:38
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Thu May 14, 2026 00:58:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
137629979472,719cyclictest0-21swapper/100:39:581
1376499764124,381cyclictest0-21swapper/323:31:383
1376499715117,381cyclictest0-21swapper/300:12:263
137639965320,7cyclictest9367-21ssh23:23:482
137639965320,7cyclictest9367-21ssh23:23:472
137639965320,7cyclictest9367-21ssh23:23:472
137619965059,48cyclictest121rcu_preempt00:04:540
137619965059,48cyclictest121rcu_preempt00:04:540
137659963483,85cyclictest121rcu_preempt23:31:384
137639962835,423cyclictest0-21swapper/223:10:012
1376199626540,84cyclictest111rcuc/023:05:340
137659962472,126cyclictest386-21diskstats23:40:174
137629961571,541cyclictest0-21swapper/123:37:031
137669960614,2cyclictest0-21swapper/523:03:155
13763996007,338cyclictest0-21swapper/222:39:082
13762995940,42cyclictest0-21swapper/122:47:541
13762995940,42cyclictest0-21swapper/100:10:571
13763995930,84cyclictest0-21swapper/200:04:212
13763995930,84cyclictest0-21swapper/200:04:212
137619959281,338cyclictest121rcu_preempt22:48:140
137649959079,508cyclictest0-21swapper/322:10:583
137649959079,508cyclictest0-21swapper/322:10:583
137649959079,508cyclictest0-21swapper/322:10:583
13762995870,77cyclictest2888-21perf21:50:021
13762995840,75cyclictest0-21swapper/122:09:491
137659958233,342cyclictest0-21swapper/400:36:484
137619958029,84cyclictest0-21swapper/023:24:250
137619958029,84cyclictest0-21swapper/023:24:240
137619958029,84cyclictest0-21swapper/023:24:240
137629957869,506cyclictest0-21swapper/100:26:461
1376599574194,3cyclictest64450irq/122-eth021:15:394
137679957324,383cyclictest0-21swapper/600:39:596
137629956712,465cyclictest0-21swapper/123:24:051
137629956712,465cyclictest0-21swapper/123:24:051
137629956712,465cyclictest0-21swapper/123:24:051
1376199567357,4cyclictest21783-21diskmemload23:55:140
13764995659,126cyclictest0-21swapper/323:06:483
137669956414,304cyclictest0-21swapper/500:27:485
1376699560218,339cyclictest21783-21diskmemload22:29:395
137639955546,126cyclictest0-21swapper/222:48:412
137639955384,338cyclictest32075-21/usr/sbin/munin23:40:192
137649955241,465cyclictest0-21swapper/323:23:253
137649955241,465cyclictest0-21swapper/323:23:243
137649955241,465cyclictest0-21swapper/323:23:243
13762995520,41cyclictest0-21swapper/122:50:211
13764995501,384cyclictest0-21swapper/322:56:053
13762995501,383cyclictest0-21swapper/122:56:061
137689954941,126cyclictest0-21swapper/700:14:427
137629954939,507cyclictest0-21swapper/122:13:451
137629954939,507cyclictest0-21swapper/122:13:441
137629954939,507cyclictest0-21swapper/122:13:441
137679954840,127cyclictest0-21swapper/622:48:426
137649954841,2cyclictest0-21swapper/323:54:323
137629954837,380cyclictest32079-21cstates21:15:161
1376899547374,84cyclictest0-21swapper/722:12:577
1376899547374,84cyclictest0-21swapper/722:12:577
1376899547374,84cyclictest0-21swapper/722:12:577
137619954734,381cyclictest0-21swapper/022:08:460
137619954712,369cyclictest0-21swapper/022:56:050
137659954677,466cyclictest0-21swapper/423:25:154
137659954677,466cyclictest0-21swapper/423:25:154
137619954579,211cyclictest0-21swapper/000:13:020
1376799544118,423cyclictest0-21swapper/623:10:036
1376399544287,126cyclictest33-21ksoftirqd/223:35:202
137689954377,126cyclictest0-21swapper/723:31:387
137679954172,211cyclictest0-21swapper/623:05:436
137679954134,7cyclictest121rcu_preempt22:38:326
137629954134,387cyclictest0-21swapper/100:04:071
137629954134,387cyclictest0-21swapper/100:04:071
1376199538197,338cyclictest19261-1kworker/u17:123:26:490
1376199538197,338cyclictest19261-1kworker/u17:123:26:490
137619953735,3cyclictest0-21swapper/022:37:400
137659953669,126cyclictest16691-21ssh22:15:114
1376599536113,84cyclictest121rcu_preempt23:23:314
1376599536113,84cyclictest121rcu_preempt23:23:304
1376599536113,84cyclictest121rcu_preempt23:23:304
137629953532,3cyclictest0-21swapper/100:05:081
137629953526,469cyclictest0-21swapper/123:25:151
137629953526,469cyclictest0-21swapper/123:25:151
137659953425,507cyclictest0-21swapper/422:10:594
137659953425,507cyclictest0-21swapper/422:10:584
137659953425,507cyclictest0-21swapper/422:10:584
137619953124,296cyclictest0-21swapper/022:11:560
137619953124,296cyclictest0-21swapper/022:11:550
137619953124,296cyclictest0-21swapper/022:11:550
137629952818,507cyclictest0-21swapper/123:10:141
137619952720,341cyclictest0-21swapper/023:01:060
137669952618,338cyclictest0-21swapper/500:14:175
137629952616,507cyclictest0-21swapper/122:40:051
137619952316,86cyclictest20558-21taskset00:27:560
1376899520182,172cyclictest721ktimersoftd/700:02:437
1376899520182,172cyclictest721ktimersoftd/700:02:437
1376599520183,255cyclictest64450irq/122-eth023:01:054
13762995209,338cyclictest0-21swapper/122:30:181
137629951913,427cyclictest0-21swapper/122:28:351
137689951811,85cyclictest0-21swapper/723:40:157
13763995169,211cyclictest0-21swapper/200:37:502
137639951595,82cyclictest0-21swapper/223:53:202
1376499512172,338cyclictest13759-21cyclictest23:04:573
137639951083,338cyclictest0-21swapper/200:14:582
137689950982,211cyclictest73-21ksoftirqd/723:03:367
13766995091,126cyclictest0-21swapper/523:40:165
137669950887,84cyclictest0-21swapper/521:15:295
13764995081,256cyclictest0-21swapper/322:08:453
137659950740,298cyclictest0-21swapper/422:08:224
137639950780,84cyclictest0-21swapper/221:13:232
1376799506119,339cyclictest0-21swapper/622:09:296
137649950582,420cyclictest0-21swapper/300:03:333
137649950582,420cyclictest0-21swapper/300:03:333
137659950482,256cyclictest13759-21cyclictest23:06:024
137679950334,211cyclictest0-21swapper/600:29:576
137669950376,261cyclictest0-21swapper/500:02:285
137669950376,261cyclictest0-21swapper/500:02:285
137659950339,296cyclictest0-21swapper/421:49:024
137639950381,168cyclictest33-21ksoftirqd/223:55:092
137689950278,168cyclictest73-21ksoftirqd/723:37:097
137679950279,84cyclictest0-21swapper/623:03:326
137679950234,211cyclictest23663-21ssh00:30:166
137629950234,465cyclictest0-21swapper/123:09:541
1376399501122,295cyclictest0-21swapper/223:04:082
1376199501290,43cyclictest101ktimersoftd/021:15:310
1376899500159,338cyclictest28633-1kworker/u17:222:39:087
137659950075,422cyclictest0-21swapper/422:46:444
1376499500115,126cyclictest41-21ksoftirqd/323:40:193
1376799499115,381cyclictest0-21swapper/621:15:376
1376399499284,84cyclictest0-21swapper/220:09:432
137669949876,256cyclictest4217-21ssh21:50:245
1376499498157,339cyclictest30558-21nscd21:15:373
137629949831,85cyclictest0-21swapper/123:40:191
1376899497198,126cyclictest121rcu_preempt23:06:457
137659949732,338cyclictest0-21swapper/422:28:544
137659949728,338cyclictest0-21swapper/400:13:024
1376399497117,377cyclictest0-21swapper/222:56:282
1376799496110,341cyclictest0-21swapper/600:03:506
1376799496110,341cyclictest0-21swapper/600:03:506
137669949569,211cyclictest57-21ksoftirqd/522:12:175
137669949569,211cyclictest57-21ksoftirqd/522:12:175
137669949569,211cyclictest57-21ksoftirqd/522:12:175
137659949380,329cyclictest0-21swapper/421:13:224
137619949326,211cyclictest8904-21ssh00:20:430
137659949126,262cyclictest0-21swapper/420:08:544
137629949122,466cyclictest0-21swapper/123:52:081
137619949123,257cyclictest0-21swapper/023:31:450
137689949026,2cyclictest0-21swapper/723:10:097
137679948967,256cyclictest24464-21ssh23:33:466
137649948923,126cyclictest0-21swapper/321:50:233
1376499489148,339cyclictest13759-21cyclictest00:27:253
1376699487188,84cyclictest121rcu_preempt22:47:325
1376699487106,84cyclictest13759-21cyclictest21:50:015
137659948721,84cyclictest0-21swapper/400:02:564
137659948721,84cyclictest0-21swapper/400:02:564
137629948761,423cyclictest0-21swapper/120:09:211
137619948761,211cyclictest3283-21ssh21:50:050
1376799486230,254cyclictest641ktimersoftd/600:05:066
137629948517,87cyclictest0-21swapper/123:32:031
137619948520,88cyclictest121rcu_preempt00:39:090
137669948420,262cyclictest0-21swapper/523:32:395
137639948420,176cyclictest0-21swapper/223:32:592
137629948458,211cyclictest0-21swapper/123:01:331
137679948313,169cyclictest30924-21chrt21:14:596
1376699483142,126cyclictest121rcu_preempt22:15:095
137619948318,84cyclictest0-21swapper/023:35:260
137659948218,296cyclictest20646-21sh00:28:054
1376499481139,126cyclictest121rcu_preempt00:37:453
137659948016,2cyclictest0-21swapper/422:39:084
137689947910,338cyclictest73-21ksoftirqd/722:28:057
137639947915,210cyclictest0-21swapper/200:27:022
137669947814,42cyclictest0-21swapper/523:38:445
137669947813,2cyclictest0-21swapper/520:07:045
137629947713,91cyclictest0-21swapper/121:14:081
13765994768,295cyclictest0-21swapper/423:38:114
13764994756,338cyclictest0-21swapper/323:38:593
137669947451,126cyclictest0-21swapper/523:08:135
137669947448,211cyclictest0-21swapper/521:14:535
1376399466125,338cyclictest0-21swapper/222:40:082
1376799465123,339cyclictest26347-21diskmemload00:12:556
1376399465124,338cyclictest0-21swapper/223:10:042
1376399465124,338cyclictest0-21swapper/222:29:092
1376399465122,340cyclictest0-21swapper/222:13:522
1376399465122,340cyclictest0-21swapper/222:13:512
1376399465122,340cyclictest0-21swapper/222:13:512
137649946441,257cyclictest0-21swapper/322:47:293
137639946438,210cyclictest321ktimersoftd/223:25:282
137639946438,210cyclictest321ktimersoftd/223:25:282
137689946340,3cyclictest0-21swapper/700:26:447
1376899463123,338cyclictest0-21swapper/723:24:227
1376899463123,338cyclictest0-21swapper/723:24:217
1376899463123,338cyclictest0-21swapper/723:24:217
1376699463380,3cyclictest551rcuc/523:52:265
1376399463122,338cyclictest0-21swapper/221:15:372
137619946341,256cyclictest0-21swapper/022:28:230
137689946240,3cyclictest0-21swapper/723:25:527
137689946240,3cyclictest0-21swapper/723:25:517
137689946240,3cyclictest0-21swapper/722:09:037
137689946240,2cyclictest0-21swapper/721:47:317
1376799462120,339cyclictest0-21swapper/622:40:146
1376799462120,126cyclictest65-21ksoftirqd/623:25:136
1376799462120,126cyclictest65-21ksoftirqd/623:25:136
137649946240,338cyclictest0-21swapper/323:25:323
137649946240,338cyclictest0-21swapper/323:25:323
1376499462124,84cyclictest0-21swapper/322:28:063
137629946278,168cyclictest121rcu_preempt00:40:171
13762994620,2cyclictest0-21swapper/121:50:251
137619946240,255cyclictest26216-21ssh23:54:260
1376799461120,338cyclictest0-21swapper/622:29:516
137649946138,256cyclictest0-21swapper/320:06:433
1376499461119,339cyclictest0-21swapper/300:15:043
1376399461120,339cyclictest0-21swapper/222:30:092
137629946134,169cyclictest25-21ksoftirqd/120:10:041
137689946037,3cyclictest0-21swapper/722:15:117
137689946037,3cyclictest0-21swapper/720:09:127
1376799460121,336cyclictest0-21swapper/621:50:086
1376799460118,340cyclictest0-21swapper/623:53:336
137649946037,256cyclictest0-21swapper/322:30:093
1376899459202,126cyclictest0-21swapper/723:52:417
1376799459118,338cyclictest0-21swapper/623:39:396
137649945977,126cyclictest13759-21cyclictest23:55:053
1376399459121,171cyclictest0-21swapper/200:20:412
137659945874,254cyclictest121rcu_preempt00:20:334
137689945734,2cyclictest0-21swapper/722:47:197
1376799457116,338cyclictest0-21swapper/622:13:526
1376799457116,338cyclictest0-21swapper/622:13:516
1376799457116,338cyclictest0-21swapper/622:13:516
1376699457120,170cyclictest0-21swapper/522:09:405
137659945736,2cyclictest0-21swapper/422:56:434
137659945735,338cyclictest0-21swapper/422:30:144
1376699456119,170cyclictest0-21swapper/500:20:415
137659945633,256cyclictest0-21swapper/421:50:244
1376499456115,339cyclictest0-21swapper/322:37:033
1376199456157,84cyclictest121rcu_preempt20:07:450
1376799455114,338cyclictest0-21swapper/623:23:146
1376799455114,338cyclictest0-21swapper/623:23:136
1376799455114,338cyclictest0-21swapper/623:23:136
137669945572,253cyclictest0-21swapper/523:23:585
137669945572,253cyclictest0-21swapper/523:23:575
137669945572,253cyclictest0-21swapper/523:23:575
137669945528,258cyclictest0-21swapper/500:05:055
137649945533,2cyclictest0-21swapper/321:49:133
1376899454115,336cyclictest0-21swapper/721:15:197
1376799454113,338cyclictest0-21swapper/622:30:096
1376399453112,126cyclictest33-21ksoftirqd/221:48:232
137689945228,126cyclictest0-21swapper/722:30:187
137669945230,256cyclictest0-21swapper/523:25:575
137669945230,256cyclictest0-21swapper/523:25:575
1376599452112,337cyclictest0-21swapper/400:30:164
1376499452112,338cyclictest15140-21sh22:50:093
137639945170,84cyclictest0-21swapper/222:08:142
1376699450109,126cyclictest27006-21seq22:36:335
137659945028,338cyclictest0-21swapper/400:40:214
1376499450110,338cyclictest0-21swapper/300:20:503
137619945028,256cyclictest0-21swapper/021:14:180
1376799448110,335cyclictest0-21swapper/622:56:276
137639944826,3cyclictest7006-21ssh00:40:052
137639944824,257cyclictest0-21swapper/221:50:242
137619944826,256cyclictest0-21swapper/022:30:080
1376599447193,84cyclictest32224-1kworker/u16:323:53:494
137639944766,84cyclictest0-21swapper/222:50:082
1376299447193,122cyclictest121rcu_preempt00:30:161
137689944622,211cyclictest0-21swapper/700:05:077
1376799444103,338cyclictest0-21swapper/621:48:306
1376799444103,338cyclictest0-21swapper/600:40:166
137669944316,295cyclictest0-21swapper/500:37:465
1376299442102,338cyclictest0-21swapper/100:20:531
137689944119,3cyclictest0-21swapper/700:38:237
1376499438270,48cyclictest30095-21ssh21:14:073
137649943711,126cyclictest0-21swapper/323:10:083
137649943696,338cyclictest0-21swapper/322:40:133
137689943594,338cyclictest16454-21if_eth022:50:157
13763994286,85cyclictest0-21swapper/200:15:052
13767994275,2cyclictest0-21swapper/620:09:366
137689942382,338cyclictest0-21swapper/700:20:507
137659942382,338cyclictest0-21swapper/400:05:064
137619942382,338cyclictest0-21swapper/023:40:060
137619942382,338cyclictest0-21swapper/022:40:090
1376199423123,84cyclictest121rcu_preempt20:10:100
137649942283,84cyclictest0-21swapper/300:30:163
137649942282,338cyclictest0-21swapper/322:15:103
137649942282,338cyclictest0-21swapper/300:40:073
137629942281,338cyclictest0-21swapper/123:55:041
137629942241,301cyclictest16634-21sh22:15:061
137679942081,84cyclictest0-21swapper/623:55:116
137679942081,84cyclictest0-21swapper/623:40:096
137679941980,84cyclictest0-21swapper/600:21:136
137619941978,338cyclictest0-21swapper/021:48:050
137659941878,84cyclictest0-21swapper/422:50:154
137659941839,295cyclictest0-21swapper/420:10:104
137619941881,126cyclictest187-21md0_raid100:30:170
137689941736,3cyclictest0-21swapper/723:55:137
137669941778,84cyclictest0-21swapper/523:10:035
137669941778,84cyclictest0-21swapper/522:50:175
137649941737,295cyclictest0-21swapper/300:05:073
137619941776,338cyclictest0-21swapper/000:05:070
137669941635,256cyclictest0-21swapper/500:30:165
137659941677,84cyclictest0-21swapper/422:40:144
137649941574,338cyclictest0-21swapper/320:10:033
137669941333,253cyclictest0-21swapper/520:10:065
137689941071,84cyclictest0-21swapper/721:13:487
137669941071,84cyclictest0-21swapper/522:56:275
137619941030,296cyclictest0-21swapper/000:40:050
137659940929,296cyclictest0-21swapper/423:10:074
137689940828,84cyclictest0-21swapper/700:15:047
137689940826,3cyclictest0-21swapper/700:40:067
137629940867,338cyclictest0-21swapper/100:15:031
137659940767,338cyclictest0-21swapper/423:55:054
137689940666,84cyclictest0-21swapper/722:40:087
1376699406111,171cyclictest13759-21cyclictest22:30:165
137669940564,211cyclictest0-21swapper/500:40:165
137629940565,338cyclictest0-21swapper/122:40:141
137679939920,85cyclictest0-21swapper/622:50:156
137689939657,85cyclictest0-21swapper/721:50:227
137669939616,211cyclictest0-21swapper/522:40:145
137619939655,338cyclictest0-21swapper/000:15:030
137619939352,338cyclictest0-21swapper/023:10:110
137679938849,84cyclictest0-21swapper/600:15:036
137639938546,84cyclictest0-21swapper/220:10:102
13763993844,217cyclictest0-21swapper/200:30:152
137689938181,127cyclictest73-21ksoftirqd/722:56:167
137679938182,127cyclictest16692-21apt22:15:116
137639937940,83cyclictest0-21swapper/200:05:072
137619937980,211cyclictest0-21swapper/022:15:090
137689937738,84cyclictest0-21swapper/720:10:037
137669937379,84cyclictest0-21swapper/523:55:035
137689937031,84cyclictest0-21swapper/700:30:157
137619936729,84cyclictest0-21swapper/022:50:150
137679936362,253cyclictest0-21swapper/620:10:106
1376399361108,171cyclictest0-21swapper/222:15:122
137669932631,175cyclictest0-21swapper/500:15:045
137659932025,211cyclictest0-21swapper/400:15:024
11333210195,3sleep10-21swapper/119:05:121
1320028756,27sleep00-21swapper/019:08:080
1305228755,28sleep30-21swapper/319:06:043
1307328655,27sleep60-21swapper/619:06:226
1322928453,27sleep20-21swapper/219:08:352
1318928453,27sleep70-21swapper/719:07:597
1310028251,27sleep40-21swapper/419:06:444
1328928149,28sleep50-21swapper/519:09:255
13765993210,22cyclictest0-21swapper/419:13:404
13765993210,21cyclictest0-21swapper/419:53:334
13765993210,0cyclictest0-21swapper/420:26:024
13764993226,5cyclictest0-21swapper/321:41:123
13765993126,4cyclictest0-21swapper/421:41:154
13765993110,20cyclictest0-21swapper/419:43:524
13764993126,5cyclictest0-21swapper/320:26:073
13764993110,21cyclictest0-21swapper/319:13:423
13765993024,5cyclictest0-21swapper/421:03:474
13767992810,18cyclictest0-21swapper/620:38:416
13765992810,18cyclictest0-21swapper/420:53:034
13764992828,0cyclictest0-21swapper/321:03:573
13764992810,18cyclictest0-21swapper/320:52:303
13764992810,0cyclictest0-21swapper/320:38:083
13764992810,0cyclictest0-21swapper/319:23:053
13763992810,18cyclictest0-21swapper/219:22:582
13761992810,18cyclictest0-21swapper/020:26:300
13761992810,18cyclictest0-21swapper/019:53:340
13768992727,0cyclictest0-21swapper/719:22:367
13768992727,0cyclictest0-21swapper/719:14:037
13768992710,17cyclictest0-21swapper/721:40:547
13768992710,17cyclictest0-21swapper/720:52:557
13768992710,17cyclictest0-21swapper/720:38:037
13768992710,17cyclictest0-21swapper/720:26:257
13768992710,17cyclictest0-21swapper/719:53:267
13768992710,17cyclictest0-21swapper/719:43:227
13768992710,17cyclictest0-21swapper/719:31:437
13768992710,0cyclictest0-21swapper/721:03:237
13767992727,0cyclictest0-21swapper/620:53:046
13767992710,17cyclictest0-21swapper/621:40:586
13767992710,17cyclictest0-21swapper/620:26:366
13767992710,17cyclictest0-21swapper/619:53:526
13767992710,17cyclictest0-21swapper/619:32:226
13767992710,17cyclictest0-21swapper/619:23:146
13767992710,17cyclictest0-21swapper/619:13:466
13766992727,0cyclictest0-21swapper/519:32:055
13766992710,17cyclictest0-21swapper/521:41:015
13766992710,17cyclictest0-21swapper/520:52:295
13766992710,17cyclictest0-21swapper/520:38:065
13766992710,17cyclictest0-21swapper/519:53:335
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional