You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-25 - 05:44
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Sat Apr 25, 2026 00:53:22)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19001997642,211cyclictest0-21swapper/721:30:027
189969968084,127cyclictest121rcu_preempt22:56:582
190019967536,550cyclictest121rcu_preempt21:55:047
189979964143,296cyclictest0-21swapper/321:54:233
1899599621583,36cyclictest231rcuc/122:34:091
190009959482,296cyclictest0-21swapper/623:29:206
18999995940,84cyclictest0-21swapper/522:33:445
18998995940,84cyclictest64450irq/122-eth022:33:444
189979959339,339cyclictest41-21ksoftirqd/323:39:253
189979959339,339cyclictest41-21ksoftirqd/323:39:253
189989958675,296cyclictest0-21swapper/400:09:464
189989958675,296cyclictest0-21swapper/400:09:464
189989958133,254cyclictest0-21swapper/420:44:384
1899699581153,296cyclictest0-21swapper/222:35:032
190019958027,127cyclictest0-21swapper/700:09:057
190019958027,127cyclictest0-21swapper/700:09:057
190019957930,465cyclictest0-21swapper/721:27:547
190019957929,468cyclictest0-21swapper/721:54:527
189999957929,380cyclictest0-21swapper/500:09:175
189999957929,380cyclictest0-21swapper/500:09:175
18998995577,126cyclictest0-21swapper/422:35:164
18998995577,126cyclictest0-21swapper/422:35:154
190019955340,380cyclictest0-21swapper/720:43:197
190019955283,466cyclictest0-21swapper/723:19:377
19001995501,465cyclictest0-21swapper/722:32:037
1899699544122,84cyclictest0-21swapper/223:29:112
189999954333,84cyclictest121rcu_preempt22:10:155
189989954332,508cyclictest0-21swapper/423:29:174
189979954132,126cyclictest0-21swapper/322:32:123
189969954031,129cyclictest0-21swapper/200:18:002
189969954031,129cyclictest0-21swapper/200:18:002
190019953929,126cyclictest0-21swapper/700:18:357
190019953929,126cyclictest0-21swapper/700:18:357
190009953871,126cyclictest0-21swapper/600:18:006
190009953871,126cyclictest0-21swapper/600:18:006
1899499538153,126cyclictest121rcu_preempt00:29:590
189959952660,84cyclictest24509-1kworker/u17:221:29:451
189959952516,127cyclictest0-21swapper/121:55:041
189959952315,86cyclictest0-21swapper/123:29:581
189959952214,84cyclictest0-21swapper/122:35:151
189959952214,84cyclictest0-21swapper/122:35:141
1899899518181,253cyclictest19181-21sh21:54:234
18995995165,508cyclictest21907-21munin-node20:43:101
18999995090,84cyclictest0-21swapper/523:39:365
18999995090,84cyclictest0-21swapper/523:39:365
18998995090,84cyclictest0-21swapper/423:39:374
18998995090,84cyclictest0-21swapper/423:39:374
1899699506207,126cyclictest121rcu_preempt21:55:092
1899699506122,381cyclictest0-21swapper/220:43:092
1899799505127,82cyclictest131rcu_sched23:57:143
1899799505127,82cyclictest131rcu_sched23:57:143
1899499505421,3cyclictest101ktimersoftd/023:29:220
190019950480,126cyclictest0-21swapper/723:29:567
1899899504378,45cyclictest471rcuc/421:55:114
1899499504162,128cyclictest26393-21kworker/0:021:28:030
189979950380,257cyclictest21403-21idleruntime21:55:163
1899799502118,381cyclictest0-21swapper/320:43:093
189969950279,84cyclictest0-21swapper/223:39:022
189969950279,84cyclictest0-21swapper/223:39:022
1899599502249,85cyclictest18149-21sshd21:53:241
1899599502247,253cyclictest241ktimersoftd/123:09:261
1899599502247,253cyclictest241ktimersoftd/123:09:261
189949950234,337cyclictest0-21swapper/000:09:070
189949950234,337cyclictest0-21swapper/000:09:070
190019950176,84cyclictest121rcu_preempt23:56:347
190019950176,84cyclictest121rcu_preempt23:56:347
189979950175,211cyclictest0-21swapper/300:10:293
189969950179,84cyclictest0-21swapper/200:29:052
189989950036,296cyclictest0-21swapper/423:56:384
189989950036,296cyclictest0-21swapper/423:56:384
189979949728,338cyclictest0-21swapper/321:28:243
1899799497114,253cyclictest0-21swapper/323:29:593
1899499497162,124cyclictest12512-21ssh22:55:420
190009949631,177cyclictest0-21swapper/600:08:096
190009949631,177cyclictest0-21swapper/600:08:096
189989949669,338cyclictest0-21swapper/421:28:174
190019949527,465cyclictest0-21swapper/723:09:007
190019949527,465cyclictest0-21swapper/723:09:007
1900199495113,84cyclictest0-21swapper/700:29:447
190009949165,168cyclictest28738-21ssh23:39:386
190009949165,168cyclictest28738-21ssh23:39:376
190009948925,296cyclictest0-21swapper/621:28:446
189999948820,295cyclictest0-21swapper/500:28:565
190019948718,466cyclictest0-21swapper/719:25:147
189979948519,296cyclictest0-21swapper/300:09:363
189979948519,296cyclictest0-21swapper/300:09:363
189949948558,338cyclictest9-21ksoftirqd/000:18:290
189949948558,338cyclictest9-21ksoftirqd/000:18:290
190019948212,169cyclictest28853-21sh23:39:457
190019948212,169cyclictest28853-21sh23:39:457
1899499482227,253cyclictest101ktimersoftd/022:09:270
189959948115,211cyclictest0-21swapper/122:09:281
189949947813,84cyclictest26799-21diskmemload22:34:210
189969947711,126cyclictest121rcu_preempt23:56:312
189969947711,126cyclictest121rcu_preempt23:56:312
189959947711,211cyclictest0-21swapper/122:25:121
18995994767,338cyclictest0-21swapper/100:17:551
18995994767,338cyclictest0-21swapper/100:17:551
190009947490,126cyclictest22638-21ssh23:19:126
1899999473131,126cyclictest121rcu_preempt21:30:015
189979947147,127cyclictest0-21swapper/322:09:543
189969946782,296cyclictest0-21swapper/223:08:512
189969946782,296cyclictest0-21swapper/223:08:502
1899899466124,169cyclictest0-21swapper/422:24:374
1900199465125,338cyclictest0-21swapper/722:56:557
1899799465125,338cyclictest0-21swapper/322:35:173
1899799465125,338cyclictest0-21swapper/322:35:163
1899799465124,339cyclictest0-21swapper/300:17:543
1899799465124,339cyclictest0-21swapper/300:17:543
190019946483,84cyclictest0-21swapper/722:35:157
190019946483,84cyclictest0-21swapper/722:35:147
1899899463122,338cyclictest0-21swapper/419:23:534
189959946278,211cyclictest25-21ksoftirqd/120:45:121
1899499462122,338cyclictest0-21swapper/021:55:140
190009946180,84cyclictest0-21swapper/622:09:276
1899999461121,338cyclictest0-21swapper/522:09:485
1899899461120,338cyclictest64450irq/122-eth022:55:454
1899799461121,338cyclictest0-21swapper/322:56:313
1900199460122,335cyclictest0-21swapper/722:10:157
190009946033,296cyclictest0-21swapper/621:53:246
190019945934,423cyclictest0-21swapper/722:09:337
189999945938,255cyclictest0-21swapper/521:55:175
1899499459287,84cyclictest0-21swapper/020:44:210
190019945772,382cyclictest0-21swapper/722:24:097
1899899457115,126cyclictest49-21ksoftirqd/400:17:544
1899899457115,126cyclictest49-21ksoftirqd/400:17:544
190019945634,338cyclictest721ktimersoftd/700:10:117
1899699456115,168cyclictest0-21swapper/222:25:122
189979945529,126cyclictest0-21swapper/323:08:503
189979945529,126cyclictest0-21swapper/323:08:503
190019945428,423cyclictest0-21swapper/720:45:127
190019945368,382cyclictest0-21swapper/719:24:147
1899999453114,336cyclictest0-21swapper/522:56:075
189979945331,7cyclictest0-21swapper/322:24:333
1899799453113,338cyclictest0-21swapper/323:19:313
189969945326,211cyclictest0-21swapper/221:27:392
189949945373,296cyclictest0-21swapper/023:19:170
189969945271,84cyclictest0-21swapper/221:30:022
1900099451110,126cyclictest20191-21ssh21:55:086
189969945127,169cyclictest0-21swapper/200:08:542
189969945127,169cyclictest0-21swapper/200:08:542
1899599451368,3cyclictest231rcuc/121:30:021
190009945027,211cyclictest0-21swapper/600:29:476
1899799450110,338cyclictest0-21swapper/300:29:493
1899699450109,338cyclictest2552-21fschecks_time22:35:162
1899699450109,338cyclictest2552-21fschecks_time22:35:162
189949945067,380cyclictest0-21swapper/021:30:020
190009944926,84cyclictest0-21swapper/619:23:426
189949944922,296cyclictest0-21swapper/021:53:570
189969944864,211cyclictest0-21swapper/221:54:342
190009944766,126cyclictest0-21swapper/622:33:236
1899799446103,253cyclictest18576-21forks22:10:133
189999944460,211cyclictest57-21ksoftirqd/523:55:575
189999944460,211cyclictest57-21ksoftirqd/523:55:575
189959944363,296cyclictest0-21swapper/123:56:451
189959944363,296cyclictest0-21swapper/123:56:451
189989944162,84cyclictest64450irq/122-eth022:09:534
1899499440357,2cyclictest111rcuc/023:57:010
1899499440357,2cyclictest111rcuc/023:57:010
190009943794,295cyclictest0-21swapper/620:43:426
1899999437139,295cyclictest25598-21nscd23:29:185
189979943653,380cyclictest0-21swapper/319:24:133
18995994369,211cyclictest25-21ksoftirqd/100:10:021
189999943392,338cyclictest0-21swapper/522:35:155
189999943392,338cyclictest0-21swapper/522:35:155
19001994326,211cyclictest0-21swapper/722:25:097
189949943151,296cyclictest0-21swapper/022:25:070
189949943147,211cyclictest0-21swapper/022:10:130
189999942741,341cyclictest121rcu_preempt00:10:165
18999994251,340cyclictest0-21swapper/521:28:395
189959942581,338cyclictest0-21swapper/100:28:551
189999942382,338cyclictest0-21swapper/519:23:275
189959942383,338cyclictest0-21swapper/122:10:231
189949942383,338cyclictest0-21swapper/023:39:000
189949942383,338cyclictest0-21swapper/023:39:000
189949942383,338cyclictest0-21swapper/022:24:540
190009942284,172cyclictest0-21swapper/623:55:526
190009942284,172cyclictest0-21swapper/623:55:526
189999942281,338cyclictest4696-21diskmemload00:18:245
189999942281,338cyclictest4696-21diskmemload00:18:245
189949942181,338cyclictest0-21swapper/023:08:520
189949942181,338cyclictest0-21swapper/023:08:520
189999942079,338cyclictest0-21swapper/521:53:585
189959942079,211cyclictest25-21ksoftirqd/122:56:381
1900099418374,41cyclictest631rcuc/623:08:516
1900099418374,41cyclictest631rcuc/623:08:506
189989941880,83cyclictest0-21swapper/422:10:224
189989941879,84cyclictest0-21swapper/422:25:124
189969941879,84cyclictest0-21swapper/222:09:182
189969941778,84cyclictest0-21swapper/222:10:132
189969941678,84cyclictest0-21swapper/222:24:112
189969941677,84cyclictest0-21swapper/223:19:162
189959941675,338cyclictest0-21swapper/100:08:531
189959941675,338cyclictest0-21swapper/100:08:531
190009941534,43cyclictest0-21swapper/622:25:076
189999941574,338cyclictest0-21swapper/520:43:245
189989941576,84cyclictest0-21swapper/423:19:174
189959941574,338cyclictest0-21swapper/123:19:141
189949941574,338cyclictest0-21swapper/000:10:110
189949941475,336cyclictest0-21swapper/019:23:540
190009941374,84cyclictest0-21swapper/622:10:216
189999941372,338cyclictest0-21swapper/523:19:175
1899999413118,126cyclictest0-21swapper/523:09:235
1899999413118,126cyclictest0-21swapper/523:09:235
189989941372,84cyclictest0-21swapper/419:25:094
189989941329,381cyclictest0-21swapper/400:28:574
189989941228,260cyclictest0-21swapper/400:10:034
189979941272,338cyclictest0-21swapper/322:25:123
189959941272,338cyclictest0-21swapper/122:24:531
1899599410367,41cyclictest231rcuc/119:25:161
1899599410112,126cyclictest121rcu_preempt19:23:261
189949941029,84cyclictest0-21swapper/020:45:130
189959940969,338cyclictest0-21swapper/123:39:321
189959940969,338cyclictest0-21swapper/123:39:311
190009940869,84cyclictest0-21swapper/622:55:446
190009940725,85cyclictest2540-21fschecks_time22:35:156
190009940725,85cyclictest2540-21fschecks_time22:35:146
189999940766,338cyclictest0-21swapper/522:24:365
190009940525,296cyclictest0-21swapper/622:24:356
1899999405148,126cyclictest121rcu_preempt19:25:155
189969940466,42cyclictest0-21swapper/200:10:062
189999940222,296cyclictest0-21swapper/522:25:085
189969940162,84cyclictest0-21swapper/219:23:412
1900099397141,253cyclictest641ktimersoftd/621:30:026
190009939354,84cyclictest0-21swapper/600:10:106
189949939314,126cyclictest9-21ksoftirqd/019:25:180
189989937878,127cyclictest49-21ksoftirqd/423:09:264
189989937878,127cyclictest49-21ksoftirqd/423:09:264
189949937837,172cyclictest0-21swapper/022:35:080
189949937837,172cyclictest0-21swapper/022:35:080
189979937133,253cyclictest0-21swapper/319:25:143
189989936830,83cyclictest0-21swapper/421:30:024
190009936325,169cyclictest0-21swapper/619:25:186
189979935817,338cyclictest0-21swapper/320:45:133
189969935555,127cyclictest12235-21taskset20:45:122
189989935093,84cyclictest49-21ksoftirqd/420:45:134
190009934891,254cyclictest0-21swapper/620:45:046
189979933982,254cyclictest2074-21apt-get21:30:043
189969933278,84cyclictest0-21swapper/219:25:012
189999932525,126cyclictest0-21swapper/520:45:055
1842629453,38sleep50-21swapper/519:08:015
1829328756,27sleep60-21swapper/619:06:096
1829128554,28sleep40-21swapper/419:06:084
1596228251,27sleep30-21swapper/319:05:033
1832528151,27sleep70-21swapper/719:06:347
1831728049,28sleep10-21swapper/119:06:281
1852227948,27sleep00-21swapper/019:09:230
1559527946,29sleep20-21swapper/219:05:012
178912690,0sleep50-21swapper/500:41:585
293752580,2sleep60-21swapper/600:31:276
293752580,2sleep60-21swapper/600:31:266
106092580,1sleep40-21swapper/400:38:544
1900099553,39cyclictest0-21swapper/623:40:116
1899799551,51cyclictest0-21swapper/323:10:053
18998995445,5cyclictest6970-21ssh00:20:134
18997995223,24cyclictest25804-21ntpq23:20:253
1900099503,33cyclictest0-21swapper/623:00:476
1900099483,32cyclictest0-21swapper/600:21:056
1900099481,32cyclictest0-21swapper/623:30:506
1900099478,26cyclictest27934-21ssh23:21:406
1900099465,30cyclictest0-21swapper/623:10:066
19000994619,17cyclictest65-21ksoftirqd/600:00:176
18998994639,4cyclictest25664-21rm23:20:234
19001994436,5cyclictest11771-21grep23:30:167
243242430,0sleep40-21swapper/421:24:204
1899899430,25cyclictest0-21swapper/423:30:044
18999994234,5cyclictest561ktimersoftd/500:30:065
18999994234,5cyclictest561ktimersoftd/500:30:065
1899899410,27cyclictest0-21swapper/400:00:064
18995994111,26cyclictest4586-21sh23:10:041
18996994012,16cyclictest33-21ksoftirqd/200:00:282
1899599402,34cyclictest0-21swapper/123:30:131
18994994033,4cyclictest101ktimersoftd/000:30:110
18994994033,4cyclictest101ktimersoftd/000:30:110
1900199391,7cyclictest24258-21tr23:20:147
18998993917,18cyclictest18141-21rm23:00:144
18997993932,4cyclictest12054-21uname23:30:183
1899599396,30cyclictest24748-21df_abs23:20:161
18999993833,2cyclictest10568-21ssh23:30:035
1899999382,32cyclictest0-21swapper/523:40:075
1899499382,32cyclictest0-21swapper/023:30:120
1899699372,3cyclictest8200-21sh00:20:192
1899699371,32cyclictest0-21swapper/223:30:042
18996993711,15cyclictest33-21ksoftirqd/223:00:172
1899499372,4cyclictest0-21swapper/023:40:070
1899499372,31cyclictest0-21swapper/023:20:120
1900199361,32cyclictest0-21swapper/723:10:087
18999993629,4cyclictest8023-21smartctl00:20:185
1899899361,31cyclictest0-21swapper/400:30:024
1899899361,31cyclictest0-21swapper/400:30:024
1899799362,30cyclictest0-21swapper/300:20:103
1899699365,21cyclictest33-21ksoftirqd/223:20:132
18994993630,3cyclictest4556-21ssh23:10:020
1900199351,2cyclictest0-21swapper/700:20:117
1899999351,2cyclictest0-21swapper/523:20:045
18998993518,12cyclictest31892-21ssh23:40:294
1899699352,3cyclictest0-21swapper/223:40:062
18995993530,2cyclictest29650-21ssh23:40:091
1899599352,3cyclictest0-21swapper/100:20:071
18998993413,17cyclictest6152-21latency23:10:184
1899699341,2cyclictest0-21swapper/200:30:042
1899699341,2cyclictest0-21swapper/200:30:032
18997993325,4cyclictest32003-21ls23:40:303
18995993326,4cyclictest27322-21wc00:30:211
18995993326,4cyclictest27322-21wc00:30:211
18999993222,6cyclictest18080-21grep23:00:145
18996993210,13cyclictest33-21ksoftirqd/223:10:092
18995993224,5cyclictest3619-21cat00:00:171
1899499321,28cyclictest0-21swapper/000:20:020
19001993124,4cyclictest28478-21ssh00:30:367
19001993124,4cyclictest28478-21ssh00:30:367
18997993126,5cyclictest0-21swapper/320:24:493
18997993124,4cyclictest26573-21ssh00:30:163
18997993124,4cyclictest26573-21ssh00:30:163
18994993124,4cyclictest17937-21tr23:00:130
159672310,1sleep10-21swapper/100:40:421
19001993023,4cyclictest4413-21sort00:00:277
19001993023,4cyclictest18041-21tr23:00:137
1899999303,13cyclictest121rcu_preempt00:00:265
18999993023,4cyclictest6640-21ssh23:10:235
18997993027,3cyclictest0-21swapper/321:24:423
18997993025,4cyclictest0-21swapper/320:16:073
18997993022,5cyclictest3147-21cut00:00:153
18994993022,5cyclictest2658-21expr00:00:120
7872291,23sleep70-21swapper/723:41:247
18997992922,4cyclictest19206-21sed23:00:193
18997992910,18cyclictest0-21swapper/319:14:173
18995992922,4cyclictest18208-21expr23:00:141
19001992828,0cyclictest0-21swapper/719:13:567
19001992827,1cyclictest0-21swapper/721:46:187
19001992827,1cyclictest0-21swapper/721:46:177
19001992827,1cyclictest0-21swapper/720:56:247
19001992827,1cyclictest0-21swapper/720:24:327
19001992810,0cyclictest0-21swapper/721:24:277
19000992810,0cyclictest0-21swapper/621:24:166
18999992810,0cyclictest0-21swapper/521:24:445
18997992810,0cyclictest0-21swapper/319:36:153
18996992828,0cyclictest0-21swapper/219:14:262
18996992810,0cyclictest0-21swapper/221:14:082
18995992810,0cyclictest0-21swapper/121:46:341
18995992810,0cyclictest0-21swapper/121:46:341
18994992810,0cyclictest0-21swapper/019:14:270
19001992727,0cyclictest0-21swapper/721:04:267
19001992727,0cyclictest0-21swapper/720:39:597
19001992727,0cyclictest0-21swapper/720:16:167
19001992727,0cyclictest0-21swapper/719:51:527
19001992710,17cyclictest0-21swapper/721:13:517
19001992710,17cyclictest0-21swapper/721:05:057
19001992710,17cyclictest0-21swapper/720:31:417
19001992710,17cyclictest0-21swapper/720:07:427
19001992710,17cyclictest0-21swapper/719:59:177
19001992710,17cyclictest0-21swapper/719:42:477
19001992710,17cyclictest0-21swapper/719:35:477
19000992727,0cyclictest0-21swapper/621:13:426
19000992727,0cyclictest0-21swapper/621:04:496
19000992710,17cyclictest0-21swapper/620:31:376
19000992710,17cyclictest0-21swapper/620:24:076
19000992710,17cyclictest0-21swapper/619:59:066
19000992710,17cyclictest0-21swapper/619:42:546
19000992710,17cyclictest0-21swapper/619:35:436
19000992710,0cyclictest0-21swapper/621:45:516
19000992710,0cyclictest0-21swapper/621:45:506
19000992710,0cyclictest0-21swapper/621:05:056
19000992710,0cyclictest0-21swapper/620:56:296
19000992710,0cyclictest0-21swapper/619:51:436
19000992710,0cyclictest0-21swapper/619:14:236
18999992727,0cyclictest0-21swapper/519:43:165
18999992710,17cyclictest0-21swapper/521:46:015
18999992710,17cyclictest0-21swapper/521:46:005
18999992710,17cyclictest0-21swapper/521:04:575
18999992710,17cyclictest0-21swapper/520:31:445
18999992710,17cyclictest0-21swapper/520:24:255
18999992710,17cyclictest0-21swapper/520:16:185
18999992710,17cyclictest0-21swapper/519:51:485
18999992710,17cyclictest0-21swapper/519:35:475
18999992710,0cyclictest0-21swapper/521:13:405
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional