You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-20 - 05:10
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Sat Jun 20, 2026 00:57:13)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22016998041,510cyclictest0-21swapper/023:27:220
22017997951,3cyclictest0-21swapper/123:27:221
22018997676,2cyclictest0-21swapper/223:27:222
2201799746111,168cyclictest0-21swapper/100:17:401
2201699719125,591cyclictest9335-1kworker/u17:000:20:020
220209970627,84cyclictest0-21swapper/400:37:104
220239970568,634cyclictest0-21swapper/700:17:307
220169970425,676cyclictest0-21swapper/022:36:440
220219969115,340cyclictest24173-21ssh23:27:225
22021996869,126cyclictest0-21swapper/521:53:575
22023996791,342cyclictest0-21swapper/723:27:227
22016996780,42cyclictest0-21swapper/022:58:010
22016996780,42cyclictest0-21swapper/022:58:010
22016996780,42cyclictest0-21swapper/022:58:000
220169966974,592cyclictest0-21swapper/021:53:260
220169965964,592cyclictest11459-21runrttasks21:33:300
220229965719,423cyclictest7970-21ssh00:39:116
220179964149,126cyclictest0-21swapper/100:09:101
22016996340,84cyclictest9833-21perf23:20:010
2201799628118,295cyclictest0-21swapper/122:46:521
220239962772,381cyclictest0-21swapper/723:16:477
2202299627120,341cyclictest0-21swapper/623:27:226
220219962470,338cyclictest0-21swapper/523:42:575
2201999623116,126cyclictest0-21swapper/300:39:353
220199962267,296cyclictest0-21swapper/323:30:133
2201999621114,341cyclictest0-21swapper/323:27:213
2201899621115,84cyclictest0-21swapper/222:36:352
2201799619108,338cyclictest0-21swapper/122:56:481
2201799619108,338cyclictest0-21swapper/122:56:481
2201799619108,338cyclictest0-21swapper/122:56:471
220169961521,591cyclictest0-21swapper/022:47:270
220169960758,88cyclictest121rcu_preempt00:39:000
220199960312,126cyclictest0-21swapper/300:06:313
220239959182,338cyclictest0-21swapper/722:56:107
220239959182,338cyclictest0-21swapper/722:56:097
220239959182,338cyclictest0-21swapper/722:56:097
2202299588330,126cyclictest0-21swapper/622:56:316
2202299588330,126cyclictest0-21swapper/622:56:316
2202299588330,126cyclictest0-21swapper/622:56:306
220189958876,338cyclictest0-21swapper/223:19:112
220169958877,127cyclictest0-21swapper/000:07:130
220169958781,422cyclictest30079-21diskmemload23:41:160
220229958332,126cyclictest121rcu_preempt22:15:156
220229958030,126cyclictest0-21swapper/623:59:246
220219958027,84cyclictest0-21swapper/500:09:105
220189958031,126cyclictest0-21swapper/223:43:062
220179958027,126cyclictest0-21swapper/123:18:341
2202199578109,338cyclictest0-21swapper/500:18:265
220209957668,126cyclictest0-21swapper/423:43:054
220199957465,126cyclictest121rcu_preempt23:18:363
220229957122,168cyclictest0-21swapper/600:17:056
220189957121,84cyclictest0-21swapper/221:51:432
2201899569103,127cyclictest0-21swapper/200:37:002
220169956916,338cyclictest0-21swapper/000:20:510
220229956818,126cyclictest0-21swapper/623:16:476
2201799567268,126cyclictest0-21swapper/123:59:241
2201899566186,84cyclictest121rcu_preempt22:57:422
2201899566186,84cyclictest121rcu_preempt22:57:422
2201899566186,84cyclictest121rcu_preempt22:57:412
2201699566518,42cyclictest101ktimersoftd/021:55:050
220179956499,346cyclictest0-21swapper/121:52:201
2201899563307,126cyclictest0-21swapper/200:18:342
220179956110,84cyclictest0-21swapper/123:43:001
220169956010,126cyclictest0-21swapper/023:59:240
220189955850,126cyclictest0-21swapper/200:26:312
220239955792,126cyclictest0-21swapper/722:46:487
220209955283,296cyclictest0-21swapper/400:09:424
22019995511,83cyclictest0-21swapper/321:31:543
22016995510,41cyclictest0-21swapper/023:45:170
22022995501,84cyclictest0-21swapper/622:46:456
220199954739,126cyclictest121rcu_preempt21:53:113
220219954680,211cyclictest0-21swapper/523:58:335
220209954678,211cyclictest49-21ksoftirqd/423:59:194
2201999546204,127cyclictest41-21ksoftirqd/300:20:343
2201899545121,84cyclictest0-21swapper/200:09:142
2202199544118,253cyclictest0-21swapper/523:18:465
2202399541117,126cyclictest0-21swapper/723:41:307
220209953868,422cyclictest0-21swapper/421:51:494
2201899538112,168cyclictest6959-21df23:00:162
220229953523,296cyclictest0-21swapper/623:42:276
2202099533196,3cyclictest481ktimersoftd/400:40:184
220179953219,254cyclictest0-21swapper/100:39:411
220219953165,295cyclictest0-21swapper/500:20:335
220209953164,126cyclictest0-21swapper/422:58:274
220209953164,126cyclictest0-21swapper/422:58:274
220209953164,126cyclictest0-21swapper/422:58:264
2202199529101,338cyclictest0-21swapper/500:39:095
220199952862,84cyclictest25126-21latency_hist00:30:023
220209952619,126cyclictest0-21swapper/422:46:514
220179952517,126cyclictest0-21swapper/100:20:311
220239952412,381cyclictest0-21swapper/700:09:037
2201799513298,84cyclictest0-21swapper/122:30:181
22019995116,80cyclictest0-21swapper/322:58:393
22019995116,80cyclictest0-21swapper/322:58:393
22019995116,80cyclictest0-21swapper/322:58:393
22016995111,84cyclictest0-21swapper/023:30:180
22016995090,42cyclictest0-21swapper/000:26:530
2202399508170,253cyclictest721ktimersoftd/723:57:437
22023995081,42cyclictest0-21swapper/700:20:367
220199950881,177cyclictest41-21ksoftirqd/300:17:343
220209950738,211cyclictest49-21ksoftirqd/423:17:114
220189950680,297cyclictest121rcu_preempt23:57:462
2202399504205,126cyclictest0-21swapper/721:51:357
220239950381,256cyclictest1911-21ssh22:36:567
220189950379,169cyclictest0-21swapper/223:30:192
220209950236,126cyclictest121rcu_preempt00:10:194
220209950176,422cyclictest27744-21kworker/4:222:36:564
220209950137,380cyclictest0-21swapper/400:20:194
220219950036,380cyclictest0-21swapper/522:56:295
220219950036,380cyclictest0-21swapper/522:56:295
220219950036,380cyclictest0-21swapper/522:56:285
220189949830,465cyclictest0-21swapper/222:46:492
220199949631,48cyclictest0-21swapper/323:44:363
220229949370,84cyclictest0-21swapper/621:51:466
220239949226,126cyclictest0-21swapper/722:10:257
2202099492154,302cyclictest64450irq/122-eth000:20:024
220179949226,84cyclictest0-21swapper/122:36:501
220209949127,299cyclictest12036-21df_abs00:00:164
2201899488104,253cyclictest33-21ksoftirqd/222:15:102
2202299487439,46cyclictest641ktimersoftd/600:20:146
220199948722,84cyclictest30079-21diskmemload00:00:133
2202299486228,88cyclictest0-21swapper/622:30:226
220229948518,89cyclictest0-21swapper/600:29:316
220219948522,252cyclictest0-21swapper/522:46:295
220189948519,84cyclictest0-21swapper/200:20:342
2202199483104,342cyclictest698-21ptp4l21:34:125
220219948297,211cyclictest24261-21ssh22:15:145
220199947857,84cyclictest121rcu_preempt23:59:273
220189947853,295cyclictest0-21swapper/222:40:112
220239947712,126cyclictest0-21swapper/721:32:117
22020994779,253cyclictest0-21swapper/423:26:594
220199947755,126cyclictest0-21swapper/322:10:223
220209947491,295cyclictest0-21swapper/421:31:554
22020994691,297cyclictest0-21swapper/400:27:234
220239946782,211cyclictest6920-21rm00:37:247
2202099466124,211cyclictest18184-21rm23:45:074
22022994651,89cyclictest0-21swapper/621:33:316
22016994651,41cyclictest0-21swapper/022:40:060
220179946441,259cyclictest0-21swapper/123:00:141
2201799463123,338cyclictest22014-21cyclictest22:15:151
2201799463123,338cyclictest0-21swapper/121:34:121
220229946281,338cyclictest0-21swapper/623:45:096
2202399461123,84cyclictest0-21swapper/700:27:417
2202399461120,338cyclictest0-21swapper/723:30:157
2201999461120,338cyclictest0-21swapper/322:15:123
220219945979,211cyclictest57-21ksoftirqd/500:00:155
220189945975,211cyclictest15555-21ntp_states22:10:252
220189945931,211cyclictest0-21swapper/223:45:232
2202199458117,338cyclictest0-21swapper/522:10:235
220239945734,339cyclictest0-21swapper/723:00:167
2202199457116,126cyclictest121rcu_preempt22:37:185
220199945734,256cyclictest1553-21runrttasks22:47:133
220229945533,3cyclictest16235-21kworker/u16:300:30:076
220169945528,85cyclictest0-21swapper/000:40:080
2202199454112,168cyclictest0-21swapper/500:27:135
2201999454239,84cyclictest0-21swapper/300:30:103
220199945331,41cyclictest18173-21ssh23:45:043
2201999453112,338cyclictest0-21swapper/322:37:093
220179945369,338cyclictest0-21swapper/123:45:091
220179945327,254cyclictest0-21swapper/100:10:181
220209945175,84cyclictest0-21swapper/422:15:134
220239945025,126cyclictest73-21ksoftirqd/721:55:067
2202199450107,340cyclictest0-21swapper/522:30:225
2201899449106,169cyclictest12064-21df_abs00:00:172
2202399448108,84cyclictest0-21swapper/723:45:177
220229944867,85cyclictest0-21swapper/600:08:306
2201899448107,338cyclictest0-21swapper/221:55:092
220219944623,257cyclictest0-21swapper/523:00:145
220239944522,256cyclictest0-21swapper/700:40:087
220179944520,211cyclictest0-21swapper/100:28:381
220179944519,84cyclictest0-21swapper/123:30:171
220239944422,256cyclictest0-21swapper/700:00:167
220189944421,256cyclictest0-21swapper/221:31:502
220169944464,127cyclictest9-21ksoftirqd/022:10:240
2201699444189,253cyclictest101ktimersoftd/022:15:150
2201999443106,171cyclictest0-21swapper/323:00:133
220169944368,3cyclictest26749-21taskset00:10:040
220209944257,211cyclictest49-21ksoftirqd/422:10:224
220169943816,256cyclictest0-21swapper/022:30:240
220229943715,84cyclictest0-21swapper/623:00:196
220189943715,84cyclictest0-21swapper/223:20:062
220229943556,82cyclictest30079-21diskmemload23:30:206
220219943312,296cyclictest0-21swapper/523:30:125
220179943311,84cyclictest0-21swapper/100:00:131
220189943210,169cyclictest0-21swapper/200:30:102
220169943210,255cyclictest0-21swapper/023:00:140
220239943044,211cyclictest24224-21ssh22:15:127
22021994297,254cyclictest0-21swapper/522:40:075
22017994264,7cyclictest0-21swapper/100:30:071
22020994252,84cyclictest0-21swapper/422:40:114
220229942483,338cyclictest0-21swapper/622:50:056
220219942482,126cyclictest0-21swapper/523:45:055
22016994240,41cyclictest0-21swapper/000:30:090
22016994230,41cyclictest0-21swapper/000:00:080
220169942281,338cyclictest0-21swapper/022:50:050
220239942180,338cyclictest0-21swapper/722:30:217
220219942180,338cyclictest0-21swapper/522:50:055
220209942182,84cyclictest0-21swapper/423:30:064
220179942181,338cyclictest0-21swapper/122:10:181
220229942080,338cyclictest0-21swapper/622:36:566
220229941939,296cyclictest0-21swapper/600:40:096
220199941935,261cyclictest0-21swapper/322:30:233
220239941838,262cyclictest0-21swapper/700:10:187
220189941877,338cyclictest0-21swapper/222:30:202
220189941876,253cyclictest28422-21rm00:10:182
220229941776,338cyclictest0-21swapper/622:10:256
220209941778,84cyclictest0-21swapper/400:30:084
220179941775,169cyclictest0-21swapper/122:40:091
220219941435,296cyclictest0-21swapper/523:20:075
220189941473,338cyclictest0-21swapper/200:40:152
220179941375,84cyclictest121rcu_preempt21:55:041
2202299409194,84cyclictest0-21swapper/600:00:166
220199940969,84cyclictest0-21swapper/322:50:043
220239940867,338cyclictest0-21swapper/700:30:117
220219940870,169cyclictest0-21swapper/521:55:065
220199940868,84cyclictest0-21swapper/300:10:023
220189940666,338cyclictest0-21swapper/222:50:052
220209940567,84cyclictest64450irq/122-eth022:50:044
220239940424,296cyclictest0-21swapper/722:40:097
220209940222,83cyclictest0-21swapper/423:00:194
220229939915,295cyclictest0-21swapper/600:10:196
220179939617,42cyclictest0-21swapper/123:20:071
220209939557,84cyclictest0-21swapper/422:30:284
220219939414,296cyclictest0-21swapper/500:40:135
220169939314,296cyclictest0-21swapper/023:20:080
22022993887,84cyclictest0-21swapper/621:55:056
22017993821,211cyclictest0-21swapper/122:50:051
220199938141,84cyclictest0-21swapper/300:40:133
220179937980,126cyclictest121rcu_preempt00:40:111
220219937777,168cyclictest18480-1kworker/u17:100:30:075
220229937638,3cyclictest0-21swapper/622:40:056
2202099376124,42cyclictest64450irq/122-eth023:20:084
220199937334,84cyclictest0-21swapper/321:55:053
220209936728,84cyclictest22014-21cyclictest21:55:054
220239936426,256cyclictest0-21swapper/723:20:087
220219935720,42cyclictest0-21swapper/500:10:185
220199935312,127cyclictest41-21ksoftirqd/323:20:083
220239935110,338cyclictest17685-1kworker/u17:122:50:057
22019993508,127cyclictest41-21ksoftirqd/322:40:103
220229929340,3cyclictest0-21swapper/623:20:056
2130429160,28sleep20-21swapper/219:05:592
1956029187,1sleep40-21swapper/419:05:124
2048228961,5sleep10-21swapper/119:05:171
2134328644,27sleep50-21swapper/519:06:345
2130028648,7sleep70-21swapper/719:05:557
2153328453,27sleep60-21swapper/619:09:166
2132328252,27sleep30-21swapper/319:06:163
2137727947,28sleep00-21swapper/019:07:010
250702670,0sleep20-21swapper/221:23:042
117502660,0sleep30-21swapper/320:35:233
277592580,4sleep42202099cyclictest23:50:084
146652400,2sleep714669-21head21:35:167
2201899392,34cyclictest0-21swapper/223:35:492
2202299372,31cyclictest0-21swapper/621:35:106
2201999372,31cyclictest0-21swapper/321:35:053
2202099362,30cyclictest0-21swapper/421:35:084
2201999369,15cyclictest41-21ksoftirqd/321:25:463
22022993425,6cyclictest27984-21grep23:50:126
22020993425,6cyclictest4372-21grep23:35:224
2201699341,3cyclictest13393-21ssh21:35:010
22023993324,5cyclictest29042-21sort23:50:187
2202099335,21cyclictest49-21ksoftirqd/421:25:424
2201999337,5cyclictest0-21swapper/323:50:163
22016993325,5cyclictest3626-21sh23:35:150
22019993210,11cyclictest0-21swapper/323:35:423
22023993124,4cyclictest5645-21ssh23:35:477
22023993124,4cyclictest1145-21ssh21:27:307
22022993123,5cyclictest31769-21ssh21:25:546
22018993124,4cyclictest29091-21ssh23:50:172
22018993124,4cyclictest167052sleep221:36:002
22017993123,5cyclictest32006-21ssh21:26:071
22017993123,5cyclictest2997-21grep23:35:131
22016993123,5cyclictest32198-21ssh21:26:170
22021993023,4cyclictest29650-21grep23:50:245
22021993023,4cyclictest1040-21ssh21:27:245
22017993023,4cyclictest16436-21ssh21:35:441
22016993022,5cyclictest27588-21chmod23:50:010
22022992922,4cyclictest2801-21egrep23:35:126
22022992910,18cyclictest0-21swapper/619:12:376
22021992924,3cyclictest15636-21ssh21:35:275
22021992921,5cyclictest2898-21grep23:35:135
22018992922,4cyclictest32027-21ssh21:26:082
22017992922,4cyclictest30669-21ssh23:50:451
22023992810,0cyclictest0-21swapper/719:12:347
22021992810,18cyclictest0-21swapper/519:12:555
152692280,0sleep30-21swapper/320:45:003
22020992727,0cyclictest0-21swapper/419:12:364
22017992710,0cyclictest0-21swapper/119:12:361
2202199262,4cyclictest0-21swapper/523:05:025
22019992626,0cyclictest0-21swapper/319:12:573
22018992626,0cyclictest0-21swapper/219:12:472
22016992626,0cyclictest0-21swapper/019:12:340
95872200,0sleep60-21swapper/621:15:156
2202299190,1cyclictest0-21swapper/620:25:026
2202299190,1cyclictest0-21swapper/619:30:026
22017991917,1cyclictest0-21swapper/120:25:171
2202399180,17cyclictest0-21swapper/719:15:007
276912170,0sleep40-21swapper/419:15:204
2202299167,5cyclictest14755-21sh23:05:016
2202299160,14cyclictest26589-21diskstats19:15:136
22022991512,2cyclictest641ktimersoftd/619:45:006
22020991512,2cyclictest481ktimersoftd/422:00:224
22022991410,3cyclictest30091-21perf21:10:006
2202099142,4cyclictest0-21swapper/423:05:024
2201999149,3cyclictest2516-21grep20:20:223
2201999143,7cyclictest41-21ksoftirqd/322:25:143
2201999140,13cyclictest0-21swapper/319:15:123
2201799149,3cyclictest3261-21df19:30:131
2201699149,3cyclictest6567-21sh21:14:120
2201699149,3cyclictest29861-21grep20:15:120
2201699144,9cyclictest0-21swapper/019:30:000
2201699140,13cyclictest0-21swapper/019:15:020
22023991310,2cyclictest16953-21grep23:05:157
22022991311,1cyclictest20453-21/usr/sbin/munin21:20:186
22022991310,2cyclictest28667-21ls20:10:216
22022991310,2cyclictest20379-21apt20:00:006
22021991311,1cyclictest13940-21unixbench-2d19:45:225
22021991311,1cyclictest11907-21ssh22:09:095
22021991311,1cyclictest0-21swapper/521:15:175
2202199130,12cyclictest0-21swapper/519:15:015
22020991311,1cyclictest49-21ksoftirqd/421:41:274
22020991311,1cyclictest29934-21tail19:20:144
22020991311,1cyclictest0-21swapper/420:20:144
22020991311,1cyclictest0-21swapper/419:50:134
22018991311,1cyclictest20288-21python20:50:192
22017991311,1cyclictest4611-21fschecks_count22:05:141
22017991311,1cyclictest22895-21ls20:55:181
22017991311,1cyclictest0-21swapper/121:21:311
2201699139,2cyclictest27692-21tr21:05:130
22016991311,1cyclictest30079-21diskmemload22:23:120
22016991311,1cyclictest30079-21diskmemload22:23:120
22016991311,1cyclictest22482-21gltestperf20:55:150
22016991311,1cyclictest0-21swapper/023:10:110
2202399129,2cyclictest929-21cat22:20:137
2202399129,2cyclictest929-21cat22:20:127
2202399129,2cyclictest7241-21uniq20:30:147
2202399129,2cyclictest7241-21uniq20:30:137
2202399129,2cyclictest721ktimersoftd/721:15:187
2202399129,2cyclictest28301-21grep21:05:147
2202399129,2cyclictest19387-21cut19:55:197
2202399128,3cyclictest5756-21taskset22:05:207
2202399128,3cyclictest25273-21sh21:40:217
22023991210,1cyclictest124032sleep722:25:227
22023991210,1cyclictest0-21swapper/723:10:057
22023991210,1cyclictest0-21swapper/722:00:207
22023991210,1cyclictest0-21swapper/721:45:037
22023991210,1cyclictest0-21swapper/721:20:397
22023991210,1cyclictest0-21swapper/721:11:117
22023991210,1cyclictest0-21swapper/721:00:217
22023991210,1cyclictest0-21swapper/720:58:357
22023991210,1cyclictest0-21swapper/720:50:077
22023991210,1cyclictest0-21swapper/720:48:247
22023991210,1cyclictest0-21swapper/720:40:127
22023991210,1cyclictest0-21swapper/720:35:217
22023991210,1cyclictest0-21swapper/720:26:357
22023991210,1cyclictest0-21swapper/720:20:057
22023991210,1cyclictest0-21swapper/720:15:427
22023991210,1cyclictest0-21swapper/720:10:027
22023991210,1cyclictest0-21swapper/720:05:207
22023991210,1cyclictest0-21swapper/720:00:117
22023991210,1cyclictest0-21swapper/719:50:347
22023991210,1cyclictest0-21swapper/719:45:217
22023991210,1cyclictest0-21swapper/719:40:237
22023991210,1cyclictest0-21swapper/719:36:357
22023991210,1cyclictest0-21swapper/719:30:207
22023991210,1cyclictest0-21swapper/719:25:007
22023991210,1cyclictest0-21swapper/719:21:007
2202299129,2cyclictest25714-21grep23:10:106
2202299129,2cyclictest15150-21expr19:50:116
2202299129,2cyclictest13650-21ssh22:25:536
2202299129,2cyclictest1344-21grep20:20:146
2202299129,2cyclictest11865-21taskset19:45:116
2202299128,3cyclictest1442-21rm21:10:466
22022991210,1cyclictest641ktimersoftd/620:00:236
22022991210,1cyclictest237582sleep620:55:296
22022991210,1cyclictest0-21swapper/622:20:336
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional