You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-08 - 07:04
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Wed Jul 08, 2026 00:58:29)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
70639975735,719cyclictest7059-21cyclictest22:55:372
70639969312,169cyclictest0-21swapper/200:00:052
706399667156,84cyclictest121rcu_preempt22:40:312
706899656536,37cyclictest721ktimersoftd/720:25:157
70649965521,2cyclictest0-21swapper/321:39:493
70649964814,2cyclictest0-21swapper/320:49:193
70619963785,340cyclictest0-21swapper/023:17:080
706899633549,3cyclictest721ktimersoftd/700:21:257
706299631162,127cyclictest121rcu_preempt20:26:321
706899629546,3cyclictest721ktimersoftd/721:37:257
70629962734,84cyclictest0-21swapper/100:10:081
706899625542,3cyclictest721ktimersoftd/721:05:317
70639962433,507cyclictest0-21swapper/223:38:492
70639962433,507cyclictest0-21swapper/223:38:492
70639962227,592cyclictest0-21swapper/220:45:452
706899621537,3cyclictest721ktimersoftd/722:11:037
706599620110,507cyclictest49-21ksoftirqd/400:35:044
706399618534,3cyclictest321ktimersoftd/222:54:162
70659961234,159cyclictest698-21ptp4l23:37:144
70659961234,159cyclictest698-21ptp4l23:37:144
70629961217,84cyclictest0-21swapper/100:09:291
70639960711,593cyclictest0-21swapper/223:02:452
70629960716,84cyclictest0-21swapper/122:03:491
70679960513,383cyclictest0-21swapper/621:39:496
70639959440,551cyclictest0-21swapper/222:11:522
7062995942,383cyclictest0-21swapper/121:39:481
7066995930,84cyclictest0-21swapper/500:16:095
7065995930,84cyclictest0-21swapper/400:16:094
706899591506,3cyclictest721ktimersoftd/721:40:547
70639958835,550cyclictest0-21swapper/220:28:172
70629958838,85cyclictest0-21swapper/120:35:281
70639958734,550cyclictest0-21swapper/221:58:572
70639958734,465cyclictest0-21swapper/223:25:262
706299587544,41cyclictest231rcuc/123:27:331
70689958674,381cyclictest0-21swapper/700:15:467
706899584501,4cyclictest721ktimersoftd/721:56:177
706499581115,126cyclictest27556-21forks23:40:173
706499580111,296cyclictest0-21swapper/323:30:233
706899575536,37cyclictest721ktimersoftd/723:37:427
706899575536,37cyclictest721ktimersoftd/723:37:427
706899575407,41cyclictest721ktimersoftd/722:52:007
706499575109,126cyclictest0-21swapper/321:56:293
70639957421,550cyclictest0-21swapper/222:00:412
70679956818,383cyclictest0-21swapper/620:49:206
706499568482,84cyclictest391rcuc/323:39:213
706499568482,84cyclictest391rcuc/323:39:213
706199568227,168cyclictest0-21swapper/022:33:550
70619956756,296cyclictest0-21swapper/020:25:150
70619956210,126cyclictest0-21swapper/022:35:200
7063995606,42cyclictest0-21swapper/200:24:142
7062995555,126cyclictest0-21swapper/100:00:201
70639955241,508cyclictest0-21swapper/221:37:432
7065995501,3cyclictest0-21swapper/421:39:494
70639954938,508cyclictest0-21swapper/200:41:522
7062995491,382cyclictest0-21swapper/120:49:201
706199549505,42cyclictest101ktimersoftd/021:05:180
70639954740,84cyclictest121rcu_preempt23:17:442
70619954782,126cyclictest0-21swapper/023:09:200
70669954577,296cyclictest0-21swapper/522:01:545
706799544204,85cyclictest10943-21kworker/u16:223:04:196
706199544287,126cyclictest0-21swapper/023:02:320
70669954235,83cyclictest0-21swapper/522:45:315
70649954133,126cyclictest0-21swapper/323:02:563
706299541119,253cyclictest0-21swapper/100:43:261
70659953974,83cyclictest121rcu_preempt22:45:144
70639953929,507cyclictest0-21swapper/223:30:272
70629953930,381cyclictest0-21swapper/123:37:151
70629953930,381cyclictest0-21swapper/123:37:151
70619953928,344cyclictest0-21swapper/020:50:020
70659953870,253cyclictest0-21swapper/400:24:064
706799537195,253cyclictest0-21swapper/620:29:206
706599537111,127cyclictest0-21swapper/422:05:014
706499537196,338cyclictest7196-1kworker/u17:023:27:213
706399537493,42cyclictest321ktimersoftd/222:05:162
70679953526,84cyclictest0-21swapper/623:56:506
70629953570,84cyclictest0-21swapper/100:34:361
70629953570,84cyclictest0-21swapper/100:34:351
70639953423,508cyclictest0-21swapper/223:57:432
70629953326,126cyclictest30983-21sed20:40:171
70669953262,296cyclictest0-21swapper/520:37:025
706699531108,256cyclictest704-21nscd00:34:365
706699531108,256cyclictest704-21nscd00:34:365
706399531492,3cyclictest321ktimersoftd/200:35:042
70649952920,84cyclictest0-21swapper/300:16:453
70629952918,126cyclictest0-21swapper/121:10:061
706299529107,3cyclictest14796-21diskmemload23:19:421
70629952817,169cyclictest30115-21ssh22:45:301
706799527228,126cyclictest29869-21kworker/u16:000:16:086
70659952719,88cyclictest0-21swapper/423:56:074
70669952615,296cyclictest0-21swapper/522:40:225
706599526188,124cyclictest121rcu_preempt00:33:274
706599526188,124cyclictest121rcu_preempt00:33:274
70639952615,42cyclictest7059-21cyclictest21:08:112
706199526101,295cyclictest0-21swapper/000:16:590
70629952519,127cyclictest0-21swapper/122:10:441
706199525102,84cyclictest121rcu_preempt22:45:210
706899524142,84cyclictest121rcu_preempt23:17:177
70659952455,296cyclictest0-21swapper/400:00:294
70639952459,380cyclictest0-21swapper/200:16:292
706599521224,295cyclictest64450irq/122-eth022:13:184
706499521477,42cyclictest401ktimersoftd/320:37:043
70639952110,42cyclictest0-21swapper/200:08:272
70669952054,85cyclictest121rcu_preempt23:29:165
70639952013,127cyclictest0-21swapper/222:34:162
706199520225,83cyclictest0-21swapper/000:23:120
70619951912,2cyclictest7011-21ssh21:39:490
70679951896,84cyclictest0-21swapper/623:19:306
7065995186,84cyclictest0-21swapper/423:30:314
706299517473,42cyclictest241ktimersoftd/121:08:381
70659951610,2cyclictest27922chrt20:49:204
7061995135,127cyclictest8645-21ssh00:42:450
7063995121,508cyclictest0-21swapper/220:40:182
7064995100,85cyclictest0-21swapper/321:30:173
7063995102,84cyclictest24286-21/usr/sbin/munin19:35:172
7066995090,84cyclictest23227-21chrt21:30:165
7066995090,84cyclictest0-21swapper/519:35:335
7065995090,84cyclictest0-21swapper/421:30:174
706299509299,42cyclictest0-21swapper/121:59:441
7067995080,84cyclictest0-21swapper/600:07:216
7066995080,83cyclictest0-21swapper/500:07:215
7068995071,382cyclictest0-21swapper/720:49:207
7065995071,340cyclictest0-21swapper/400:42:124
70659950679,338cyclictest0-21swapper/423:03:594
70649950578,126cyclictest0-21swapper/320:25:523
70639950439,386cyclictest0-21swapper/200:32:452
70639950439,386cyclictest0-21swapper/200:32:452
70619950481,126cyclictest0-21swapper/022:11:050
706899503420,3cyclictest721ktimersoftd/722:32:427
70669950382,83cyclictest0-21swapper/523:19:305
706399503118,169cyclictest303-21systemd-journal22:15:122
70629950334,126cyclictest0-21swapper/123:24:311
706899502463,3cyclictest721ktimersoftd/722:00:267
706899502418,2cyclictest721ktimersoftd/723:26:587
70639950234,127cyclictest0-21swapper/223:24:582
70629950280,254cyclictest0-21swapper/100:21:231
706299502120,380cyclictest25-21ksoftirqd/100:35:041
706799501458,41cyclictest631rcuc/620:35:566
706199501418,3cyclictest101ktimersoftd/022:03:050
70679950035,84cyclictest0-21swapper/622:00:586
70669950074,338cyclictest0-21swapper/523:59:195
706599500241,84cyclictest0-21swapper/423:25:444
70649950035,298cyclictest0-21swapper/300:42:123
706199500116,296cyclictest22669-21kworker/u16:123:56:230
706899499461,2cyclictest721ktimersoftd/722:15:277
70619949933,126cyclictest0-21swapper/022:54:360
706599497283,126cyclictest0-21swapper/422:34:084
70639949732,381cyclictest0-21swapper/222:35:272
706699496155,338cyclictest32139-1kworker/u17:021:39:335
706199496115,126cyclictest0-21swapper/000:35:040
706899495411,3cyclictest711rcuc/700:07:197
70669949569,211cyclictest0-21swapper/523:40:025
70659949531,258cyclictest0-21swapper/421:59:214
706499495411,3cyclictest391rcuc/321:05:443
70639949525,126cyclictest0-21swapper/220:30:182
70629949530,84cyclictest0-21swapper/100:25:181
70619949531,84cyclictest5905-21ssh23:25:390
70619949530,89cyclictest0-21swapper/021:56:120
70669949428,211cyclictest7059-21cyclictest21:56:125
70659949468,295cyclictest0-21swapper/420:29:514
70639949427,464cyclictest0-21swapper/223:40:042
70679949337,84cyclictest0-21swapper/623:28:306
70629949371,126cyclictest0-21swapper/123:58:221
70659949169,211cyclictest0-21swapper/420:20:214
70619949067,126cyclictest7742-21apt-get21:40:150
70649948925,42cyclictest0-21swapper/322:11:053
70639948923,88cyclictest0-21swapper/220:20:302
706399489107,126cyclictest29099-21taskset20:37:272
70689948862,211cyclictest0-21swapper/719:50:397
70679948722,3cyclictest8027-21ssh00:42:036
70669948765,256cyclictest7059-21cyclictest23:39:085
706599487105,125cyclictest0-21swapper/420:37:534
70649948723,84cyclictest0-21swapper/321:10:133
70639948719,465cyclictest0-21swapper/223:47:092
70629948764,126cyclictest11362-21ssh23:30:221
706199487233,166cyclictest0-21swapper/000:32:030
706199487233,166cyclictest0-21swapper/000:32:030
706599486232,82cyclictest0-21swapper/422:40:024
706799485147,3cyclictest641ktimersoftd/622:55:186
70629948462,380cyclictest0-21swapper/122:40:401
70689948318,3cyclictest0-21swapper/720:11:277
70659948212,86cyclictest0-21swapper/421:10:084
70649948259,257cyclictest8406-21ssh23:46:563
70649948155,211cyclictest0-21swapper/322:52:253
70639948116,297cyclictest0-21swapper/219:50:212
706799480441,3cyclictest631rcuc/621:57:036
706699480435,42cyclictest561ktimersoftd/520:29:405
70619948054,296cyclictest0-21swapper/000:00:400
70629947852,295cyclictest0-21swapper/122:54:371
706299478226,126cyclictest13816-1kworker/u17:223:02:411
70619947812,84cyclictest0-21swapper/022:40:390
70659947410,49cyclictest6330-21ssh22:15:364
70679947347,177cyclictest65-21ksoftirqd/622:40:246
7063994735,43cyclictest0-21swapper/219:27:422
7067994717,2cyclictest0-21swapper/621:05:396
7063994717,7cyclictest0-21swapper/222:25:172
706899470431,37cyclictest711rcuc/723:03:037
70649946786,126cyclictest27276-21kworker/u16:122:40:353
70689946681,211cyclictest73-21ksoftirqd/700:35:277
70679946639,296cyclictest65-21ksoftirqd/621:40:136
706499465123,126cyclictest0-21swapper/323:24:583
70619946581,253cyclictest0-21swapper/019:14:100
706899463379,3cyclictest721ktimersoftd/722:40:127
70669946382,85cyclictest7059-21cyclictest20:48:105
70689946240,256cyclictest0-21swapper/720:54:447
70679946240,2cyclictest0-21swapper/623:39:526
70679946240,2cyclictest0-21swapper/623:39:526
70679946240,2cyclictest0-21swapper/600:34:266
70679946240,2cyclictest0-21swapper/600:34:266
706799462121,338cyclictest0-21swapper/600:25:146
706499462126,248cyclictest29081-1kworker/u17:100:24:233
70689946139,256cyclictest0-21swapper/723:58:207
706899461382,76cyclictest721ktimersoftd/722:45:247
706599461124,41cyclictest0-21swapper/423:40:214
70639946135,3cyclictest0-21swapper/220:16:052
70619946138,256cyclictest0-21swapper/023:36:520
70619946138,256cyclictest0-21swapper/023:36:520
70619946133,338cyclictest0-21swapper/020:37:270
70679946039,2cyclictest0-21swapper/622:34:326
70679946038,2cyclictest0-21swapper/600:35:286
706799460160,297cyclictest5799-1kworker/u17:000:24:216
70669946081,134cyclictest0-21swapper/500:21:505
70619946034,85cyclictest0-21swapper/023:30:220
70669945935,42cyclictest0-21swapper/522:53:285
706699459202,84cyclictest121rcu_preempt20:50:145
70689945874,381cyclictest0-21swapper/720:37:387
70619945874,126cyclictest0-21swapper/023:20:100
70679945735,42cyclictest0-21swapper/623:46:576
706599457377,3cyclictest471rcuc/421:05:134
706599457116,338cyclictest9069-21ssh00:25:224
70619945736,2cyclictest0-21swapper/021:30:140
70619945734,126cyclictest0-21swapper/021:10:210
706699456158,126cyclictest121rcu_preempt20:30:335
706499456115,338cyclictest0-21swapper/319:35:243
70639945634,343cyclictest0-21swapper/222:45:372
70629945635,2cyclictest0-21swapper/100:16:031
70689945533,256cyclictest0-21swapper/700:42:227
70639945530,126cyclictest0-21swapper/200:25:272
70669945432,258cyclictest0-21swapper/523:25:005
70659945432,3cyclictest0-21swapper/423:19:474
70659945429,422cyclictest0-21swapper/400:07:544
70649945470,211cyclictest0-21swapper/323:59:173
70649945470,211cyclictest0-21swapper/320:20:163
70649945368,254cyclictest0-21swapper/322:32:513
70619945330,257cyclictest3964-21cpu22:15:150
70689945230,256cyclictest0-21swapper/700:25:117
70679945230,2cyclictest0-21swapper/622:51:466
70689945167,211cyclictest73-21ksoftirqd/700:33:217
70689945167,211cyclictest73-21ksoftirqd/700:33:217
70659945129,257cyclictest0-21swapper/423:25:004
706499451114,171cyclictest401ktimersoftd/320:54:443
706299451114,211cyclictest6079-21sh22:32:131
70669945029,4cyclictest0-21swapper/522:14:065
706799449194,84cyclictest641ktimersoftd/620:40:176
70669944927,42cyclictest0-21swapper/521:07:005
70669944927,256cyclictest0-21swapper/523:30:275
706499449106,85cyclictest0-21swapper/323:19:163
70639944923,169cyclictest0-21swapper/223:10:122
70649944825,257cyclictest16138-21ssh22:01:463
706499448149,126cyclictest22669-21kworker/u16:100:25:123
70679944725,127cyclictest0-21swapper/622:12:546
70659944725,44cyclictest0-21swapper/422:51:464
706299447106,338cyclictest0-21swapper/122:05:101
70619944732,36cyclictest0-21swapper/000:08:580
706199447106,338cyclictest0-21swapper/020:20:140
70669944624,43cyclictest0-21swapper/523:03:465
706699446193,84cyclictest0-21swapper/521:40:185
706599446107,49cyclictest19341-21ssh22:05:104
70639944621,388cyclictest0-21swapper/221:40:122
70679944521,258cyclictest0-21swapper/622:15:156
70639944519,169cyclictest0-21swapper/220:11:352
70629944523,2cyclictest0-21swapper/123:10:141
70619944561,126cyclictest0-21swapper/000:10:280
706199445105,338cyclictest0-21swapper/023:46:560
70689944422,256cyclictest0-21swapper/720:20:277
706899444103,338cyclictest0-21swapper/720:40:027
70669944423,255cyclictest0-21swapper/520:40:175
70649944421,42cyclictest0-21swapper/300:00:393
70649944419,127cyclictest0-21swapper/300:09:263
70679944321,256cyclictest0-21swapper/623:25:006
70669944321,84cyclictest0-21swapper/522:55:035
70629944321,255cyclictest0-21swapper/121:40:531
70689944219,256cyclictest0-21swapper/723:25:007
706299442356,84cyclictest241ktimersoftd/122:55:081
70679944019,2cyclictest0-21swapper/622:45:126
70679944017,126cyclictest11392-21latency23:30:236
70649944055,211cyclictest0-21swapper/322:35:153
706299440100,337cyclictest18134-21cstates20:20:171
70619944099,338cyclictest0-21swapper/020:11:550
70669943957,85cyclictest0-21swapper/520:20:175
70669943614,84cyclictest0-21swapper/522:25:245
70659943552,380cyclictest0-21swapper/422:55:124
70669943412,256cyclictest0-21swapper/522:34:325
7066994327,126cyclictest0-21swapper/523:46:305
7065994329,84cyclictest0-21swapper/419:14:104
7068994318,211cyclictest0-21swapper/723:09:207
7065994308,42cyclictest0-21swapper/423:47:114
70649942948,84cyclictest0-21swapper/322:15:183
7066994287,47cyclictest32076-21pmu-power00:00:355
706599428346,3cyclictest471rcuc/419:35:574
7063994271,423cyclictest0-21swapper/221:10:072
7061994271,253cyclictest8033-21ssh00:25:160
7066994264,256cyclictest0-21swapper/500:42:115
706599426127,253cyclictest0-21swapper/419:50:374
7063994261,126cyclictest0-21swapper/220:50:062
7062994253,256cyclictest0-21swapper/122:15:151
70689942483,338cyclictest0-21swapper/722:35:277
7067994241,83cyclictest0-21swapper/622:35:276
70639942483,338cyclictest0-21swapper/221:30:142
7065994231,2cyclictest0-21swapper/420:40:174
70629942382,338cyclictest0-21swapper/123:40:111
7062994231,84cyclictest25164-21awk22:25:181
70679942284,3cyclictest31960-21ssh00:00:316
70649942283,84cyclictest0-21swapper/300:35:073
70649942283,84cyclictest0-21swapper/300:34:483
70649942283,84cyclictest0-21swapper/300:34:483
70619942280,211cyclictest9-21ksoftirqd/023:40:100
706899421122,126cyclictest0-21swapper/722:25:187
70679942139,2cyclictest0-21swapper/600:10:246
70669942182,84cyclictest0-21swapper/521:10:085
70689942080,338cyclictest0-21swapper/723:40:117
70679942040,2cyclictest0-21swapper/620:30:256
70649942081,84cyclictest0-21swapper/321:40:133
70629942080,338cyclictest0-21swapper/123:47:101
70629942079,338cyclictest14176-21taskset20:11:541
70689941979,337cyclictest0-21swapper/719:35:337
70689941939,296cyclictest7059-21cyclictest23:46:097
70689941939,256cyclictest0-21swapper/722:05:067
706899419120,296cyclictest7059-21cyclictest23:30:127
70669941981,83cyclictest0-21swapper/522:35:205
70649941980,84cyclictest0-21swapper/322:05:103
70649941939,3cyclictest0-21swapper/320:30:343
70639941931,385cyclictest0-21swapper/200:10:242
706299419118,126cyclictest0-21swapper/121:30:131
70619941978,253cyclictest0-21swapper/019:35:140
70689941877,338cyclictest0-21swapper/720:30:327
70669941879,84cyclictest0-21swapper/519:50:355
70649941879,84cyclictest0-21swapper/300:10:263
70629941834,47cyclictest0-21swapper/120:50:091
70679941776,338cyclictest0-21swapper/620:20:196
70679941737,2cyclictest0-21swapper/623:40:106
70659941776,126cyclictest49-21ksoftirqd/421:40:464
70619941737,3cyclictest0-21swapper/022:25:170
70679941676,338cyclictest0-21swapper/621:10:086
70659941677,84cyclictest0-21swapper/419:27:434
70649941677,84cyclictest0-21swapper/320:16:223
70629941675,126cyclictest0-21swapper/119:36:191
70629941634,214cyclictest7059-21cyclictest22:35:201
70679941572,126cyclictest65-21ksoftirqd/623:10:166
70689941473,338cyclictest0-21swapper/700:00:337
70669941473,168cyclictest17314-21gunzip00:10:245
70659941476,84cyclictest0-21swapper/422:35:204
70649941435,255cyclictest0-21swapper/322:55:183
70629941435,2cyclictest0-21swapper/119:14:091
706299414288,42cyclictest241ktimersoftd/119:50:271
70669941375,83cyclictest0-21swapper/500:25:285
70669941374,84cyclictest5149-21sh22:15:235
70659941333,45cyclictest0-21swapper/400:10:164
70659941331,85cyclictest0-21swapper/423:09:204
70619941372,338cyclictest0-21swapper/020:30:230
70689941130,256cyclictest0-21swapper/721:10:077
70659940970,84cyclictest0-21swapper/420:50:054
70619940978,126cyclictest121rcu_preempt20:54:440
70619940928,84cyclictest19292-21sh22:05:050
70649940667,84cyclictest0-21swapper/323:09:203
70679940563,339cyclictest0-21swapper/619:35:146
70649940462,168cyclictest41-21ksoftirqd/320:11:533
70629940463,338cyclictest0-21swapper/120:30:231
706799403232,168cyclictest641ktimersoftd/623:09:196
706699401233,42cyclictest27624-21/usr/sbin/munin23:40:205
70649940120,43cyclictest0-21swapper/322:45:303
70619940122,2cyclictest0-21swapper/022:55:070
70619940121,3cyclictest0-21swapper/023:10:140
70669940018,45cyclictest0-21swapper/522:05:165
706899399143,126cyclictest121rcu_preempt22:55:117
706799399188,83cyclictest641ktimersoftd/622:05:156
70659939960,84cyclictest0-21swapper/420:30:204
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional