You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-03 - 19:21
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot6.osadl.org (updated Fri Apr 03, 2026 12:44:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25804521120,104pltrace0-21swapper/307:10:003
23020521100,97pltrace0-21swapper/607:05:146
1814652970,0pltrace0-21swapper/011:44:380
2302052920,83pltrace0-21swapper/107:05:101
2435852890,1pltrace0-21swapper/709:59:377
2302052830,77pltrace0-21swapper/507:07:535
2302052820,39pltrace0-21swapper/707:08:397
2556327157,10sleep20-21swapper/207:06:262
2552926240,17sleep40-21swapper/407:05:574
2565626046,10sleep00-21swapper/007:07:460
136932250,0sleep20-21swapper/210:35:222
3211152240,0pltrace0-21swapper/712:10:137
2606199221,15cyclictest0-21swapper/708:39:047
2603199220,15cyclictest0-21swapper/009:20:350
1706952220,3pltrace18987-21grep10:45:197
2651752210,6pltrace78-21ksoftirqd/712:01:037
1985552210,0pltrace0-21swapper/710:50:157
592252200,0pltrace0-21swapper/710:25:147
2604899208,2cyclictest54-21ksoftirqd/411:50:314
2603999200,14cyclictest0-21swapper/209:35:212
1535752200,0pltrace0-21swapper/711:39:587
1535752200,0pltrace0-21swapper/711:39:587
1428052200,4pltrace0-21swapper/710:40:157
2605399191,15cyclictest0-21swapper/507:47:115
2605399190,2cyclictest32427-21ntp_states08:15:165
2603999190,7cyclictest0-21swapper/210:00:002
1490452190,14pltrace0-21swapper/008:46:030
870652180,14pltrace0-21swapper/110:33:021
2606199180,14cyclictest0-21swapper/708:49:437
26048991816,1cyclictest0-21swapper/410:38:524
26044991813,3cyclictest0-21swapper/308:19:203
26031991810,7cyclictest0-21swapper/010:29:430
1535752180,4pltrace0-21swapper/711:41:457
825352170,0pltrace0-21swapper/307:38:433
2605799171,15cyclictest0-21swapper/610:33:496
2605799171,15cyclictest0-21swapper/610:24:066
2605799170,16cyclictest27799-21ps07:10:196
26053991712,4cyclictest0-21swapper/508:20:175
26053991712,4cyclictest0-21swapper/508:20:175
2605399171,15cyclictest0-21swapper/509:40:335
2605399170,1cyclictest0-21swapper/509:53:435
26048991711,5cyclictest0-21swapper/411:38:014
26048991711,5cyclictest0-21swapper/411:38:014
2604899171,7cyclictest191rcu_preempt07:55:234
2604899171,4cyclictest0-21swapper/407:34:044
2604899170,1cyclictest0-21swapper/407:39:144
26044991713,2cyclictest0-21swapper/309:00:473
26044991711,4cyclictest0-21swapper/309:48:333
2603999179,7cyclictest1047-21NetworkManager09:15:012
26036991711,5cyclictest0-21swapper/107:17:221
2435852170,4pltrace0-21swapper/710:04:267
2326552170,2pltrace0-21swapper/709:00:187
2264652170,0pltrace0-21swapper/710:58:177
1085852170,7pltrace0-21swapper/712:29:527
375752160,10pltrace1146-21snmpd08:29:046
2606199161,4cyclictest0-21swapper/707:54:447
2606199160,15cyclictest0-21swapper/707:49:157
2605799161,7cyclictest0-21swapper/608:18:306
2605799161,3cyclictest0-21swapper/611:51:556
2605799160,8cyclictest0-21swapper/610:48:206
2605799160,8cyclictest0-21swapper/609:39:036
26053991612,3cyclictest0-21swapper/507:34:385
2605399160,2cyclictest4255-21munin-run11:20:005
2605399160,2cyclictest4255-21munin-run11:20:005
2605399160,10cyclictest29375-21idleruntime-cro12:05:005
2605399160,1cyclictest0-21swapper/510:03:355
2604899166,9cyclictest0-21swapper/412:20:234
26048991612,3cyclictest0-21swapper/408:44:344
2604899161,7cyclictest0-21swapper/409:07:534
2604899161,14cyclictest0-21swapper/408:37:554
2604899160,14cyclictest0-21swapper/412:36:304
2603999164,7cyclictest1178-21runrttasks12:10:542
26039991610,5cyclictest0-21swapper/207:29:362
2603999161,8cyclictest0-21swapper/208:03:162
2603999160,8cyclictest0-21swapper/212:15:282
26036991610,5cyclictest0-21swapper/111:35:181
26036991610,5cyclictest0-21swapper/111:35:181
2603699160,2cyclictest0-21swapper/111:45:001
2499552160,0pltrace0-21swapper/708:07:407
2372052160,6pltrace0-21swapper/511:57:555
528352150,12pltrace1146-21snmpd12:23:506
313552150,2pltrace0-21swapper/710:20:257
26061991511,3cyclictest0-21swapper/709:09:597
2606199150,8cyclictest0-21swapper/710:39:597
2606199150,8cyclictest0-21swapper/708:50:227
2606199150,14cyclictest0-21swapper/711:12:247
2606199150,14cyclictest0-21swapper/708:16:097
2605799159,5cyclictest0-21swapper/612:16:326
2605799159,5cyclictest0-21swapper/611:18:066
2605799159,5cyclictest0-21swapper/611:18:066
26057991510,4cyclictest0-21swapper/612:31:056
2605799150,8cyclictest0-21swapper/612:00:376
2605799150,7cyclictest0-21swapper/609:58:516
2605799150,14cyclictest0-21swapper/609:28:596
2605799150,14cyclictest0-21swapper/608:07:126
2605799150,14cyclictest0-21swapper/607:50:236
2605399159,5cyclictest0-21swapper/509:48:485
2605399157,7cyclictest1146-21snmpd08:59:015
26053991510,3cyclictest0-21swapper/507:14:435
2605399151,7cyclictest0-21swapper/507:20:195
2605399150,8cyclictest0-21swapper/507:18:495
2605399150,14cyclictest0-21swapper/511:53:255
2605399150,14cyclictest0-21swapper/509:08:525
2605399150,14cyclictest0-21swapper/508:06:155
2605399150,13cyclictest0-21swapper/511:24:245
2605399150,13cyclictest0-21swapper/511:11:095
2605399150,1cyclictest0-21swapper/511:45:555
2605399150,1cyclictest0-21swapper/511:30:415
2605399150,1cyclictest0-21swapper/511:30:415
2605399150,1cyclictest0-21swapper/510:56:325
2605399150,1cyclictest0-21swapper/510:13:285
2604899159,5cyclictest12376-21irqstats_TLB12:30:144
2604899155,6cyclictest54-21ksoftirqd/409:50:134
26048991512,2cyclictest9298-21perl12:25:144
26048991511,3cyclictest0-21swapper/410:33:044
26048991510,3cyclictest0-21swapper/410:52:024
26048991510,2cyclictest54-21ksoftirqd/410:25:204
26048991510,2cyclictest191rcu_preempt08:15:204
2604899150,7cyclictest0-21swapper/411:15:204
2604899150,7cyclictest0-21swapper/411:15:204
2604899150,7cyclictest0-21swapper/409:45:504
2604899150,4cyclictest0-21swapper/407:16:124
2604899150,14cyclictest0-21swapper/411:10:174
2604899150,1cyclictest0-21swapper/410:55:184
2604499150,8cyclictest0-21swapper/308:32:173
2604499150,7cyclictest0-21swapper/312:04:533
2604499150,14cyclictest0-21swapper/307:52:523
26039991510,4cyclictest0-21swapper/210:43:212
26039991510,3cyclictest0-21swapper/212:06:132
2603999150,3cyclictest7674-21latency_hist09:30:012
2603999150,3cyclictest0-21swapper/210:10:202
2603999150,14cyclictest0-21swapper/212:36:002
2603999150,14cyclictest0-21swapper/211:43:312
2603999150,14cyclictest0-21swapper/208:15:312
26036991511,2cyclictest0-21swapper/112:17:211
2603699150,7cyclictest0-21swapper/108:38:101
2603199150,8cyclictest0-21swapper/011:56:160
2603199150,14cyclictest18767-21missed_timers10:45:170
2603199150,14cyclictest0-21swapper/010:16:010
2603199150,13cyclictest0-21swapper/011:15:510
2603199150,13cyclictest0-21swapper/011:15:510
2603199150,13cyclictest0-21swapper/011:02:110
2264652150,0pltrace0-21swapper/610:55:216
2219052150,11pltrace0-21swapper/008:03:450
2219052150,0pltrace0-21swapper/708:04:197
1878352150,0pltrace0-21swapper/709:53:017
3275052140,10pltrace0-21swapper/310:16:373
2779352140,11pltrace0-21swapper/608:14:366
2605799140,8cyclictest0-21swapper/611:13:216
2605799140,7cyclictest0-21swapper/610:08:286
2605799140,7cyclictest0-21swapper/607:43:156
2605799140,13cyclictest0-21swapper/608:21:436
2605799140,13cyclictest0-21swapper/608:21:436
2605399149,4cyclictest0-21swapper/510:40:165
2605399149,4cyclictest0-21swapper/510:32:525
2605399149,4cyclictest0-21swapper/510:27:045
2605399149,3cyclictest0-21swapper/507:43:595
2605399148,5cyclictest0-21swapper/512:30:355
2605399148,5cyclictest0-21swapper/512:06:265
2605399143,9cyclictest8964-21gzip11:25:195
2605399143,9cyclictest8964-21gzip11:25:195
26053991410,3cyclictest0-21swapper/509:29:165
2605399141,3cyclictest2543252pltrace11:00:225
2605399140,13cyclictest0-21swapper/511:44:215
2605399140,13cyclictest0-21swapper/510:36:505
2605399140,13cyclictest0-21swapper/507:39:445
2605399140,1cyclictest0-21swapper/507:56:295
2604899144,6cyclictest54-21ksoftirqd/410:05:224
2604899144,6cyclictest54-21ksoftirqd/407:40:134
2604899143,7cyclictest54-21ksoftirqd/410:45:134
2604899143,6cyclictest54-21ksoftirqd/410:10:154
26048991410,2cyclictest54-21ksoftirqd/411:05:124
26048991410,2cyclictest54-21ksoftirqd/408:45:144
2604899140,7cyclictest0-21swapper/409:11:324
26044991410,3cyclictest1146-21snmpd10:30:223
2604499140,3cyclictest0-21swapper/309:33:173
2604499140,1cyclictest0-21swapper/311:39:383
2604499140,1cyclictest0-21swapper/311:39:383
2603999147,6cyclictest1146-21snmpd11:53:132
2603999147,6cyclictest1146-21snmpd09:16:592
2603999146,7cyclictest1146-21snmpd09:23:592
2603999143,9cyclictest29317-21grep11:05:122
2603999143,5cyclictest24169-21aten_r4power_en11:55:102
26039991410,3cyclictest0-21swapper/208:48:032
2603999141,12cyclictest0-21swapper/210:50:002
2603999140,8cyclictest0-21swapper/207:30:102
2603999140,7cyclictest0-21swapper/212:23:232
2603999140,6cyclictest0-21swapper/212:29:532
2603999140,3cyclictest29202-21cat09:10:012
2603699148,5cyclictest0-21swapper/107:13:461
2603199149,3cyclictest0-21swapper/011:13:300
26031991410,2cyclictest0-21swapper/009:59:260
26031991410,2cyclictest0-21swapper/009:06:470
2603199140,6cyclictest0-21swapper/012:19:170
2603199140,3cyclictest0-21swapper/010:50:560
825352130,6pltrace0-21swapper/707:38:507
2717752130,0pltrace0-21swapper/710:08:147
2606199137,5cyclictest0-21swapper/707:23:347
26061991310,2cyclictest1055-17auditd11:50:007
2606199130,12cyclictest0-21swapper/712:20:187
2606199130,12cyclictest0-21swapper/711:29:217
2606199130,12cyclictest0-21swapper/711:29:217
2606199130,12cyclictest0-21swapper/708:42:317
2605799139,3cyclictest1063-17auditd07:30:006
2605799138,3cyclictest0-21swapper/610:53:056
26057991311,1cyclictest0-21swapper/609:15:476
2605799130,12cyclictest0-21swapper/610:27:486
2605799130,12cyclictest0-21swapper/610:16:206
2605799130,12cyclictest0-21swapper/609:00:206
2605799130,1cyclictest0-21swapper/609:48:006
2605399136,6cyclictest0-21swapper/510:23:225
2605399132,9cyclictest23520-21head08:00:135
2605399132,9cyclictest19323-21cat08:50:165
2605399132,6cyclictest1535752pltrace11:39:535
2605399132,6cyclictest1535752pltrace11:39:535
26053991311,1cyclictest0-21swapper/509:18:155
2605399131,10cyclictest29544-21cat08:10:175
2605399130,5cyclictest16721-21ntpdc08:45:155
2605399130,2cyclictest0-21swapper/509:31:175
2605399130,12cyclictest0-21swapper/512:15:035
2605399130,1cyclictest0-21swapper/512:21:125
2604899139,2cyclictest54-21ksoftirqd/412:05:214
2604899139,2cyclictest54-21ksoftirqd/410:20:214
2604899139,2cyclictest54-21ksoftirqd/409:00:124
2604899139,2cyclictest54-21ksoftirqd/408:10:144
2604899139,2cyclictest54-21ksoftirqd/407:45:174
2604899139,2cyclictest191rcu_preempt08:07:404
2604899137,3cyclictest54-21ksoftirqd/407:53:024
2604899136,6cyclictest16544-21grep11:40:164
2604899133,7cyclictest54-21ksoftirqd/408:50:114
2604899133,6cyclictest54-21ksoftirqd/412:10:114
2604899133,6cyclictest54-21ksoftirqd/411:25:184
2604899133,6cyclictest54-21ksoftirqd/411:25:184
2604899133,6cyclictest54-21ksoftirqd/410:15:174
26048991310,2cyclictest8752-21tune2fs09:30:134
26048991310,2cyclictest790-21df07:20:114
26048991310,2cyclictest32616-21cat09:15:124
2604899130,12cyclictest0-21swapper/411:47:214
26044991312,1cyclictest0-21swapper/310:20:193
26044991311,1cyclictest0-21swapper/312:25:093
2604499130,13cyclictest0-21swapper/308:40:453
2604499130,12cyclictest0-21swapper/308:55:393
2603999139,4cyclictest0-21swapper/208:27:012
2603999137,5cyclictest1146-21snmpd10:27:252
2603999135,7cyclictest1146-21snmpd07:52:022
2603999133,9cyclictest18140-21cut08:50:102
2603999133,4cyclictest9771-21tr11:29:382
2603999133,4cyclictest9771-21tr11:29:382
2603999133,4cyclictest26965-21irqstats_TLB11:00:162
2603999132,4cyclictest20884-21sed09:50:212
26039991311,1cyclictest0-21swapper/210:55:192
26039991310,2cyclictest0-21swapper/210:09:202
26039991310,2cyclictest0-21swapper/208:33:492
2603999131,5cyclictest26748-21expr07:10:122
2603999130,2cyclictest0-21swapper/209:00:002
2603999130,12cyclictest0-21swapper/208:13:162
2603999130,12cyclictest0-21swapper/207:45:002
2603699139,3cyclictest0-21swapper/107:50:451
2603699134,8cyclictest0-21swapper/110:12:211
26036991312,1cyclictest0-21swapper/109:50:131
2603699130,12cyclictest0-21swapper/108:02:461
26031991310,2cyclictest0-21swapper/012:08:210
2603199130,2cyclictest0-21swapper/011:26:150
2603199130,2cyclictest0-21swapper/011:26:150
2603199130,12cyclictest0-21swapper/010:22:040
2603199130,1cyclictest0-21swapper/008:31:430
205852130,9pltrace54-21ksoftirqd/409:23:364
1706952130,1pltrace19679-21taskset10:47:233
1490452130,9pltrace0-21swapper/108:45:451
1320752130,5pltrace0-21swapper/709:40:387
1320752130,1pltrace15254-21ls09:40:203
699552120,6pltrace0-21swapper/311:27:113
699552120,6pltrace0-21swapper/311:27:113
2932752120,9pltrace0-21swapper/312:05:073
2932752120,1pltrace29366-21crond12:05:001
2606199127,3cyclictest0-21swapper/711:56:437
2606199127,2cyclictest0-21swapper/709:47:487
26061991210,1cyclictest0-21swapper/708:20:027
26061991210,1cyclictest0-21swapper/708:20:027
2606199120,11cyclictest0-21swapper/712:15:467
26057991211,1cyclictest0-21swapper/608:55:436
26057991211,1cyclictest0-21swapper/608:40:206
2605799120,12cyclictest0-21swapper/608:45:196
2605799120,11cyclictest0-21swapper/608:51:116
2605799120,11cyclictest0-21swapper/607:37:316
2605399122,8cyclictest4654-21expr08:25:125
2605399121,9cyclictest11681-21grep09:35:145
2605399120,5cyclictest8912-21expr12:25:135
2605399120,5cyclictest8905-21grep08:30:215
2605399120,5cyclictest3254-21cpuspeed_turbos07:25:115
2605399120,5cyclictest30029-21cut11:05:155
2605399120,5cyclictest29596-21unixbench_singl10:05:235
2605399120,5cyclictest29484-21cpuspeed_turbos09:10:125
2605399120,5cyclictest25862-21chrt09:01:515
2605399120,5cyclictest2565-21cut09:20:115
2605399120,5cyclictest22926-21tail09:55:145
2605399120,5cyclictest21200-21grep10:50:135
2605399120,5cyclictest2073-21tail12:10:215
2605399120,5cyclictest17247-21cut07:50:115
2605399120,5cyclictest15658-21chrt12:35:195
2605399120,5cyclictest14322-21cat08:40:215
2605399120,5cyclictest1146-21snmpd10:45:135
2605399120,5cyclictest11098-21needreboot08:35:165
2605399120,4cyclictest1674-21grep10:15:135
2604899129,2cyclictest521rcuc/408:30:114
2604899129,2cyclictest4381-21cut08:25:114
2604899129,2cyclictest3275-21grep07:25:104
2604899129,2cyclictest3183-21timerandwakeup08:20:204
2604899129,2cyclictest3183-21timerandwakeup08:20:204
2604899129,2cyclictest28440-21cut12:00:184
2604899129,2cyclictest25853-21interrupts10:00:144
2604899129,2cyclictest22415-21expr09:55:124
2604899129,2cyclictest15478-21fschecks_time10:40:124
2604899129,2cyclictest1146-21snmpd08:55:104
2604899129,1cyclictest191rcu_preempt11:55:154
2604899127,2cyclictest191rcu_preempt08:00:134
2604899124,5cyclictest54-21ksoftirqd/409:25:154
2604899124,3cyclictest54-21ksoftirqd/409:35:174
2604899123,6cyclictest54-21ksoftirqd/409:40:174
2604899122,6cyclictest54-21ksoftirqd/411:20:194
2604899122,6cyclictest54-21ksoftirqd/411:00:114
2604499127,3cyclictest0-21swapper/307:14:073
26044991211,1cyclictest0-21swapper/311:45:483
26044991211,1cyclictest0-21swapper/311:40:273
26044991211,1cyclictest0-21swapper/310:09:383
26044991211,1cyclictest0-21swapper/310:01:033
26044991211,1cyclictest0-21swapper/309:24:183
26044991211,1cyclictest0-21swapper/307:55:533
26044991211,1cyclictest0-21swapper/307:30:293
2604499120,12cyclictest0-21swapper/309:18:483
2604499120,11cyclictest0-21swapper/311:56:513
2604499120,11cyclictest0-21swapper/309:52:283
2604499120,1cyclictest0-21swapper/312:37:273
2604499120,1cyclictest0-21swapper/311:34:193
2604499120,1cyclictest0-21swapper/311:34:183
2604499120,1cyclictest0-21swapper/308:04:093
2604499120,1cyclictest0-21swapper/307:40:503
2603999126,4cyclictest0-21swapper/210:01:092
2603999124,7cyclictest4981-21cstates11:20:122
2603999123,8cyclictest2093-21cut11:15:102
2603999123,8cyclictest2093-21cut11:15:102
2603999123,3cyclictest25368-21cat09:00:202
2603999123,2cyclictest21924-21sed10:50:202
2603999123,2cyclictest15962-21timerandwakeup07:45:202
2603999122,5cyclictest8778-21cut07:35:112
2603999122,3cyclictest32322-21fschecks_time11:10:142
2603999121,4cyclictest1595-21sed07:20:152
2603999121,4cyclictest10528-21cut11:30:112
2603999121,4cyclictest10528-21cut11:30:112
2603999120,5cyclictest8085-21cut09:30:112
2603999120,5cyclictest30436-21expr07:15:112
2603999120,5cyclictest25435-21cut08:05:102
2603999120,5cyclictest1369-21taskset08:20:082
2603999120,5cyclictest1369-21taskset08:20:082
2603999120,5cyclictest13207-21grep12:30:192
2603999120,5cyclictest12525-21taskset08:40:022
2603999120,5cyclictest1214-21ls10:15:112
2603999120,5cyclictest1146-21snmpd11:37:252
2603999120,5cyclictest1146-21snmpd11:37:252
2603999120,4cyclictest20512-21grep11:45:202
2603699129,2cyclictest15229-21expr10:40:111
2603699123,4cyclictest31018-21missed_timers12:05:161
26036991211,1cyclictest0-21swapper/111:20:141
26036991210,1cyclictest0-21swapper/111:50:211
26036991210,1cyclictest0-21swapper/108:16:281
2603699120,11cyclictest0-21swapper/112:22:531
2603699120,11cyclictest0-21swapper/111:12:211
2603699120,11cyclictest0-21swapper/110:55:281
2603199128,2cyclictest0-21swapper/012:38:320
2603199126,2cyclictest2544-21awk12:15:010
2603199126,2cyclictest1146-21snmpd11:50:310
2603199121,1cyclictest0-21swapper/010:31:310
2603199120,12cyclictest0-21swapper/008:54:420
2603199120,11cyclictest0-21swapper/009:39:040
2603199120,1cyclictest0-21swapper/011:39:220
2603199120,1cyclictest0-21swapper/011:39:220
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional