You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-03 - 11:48
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot6.osadl.org (updated Wed Jun 03, 2026 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25589521080,90pltrace0-21swapper/519:07:165
25589521010,93pltrace0-21swapper/219:06:282
25589521000,92pltrace0-21swapper/619:05:166
2558952830,67pltrace0-21swapper/119:07:191
2839126854,10sleep30-21swapper/319:08:123
2844426442,5sleep00-21swapper/019:08:570
290282630,1sleep10-21swapper/122:20:561
2848025945,10sleep70-21swapper/719:09:297
1658252590,10pltrace78-21ksoftirqd/700:31:087
2711325842,11sleep40-21swapper/419:05:164
719052530,6pltrace78-21ksoftirqd/721:59:447
2149452500,6pltrace78-21ksoftirqd/723:59:327
2149452500,6pltrace78-21ksoftirqd/723:59:327
28710994745,1cyclictest0-21swapper/222:11:262
1622552470,7pltrace78-21ksoftirqd/723:54:517
298752440,7pltrace78-21ksoftirqd/700:16:397
2627952440,7pltrace78-21ksoftirqd/700:03:317
196752440,3pltrace1984-21sh21:53:087
2018552430,7pltrace78-21ksoftirqd/722:50:447
113082430,0sleep60-21swapper/600:21:276
3085752400,3pltrace0-21swapper/700:09:317
298752390,2pltrace2991-21pltrace-run00:13:077
28882350,2sleep454-21ksoftirqd/423:39:424
1558052350,7pltrace78-21ksoftirqd/721:35:367
28727993432,1cyclictest0-21swapper/622:11:256
1658252340,0pltrace0-21swapper/700:28:167
2837952330,5pltrace0-21swapper/719:10:227
2018552330,0pltrace0-21swapper/722:48:267
2871899310,1cyclictest0-21swapper/421:15:014
2531152310,6pltrace78-21ksoftirqd/721:10:117
2476352310,9pltrace78-21ksoftirqd/722:55:037
605352300,0pltrace0-21swapper/723:09:227
1561452300,1pltrace16247-21ssh22:44:327
28722992926,2cyclictest601rcuc/522:11:255
644352280,0pltrace0-21swapper/721:25:477
605352270,6pltrace78-21ksoftirqd/723:11:577
2475952270,0pltrace0-21swapper/721:47:467
2018552270,4pltrace20189-21pltrace-run22:48:073
220152260,13pltrace0-21swapper/723:42:537
2872799240,4cyclictest0-21swapper/622:30:266
2871899240,12cyclictest23085-21sh00:35:124
2871899230,22cyclictest0-21swapper/422:46:294
152652230,1pltrace0-21swapper/723:04:547
28713992215,3cyclictest191rcu_preempt23:14:123
2101452220,0pltrace0-21swapper/723:24:227
321552210,17pltrace54-21ksoftirqd/419:27:144
2872299211,1cyclictest0-21swapper/523:10:205
157652210,0pltrace0-21swapper/721:19:097
1558052210,0pltrace0-21swapper/721:34:037
719052200,3pltrace0-21swapper/722:00:357
2871099202,14cyclictest191rcu_preempt23:07:142
28710992017,2cyclictest1146-21snmpd22:01:382
2870199200,8cyclictest1414452pltrace20:50:010
28713991916,2cyclictest25406-21taskset00:01:093
28713991916,2cyclictest12354-21taskset22:03:513
28713991914,2cyclictest46-21ksoftirqd/320:37:243
27092190,1sleep30-21swapper/323:39:183
1136052190,1pltrace13802-21unixbench-2d20:45:227
28718991815,2cyclictest0-21swapper/421:29:464
28710991815,2cyclictest31893-21ntpdc21:15:202
749252170,0pltrace0-21swapper/719:39:427
2872299171,15cyclictest0-21swapper/523:47:545
2872299171,15cyclictest0-21swapper/523:47:545
2871399178,6cyclictest46-21ksoftirqd/322:17:283
28713991714,2cyclictest4658-21latency21:55:153
28713991714,2cyclictest31230-21tr19:13:053
28713991714,2cyclictest29628-21seq20:15:383
28713991714,2cyclictest24331-21sshd20:09:283
28713991714,2cyclictest15033-21cstates20:50:123
28713991714,2cyclictest15033-21cstates20:50:123
28713991714,2cyclictest1146-21snmpd23:46:593
28713991714,2cyclictest1146-21snmpd23:46:593
28713991712,2cyclictest46-21ksoftirqd/323:55:123
28713991712,2cyclictest46-21ksoftirqd/323:55:123
28713991710,6cyclictest0-21swapper/322:34:323
28710991716,1cyclictest0-21swapper/223:50:162
28710991713,2cyclictest38-21ksoftirqd/221:00:312
2870599171,8cyclictest0-21swapper/121:37:461
2870599171,15cyclictest0-21swapper/123:11:241
2870199171,8cyclictest0-21swapper/000:15:280
2531152170,0pltrace0-21swapper/721:09:587
749252160,1pltrace0-21swapper/319:42:453
28731991611,3cyclictest0-21swapper/720:42:537
2873199160,2cyclictest32166-21idleruntime22:25:137
2872799160,4cyclictest0-21swapper/623:45:546
2872799160,4cyclictest0-21swapper/623:45:546
28722991614,2cyclictest62-21ksoftirqd/519:54:515
28722991611,4cyclictest26028-21/usr/sbin/munin21:45:155
28722991610,5cyclictest0-21swapper/500:16:045
28718991612,3cyclictest1146-21snmpd20:38:444
28713991613,2cyclictest5017-21sed21:20:223
28713991613,2cyclictest1178-21runrttasks19:48:303
28713991613,2cyclictest1146-21snmpd19:19:233
28713991611,2cyclictest46-21ksoftirqd/322:41:433
28713991610,5cyclictest0-21swapper/319:57:373
28713991610,2cyclictest46-21ksoftirqd/323:15:193
2871399161,8cyclictest0-21swapper/323:02:453
2871399161,3cyclictest0-21swapper/319:36:213
28710991615,1cyclictest0-21swapper/200:03:152
28710991614,1cyclictest32451-21sed22:25:162
28710991614,1cyclictest1146-21snmpd23:43:222
28710991613,2cyclictest32102-21/usr/sbin/munin00:10:152
28710991613,2cyclictest31391-21grep23:00:142
28710991613,2cyclictest27099-21expr23:30:132
28710991613,2cyclictest1474-21lldpd20:55:162
28710991613,2cyclictest1474-21lldpd20:55:152
28710991613,2cyclictest1146-21snmpd00:36:452
2870599160,15cyclictest0-21swapper/120:56:061
2870599160,15cyclictest0-21swapper/120:56:061
2870199161,8cyclictest0-21swapper/020:21:050
2870199160,15cyclictest0-21swapper/021:55:110
2870199160,15cyclictest0-21swapper/021:45:140
1094052160,0pltrace0-21swapper/723:17:287
753752150,11pltrace0-21swapper/100:19:411
36152150,0pltrace0-21swapper/719:21:287
2873199150,4cyclictest0-21swapper/719:19:307
2873199150,14cyclictest0-21swapper/720:59:417
2873199150,14cyclictest0-21swapper/720:59:417
2872799159,2cyclictest7465-21cpuspeed_turbos23:10:116
2872799150,8cyclictest0-21swapper/620:19:526
2872799150,7cyclictest0-21swapper/619:46:066
2872799150,14cyclictest0-21swapper/622:23:466
28722991510,3cyclictest0-21swapper/523:39:485
28722991510,3cyclictest0-21swapper/522:55:025
2872299150,7cyclictest0-21swapper/523:27:485
2872299150,4cyclictest0-21swapper/522:28:165
2872299150,14cyclictest0-21swapper/522:48:415
2872299150,14cyclictest0-21swapper/519:56:405
2872299150,14cyclictest0-21swapper/500:08:465
2871899155,9cyclictest0-21swapper/423:55:034
2871899155,9cyclictest0-21swapper/423:55:034
2871899154,10cyclictest0-21swapper/420:55:174
2871899154,10cyclictest0-21swapper/420:55:174
2871899150,7cyclictest0-21swapper/400:15:414
2871899150,4cyclictest0-21swapper/419:41:454
2871899150,3cyclictest0-21swapper/420:53:154
2871899150,3cyclictest0-21swapper/420:53:154
2871899150,14cyclictest0-21swapper/423:47:354
2871899150,14cyclictest0-21swapper/423:47:354
2871899150,14cyclictest0-21swapper/421:10:034
2871899150,14cyclictest0-21swapper/400:26:474
2871399156,5cyclictest0-21swapper/323:20:133
28713991513,1cyclictest4096-21grep23:40:143
28713991512,2cyclictest5172-21cut19:30:093
28713991512,2cyclictest474-21grep00:10:153
28713991512,2cyclictest26860-21sendmail23:08:203
28713991512,2cyclictest26177-21ntpq20:10:173
28713991512,2cyclictest23588-21missed_timers23:25:173
28713991512,2cyclictest22857-21netstat22:50:183
28713991512,2cyclictest20728-21fgrep20:00:203
28713991512,2cyclictest19116-21grep00:30:183
28713991512,2cyclictest18320-21expr22:10:123
28713991512,2cyclictest17203-21ls21:35:113
28713991512,2cyclictest14334-21cat22:05:153
28713991512,2cyclictest13045-21cut21:30:223
28713991512,2cyclictest1146-21snmpd22:36:443
28713991512,2cyclictest1076-21systemd-logind19:55:003
28713991512,2cyclictest10680-21cat00:20:243
28713991511,2cyclictest46-21ksoftirqd/320:26:513
28713991510,2cyclictest0-21swapper/322:55:183
28713991510,1cyclictest191rcu_preempt21:25:143
2871399151,7cyclictest0-21swapper/319:25:593
2871399150,8cyclictest0-21swapper/321:05:213
2871399150,8cyclictest0-21swapper/320:43:323
28710991513,2cyclictest1474-21lldpd20:50:462
28710991513,2cyclictest1474-21lldpd20:50:452
28710991513,2cyclictest1306-21expr19:20:082
28710991513,1cyclictest0-21swapper/200:25:142
28710991512,2cyclictest9818-21idleruntime00:20:172
28710991512,2cyclictest8079-21df22:35:122
28710991512,2cyclictest7800-21hostname21:25:132
28710991512,2cyclictest754-21cat23:35:212
28710991512,2cyclictest5985-21processes19:30:152
28710991512,2cyclictest38-21ksoftirqd/221:44:112
28710991512,2cyclictest31547-21cat20:20:162
28710991512,2cyclictest28748-21awk22:20:232
28710991512,2cyclictest25961-21aten_r4power_po21:10:132
28710991512,2cyclictest19655-21cat00:30:232
28710991512,2cyclictest19110-21strings23:20:182
28710991512,2cyclictest18987-21/usr/sbin/munin22:45:232
28710991512,2cyclictest18256-21expr23:55:132
28710991512,2cyclictest18256-21expr23:55:132
28710991512,2cyclictest16907-21df19:55:132
28710991512,2cyclictest1478-21lldpd23:28:472
28710991512,2cyclictest14199-21gltestperf22:05:152
28710991512,2cyclictest12751-21expr22:40:122
28710991512,2cyclictest10518-21ntpq20:40:182
28710991512,2cyclictest1002-21cpuspeed_turbos20:25:112
28710991512,1cyclictest191rcu_preempt20:19:412
28710991511,3cyclictest0-21swapper/222:32:032
28710991511,3cyclictest0-21swapper/220:00:042
2871099150,7cyclictest0-21swapper/222:18:152
2871099150,15cyclictest0-21swapper/221:20:132
2871099150,14cyclictest0-21swapper/223:19:282
2871099150,14cyclictest0-21swapper/219:52:152
2870599150,7cyclictest0-21swapper/123:56:161
2870599150,7cyclictest0-21swapper/123:56:161
2870599150,7cyclictest0-21swapper/119:58:141
2870599150,7cyclictest0-21swapper/100:10:401
28701991511,3cyclictest0-21swapper/021:02:160
2870199150,8cyclictest0-21swapper/022:45:140
2870199150,14cyclictest0-21swapper/022:57:500
220152150,11pltrace54-21ksoftirqd/423:42:454
220152150,10pltrace0-21swapper/023:42:510
194452150,0pltrace0-21swapper/722:32:047
672452140,0pltrace0-21swapper/723:45:437
672452140,0pltrace0-21swapper/723:45:437
2919052140,6pltrace31297-21perl21:15:150
2873199149,3cyclictest0-21swapper/721:00:507
28731991412,1cyclictest0-21swapper/722:19:237
28731991412,1cyclictest0-21swapper/722:11:277
2873199140,7cyclictest0-21swapper/723:30:467
28727991413,1cyclictest0-21swapper/600:05:256
2872799140,8cyclictest0-21swapper/620:30:006
2872799140,3cyclictest0-21swapper/600:30:526
2872799140,13cyclictest20462-21ssh23:56:216
2872799140,13cyclictest20462-21ssh23:56:216
2872799140,13cyclictest0-21swapper/623:00:136
2872799140,13cyclictest0-21swapper/620:55:196
2872799140,13cyclictest0-21swapper/620:55:196
2872799140,1cyclictest0-21swapper/621:53:406
2872299149,2cyclictest17048-21cpuspeed_turbos21:35:115
28722991413,1cyclictest0-21swapper/522:50:495
28722991410,3cyclictest0-21swapper/520:25:165
2872299141,12cyclictest0-21swapper/521:02:245
2872299140,7cyclictest0-21swapper/520:00:255
2872299140,3cyclictest0-21swapper/522:05:465
2872299140,14cyclictest0-21swapper/523:15:165
2872299140,14cyclictest0-21swapper/521:57:525
2872299140,13cyclictest0-21swapper/523:00:175
2872299140,1cyclictest0-21swapper/523:51:325
2872299140,1cyclictest0-21swapper/521:18:325
28718991413,1cyclictest0-21swapper/421:57:004
2871899140,3cyclictest0-21swapper/422:08:194
2871899140,3cyclictest0-21swapper/422:04:584
2871899140,13cyclictest0-21swapper/420:05:244
2871899140,13cyclictest0-21swapper/400:09:564
2871899140,1cyclictest0-21swapper/423:54:044
2871899140,1cyclictest0-21swapper/422:43:384
2871899140,1cyclictest0-21swapper/422:13:214
28713991411,2cyclictest32277-21sed21:15:223
28713991411,2cyclictest31972-21sed21:50:183
28713991411,2cyclictest28462-21runrttasks21:46:083
28713991411,2cyclictest27975-21cstates00:05:173
28713991411,2cyclictest26054-21cpuspeed_turbos21:10:123
28713991411,2cyclictest21521-21aten_r4power_vo21:40:113
28713991411,2cyclictest2086-21wc19:20:113
28713991411,2cyclictest1474-21lldpd20:56:153
28713991411,2cyclictest1474-21lldpd20:56:153
28713991411,2cyclictest12971-21cat23:50:143
28713991411,2cyclictest1146-21snmpd22:20:203
2871399140,1cyclictest0-21swapper/323:33:223
2871399140,1cyclictest0-21swapper/300:15:183
28710991413,1cyclictest0-21swapper/219:40:082
28710991412,1cyclictest0-21swapper/221:56:192
28710991411,2cyclictest32232-21aten_r4power_vo19:15:012
28710991411,2cyclictest29398-21expr19:10:112
28710991411,2cyclictest27893-21timerandwakeup21:45:222
28710991411,2cyclictest25107-21cstates20:10:122
28710991411,2cyclictest24024-21mktemp21:05:162
28710991411,2cyclictest12007-21sed19:45:152
28710991411,2cyclictest1178-21runrttasks21:37:262
28710991411,2cyclictest0-21swapper/222:50:032
28710991410,3cyclictest0-21swapper/223:45:372
28710991410,3cyclictest0-21swapper/223:45:372
2871099140,8cyclictest0-21swapper/223:12:482
2871099140,13cyclictest0-21swapper/221:53:442
2870599147,2cyclictest1065-17sedispatch22:55:011
28705991413,1cyclictest0-21swapper/121:56:121
28705991412,1cyclictest0-21swapper/122:55:171
2870599140,3cyclictest0-21swapper/119:12:051
2870599140,14cyclictest0-21swapper/100:06:311
2870599140,13cyclictest0-21swapper/123:00:241
2870599140,13cyclictest0-21swapper/119:16:261
2870599140,1cyclictest0-21swapper/123:50:401
2870599140,1cyclictest0-21swapper/122:37:291
2870599140,1cyclictest0-21swapper/122:00:241
2870599140,1cyclictest0-21swapper/121:17:491
28701991411,2cyclictest0-21swapper/022:00:220
2870199140,7cyclictest0-21swapper/022:06:380
2870199140,6cyclictest0-21swapper/000:10:490
2870199140,3cyclictest0-21swapper/000:28:160
2870199140,13cyclictest0-21swapper/019:16:030
2423052140,6pltrace24314-21taskset20:09:140
196752140,6pltrace0-21swapper/121:53:131
1306452140,6pltrace14399-21fschecks_time19:50:140
644352130,7pltrace0-21swapper/021:27:140
28731991312,1cyclictest0-21swapper/721:40:427
2873199130,12cyclictest0-21swapper/720:16:457
2872799135,7cyclictest0-21swapper/619:29:006
28727991312,1cyclictest0-21swapper/622:54:306
28727991311,1cyclictest1146-21snmpd20:54:056
28727991311,1cyclictest1146-21snmpd20:54:056
2872799130,12cyclictest0-21swapper/623:20:546
2872799130,12cyclictest0-21swapper/621:15:356
2872799130,1cyclictest0-21swapper/621:24:516
2872299138,3cyclictest0-21swapper/520:31:205
2872299137,5cyclictest5488-21unixbench_singl23:40:215
2872299130,13cyclictest0-21swapper/521:27:075
2872299130,12cyclictest0-21swapper/523:21:095
2872299130,12cyclictest0-21swapper/521:14:155
2872299130,12cyclictest0-21swapper/500:25:465
2871899135,4cyclictest14292-21sadc20:50:004
2871899130,13cyclictest0-21swapper/423:28:004
2871899130,13cyclictest0-21swapper/421:49:044
2871899130,13cyclictest0-21swapper/421:44:524
2871899130,12cyclictest0-21swapper/421:51:374
2871899130,12cyclictest0-21swapper/420:15:044
2871899130,12cyclictest0-21swapper/419:52:184
2871899130,1cyclictest0-21swapper/400:02:004
28713991311,1cyclictest23309-21cpuspeed_turbos00:35:133
28713991311,1cyclictest0-21swapper/320:20:123
28713991310,2cyclictest4547-21cat20:30:153
28713991310,2cyclictest32082-21expr22:25:133
28713991310,2cyclictest21639-21ntp_kernel_err21:00:183
28713991310,2cyclictest13435-21cpuspeed_turbos00:25:133
28713991310,2cyclictest11878-21taskset20:45:113
2871099139,3cyclictest0-21swapper/219:26:052
2871099136,4cyclictest0-21swapper/221:32:362
28710991312,1cyclictest0-21swapper/220:05:242
28710991311,1cyclictest29810-21ssh00:05:452
28710991310,2cyclictest1178-21runrttasks22:58:192
28710991310,2cyclictest1146-21snmpd00:17:072
28705991312,1cyclictest0-21swapper/121:47:461
28705991312,1cyclictest0-21swapper/121:11:301
28705991312,1cyclictest0-21swapper/121:06:291
2870599130,13cyclictest0-21swapper/123:45:091
2870599130,13cyclictest0-21swapper/123:45:091
2870599130,13cyclictest0-21swapper/123:40:371
2870599130,13cyclictest0-21swapper/122:16:161
2870599130,12cyclictest0-21swapper/122:08:171
2870199138,4cyclictest0-21swapper/019:21:090
2870199131,11cyclictest0-21swapper/021:51:170
2870199130,12cyclictest0-21swapper/020:38:390
2870199130,1cyclictest0-21swapper/000:35:470
2870199130,1cyclictest0-21swapper/000:02:360
2702252130,4pltrace0-21swapper/620:13:236
2550752130,6pltrace0-21swapper/022:20:590
194452130,5pltrace0-21swapper/122:31:531
1658252130,5pltrace0-21swapper/100:32:001
1622552130,5pltrace0-21swapper/523:55:265
1622552130,5pltrace0-21swapper/523:55:265
1558052130,8pltrace0-21swapper/621:35:506
1558052130,6pltrace0-21swapper/021:35:500
152652130,7pltrace0-21swapper/123:05:051
1306452130,0pltrace0-21swapper/719:48:287
1094052130,2pltrace15546-21chrt23:17:051
1075252130,6pltrace0-21swapper/022:42:270
1075252130,2pltrace15289-21ssh22:42:256
1028052130,6pltrace0-21swapper/019:45:550
674852120,3pltrace0-21swapper/119:35:201
644352120,5pltrace0-21swapper/121:27:191
299852120,8pltrace3126-21taskset20:29:511
2873199129,2cyclictest1146-21snmpd19:40:167
2872799129,2cyclictest5191-21fschecks_time19:30:096
2872799129,2cyclictest23951-21sed20:05:256
2872799129,2cyclictest13687-21missed_timers21:30:256
2872799127,3cyclictest0-21swapper/622:18:506
2872799127,3cyclictest0-21swapper/600:27:106
28727991211,1cyclictest0-21swapper/622:03:016
28727991210,1cyclictest840-21ssh22:25:316
28727991210,1cyclictest0-21swapper/622:36:226
28727991210,1cyclictest0-21swapper/619:50:156
2872799120,12cyclictest0-21swapper/600:19:016
2872799120,11cyclictest0-21swapper/622:06:166
2872799120,11cyclictest0-21swapper/619:21:416
2872799120,1cyclictest0-21swapper/621:56:236
2872299129,2cyclictest28060-21expr20:15:135
2872299129,2cyclictest1178-21runrttasks19:11:355
2872299127,3cyclictest0-21swapper/519:36:245
2872299126,5cyclictest0-21swapper/522:02:045
2872299126,5cyclictest0-21swapper/520:38:115
2872299122,8cyclictest0-21swapper/520:10:165
2872299122,8cyclictest0-21swapper/500:04:495
28722991211,1cyclictest0-21swapper/523:32:555
2871899129,2cyclictest1178-21runrttasks23:02:334
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional