You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-31 - 07:51
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot6.osadl.org (updated Mon Aug 31, 2026 00:44:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5973521110,59pltrace0-21swapper/719:05:137
597352950,91pltrace0-21swapper/619:05:226
95802900,0sleep20-21swapper/223:40:202
597352870,78pltrace0-21swapper/119:05:561
597352850,82pltrace0-21swapper/519:05:245
790727942,10sleep00-21swapper/019:05:210
623052780,1pltrace0-21swapper/020:58:360
597352780,73pltrace0-21swapper/319:05:183
872027258,10sleep20-21swapper/219:08:202
1361052690,7pltrace78-21ksoftirqd/722:46:327
218452630,0pltrace0-21swapper/723:06:177
883025844,10sleep40-21swapper/419:09:344
3030352580,7pltrace0-21swapper/722:31:117
2946952530,8pltrace78-21ksoftirqd/723:03:177
1176452510,6pltrace78-21ksoftirqd/723:43:457
1462252500,9pltrace78-21ksoftirqd/722:17:177
1090352490,6pltrace0-21swapper/700:14:447
3267452460,7pltrace78-21ksoftirqd/700:06:547
505852440,2pltrace0-21swapper/721:36:337
505852440,2pltrace0-21swapper/721:36:337
2418052440,6pltrace78-21ksoftirqd/722:57:327
3196752430,6pltrace78-21ksoftirqd/700:36:587
3131852430,1pltrace0-21swapper/722:01:557
132352430,9pltrace78-21ksoftirqd/723:37:557
1275652420,1pltrace0-21swapper/723:15:247
1275652410,2pltrace0-21swapper/723:14:507
849752400,0pltrace0-21swapper/722:42:027
1462252390,5pltrace78-21ksoftirqd/722:14:477
933152380,9pltrace78-21ksoftirqd/722:09:327
3131852380,2pltrace0-21swapper/721:59:077
2688452380,0pltrace0-21swapper/721:27:127
2688452380,0pltrace0-21swapper/721:27:127
909999370,2cyclictest0-21swapper/522:35:045
59912370,0sleep1281rcuc/123:06:061
2179252370,6pltrace78-21ksoftirqd/721:21:537
344452310,0pltrace0-21swapper/720:55:187
3267452300,1pltrace0-21swapper/700:04:577
2843752300,9pltrace78-21ksoftirqd/723:28:427
2765352290,1pltrace27957-21ssh23:59:087
1090352290,1pltrace15959-21ssh00:18:147
318252280,0sleep40-21swapper/420:45:154
1645252270,6pltrace78-21ksoftirqd/721:13:557
907999240,23cyclictest0-21swapper/022:42:260
907999240,23cyclictest0-21swapper/019:30:280
52142220,1sleep2361rcuc/223:36:112
1888052220,2pltrace0-21swapper/722:52:457
909599210,20cyclictest0-21swapper/420:27:394
9095992018,1cyclictest0-21swapper/423:41:494
2747852200,3pltrace0-21swapper/720:38:587
1803452200,4pltrace0-21swapper/719:25:207
9104991912,2cyclictest3196752pltrace00:35:116
910499190,18cyclictest14717-21ssh22:13:466
9099991817,1cyclictest611ktimers/523:00:105
9091991812,3cyclictest0-21swapper/300:11:583
9109991715,1cyclictest0-21swapper/723:53:477
910999171,8cyclictest0-21swapper/720:25:147
910499179,4cyclictest10284-21expr22:40:116
909199170,15cyclictest3793-21ssh21:31:273
9087991714,2cyclictest0-21swapper/221:31:262
9079991711,5cyclictest0-21swapper/022:37:500
3030352170,3pltrace0-21swapper/722:28:457
1704752170,0pltrace78-21ksoftirqd/723:49:237
1524652170,5pltrace54-21ksoftirqd/419:20:544
1183452170,12pltrace54-21ksoftirqd/421:09:494
933152160,5pltrace0-21swapper/522:12:285
910999160,8cyclictest0-21swapper/719:18:537
910499168,4cyclictest28174-21fschecks_time21:55:146
9095991611,3cyclictest0-21swapper/423:07:424
909199161,7cyclictest0-21swapper/300:07:433
909199161,14cyclictest0-21swapper/323:56:403
9087991611,4cyclictest13464-21chrt20:12:442
908799160,15cyclictest0-21swapper/222:13:462
9079991610,3cyclictest0-21swapper/022:00:090
907999161,3cyclictest0-21swapper/020:13:360
907999160,15cyclictest0-21swapper/021:25:590
907999160,15cyclictest0-21swapper/021:25:590
746252160,12pltrace0-21swapper/023:10:260
3131852160,10pltrace0-21swapper/322:02:553
2747852160,3pltrace0-21swapper/720:40:167
218452160,7pltrace0-21swapper/323:07:033
2134052160,11pltrace0-21swapper/000:24:200
910999159,2cyclictest11240-21/usr/sbin/munin20:10:127
910999159,1cyclictest125831gltestperf21:40:147
910999159,1cyclictest125831gltestperf21:40:147
910999154,6cyclictest78-21ksoftirqd/723:30:187
9109991513,1cyclictest0-21swapper/723:24:007
9109991512,2cyclictest18140-21crond19:25:007
9109991510,3cyclictest78-21ksoftirqd/721:00:197
9109991510,2cyclictest78-21ksoftirqd/719:50:127
9109991510,2cyclictest0-21swapper/722:37:597
910999150,8cyclictest0-21swapper/719:49:197
910999150,2cyclictest0-21swapper/721:50:147
910999150,14cyclictest0-21swapper/700:32:327
910499159,2cyclictest1993-21cut20:50:136
910499158,2cyclictest25556-21grep23:25:166
910499157,4cyclictest9359-21taskset00:11:016
910499157,4cyclictest24642-21idleruntime23:55:156
910499157,4cyclictest16873-21grep22:45:206
910499156,4cyclictest9737-21timerandwakeup23:40:216
910499155,2cyclictest191rcu_preempt00:05:156
9104991512,2cyclictest28739-21cut20:40:126
9099991513,1cyclictest0-21swapper/500:20:145
9099991512,2cyclictest14159-21iostat_ios23:45:165
9095991511,2cyclictest0-21swapper/422:23:494
909599151,7cyclictest0-21swapper/422:10:374
909599150,8cyclictest0-21swapper/423:03:274
909599150,8cyclictest0-21swapper/421:36:564
909599150,8cyclictest0-21swapper/421:36:564
909599150,8cyclictest0-21swapper/420:56:044
909599150,7cyclictest0-21swapper/422:57:504
909599150,7cyclictest0-21swapper/419:54:294
909599150,14cyclictest0-21swapper/423:39:014
909599150,14cyclictest0-21swapper/421:16:294
909599150,14cyclictest0-21swapper/421:13:174
909599150,14cyclictest0-21swapper/420:33:184
909599150,14cyclictest0-21swapper/400:32:474
909599150,14cyclictest0-21swapper/400:16:504
909599150,13cyclictest0-21swapper/400:06:164
9091991513,1cyclictest0-21swapper/319:30:013
9091991512,2cyclictest0-21swapper/321:00:583
909199151,3cyclictest0-21swapper/320:16:213
909199150,14cyclictest0-21swapper/323:38:283
909199150,14cyclictest0-21swapper/320:07:553
909199150,14cyclictest0-21swapper/320:03:253
909199150,14cyclictest0-21swapper/300:01:403
908799157,3cyclictest8152-21ntp_states21:00:172
9087991514,1cyclictest0-21swapper/223:31:132
9087991513,1cyclictest0-21swapper/223:07:452
9087991511,1cyclictest191rcu_preempt22:35:222
908799150,14cyclictest0-21swapper/200:13:202
908799150,1cyclictest0-21swapper/221:42:072
908799150,1cyclictest0-21swapper/221:42:072
908799150,1cyclictest0-21swapper/200:06:322
907999159,5cyclictest0-21swapper/021:05:110
907999157,6cyclictest0-21swapper/023:19:210
9079991513,1cyclictest0-21swapper/000:15:000
9079991510,3cyclictest0-21swapper/021:47:420
907999151,3cyclictest0-21swapper/023:22:540
907999151,13cyclictest0-21swapper/022:28:550
907999150,8cyclictest0-21swapper/021:02:280
907999150,8cyclictest0-21swapper/020:06:420
907999150,7cyclictest0-21swapper/021:59:540
907999150,2cyclictest0-21swapper/021:50:220
907999150,14cyclictest0-21swapper/021:39:250
907999150,14cyclictest0-21swapper/021:39:250
904052150,3pltrace0-21swapper/721:06:207
849752150,2pltrace13068-21ssh22:42:433
746252150,1pltrace0-21swapper/323:10:193
2074652150,1pltrace78-21ksoftirqd/721:48:387
1911852150,6pltrace78-21ksoftirqd/720:23:477
1911852150,11pltrace0-21swapper/020:28:150
1704752150,9pltrace1047-21NetworkManager23:50:003
1704752150,10pltrace0-21swapper/523:50:095
1462252150,12pltrace54-21ksoftirqd/422:15:414
910999148,2cyclictest32217-21awk20:45:207
910999147,5cyclictest1146-21snmpd00:26:157
910999147,4cyclictest78-21ksoftirqd/719:55:217
910999144,4cyclictest0-21swapper/700:24:107
9109991412,1cyclictest24513-21seq22:22:467
9109991411,2cyclictest9318-21/usr/sbin/munin19:10:177
9109991411,2cyclictest8034-21seq20:04:407
9109991411,2cyclictest27238-21cstates19:40:127
910499147,4cyclictest319-21cpuspeed_turbos22:00:116
910499147,2cyclictest32590-21cpu19:50:116
910499147,2cyclictest30779-21if_err_eth023:30:156
910499146,5cyclictest4240-21fgrep23:35:216
910499146,4cyclictest13259-21grep19:15:126
910499146,1cyclictest191rcu_preempt20:46:076
9104991411,2cyclictest7288-21gltestperf21:35:156
9104991411,2cyclictest7288-21gltestperf21:35:156
9104991411,2cyclictest29150-21latency00:30:186
9099991413,1cyclictest0-21swapper/523:30:325
9099991413,1cyclictest0-21swapper/523:28:365
9099991413,1cyclictest0-21swapper/521:44:005
9099991413,1cyclictest0-21swapper/521:44:005
9099991413,1cyclictest0-21swapper/521:26:595
9099991413,1cyclictest0-21swapper/521:26:595
9099991413,1cyclictest0-21swapper/521:19:035
9099991413,1cyclictest0-21swapper/519:20:145
909999140,14cyclictest0-21swapper/522:42:465
909999140,14cyclictest0-21swapper/522:09:455
909999140,14cyclictest0-21swapper/500:31:155
909999140,14cyclictest0-21swapper/500:17:585
909999140,1cyclictest0-21swapper/523:19:125
909999140,1cyclictest0-21swapper/522:24:295
909599148,3cyclictest0-21swapper/420:05:084
909599140,7cyclictest0-21swapper/423:14:274
909599140,6cyclictest0-21swapper/422:28:194
909599140,14cyclictest0-21swapper/419:25:224
909599140,13cyclictest0-21swapper/421:49:364
909599140,12cyclictest0-21swapper/422:45:084
909599140,12cyclictest0-21swapper/422:32:444
909599140,1cyclictest0-21swapper/421:31:204
9091991411,2cyclictest1146-21snmpd23:27:243
909199141,12cyclictest0-21swapper/321:18:023
909199140,7cyclictest0-21swapper/319:16:013
909199140,13cyclictest0-21swapper/300:32:183
909199140,1cyclictest0-21swapper/323:42:263
908799146,4cyclictest38-21ksoftirqd/221:50:002
9087991413,1cyclictest0-21swapper/223:58:542
9087991413,1cyclictest0-21swapper/223:10:302
9087991413,1cyclictest0-21swapper/222:15:572
9087991413,1cyclictest0-21swapper/222:05:552
9087991413,1cyclictest0-21swapper/221:05:222
9087991413,1cyclictest0-21swapper/219:50:242
9087991413,1cyclictest0-21swapper/200:33:252
9087991413,1cyclictest0-21swapper/200:17:522
9087991411,2cyclictest1146-21snmpd21:28:142
9087991411,2cyclictest1146-21snmpd21:28:142
9087991411,2cyclictest1146-21snmpd19:45:322
908799140,14cyclictest0-21swapper/223:51:182
908799140,14cyclictest0-21swapper/221:56:122
908799140,1cyclictest0-21swapper/222:52:022
908799140,1cyclictest0-21swapper/222:43:082
9083991413,1cyclictest0-21swapper/121:31:341
9083991413,1cyclictest0-21swapper/121:20:241
9083991413,1cyclictest0-21swapper/100:21:231
9083991411,2cyclictest24301-21sh22:22:241
908399140,1cyclictest0-21swapper/123:16:001
908399140,1cyclictest0-21swapper/122:35:411
908399140,1cyclictest0-21swapper/100:30:091
907999148,5cyclictest0-21swapper/020:45:340
9079991412,1cyclictest0-21swapper/023:37:070
9079991411,2cyclictest0-21swapper/023:54:340
9079991411,2cyclictest0-21swapper/022:11:280
907999140,7cyclictest0-21swapper/023:59:160
907999140,7cyclictest0-21swapper/019:26:160
907999140,14cyclictest0-21swapper/023:46:500
907999140,1cyclictest0-21swapper/021:42:590
907999140,1cyclictest0-21swapper/021:42:590
907999140,1cyclictest0-21swapper/021:13:500
514852140,2pltrace5266-21sleep20:00:000
3196752140,8pltrace3232-21ssh00:35:481
3196752140,8pltrace0-21swapper/000:35:480
267232140,1sleep40-21swapper/423:25:404
2418052140,6pltrace0-21swapper/322:55:503
2418052140,3pltrace27794-21ssh22:55:450
2418052140,2pltrace26679-21latency22:55:175
2134052140,10pltrace54-21ksoftirqd/400:24:104
1462252140,8pltrace18346-21ssh22:15:565
1462252140,8pltrace15-21ksoftirqd/022:15:130
132352140,3pltrace5835-21sh23:37:165
9109991311,1cyclictest26219-21taskset19:36:227
9109991311,1cyclictest1125-21aten_r4power_cu20:50:117
9109991311,1cyclictest0-21swapper/721:15:177
9109991311,1cyclictest0-21swapper/719:30:137
910499137,2cyclictest8131-21grep21:00:166
910499135,4cyclictest0-21swapper/623:50:116
910499135,1cyclictest191rcu_preempt20:02:176
910499133,7cyclictest70-21ksoftirqd/620:23:196
910499132,7cyclictest70-21ksoftirqd/620:05:026
910499132,7cyclictest70-21ksoftirqd/619:44:536
9104991310,2cyclictest9070-21cstates23:10:126
9104991310,2cyclictest4882-21taskset23:05:176
9104991310,2cyclictest452-21sed22:30:196
9104991310,2cyclictest22812-21sensors_fan22:20:216
9104991310,2cyclictest20093-21cstates20:25:126
9104991310,2cyclictest1178-21runrttasks19:28:066
9104991310,2cyclictest1178-21runrttasks00:03:166
910499131,6cyclictest70-21ksoftirqd/623:00:116
910499131,4cyclictest70-21ksoftirqd/622:55:256
910499130,13cyclictest0-21swapper/621:45:116
910499130,12cyclictest5001-21ssh21:33:326
910499130,12cyclictest0-21swapper/621:50:136
9099991312,1cyclictest0-21swapper/523:23:545
9099991312,1cyclictest0-21swapper/522:26:255
9099991312,1cyclictest0-21swapper/521:46:455
9099991312,1cyclictest0-21swapper/521:20:125
909999130,13cyclictest0-21swapper/520:14:575
909999130,1cyclictest0-21swapper/521:34:075
909999130,1cyclictest0-21swapper/520:16:255
909999130,1cyclictest0-21swapper/520:01:025
9095991312,1cyclictest0-21swapper/423:18:014
909599130,12cyclictest0-21swapper/422:51:384
909599130,12cyclictest0-21swapper/419:44:504
909599130,12cyclictest0-21swapper/400:27:564
909599130,1cyclictest0-21swapper/422:43:424
909599130,1cyclictest0-21swapper/400:13:044
9091991312,1cyclictest0-21swapper/321:48:263
9091991312,1cyclictest0-21swapper/321:42:573
9091991312,1cyclictest0-21swapper/321:42:573
9091991311,1cyclictest0-21swapper/323:30:473
909199130,13cyclictest0-21swapper/323:24:083
909199130,13cyclictest0-21swapper/323:16:413
909199130,13cyclictest0-21swapper/323:02:073
909199130,13cyclictest0-21swapper/322:23:053
909199130,13cyclictest0-21swapper/321:51:123
909199130,13cyclictest0-21swapper/300:22:253
908799137,2cyclictest5437-21processes20:55:192
908799135,4cyclictest27571-21ssh22:55:222
908799134,5cyclictest38-21ksoftirqd/200:25:142
9087991312,1cyclictest0-21swapper/222:20:582
9087991312,1cyclictest0-21swapper/221:13:272
9087991311,1cyclictest0-21swapper/221:50:152
9087991310,2cyclictest9532-21tr19:10:122
9087991310,2cyclictest7982-21sensors_temp21:35:222
9087991310,2cyclictest7982-21sensors_temp21:35:222
9087991310,2cyclictest6182-21diskstats20:00:142
9087991310,2cyclictest3627-21tune2fs19:55:142
9087991310,2cyclictest32069-21sed23:00:152
9087991310,2cyclictest25692-21smart_sda19:35:202
9087991310,2cyclictest24150-21sed20:30:222
9087991310,2cyclictest19954-21ntp_states19:25:182
9087991310,2cyclictest14557-21df23:15:122
9087991310,2cyclictest13924-21perl23:45:142
9087991310,2cyclictest12789-21perl19:15:022
9087991310,2cyclictest1146-21snmpd20:49:312
9087991310,2cyclictest1146-21snmpd19:20:272
9087991310,1cyclictest191rcu_preempt23:23:022
908799130,13cyclictest0-21swapper/222:27:372
9083991312,1cyclictest0-21swapper/123:23:261
9083991312,1cyclictest0-21swapper/123:04:101
9083991312,1cyclictest0-21swapper/122:27:061
9083991312,1cyclictest0-21swapper/122:11:331
9083991312,1cyclictest0-21swapper/121:51:191
9083991312,1cyclictest0-21swapper/121:43:041
9083991312,1cyclictest0-21swapper/121:43:041
9083991312,1cyclictest0-21swapper/100:04:431
9083991310,2cyclictest0-21swapper/120:46:191
908399130,13cyclictest0-21swapper/123:54:021
908399130,13cyclictest0-21swapper/123:26:111
908399130,13cyclictest0-21swapper/123:13:141
908399130,13cyclictest0-21swapper/122:47:201
908399130,13cyclictest0-21swapper/121:35:561
908399130,13cyclictest0-21swapper/121:35:561
908399130,1cyclictest0-21swapper/120:40:431
9079991312,1cyclictest0-21swapper/022:54:420
9079991311,1cyclictest0-21swapper/023:31:180
907999130,13cyclictest0-21swapper/023:29:210
907999130,12cyclictest0-21swapper/023:40:400
907999130,1cyclictest0-21swapper/021:24:510
746252130,1pltrace10885-21ssh23:10:245
505852130,7pltrace5602-21ssh21:34:320
3197252130,1pltrace0-21swapper/119:52:361
3030352130,1pltrace30711-21ssh22:29:183
2639352130,10pltrace0-21swapper/019:41:030
2179252130,5pltrace0-21swapper/321:21:373
1803452130,5pltrace0-21swapper/319:27:363
1704752130,1pltrace0-21swapper/323:50:043
1462252130,5pltrace0-21swapper/322:15:563
132352130,5pltrace0-21swapper/123:37:201
1033952130,9pltrace54-21ksoftirqd/421:43:274
1033952130,9pltrace54-21ksoftirqd/421:43:274
910999129,2cyclictest22523-21cpu20:30:137
910999129,2cyclictest15432-21sshd20:15:167
910999129,2cyclictest10044-21sensors20:05:197
9109991211,1cyclictest0-21swapper/721:30:117
910499129,2cyclictest4676-21fschecks_time20:55:136
910499129,2cyclictest3457-21cut19:55:146
910499129,2cyclictest27914-21wc22:25:216
910499129,2cyclictest24774-21switchtime21:20:226
910499129,2cyclictest22936-21taskset22:51:296
910499129,2cyclictest22390-21/usr/sbin/munin20:30:186
910499129,2cyclictest21897-21cut19:30:146
910499129,2cyclictest20933-21sensors23:20:216
910499129,2cyclictest18885-21uname21:15:166
910499129,2cyclictest16542-21fschecks_count22:15:136
910499129,2cyclictest16104-21cstates19:20:156
910499129,2cyclictest12600-21cat00:15:146
910499129,2cyclictest1178-21runrttasks23:18:326
910499129,2cyclictest1146-21snmpd23:49:346
910499129,2cyclictest1146-21snmpd21:05:466
910499129,2cyclictest1146-21snmpd19:45:326
910499126,2cyclictest13754-21missed_timers21:10:176
910499126,2cyclictest11955-21munin-node00:28:156
910499124,5cyclictest0-21swapper/621:40:196
910499124,5cyclictest0-21swapper/621:40:196
909999129,2cyclictest2422-21perl00:05:145
9099991211,1cyclictest0-21swapper/521:37:415
9099991211,1cyclictest0-21swapper/521:37:415
9099991211,1cyclictest0-21swapper/520:50:345
9099991211,1cyclictest0-21swapper/500:01:185
909999120,11cyclictest0-21swapper/519:45:105
909599127,4cyclictest7271-21forks21:00:124
909599126,5cyclictest0-21swapper/421:50:354
9095991211,1cyclictest0-21swapper/423:59:194
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional