You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-06 - 12:33
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot6.osadl.org (updated Sat Jun 06, 2026 00:44:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24387521140,49pltrace0-21swapper/719:05:127
27174521120,108pltrace0-21swapper/519:10:005
24387521080,103pltrace0-21swapper/619:05:146
199182970,0sleep30-21swapper/322:40:203
2438752810,77pltrace0-21swapper/219:05:162
405152790,2pltrace0-21swapper/719:24:597
2077452780,0pltrace0-21swapper/619:54:596
2700127157,10sleep30-21swapper/319:07:483
2717452690,62pltrace0-21swapper/119:09:581
9802660,1sleep40-21swapper/421:11:514
2711925941,14sleep00-21swapper/019:09:310
2695525743,10sleep40-21swapper/419:07:094
2726452540,7pltrace78-21ksoftirqd/700:35:437
1819852490,6pltrace78-21ksoftirqd/723:52:317
2319352470,2pltrace0-21swapper/723:21:337
451252460,9pltrace78-21ksoftirqd/723:39:557
2539252450,0pltrace0-21swapper/721:41:077
393052430,7pltrace78-21ksoftirqd/723:03:167
1254852430,10pltrace78-21ksoftirqd/722:38:157
3175652420,7pltrace78-21ksoftirqd/722:58:427
1364252420,5pltrace78-21ksoftirqd/723:48:487
1703352410,6pltrace78-21ksoftirqd/722:08:537
238952390,8pltrace78-21ksoftirqd/721:16:237
208752390,8pltrace78-21ksoftirqd/721:52:437
1299452390,0pltrace0-21swapper/723:14:177
2233952380,8pltrace78-21ksoftirqd/722:48:407
712452370,0pltrace0-21swapper/721:20:117
3076052370,6pltrace78-21ksoftirqd/722:24:507
2994452370,6pltrace78-21ksoftirqd/721:48:587
2777352370,7pltrace78-21ksoftirqd/723:27:277
2616652370,8pltrace78-21ksoftirqd/722:16:147
1774452370,3pltrace0-21swapper/722:43:047
714852350,8pltrace78-21ksoftirqd/721:56:057
1630152350,7pltrace78-21ksoftirqd/721:32:097
52652340,0pltrace78-21ksoftirqd/700:09:237
1851852340,7pltrace78-21ksoftirqd/700:28:417
850452330,9pltrace78-21ksoftirqd/723:09:137
1797152320,8pltrace78-21ksoftirqd/723:18:587
2854352310,2pltrace30276-21processes00:00:187
2740799250,16cyclictest0-21swapper/400:22:394
293052240,5pltrace0-21swapper/722:25:157
2740799210,20cyclictest0-21swapper/423:34:414
2742199200,14cyclictest0-21swapper/719:16:537
2741699200,19cyclictest0-21swapper/623:10:186
2233952200,10pltrace0-21swapper/222:48:412
1349252200,1pltrace13919-21aten_r4power_en20:40:127
962352190,3pltrace0-21swapper/719:39:437
748552190,0pltrace0-21swapper/722:34:217
2159352190,9pltrace25682-21ssh22:13:552
2159352190,11pltrace25686-21ssh22:13:566
1299452190,11pltrace16555-21ssh23:12:592
2746452180,4pltrace0-21swapper/721:05:427
962352170,1pltrace9625-21pltrace-run19:34:594
791952170,3pltrace0-21swapper/720:30:217
27407991711,3cyclictest0-21swapper/421:46:514
27386991712,3cyclictest0-21swapper/020:42:560
2726452170,11pltrace0-21swapper/600:39:426
1519952170,0pltrace0-21swapper/719:49:177
513152160,0pltrace0-21swapper/720:26:047
278322160,0sleep70-21swapper/723:59:167
27421991611,4cyclictest0-21swapper/720:16:507
2742199161,14cyclictest0-21swapper/719:29:087
27407991611,3cyclictest0-21swapper/420:51:534
27407991610,5cyclictest0-21swapper/422:38:094
2740799160,8cyclictest0-21swapper/420:00:364
2740799160,15cyclictest0-21swapper/422:25:004
2740199161,4cyclictest0-21swapper/319:53:523
2740199160,15cyclictest0-21swapper/322:27:293
2739199167,5cyclictest25981-21cpuspeed_turbos21:40:101
27386991611,3cyclictest0-21swapper/022:00:010
2738699161,8cyclictest0-21swapper/019:47:540
132162160,1sleep562-21ksoftirqd/523:44:415
1241652160,0pltrace0-21swapper/722:00:077
293052150,7pltrace5595-21ssh22:25:392
2916852150,2pltrace0-21swapper/720:11:507
2742199155,4cyclictest0-21swapper/720:04:457
27421991513,1cyclictest0-21swapper/722:51:537
2742199150,8cyclictest0-21swapper/700:20:237
2742199150,7cyclictest0-21swapper/720:48:097
2742199150,14cyclictest0-21swapper/721:12:427
27416991513,1cyclictest0-21swapper/622:06:306
2741699150,14cyclictest0-21swapper/623:26:186
2741699150,14cyclictest0-21swapper/622:01:096
27407991511,3cyclictest0-21swapper/420:56:054
27407991510,3cyclictest0-21swapper/420:50:004
2740799150,8cyclictest0-21swapper/420:42:574
2740799150,7cyclictest0-21swapper/423:26:084
2740799150,4cyclictest0-21swapper/419:40:254
2740799150,14cyclictest0-21swapper/422:57:014
2740799150,14cyclictest0-21swapper/422:16:294
2740799150,14cyclictest0-21swapper/420:38:104
2740799150,14cyclictest0-21swapper/419:36:444
2740799150,13cyclictest0-21swapper/421:15:144
2740799150,13cyclictest0-21swapper/420:31:284
2740799150,13cyclictest0-21swapper/420:20:494
2740799150,13cyclictest0-21swapper/400:28:394
2740199159,4cyclictest0-21swapper/323:10:313
27401991512,2cyclictest0-21swapper/322:09:503
2740199151,7cyclictest0-21swapper/300:15:063
2740199150,3cyclictest0-21swapper/322:57:013
2740199150,3cyclictest0-21swapper/321:00:583
2740199150,1cyclictest0-21swapper/300:25:173
2739699151,2cyclictest0-21swapper/221:11:022
2739699150,1cyclictest0-21swapper/221:58:502
2739199158,4cyclictest12732-21cat20:35:211
2739199157,4cyclictest17852-21iostat_ios21:30:151
2739199150,1cyclictest0-21swapper/122:33:231
2739199150,1cyclictest0-21swapper/100:25:161
27386991513,1cyclictest0-21swapper/000:17:380
27386991511,3cyclictest0-21swapper/022:36:230
27386991510,3cyclictest0-21swapper/021:30:000
2738699151,8cyclictest0-21swapper/019:37:290
2738699150,14cyclictest0-21swapper/023:34:030
2738699150,14cyclictest0-21swapper/023:29:340
2738699150,14cyclictest0-21swapper/023:21:370
968852140,3pltrace13615-21taskset00:19:276
451252140,2pltrace7259-21sh23:36:286
2994452140,1pltrace0-21swapper/521:45:215
2742199148,2cyclictest11094-21/usr/sbin/munin20:35:227
2742199145,5cyclictest0-21swapper/700:11:507
27421991410,3cyclictest0-21swapper/723:32:587
2742199140,8cyclictest0-21swapper/723:43:557
2742199140,1cyclictest0-21swapper/700:19:097
27416991413,1cyclictest0-21swapper/623:51:046
27416991413,1cyclictest0-21swapper/623:44:276
27416991413,1cyclictest0-21swapper/621:28:366
27416991413,1cyclictest0-21swapper/621:19:326
27416991413,1cyclictest0-21swapper/600:22:586
27416991413,1cyclictest0-21swapper/600:07:456
27416991412,1cyclictest0-21swapper/620:30:196
27416991411,2cyclictest1146-21snmpd21:11:036
2741699141,13cyclictest0-21swapper/621:36:136
2741699141,13cyclictest0-21swapper/621:36:136
2741699140,14cyclictest0-21swapper/600:28:386
2741699140,1cyclictest0-21swapper/623:09:476
2741699140,1cyclictest0-21swapper/622:26:246
2741699140,1cyclictest0-21swapper/622:18:286
2741699140,1cyclictest0-21swapper/621:55:486
2741699140,1cyclictest0-21swapper/621:24:346
2741299148,2cyclictest25202-21cat21:00:105
2741299147,2cyclictest24352-21grep23:55:145
2741299146,4cyclictest8870-21wc21:55:165
2741299146,4cyclictest20097-21ssh21:33:175
27412991412,1cyclictest0-21swapper/519:30:135
27412991411,2cyclictest25689-21taskset23:20:525
27412991411,2cyclictest22293-21taskset00:29:155
27412991411,2cyclictest17586-21runrttasks23:49:145
27412991411,2cyclictest14837-21vmstat19:40:205
2741299140,13cyclictest0-21swapper/500:20:545
2740799149,4cyclictest1055-17auditd19:49:594
27407991413,1cyclictest54-21ksoftirqd/420:15:164
2740799141,12cyclictest0-21swapper/419:53:424
2740799140,3cyclictest0-21swapper/423:15:434
2740799140,13cyclictest0-21swapper/423:42:224
2740799140,12cyclictest0-21swapper/423:57:204
2740799140,1cyclictest0-21swapper/400:30:174
2740199149,1cyclictest62461fschecks_time20:25:123
27401991413,1cyclictest0-21swapper/323:06:023
27401991413,1cyclictest0-21swapper/300:11:073
27401991412,1cyclictest0-21swapper/322:18:163
27401991412,1cyclictest0-21swapper/300:30:433
2740199140,3cyclictest0-21swapper/323:00:313
2740199140,14cyclictest0-21swapper/300:02:523
2740199140,13cyclictest0-21swapper/322:11:373
2740199140,13cyclictest0-21swapper/321:36:003
2740199140,13cyclictest0-21swapper/321:36:003
2740199140,13cyclictest0-21swapper/321:19:203
2740199140,13cyclictest0-21swapper/319:17:393
2740199140,13cyclictest0-21swapper/300:23:453
2740199140,1cyclictest0-21swapper/323:46:303
2740199140,1cyclictest0-21swapper/323:24:493
2740199140,1cyclictest0-21swapper/322:37:123
2740199140,1cyclictest0-21swapper/321:10:063
2740199140,1cyclictest0-21swapper/300:35:053
27396991413,1cyclictest0-21swapper/222:39:192
27396991413,1cyclictest0-21swapper/221:37:472
27396991413,1cyclictest0-21swapper/221:37:472
27396991413,1cyclictest0-21swapper/219:25:222
2739699140,14cyclictest0-21swapper/223:19:092
2739699140,14cyclictest0-21swapper/200:18:342
2739199147,2cyclictest30267-21munin-node19:15:021
2739199147,2cyclictest10922-21grep19:35:141
2739199146,4cyclictest28468-21idleruntime00:35:141
2739199146,4cyclictest26627-21sensors_volt21:00:191
27391991413,1cyclictest0-21swapper/123:05:461
2739199140,14cyclictest0-21swapper/123:54:401
2739199140,14cyclictest0-21swapper/123:48:021
2739199140,14cyclictest0-21swapper/123:15:101
2739199140,14cyclictest0-21swapper/122:51:021
2739199140,14cyclictest0-21swapper/122:41:191
2739199140,14cyclictest0-21swapper/121:38:201
2739199140,14cyclictest0-21swapper/121:38:201
2739199140,14cyclictest0-21swapper/100:15:141
2739199140,14cyclictest0-21swapper/100:08:161
2739199140,14cyclictest0-21swapper/100:01:181
2739199140,1cyclictest0-21swapper/123:13:231
2739199140,1cyclictest0-21swapper/122:46:111
2739199140,1cyclictest0-21swapper/100:20:541
27386991413,1cyclictest0-21swapper/000:37:430
27386991412,1cyclictest0-21swapper/023:15:470
27386991412,1cyclictest0-21swapper/022:40:100
27386991412,1cyclictest0-21swapper/021:16:480
2738699140,14cyclictest0-21swapper/023:38:560
2738699140,14cyclictest0-21swapper/022:29:100
2738699140,14cyclictest0-21swapper/022:19:270
2738699140,14cyclictest0-21swapper/022:02:070
2738699140,1cyclictest0-21swapper/022:51:400
2738699140,1cyclictest0-21swapper/021:20:030
2738699140,1cyclictest0-21swapper/000:11:580
2738699140,1cyclictest0-21swapper/000:06:180
2636752140,4pltrace0-21swapper/720:06:527
2233952140,3pltrace26282-21chrt22:48:373
2077452140,0pltrace0-21swapper/719:55:147
155212140,0sleep60-21swapper/622:36:076
451252130,2pltrace7285-21ssh23:36:302
393052130,1pltrace0-21swapper/623:00:486
27421991312,1cyclictest0-21swapper/700:34:517
2742199130,13cyclictest0-21swapper/721:29:497
2742199130,1cyclictest0-21swapper/721:35:387
2742199130,1cyclictest0-21swapper/721:35:387
27416991312,1cyclictest0-21swapper/621:52:156
27416991310,2cyclictest1076-21systemd-logind19:59:596
2741699130,1cyclictest0-21swapper/623:22:446
2741299139,2cyclictest191rcu_preempt21:53:365
2741299139,1cyclictest191rcu_preempt21:20:535
2741299136,2cyclictest16630-21if_err_eth019:45:145
27412991312,1cyclictest0-21swapper/523:00:135
27412991310,2cyclictest817-21unixbench_singl22:20:225
27412991310,2cyclictest6815-21open_inodes00:10:185
27412991310,2cyclictest5868-21ntp_states19:25:185
27412991310,2cyclictest29442-21timerandwakeup19:10:215
27412991310,2cyclictest29045-21iostat23:25:155
27412991310,2cyclictest2675-21cat00:05:215
27412991310,2cyclictest25642-21cat20:00:215
27412991310,2cyclictest23755-21switchtime22:10:205
27412991310,2cyclictest22741-21grep21:35:165
27412991310,2cyclictest22741-21grep21:35:165
27412991310,2cyclictest22464-21missed_timers19:55:165
27412991310,2cyclictest20481-21taskset19:51:055
27412991310,2cyclictest19491-21tail22:05:235
27412991310,2cyclictest18373-21taskset23:50:095
27412991310,2cyclictest16287-21tr21:29:595
27412991310,2cyclictest14889-21basename20:40:145
27412991310,2cyclictest1178-21runrttasks22:58:335
2741299130,13cyclictest0-21swapper/522:39:305
2740799138,3cyclictest0-21swapper/420:12:284
2740799131,5cyclictest4529-21aten_r4power_vo19:25:114
2740799131,4cyclictest13903-21fschecks_count23:10:134
2740799130,6cyclictest0-21swapper/420:29:574
2740799130,4cyclictest24733-21sed22:45:224
2740799130,13cyclictest0-21swapper/423:20:204
2740799130,12cyclictest3933-21grep23:05:014
2740799130,12cyclictest0-21swapper/423:49:354
2740799130,12cyclictest0-21swapper/421:56:304
2740799130,12cyclictest0-21swapper/421:50:484
2740799130,1cyclictest0-21swapper/421:23:454
2740799130,1cyclictest0-21swapper/400:36:554
2740199138,2cyclictest0-21swapper/323:36:583
27401991312,1cyclictest0-21swapper/323:15:263
27401991312,1cyclictest0-21swapper/322:21:203
27401991312,1cyclictest0-21swapper/322:02:433
27401991312,1cyclictest0-21swapper/319:20:213
27401991310,2cyclictest12731-21fschecks_count21:25:133
27401991310,2cyclictest0-21swapper/319:32:483
2740199130,7cyclictest0-21swapper/321:44:123
2740199130,1cyclictest0-21swapper/320:32:323
27396991312,1cyclictest0-21swapper/223:57:022
27396991312,1cyclictest0-21swapper/222:34:272
27396991312,1cyclictest0-21swapper/222:06:272
27396991312,1cyclictest0-21swapper/200:29:062
27396991312,1cyclictest0-21swapper/200:08:022
2739699130,13cyclictest0-21swapper/223:07:002
2739699130,13cyclictest0-21swapper/222:15:502
2739699130,13cyclictest0-21swapper/221:47:402
2739699130,1cyclictest0-21swapper/223:23:112
2739699130,1cyclictest0-21swapper/221:16:542
2739699130,1cyclictest0-21swapper/200:00:352
2739199137,2cyclictest8512-21gltestperf21:55:131
2739199137,2cyclictest14097-21missed_timers19:40:151
2739199136,2cyclictest6374-21unixbench_multi19:25:221
27391991312,1cyclictest0-21swapper/121:26:411
2739199130,13cyclictest0-21swapper/122:20:361
2739199130,12cyclictest0-21swapper/121:08:061
2739199130,1cyclictest0-21swapper/121:13:541
27386991311,1cyclictest0-21swapper/022:48:060
27386991311,1cyclictest0-21swapper/000:33:200
2738699130,2cyclictest0-21swapper/021:36:550
2738699130,2cyclictest0-21swapper/021:36:550
2738699130,2cyclictest0-21swapper/020:25:580
2738699130,13cyclictest0-21swapper/023:47:220
2738699130,13cyclictest0-21swapper/021:47:530
2738699130,13cyclictest0-21swapper/021:10:200
2738699130,13cyclictest0-21swapper/000:00:380
2738699130,12cyclictest0-21swapper/023:00:210
2738699130,12cyclictest0-21swapper/021:09:450
2738699130,1cyclictest0-21swapper/021:51:270
2738699130,1cyclictest0-21swapper/021:30:530
1630152130,2pltrace20181-21sshd21:33:306
1364252130,10pltrace16818-21ssh23:47:296
52652120,1pltrace0-21swapper/300:09:223
2994452120,1pltrace0-21swapper/321:47:173
2742199127,3cyclictest0-21swapper/722:10:597
2742199127,3cyclictest0-21swapper/721:03:157
2741699129,2cyclictest1076-21systemd-logind00:00:006
2741699129,2cyclictest0-21swapper/620:35:206
2741699127,4cyclictest28342-21gltestperf22:50:146
27416991211,1cyclictest0-21swapper/622:40:306
2741699120,12cyclictest0-21swapper/619:28:376
2741699120,1cyclictest0-21swapper/620:19:276
2741299129,2cyclictest6604-21sed23:35:205
2741299129,2cyclictest29355-21fschecks_count00:00:125
2741299129,2cyclictest28008-21missed_timers20:05:155
2741299129,2cyclictest22903-21cpu22:45:125
2741299129,2cyclictest1178-21runrttasks23:11:595
2741299126,2cyclictest31992-21missed_timers21:10:165
2741299126,2cyclictest30514-21ssh22:19:305
2741299125,2cyclictest5748-21cpuspeed_turbos20:25:115
2741299125,2cyclictest3696-21fschecks_time21:15:135
2741299125,2cyclictest30052-21ssh22:51:435
2741299125,2cyclictest21182-21ssh23:16:575
2741299120,12cyclictest0-21swapper/523:07:015
2740799129,2cyclictest0-21swapper/421:37:214
2740799129,2cyclictest0-21swapper/421:37:214
2740799122,2cyclictest17228-21expr21:30:124
2740799121,4cyclictest28800-21tail21:05:134
2740799121,3cyclictest13475-21ls21:25:164
2740799120,11cyclictest0-21swapper/421:43:154
2740799120,11cyclictest0-21swapper/400:02:554
2740799120,1cyclictest0-21swapper/422:12:584
2740199129,2cyclictest24248-21tr20:00:123
2740199129,2cyclictest14179-21tr20:40:113
2740199128,3cyclictest4581-21unixbench_singl21:50:213
2740199126,5cyclictest0-21swapper/323:31:173
2740199126,5cyclictest0-21swapper/320:57:533
2740199126,5cyclictest0-21swapper/319:58:373
2740199126,5cyclictest0-21swapper/319:47:433
27401991211,1cyclictest0-21swapper/323:44:433
27401991211,1cyclictest0-21swapper/321:30:103
27401991211,1cyclictest0-21swapper/320:24:363
27401991211,1cyclictest0-21swapper/319:42:313
2740199120,1cyclictest0-21swapper/320:11:103
2739699129,2cyclictest0-21swapper/220:05:222
27396991211,1cyclictest0-21swapper/222:53:052
27396991211,1cyclictest0-21swapper/221:22:542
2739699120,12cyclictest0-21swapper/221:25:102
2739699120,11cyclictest0-21swapper/220:57:582
2739699120,1cyclictest0-21swapper/220:15:032
2739199129,2cyclictest23295-21tr00:30:111
2739199129,2cyclictest23290-21taskset19:56:271
2739199129,2cyclictest16034-21taskset20:41:451
2739199126,2cyclictest7320-21ssh00:10:201
2739199126,2cyclictest29705-21threads23:25:201
2739199126,2cyclictest15942-21cstates19:45:131
2739199126,2cyclictest10350-21iostat23:40:131
27391991211,1cyclictest0-21swapper/123:37:501
27391991211,1cyclictest0-21swapper/122:36:281
27391991211,1cyclictest0-21swapper/122:15:441
27391991211,1cyclictest0-21swapper/122:03:161
27391991211,1cyclictest0-21swapper/121:22:381
2738699127,2cyclictest0-21swapper/019:52:210
2738699126,5cyclictest0-21swapper/019:19:470
2738699122,8cyclictest0-21swapper/020:36:320
27386991211,1cyclictest0-21swapper/022:32:140
27386991211,1cyclictest0-21swapper/022:05:410
27386991211,1cyclictest0-21swapper/020:50:150
27386991211,1cyclictest0-21swapper/000:23:180
27386991210,1cyclictest0-21swapper/022:24:180
27386991210,1cyclictest0-21swapper/020:33:540
1797152120,3pltrace21734-21taskset23:17:546
1519952120,1pltrace0-21swapper/219:47:412
1241652120,1pltrace0-21swapper/222:00:132
3196052110,7pltrace1178-21runrttasks20:15:560
2742199115,5cyclictest0-21swapper/720:24:417
27421991110,1cyclictest0-21swapper/720:57:457
2742199110,1cyclictest0-21swapper/719:30:387
2741699118,2cyclictest23357-21cut00:30:116
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional