You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-07 - 05:39
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot6.osadl.org (updated Mon Jul 07, 2025 00:44:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21396521010,93pltrace0-21swapper/119:08:271
21396521000,92pltrace0-21swapper/519:08:345
3005952940,4pltrace30063-21pltrace-run21:18:086
1861052940,90pltrace0-21swapper/219:05:162
1499752800,0pltrace0-21swapper/219:53:072
1861052790,75pltrace0-21swapper/619:05:156
2144627763,10sleep30-21swapper/319:08:463
2149326641,5sleep40-21swapper/419:09:274
2124626248,10sleep70-21swapper/719:06:177
2128626046,10sleep00-21swapper/019:06:500
385052340,5pltrace0-21swapper/719:35:217
73942300,1sleep3441rcuc/321:35:153
73942300,1sleep3441rcuc/321:35:153
2494852300,4pltrace0-21swapper/700:05:167
2173399240,23cyclictest0-21swapper/300:07:063
2173099220,15cyclictest0-21swapper/220:49:092
261252200,0pltrace0-21swapper/723:25:237
2172199190,1cyclictest0-21swapper/022:00:220
1995552190,2pltrace191rcu_preempt22:01:457
1546252190,0pltrace0-21swapper/722:52:277
2338952180,4pltrace0-21swapper/720:10:157
2172699181,2cyclictest0-21swapper/121:23:131
1097152180,0pltrace0-21swapper/723:42:047
85852170,0pltrace0-21swapper/700:20:147
37852170,3pltrace0-21swapper/721:23:507
21741991711,5cyclictest0-21swapper/519:41:465
2174199171,8cyclictest0-21swapper/523:21:145
2173799172,14cyclictest0-21swapper/419:30:014
21737991712,3cyclictest0-21swapper/420:39:484
2173799171,8cyclictest0-21swapper/419:43:284
2173799171,4cyclictest0-21swapper/421:10:314
2173799170,2cyclictest0-21swapper/422:10:134
2173799170,12cyclictest0-21swapper/421:55:014
2173799170,12cyclictest0-21swapper/400:04:184
2172199171,15cyclictest0-21swapper/022:43:310
1159852170,0pltrace0-21swapper/721:45:157
21749991611,3cyclictest0-21swapper/722:34:127
2174999161,8cyclictest0-21swapper/722:26:097
2174999161,14cyclictest0-21swapper/720:07:387
2174699161,7cyclictest0-21swapper/622:24:576
2174699160,15cyclictest0-21swapper/622:00:206
21741991611,3cyclictest0-21swapper/522:09:485
21741991611,3cyclictest0-21swapper/520:56:545
21741991610,5cyclictest0-21swapper/523:02:135
2174199160,4cyclictest0-21swapper/523:13:415
2173799160,5cyclictest0-21swapper/423:16:144
2173799160,4cyclictest20280-21tail20:00:244
2173799160,4cyclictest10300-21cstates19:45:134
2173799160,4cyclictest0-21swapper/423:03:134
2173799160,14cyclictest20028-21tr23:55:154
21733991611,3cyclictest0-21swapper/323:07:563
21733991611,3cyclictest0-21swapper/321:27:163
2173399161,4cyclictest0-21swapper/322:09:543
2173399161,14cyclictest0-21swapper/323:00:253
2173099161,4cyclictest0-21swapper/221:12:532
21726991610,5cyclictest0-21swapper/123:10:351
21726991610,5cyclictest0-21swapper/120:49:241
2172699161,14cyclictest0-21swapper/123:42:451
21721991611,3cyclictest0-21swapper/000:04:510
21721991610,3cyclictest0-21swapper/021:34:490
2172199161,8cyclictest0-21swapper/019:56:010
1995552160,1pltrace0-21swapper/322:01:353
1159852160,1pltrace0-21swapper/521:46:205
492852150,0pltrace0-21swapper/220:37:452
3067552150,4pltrace31347-21/usr/sbin/munin19:25:136
3067552150,0pltrace0-21swapper/119:25:131
21749991510,3cyclictest0-21swapper/721:42:507
21749991510,2cyclictest0-21swapper/723:00:007
2174999150,7cyclictest0-21swapper/720:17:227
2174999150,7cyclictest0-21swapper/719:16:457
2174999150,7cyclictest0-21swapper/700:04:067
2174999150,14cyclictest0-21swapper/719:24:097
2174699151,7cyclictest0-21swapper/620:33:476
2174699150,14cyclictest0-21swapper/622:00:016
2174699150,14cyclictest0-21swapper/620:38:226
2174199159,5cyclictest0-21swapper/519:39:195
21741991510,3cyclictest0-21swapper/521:28:165
2174199151,7cyclictest0-21swapper/520:32:225
2174199150,7cyclictest0-21swapper/523:44:455
2174199150,7cyclictest0-21swapper/520:00:005
2174199150,14cyclictest0-21swapper/521:11:195
2174199150,14cyclictest0-21swapper/520:15:595
2173799159,5cyclictest0-21swapper/419:13:234
21737991511,2cyclictest0-21swapper/421:35:434
21737991511,2cyclictest0-21swapper/421:35:434
21737991510,4cyclictest0-21swapper/421:29:464
2173799151,2cyclictest0-21swapper/420:10:234
2173799150,8cyclictest0-21swapper/422:30:304
2173799150,7cyclictest0-21swapper/419:37:224
2173799150,4cyclictest0-21swapper/421:09:524
2173799150,14cyclictest0-21swapper/423:10:194
2173799150,14cyclictest0-21swapper/420:56:574
2173799150,11cyclictest0-21swapper/423:25:034
2173799150,1cyclictest0-21swapper/420:45:054
2173399159,5cyclictest0-21swapper/300:17:013
2173399150,8cyclictest0-21swapper/321:19:373
2173399150,7cyclictest0-21swapper/320:43:093
2173399150,4cyclictest0-21swapper/320:24:443
2173399150,14cyclictest0-21swapper/323:29:173
2173399150,14cyclictest0-21swapper/322:17:003
2173399150,13cyclictest0-21swapper/320:05:163
2173099150,8cyclictest0-21swapper/223:59:122
2173099150,8cyclictest0-21swapper/221:33:522
2173099150,14cyclictest0-21swapper/223:43:002
21726991510,3cyclictest0-21swapper/119:59:311
2172699150,14cyclictest0-21swapper/123:05:131
2172699150,14cyclictest0-21swapper/122:32:241
2172199150,7cyclictest0-21swapper/020:30:260
2167352150,1pltrace0-21swapper/621:05:136
2167352150,1pltrace0-21swapper/121:05:121
1267852150,3pltrace0-21swapper/722:46:587
710452140,9pltrace8522-21gltestperf22:35:152
2385552140,0pltrace0-21swapper/123:03:271
2174999149,3cyclictest0-21swapper/700:18:197
2174999140,7cyclictest0-21swapper/720:46:527
2174999140,7cyclictest0-21swapper/720:43:307
2174999140,7cyclictest0-21swapper/720:21:227
2174999140,7cyclictest0-21swapper/719:46:517
2174999140,7cyclictest0-21swapper/719:33:527
2174999140,7cyclictest0-21swapper/700:11:557
2174999140,3cyclictest0-21swapper/722:20:017
2174699141,12cyclictest0-21swapper/621:14:106
2174699140,7cyclictest0-21swapper/621:46:416
2174199149,3cyclictest0-21swapper/522:23:245
2174199149,3cyclictest0-21swapper/520:28:415
2174199140,7cyclictest0-21swapper/500:32:135
2174199140,3cyclictest0-21swapper/500:22:015
2174199140,13cyclictest0-21swapper/521:21:235
2173799143,10cyclictest11432-21cat22:40:164
2173799142,4cyclictest15767-21tr19:55:144
2173799142,10cyclictest7661-21cut00:30:164
2173799142,10cyclictest5461-21switchtime21:30:234
2173799141,12cyclictest0-21swapper/419:31:044
2173799140,4cyclictest3557-21taskset00:21:184
2173799140,4cyclictest13704-21chrt23:42:394
2173799140,3cyclictest5591-21uname00:25:384
2173799140,3cyclictest23534-21latency_hist20:10:004
2173799140,2cyclictest0-21swapper/423:05:264
2173799140,2cyclictest0-21swapper/422:30:014
2173799140,14cyclictest0-21swapper/421:52:164
2173799140,13cyclictest30721-21cat00:15:004
2173799140,13cyclictest0-21swapper/400:07:374
2173799140,10cyclictest0-21swapper/421:46:564
2173399140,7cyclictest0-21swapper/321:41:413
2173399140,13cyclictest0-21swapper/322:38:413
2173099149,3cyclictest0-21swapper/221:52:172
2173099140,7cyclictest0-21swapper/219:27:522
2173099140,6cyclictest0-21swapper/220:44:212
2173099140,2cyclictest0-21swapper/222:25:162
2172699149,4cyclictest1028-17auditd22:25:011
2172699141,12cyclictest0-21swapper/100:28:431
2172699140,8cyclictest0-21swapper/100:01:271
2172699140,7cyclictest0-21swapper/120:17:471
2172699140,7cyclictest0-21swapper/119:39:311
2172199147,6cyclictest1166-21runrttasks20:48:330
21721991411,2cyclictest1125-21polkitd22:50:010
1934152140,0pltrace0-21swapper/323:55:003
1934152140,0pltrace0-21swapper/123:55:001
1778452140,0pltrace0-21swapper/319:58:583
323652130,0pltrace0-21swapper/321:32:353
3175352130,0pltrace0-21swapper/620:23:266
3005952130,1pltrace0-21swapper/021:18:330
2943252130,0pltrace0-21swapper/123:16:591
2385552130,0pltrace0-21swapper/023:03:290
2174999130,12cyclictest0-21swapper/723:10:297
2174999130,12cyclictest0-21swapper/722:08:487
2174699138,2cyclictest0-21swapper/622:42:106
2174699137,5cyclictest0-21swapper/619:48:166
2174699135,6cyclictest0-21swapper/620:06:536
21746991312,1cyclictest0-21swapper/620:55:156
2174699130,12cyclictest0-21swapper/619:34:406
2174199138,3cyclictest0-21swapper/523:58:125
2174199136,5cyclictest31194-21/usr/sbin/munin19:25:215
2173799133,9cyclictest8811-21fschecks_count20:40:184
2173799133,9cyclictest18083-21basename23:50:164
2173799133,9cyclictest13810-21fschecks_count22:45:134
2173799133,8cyclictest0-21swapper/422:16:394
2173799132,9cyclictest26791-21aten_r4power_vo20:15:124
2173799132,4cyclictest19941-21latency22:55:174
2173799132,10cyclictest1750-21tr23:20:214
21737991310,3cyclictest0-21swapper/422:36:284
2173799131,5cyclictest30327-21cut20:20:154
2173799131,2cyclictest0-21swapper/420:30:184
2173799131,1cyclictest0-21swapper/421:40:234
2173799130,5cyclictest6434-21cat23:30:154
2173799130,5cyclictest24650-21needreboot22:05:174
2173799130,5cyclictest10883-21chrt23:37:104
2173799130,5cyclictest10670-21fschecks_time00:35:184
2173799130,4cyclictest28775-21expr19:20:134
2173799130,4cyclictest19724-21expr21:00:164
2173799130,2cyclictest0-21swapper/422:00:194
2173799130,2cyclictest0-21swapper/421:15:144
2173799130,2cyclictest0-21swapper/420:25:154
2173399139,3cyclictest0-21swapper/321:20:283
2173399138,3cyclictest0-21swapper/300:33:273
21733991311,1cyclictest0-21swapper/323:15:203
2173399130,12cyclictest0-21swapper/320:35:293
2173399130,12cyclictest0-21swapper/319:40:043
2173399130,12cyclictest0-21swapper/319:39:163
21730991312,1cyclictest0-21swapper/221:35:132
21730991312,1cyclictest0-21swapper/221:35:132
2172699138,3cyclictest0-21swapper/123:59:451
2172699138,2cyclictest0-21swapper/122:16:541
2172699133,8cyclictest0-21swapper/123:31:021
21726991311,1cyclictest0-21swapper/121:44:281
2172699130,12cyclictest0-21swapper/121:32:271
2172699130,1cyclictest0-21swapper/123:35:131
2172199138,3cyclictest0-21swapper/019:25:090
2172199133,8cyclictest0-21swapper/000:37:100
21721991312,1cyclictest0-21swapper/023:07:280
21721991311,1cyclictest0-21swapper/019:20:290
2172199130,13cyclictest0-21swapper/020:28:090
2172199130,12cyclictest0-21swapper/019:40:140
2172199130,12cyclictest0-21swapper/019:36:430
2172199130,12cyclictest0-21swapper/019:13:530
2167352130,0pltrace0-21swapper/221:05:142
1778452130,0pltrace0-21swapper/719:58:597
1654352130,0pltrace0-21swapper/723:51:147
1328852130,1pltrace0-21swapper/020:51:460
1328852130,0pltrace0-21swapper/720:51:287
1328852130,0pltrace0-21swapper/620:51:346
1159852130,0pltrace0-21swapper/121:46:571
85852120,0pltrace0-21swapper/300:21:023
539552120,0pltrace0-21swapper/623:29:326
492852120,2pltrace0-21swapper/720:35:337
3067552120,0pltrace0-21swapper/719:25:147
2943252120,1pltrace0-21swapper/723:16:597
2385552120,0pltrace0-21swapper/223:03:312
2174999120,11cyclictest0-21swapper/721:54:017
2174999120,11cyclictest0-21swapper/721:13:537
2174699129,2cyclictest0-21swapper/623:23:086
2174699127,3cyclictest0-21swapper/620:01:186
2174699126,4cyclictest0-21swapper/622:55:196
2174699123,8cyclictest0-21swapper/622:51:316
2174699120,11cyclictest0-21swapper/620:40:536
2174199129,2cyclictest0-21swapper/500:39:475
2174199127,4cyclictest0-21swapper/522:18:095
2174199127,3cyclictest0-21swapper/521:59:145
2174199126,5cyclictest0-21swapper/522:13:395
2174199126,5cyclictest0-21swapper/519:48:585
2173799120,5cyclictest32181-21cstates22:20:144
2173799120,5cyclictest31220-21cat00:15:134
2173799120,5cyclictest30930-21tr21:20:204
2173799120,5cyclictest25800-21cpuspeed_turbos19:15:144
2173799120,5cyclictest16330-21cat22:50:124
2173799120,5cyclictest14351-21wget23:45:104
2173799120,5cyclictest14291-21cstates20:50:134
2173799120,5cyclictest13098-21cstates19:50:134
2173399127,4cyclictest14362-21switchtime19:50:223
2173399127,3cyclictest0-21swapper/321:07:203
2173399126,5cyclictest0-21swapper/319:48:103
2173099127,2cyclictest0-21swapper/219:57:072
2173099126,5cyclictest0-21swapper/222:21:032
2173099120,11cyclictest0-21swapper/219:47:132
2173099120,1cyclictest0-21swapper/220:12:582
2172699128,3cyclictest14469-21fschecks_count20:50:151
21726991211,1cyclictest0-21swapper/121:12:251
2172699120,11cyclictest0-21swapper/122:35:091
2172699120,11cyclictest0-21swapper/121:02:521
2172199129,2cyclictest0-21swapper/000:10:130
2172199126,5cyclictest0-21swapper/023:25:440
21721991210,1cyclictest0-21swapper/023:50:130
21721991210,1cyclictest0-21swapper/021:57:030
21721991210,1cyclictest0-21swapper/020:10:000
2172199120,11cyclictest0-21swapper/023:17:400
2172199120,11cyclictest0-21swapper/022:59:130
2172199120,1cyclictest0-21swapper/020:11:000
1328852120,0pltrace0-21swapper/220:51:352
1206752120,3pltrace1132-21snmpd00:38:501
2385552110,0pltrace0-21swapper/623:03:316
2174999119,1cyclictest0-21swapper/723:00:087
2174999116,3cyclictest0-21swapper/721:33:287
2174999115,5cyclictest0-21swapper/722:23:367
2174999115,5cyclictest0-21swapper/721:35:347
2174999115,5cyclictest0-21swapper/721:35:347
2174999110,9cyclictest0-21swapper/721:29:047
2174999110,11cyclictest0-21swapper/723:59:147
2174999110,11cyclictest0-21swapper/720:57:057
2174999110,10cyclictest0-21swapper/719:54:167
2174999110,1cyclictest0-21swapper/721:05:117
2174999110,1cyclictest0-21swapper/700:32:457
2174699119,1cyclictest0-21swapper/622:05:026
2174699118,2cyclictest0-21swapper/623:31:226
2174699114,3cyclictest8496-21/usr/sbin/munin00:30:226
2174699112,8cyclictest0-21swapper/600:39:386
2174699110,9cyclictest0-21swapper/621:39:566
2174699110,9cyclictest0-21swapper/621:39:566
2174699110,1cyclictest0-21swapper/623:58:036
2174699110,1cyclictest0-21swapper/622:30:276
2174699110,1cyclictest0-21swapper/622:13:476
2174199119,1cyclictest0-21swapper/520:08:395
2174199116,3cyclictest0-21swapper/500:02:215
2174199115,4cyclictest17311-21crond21:55:005
2174199113,7cyclictest0-21swapper/519:19:235
2174199112,8cyclictest0-21swapper/522:34:105
2174199112,8cyclictest0-21swapper/519:20:235
2174199111,8cyclictest0-21swapper/519:14:155
2174199110,1cyclictest0-21swapper/522:26:365
2174199110,1cyclictest0-21swapper/500:10:425
2173399119,1cyclictest0-21swapper/322:23:123
2173399116,4cyclictest1038-21systemd-logind22:30:013
2173399116,3cyclictest0-21swapper/323:36:293
2173399116,3cyclictest0-21swapper/320:19:383
2173399110,1cyclictest0-21swapper/323:57:163
2173399110,1cyclictest0-21swapper/322:53:283
2173399110,1cyclictest0-21swapper/321:00:573
2173099119,1cyclictest0-21swapper/220:00:492
2173099118,2cyclictest1028-17auditd20:27:272
2173099117,3cyclictest1132-21snmpd22:15:062
2173099117,3cyclictest1132-21snmpd00:09:492
2173099115,5cyclictest0-21swapper/200:33:102
21730991110,1cyclictest0-21swapper/221:44:262
21730991110,1cyclictest0-21swapper/220:31:062
21730991110,1cyclictest0-21swapper/220:15:332
2173099110,1cyclictest0-21swapper/222:53:542
2173099110,1cyclictest0-21swapper/221:56:252
2172699118,2cyclictest1166-21runrttasks19:15:171
2172699118,2cyclictest1028-17auditd19:50:011
2172699118,2cyclictest0-21swapper/121:25:241
2172699118,2cyclictest0-21swapper/119:30:171
2172699117,3cyclictest0-21swapper/120:11:521
21726991110,1cyclictest0-21swapper/123:22:061
21726991110,1cyclictest0-21swapper/122:47:171
21726991110,1cyclictest0-21swapper/122:43:151
21726991110,1cyclictest0-21swapper/121:37:501
21726991110,1cyclictest0-21swapper/121:37:501
21726991110,1cyclictest0-21swapper/120:58:491
21726991110,1cyclictest0-21swapper/120:42:471
21726991110,1cyclictest0-21swapper/119:54:321
2172699110,11cyclictest0-21swapper/121:56:271
2172699110,11cyclictest0-21swapper/121:51:061
2172699110,1cyclictest0-21swapper/120:30:581
2172699110,1cyclictest0-21swapper/120:05:131
2172199118,2cyclictest1023-21NetworkManager20:15:140
2172199118,2cyclictest0-21swapper/021:06:220
2172199118,2cyclictest0-21swapper/021:00:250
2172199116,4cyclictest0-21swapper/022:15:420
2172199115,5cyclictest0-21swapper/023:31:470
2172199115,3cyclictest0-21swapper/021:35:130
2172199115,3cyclictest0-21swapper/021:35:130
21721991110,1cyclictest0-21swapper/022:28:080
21721991110,1cyclictest0-21swapper/020:36:050
2172199110,11cyclictest0-21swapper/000:24:130
2172199110,11cyclictest0-21swapper/000:06:340
2172199110,1cyclictest0-21swapper/021:20:560
1097152110,0pltrace0-21swapper/323:42:033
2174999109,1cyclictest0-21swapper/722:42:397
2174999108,1cyclictest0-21swapper/719:42:177
2174999105,4cyclictest4509-21sed20:30:227
2174999105,3cyclictest0-21swapper/720:25:477
2174999104,5cyclictest0-21swapper/723:36:027
2174999104,5cyclictest0-21swapper/722:11:097
2174999103,6cyclictest0-21swapper/700:26:357
2174999101,2cyclictest0-21swapper/722:35:157
2174999100,8cyclictest0-21swapper/723:23:267
2174999100,5cyclictest17690-21/usr/sbin/munin21:55:157
2174699109,1cyclictest0-21swapper/620:46:016
2174699109,1cyclictest0-21swapper/619:11:486
2174699109,1cyclictest0-21swapper/600:21:126
2174699109,1cyclictest0-21swapper/600:05:206
2174699107,2cyclictest0-21swapper/623:40:146
2174699106,2cyclictest0-21swapper/622:25:086
2174699106,2cyclictest0-21swapper/621:42:596
2174699105,3cyclictest0-21swapper/621:30:586
2174699104,5cyclictest0-21swapper/620:18:076
2174699104,5cyclictest0-21swapper/600:11:226
2174699100,8cyclictest0-21swapper/623:35:266
2174699100,8cyclictest0-21swapper/622:46:466
2174699100,8cyclictest0-21swapper/621:53:236
2174699100,10cyclictest0-21swapper/621:01:536
2174699100,10cyclictest0-21swapper/619:57:276
2174699100,10cyclictest0-21swapper/600:00:586
2174699100,1cyclictest0-21swapper/623:08:596
2174199109,1cyclictest0-21swapper/523:31:515
2174199109,1cyclictest0-21swapper/523:08:515
2174199109,1cyclictest0-21swapper/522:35:405
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional