You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-25 - 08:32
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot6.osadl.org (updated Thu Jul 25, 2024 00:44:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29974521340,125pltrace0-21swapper/519:07:585
32766521330,129pltrace0-21swapper/719:10:007
29974521320,123pltrace0-21swapper/119:08:081
32766521240,116pltrace0-21swapper/619:08:186
29974521200,111pltrace0-21swapper/319:06:433
29974521170,108pltrace0-21swapper/219:06:592
888052790,0pltrace0-21swapper/700:23:117
3255527044,12sleep40-21swapper/419:05:254
3257125844,10sleep00-21swapper/019:05:390
614992113,2cyclictest0-21swapper/700:01:077
888052190,14pltrace54-21ksoftirqd/400:25:004
594991915,2cyclictest0-21swapper/221:42:422
2121152190,0pltrace0-21swapper/722:40:137
1905752190,0pltrace0-21swapper/720:39:567
599991811,6cyclictest0-21swapper/320:00:113
59199180,16cyclictest18871-21idleruntime21:35:171
1457152180,4pltrace0-21swapper/721:32:467
61499170,2cyclictest0-21swapper/721:25:567
60899179,7cyclictest1069-17sedispatch00:05:205
608991711,5cyclictest0-21swapper/522:39:445
60899170,1cyclictest0-21swapper/523:40:205
59999171,8cyclictest0-21swapper/322:44:013
59999171,4cyclictest0-21swapper/321:14:163
591991712,3cyclictest0-21swapper/121:14:191
591991711,5cyclictest0-21swapper/123:09:291
2850352170,3pltrace0-21swapper/321:53:563
1670852170,0pltrace0-21swapper/723:29:597
614991612,3cyclictest1028-17auditd21:40:007
614991611,3cyclictest0-21swapper/723:07:177
61499161,8cyclictest0-21swapper/723:12:597
61499160,8cyclictest0-21swapper/723:54:407
612991611,3cyclictest0-21swapper/620:23:276
612991610,3cyclictest0-21swapper/622:26:026
61299161,14cyclictest0-21swapper/623:55:166
61299160,8cyclictest0-21swapper/623:48:106
60899165,4cyclictest5879-21tr20:15:125
60899164,3cyclictest1070-13audispd22:55:015
60899161,14cyclictest0-21swapper/523:00:455
599991611,3cyclictest0-21swapper/321:46:233
59999161,8cyclictest0-21swapper/322:03:513
59999161,8cyclictest0-21swapper/322:03:513
59999161,8cyclictest0-21swapper/320:39:573
59999160,15cyclictest0-21swapper/300:09:353
594991610,5cyclictest0-21swapper/219:11:372
59499161,14cyclictest0-21swapper/220:16:392
59499160,15cyclictest0-21swapper/200:03:442
591991612,3cyclictest0-21swapper/120:33:571
591991611,3cyclictest0-21swapper/122:47:081
591991611,3cyclictest0-21swapper/100:28:511
322752160,3pltrace0-21swapper/519:16:195
277952160,6pltrace0-21swapper/323:08:033
888052150,10pltrace54-21ksoftirqd/400:25:014
61499159,5cyclictest0-21swapper/719:46:087
61499150,8cyclictest0-21swapper/722:37:047
61499150,2cyclictest0-21swapper/720:40:157
61499150,14cyclictest0-21swapper/723:02:327
61499150,14cyclictest0-21swapper/721:01:557
61499150,14cyclictest0-21swapper/720:47:437
61499150,14cyclictest0-21swapper/700:31:237
61299151,4cyclictest0-21swapper/600:18:536
61299150,8cyclictest0-21swapper/600:00:526
61299150,7cyclictest0-21swapper/621:55:216
61299150,7cyclictest0-21swapper/621:55:216
61299150,2cyclictest0-21swapper/620:40:186
60899157,7cyclictest1132-21snmpd23:07:415
608991511,3cyclictest0-21swapper/523:51:435
608991511,3cyclictest0-21swapper/500:20:235
60899150,7cyclictest0-21swapper/521:38:545
60899150,1cyclictest0-21swapper/523:23:595
59999150,8cyclictest0-21swapper/321:58:563
59999150,8cyclictest0-21swapper/321:58:563
59999150,4cyclictest0-21swapper/322:14:283
594991510,3cyclictest0-21swapper/222:40:312
594991510,3cyclictest0-21swapper/200:17:262
59499150,8cyclictest0-21swapper/223:58:412
59499150,7cyclictest0-21swapper/221:53:462
59499150,7cyclictest0-21swapper/221:16:442
59499150,3cyclictest0-21swapper/222:57:332
59499150,14cyclictest0-21swapper/222:25:312
59499150,14cyclictest0-21swapper/220:39:452
591991510,3cyclictest0-21swapper/122:27:011
591991510,3cyclictest0-21swapper/119:27:401
59199150,8cyclictest0-21swapper/120:01:451
59199150,14cyclictest0-21swapper/121:27:021
3194652150,4pltrace1023-21NetworkManager20:04:460
2957152150,11pltrace54-21ksoftirqd/422:56:264
1735552150,10pltrace54-21ksoftirqd/421:34:264
1349052150,0pltrace0-21swapper/720:28:187
1037052150,4pltrace11022-21taskset00:37:093
685052140,1pltrace6854-21pltrace-run19:18:103
61499140,7cyclictest0-21swapper/721:42:247
61499140,7cyclictest0-21swapper/721:11:257
61499140,7cyclictest0-21swapper/719:52:507
61499140,7cyclictest0-21swapper/719:33:197
61499140,3cyclictest0-21swapper/723:55:257
61499140,14cyclictest0-21swapper/720:50:157
61299149,3cyclictest0-21swapper/623:38:126
61299149,3cyclictest0-21swapper/621:50:366
612991410,2cyclictest20276-21crond21:40:006
61299140,7cyclictest0-21swapper/621:13:516
61299140,7cyclictest0-21swapper/620:15:506
61299140,13cyclictest0-21swapper/622:35:026
61299140,13cyclictest0-21swapper/621:30:156
60899147,6cyclictest17494-21latency_hist21:35:005
60899143,4cyclictest16698-21sh22:30:115
608991412,1cyclictest0-21swapper/523:10:235
608991410,3cyclictest0-21swapper/500:27:075
60899140,13cyclictest0-21swapper/522:20:215
60899140,12cyclictest0-21swapper/520:50:185
60899140,11cyclictest0-21swapper/521:21:445
60899140,1cyclictest0-21swapper/519:25:005
603991411,2cyclictest0-21swapper/420:13:334
603991411,2cyclictest0-21swapper/400:38:514
603991411,2cyclictest0-21swapper/400:07:584
599991411,2cyclictest0-21swapper/322:30:103
59999140,8cyclictest0-21swapper/323:14:043
59999140,7cyclictest0-21swapper/322:35:313
59999140,7cyclictest0-21swapper/320:24:503
59999140,2cyclictest0-21swapper/320:29:543
59499149,2cyclictest0-21swapper/222:06:542
59499149,2cyclictest0-21swapper/222:06:542
59499140,8cyclictest0-21swapper/223:22:332
59499140,7cyclictest0-21swapper/223:47:532
59499140,7cyclictest0-21swapper/223:35:352
59499140,7cyclictest0-21swapper/223:17:362
59499140,1cyclictest0-21swapper/222:50:172
59499140,1cyclictest0-21swapper/220:20:132
59199140,6cyclictest0-21swapper/122:44:041
59199140,6cyclictest0-21swapper/120:21:151
59199140,1cyclictest0-21swapper/120:40:171
55252140,5pltrace2400-21missed_timers21:05:286
3020452140,10pltrace32123-21ntp_kernel_err21:00:191
2850352140,4pltrace0-21swapper/721:56:487
2850352140,4pltrace0-21swapper/721:56:487
2506852140,2pltrace0-21swapper/723:44:447
1735552140,2pltrace17456-21taskset21:34:310
171052140,11pltrace3598-21netstat22:05:171
171052140,11pltrace3598-21netstat22:05:171
1285852140,2pltrace0-21swapper/722:26:307
1070652140,10pltrace54-21ksoftirqd/420:27:334
1037052140,4pltrace0-21swapper/700:35:217
899952130,9pltrace54-21ksoftirqd/421:19:214
728652130,7pltrace0-21swapper/322:15:483
61499132,9cyclictest0-21swapper/722:23:087
61499130,1cyclictest0-21swapper/720:20:197
61299137,5cyclictest0-21swapper/600:33:566
612991310,2cyclictest13420-21timerandwakeup23:20:216
60899137,4cyclictest1700-21lldpd21:52:265
60899136,6cyclictest1700-21lldpd19:31:495
60899135,6cyclictest0-21swapper/519:40:015
60899134,3cyclictest1132-21snmpd21:19:235
60899133,4cyclictest27582-21grep19:55:135
60899133,4cyclictest15227-21cut23:25:115
60899133,4cyclictest1321-21cpuspeed_turbos21:05:135
60899133,4cyclictest13163-21perl21:25:115
608991311,1cyclictest0-21swapper/522:05:175
608991311,1cyclictest0-21swapper/522:05:175
608991311,1cyclictest0-21swapper/519:45:145
608991310,2cyclictest1132-21snmpd23:45:105
60899131,4cyclictest0-21swapper/520:41:545
60899131,2cyclictest0-21swapper/521:40:295
60899130,10cyclictest0-21swapper/522:42:135
603991312,1cyclictest0-21swapper/422:23:204
603991311,1cyclictest0-21swapper/422:45:124
603991310,2cyclictest0-21swapper/423:05:154
59999139,3cyclictest0-21swapper/323:15:043
59999137,5cyclictest0-21swapper/320:55:493
59999135,7cyclictest1065-17auditd22:30:003
59999130,7cyclictest0-21swapper/321:25:553
59999130,13cyclictest0-21swapper/323:30:193
59999130,12cyclictest0-21swapper/323:24:463
59999130,12cyclictest0-21swapper/320:16:333
59999130,12cyclictest0-21swapper/319:46:323
59999130,1cyclictest0-21swapper/319:26:223
59499138,3cyclictest0-21swapper/200:23:232
594991312,1cyclictest0-21swapper/223:05:112
594991312,1cyclictest0-21swapper/221:25:172
59499130,13cyclictest0-21swapper/221:06:592
59499130,1cyclictest0-21swapper/220:27:102
59199133,8cyclictest0-21swapper/122:15:221
591991312,1cyclictest0-21swapper/119:15:151
59199130,6cyclictest0-21swapper/100:36:391
59199130,12cyclictest0-21swapper/123:42:381
59199130,12cyclictest0-21swapper/122:20:291
586991312,1cyclictest0-21swapper/020:39:230
586991310,2cyclictest0-21swapper/019:23:460
55252130,5pltrace0-21swapper/721:05:287
55252130,4pltrace2241-21/usr/sbin/munin21:05:221
55252130,4pltrace0-21swapper/321:05:283
3276652130,0pltrace0-21swapper/719:10:167
2785152130,7pltrace0-21swapper/723:48:147
2678652130,4pltrace1166-21runrttasks22:51:231
2635352130,0pltrace0-21swapper/619:54:026
2463152130,1pltrace0-21swapper/120:52:341
1670852130,3pltrace0-21swapper/723:32:337
1457152130,10pltrace0-21swapper/121:30:321
1006952130,11pltrace0-21swapper/022:19:260
100152130,4pltrace0-21swapper/300:00:393
888052120,6pltrace0-21swapper/000:25:100
621652120,0pltrace0-21swapper/721:15:117
61499129,2cyclictest1070-13audispd22:05:007
61499129,2cyclictest1070-13audispd22:05:007
61499129,2cyclictest0-21swapper/720:04:297
61299127,3cyclictest0-21swapper/623:31:036
61299122,8cyclictest0-21swapper/622:32:326
61299120,11cyclictest0-21swapper/621:48:566
61299120,11cyclictest0-21swapper/620:00:016
60899127,4cyclictest0-21swapper/500:16:505
60899126,5cyclictest0-21swapper/522:45:145
60899125,6cyclictest1132-21snmpd21:48:395
60899125,6cyclictest0-21swapper/522:10:085
60899124,2cyclictest17578-21cstates23:30:105
60899124,2cyclictest11538-21cpuspeed_turbos20:25:115
60899123,4cyclictest5642-21ls21:10:405
60899123,4cyclictest30874-21cut22:55:145
60899123,3cyclictest24240-21cpuspeed_turbos19:50:115
60899122,4cyclictest29239-21tr21:55:135
60899122,4cyclictest29239-21tr21:55:135
60899120,5cyclictest14075-21cut22:25:105
60899120,5cyclictest11093-21taskset00:38:115
60899120,5cyclictest10468-21sshd23:15:205
60899120,5cyclictest10137-21taskset00:30:085
60899120,4cyclictest7900-21hostname22:15:115
60899120,4cyclictest5228-21chrt00:11:405
60899120,4cyclictest4197-21cat20:10:175
60899120,4cyclictest31158-21proc_pri20:00:205
60899120,4cyclictest30931-21tr21:00:125
60899120,4cyclictest3059-21/usr/sbin/munin00:00:215
60899120,4cyclictest28870-21sed20:55:145
60899120,4cyclictest22916-21forks20:45:125
60899120,4cyclictest18909-21expr19:40:125
60899120,4cyclictest12416-21tr19:28:105
60899120,4cyclictest1045-21taskset23:58:345
60899120,4cyclictest1011-21cut19:10:105
60399129,2cyclictest0-21swapper/400:11:534
60399125,6cyclictest1070-13audispd20:35:004
603991210,1cyclictest0-21swapper/419:49:504
60399120,11cyclictest0-21swapper/422:50:034
59999128,2cyclictest0-21swapper/323:50:003
59999126,5cyclictest0-21swapper/320:52:523
59999126,5cyclictest0-21swapper/319:50:233
59999126,5cyclictest0-21swapper/300:27:083
59499128,3cyclictest0-21swapper/223:01:032
59499126,5cyclictest0-21swapper/219:43:572
59499120,11cyclictest0-21swapper/219:26:202
59499120,1cyclictest0-21swapper/221:04:242
59199129,2cyclictest0-21swapper/123:30:031
59199126,5cyclictest0-21swapper/123:11:111
59199126,3cyclictest0-21swapper/119:51:151
59199126,2cyclictest0-21swapper/120:06:031
59199123,8cyclictest0-21swapper/121:43:541
59199120,1cyclictest0-21swapper/123:48:451
58699120,8cyclictest1132-21snmpd23:04:410
58699120,12cyclictest0-21swapper/023:05:060
58699120,1cyclictest0-21swapper/023:18:350
3128752120,5pltrace0-21swapper/022:03:010
3128752120,5pltrace0-21swapper/022:03:010
2957152120,3pltrace31276-21grep22:55:173
2635352120,10pltrace0-21swapper/219:55:062
2121152120,5pltrace0-21swapper/022:42:520
1949252120,4pltrace0-21swapper/523:35:425
1949252120,2pltrace22078-21runrttasks23:35:351
1349052120,0pltrace0-21swapper/720:30:187
1113852120,8pltrace0-21swapper/023:22:480
1070652120,5pltrace0-21swapper/020:27:410
899952110,4pltrace0-21swapper/621:19:356
899952110,4pltrace0-21swapper/121:19:341
899952110,1pltrace0-21swapper/021:19:310
61499119,1cyclictest0-21swapper/722:54:557
61499119,1cyclictest0-21swapper/700:15:047
61499118,2cyclictest1132-21snmpd20:06:417
61499116,4cyclictest0-21swapper/700:14:177
61499115,5cyclictest0-21swapper/719:16:227
614991110,1cyclictest0-21swapper/722:14:377
614991110,1cyclictest0-21swapper/720:55:167
61499110,11cyclictest0-21swapper/721:45:577
61499110,11cyclictest0-21swapper/719:20:337
61299119,1cyclictest0-21swapper/600:10:426
61299117,3cyclictest1132-21snmpd23:09:176
61299117,2cyclictest0-21swapper/619:22:286
61299116,3cyclictest0-21swapper/622:02:426
61299116,3cyclictest0-21swapper/622:02:426
61299116,3cyclictest0-21swapper/619:10:346
612991110,1cyclictest0-21swapper/620:54:596
612991110,1cyclictest0-21swapper/619:25:266
61299110,9cyclictest0-21swapper/620:00:206
61299110,11cyclictest0-21swapper/621:43:136
61299110,1cyclictest0-21swapper/620:58:426
61299110,1cyclictest0-21swapper/619:32:336
60899110,5cyclictest32531-21cut20:05:105
60899110,4cyclictest8515-21wget20:20:115
60899110,4cyclictest31919-21egrep22:00:125
60899110,4cyclictest31919-21egrep22:00:125
60899110,4cyclictest17132-21cut20:35:135
60899110,4cyclictest13997-21chrt20:30:035
60399117,1cyclictest54-21ksoftirqd/400:18:164
603991110,1cyclictest0-21swapper/422:18:194
603991110,1cyclictest0-21swapper/421:38:394
603991110,1cyclictest0-21swapper/420:59:574
603991110,1cyclictest0-21swapper/420:54:564
603991110,1cyclictest0-21swapper/420:42:284
603991110,1cyclictest0-21swapper/420:36:194
60399110,11cyclictest0-21swapper/421:48:124
60399110,11cyclictest0-21swapper/419:22:204
60399110,1cyclictest0-21swapper/423:58:524
60399110,1cyclictest0-21swapper/423:33:274
60399110,1cyclictest0-21swapper/423:24:424
60399110,1cyclictest0-21swapper/420:46:124
60399110,1cyclictest0-21swapper/419:13:354
59999118,2cyclictest1132-21snmpd20:48:283
59999117,3cyclictest1132-21snmpd20:11:303
59999117,3cyclictest1132-21snmpd19:20:493
59999117,3cyclictest1132-21snmpd00:22:443
599991110,1cyclictest0-21swapper/322:07:253
599991110,1cyclictest0-21swapper/322:07:253
599991110,1cyclictest0-21swapper/321:37:483
599991110,1cyclictest0-21swapper/319:40:443
59999110,11cyclictest0-21swapper/323:37:563
59999110,11cyclictest0-21swapper/321:41:213
59999110,11cyclictest0-21swapper/300:30:043
59999110,10cyclictest0-21swapper/320:33:413
59999110,1cyclictest0-21swapper/322:23:463
59999110,1cyclictest0-21swapper/321:21:073
59999110,1cyclictest0-21swapper/321:15:073
59499117,3cyclictest31269-21ifconfig21:58:112
59499117,3cyclictest31269-21ifconfig21:58:102
59499116,4cyclictest11623-21cat22:20:142
59499115,5cyclictest0-21swapper/221:46:002
59499115,5cyclictest0-21swapper/200:07:262
59499112,8cyclictest0-21swapper/221:31:542
594991110,1cyclictest0-21swapper/222:30:422
594991110,1cyclictest0-21swapper/220:08:132
594991110,1cyclictest0-21swapper/219:19:502
594991110,1cyclictest0-21swapper/200:37:482
59499111,9cyclictest0-21swapper/219:25:012
59499110,8cyclictest0-21swapper/200:30:112
59499110,11cyclictest0-21swapper/221:22:522
59499110,11cyclictest0-21swapper/220:51:462
59499110,11cyclictest0-21swapper/220:41:442
59499110,1cyclictest0-21swapper/223:27:422
59499110,1cyclictest0-21swapper/222:14:302
59499110,1cyclictest0-21swapper/222:00:352
59499110,1cyclictest0-21swapper/222:00:352
59199119,1cyclictest0-21swapper/119:10:231
59199116,4cyclictest0-21swapper/100:34:411
59199115,5cyclictest0-21swapper/122:35:161
59199115,5cyclictest0-21swapper/119:59:571
59199111,8cyclictest0-21swapper/123:52:291
59199110,11cyclictest0-21swapper/120:29:151
59199110,11cyclictest0-21swapper/119:23:241
59199110,11cyclictest0-21swapper/100:03:451
58699119,1cyclictest0-21swapper/022:45:120
586991110,1cyclictest0-21swapper/022:13:180
586991110,1cyclictest0-21swapper/021:27:470
586991110,1cyclictest0-21swapper/020:48:070
586991110,1cyclictest0-21swapper/020:31:270
586991110,1cyclictest0-21swapper/019:52:550
58699110,11cyclictest0-21swapper/022:34:200
58699110,11cyclictest0-21swapper/021:55:380
58699110,11cyclictest0-21swapper/021:55:380
58699110,11cyclictest0-21swapper/020:17:520
58699110,11cyclictest0-21swapper/019:41:060
58699110,11cyclictest0-21swapper/000:36:260
58699110,10cyclictest0-21swapper/019:27:490
58699110,1cyclictest0-21swapper/023:50:350
58699110,1cyclictest0-21swapper/023:49:180
58699110,1cyclictest0-21swapper/022:51:300
58699110,1cyclictest0-21swapper/021:13:230
58699110,1cyclictest0-21swapper/020:53:080
58699110,1cyclictest0-21swapper/020:21:550
58699110,1cyclictest0-21swapper/000:19:550
3276652110,6pltrace1378-21cstates19:10:113
3235952110,9pltrace0-21swapper/623:02:336
322752110,6pltrace0-21swapper/019:15:140
2957152110,7pltrace0-21swapper/022:56:320
2399852110,0pltrace0-21swapper/722:46:207
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional