You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-09-21 - 23:55
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack5slot3.osadl.org (updated Thu Sep 21, 2023 12:50:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1412229187,2sleep20-21swapper/207:05:272
1546028047,29sleep10-21swapper/107:09:341
1290328045,30sleep70-21swapper/707:05:167
1543627945,30sleep60-21swapper/607:09:156
1539927743,30sleep50-21swapper/507:08:425
1549027642,29sleep00-21swapper/007:09:570
1291227570,2sleep40-21swapper/407:05:204
1289027540,30sleep30-21swapper/307:05:033
145662490,0sleep60-21swapper/611:15:266
156762450,0sleep50-21swapper/510:39:065
1592499360,0cyclictest0-21swapper/308:40:253
15925993130,1cyclictest0-21swapper/409:57:034
15924993131,0cyclictest0-21swapper/312:37:413
15923993030,0cyclictest0-21swapper/210:37:542
15927992929,0cyclictest0-21swapper/608:20:236
15927992624,1cyclictest0-21swapper/608:45:186
15928992310,0cyclictest0-21swapper/707:15:347
15925992322,0cyclictest0-21swapper/407:20:194
1592399230,23cyclictest0-21swapper/211:24:532
234472210,0sleep40-21swapper/412:18:104
15927992117,4cyclictest1657-21lldpd12:00:006
15926992019,1cyclictest73550irq/125-eth012:05:325
15926992014,2cyclictest26204-21perf09:34:565
15926992014,2cyclictest26204-21perf09:34:555
15926991919,0cyclictest73550irq/125-eth012:36:515
15924991910,4cyclictest0-21swapper/308:20:283
15926991818,0cyclictest0-21swapper/508:00:235
15923991818,0cyclictest0-21swapper/208:40:262
15923991810,5cyclictest0-21swapper/208:45:192
15923991810,5cyclictest0-21swapper/207:55:272
15922991814,2cyclictest682-21nscd10:32:071
15922991814,2cyclictest682-21nscd10:32:071
15925991710,5cyclictest0-21swapper/408:20:244
15925991710,0cyclictest0-21swapper/411:41:454
15924991716,1cyclictest9734-21ssh11:31:353
15924991710,4cyclictest0-21swapper/307:40:293
15923991710,0cyclictest0-21swapper/209:11:122
15922991710,3cyclictest0-21swapper/108:25:011
15921991716,1cyclictest1803-21sendmail-mta11:29:580
15921991710,4cyclictest0-21swapper/008:40:280
15921991710,2cyclictest0-21swapper/008:20:250
15921991710,0cyclictest0-21swapper/012:35:130
15928991613,3cyclictest704-21ptp4l11:40:367
15928991610,6cyclictest0-21swapper/711:25:427
15927991610,5cyclictest0-21swapper/611:24:076
15926991616,0cyclictest73550irq/125-eth011:07:535
15926991610,0cyclictest0-21swapper/508:45:195
15925991610,4cyclictest0-21swapper/411:47:004
15925991610,1cyclictest0-21swapper/410:18:374
15925991610,0cyclictest0-21swapper/412:10:314
15925991610,0cyclictest0-21swapper/411:26:594
15925991610,0cyclictest0-21swapper/410:07:124
15925991610,0cyclictest0-21swapper/408:00:184
15924991615,1cyclictest28821-21ssh10:08:523
15924991615,1cyclictest14854-21ssh10:38:273
15924991615,1cyclictest10757-21ssh10:55:043
15923991610,0cyclictest0-21swapper/210:05:132
15922991615,1cyclictest18928-21munin-node10:38:311
15922991610,5cyclictest0-21swapper/112:35:251
15922991610,2cyclictest0-21swapper/111:21:131
15922991610,0cyclictest0-21swapper/111:17:271
15922991610,0cyclictest0-21swapper/109:05:221
15921991615,1cyclictest23915-21kworker/u16:009:45:080
15921991611,1cyclictest19616-21ls09:25:250
15921991610,6cyclictest0-21swapper/012:30:500
15921991610,0cyclictest0-21swapper/011:08:430
15921991610,0cyclictest0-21swapper/009:58:440
1026221610,0sleep30-21swapper/310:54:593
15928991515,0cyclictest0-21swapper/711:53:277
15928991510,5cyclictest0-21swapper/710:15:257
15928991510,5cyclictest0-21swapper/707:20:017
15928991510,0cyclictest0-21swapper/711:15:327
15928991510,0cyclictest0-21swapper/711:08:337
15928991510,0cyclictest0-21swapper/709:56:127
15928991510,0cyclictest0-21swapper/709:50:457
15927991515,0cyclictest7450-21ls07:45:316
15927991513,1cyclictest701-21avahi-daemon10:18:326
15927991510,2cyclictest0-21swapper/607:20:016
15927991510,0cyclictest0-21swapper/611:31:276
15927991510,0cyclictest0-21swapper/609:19:146
15926991515,0cyclictest26210-21strings09:10:225
15926991510,5cyclictest0-21swapper/512:13:125
1592599155,9cyclictest1657-21lldpd11:58:014
15925991514,1cyclictest8752-21ssh09:37:594
15925991510,5cyclictest0-21swapper/412:06:344
15925991510,5cyclictest0-21swapper/411:35:564
15925991510,5cyclictest0-21swapper/411:05:214
15925991510,5cyclictest0-21swapper/407:55:294
15925991510,2cyclictest0-21swapper/408:40:284
15925991510,0cyclictest0-21swapper/412:35:274
15925991510,0cyclictest0-21swapper/412:27:474
15925991510,0cyclictest0-21swapper/410:36:494
15925991510,0cyclictest0-21swapper/409:10:264
15924991514,0cyclictest29164-1kworker/u17:011:45:463
15924991513,1cyclictest690-21nscd09:00:203
15924991513,0cyclictest21649-21ls11:00:293
15924991513,0cyclictest21649-21ls11:00:283
15924991510,2cyclictest0-21swapper/309:50:303
15924991510,0cyclictest0-21swapper/310:32:323
15924991510,0cyclictest0-21swapper/310:32:313
15923991515,0cyclictest25140-21strings08:15:252
15923991515,0cyclictest0-21swapper/212:29:562
15923991514,1cyclictest23964-21diskmemload11:49:162
15923991510,3cyclictest0-21swapper/212:24:332
15923991510,3cyclictest0-21swapper/209:41:072
15923991510,2cyclictest0-21swapper/207:10:252
15923991510,0cyclictest0-21swapper/211:50:182
15923991510,0cyclictest0-21swapper/211:26:222
15923991510,0cyclictest0-21swapper/211:19:382
15923991510,0cyclictest0-21swapper/211:09:322
15923991510,0cyclictest0-21swapper/210:28:062
15923991510,0cyclictest0-21swapper/210:15:302
15923991510,0cyclictest0-21swapper/208:30:332
1592299158,6cyclictest11299-21nscd12:15:421
15922991515,0cyclictest0-21swapper/112:05:331
15922991512,1cyclictest673-21irqbalance10:46:301
15922991510,5cyclictest0-21swapper/107:15:011
15922991510,4cyclictest0-21swapper/112:20:121
15922991510,1cyclictest0-21swapper/108:25:161
15922991510,0cyclictest0-21swapper/110:51:421
15922991510,0cyclictest0-21swapper/110:11:321
15921991513,1cyclictest2385-21apt-get09:35:010
15921991510,5cyclictest0-21swapper/011:00:010
15921991510,5cyclictest0-21swapper/011:00:010
15921991510,5cyclictest0-21swapper/010:42:110
15921991510,2cyclictest0-21swapper/007:15:340
15921991510,0cyclictest0-21swapper/012:17:550
15921991510,0cyclictest0-21swapper/011:33:390
15921991510,0cyclictest0-21swapper/011:22:470
15921991510,0cyclictest0-21swapper/010:15:170
15921991510,0cyclictest0-21swapper/010:10:130
15921991510,0cyclictest0-21swapper/010:03:590
15921991510,0cyclictest0-21swapper/010:03:590
15921991510,0cyclictest0-21swapper/007:55:040
15921991510,0cyclictest0-21swapper/007:38:040
365721410,3sleep60-21swapper/610:50:216
15928991414,0cyclictest30280-1kworker/u17:212:12:027
15928991414,0cyclictest0-21swapper/712:37:307
15928991414,0cyclictest0-21swapper/712:16:127
15928991414,0cyclictest0-21swapper/711:00:577
15928991414,0cyclictest0-21swapper/711:00:567
15928991414,0cyclictest0-21swapper/710:55:557
15928991412,1cyclictest682-21nscd12:23:137
15928991410,4cyclictest0-21swapper/711:30:137
15928991410,3cyclictest11299-21nscd12:31:187
15928991410,2cyclictest0-21swapper/708:20:247
15928991410,0cyclictest0-21swapper/711:11:087
15928991410,0cyclictest0-21swapper/710:05:247
15928991410,0cyclictest0-21swapper/710:00:267
15928991410,0cyclictest0-21swapper/710:00:267
15928991410,0cyclictest0-21swapper/709:25:007
15928991410,0cyclictest0-21swapper/709:14:247
15928991410,0cyclictest0-21swapper/707:25:217
1592799144,9cyclictest18928-21munin-node09:35:026
15927991412,1cyclictest18635-21nscd09:26:576
15927991411,1cyclictest23915-21kworker/u16:009:55:006
15927991410,4cyclictest0-21swapper/611:51:086
15927991410,4cyclictest0-21swapper/611:04:136
15927991410,4cyclictest0-21swapper/611:04:136
15927991410,4cyclictest0-21swapper/609:30:416
15927991410,4cyclictest0-21swapper/609:30:406
15927991410,4cyclictest0-21swapper/609:13:016
15927991410,3cyclictest0-21swapper/607:35:276
15927991410,2cyclictest0-21swapper/607:50:226
15927991410,0cyclictest0-21swapper/612:38:176
15927991410,0cyclictest0-21swapper/612:20:116
15927991410,0cyclictest0-21swapper/612:15:436
15927991410,0cyclictest0-21swapper/612:05:126
15927991410,0cyclictest0-21swapper/611:40:316
15927991410,0cyclictest0-21swapper/611:37:356
15927991410,0cyclictest0-21swapper/610:45:326
15927991410,0cyclictest0-21swapper/610:42:566
15927991410,0cyclictest0-21swapper/610:14:276
15927991410,0cyclictest0-21swapper/610:07:516
15927991410,0cyclictest0-21swapper/609:40:296
15926991414,0cyclictest24438-21ls11:40:235
15926991414,0cyclictest0-21swapper/511:57:555
15926991414,0cyclictest0-21swapper/510:50:035
15926991410,3cyclictest0-21swapper/512:17:445
15926991410,3cyclictest0-21swapper/507:20:225
15926991410,0cyclictest0-21swapper/508:20:305
15925991414,0cyclictest4056-21ntpq10:50:234
15925991414,0cyclictest0-21swapper/407:45:234
15925991414,0cyclictest0-21swapper/407:15:284
15925991410,3cyclictest0-21swapper/412:00:304
15925991410,2cyclictest0-21swapper/410:30:264
15925991410,2cyclictest0-21swapper/410:30:264
15925991410,1cyclictest23915-21kworker/u16:010:26:494
15925991410,0cyclictest0-21swapper/412:30:414
15925991410,0cyclictest0-21swapper/412:22:344
15925991410,0cyclictest0-21swapper/411:50:294
15925991410,0cyclictest0-21swapper/411:21:154
15925991410,0cyclictest0-21swapper/411:13:344
15925991410,0cyclictest0-21swapper/411:00:154
15925991410,0cyclictest0-21swapper/411:00:144
15925991410,0cyclictest0-21swapper/410:20:434
15925991410,0cyclictest0-21swapper/410:10:194
15925991410,0cyclictest0-21swapper/410:01:264
15925991410,0cyclictest0-21swapper/410:01:264
15925991410,0cyclictest0-21swapper/409:30:294
15925991410,0cyclictest0-21swapper/409:30:294
15925991410,0cyclictest0-21swapper/409:20:154
15925991410,0cyclictest0-21swapper/407:53:444
15924991413,1cyclictest19411-21ssh11:37:353
15924991412,0cyclictest5938-1kworker/u17:111:57:563
15924991410,0cyclictest0-21swapper/310:25:193
15924991410,0cyclictest0-21swapper/309:56:063
15923991414,0cyclictest0-21swapper/212:32:272
15923991414,0cyclictest0-21swapper/211:30:072
15923991414,0cyclictest0-21swapper/209:45:172
15923991414,0cyclictest0-21swapper/209:38:182
15923991411,3cyclictest15919-21cyclictest12:35:162
15923991410,4cyclictest0-21swapper/212:10:202
15923991410,4cyclictest0-21swapper/208:10:232
15923991410,3cyclictest0-21swapper/207:15:302
15923991410,1cyclictest23964-21diskmemload09:18:412
15923991410,0cyclictest0-21swapper/212:15:242
15923991410,0cyclictest0-21swapper/212:05:232
15923991410,0cyclictest0-21swapper/212:04:312
15923991410,0cyclictest0-21swapper/211:55:212
15923991410,0cyclictest0-21swapper/210:40:202
15923991410,0cyclictest0-21swapper/210:12:412
15923991410,0cyclictest0-21swapper/209:57:342
15923991410,0cyclictest0-21swapper/209:50:242
15923991410,0cyclictest0-21swapper/209:21:152
15923991410,0cyclictest0-21swapper/208:25:202
15923991410,0cyclictest0-21swapper/208:20:252
15923991410,0cyclictest0-21swapper/208:00:192
1592299143,5cyclictest0-21swapper/112:00:011
15922991414,0cyclictest11426-21ls08:45:281
15922991410,4cyclictest0-21swapper/110:06:041
15922991410,4cyclictest0-21swapper/109:55:211
15922991410,3cyclictest0-21swapper/108:40:251
15922991410,0cyclictest15919-21cyclictest08:13:081
15922991410,0cyclictest0-21swapper/112:32:071
15922991410,0cyclictest0-21swapper/112:26:531
15922991410,0cyclictest0-21swapper/111:56:501
15922991410,0cyclictest0-21swapper/111:45:391
15922991410,0cyclictest0-21swapper/111:28:251
15922991410,0cyclictest0-21swapper/110:56:201
15922991410,0cyclictest0-21swapper/110:43:201
15922991410,0cyclictest0-21swapper/110:03:311
15922991410,0cyclictest0-21swapper/110:03:301
15922991410,0cyclictest0-21swapper/109:40:201
15922991410,0cyclictest0-21swapper/109:12:481
15922991410,0cyclictest0-21swapper/108:55:001
15922991410,0cyclictest0-21swapper/108:04:591
15921991414,0cyclictest13347-21ls10:55:310
15921991414,0cyclictest0-21swapper/010:30:410
15921991414,0cyclictest0-21swapper/010:30:410
15921991414,0cyclictest0-21swapper/010:27:020
15921991414,0cyclictest0-21swapper/010:21:050
15921991410,4cyclictest0-21swapper/011:12:020
15921991410,4cyclictest0-21swapper/009:30:380
15921991410,4cyclictest0-21swapper/009:30:380
15921991410,4cyclictest0-21swapper/009:23:270
15921991410,2cyclictest0-21swapper/008:49:590
15921991410,2cyclictest0-21swapper/007:20:180
15921991410,0cyclictest0-21swapper/012:11:070
15921991410,0cyclictest0-21swapper/012:01:590
15921991410,0cyclictest0-21swapper/011:59:040
15921991410,0cyclictest0-21swapper/011:41:010
15921991410,0cyclictest0-21swapper/010:37:080
15921991410,0cyclictest0-21swapper/009:50:210
15921991410,0cyclictest0-21swapper/009:13:410
577121310,2sleep10-21swapper/108:36:451
461721310,2sleep00-21swapper/009:16:310
2034521310,2sleep00-21swapper/011:19:460
15928991312,1cyclictest2977-21ssh09:35:037
15928991312,1cyclictest1624-21runrttasks08:05:217
15928991312,1cyclictest1624-21runrttasks08:05:217
15928991311,1cyclictest7621-21runrttasks09:19:007
15928991311,1cyclictest31611-21nscd10:10:567
15928991311,1cyclictest23964-21diskmemload10:52:427
15928991311,1cyclictest17733-21cpuspeed_turbos10:40:187
15928991310,3cyclictest0-21swapper/712:05:037
15928991310,3cyclictest0-21swapper/712:00:057
15928991310,3cyclictest0-21swapper/711:55:197
15928991310,3cyclictest0-21swapper/711:20:337
15928991310,3cyclictest0-21swapper/709:40:227
15928991310,3cyclictest0-21swapper/709:05:237
15928991310,3cyclictest0-21swapper/708:10:207
15928991310,2cyclictest704-21ptp4l09:30:217
15928991310,2cyclictest704-21ptp4l09:30:217
15928991310,2cyclictest0-21swapper/708:30:187
15928991310,1cyclictest0-21swapper/709:28:147
15928991310,0cyclictest0-21swapper/712:25:257
15928991310,0cyclictest0-21swapper/711:49:507
15928991310,0cyclictest0-21swapper/710:45:117
15928991310,0cyclictest0-21swapper/710:35:067
15928991310,0cyclictest0-21swapper/710:30:277
15928991310,0cyclictest0-21swapper/710:30:267
15928991310,0cyclictest0-21swapper/709:45:257
15928991310,0cyclictest0-21swapper/708:40:297
15928991310,0cyclictest0-21swapper/708:00:207
15928991310,0cyclictest0-21swapper/707:10:247
1592799136,6cyclictest0-21swapper/608:40:296
15927991313,0cyclictest31249-21ls09:50:306
15927991313,0cyclictest0-21swapper/611:59:526
15927991313,0cyclictest0-21swapper/610:55:036
15927991313,0cyclictest0-21swapper/608:00:296
15927991311,1cyclictest689-21nscd10:00:126
15927991311,1cyclictest689-21nscd10:00:116
15927991311,1cyclictest16669-21latency08:55:236
15927991310,3cyclictest0-21swapper/611:25:276
15927991310,3cyclictest0-21swapper/611:11:456
15927991310,3cyclictest0-21swapper/611:08:286
15927991310,3cyclictest0-21swapper/610:25:186
15927991310,3cyclictest0-21swapper/608:16:266
15927991310,3cyclictest0-21swapper/608:10:216
15927991310,3cyclictest0-21swapper/607:33:566
15927991310,0cyclictest0-21swapper/612:30:186
15927991310,0cyclictest0-21swapper/612:29:556
15927991310,0cyclictest0-21swapper/612:12:126
15927991310,0cyclictest0-21swapper/611:45:546
15927991310,0cyclictest0-21swapper/610:35:596
15927991310,0cyclictest0-21swapper/610:31:506
15927991310,0cyclictest0-21swapper/610:31:496
15927991310,0cyclictest0-21swapper/610:20:156
15927991310,0cyclictest0-21swapper/609:45:436
15927991310,0cyclictest0-21swapper/609:20:306
15927991310,0cyclictest0-21swapper/609:06:016
15927991310,0cyclictest0-21swapper/608:30:226
15927991310,0cyclictest0-21swapper/607:40:376
15926991313,0cyclictest73550irq/125-eth012:28:375
15926991313,0cyclictest27746-21ls12:20:215
15926991312,1cyclictest73550irq/125-eth011:16:205
15926991312,1cyclictest73550irq/125-eth010:41:135
15926991312,1cyclictest73550irq/125-eth010:27:585
15926991312,1cyclictest73550irq/125-eth009:58:075
15926991312,1cyclictest73550irq/125-eth009:46:485
15926991312,1cyclictest73550irq/125-eth009:41:385
15926991312,1cyclictest73550irq/125-eth009:24:315
15926991312,1cyclictest25058-21diskmemload11:03:055
15926991312,1cyclictest25058-21diskmemload11:03:055
15926991312,1cyclictest21556-21ntp_states07:15:305
15926991311,1cyclictest73550irq/125-eth011:53:295
15926991311,1cyclictest15820-21grep10:00:275
15926991311,1cyclictest15820-21grep10:00:265
15926991310,3cyclictest505-21ls10:10:275
15926991310,3cyclictest0-21swapper/512:00:275
15926991310,3cyclictest0-21swapper/509:25:105
15926991310,3cyclictest0-21swapper/508:30:155
15926991310,3cyclictest0-21swapper/508:15:205
15926991310,3cyclictest0-21swapper/507:25:195
15926991310,0cyclictest0-21swapper/511:28:565
15926991310,0cyclictest0-21swapper/511:10:275
15926991310,0cyclictest0-21swapper/510:46:535
15926991310,0cyclictest0-21swapper/510:16:505
15926991310,0cyclictest0-21swapper/510:08:525
15926991310,0cyclictest0-21swapper/509:50:115
15926991310,0cyclictest0-21swapper/509:00:175
15925991313,0cyclictest0-21swapper/409:50:034
15925991312,1cyclictest682-21nscd10:46:104
15925991310,3cyclictest0-21swapper/411:30:454
15925991310,3cyclictest0-21swapper/408:55:254
15925991310,3cyclictest0-21swapper/408:50:244
15925991310,2cyclictest0-21swapper/411:15:274
15925991310,2cyclictest0-21swapper/408:25:164
15925991310,0cyclictest0-21swapper/410:57:124
15925991310,0cyclictest0-21swapper/410:40:074
15925991310,0cyclictest0-21swapper/409:46:374
15925991310,0cyclictest0-21swapper/409:44:234
15925991310,0cyclictest0-21swapper/409:26:564
15925991310,0cyclictest0-21swapper/409:15:304
15925991310,0cyclictest0-21swapper/408:45:194
15925991310,0cyclictest0-21swapper/408:30:184
15925991310,0cyclictest0-21swapper/407:35:234
15924991313,0cyclictest23964-21diskmemload10:49:093
15924991312,1cyclictest8527-21ssh11:12:053
15924991312,1cyclictest7155-1kworker/u17:209:46:153
15924991312,1cyclictest32390-21ssh09:14:493
15924991312,1cyclictest23964-21diskmemload12:12:433
15924991312,1cyclictest23964-21diskmemload09:35:153
15924991312,1cyclictest16941-21awk08:55:273
15924991312,1cyclictest15392-21ssh12:32:163
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional