You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-08-13 - 02:50

x86 Intel Core i9-9900K @3600 MHz, Linux 5.15.2-rt19 (Profile)

Latency plot of system in rack #0, slot #3
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot3.osadl.org (updated Sat Aug 13, 2022 00:45:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
40442332374354,13sleep150-21swapper/1519:08:297
40441242361354,5sleep40-21swapper/419:06:5610
40441962356322,28sleep140-21swapper/1419:07:556
40441342351311,14sleep120-21swapper/1219:07:054
40441092349309,13sleep70-21swapper/719:06:4413
40443202348328,13sleep100-21swapper/1019:09:442
40440672348309,31sleep50-21swapper/519:06:1011
40443342343322,14sleep60-21swapper/619:09:5612
40441712343323,14sleep80-21swapper/819:07:3314
40441032342304,31sleep10-21swapper/119:06:381
40442942341320,14sleep30-21swapper/319:09:219
40440362340320,14sleep130-21swapper/1319:05:465
40440732339319,14sleep110-21swapper/1119:06:163
40443282337305,19sleep00-21swapper/019:09:490
40443002335316,13sleep90-21swapper/919:09:2715
40442762334315,13sleep20-21swapper/219:09:048
379132710,0sleep70-21swapper/722:03:1113
157892580,0sleep10-21swapper/121:50:001
4044564993813,23cyclictest0-21swapper/1100:29:533
41766732360,0sleep60-21swapper/621:27:3912
4044564993410,11cyclictest0-21swapper/1121:25:213
2636042340,0sleep80-21swapper/800:30:0114
404456499339,24cyclictest0-21swapper/1100:21:053
404456499339,11cyclictest0-21swapper/1123:37:093
404456499339,11cyclictest0-21swapper/1121:47:193
404456499339,11cyclictest0-21swapper/1121:03:233
404456499339,11cyclictest0-21swapper/1120:08:283
404456499339,11cyclictest0-21swapper/1119:35:313
404456499339,11cyclictest0-21swapper/1119:24:323
404456399336,24cyclictest0-21swapper/1000:20:282
2762912320,0sleep150-21swapper/1500:37:227
2440902310,0sleep130-21swapper/1300:16:295
1792432310,0sleep40-21swapper/423:35:1410
41805422300,0sleep552-21ksoftirqd/521:30:1911
41727982300,0sleep50-21swapper/521:25:1411
404456499283,25cyclictest0-21swapper/1122:20:163
404456499283,0cyclictest0-21swapper/1123:26:103
2316022280,0sleep80-21swapper/800:10:0014
404456499272,0cyclictest0-21swapper/1123:04:123
404456499272,0cyclictest0-21swapper/1122:31:153
404456499272,0cyclictest0-21swapper/1122:09:173
404456499272,0cyclictest0-21swapper/1121:14:223
404456499262,0cyclictest0-21swapper/1119:46:293
1052872240,0sleep100-21swapper/1022:45:242
4044564992310,0cyclictest0-21swapper/1123:48:083
4044564992310,0cyclictest0-21swapper/1120:41:253
4044564992310,0cyclictest0-21swapper/1120:30:263
404456499229,0cyclictest0-21swapper/1121:36:203
404456499229,0cyclictest0-21swapper/1121:36:203
404456499229,0cyclictest0-21swapper/1119:13:333
4044563992212,7cyclictest0-21swapper/1000:30:192
404456499218,0cyclictest0-21swapper/1123:59:073
404456499218,0cyclictest0-21swapper/1120:52:243
404456499218,0cyclictest0-21swapper/1119:57:293
4044564992110,11cyclictest0-21swapper/1122:53:133
4044564992110,11cyclictest0-21swapper/1122:42:153
404456399216,2cyclictest0-21swapper/1022:06:112
404456299210,21cyclictest0-21swapper/900:29:5315
404456199211,0cyclictest721rcuc/821:18:0014
404456599200,17cyclictest0-21swapper/1221:35:404
404456599200,17cyclictest0-21swapper/1221:35:404
4044564992010,10cyclictest0-21swapper/1123:15:113
4044564992010,10cyclictest0-21swapper/1123:15:113
404455599200,17cyclictest0-21swapper/223:28:128
404456599190,2cyclictest142958-21grep23:10:224
404456599190,16cyclictest0-21swapper/1200:15:024
4044563991913,3cyclictest0-21swapper/1023:41:222
4044563991913,3cyclictest0-21swapper/1023:22:402
404456399190,1cyclictest0-21swapper/1023:34:162
404456399190,14cyclictest0-21swapper/1000:15:582
404456199191,18cyclictest0-21swapper/800:39:2814
4044559991918,1cyclictest0-21swapper/622:21:0612
404455599190,0cyclictest0-21swapper/223:32:118
4044554991919,0cyclictest3988168-21kworker/1:100:29:531
404455499190,16cyclictest0-21swapper/119:25:151
404456899180,1cyclictest3597-21gnome-shell19:10:097
4044566991818,0cyclictest0-21swapper/1320:20:005
404456599180,18cyclictest0-21swapper/1223:03:164
404456599180,15cyclictest0-21swapper/1200:13:384
404456499188,10cyclictest0-21swapper/1120:19:273
404456499188,10cyclictest0-21swapper/1100:10:073
4044563991816,2cyclictest0-21swapper/1019:15:292
4044563991812,3cyclictest0-21swapper/1021:07:032
404456399180,18cyclictest0-21swapper/1023:56:082
404456399180,18cyclictest0-21swapper/1021:45:222
404456399180,18cyclictest0-21swapper/1021:37:552
404456399180,18cyclictest0-21swapper/1021:37:552
4044562991817,1cyclictest0-21swapper/919:12:5115
404456199182,16cyclictest73-21ksoftirqd/823:57:5914
4044561991816,2cyclictest0-21swapper/822:21:0614
404456199180,1cyclictest0-21swapper/822:38:1814
404456099180,0cyclictest0-21swapper/719:10:0813
404455999181,17cyclictest581rcuc/622:39:1212
404455999180,18cyclictest0-21swapper/619:10:0412
4044555991816,0cyclictest0-21swapper/223:57:508
4044555991813,2cyclictest892016-21snmpd21:17:308
404455599180,16cyclictest0-21swapper/220:12:028
404455499180,18cyclictest0-21swapper/122:37:231
404455499180,16cyclictest0-21swapper/122:45:281
4044568991717,0cyclictest0-21swapper/1522:15:237
4044568991717,0cyclictest0-21swapper/1521:53:377
4044568991717,0cyclictest0-21swapper/1521:44:107
4044568991717,0cyclictest0-21swapper/1521:19:077
404456899170,17cyclictest0-21swapper/1523:43:027
404456899170,17cyclictest0-21swapper/1522:32:267
404456899170,0cyclictest0-21swapper/1523:18:497
404456899170,0cyclictest0-21swapper/1523:18:497
404456899170,0cyclictest0-21swapper/1522:51:387
4044567991717,0cyclictest0-21swapper/1423:36:056
4044567991717,0cyclictest0-21swapper/1422:09:106
404456799170,17cyclictest0-21swapper/1423:43:146
404456799170,17cyclictest0-21swapper/1422:12:366
404456799170,17cyclictest0-21swapper/1421:13:286
404456799170,17cyclictest0-21swapper/1419:10:026
404456799170,14cyclictest0-21swapper/1400:31:536
404456799170,0cyclictest0-21swapper/1421:51:246
4044566991717,0cyclictest0-21swapper/1322:11:155
404456699170,17cyclictest0-21swapper/1323:43:545
404456699170,0cyclictest0-21swapper/1319:10:055
4044565991716,1cyclictest892016-21snmpd00:02:324
4044565991715,2cyclictest0-21swapper/1219:26:044
404456599170,17cyclictest0-21swapper/1221:51:324
404456599170,17cyclictest0-21swapper/1220:48:574
404456599170,17cyclictest0-21swapper/1200:30:414
404456599170,17cyclictest0-21swapper/1200:23:294
404456599170,15cyclictest229502-21ssh00:07:204
404456599170,15cyclictest0-21swapper/1221:44:104
404456599170,15cyclictest0-21swapper/1221:34:554
404456599170,0cyclictest0-21swapper/1219:10:024
4044564991717,0cyclictest0-21swapper/1122:35:243
4044564991717,0cyclictest0-21swapper/1121:53:533
4044564991716,1cyclictest892016-21snmpd23:52:443
4044564991716,1cyclictest892016-21snmpd21:18:573
404456499170,17cyclictest0-21swapper/1122:56:273
404456499170,17cyclictest0-21swapper/1122:16:043
404456499170,17cyclictest0-21swapper/1100:05:373
404456499170,14cyclictest0-21swapper/1100:39:383
4044563991716,1cyclictest0-21swapper/1020:34:472
4044563991716,1cyclictest0-21swapper/1020:16:032
4044563991716,1cyclictest0-21swapper/1019:33:232
4044563991714,3cyclictest3996-21Qt12:00:082
4044563991713,2cyclictest92710-21ssh22:38:482
4044563991712,3cyclictest0-21swapper/1021:32:072
4044563991711,3cyclictest0-21swapper/1023:19:282
4044563991711,3cyclictest0-21swapper/1023:19:282
4044563991711,3cyclictest0-21swapper/1022:00:432
4044563991711,3cyclictest0-21swapper/1021:25:552
404456399170,17cyclictest0-21swapper/1022:56:252
404456399170,17cyclictest0-21swapper/1021:11:022
404456399170,17cyclictest0-21swapper/1019:10:022
404456399170,14cyclictest0-21swapper/1020:20:502
4044562991717,0cyclictest0-21swapper/920:47:5015
404456299170,14cyclictest0-21swapper/923:06:0415
404456299170,0cyclictest0-21swapper/919:16:1215
4044561991717,0cyclictest0-21swapper/823:15:2514
4044561991717,0cyclictest0-21swapper/823:15:2514
4044561991717,0cyclictest0-21swapper/822:17:3414
4044561991717,0cyclictest0-21swapper/821:48:3014
404456199171,16cyclictest73-21ksoftirqd/820:35:2414
404456199170,17cyclictest0-21swapper/823:02:4114
404456199170,16cyclictest0-21swapper/819:10:0314
404456199170,0cyclictest0-21swapper/823:10:5914
4044560991716,1cyclictest892016-21snmpd22:28:3513
404456099170,14cyclictest0-21swapper/723:19:1013
404456099170,14cyclictest0-21swapper/723:19:1013
404456099170,14cyclictest0-21swapper/722:51:3313
404456099170,0cyclictest0-21swapper/700:27:2613
404455999172,15cyclictest59-21ksoftirqd/622:15:4312
4044559991716,1cyclictest0-21swapper/622:55:2412
4044559991715,2cyclictest0-21swapper/621:50:0812
404455999171,16cyclictest59-21ksoftirqd/623:49:3612
404455999170,17cyclictest0-21swapper/622:40:3112
404455999170,0cyclictest0-21swapper/622:14:4312
4044558991717,0cyclictest0-21swapper/521:17:1011
404455899170,17cyclictest0-21swapper/523:08:5111
404455899170,17cyclictest0-21swapper/519:10:0211
404455899170,17cyclictest0-21swapper/500:20:0111
4044557991716,1cyclictest4134055-21kworker/4:223:45:4610
404455799170,17cyclictest0-21swapper/423:59:5210
404455799170,17cyclictest0-21swapper/422:40:3210
404455799170,17cyclictest0-21swapper/422:04:1010
404455799170,17cyclictest0-21swapper/421:15:2010
404455799170,0cyclictest0-21swapper/419:10:0210
4044556991716,1cyclictest892016-21snmpd21:02:429
4044556991716,1cyclictest892016-21snmpd00:27:209
4044556991716,1cyclictest892016-21snmpd00:10:509
404455699170,1cyclictest0-21swapper/321:53:019
404455699170,17cyclictest0-21swapper/319:10:029
404455699170,14cyclictest0-21swapper/321:08:309
4044555991717,0cyclictest0-21swapper/219:10:028
4044555991716,1cyclictest892016-21snmpd21:11:098
4044555991714,3cyclictest892016-21snmpd21:05:218
404455599170,17cyclictest0-21swapper/222:26:548
404455599170,17cyclictest0-21swapper/222:03:168
404455599170,17cyclictest0-21swapper/221:58:508
404455599170,17cyclictest0-21swapper/200:38:208
4044554991716,1cyclictest892016-21snmpd23:11:511
4044554991716,1cyclictest892016-21snmpd22:59:241
4044554991716,1cyclictest3988168-21kworker/1:123:50:011
4044554991716,1cyclictest3988168-21kworker/1:121:58:371
4044554991713,4cyclictest892016-21snmpd23:08:001
404455499170,17cyclictest0-21swapper/122:34:561
404455499170,14cyclictest0-21swapper/122:41:081
404455499170,14cyclictest0-21swapper/119:54:571
404455499170,14cyclictest0-21swapper/119:35:131
404455499170,14cyclictest0-21swapper/119:24:271
4044551991717,0cyclictest0-21swapper/021:43:090
4044551991717,0cyclictest0-21swapper/000:35:260
4044551991717,0cyclictest0-21swapper/000:05:200
4044551991716,1cyclictest892016-21snmpd19:46:400
4044551991716,1cyclictest892016-21snmpd19:44:070
404455199170,17cyclictest0-21swapper/023:37:000
404455199170,17cyclictest0-21swapper/022:24:060
404455199170,17cyclictest0-21swapper/021:48:360
404455199170,0cyclictest0-21swapper/019:10:220
4044568991616,0cyclictest0-21swapper/1523:29:347
4044568991616,0cyclictest0-21swapper/1523:14:497
4044568991616,0cyclictest0-21swapper/1520:40:017
4044568991616,0cyclictest0-21swapper/1500:17:067
4044568991610,3cyclictest0-21swapper/1523:30:207
404456899160,16cyclictest0-21swapper/1523:01:187
404456899160,16cyclictest0-21swapper/1522:59:387
404456899160,16cyclictest0-21swapper/1522:28:237
404456899160,16cyclictest0-21swapper/1522:14:117
404456899160,14cyclictest0-21swapper/1520:25:177
404456899160,0cyclictest0-21swapper/1523:46:107
4044567991616,0cyclictest0-21swapper/1423:15:366
4044567991616,0cyclictest0-21swapper/1423:15:366
4044567991616,0cyclictest0-21swapper/1422:25:556
4044567991616,0cyclictest0-21swapper/1422:01:176
4044567991616,0cyclictest0-21swapper/1421:56:516
4044567991616,0cyclictest0-21swapper/1421:45:586
4044567991616,0cyclictest0-21swapper/1421:29:046
4044567991613,2cyclictest1630-21ladvd20:46:126
404456799160,16cyclictest0-21swapper/1423:10:446
404456799160,16cyclictest0-21swapper/1423:05:436
404456799160,16cyclictest0-21swapper/1422:55:516
404456799160,16cyclictest0-21swapper/1420:20:296
404456799160,16cyclictest0-21swapper/1419:26:136
404456799160,16cyclictest0-21swapper/1400:01:436
404456799160,0cyclictest0-21swapper/1422:50:416
404456799160,0cyclictest0-21swapper/1421:20:036
4044566991616,0cyclictest0-21swapper/1323:33:095
4044566991616,0cyclictest0-21swapper/1323:01:395
4044566991616,0cyclictest0-21swapper/1322:58:225
4044566991616,0cyclictest0-21swapper/1322:25:345
4044566991616,0cyclictest0-21swapper/1321:58:565
4044566991616,0cyclictest0-21swapper/1321:40:375
4044566991616,0cyclictest0-21swapper/1321:29:095
4044566991616,0cyclictest0-21swapper/1321:05:235
4044566991616,0cyclictest0-21swapper/1300:20:595
4044566991614,2cyclictest180386-21awk23:35:175
404456699160,0cyclictest0-21swapper/1322:49:555
4044565991616,0cyclictest0-21swapper/1223:37:324
4044565991616,0cyclictest0-21swapper/1222:34:574
4044565991616,0cyclictest0-21swapper/1221:59:274
4044565991616,0cyclictest0-21swapper/1221:11:034
4044565991615,1cyclictest892016-21snmpd23:51:344
404456599160,16cyclictest0-21swapper/1223:47:504
404456599160,16cyclictest0-21swapper/1222:53:084
404456599160,16cyclictest0-21swapper/1222:46:074
404456599160,16cyclictest0-21swapper/1220:52:094
404456599160,16cyclictest0-21swapper/1220:44:004
404456599160,15cyclictest0-21swapper/1219:31:544
404456599160,14cyclictest0-21swapper/1200:35:544
404456599160,13cyclictest0-21swapper/1222:55:304
4044564991616,0cyclictest0-21swapper/1123:42:083
4044564991616,0cyclictest0-21swapper/1122:28:493
4044564991616,0cyclictest0-21swapper/1100:15:123
4044564991615,1cyclictest892016-21snmpd00:33:173
4044564991614,2cyclictest892016-21snmpd21:55:103
4044564991614,2cyclictest892016-21snmpd21:20:283
4044564991614,2cyclictest892016-21snmpd20:37:233
4044564991614,1cyclictest1556-21ladvd19:53:403
4044563991615,1cyclictest892016-21snmpd20:14:372
4044563991615,1cyclictest0-21swapper/1022:23:212
4044563991615,1cyclictest0-21swapper/1021:51:502
4044563991614,2cyclictest3996-21Qt12:00:082
4044563991614,2cyclictest3996-21Qt12:00:082
4044563991610,3cyclictest0-21swapper/1022:32:082
4044563991610,3cyclictest0-21swapper/1020:50:062
404456399160,16cyclictest0-21swapper/1022:18:112
404456399160,14cyclictest0-21swapper/1022:29:482
4044562991616,0cyclictest0-21swapper/923:54:5115
4044562991616,0cyclictest0-21swapper/923:44:0615
4044562991616,0cyclictest0-21swapper/922:24:2915
4044562991616,0cyclictest0-21swapper/900:34:1315
4044562991615,1cyclictest892016-21snmpd19:59:3115
4044562991615,1cyclictest892016-21snmpd19:52:4815
4044562991615,1cyclictest892016-21snmpd19:31:3315
4044562991615,1cyclictest892016-21snmpd19:25:3615
4044562991614,2cyclictest892016-21snmpd22:46:4815
4044562991614,2cyclictest892016-21snmpd19:23:5115
404456299160,16cyclictest0-21swapper/922:17:2815
404456299160,16cyclictest0-21swapper/900:20:0315
404456299160,14cyclictest0-21swapper/922:40:3215
4044561991616,0cyclictest0-21swapper/822:30:1914
4044561991616,0cyclictest0-21swapper/822:25:3514
4044561991616,0cyclictest0-21swapper/821:40:0314
4044561991616,0cyclictest0-21swapper/800:32:1914
4044561991615,1cyclictest892016-21snmpd22:05:0514
4044561991615,1cyclictest721rcuc/800:13:0214
4044561991615,1cyclictest0-21swapper/823:41:0014
4044561991614,2cyclictest892016-21snmpd22:02:5514
4044561991613,3cyclictest0-21swapper/821:50:0914
4044561991610,3cyclictest0-21swapper/820:07:5914
404456199160,1cyclictest0-21swapper/819:22:3014
404456199160,16cyclictest721rcuc/823:39:1514
404456199160,16cyclictest0-21swapper/820:34:1114
404456199160,16cyclictest0-21swapper/820:22:1414
404456199160,16cyclictest0-21swapper/820:03:0414
404456199160,16cyclictest0-21swapper/819:50:3714
404456199160,16cyclictest0-21swapper/819:27:1014
404456199160,16cyclictest0-21swapper/800:17:0814
404456199160,14cyclictest0-21swapper/820:10:2614
4044560991616,0cyclictest0-21swapper/721:00:2213
4044560991615,1cyclictest892016-21snmpd22:38:2613
4044560991614,2cyclictest892016-21snmpd23:03:5413
4044560991613,2cyclictest1630-21ladvd00:01:5213
404456099160,16cyclictest0-21swapper/722:40:1913
404456099160,16cyclictest0-21swapper/722:15:5013
404456099160,16cyclictest0-21swapper/722:08:4913
404456099160,16cyclictest0-21swapper/700:17:1613
4044559991616,0cyclictest0-21swapper/623:30:3812
4044559991616,0cyclictest0-21swapper/623:25:2012
4044559991616,0cyclictest0-21swapper/622:08:2612
4044559991616,0cyclictest0-21swapper/621:31:2912
4044559991616,0cyclictest0-21swapper/621:19:0212
4044559991616,0cyclictest0-21swapper/600:24:3412
4044559991615,1cyclictest0-21swapper/623:40:5912
4044559991614,2cyclictest4052070-21awk19:15:2312
4044559991614,2cyclictest241691-21awk00:15:1812
404455999160,16cyclictest0-21swapper/621:20:0212
404455999160,16cyclictest0-21swapper/600:31:0312
404455999160,16cyclictest0-21swapper/600:26:3712
4044558991616,0cyclictest0-21swapper/523:23:0211
4044558991616,0cyclictest0-21swapper/522:14:3511
4044558991616,0cyclictest0-21swapper/521:42:0511
4044558991613,2cyclictest1630-21ladvd19:50:1011
404455899160,16cyclictest0-21swapper/521:35:3811
404455899160,16cyclictest0-21swapper/521:35:3811
404455899160,16cyclictest0-21swapper/500:36:5511
404455899160,0cyclictest0-21swapper/522:59:5911
404455899160,0cyclictest0-21swapper/500:28:5511
4044557991616,0cyclictest0-21swapper/423:26:3010
4044557991616,0cyclictest0-21swapper/420:56:3510
4044557991616,0cyclictest0-21swapper/420:56:3510
4044557991616,0cyclictest0-21swapper/419:18:1310
4044557991615,1cyclictest892016-21snmpd00:25:2010
4044557991615,1cyclictest892016-21snmpd00:13:4510
404455799160,16cyclictest0-21swapper/423:10:1110
404455799160,16cyclictest0-21swapper/422:36:2310
404455799160,16cyclictest0-21swapper/421:34:1410
404455799160,16cyclictest0-21swapper/419:57:4410
404455799160,16cyclictest0-21swapper/400:17:5410
404455799160,16cyclictest0-21swapper/400:02:0210
404455799160,0cyclictest0-21swapper/420:33:1510
4044556991616,0cyclictest0-21swapper/322:04:549
4044556991616,0cyclictest0-21swapper/300:20:229
4044556991615,1cyclictest892016-21snmpd20:26:269
4044556991615,1cyclictest892016-21snmpd00:08:209
4044556991614,2cyclictest892016-21snmpd23:35:439
4044556991614,2cyclictest892016-21snmpd21:10:219
4044556991614,2cyclictest892016-21snmpd20:37:539
4044556991610,3cyclictest0-21swapper/321:48:499
404455699160,1cyclictest0-21swapper/300:35:299
404455699160,16cyclictest0-21swapper/323:46:529
404455699160,16cyclictest0-21swapper/323:34:159
404455699160,16cyclictest0-21swapper/322:51:269
404455699160,16cyclictest0-21swapper/322:37:339
404455699160,16cyclictest0-21swapper/321:42:519
404455699160,16cyclictest0-21swapper/321:23:069
404455699160,16cyclictest0-21swapper/321:16:059
404455699160,16cyclictest0-21swapper/300:16:219
404455699160,0cyclictest0-21swapper/300:04:379
4044555991616,0cyclictest0-21swapper/223:39:228
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional