You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-11-26 - 17:44

x86 Intel Core i9-9900K @3600 MHz, Linux 5.15.49-rt47 (Profile)

Latency plot of system in rack #0, slot #4
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot4.osadl.org (updated Sat Nov 26, 2022 00:45:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29384422404398,3sleep140-21swapper/1419:08:096
29384412378360,10sleep130-21swapper/1319:08:085
29384102369337,12sleep50-21swapper/519:07:4511
29384372363343,10sleep90-21swapper/919:08:0415
29384152362344,9sleep100-21swapper/1019:07:502
29384002359340,10sleep120-21swapper/1219:07:364
29384732356338,10sleep80-21swapper/819:08:3614
29384392356337,10sleep110-21swapper/1119:08:073
29384112355334,13sleep60-21swapper/619:07:4612
29384262353332,11sleep150-21swapper/1519:07:547
29384092353332,12sleep40-21swapper/419:07:4410
29384522346328,10sleep70-21swapper/719:08:1913
29383362339321,10sleep10-21swapper/119:06:521
29347632335317,10sleep30-21swapper/319:05:029
29382242330312,10sleep20-21swapper/219:05:328
29382392327309,10sleep00-21swapper/019:05:470
127599370,0ntpd0-21swapper/620:44:5212
2939015993414,20cyclictest0-21swapper/1520:39:167
293898599290,0cyclictest0-21swapper/720:15:1813
293901599240,14cyclictest0-21swapper/1520:55:597
2938988992110,11cyclictest0-21swapper/800:15:3514
293896699210,11cyclictest0-21swapper/220:57:598
293898899206,13cyclictest74-21ksoftirqd/800:00:3914
2938957992020,0cyclictest2825702-21kworker/0:020:39:170
293899999190,16cyclictest0-21swapper/1100:23:423
293897699190,17cyclictest0-21swapper/520:56:0611
293899999180,15cyclictest0-21swapper/1119:26:483
293899599180,18cyclictest0-21swapper/1019:22:182
293899199187,3cyclictest0-21swapper/921:41:4315
293897699180,16cyclictest0-21swapper/523:31:1011
293896199188,10cyclictest0-21swapper/122:20:591
293896199188,10cyclictest0-21swapper/119:59:591
2939015991716,1cyclictest3109408-21ethtool22:00:257
2939015991716,1cyclictest1700-21snmpd23:35:337
293901599170,17cyclictest0-21swapper/1520:45:317
2939004991716,1cyclictest3090488-21nodedev-init21:45:194
293900499171,0cyclictest0-21swapper/1220:35:024
293899599170,14cyclictest0-21swapper/1019:53:312
2938991991714,3cyclictest0-21swapper/922:12:1715
293899199170,14cyclictest0-21swapper/922:42:3615
293899199170,14cyclictest0-21swapper/900:17:4715
2938988991717,0cyclictest0-21swapper/820:39:1714
293898899170,14cyclictest3213491-21ssh23:20:5514
293898899170,14cyclictest3048351-21ssh21:13:1414
293898899170,14cyclictest3048351-21ssh21:13:1414
293898599170,17cyclictest0-21swapper/720:25:1913
293898099170,14cyclictest0-21swapper/623:54:2912
293898099170,14cyclictest0-21swapper/621:05:0812
2938976991716,1cyclictest1700-21snmpd20:42:2011
293897699170,14cyclictest0-21swapper/500:01:5011
293896899170,17cyclictest0-21swapper/319:23:489
293896699170,14cyclictest0-21swapper/200:24:488
293896199177,10cyclictest0-21swapper/123:18:581
293896199170,14cyclictest0-21swapper/100:20:301
293896199170,0cyclictest0-21swapper/120:46:161
293896199170,0cyclictest0-21swapper/100:32:571
2939015991616,0cyclictest1700-21snmpd23:08:437
2939015991616,0cyclictest1700-21snmpd22:36:127
2939015991615,1cyclictest1700-21snmpd19:32:237
2939015991610,3cyclictest0-21swapper/1520:23:087
293901599160,16cyclictest3048552-21ssh21:13:397
293901599160,16cyclictest3048552-21ssh21:13:387
293901599160,16cyclictest0-21swapper/1519:59:367
293901599160,13cyclictest0-21swapper/1520:10:167
2939010991616,0cyclictest1700-21snmpd22:14:026
2939010991615,1cyclictest1700-21snmpd21:00:576
293901099160,16cyclictest0-21swapper/1400:25:206
293901099160,0cyclictest0-21swapper/1423:55:166
293901099160,0cyclictest0-21swapper/1422:46:546
2939007991616,0cyclictest1700-21snmpd00:02:355
293900799160,16cyclictest0-21swapper/1323:10:165
293900799160,16cyclictest0-21swapper/1300:10:235
293900799160,13cyclictest0-21swapper/1320:55:365
2939004991614,2cyclictest1700-21snmpd21:15:424
2939004991614,2cyclictest1700-21snmpd21:15:424
2938999991615,1cyclictest2950854-21ethtool19:20:183
2938999991615,1cyclictest1700-21snmpd22:33:343
2938999991614,2cyclictest1700-21snmpd21:15:063
2938999991614,2cyclictest1700-21snmpd21:15:063
293899999160,13cyclictest0-21swapper/1122:10:203
2938995991616,0cyclictest1700-21snmpd22:23:112
293899599160,16cyclictest0-21swapper/1000:16:442
2938991991615,1cyclictest3237575-21nodedev-init23:40:2015
2938991991615,1cyclictest1700-21snmpd21:13:4715
2938991991615,1cyclictest1700-21snmpd21:13:4715
2938991991612,4cyclictest0-21swapper/919:40:1715
293899199160,15cyclictest0-21swapper/922:39:0615
293899199160,0cyclictest0-21swapper/900:26:3615
293898899160,13cyclictest3092473-21ssh21:46:5714
2938985991615,1cyclictest1700-21snmpd20:42:3213
293898599160,0cyclictest0-21swapper/723:19:1313
293898099160,16cyclictest0-21swapper/600:07:0512
293898099160,0cyclictest0-21swapper/623:40:5312
2938976991616,0cyclictest3160557-21ethtool22:40:1511
2938976991616,0cyclictest1700-21snmpd23:38:3711
2938976991616,0cyclictest1700-21snmpd22:56:3611
293897699160,0cyclictest0-21swapper/521:55:3711
293897699160,0cyclictest0-21swapper/519:31:5111
293897699160,0cyclictest0-21swapper/500:06:2011
2938972991616,0cyclictest1700-21snmpd20:35:0210
2938968991614,2cyclictest1700-21snmpd00:22:519
2938968991610,3cyclictest0-21swapper/319:41:489
2938968991610,3cyclictest0-21swapper/300:35:459
293896899160,14cyclictest0-21swapper/322:29:329
293896899160,0cyclictest0-21swapper/319:56:529
293896699160,16cyclictest0-21swapper/219:40:238
293896699160,16cyclictest0-21swapper/219:26:068
293896699160,13cyclictest0-21swapper/220:11:588
2938961991615,1cyclictest1700-21snmpd00:18:141
2938961991610,3cyclictest0-21swapper/121:14:151
2938961991610,3cyclictest0-21swapper/121:14:151
2938957991616,0cyclictest3278448-21kworker/0:100:24:430
2938957991616,0cyclictest3104976-21kworker/0:122:26:250
2938957991616,0cyclictest2825702-21kworker/0:019:10:120
2938957991616,0cyclictest1700-21snmpd23:20:250
2938957991616,0cyclictest1700-21snmpd19:49:220
2938957991615,1cyclictest3104976-21kworker/0:123:44:550
2938957991615,1cyclictest2825702-21kworker/0:020:05:240
127599160,10ntpd0-21swapper/1423:10:226
2939015991515,0cyclictest3083755-21mii-tool21:40:177
2939015991514,1cyclictest1700-21snmpd23:10:527
2939015991514,1cyclictest1700-21snmpd22:50:487
2939015991514,1cyclictest1700-21snmpd19:29:007
293901599150,15cyclictest0-21swapper/1521:52:467
293901599150,15cyclictest0-21swapper/1520:44:407
293901599150,15cyclictest0-21swapper/1500:39:157
2939010991515,0cyclictest1700-21snmpd19:50:246
2939010991515,0cyclictest0-21swapper/1421:35:456
2939010991515,0cyclictest0-21swapper/1420:25:266
2939010991514,1cyclictest1700-21snmpd23:53:416
2939010991513,2cyclictest1700-21snmpd21:18:246
2939010991513,2cyclictest1700-21snmpd21:18:246
293901099150,15cyclictest0-21swapper/1423:22:556
293901099150,15cyclictest0-21swapper/1421:30:356
293901099150,15cyclictest0-21swapper/1420:36:556
293901099150,0cyclictest0-21swapper/1423:38:096
293900799159,3cyclictest0-21swapper/1319:44:555
2939007991514,1cyclictest1700-21snmpd19:31:155
2939007991513,2cyclictest1700-21snmpd00:28:155
2939007991512,1cyclictest0-21swapper/1320:05:205
2939007991510,2cyclictest0-21swapper/1321:22:545
2939007991510,2cyclictest0-21swapper/1321:22:535
293900799150,15cyclictest0-21swapper/1320:34:315
293900799150,12cyclictest0-21swapper/1320:37:475
2939004991514,1cyclictest1700-21snmpd23:48:534
2939004991514,1cyclictest1700-21snmpd00:00:504
2939004991513,2cyclictest1700-21snmpd22:11:164
293900499150,15cyclictest0-21swapper/1223:27:284
293900499150,15cyclictest0-21swapper/1220:54:064
293900499150,0cyclictest0-21swapper/1219:23:084
293899999159,3cyclictest0-21swapper/1121:52:133
2938999991515,0cyclictest1700-21snmpd23:49:143
2938999991515,0cyclictest1700-21snmpd20:26:583
2938999991514,1cyclictest3115811-21cat22:05:203
2938999991513,2cyclictest1700-21snmpd23:02:453
2938999991510,2cyclictest0-21swapper/1123:27:043
293899999150,15cyclictest0-21swapper/1123:55:223
293899999150,15cyclictest0-21swapper/1123:20:153
293899999150,15cyclictest0-21swapper/1119:40:203
293899999150,12cyclictest0-21swapper/1121:10:393
293899999150,12cyclictest0-21swapper/1121:10:393
293899599159,3cyclictest0-21swapper/1023:30:042
293899599159,3cyclictest0-21swapper/1020:33:202
2938995991515,0cyclictest1700-21snmpd20:10:392
2938995991515,0cyclictest1700-21snmpd20:06:152
2938995991515,0cyclictest1700-21snmpd19:41:412
293899599150,15cyclictest0-21swapper/1023:27:192
293899599150,15cyclictest0-21swapper/1022:42:412
293899599150,15cyclictest0-21swapper/1021:43:342
293899599150,12cyclictest0-21swapper/1021:52:072
293899599150,0cyclictest0-21swapper/1019:15:002
2938991991514,1cyclictest1700-21snmpd00:32:3315
2938991991513,2cyclictest1700-21snmpd20:15:1415
2938991991513,2cyclictest0-21swapper/922:24:5315
293899199150,15cyclictest0-21swapper/922:05:0515
2938988991515,0cyclictest1700-21snmpd21:35:1614
293898899150,15cyclictest0-21swapper/823:10:1314
293898899150,15cyclictest0-21swapper/819:50:2214
293898899150,0cyclictest2941817-21ntp_states19:10:2114
2938985991515,0cyclictest1700-21snmpd20:13:1413
2938985991514,1cyclictest1700-21snmpd22:19:4713
2938985991514,1cyclictest1700-21snmpd20:03:0713
2938985991510,2cyclictest0-21swapper/721:39:5513
293898599150,15cyclictest0-21swapper/723:25:1913
293898599150,15cyclictest0-21swapper/722:48:1713
293898599150,15cyclictest0-21swapper/700:25:2613
293898599150,0cyclictest0-21swapper/722:34:3513
2938980991515,0cyclictest0-21swapper/622:10:4812
2938980991515,0cyclictest0-21swapper/620:25:1612
2938980991514,1cyclictest1700-21snmpd23:16:5512
2938980991514,1cyclictest1700-21snmpd21:32:5212
293898099150,15cyclictest0-21swapper/623:35:2612
293898099150,12cyclictest0-21swapper/621:57:4912
293898099150,12cyclictest0-21swapper/600:15:5312
293898099150,0cyclictest0-21swapper/619:34:3312
293898099150,0cyclictest0-21swapper/600:35:1712
2938976991515,0cyclictest3109412-21mii-tool22:00:2411
2938976991515,0cyclictest1700-21snmpd23:50:0511
2938976991514,1cyclictest1700-21snmpd22:46:0911
2938976991514,1cyclictest1700-21snmpd21:11:2411
2938976991514,1cyclictest1700-21snmpd21:11:2411
2938976991514,1cyclictest1700-21snmpd19:45:2411
2938976991510,2cyclictest0-21swapper/522:30:5811
293897699150,15cyclictest0-21swapper/500:35:4011
293897699150,13cyclictest0-21swapper/521:17:5711
293897699150,13cyclictest0-21swapper/521:17:5611
293897699150,12cyclictest0-21swapper/521:00:2611
2938972991515,0cyclictest3077402-21mii-tool21:35:2210
2938972991514,1cyclictest1700-21snmpd21:29:5510
2938968991515,0cyclictest1700-21snmpd22:45:099
2938968991514,1cyclictest1700-21snmpd23:36:169
2938968991510,3cyclictest0-21swapper/322:23:499
293896899150,15cyclictest0-21swapper/319:27:189
293896899150,15cyclictest0-21swapper/300:33:479
293896699155,10cyclictest0-21swapper/220:45:588
2938966991515,0cyclictest3224510-21mii-tool23:30:238
2938966991515,0cyclictest1700-21snmpd21:43:438
2938966991515,0cyclictest1700-21snmpd20:08:258
2938966991515,0cyclictest1700-21snmpd19:19:518
2938966991515,0cyclictest0-21swapper/222:23:198
2938966991514,1cyclictest1700-21snmpd22:53:308
2938966991513,2cyclictest1700-21snmpd22:44:098
2938966991513,2cyclictest1700-21snmpd20:29:208
293896699150,15cyclictest0-21swapper/223:05:528
293896699150,12cyclictest0-21swapper/200:33:058
293896699150,0cyclictest0-21swapper/222:10:448
293896699150,0cyclictest0-21swapper/221:37:488
293896699150,0cyclictest0-21swapper/221:14:598
293896699150,0cyclictest0-21swapper/221:14:598
2938961991515,0cyclictest1700-21snmpd22:41:571
2938961991515,0cyclictest1700-21snmpd22:07:081
2938961991515,0cyclictest1700-21snmpd21:20:091
2938961991515,0cyclictest1700-21snmpd21:20:091
2938961991515,0cyclictest1700-21snmpd19:15:351
2938961991515,0cyclictest0-21swapper/121:25:221
2938961991514,1cyclictest1700-21snmpd23:24:101
2938961991514,1cyclictest1700-21snmpd21:45:101
2938961991510,2cyclictest0-21swapper/123:57:431
293896199150,13cyclictest0-21swapper/121:52:041
293896199150,0cyclictest0-21swapper/123:40:091
2938957991515,0cyclictest3246763-21kworker/0:023:50:170
2938957991515,0cyclictest3104976-21kworker/0:123:31:130
2938957991515,0cyclictest3104976-21kworker/0:123:25:270
2938957991515,0cyclictest3104976-21kworker/0:123:06:060
2938957991515,0cyclictest3104976-21kworker/0:122:50:130
2938957991515,0cyclictest3104976-21kworker/0:122:46:150
2938957991515,0cyclictest3104976-21kworker/0:122:36:570
2938957991515,0cyclictest3104976-21kworker/0:122:34:370
2938957991515,0cyclictest3104976-21kworker/0:122:10:190
2938957991515,0cyclictest3098304-21kworker/0:221:55:210
2938957991515,0cyclictest2825702-21kworker/0:021:45:250
2938957991515,0cyclictest2825702-21kworker/0:021:35:110
2938957991515,0cyclictest2825702-21kworker/0:021:34:020
2938957991515,0cyclictest2825702-21kworker/0:021:23:210
2938957991515,0cyclictest2825702-21kworker/0:021:23:210
2938957991515,0cyclictest2825702-21kworker/0:020:20:250
2938957991515,0cyclictest2825702-21kworker/0:019:25:210
2938957991515,0cyclictest1700-21snmpd00:10:200
2938957991514,1cyclictest3278448-21kworker/0:100:29:360
2938957991514,1cyclictest3104976-21kworker/0:123:10:210
2938957991514,1cyclictest1700-21snmpd22:02:100
2938957991514,1cyclictest1700-21snmpd20:19:500
2938957991514,1cyclictest1700-21snmpd19:19:450
2938957991513,2cyclictest1700-21snmpd00:07:260
127599150,9ntpd0-21swapper/1419:45:216
2939015991414,0cyclictest1700-21snmpd21:59:407
2939015991414,0cyclictest0-21swapper/1523:55:207
2939015991414,0cyclictest0-21swapper/1523:29:427
2939015991414,0cyclictest0-21swapper/1523:03:377
2939015991414,0cyclictest0-21swapper/1522:55:207
2939015991414,0cyclictest0-21swapper/1522:24:247
2939015991414,0cyclictest0-21swapper/1522:13:237
2939015991414,0cyclictest0-21swapper/1521:35:267
2939015991414,0cyclictest0-21swapper/1521:17:327
2939015991414,0cyclictest0-21swapper/1521:17:317
2939015991414,0cyclictest0-21swapper/1520:15:157
2939015991414,0cyclictest0-21swapper/1500:30:087
2939015991413,1cyclictest2941171-21cat19:10:167
293901599140,14cyclictest0-21swapper/1523:49:537
293901599140,0cyclictest0-21swapper/1520:05:257
293901599140,0cyclictest0-21swapper/1520:00:187
2939010991414,0cyclictest1700-21snmpd23:05:456
2939010991414,0cyclictest0-21swapper/1422:32:536
2939010991414,0cyclictest0-21swapper/1421:51:306
2939010991414,0cyclictest0-21swapper/1400:30:016
2939010991413,1cyclictest1700-21snmpd23:34:066
2939010991413,1cyclictest1700-21snmpd21:58:136
293901099140,14cyclictest0-21swapper/1423:01:486
293901099140,14cyclictest0-21swapper/1422:56:146
293901099140,0cyclictest0-21swapper/1422:21:516
293901099140,0cyclictest0-21swapper/1422:15:586
293901099140,0cyclictest0-21swapper/1400:12:076
2939007991414,0cyclictest3070986-21mii-tool21:30:215
2939007991414,0cyclictest0-21swapper/1322:58:175
2939007991414,0cyclictest0-21swapper/1300:36:495
2939007991414,0cyclictest0-21swapper/1300:31:065
2939007991412,1cyclictest0-21swapper/1323:00:235
293900799140,14cyclictest0-21swapper/1323:25:185
293900799140,14cyclictest0-21swapper/1323:05:195
293900799140,14cyclictest0-21swapper/1320:42:025
293900799140,0cyclictest0-21swapper/1323:40:415
293900799140,0cyclictest0-21swapper/1323:30:065
293900799140,0cyclictest0-21swapper/1321:25:565
293900799140,0cyclictest0-21swapper/1321:11:575
293900799140,0cyclictest0-21swapper/1321:11:565
293900799140,0cyclictest0-21swapper/1319:58:225
2939004991414,0cyclictest1700-21snmpd20:57:334
2939004991414,0cyclictest1700-21snmpd20:29:284
2939004991414,0cyclictest0-21swapper/1223:30:394
2939004991414,0cyclictest0-21swapper/1221:57:514
2939004991414,0cyclictest0-21swapper/1221:38:234
2939004991413,1cyclictest1700-21snmpd00:27:514
293900499140,14cyclictest0-21swapper/1223:50:164
293900499140,14cyclictest0-21swapper/1223:23:554
293900499140,14cyclictest0-21swapper/1223:05:184
293900499140,14cyclictest0-21swapper/1222:44:154
293900499140,14cyclictest0-21swapper/1222:25:154
293900499140,14cyclictest0-21swapper/1222:03:524
293900499140,14cyclictest0-21swapper/1221:22:274
293900499140,14cyclictest0-21swapper/1221:22:264
293900499140,0cyclictest0-21swapper/1221:32:574
293899999149,2cyclictest0-21swapper/1121:22:473
293899999149,2cyclictest0-21swapper/1121:22:473
2938999991414,0cyclictest2963730-21mii-tool19:35:203
2938999991414,0cyclictest1700-21snmpd23:44:293
2938999991413,1cyclictest0-21swapper/1122:51:133
293899999140,14cyclictest0-21swapper/1122:45:263
293899999140,14cyclictest0-21swapper/1120:50:483
293899999140,0cyclictest0-21swapper/1123:36:073
293899999140,0cyclictest0-21swapper/1100:37:593
293899599149,2cyclictest0-21swapper/1021:45:222
2938995991414,0cyclictest1700-21snmpd19:26:092
2938995991414,0cyclictest0-21swapper/1023:36:492
2938995991414,0cyclictest0-21swapper/1023:18:122
2938995991413,1cyclictest1700-21snmpd22:59:422
2938995991413,1cyclictest1700-21snmpd21:36:182
2938995991413,1cyclictest1700-21snmpd20:50:032
293899599140,14cyclictest0-21swapper/1023:50:332
293899599140,14cyclictest0-21swapper/1023:14:192
293899599140,14cyclictest0-21swapper/1021:20:272
293899599140,14cyclictest0-21swapper/1021:20:262
293899599140,14cyclictest0-21swapper/1000:37:512
293899599140,14cyclictest0-21swapper/1000:27:132
293899599140,0cyclictest0-21swapper/1022:09:272
293899599140,0cyclictest0-21swapper/1020:29:372
2938991991413,1cyclictest1700-21snmpd00:22:0315
2938991991413,1cyclictest0-21swapper/921:50:0515
2938991991411,3cyclictest1700-21snmpd20:00:2215
2938991991411,3cyclictest0-21swapper/923:45:1515
293899199140,14cyclictest0-21swapper/923:12:1915
293899199140,14cyclictest0-21swapper/919:55:0515
293899199140,14cyclictest0-21swapper/919:16:4215
293899199140,14cyclictest0-21swapper/900:06:4915
293899199140,0cyclictest0-21swapper/922:52:1415
293899199140,0cyclictest0-21swapper/921:26:1815
293899199140,0cyclictest0-21swapper/900:36:2715
293899199140,0cyclictest0-21swapper/900:13:1115
2938988991414,0cyclictest1700-21snmpd20:03:1614
2938988991414,0cyclictest0-21swapper/822:41:4014
293898899140,14cyclictest3258116-21ssh23:55:4314
293898899140,14cyclictest3220479-21ssh23:27:0714
293898899140,14cyclictest3111116-21ssh22:00:5114
293898899140,14cyclictest0-21swapper/821:20:1514
293898899140,14cyclictest0-21swapper/821:20:1414
293898899140,0cyclictest3105351-21ssh21:57:0514
293898899140,0cyclictest0-21swapper/819:20:2014
293898599149,2cyclictest0-21swapper/721:00:4513
293898599148,3cyclictest0-21swapper/719:30:1913
2938985991414,0cyclictest1700-21snmpd20:20:5913
2938985991414,0cyclictest1700-21snmpd20:07:5213
2938985991413,1cyclictest1700-21snmpd20:45:0513
293898599140,14cyclictest0-21swapper/722:37:0613
293898599140,14cyclictest0-21swapper/722:08:0713
293898599140,14cyclictest0-21swapper/721:30:1013
293898599140,14cyclictest0-21swapper/721:25:3413
293898599140,14cyclictest0-21swapper/719:36:1213
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional