You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-12-04 - 02:01

x86 Intel Core i7-X990 @3467 MHz, Linux 4.9.47-rt37 (Profile)

Latency plot of system in rack #c, slot #1
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackcslot1.osadl.org (updated Sat Dec 03, 2022 12:48:19)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2389321270,1sleep40-21swapper/407:05:126
390799980,93rtkit-daemon0-21swapper/507:06:377
293772950,1sleep100-21swapper/1011:41:362
2573628565,6sleep90-21swapper/907:05:5511
2517827768,6sleep10-21swapper/107:05:231
2573027566,6sleep30-21swapper/307:05:495
2580527460,12sleep110-21swapper/1107:06:563
117822710,2sleep90-21swapper/911:20:2111
2586526960,6sleep20-21swapper/207:07:484
2580426758,6sleep100-21swapper/1007:06:562
2600026557,6sleep80-21swapper/807:09:4210
2583726456,6sleep00-21swapper/007:07:220
2570726444,7sleep70-21swapper/707:05:289
2585626343,6sleep60-21swapper/607:07:408
80342600,1sleep60-21swapper/607:30:188
174112550,2sleep22643599cyclictest12:06:184
2644099293,21cyclictest0-21swapper/311:25:005
2647099284,18cyclictest0-21swapper/810:42:1210
2647099275,15cyclictest0-21swapper/809:25:0110
2646399275,15cyclictest0-21swapper/712:05:019
2645799254,15cyclictest0-21swapper/609:05:028
2647599245,13cyclictest0-21swapper/911:52:3411
2646399243,15cyclictest0-21swapper/712:15:029
2644099244,15cyclictest0-21swapper/308:35:015
2642899244,14cyclictest0-21swapper/110:20:001
2647099234,13cyclictest0-21swapper/807:35:0210
2647099233,15cyclictest0-21swapper/811:55:0210
2646399235,13cyclictest0-21swapper/711:40:019
2646399234,14cyclictest0-21swapper/707:45:029
2645799235,15cyclictest0-21swapper/612:20:028
2645799234,13cyclictest0-21swapper/607:55:018
2645399235,13cyclictest0-21swapper/511:30:017
2648799224,13cyclictest0-21swapper/1112:35:023
2648799224,13cyclictest0-21swapper/1108:10:023
2647599224,13cyclictest0-21swapper/911:15:0111
2646399224,13cyclictest0-21swapper/711:20:019
2646399223,14cyclictest0-21swapper/712:40:029
2646399223,14cyclictest0-21swapper/712:40:019
2645799225,12cyclictest0-21swapper/612:10:028
2645399223,12cyclictest0-21swapper/511:05:017
2644699221,5cyclictest0-21swapper/411:44:406
2642899224,13cyclictest0-21swapper/107:30:021
2647599214,12cyclictest0-21swapper/910:05:0111
2647099214,12cyclictest0-21swapper/808:05:0210
2646399214,12cyclictest0-21swapper/711:45:019
2646399214,12cyclictest0-21swapper/709:15:019
2646399213,13cyclictest0-21swapper/709:55:009
2645799214,12cyclictest0-21swapper/608:30:028
2645799214,12cyclictest0-21swapper/608:15:028
2645799213,13cyclictest0-21swapper/611:40:018
2645799213,13cyclictest0-21swapper/609:00:028
2645399214,12cyclictest0-21swapper/511:00:007
2644099214,13cyclictest0-21swapper/310:20:015
2644099214,12cyclictest0-21swapper/308:30:025
2644099213,13cyclictest0-21swapper/309:15:015
2644099213,13cyclictest0-21swapper/308:10:015
2643599214,13cyclictest0-21swapper/208:15:014
2648799207,12cyclictest0-21swapper/1111:20:003
2648799204,11cyclictest0-21swapper/1111:05:003
2648799204,11cyclictest0-21swapper/1107:30:013
2648799203,12cyclictest0-21swapper/1108:50:023
2647099203,13cyclictest0-21swapper/809:35:0010
2645799203,12cyclictest0-21swapper/608:55:028
2645399204,12cyclictest0-21swapper/512:35:017
2645399202,13cyclictest0-21swapper/510:54:367
2644099205,10cyclictest0-21swapper/311:35:015
2644099204,12cyclictest0-21swapper/307:35:015
2642399204,12cyclictest0-21swapper/009:25:000
2648799194,12cyclictest0-21swapper/1109:35:013
2646399194,10cyclictest0-21swapper/710:10:019
2646399190,2cyclictest0-21swapper/712:23:289
2646399190,18cyclictest0-21swapper/712:19:199
2645799196,7cyclictest22156-1kworker/6:0H11:45:018
2645799193,12cyclictest0-21swapper/610:50:018
2645399193,11cyclictest0-21swapper/507:35:027
2644099193,13cyclictest0-21swapper/309:55:015
2644099191,13cyclictest0-21swapper/312:35:025
2644099191,13cyclictest0-21swapper/312:35:015
2642399193,12cyclictest0-21swapper/008:25:020
2642399193,11cyclictest0-21swapper/012:35:010
2642399193,11cyclictest0-21swapper/007:55:010
2642399191,13cyclictest0-21swapper/009:10:010
26487991814,3cyclictest0-21swapper/1111:26:493
2648299183,10cyclictest0-21swapper/1010:50:012
2647099185,10cyclictest0-21swapper/810:00:0110
2646399184,13cyclictest0-21swapper/709:20:009
2646399180,12cyclictest0-21swapper/711:00:589
2643599183,11cyclictest0-21swapper/210:45:004
2642399184,8cyclictest0-21swapper/011:25:010
2648799173,10cyclictest0-21swapper/1108:45:013
2648799170,16cyclictest0-21swapper/1107:11:233
2648299170,17cyclictest0-21swapper/1007:12:362
2647599170,2cyclictest14070-21ssh10:43:5611
2646399170,2cyclictest0-21swapper/711:51:479
2646399170,11cyclictest0-21swapper/711:26:029
2645799173,9cyclictest0-21swapper/609:45:008
2645399170,3cyclictest0-21swapper/510:37:437
2645399170,1cyclictest29877-21ssh12:22:267
2645399170,13cyclictest0-21swapper/512:08:147
2645399170,11cyclictest0-21swapper/511:45:107
2643599173,13cyclictest0-21swapper/212:30:014
2642399179,3cyclictest0-21swapper/010:35:090
2642399170,12cyclictest0-21swapper/011:56:120
2642399170,11cyclictest17259-21ssh12:05:490
39892160,0sleep40-21swapper/411:50:196
2647599161,14cyclictest0-21swapper/908:30:0111
2647599160,1cyclictest9616-21ssh11:57:1511
2647599160,15cyclictest0-21swapper/912:22:3811
2647599160,11cyclictest30187-21ssh11:44:4511
2647099163,10cyclictest0-21swapper/810:47:2310
2647099161,10cyclictest0-21swapper/811:50:0110
26463991610,5cyclictest6937-21ssh11:55:229
2646399160,2cyclictest0-21swapper/710:40:459
2646399160,15cyclictest0-21swapper/712:28:499
2646399160,11cyclictest0-21swapper/712:06:049
2645399164,10cyclictest0-21swapper/512:10:027
2645399160,15cyclictest0-21swapper/512:17:037
2645399160,15cyclictest0-21swapper/511:24:497
2645399160,15cyclictest0-21swapper/509:25:207
2645399160,12cyclictest0-21swapper/511:51:447
26440991612,3cyclictest0-21swapper/312:17:595
26435991611,5cyclictest0-21swapper/212:15:254
2643599161,10cyclictest0-21swapper/209:20:014
2642899163,9cyclictest0-21swapper/108:52:111
2642399163,9cyclictest0-21swapper/009:05:020
2642399161,9cyclictest0-21swapper/007:30:020
2642399160,11cyclictest0-21swapper/010:47:390
26487991511,3cyclictest0-21swapper/1112:01:273
2647599159,5cyclictest31945-21ssh11:45:2211
2647099153,11cyclictest0-21swapper/808:40:0210
26470991512,2cyclictest18454-21ssh12:10:0110
2646399150,4cyclictest0-21swapper/711:24:569
2646399150,14cyclictest30279-21scp11:45:039
2646399150,14cyclictest0-21swapper/709:50:019
2646399150,11cyclictest0-21swapper/710:34:459
2646399150,10cyclictest0-21swapper/709:34:509
2645799150,14cyclictest0-21swapper/612:14:318
2645399150,9cyclictest0-21swapper/509:58:057
2645399150,9cyclictest0-21swapper/508:18:457
2645399150,9cyclictest0-21swapper/507:45:197
2645399150,2cyclictest0-21swapper/510:24:347
2645399150,2cyclictest0-21swapper/510:11:197
2645399150,14cyclictest0-21swapper/509:47:167
2645399150,14cyclictest0-21swapper/509:31:057
2645399150,11cyclictest0-21swapper/510:33:457
2645399150,10cyclictest0-21swapper/512:27:447
2644699153,11cyclictest0-21swapper/411:35:016
2644099151,13cyclictest0-21swapper/312:00:185
2644099150,14cyclictest0-21swapper/312:21:305
2644099150,14cyclictest0-21swapper/312:12:485
2644099150,14cyclictest0-21swapper/312:09:555
2643599153,8cyclictest0-21swapper/208:40:014
2643599150,14cyclictest0-21swapper/212:20:044
26428991511,3cyclictest0-21swapper/112:19:071
2642899150,13cyclictest0-21swapper/109:40:191
26423991510,4cyclictest0-21swapper/011:50:160
2642399150,10cyclictest21514-21ssh10:51:420
102282150,1sleep40-21swapper/411:59:316
26487991410,3cyclictest0-21swapper/1111:35:233
26487991410,3cyclictest0-21swapper/1110:59:203
2648799140,13cyclictest0-21swapper/1112:20:293
2648799140,13cyclictest0-21swapper/1111:33:523
2648799140,13cyclictest0-21swapper/1111:21:303
2648299140,3cyclictest0-21swapper/1012:15:232
2648299140,13cyclictest0-21swapper/1011:56:252
26475991410,3cyclictest0-21swapper/912:15:4411
2647599140,1cyclictest21921-21ssh10:53:2011
2647599140,13cyclictest0-21swapper/912:34:1511
2647599140,13cyclictest0-21swapper/910:29:5611
2647099141,12cyclictest0-21swapper/810:05:2010
2647099140,13cyclictest0-21swapper/812:18:3310
2647099140,13cyclictest0-21swapper/811:44:3510
2647099140,13cyclictest0-21swapper/811:30:2710
2647099140,13cyclictest0-21swapper/811:13:5010
2647099140,12cyclictest0-21swapper/810:38:0010
2646399141,7cyclictest0-21swapper/709:05:149
2646399140,9cyclictest0-21swapper/707:22:299
2646399140,2cyclictest0-21swapper/709:44:339
2646399140,2cyclictest0-21swapper/709:19:319
2646399140,2cyclictest0-21swapper/707:39:419
2646399140,2cyclictest0-21swapper/707:31:119
2646399140,2cyclictest0-21swapper/707:17:109
2646399140,13cyclictest0-21swapper/708:49:369
26457991410,3cyclictest0-21swapper/612:23:248
26457991410,3cyclictest0-21swapper/612:01:208
26457991410,3cyclictest0-21swapper/611:56:448
26457991410,3cyclictest0-21swapper/610:51:528
26457991410,3cyclictest0-21swapper/609:25:018
2645399142,6cyclictest0-21swapper/512:39:237
2645399142,6cyclictest0-21swapper/512:39:237
2645399142,6cyclictest0-21swapper/507:26:017
2645399142,5cyclictest0-21swapper/508:32:317
2645399142,5cyclictest0-21swapper/507:50:317
2645399142,11cyclictest0-21swapper/509:11:017
2645399142,11cyclictest0-21swapper/508:07:227
2645399141,7cyclictest0-21swapper/509:01:017
2645399141,6cyclictest0-21swapper/508:40:047
2645399141,5cyclictest0-21swapper/507:11:437
26453991410,3cyclictest0-21swapper/511:55:277
26453991410,3cyclictest0-21swapper/511:43:367
26453991410,3cyclictest0-21swapper/511:30:197
2645399140,9cyclictest0-21swapper/511:16:167
2645399140,9cyclictest0-21swapper/507:22:107
2645399140,9cyclictest0-21swapper/507:16:377
2645399140,2cyclictest0-21swapper/510:40:117
2645399140,2cyclictest0-21swapper/509:40:397
2645399140,2cyclictest0-21swapper/508:50:307
2645399140,1cyclictest0-21swapper/510:00:527
2645399140,13cyclictest0-21swapper/512:04:397
2645399140,13cyclictest0-21swapper/511:37:377
2645399140,13cyclictest0-21swapper/509:50:577
2644699144,9cyclictest0-21swapper/411:10:016
2644699143,9cyclictest0-21swapper/408:20:026
2644699143,9cyclictest0-21swapper/407:55:016
2644699143,9cyclictest0-21swapper/407:35:016
2644699143,10cyclictest0-21swapper/412:40:026
2644699143,10cyclictest0-21swapper/412:40:016
2644699143,10cyclictest0-21swapper/412:30:026
2644699142,11cyclictest0-21swapper/408:40:026
2644699140,13cyclictest0-21swapper/412:00:156
2644699140,13cyclictest0-21swapper/411:19:256
2643599140,14cyclictest0-21swapper/212:00:524
2643599140,13cyclictest21336-21ssh12:10:464
2643599140,13cyclictest0-21swapper/210:53:284
2642899140,13cyclictest0-21swapper/111:54:461
26423991410,3cyclictest0-21swapper/011:47:250
2642399140,13cyclictest0-21swapper/012:20:190
24422140,0sleep90-21swapper/907:20:1811
200602140,1sleep110-21swapper/1107:50:133
2648799139,3cyclictest0-21swapper/1110:32:533
26487991310,2cyclictest17890-21ssh12:08:103
2648799130,3cyclictest0-21swapper/1111:41:513
2648799130,12cyclictest0-21swapper/1112:10:183
2648799130,12cyclictest0-21swapper/1111:55:193
2648799130,12cyclictest0-21swapper/1111:53:313
2648799130,12cyclictest0-21swapper/1110:52:423
2648799130,12cyclictest0-21swapper/1107:45:203
26482991310,2cyclictest0-21swapper/1011:14:102
2648299130,12cyclictest0-21swapper/1012:11:112
2648299130,12cyclictest0-21swapper/1012:08:462
2648299130,12cyclictest0-21swapper/1011:06:202
2648299130,12cyclictest0-21swapper/1011:00:442
2648299130,12cyclictest0-21swapper/1010:35:332
2647599139,3cyclictest0-21swapper/912:06:2711
2647599138,4cyclictest0-21swapper/912:13:3811
2647599130,9cyclictest0-21swapper/910:10:2311
2647599130,9cyclictest0-21swapper/909:42:1711
2647599130,9cyclictest0-21swapper/907:52:0011
2647599130,9cyclictest0-21swapper/907:33:0711
2647599130,1cyclictest26068-21ssh10:59:0011
2647599130,12cyclictest0-21swapper/912:01:3511
2647599130,12cyclictest0-21swapper/911:33:5911
2647599130,12cyclictest0-21swapper/909:20:0111
2647099134,8cyclictest0-21swapper/807:15:0210
2647099134,7cyclictest0-21swapper/810:32:3110
2647099132,8cyclictest0-21swapper/811:38:2110
2647099131,11cyclictest0-21swapper/811:05:5110
2647099131,11cyclictest0-21swapper/809:50:1910
2647099130,12cyclictest0-21swapper/812:10:4010
2647099130,12cyclictest0-21swapper/811:29:2110
2647099130,12cyclictest0-21swapper/811:23:5310
2647099130,12cyclictest0-21swapper/811:02:1010
2646399139,3cyclictest0-21swapper/710:57:449
2646399131,5cyclictest0-21swapper/710:25:569
2646399131,4cyclictest0-21swapper/710:25:019
2646399131,4cyclictest0-21swapper/707:55:209
2646399131,4cyclictest0-21swapper/707:45:139
2646399130,3cyclictest0-21swapper/711:10:119
2646399130,3cyclictest0-21swapper/710:35:469
2646399130,2cyclictest0-21swapper/710:15:289
2646399130,2cyclictest0-21swapper/708:58:199
2646399130,2cyclictest0-21swapper/708:15:049
2646399130,13cyclictest0-21swapper/712:32:179
2646399130,13cyclictest0-21swapper/708:53:589
2646399130,12cyclictest0-21swapper/711:08:379
2646399130,12cyclictest0-21swapper/710:45:059
2646399130,12cyclictest0-21swapper/710:12:339
26457991310,2cyclictest12540-21sensors_temp11:20:278
2645799130,12cyclictest0-21swapper/611:50:128
2645799130,12cyclictest0-21swapper/611:45:498
2645799130,12cyclictest0-21swapper/611:34:478
2645799130,12cyclictest0-21swapper/611:10:228
2645799130,12cyclictest0-21swapper/610:55:158
2645399139,3cyclictest0-21swapper/511:00:257
2645399133,3cyclictest0-21swapper/510:16:547
2645399133,3cyclictest0-21swapper/509:24:297
2645399132,4cyclictest0-21swapper/508:47:377
2645399131,6cyclictest0-21swapper/509:36:137
2645399131,6cyclictest0-21swapper/508:56:197
2645399131,6cyclictest0-21swapper/508:02:287
2645399131,6cyclictest0-21swapper/507:41:307
2645399131,11cyclictest0-21swapper/508:15:017
2645399130,1cyclictest0-21swapper/509:07:197
2645399130,1cyclictest0-21swapper/508:39:127
2645399130,13cyclictest0-21swapper/510:05:037
2645399130,12cyclictest0-21swapper/511:11:317
2644699130,12cyclictest0-21swapper/412:18:316
2644699130,12cyclictest0-21swapper/412:07:366
2644699130,12cyclictest0-21swapper/411:47:256
2644699130,12cyclictest0-21swapper/411:26:306
2644099139,3cyclictest0-21swapper/311:52:415
2644099131,11cyclictest0-21swapper/311:10:205
2644099130,13cyclictest0-21swapper/312:25:215
2644099130,12cyclictest0-21swapper/312:32:295
2642899130,2cyclictest0-21swapper/112:05:471
2642899130,2cyclictest0-21swapper/111:39:191
2642899130,12cyclictest2612-21cat10:30:011
2642899130,12cyclictest13290-21awk12:01:021
2642899130,12cyclictest0-21swapper/112:21:391
2642899130,12cyclictest0-21swapper/112:11:301
2642899130,12cyclictest0-21swapper/111:57:201
2642899130,12cyclictest0-21swapper/111:48:001
2642899130,12cyclictest0-21swapper/111:43:401
2642899130,12cyclictest0-21swapper/110:42:101
2642899130,12cyclictest0-21swapper/110:36:241
2642399139,3cyclictest0-21swapper/011:31:410
26423991310,2cyclictest0-21swapper/012:16:240
2642399130,12cyclictest0-21swapper/012:01:050
2642399130,12cyclictest0-21swapper/011:40:180
2642399130,12cyclictest0-21swapper/010:41:160
2642399130,11cyclictest0-21swapper/008:45:220
2648799129,2cyclictest671-21ssh11:45:423
2648799129,2cyclictest26409-21awk10:20:013
26487991210,2cyclictest0-21swapper/1110:46:333
2648799120,3cyclictest0-21swapper/1109:57:463
2648799120,11cyclictest0-21swapper/1112:15:533
2648799120,11cyclictest0-21swapper/1111:13:263
2648799120,11cyclictest0-21swapper/1109:38:563
2648799120,11cyclictest0-21swapper/1107:40:223
26482991210,2cyclictest0-21swapper/1012:00:442
26482991210,2cyclictest0-21swapper/1011:25:222
2648299120,2cyclictest0-21swapper/1011:24:132
2648299120,2cyclictest0-21swapper/1011:16:552
2648299120,12cyclictest0-21swapper/1011:51:582
2648299120,11cyclictest0-21swapper/1011:46:032
2648299120,11cyclictest0-21swapper/1011:38:582
2648299120,11cyclictest0-21swapper/1010:42:542
2648299120,11cyclictest0-21swapper/1009:55:132
2647599121,9cyclictest0-21swapper/911:01:2811
2647599120,8cyclictest0-21swapper/908:46:1711
2647599120,1cyclictest0-21swapper/911:15:0411
2647599120,1cyclictest0-21swapper/909:26:1511
2647599120,1cyclictest0-21swapper/909:22:5311
2647599120,1cyclictest0-21swapper/908:56:0211
2647599120,1cyclictest0-21swapper/908:30:3111
2647599120,1cyclictest0-21swapper/908:24:2011
2647599120,1cyclictest0-21swapper/908:19:0011
2647599120,1cyclictest0-21swapper/908:07:4111
2647599120,1cyclictest0-21swapper/907:58:1711
2647599120,1cyclictest0-21swapper/907:38:5511
2647599120,1cyclictest0-21swapper/907:12:0311
2647599120,12cyclictest0-21swapper/909:55:3411
2647599120,12cyclictest0-21swapper/909:51:1111
2647599120,12cyclictest0-21swapper/908:54:1311
2647599120,12cyclictest0-21swapper/907:41:2811
2647599120,11cyclictest0-21swapper/911:38:2711
2647599120,11cyclictest0-21swapper/911:27:3811
2647599120,11cyclictest0-21swapper/911:07:5911
2647599120,11cyclictest0-21swapper/910:33:5911
2647099129,3cyclictest0-21swapper/811:58:4410
2647099128,3cyclictest0-21swapper/810:57:3010
2647099122,8cyclictest0-21swapper/811:18:0910
2647099120,3cyclictest0-21swapper/812:23:1810
2647099120,11cyclictest0-21swapper/812:01:5310
2647099120,11cyclictest0-21swapper/810:51:4310
2647099120,11cyclictest0-21swapper/809:25:1610
2646399124,4cyclictest0-21swapper/710:05:019
2646399122,1cyclictest81rcu_preempt09:35:129
2646399121,4cyclictest0-21swapper/711:30:249
2646399121,4cyclictest0-21swapper/709:55:179
2646399121,4cyclictest0-21swapper/709:00:179
2646399121,4cyclictest0-21swapper/708:30:059
2646399121,4cyclictest0-21swapper/708:25:189
2646399121,4cyclictest0-21swapper/708:14:049
2646399121,4cyclictest0-21swapper/708:05:289
2646399121,4cyclictest0-21swapper/708:03:529
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional