You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-08-09 - 00:16

x86 Intel Core i7-2600K @3400 MHz, Linux 5.9.0-rc2-rt1 (Profile)

Latency plot of system in rack #4, slot #6
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h200 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot6.osadl.org (updated Mon Aug 08, 2022 12:44:34)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
890521480,0sleep40-21swapper/410:26:284
21773521090,105pltrace0-21swapper/407:05:344
2177352980,87pltrace0-21swapper/007:07:090
2409327761,12sleep20-21swapper/207:08:312
2402327761,12sleep60-21swapper/607:07:316
2404727357,12sleep30-21swapper/307:07:523
2396327146,12sleep50-21swapper/507:06:415
129462710,0sleep40-21swapper/409:59:524
105252690,0pltrace0-21swapper/110:20:021
1891652670,0pltrace0-21swapper/411:10:034
2418326649,12sleep70-21swapper/707:09:487
2407226145,12sleep10-21swapper/107:08:141
234282600,0sleep00-21swapper/009:36:160
2822752470,1pltrace28231-21pltrace-run09:10:013
2437899230,22cyclictest0-21swapper/409:35:284
24378992221,1cyclictest0-21swapper/411:45:254
2437899221,20cyclictest0-21swapper/408:40:254
2437899221,20cyclictest0-21swapper/408:40:254
2432852210,7pltrace2436499cyclictest12:20:022
285462200,2sleep225-21ksoftirqd/210:15:122
1616252200,1pltrace16166-21pltrace-run10:35:012
1942152190,2pltrace49-21ksoftirqd/612:18:296
43202180,3sleep02435199cyclictest08:25:180
386852180,1pltrace37-21ksoftirqd/410:56:524
1666652180,1pltrace37-21ksoftirqd/411:44:264
1114752180,1pltrace49-21ksoftirqd/610:32:466
939552170,9pltrace0-21swapper/212:06:492
664052170,2pltrace37-21ksoftirqd/411:31:184
386852170,1pltrace49-21ksoftirqd/610:58:226
16612170,0sleep4361rcuc/410:20:194
155852170,0sleep10-21swapper/108:45:221
149822170,2sleep52438499cyclictest10:00:255
1341352170,1pltrace13417-21pltrace-run10:00:013
1114752170,1pltrace25-21ksoftirqd/210:33:222
715852160,1pltrace9631-21cat07:40:010
664052160,1pltrace25-21ksoftirqd/211:31:532
222172160,0sleep10-21swapper/110:40:191
1942152160,10pltrace22857-21ssh12:17:442
1839452160,1pltrace37-21ksoftirqd/407:55:204
1666652160,6pltrace0-21swapper/211:44:172
1573752160,2pltrace25-21ksoftirqd/209:32:492
939552150,1pltrace14374-21cat12:10:000
76152150,8pltrace0-21swapper/509:15:135
664052150,1pltrace49-21ksoftirqd/611:32:116
434652150,0pltrace0-21swapper/207:31:272
3170652150,1pltrace25-21ksoftirqd/211:55:382
3170652150,1pltrace0-21swapper/311:55:203
2967052150,1pltrace37-21ksoftirqd/408:18:524
1942152150,1pltrace0-21swapper/012:19:320
1573752150,0pltrace0-21swapper/509:30:195
1573752150,0pltrace0-21swapper/409:30:274
1114752150,11pltrace0-21swapper/010:35:010
1072852150,9pltrace0-21swapper/509:28:005
852152140,0pltrace0-21swapper/008:37:400
852152140,0pltrace0-21swapper/008:37:400
76152140,5pltrace0-21swapper/409:15:214
76152140,5pltrace0-21swapper/209:15:472
715852140,0pltrace0-21swapper/207:37:582
675452140,5pltrace0-21swapper/212:38:322
571152140,5pltrace0-21swapper/308:33:193
571152140,2pltrace111rcu_preempt08:33:364
386852140,1pltrace0-21swapper/210:57:012
3170652140,2pltrace489-21irqstats_TLB11:55:217
3170652140,0pltrace0-21swapper/511:56:035
3170652140,0pltrace0-21swapper/411:56:044
2923852140,5pltrace0-21swapper/512:28:385
2923852140,2pltrace111rcu_preempt12:29:202
2923852140,1pltrace858-21ssh12:29:034
2822752140,0pltrace0-21swapper/509:14:555
2576252140,0pltrace0-21swapper/009:41:540
2465652140,1pltrace0-21swapper/207:10:342
2438499141,2cyclictest30365-21ssh10:49:045
24378991412,1cyclictest0-21swapper/407:10:344
2435199141,11cyclictest20824-21crond08:00:000
2394152140,5pltrace0-21swapper/211:16:372
2257152140,1pltrace24200-21smart_sda09:00:246
2120852140,0pltrace0-21swapper/408:00:214
1942152140,1pltrace37-21ksoftirqd/412:17:184
1839452140,0pltrace0-21swapper/207:55:192
1616252140,0pltrace0-21swapper/610:36:156
1573752140,1pltrace18253-21sh09:30:567
1114752140,0pltrace14674-21ssh10:32:544
1114752140,0pltrace0-21swapper/510:32:275
1072852140,0pltrace0-21swapper/209:28:472
939552130,2pltrace111rcu_preempt12:06:496
939552130,0pltrace0-21swapper/412:05:484
675452130,4pltrace0-21swapper/512:38:155
675452130,1pltrace10877-21sh12:39:066
675452130,0pltrace0-21swapper/412:38:164
664052130,6pltrace0-21swapper/511:31:015
434652130,5pltrace0-21swapper/507:30:575
3124552130,0pltrace0-21swapper/010:52:050
3111852130,10pltrace0-21swapper/007:20:420
2669252130,0pltrace0-21swapper/211:50:082
2621352130,10pltrace0-21swapper/010:49:040
24384991311,1cyclictest0-21swapper/510:10:225
2437899130,12cyclictest0-21swapper/407:25:324
2437099131,11cyclictest0-21swapper/310:35:303
24364991311,1cyclictest0-21swapper/208:30:202
24364991310,2cyclictest0-21swapper/212:10:222
2435199131,11cyclictest0-21swapper/011:51:190
2435199130,11cyclictest0-21swapper/011:40:180
2403752130,5pltrace0-21swapper/408:10:014
2403752130,4pltrace0-21swapper/408:10:014
2394152130,5pltrace0-21swapper/511:16:285
2394152130,0pltrace0-21swapper/411:16:554
1844552130,7pltrace0-21swapper/010:06:140
1839452130,4pltrace0-21swapper/607:55:206
1666652130,4pltrace0-21swapper/511:44:435
1666652130,0pltrace0-21swapper/711:44:007
1666652130,0pltrace0-21swapper/611:44:516
1616252130,0pltrace0-21swapper/410:35:474
1558652130,0pltrace0-21swapper/007:51:320
1413852130,5pltrace0-21swapper/208:46:082
1114752130,0pltrace0-21swapper/710:32:457
1072852130,5pltrace0-21swapper/709:28:097
1072852130,0pltrace0-21swapper/409:28:184
105252130,1pltrace1056-21tail10:20:015
2967052120,0pltrace0-21swapper/208:18:102
2923852120,0pltrace0-21swapper/612:29:206
290052120,0pltrace0-21swapper/408:26:164
290052120,0pltrace0-21swapper/308:25:503
2465652120,0pltrace0-21swapper/607:10:176
24396991210,1cyclictest0-21swapper/712:11:337
24396991210,1cyclictest0-21swapper/709:18:287
24396991210,1cyclictest0-21swapper/708:30:137
24396991210,1cyclictest0-21swapper/707:35:207
2439699121,10cyclictest0-21swapper/709:59:267
2439699120,11cyclictest0-21swapper/711:48:247
2439699120,11cyclictest0-21swapper/711:24:167
2439699120,11cyclictest0-21swapper/710:29:527
2438499120,11cyclictest0-21swapper/511:38:565
2438499120,11cyclictest0-21swapper/510:54:155
2438499120,11cyclictest0-21swapper/510:44:125
2438499120,11cyclictest0-21swapper/510:08:165
2438499120,11cyclictest0-21swapper/509:56:365
2438499120,1cyclictest0-21swapper/512:23:375
2438499120,1cyclictest0-21swapper/512:02:445
2438499120,1cyclictest0-21swapper/510:28:205
2438499120,1cyclictest0-21swapper/509:35:435
24378991210,1cyclictest0-21swapper/411:09:114
24378991210,1cyclictest0-21swapper/407:40:014
2437899120,11cyclictest0-21swapper/411:03:514
2437899120,11cyclictest0-21swapper/410:14:184
2437899120,11cyclictest0-21swapper/410:03:184
2437899120,1cyclictest0-21swapper/411:37:514
2437899120,1cyclictest0-21swapper/411:24:444
2437899120,1cyclictest0-21swapper/407:15:244
2437099127,3cyclictest765-21tail09:15:013
24364991210,1cyclictest0-21swapper/210:44:042
24364991210,1cyclictest0-21swapper/210:12:112
24364991210,1cyclictest0-21swapper/210:05:532
24364991210,1cyclictest0-21swapper/209:59:232
24364991210,1cyclictest0-21swapper/209:40:172
24364991210,1cyclictest0-21swapper/209:13:242
24364991210,1cyclictest0-21swapper/208:40:232
24364991210,1cyclictest0-21swapper/208:40:232
2436499120,11cyclictest0-21swapper/210:49:052
2436499120,11cyclictest0-21swapper/210:21:152
2436499120,1cyclictest0-21swapper/208:50:252
2436499120,1cyclictest0-21swapper/207:25:152
2435899120,11cyclictest0-21swapper/109:25:161
24351991210,1cyclictest0-21swapper/011:35:260
24351991210,1cyclictest0-21swapper/010:59:200
2435199121,2cyclictest0-21swapper/007:45:010
2435199121,10cyclictest0-21swapper/009:18:050
2435199121,10cyclictest0-21swapper/009:10:010
2435199121,1cyclictest1403-21turbostat.cron12:34:550
2435199121,1cyclictest0-21swapper/011:55:060
2435199120,3cyclictest14422-21cp12:10:020
2435199120,11cyclictest20132-21sh09:34:330
2435199120,11cyclictest0-21swapper/010:17:240
2435199120,11cyclictest0-21swapper/010:13:010
2435199120,11cyclictest0-21swapper/009:20:440
2435199120,10cyclictest6476-21chrt07:30:470
2394152120,4pltrace0-21swapper/611:16:196
2257152120,5pltrace0-21swapper/409:01:184
2166752120,7pltrace0-21swapper/011:46:310
1573752120,4pltrace0-21swapper/309:32:493
1573752120,0pltrace0-21swapper/609:32:026
1072852120,0pltrace0-21swapper/609:27:436
996952110,0pltrace0-21swapper/607:44:046
76152110,0pltrace0-21swapper/609:16:246
715852110,0pltrace0-21swapper/407:37:434
675452110,4pltrace0-21swapper/712:39:157
386852110,0pltrace0-21swapper/710:58:047
3111852110,0pltrace0-21swapper/407:22:524
290052110,4pltrace0-21swapper/708:25:597
2822752110,0pltrace0-21swapper/409:14:554
2439699119,1cyclictest0-21swapper/710:50:167
2439699119,1cyclictest0-21swapper/709:46:387
2439699119,1cyclictest0-21swapper/708:15:207
2439699110,10cyclictest0-21swapper/711:04:507
2439699110,10cyclictest0-21swapper/710:45:257
2439699110,10cyclictest0-21swapper/710:36:517
2439699110,10cyclictest0-21swapper/709:38:237
2439699110,1cyclictest0-21swapper/711:34:097
2438499119,1cyclictest0-21swapper/511:25:595
2438499119,1cyclictest0-21swapper/510:59:275
2438499119,1cyclictest0-21swapper/510:35:385
2438499119,1cyclictest0-21swapper/509:46:545
2438499119,1cyclictest0-21swapper/509:04:095
2438499119,1cyclictest0-21swapper/508:55:445
2438499119,1cyclictest0-21swapper/507:17:475
2438499110,10cyclictest0-21swapper/509:09:205
2438499110,10cyclictest0-21swapper/508:50:235
2438499110,10cyclictest0-21swapper/507:48:425
2437899119,1cyclictest0-21swapper/409:40:294
2437899110,10cyclictest0-21swapper/410:43:274
2437899110,10cyclictest0-21swapper/409:23:284
2437899110,1cyclictest0-21swapper/412:24:194
2437899110,1cyclictest0-21swapper/412:02:184
2437099119,1cyclictest0-21swapper/309:45:283
2436499119,1cyclictest0-21swapper/211:26:592
2436499119,1cyclictest0-21swapper/211:10:082
2436499119,1cyclictest0-21swapper/211:04:482
2436499119,1cyclictest0-21swapper/209:52:252
2436499110,10cyclictest0-21swapper/210:03:162
2436499110,10cyclictest0-21swapper/209:22:282
2435899119,1cyclictest0-21swapper/109:39:211
2435199119,1cyclictest9-21ksoftirqd/009:47:560
2435199119,1cyclictest0-21swapper/010:22:150
2435199119,1cyclictest0-21swapper/007:10:330
2435199111,9cyclictest8149-21munin-run08:35:000
2435199111,9cyclictest0-21swapper/012:25:120
2435199111,9cyclictest0-21swapper/008:15:160
2435199111,1cyclictest0-21swapper/010:25:070
2435199110,2cyclictest25346-21cat09:05:020
2435199110,2cyclictest24535-21sh10:42:310
2435199110,2cyclictest0-21swapper/009:30:010
2435199110,10cyclictest0-21swapper/012:37:120
2435199110,10cyclictest0-21swapper/011:34:240
2435199110,10cyclictest0-21swapper/011:24:060
2435199110,10cyclictest0-21swapper/010:36:010
2435199110,10cyclictest0-21swapper/009:57:320
2435199110,10cyclictest0-21swapper/009:52:470
2435199110,10cyclictest0-21swapper/007:15:320
2435199110,1cyclictest0-21swapper/011:09:220
2435199110,1cyclictest0-21swapper/011:01:250
2435199110,1cyclictest0-21swapper/010:01:330
2435199110,1cyclictest0-21swapper/009:10:370
2394152110,5pltrace0-21swapper/011:15:310
2257152110,4pltrace0-21swapper/209:01:092
1413852110,0pltrace0-21swapper/408:45:504
2685952100,0pltrace0-21swapper/208:10:012
2439699108,1cyclictest0-21swapper/711:14:537
2439699108,1cyclictest0-21swapper/710:40:537
2439699108,1cyclictest0-21swapper/710:20:297
2439699108,1cyclictest0-21swapper/710:07:417
2439699108,1cyclictest0-21swapper/709:52:487
2439699108,1cyclictest0-21swapper/707:45:437
2439699100,9cyclictest0-21swapper/710:13:507
2439699100,9cyclictest0-21swapper/708:40:067
2439699100,9cyclictest0-21swapper/708:40:067
2439199108,1cyclictest31438-21ssh11:20:506
2439199108,1cyclictest0-21swapper/611:05:176
2439199108,1cyclictest0-21swapper/609:56:196
2439199100,9cyclictest0-21swapper/611:28:076
2438499108,1cyclictest0-21swapper/511:04:375
2438499108,1cyclictest0-21swapper/508:13:485
2438499103,4cyclictest1442652pltrace12:10:185
2438499100,1cyclictest0-21swapper/508:21:245
2438499100,1cyclictest0-21swapper/507:37:035
2437899108,1cyclictest0-21swapper/410:16:344
2437899108,1cyclictest0-21swapper/410:05:534
2437899100,9cyclictest0-21swapper/408:23:344
2437099108,1cyclictest0-21swapper/312:20:543
2437099108,1cyclictest0-21swapper/312:08:263
2437099108,1cyclictest0-21swapper/311:05:073
2437099108,1cyclictest0-21swapper/310:29:013
2437099108,1cyclictest0-21swapper/310:06:023
2437099108,1cyclictest0-21swapper/309:54:033
2437099108,1cyclictest0-21swapper/308:59:103
2437099108,1cyclictest0-21swapper/308:51:143
2437099108,1cyclictest0-21swapper/308:45:533
2437099100,9cyclictest0-21swapper/308:21:363
2437099100,1cyclictest0-21swapper/311:50:373
2437099100,1cyclictest0-21swapper/311:39:463
2437099100,1cyclictest0-21swapper/311:14:113
2437099100,1cyclictest0-21swapper/310:22:423
2437099100,1cyclictest0-21swapper/309:27:493
2436499108,1cyclictest0-21swapper/209:46:072
2436499108,1cyclictest0-21swapper/207:42:252
2436499103,4cyclictest2762952pltrace07:15:012
2436499100,9cyclictest0-21swapper/207:22:592
2435899108,1cyclictest0-21swapper/111:41:461
2435899108,1cyclictest0-21swapper/111:06:371
2435899108,1cyclictest0-21swapper/110:27:261
2435899108,1cyclictest0-21swapper/109:30:071
2435899108,1cyclictest0-21swapper/107:10:241
2435899101,7cyclictest12798-21ntp_states08:40:221
2435899101,7cyclictest12798-21ntp_states08:40:221
2435899101,7cyclictest0-21swapper/112:25:371
2435899101,7cyclictest0-21swapper/111:58:261
2435899101,2cyclictest0-21swapper/112:05:001
2435899101,1cyclictest0-21swapper/107:47:091
2435899100,9cyclictest0-21swapper/111:31:131
2435899100,9cyclictest0-21swapper/111:23:371
2435899100,9cyclictest0-21swapper/109:55:521
2435899100,9cyclictest0-21swapper/107:43:061
2435199101,8cyclictest0-21swapper/008:45:320
2435199101,1cyclictest0-21swapper/008:00:280
2435199100,9cyclictest101rcuc/008:50:010
2435199100,9cyclictest0-21swapper/012:24:350
2435199100,9cyclictest0-21swapper/011:26:070
2435199100,9cyclictest0-21swapper/008:20:460
2435199100,9cyclictest0-21swapper/007:45:330
2435199100,2cyclictest987-17auditd07:25:000
2435199100,2cyclictest25423-21ls08:05:240
2435199100,2cyclictest11962-21fschecks_count08:40:170
2435199100,2cyclictest11962-21fschecks_count08:40:170
2435199100,1cyclictest29299-21munin-run08:15:000
2435199100,1cyclictest0-21swapper/012:01:450
2403752100,4pltrace0-21swapper/208:09:362
2120852100,1pltrace22601-21strings08:00:226
312455290,1pltrace3380-21ssh10:54:484
292385290,0pltrace0-21swapper/712:29:047
243969997,1cyclictest0-21swapper/710:18:137
243969997,1cyclictest0-21swapper/709:11:307
243969997,1cyclictest0-21swapper/708:23:457
243969994,4cyclictest1141-21runrttasks09:21:567
243969990,8cyclictest0-21swapper/708:35:447
243969990,8cyclictest0-21swapper/708:35:447
243969990,8cyclictest0-21swapper/707:57:217
243969990,1cyclictest0-21swapper/708:01:057
243969990,1cyclictest0-21swapper/707:24:107
243969990,1cyclictest0-21swapper/707:19:287
243919997,1cyclictest0-21swapper/610:48:186
243919997,1cyclictest0-21swapper/609:47:346
243919997,1cyclictest0-21swapper/608:47:306
243919997,1cyclictest0-21swapper/608:42:486
243919997,1cyclictest0-21swapper/608:42:486
243919997,1cyclictest0-21swapper/608:25:396
243919990,8cyclictest4397-21sh12:00:106
243919990,8cyclictest0-21swapper/612:23:396
243919990,8cyclictest0-21swapper/611:00:556
243919990,8cyclictest0-21swapper/610:11:036
243919990,8cyclictest0-21swapper/609:35:066
243919990,8cyclictest0-21swapper/608:56:346
243919990,8cyclictest0-21swapper/607:47:466
243919990,8cyclictest0-21swapper/607:37:246
243919990,8cyclictest0-21swapper/607:16:316
243919990,1cyclictest0-21swapper/612:12:486
243919990,1cyclictest0-21swapper/611:36:426
243919990,1cyclictest0-21swapper/610:19:086
243919990,1cyclictest0-21swapper/610:09:066
243919990,1cyclictest0-21swapper/609:53:146
243919990,1cyclictest0-21swapper/609:13:346
243849997,1cyclictest0-21swapper/507:54:125
243849997,1cyclictest0-21swapper/507:20:035
243849995,2cyclictest1141-21runrttasks10:18:315
243789997,1cyclictest0-21swapper/409:08:054
243789997,1cyclictest0-21swapper/408:59:214
243789997,1cyclictest0-21swapper/408:51:054
243789997,1cyclictest0-21swapper/408:35:134
243789997,1cyclictest0-21swapper/408:35:134
243789997,1cyclictest0-21swapper/407:51:504
243789997,1cyclictest0-21swapper/407:46:004
243709997,1cyclictest0-21swapper/311:29:353
243709997,1cyclictest0-21swapper/311:21:493
243709997,1cyclictest0-21swapper/310:47:193
243709997,1cyclictest0-21swapper/308:37:183
243709997,1cyclictest0-21swapper/308:37:183
243709997,1cyclictest0-21swapper/308:11:433
243709997,1cyclictest0-21swapper/307:18:373
243709994,3cyclictest987-17auditd07:59:553
243709991,5cyclictest2923852pltrace12:25:133
243709990,8cyclictest0-21swapper/307:43:043
243709990,8cyclictest0-21swapper/307:30:163
243709990,8cyclictest0-21swapper/307:10:413
243709990,1cyclictest0-21swapper/310:56:033
243709990,1cyclictest0-21swapper/310:42:563
243709990,1cyclictest0-21swapper/309:07:253
243709990,1cyclictest0-21swapper/309:03:033
243649997,1cyclictest0-21swapper/208:55:262
243649997,1cyclictest0-21swapper/208:35:312
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional