You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-24 - 07:09

x86 Intel Core i7-3632QM @2200 MHz, Linux 4.19.1-rt3 (Profile)

Latency plot of system in rack #6, slot #3
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack6slot3.osadl.org (updated Fri Mar 24, 2023 00:46:41)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1885626935,0sleep50-21swapper/519:06:385
1885226135,0sleep30-21swapper/319:06:363
1882625437,0sleep70-21swapper/719:06:167
1878625438,0sleep40-21swapper/419:05:424
1905425336,0sleep00-21swapper/019:09:300
1879225336,0sleep10-21swapper/119:05:471
1708125336,0sleep60-21swapper/619:05:106
1899125134,0sleep20-21swapper/219:08:352
391350,0ktimersoftd/315861-21sh23:25:083
711330,0ktimersoftd/70-21swapper/722:58:007
631330,0ktimersoftd/60-21swapper/620:51:526
631330,0ktimersoftd/60-21swapper/620:51:526
551330,0ktimersoftd/50-21swapper/521:51:325
471330,0ktimersoftd/40-21swapper/422:26:514
391330,0ktimersoftd/3801-21grep00:15:183
391330,0ktimersoftd/36868-21sh22:41:533
391330,0ktimersoftd/315984-21ssh22:50:443
391330,0ktimersoftd/30-21swapper/323:59:483
391330,0ktimersoftd/30-21swapper/321:19:313
391330,0ktimersoftd/30-21swapper/319:20:223
231330,0ktimersoftd/10-21swapper/122:26:511
711320,0ktimersoftd/70-21swapper/723:28:107
711320,0ktimersoftd/70-21swapper/723:17:117
711320,0ktimersoftd/70-21swapper/722:54:387
711320,0ktimersoftd/70-21swapper/720:51:537
711320,0ktimersoftd/70-21swapper/720:51:537
631320,0ktimersoftd/60-21swapper/622:48:416
551320,0ktimersoftd/50-21swapper/523:08:105
551320,0ktimersoftd/50-21swapper/522:48:415
551320,0ktimersoftd/50-21swapper/520:51:525
551320,0ktimersoftd/50-21swapper/520:51:525
551320,0ktimersoftd/50-21swapper/500:29:485
471320,0ktimersoftd/419451-21cyclictest21:44:124
471320,0ktimersoftd/40-21swapper/421:51:324
471320,0ktimersoftd/40-21swapper/421:28:264
391320,0ktimersoftd/37151-21ssh22:10:083
391320,0ktimersoftd/30-21swapper/323:51:423
391320,0ktimersoftd/30-21swapper/323:47:403
391320,0ktimersoftd/30-21swapper/323:40:403
391320,0ktimersoftd/30-21swapper/323:39:033
391320,0ktimersoftd/30-21swapper/323:34:353
391320,0ktimersoftd/30-21swapper/323:20:223
391320,0ktimersoftd/30-21swapper/323:18:443
391320,0ktimersoftd/30-21swapper/323:01:373
391320,0ktimersoftd/30-21swapper/322:45:383
391320,0ktimersoftd/30-21swapper/322:32:473
391320,0ktimersoftd/30-21swapper/322:26:183
391320,0ktimersoftd/30-21swapper/322:06:183
391320,0ktimersoftd/30-21swapper/322:03:303
391320,0ktimersoftd/30-21swapper/321:54:083
391320,0ktimersoftd/30-21swapper/321:46:213
391320,0ktimersoftd/30-21swapper/321:43:123
391320,0ktimersoftd/30-21swapper/321:33:003
391320,0ktimersoftd/30-21swapper/321:25:573
391320,0ktimersoftd/30-21swapper/321:12:213
391320,0ktimersoftd/30-21swapper/321:02:493
391320,0ktimersoftd/30-21swapper/320:55:023
391320,0ktimersoftd/30-21swapper/320:50:513
391320,0ktimersoftd/30-21swapper/320:50:513
391320,0ktimersoftd/30-21swapper/320:45:183
391320,0ktimersoftd/30-21swapper/320:36:263
391320,0ktimersoftd/30-21swapper/320:32:403
391320,0ktimersoftd/30-21swapper/320:24:133
391320,0ktimersoftd/30-21swapper/320:15:003
391320,0ktimersoftd/30-21swapper/320:06:153
391320,0ktimersoftd/30-21swapper/319:50:343
391320,0ktimersoftd/30-21swapper/319:37:563
391320,0ktimersoftd/30-21swapper/319:34:563
391320,0ktimersoftd/30-21swapper/319:25:373
391320,0ktimersoftd/30-21swapper/319:10:073
391320,0ktimersoftd/30-21swapper/300:35:403
391320,0ktimersoftd/30-21swapper/300:33:303
391320,0ktimersoftd/30-21swapper/300:25:283
391320,0ktimersoftd/30-21swapper/300:11:003
391320,0ktimersoftd/30-21swapper/300:03:173
231320,0ktimersoftd/10-21swapper/122:58:001
231320,0ktimersoftd/10-21swapper/121:56:301
231320,0ktimersoftd/10-21swapper/121:56:301
231320,0ktimersoftd/10-21swapper/121:44:121
231320,0ktimersoftd/10-21swapper/121:33:111
231320,0ktimersoftd/10-21swapper/121:28:261
231320,0ktimersoftd/10-21swapper/119:19:381
231320,0ktimersoftd/10-21swapper/100:34:501
231320,0ktimersoftd/10-21swapper/100:29:481
101320,0ktimersoftd/014185-21sh00:29:350
101320,0ktimersoftd/00-21swapper/022:58:010
101320,0ktimersoftd/00-21swapper/021:28:260
101320,0ktimersoftd/00-21swapper/000:34:500
551310,0ktimersoftd/50-21swapper/521:44:135
471310,0ktimersoftd/40-21swapper/423:08:104
391310,0ktimersoftd/37943-21sh00:22:163
391310,0ktimersoftd/30-21swapper/323:14:023
391310,0ktimersoftd/30-21swapper/300:06:213
19462993029,0cyclictest9-21ksoftirqd/021:14:380
19462993029,0cyclictest11-21rcuc/022:54:380
19462992928,0cyclictest9-21ksoftirqd/023:17:110
19462992928,0cyclictest11-21rcuc/021:35:150
19462992928,0cyclictest11-21rcuc/000:13:390
19462992928,0cyclictest11-21rcuc/000:09:180
19462992928,0cyclictest11-21rcuc/000:03:430
19462992928,0cyclictest101ktimersoftd/021:33:100
19462992928,0cyclictest0-21swapper/022:48:410
19462992928,0cyclictest0-21swapper/021:56:300
19462992928,0cyclictest0-21swapper/021:56:300
19462992928,0cyclictest0-21swapper/020:51:520
19462992928,0cyclictest0-21swapper/020:51:520
19462992828,0cyclictest11-21rcuc/019:39:570
19462992828,0cyclictest11-21rcuc/000:21:060
19462992828,0cyclictest0-21swapper/022:03:290
19462992827,0cyclictest11-21rcuc/023:36:380
19462992827,0cyclictest11-21rcuc/023:31:450
19462992827,0cyclictest11-21rcuc/023:00:450
19462992827,0cyclictest11-21rcuc/019:21:100
19462992827,0cyclictest11-21rcuc/000:16:310
19462992827,0cyclictest101ktimersoftd/021:51:280
19462992727,0cyclictest11-21rcuc/022:26:510
19462992727,0cyclictest0-21swapper/023:44:050
19462992727,0cyclictest0-21swapper/023:08:100
19462992727,0cyclictest0-21swapper/022:15:560
19462992727,0cyclictest0-21swapper/022:12:350
19462992726,0cyclictest11-21rcuc/023:28:100
19462992626,0cyclictest0-21swapper/021:44:120
19462992626,0cyclictest0-21swapper/019:19:380
19487992524,0cyclictest48-21ksoftirqd/423:54:354
19487992524,0cyclictest46-21rcuc/422:38:174
19487992524,0cyclictest46-21rcuc/400:10:144
19487992524,0cyclictest12-21rcu_preempt00:39:394
19462992525,0cyclictest0-21swapper/021:48:510
19504992423,0cyclictest72-21ksoftirqd/722:47:447
19487992424,0cyclictest0-21swapper/423:01:024
19487992423,0cyclictest48-21ksoftirqd/400:21:074
19487992423,0cyclictest0-21swapper/400:29:494
19504992323,0cyclictest0-21swapper/723:01:507
19487992323,0cyclictest0-21swapper/422:24:104
19487992322,0cyclictest48-21ksoftirqd/423:36:394
19487992322,0cyclictest48-21ksoftirqd/423:17:124
19487992322,0cyclictest46-21rcuc/422:47:344
19487992322,0cyclictest46-21rcuc/422:32:024
19487992322,0cyclictest46-21rcuc/422:03:294
19487992322,0cyclictest46-21rcuc/421:33:114
19487992322,0cyclictest46-21rcuc/421:15:494
19487992322,0cyclictest46-21rcuc/419:39:574
19487992322,0cyclictest46-21rcuc/400:16:324
19487992322,0cyclictest46-21rcuc/400:07:244
19487992322,0cyclictest46-21rcuc/400:03:424
19487992322,0cyclictest0-21swapper/423:31:464
19504992222,0cyclictest711ktimersoftd/722:18:197
19504992221,0cyclictest711ktimersoftd/723:30:167
19504992221,0cyclictest70-21rcuc/721:28:267
19504992221,0cyclictest70-21rcuc/700:21:077
19504992221,0cyclictest0-21swapper/722:24:107
19487992222,0cyclictest46-21rcuc/422:58:004
19487992222,0cyclictest0-21swapper/423:44:054
19487992221,0cyclictest48-21ksoftirqd/420:12:334
19487992221,0cyclictest48-21ksoftirqd/400:34:504
19487992221,0cyclictest46-21rcuc/421:24:084
19504992121,0cyclictest711ktimersoftd/723:40:407
19504992121,0cyclictest711ktimersoftd/721:30:407
19504992121,0cyclictest711ktimersoftd/700:33:317
19504992121,0cyclictest711ktimersoftd/700:29:127
19504992120,0cyclictest711ktimersoftd/723:53:397
19504992120,0cyclictest711ktimersoftd/722:05:207
19504992120,0cyclictest711ktimersoftd/719:19:477
19504992120,0cyclictest70-21rcuc/722:44:567
19504992120,0cyclictest70-21rcuc/700:16:327
19504992120,0cyclictest70-21rcuc/700:09:187
19504992120,0cyclictest24526-21ssh22:26:517
19487992121,0cyclictest0-21swapper/422:12:354
19487992121,0cyclictest0-21swapper/419:19:384
19504992020,0cyclictest711ktimersoftd/723:49:197
19504992020,0cyclictest711ktimersoftd/723:21:367
19504992020,0cyclictest711ktimersoftd/723:10:197
19504992020,0cyclictest711ktimersoftd/723:05:077
19504992020,0cyclictest711ktimersoftd/722:11:237
19504992020,0cyclictest711ktimersoftd/722:03:357
19504992020,0cyclictest711ktimersoftd/721:43:397
19504992020,0cyclictest711ktimersoftd/721:36:447
19504992020,0cyclictest711ktimersoftd/721:11:357
19504992020,0cyclictest711ktimersoftd/721:02:037
19504992020,0cyclictest711ktimersoftd/720:36:037
19504992020,0cyclictest711ktimersoftd/720:25:397
19504992020,0cyclictest711ktimersoftd/720:21:197
19504992020,0cyclictest711ktimersoftd/720:16:077
19504992020,0cyclictest711ktimersoftd/720:03:087
19504992020,0cyclictest711ktimersoftd/719:56:127
19504992020,0cyclictest711ktimersoftd/719:48:237
19504992020,0cyclictest711ktimersoftd/719:44:557
19504992020,0cyclictest711ktimersoftd/700:37:517
19504992020,0cyclictest0-21swapper/721:56:307
19504992020,0cyclictest0-21swapper/721:56:307
19504992019,0cyclictest70-21rcuc/723:36:397
19504992019,0cyclictest70-21rcuc/722:38:117
19504992019,0cyclictest70-21rcuc/722:32:027
19504992019,0cyclictest70-21rcuc/721:51:287
19504992019,0cyclictest70-21rcuc/721:15:497
19504992019,0cyclictest70-21rcuc/719:21:107
19504992019,0cyclictest70-21rcuc/700:03:357
19504991919,0cyclictest711ktimersoftd/723:56:157
19504991919,0cyclictest711ktimersoftd/721:47:077
19504991919,0cyclictest711ktimersoftd/721:09:517
19504991919,0cyclictest711ktimersoftd/720:59:287
19504991919,0cyclictest711ktimersoftd/720:49:557
19504991919,0cyclictest711ktimersoftd/720:44:437
19504991919,0cyclictest711ktimersoftd/720:32:357
19504991919,0cyclictest711ktimersoftd/720:10:037
19504991919,0cyclictest711ktimersoftd/720:07:277
19504991919,0cyclictest711ktimersoftd/719:54:277
19504991919,0cyclictest711ktimersoftd/719:37:597
19504991919,0cyclictest711ktimersoftd/719:12:527
19504991918,0cyclictest711ktimersoftd/700:14:277
19504991818,0cyclictest711ktimersoftd/719:31:037
19504991818,0cyclictest711ktimersoftd/719:26:437
19470991818,0cyclictest24-21ksoftirqd/122:47:441
19470991818,0cyclictest0-21swapper/123:01:501
19470991716,0cyclictest24-21ksoftirqd/100:10:141
19470991716,0cyclictest22-21rcuc/122:54:381
19470991716,0cyclictest22-21rcuc/100:21:071
19470991616,0cyclictest22-21rcuc/123:25:021
19470991615,0cyclictest24-21ksoftirqd/122:15:551
19470991615,0cyclictest22-21rcuc/123:36:381
19470991615,0cyclictest22-21rcuc/123:14:071
19470991615,0cyclictest22-21rcuc/122:40:451
19470991615,0cyclictest22-21rcuc/121:35:151
19470991615,0cyclictest22-21rcuc/119:39:571
19470991615,0cyclictest22-21rcuc/100:16:321
19470991515,0cyclictest24-21ksoftirqd/122:03:301
19470991515,0cyclictest22-21rcuc/100:03:421
19470991515,0cyclictest0-21swapper/123:17:121
19470991515,0cyclictest0-21swapper/120:51:531
19470991515,0cyclictest0-21swapper/120:51:521
19470991514,0cyclictest24-21ksoftirqd/120:28:561
19470991514,0cyclictest22-21rcuc/123:31:461
19470991514,0cyclictest22-21rcuc/122:11:421
19470991514,0cyclictest22-21rcuc/121:51:281
19470991514,0cyclictest22-21rcuc/121:15:491
19470991514,0cyclictest22-21rcuc/119:21:101
19470991514,0cyclictest22-21rcuc/100:35:201
19470991514,0cyclictest22-21rcuc/100:09:011
19470991414,0cyclictest0-21swapper/123:44:051
19470991413,0cyclictest24-21ksoftirqd/120:12:331
19470991413,0cyclictest0-21swapper/123:08:111
19470991313,0cyclictest0-21swapper/121:48:511
63180,0ktimersoftd/60-21swapper/623:16:356
63180,0ktimersoftd/60-21swapper/600:06:056
55180,0ktimersoftd/50-21swapper/500:06:045
31180,0ktimersoftd/20-21swapper/219:47:552
199985080,0irq/32-enp2s0f00-21swapper/122:21:271
199955080,0irq/30-enp2s0f00-21swapper/523:00:225
199955080,0irq/30-enp2s0f00-21swapper/522:35:475
194809988,0cyclictest0-21swapper/321:22:083
63170,0ktimersoftd/60-21swapper/622:01:396
63170,0ktimersoftd/60-21swapper/621:52:386
63170,0ktimersoftd/60-21swapper/621:32:236
63170,0ktimersoftd/60-21swapper/620:20:146
63170,0ktimersoftd/60-21swapper/619:47:556
55170,0ktimersoftd/50-21swapper/523:35:205
55170,0ktimersoftd/50-21swapper/522:01:385
55170,0ktimersoftd/50-21swapper/521:38:085
55170,0ktimersoftd/50-21swapper/520:20:145
55170,0ktimersoftd/50-21swapper/519:13:205
47170,0ktimersoftd/40-21swapper/423:58:264
47170,0ktimersoftd/40-21swapper/421:57:484
47170,0ktimersoftd/40-21swapper/421:57:484
47170,0ktimersoftd/40-21swapper/419:47:554
47170,0ktimersoftd/40-21swapper/419:13:214
39170,0ktimersoftd/30-21swapper/319:47:553
31170,0ktimersoftd/20-21swapper/223:58:262
31170,0ktimersoftd/20-21swapper/223:35:212
31170,0ktimersoftd/20-21swapper/222:26:092
31170,0ktimersoftd/20-21swapper/222:00:382
31170,0ktimersoftd/20-21swapper/221:38:082
199975070,0irq/31-enp2s0f00-21swapper/422:19:374
199975070,0irq/31-enp2s0f00-21swapper/421:12:004
199955070,0irq/30-enp2s0f04063-21timerwakeupswit22:05:265
199955070,0irq/30-enp2s0f00-21swapper/523:56:455
199955070,0irq/30-enp2s0f00-21swapper/523:49:095
199955070,0irq/30-enp2s0f00-21swapper/523:30:535
199955070,0irq/30-enp2s0f00-21swapper/523:25:515
199955070,0irq/30-enp2s0f00-21swapper/523:21:045
199955070,0irq/30-enp2s0f00-21swapper/523:16:465
199955070,0irq/30-enp2s0f00-21swapper/523:10:165
199955070,0irq/30-enp2s0f00-21swapper/522:57:165
199955070,0irq/30-enp2s0f00-21swapper/522:50:065
199955070,0irq/30-enp2s0f00-21swapper/522:40:365
199955070,0irq/30-enp2s0f00-21swapper/522:31:305
199955070,0irq/30-enp2s0f00-21swapper/522:27:045
199955070,0irq/30-enp2s0f00-21swapper/522:23:155
199955070,0irq/30-enp2s0f00-21swapper/522:15:515
199955070,0irq/30-enp2s0f00-21swapper/521:56:085
199955070,0irq/30-enp2s0f00-21swapper/521:56:085
199955070,0irq/30-enp2s0f00-21swapper/521:30:455
199955070,0irq/30-enp2s0f00-21swapper/521:25:575
199955070,0irq/30-enp2s0f00-21swapper/521:20:485
199955070,0irq/30-enp2s0f00-21swapper/521:16:315
199955070,0irq/30-enp2s0f00-21swapper/521:11:585
199955070,0irq/30-enp2s0f00-21swapper/500:35:555
199955070,0irq/30-enp2s0f00-21swapper/500:30:255
199955070,0irq/30-enp2s0f00-21swapper/500:16:005
199955070,0irq/30-enp2s0f00-21swapper/500:14:145
199955070,0irq/30-enp2s0f00-21swapper/500:01:365
199935070,0irq/28-enp2s0f08155-21ssh21:37:303
199935070,0irq/28-enp2s0f018876-21ssh22:20:353
199935070,0irq/28-enp2s0f00-21swapper/322:55:383
199935070,0irq/28-enp2s0f00-21swapper/322:36:183
194979976,0cyclictest0-21swapper/622:25:136
194979970,0cyclictest0-21swapper/600:33:406
194919970,0cyclictest0-21swapper/500:20:035
194879976,0cyclictest0-21swapper/423:29:474
194879970,0cyclictest0-21swapper/422:50:004
194749970,0cyclictest0-21swapper/223:20:182
194709975,0cyclictest24264-21smartctl21:20:221
194709970,0cyclictest0-21swapper/123:46:321
194709970,0cyclictest0-21swapper/122:05:541
169970,0migration/06585-21taskset22:41:240
16079270,0sleep612526-21diskmemload21:12:396
1355070,0irq/24-ahci[00020362-21latency_hist23:29:592
10170,0ktimersoftd/00-21swapper/020:46:340
71160,0ktimersoftd/70-21swapper/721:20:107
63160,0ktimersoftd/60-21swapper/620:05:016
63160,0ktimersoftd/60-21swapper/600:28:146
609960,0migration/616339-21taskset22:51:206
529960,0migration/530572-21taskset23:40:055
31160,0ktimersoftd/20-21swapper/222:35:572
31160,0ktimersoftd/20-21swapper/219:54:002
31160,0ktimersoftd/20-21swapper/200:26:112
23160,0ktimersoftd/10-21swapper/120:46:351
209960,0migration/15630-21latency_hist20:55:001
199985060,0irq/32-enp2s0f00-21swapper/123:55:141
199985060,0irq/32-enp2s0f00-21swapper/123:52:071
199985060,0irq/32-enp2s0f00-21swapper/122:36:021
199985060,0irq/32-enp2s0f00-21swapper/122:33:251
199975060,0irq/31-enp2s0f00-21swapper/423:45:114
199975060,0irq/31-enp2s0f00-21swapper/422:40:354
199975060,0irq/31-enp2s0f00-21swapper/421:45:334
199975060,0irq/31-enp2s0f00-21swapper/421:37:294
199955060,0irq/30-enp2s0f00-21swapper/523:51:085
199955060,0irq/30-enp2s0f00-21swapper/521:45:355
199955060,0irq/30-enp2s0f00-21swapper/520:26:275
199955060,0irq/30-enp2s0f00-21swapper/519:35:555
199935060,0irq/28-enp2s0f00-21swapper/323:05:123
199935060,0irq/28-enp2s0f00-21swapper/322:15:463
199935060,0irq/28-enp2s0f00-21swapper/321:55:013
199935060,0irq/28-enp2s0f00-21swapper/321:55:013
199935060,0irq/28-enp2s0f00-21swapper/321:05:133
199935060,0irq/28-enp2s0f00-21swapper/320:10:183
194979966,0cyclictest0-21swapper/623:23:536
194979966,0cyclictest0-21swapper/622:30:016
194979966,0cyclictest0-21swapper/622:15:376
194979966,0cyclictest0-21swapper/622:05:376
194979966,0cyclictest0-21swapper/600:20:016
194979965,0cyclictest0-21swapper/621:02:396
194979964,0cyclictest9648-21ls21:40:006
194979964,0cyclictest8175-21tr23:50:146
194979964,0cyclictest494-21iwlist19:35:186
194979964,0cyclictest30884-21sh21:27:466
194979964,0cyclictest21435-21sh00:03:266
194979964,0cyclictest12526-21diskmemload23:55:156
194979964,0cyclictest12453-21sh21:41:266
194979960,0cyclictest0-21swapper/621:59:496
194979960,0cyclictest0-21swapper/621:59:496
194979960,0cyclictest0-21swapper/600:39:086
194919964,0cyclictest1999550irq/30-enp2s0f022:10:195
194919964,0cyclictest1374-21sh20:45:135
194919960,0cyclictest0-21swapper/520:07:225
194809966,0cyclictest0-21swapper/320:00:003
194809960,0cyclictest0-21swapper/320:40:223
194809960,0cyclictest0-21swapper/320:29:403
194749964,0cyclictest9281-21expr22:45:122
194749964,0cyclictest878-21sh00:15:182
194749964,0cyclictest7736-21idleruntime-cro21:00:002
194749964,0cyclictest27966-21cut22:30:202
194749964,0cyclictest20702-21if_err_wlp3s000:35:262
194749964,0cyclictest10708-21fgrep19:55:262
194749963,0cyclictest30869-21cstates23:40:182
194749960,0cyclictest12526-21diskmemload23:45:402
194749960,0cyclictest12526-21diskmemload22:23:052
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional