You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-08-16 - 06:05

x86 Intel Core i7-3632QM @2200 MHz, Linux 4.19.1-rt3 (Profile)

Latency plot of system in rack #6, slot #3
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rack6slot3.osadl.org (updated Tue Aug 16, 2022 00:46:46)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
624026044,0sleep30-21swapper/319:05:333
644025842,0sleep50-21swapper/519:08:235
649925741,0sleep60-21swapper/619:09:136
627625741,0sleep10-21swapper/119:06:021
642625539,0sleep00-21swapper/019:08:100
630625538,0sleep40-21swapper/419:06:294
645125336,0sleep70-21swapper/719:08:337
632525134,0sleep20-21swapper/219:06:442
391350,0ktimersoftd/30-21swapper/321:33:423
391350,0ktimersoftd/30-21swapper/319:21:483
391350,0ktimersoftd/30-21swapper/319:21:483
391330,0ktimersoftd/30-21swapper/322:40:503
391330,0ktimersoftd/30-21swapper/322:29:183
711320,0ktimersoftd/70-21swapper/719:31:517
391320,0ktimersoftd/30-21swapper/323:34:033
391320,0ktimersoftd/30-21swapper/323:11:183
391320,0ktimersoftd/30-21swapper/323:05:353
391320,0ktimersoftd/30-21swapper/323:00:193
391320,0ktimersoftd/30-21swapper/322:24:113
391320,0ktimersoftd/30-21swapper/322:09:343
391320,0ktimersoftd/30-21swapper/321:57:393
391320,0ktimersoftd/30-21swapper/321:28:593
391320,0ktimersoftd/30-21swapper/321:22:033
391320,0ktimersoftd/30-21swapper/321:18:523
391320,0ktimersoftd/30-21swapper/320:32:053
391320,0ktimersoftd/30-21swapper/320:21:483
391320,0ktimersoftd/30-21swapper/320:16:153
391320,0ktimersoftd/30-21swapper/320:10:353
391320,0ktimersoftd/30-21swapper/320:09:093
391320,0ktimersoftd/30-21swapper/319:56:213
391320,0ktimersoftd/30-21swapper/319:54:173
391320,0ktimersoftd/30-21swapper/319:49:373
391310,0ktimersoftd/30-21swapper/320:27:363
101310,0ktimersoftd/00-21swapper/019:31:500
6918992929,0cyclictest0-21swapper/322:15:223
6906992423,0cyclictest24-21ksoftirqd/122:44:141
6906992423,0cyclictest24-21ksoftirqd/122:15:291
6906992323,0cyclictest22-21rcuc/122:55:521
6906992322,0cyclictest24-21ksoftirqd/122:24:041
6906992322,0cyclictest22-21rcuc/122:14:211
6930992221,0cyclictest0-21swapper/521:17:485
6906992222,0cyclictest0-21swapper/121:17:481
6906992221,0cyclictest24-21ksoftirqd/121:49:071
6906992221,0cyclictest22-21rcuc/123:55:141
6906992221,0cyclictest22-21rcuc/123:03:561
6906992221,0cyclictest22-21rcuc/119:31:511
6906992221,0cyclictest12-21rcu_preempt20:25:521
6906992221,0cyclictest12-21rcu_preempt00:29:321
6930992120,0cyclictest54-21rcuc/521:13:065
6906992120,0cyclictest24-21ksoftirqd/121:28:031
6906992120,0cyclictest22-21rcuc/100:38:321
6906992120,0cyclictest12-21rcu_preempt23:54:031
6906992120,0cyclictest0-21swapper/120:09:141
6906992019,0cyclictest24-21ksoftirqd/122:26:181
6906992019,0cyclictest22-21rcuc/123:17:571
6930991919,0cyclictest12-21rcu_preempt23:49:575
6930991919,0cyclictest0-21swapper/523:03:555
6930991919,0cyclictest0-21swapper/521:49:065
6930991919,0cyclictest0-21swapper/519:31:515
6930991918,0cyclictest56-21ksoftirqd/521:28:045
6930991918,0cyclictest54-21rcuc/521:02:025
6930991818,0cyclictest54-21rcuc/500:38:325
6930991818,0cyclictest0-21swapper/522:55:445
6930991818,0cyclictest0-21swapper/522:29:185
6930991818,0cyclictest0-21swapper/520:09:135
6930991817,0cyclictest54-21rcuc/523:17:575
6930991817,0cyclictest12-21rcu_preempt22:17:265
6930991717,0cyclictest0-21swapper/519:18:535
13550100,0irq/24-ahci[0006568-21latencyplot19:10:002
71180,0ktimersoftd/70-21swapper/719:57:067
69389980,0cyclictest0-21swapper/622:25:136
689980,0migration/73002-21taskset23:41:327
63180,0ktimersoftd/60-21swapper/623:35:346
63180,0ktimersoftd/60-21swapper/619:57:076
47180,0ktimersoftd/40-21swapper/423:35:344
39180,0ktimersoftd/30-21swapper/323:35:343
31180,0ktimersoftd/20-21swapper/219:25:152
23180,0ktimersoftd/10-21swapper/123:35:341
23180,0ktimersoftd/10-21swapper/119:57:061
23180,0ktimersoftd/10-21swapper/119:44:121
209980,0migration/122506-21cpuspeed_turbos00:05:191
199985080,0irq/32-enp2s0f017004-21ntp_states23:20:185
199955080,0irq/30-enp2s0f00-21swapper/122:51:411
199935080,0irq/28-enp2s0f06158-1kworker/u17:121:57:214
199935080,0irq/28-enp2s0f023188-21kworker/u16:223:32:454
199935080,0irq/28-enp2s0f00-21swapper/422:50:114
199935080,0irq/28-enp2s0f00-21swapper/422:38:114
199935080,0irq/28-enp2s0f00-21swapper/421:47:494
10180,0ktimersoftd/00-21swapper/023:35:340
10180,0ktimersoftd/00-21swapper/019:57:070
71170,0ktimersoftd/70-21swapper/723:47:177
71170,0ktimersoftd/70-21swapper/723:24:107
71170,0ktimersoftd/70-21swapper/722:03:327
71170,0ktimersoftd/70-21swapper/720:26:427
71170,0ktimersoftd/70-21swapper/719:44:137
71170,0ktimersoftd/70-21swapper/719:25:157
69449976,0cyclictest0-21swapper/721:36:497
69449976,0cyclictest0-21swapper/721:29:307
69449970,0cyclictest0-21swapper/723:26:197
69449970,0cyclictest0-21swapper/723:26:187
69449970,0cyclictest0-21swapper/722:06:027
69449970,0cyclictest0-21swapper/721:40:297
69449970,0cyclictest0-21swapper/721:22:107
69389977,0cyclictest0-21swapper/622:09:256
69389976,0cyclictest0-21swapper/622:20:166
69389976,0cyclictest0-21swapper/621:10:526
69389970,0cyclictest0-21swapper/621:15:326
69389970,0cyclictest0-21swapper/600:18:456
69309976,0cyclictest3521-21grep00:20:185
69309976,0cyclictest3521-21grep00:20:175
69309976,0cyclictest0-21swapper/523:59:265
69309976,0cyclictest0-21swapper/522:43:475
69309970,0cyclictest0-21swapper/523:11:395
69239970,0cyclictest0-21swapper/423:20:014
69239970,0cyclictest0-21swapper/400:35:454
69189976,0cyclictest0-21swapper/321:10:353
69189976,0cyclictest0-21swapper/300:34:493
69189970,0cyclictest0-21swapper/323:54:533
69189970,0cyclictest0-21swapper/323:44:083
69189970,0cyclictest0-21swapper/323:20:223
69119976,0cyclictest0-21swapper/222:45:222
69119970,0cyclictest9176-21ssh23:10:522
69119970,0cyclictest0-21swapper/223:50:042
69119970,0cyclictest0-21swapper/221:45:002
69069976,0cyclictest0-21swapper/123:11:111
69069976,0cyclictest0-21swapper/121:22:231
689970,0migration/78731-21switchtime23:10:317
689970,0migration/77362-21cpuspeed_turbos00:25:147
689970,0migration/719839-21cpuspeed_turbos19:35:127
63170,0ktimersoftd/60-21swapper/622:03:316
63170,0ktimersoftd/60-21swapper/621:54:456
63170,0ktimersoftd/60-21swapper/620:26:416
55170,0ktimersoftd/50-21swapper/522:03:325
55170,0ktimersoftd/50-21swapper/521:44:245
55170,0ktimersoftd/50-21swapper/520:19:255
55170,0ktimersoftd/50-21swapper/519:44:135
47170,0ktimersoftd/40-21swapper/422:03:324
47170,0ktimersoftd/40-21swapper/419:25:154
47170,0ktimersoftd/40-21swapper/400:10:514
39170,0ktimersoftd/30-21swapper/323:47:163
39170,0ktimersoftd/30-21swapper/322:45:203
39170,0ktimersoftd/30-21swapper/321:44:243
31170,0ktimersoftd/20-21swapper/223:29:122
31170,0ktimersoftd/20-21swapper/223:29:122
31170,0ktimersoftd/20-21swapper/222:05:372
31170,0ktimersoftd/20-21swapper/221:44:232
31170,0ktimersoftd/20-21swapper/221:12:552
31170,0ktimersoftd/20-21swapper/219:44:132
23170,0ktimersoftd/10-21swapper/119:20:371
23170,0ktimersoftd/10-21swapper/119:20:371
199985070,0irq/32-enp2s0f00-21swapper/521:50:185
199985070,0irq/32-enp2s0f00-21swapper/521:38:445
199975070,0irq/31-enp2s0f00-21swapper/622:11:126
199975070,0irq/31-enp2s0f00-21swapper/621:48:366
199955070,0irq/30-enp2s0f09805-21fw_forwarded_lo22:35:151
199955070,0irq/30-enp2s0f00-21swapper/123:45:011
199955070,0irq/30-enp2s0f00-21swapper/123:25:301
199955070,0irq/30-enp2s0f00-21swapper/123:25:291
199955070,0irq/30-enp2s0f00-21swapper/123:20:451
199955070,0irq/30-enp2s0f00-21swapper/122:34:211
199955070,0irq/30-enp2s0f00-21swapper/122:08:271
199955070,0irq/30-enp2s0f00-21swapper/122:03:021
199955070,0irq/30-enp2s0f00-21swapper/121:55:201
199955070,0irq/30-enp2s0f00-21swapper/121:50:221
199955070,0irq/30-enp2s0f00-21swapper/121:42:361
199955070,0irq/30-enp2s0f00-21swapper/121:36:221
199955070,0irq/30-enp2s0f00-21swapper/121:10:591
199955070,0irq/30-enp2s0f00-21swapper/100:32:111
199955070,0irq/30-enp2s0f00-21swapper/100:14:131
199945070,0irq/29-enp2s0f00-21swapper/721:48:417
199945070,0irq/29-enp2s0f00-21swapper/721:19:547
199945070,0irq/29-enp2s0f00-21swapper/700:14:167
199935070,0irq/28-enp2s0f032376-21diskmemload23:48:294
199935070,0irq/28-enp2s0f03226-21df21:50:194
199935070,0irq/28-enp2s0f029882-21/usr/sbin/munin22:20:194
199935070,0irq/28-enp2s0f010991-1kworker/u17:122:40:234
199935070,0irq/28-enp2s0f01009-21ssh23:02:024
199935070,0irq/28-enp2s0f00-21swapper/423:57:114
199935070,0irq/28-enp2s0f00-21swapper/423:52:274
199935070,0irq/28-enp2s0f00-21swapper/423:40:054
199935070,0irq/28-enp2s0f00-21swapper/423:25:484
199935070,0irq/28-enp2s0f00-21swapper/423:25:474
199935070,0irq/28-enp2s0f00-21swapper/423:15:564
199935070,0irq/28-enp2s0f00-21swapper/423:12:104
199935070,0irq/28-enp2s0f00-21swapper/422:57:374
199935070,0irq/28-enp2s0f00-21swapper/422:46:214
199935070,0irq/28-enp2s0f00-21swapper/422:19:094
199935070,0irq/28-enp2s0f00-21swapper/422:14:384
199935070,0irq/28-enp2s0f00-21swapper/422:05:274
199935070,0irq/28-enp2s0f00-21swapper/421:40:044
199935070,0irq/28-enp2s0f00-21swapper/421:20:194
199935070,0irq/28-enp2s0f00-21swapper/421:15:234
199935070,0irq/28-enp2s0f00-21swapper/400:25:194
199935070,0irq/28-enp2s0f00-21swapper/400:22:454
199935070,0irq/28-enp2s0f00-21swapper/400:22:454
199935070,0irq/28-enp2s0f00-21swapper/400:06:204
199935070,0irq/28-enp2s0f00-21swapper/400:01:034
199935070,0irq/28-enp2s0f00-21swapper/400:01:024
169970,0migration/024405-21taskset22:15:130
169970,0migration/022654-21chrt20:48:060
169970,0migration/018435-21df00:00:140
169970,0migration/018435-21df00:00:140
1355070,0irq/24-ahci[0006312-21smart_sda22:30:222
1355070,0irq/24-ahci[0002389-21latency_hist23:05:012
10170,0ktimersoftd/00-21swapper/019:14:120
69449966,0cyclictest0-21swapper/722:20:127
69449966,0cyclictest0-21swapper/700:39:167
69449964,0cyclictest9648-21uname20:20:177
69449964,0cyclictest1703-21grep21:10:237
69449964,0cyclictest15873-21sh23:56:147
69449960,0cyclictest0-21swapper/723:35:157
69449960,0cyclictest0-21swapper/723:31:217
69449960,0cyclictest0-21swapper/722:56:217
69449960,0cyclictest0-21swapper/722:41:007
69449960,0cyclictest0-21swapper/722:33:597
69449960,0cyclictest0-21swapper/721:32:317
69449960,0cyclictest0-21swapper/721:05:587
69449960,0cyclictest0-21swapper/720:56:537
69449960,0cyclictest0-21swapper/720:51:027
69449960,0cyclictest0-21swapper/720:36:437
69449960,0cyclictest0-21swapper/720:33:177
69449960,0cyclictest0-21swapper/720:06:337
69449960,0cyclictest0-21swapper/719:53:127
69449960,0cyclictest0-21swapper/719:18:547
69449960,0cyclictest0-21swapper/719:12:337
69449960,0cyclictest0-21swapper/700:30:347
69449960,0cyclictest0-21swapper/700:22:327
69449960,0cyclictest0-21swapper/700:22:327
69449960,0cyclictest0-21swapper/700:16:247
69449960,0cyclictest0-21swapper/700:07:037
69389966,0cyclictest0-21swapper/622:57:506
69389966,0cyclictest0-21swapper/622:50:156
69389966,0cyclictest0-21swapper/622:42:406
69389966,0cyclictest0-21swapper/621:56:306
69389966,0cyclictest0-21swapper/620:38:536
69389966,0cyclictest0-21swapper/600:22:416
69389966,0cyclictest0-21swapper/600:22:416
69389965,0cyclictest2163-21rm23:04:326
69389960,0cyclictest32084-21sshd21:06:036
69389960,0cyclictest0-21swapper/623:54:096
69309966,0cyclictest0-21swapper/523:43:305
69309966,0cyclictest0-21swapper/521:34:255
69309966,0cyclictest0-21swapper/520:46:525
69309966,0cyclictest0-21swapper/500:09:105
69309965,0cyclictest0-21swapper/521:21:085
69309960,0cyclictest0-21swapper/522:50:095
69309960,0cyclictest0-21swapper/500:26:225
69309960,0cyclictest0-21swapper/500:02:215
69309960,0cyclictest0-21swapper/500:02:215
69239964,0cyclictest23701-21grep20:50:124
69239964,0cyclictest13580-21cstates21:25:134
69189966,0cyclictest0-21swapper/323:18:053
69189966,0cyclictest0-21swapper/321:54:073
69189965,0cyclictest10792-21ssh00:29:183
69189960,0cyclictest0-21swapper/322:52:403
69189960,0cyclictest0-21swapper/322:01:053
69119966,0cyclictest0-21swapper/222:25:212
69119965,0cyclictest0-21swapper/222:50:072
69119965,0cyclictest0-21swapper/221:20:142
69119964,0cyclictest8973-21tr20:20:122
69119964,0cyclictest25615-21ntpq19:45:202
69119964,0cyclictest24527-21cpu23:30:152
69119960,0cyclictest0-21swapper/222:44:342
69069966,0cyclictest0-21swapper/121:30:161
69019966,0cyclictest0-21swapper/000:17:200
69019964,0cyclictest9005-21sh00:25:250
69019964,0cyclictest3457-21fschecks_time00:20:170
69019964,0cyclictest3457-21fschecks_time00:20:160
69019964,0cyclictest32592-21missed_timers20:00:180
69019962,0cyclictest2411-21perf23:05:000
69019960,0cyclictest0-21swapper/023:45:050
69019960,0cyclictest0-21swapper/023:43:000
69019960,0cyclictest0-21swapper/023:25:380
69019960,0cyclictest0-21swapper/023:25:380
69019960,0cyclictest0-21swapper/023:10:410
69019960,0cyclictest0-21swapper/022:57:310
69019960,0cyclictest0-21swapper/022:51:210
69019960,0cyclictest0-21swapper/022:45:060
69019960,0cyclictest0-21swapper/022:39:410
69019960,0cyclictest0-21swapper/022:29:510
69019960,0cyclictest0-21swapper/022:05:270
69019960,0cyclictest0-21swapper/021:20:110
69019960,0cyclictest0-21swapper/021:02:490
69019960,0cyclictest0-21swapper/020:56:070
69019960,0cyclictest0-21swapper/020:54:270
69019960,0cyclictest0-21swapper/020:41:090
69019960,0cyclictest0-21swapper/020:35:110
69019960,0cyclictest0-21swapper/020:31:190
69019960,0cyclictest0-21swapper/020:20:100
69019960,0cyclictest0-21swapper/020:15:280
69019960,0cyclictest0-21swapper/019:38:030
69019960,0cyclictest0-21swapper/019:29:040
69019960,0cyclictest0-21swapper/019:20:350
69019960,0cyclictest0-21swapper/019:20:340
69019960,0cyclictest0-21swapper/019:18:270
69019960,0cyclictest0-21swapper/000:34:160
69019960,0cyclictest0-21swapper/000:07:100
63160,0ktimersoftd/60-21swapper/623:42:316
63160,0ktimersoftd/60-21swapper/621:23:146
63160,0ktimersoftd/60-21swapper/620:56:546
63160,0ktimersoftd/60-21swapper/619:20:376
63160,0ktimersoftd/60-21swapper/619:20:376
609960,0migration/626838-21cpuspeed_turbos00:10:186
55160,0ktimersoftd/50-21swapper/523:05:495
55160,0ktimersoftd/50-21swapper/522:48:095
55160,0ktimersoftd/50-21swapper/522:30:425
55160,0ktimersoftd/50-21swapper/519:14:125
47160,0ktimersoftd/40-21swapper/422:30:434
47160,0ktimersoftd/40-21swapper/419:57:064
47160,0ktimersoftd/40-21swapper/419:14:114
449960,0migration/431369-21cstates00:15:144
39160,0ktimersoftd/30-21swapper/323:28:503
39160,0ktimersoftd/30-21swapper/323:28:503
39160,0ktimersoftd/30-21swapper/320:47:123
369960,0migration/39617-21gltestperf19:15:013
31160,0ktimersoftd/2820-21snmpd00:21:072
31160,0ktimersoftd/2820-21snmpd00:21:072
31160,0ktimersoftd/20-21swapper/222:17:102
31160,0ktimersoftd/20-21swapper/221:54:512
31160,0ktimersoftd/20-21swapper/220:47:122
31160,0ktimersoftd/20-21swapper/200:07:102
289960,0migration/213237-21latency_hist21:25:012
199985060,0irq/32-enp2s0f00-21swapper/522:20:195
199985060,0irq/32-enp2s0f00-21swapper/521:55:495
199975060,0irq/31-enp2s0f026345-21ssh21:40:016
199975060,0irq/31-enp2s0f00-21swapper/623:27:586
199975060,0irq/31-enp2s0f00-21swapper/623:27:586
199975060,0irq/31-enp2s0f00-21swapper/623:17:276
199975060,0irq/31-enp2s0f00-21swapper/623:07:006
199975060,0irq/31-enp2s0f00-21swapper/622:45:206
199975060,0irq/31-enp2s0f00-21swapper/622:36:206
199975060,0irq/31-enp2s0f00-21swapper/622:32:076
199975060,0irq/31-enp2s0f00-21swapper/621:30:186
199975060,0irq/31-enp2s0f00-21swapper/619:35:226
199975060,0irq/31-enp2s0f00-21swapper/600:39:046
199955060,0irq/30-enp2s0f00-21swapper/123:40:191
199955060,0irq/30-enp2s0f00-21swapper/123:30:021
199955060,0irq/30-enp2s0f00-21swapper/123:05:121
199955060,0irq/30-enp2s0f00-21swapper/122:45:351
199955060,0irq/30-enp2s0f00-21swapper/100:23:021
199955060,0irq/30-enp2s0f00-21swapper/100:23:021
199955060,0irq/30-enp2s0f00-21swapper/100:19:321
199955060,0irq/30-enp2s0f00-21swapper/100:00:351
199955060,0irq/30-enp2s0f00-21swapper/100:00:351
199945060,0irq/29-enp2s0f07520-21cut21:55:197
199945060,0irq/29-enp2s0f023163-21ssh22:13:187
199945060,0irq/29-enp2s0f00-21swapper/723:19:497
199945060,0irq/29-enp2s0f00-21swapper/723:09:297
199945060,0irq/29-enp2s0f00-21swapper/723:02:587
199945060,0irq/29-enp2s0f00-21swapper/722:50:257
199945060,0irq/29-enp2s0f00-21swapper/722:45:137
199945060,0irq/29-enp2s0f00-21swapper/722:38:357
199945060,0irq/29-enp2s0f00-21swapper/720:01:317
199945060,0irq/29-enp2s0f00-21swapper/700:02:157
199945060,0irq/29-enp2s0f00-21swapper/700:02:157
199935060,0irq/28-enp2s0f0553-21/usr/sbin/munin22:25:124
199935060,0irq/28-enp2s0f00-21swapper/423:06:474
199935060,0irq/28-enp2s0f00-21swapper/421:36:094
199935060,0irq/28-enp2s0f00-21swapper/421:30:424
199935060,0irq/28-enp2s0f00-21swapper/421:10:164
199935060,0irq/28-enp2s0f00-21swapper/421:00:184
199935060,0irq/28-enp2s0f00-21swapper/420:02:534
199935060,0irq/28-enp2s0f00-21swapper/400:30:124
169960,0migration/07851-21ssh21:18:210
169960,0migration/032376-21diskmemload21:25:380
169960,0migration/015963-21cpuspeed_turbos00:35:150
169960,0migration/011654-21diskstats20:25:130
169960,0migration/00-21swapper/023:05:580
169960,0migration/00-21swapper/019:51:300
1355060,0irq/24-ahci[00015399-21idleruntime-cro00:35:002
1355060,0irq/24-ahci[0000-21swapper/223:50:002
1355060,0irq/24-ahci[0000-21swapper/221:35:262
1355060,0irq/24-ahci[0000-21swapper/221:00:202
1355060,0irq/24-ahci[0000-21swapper/220:35:242
1355060,0irq/24-ahci[0000-21swapper/220:15:002
1355060,0irq/24-ahci[0000-21swapper/219:25:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional