You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-04-01 - 18:39

x86 Intel Atom E3845 @1910 MHz, Linux 4.19.37-rt19 (Profile)

Latency plot of system in rack #d, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Up99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackdslot2.osadl.org (updated Sat Apr 01, 2023 12:46:18)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2523224430,10sleep341-21ksoftirqd/307:05:183
109792760,9sleep02709299cyclictest11:20:000
18612730,8sleep22709499cyclictest08:35:382
27094997157,7cyclictest33-21ksoftirqd/207:45:182
247192701,17sleep00-21swapper/007:05:070
27094996829,8cyclictest33-21ksoftirqd/209:15:332
27094996829,7cyclictest33-21ksoftirqd/208:15:232
55498660,8rtkit-daemon553-21rtkit-daemon07:06:072
27094996631,5cyclictest33-21ksoftirqd/207:25:322
27094996629,5cyclictest33-21ksoftirqd/211:25:382
27094996628,10cyclictest33-21ksoftirqd/208:40:252
25512660,3sleep30-21swapper/311:00:273
52642650,5sleep125-21ksoftirqd/111:05:361
27094996530,5cyclictest33-21ksoftirqd/211:45:012
27094996529,5cyclictest33-21ksoftirqd/211:30:272
27094996529,5cyclictest33-21ksoftirqd/211:30:272
123742650,3sleep20-21swapper/209:00:182
27094996432,10cyclictest33-21ksoftirqd/207:40:292
27094996429,5cyclictest33-21ksoftirqd/210:55:002
27094996429,4cyclictest33-21ksoftirqd/211:35:242
27094996428,5cyclictest33-21ksoftirqd/208:00:312
27094996331,4cyclictest33-21ksoftirqd/209:40:382
27094996328,5cyclictest33-21ksoftirqd/209:50:192
27094996328,5cyclictest33-21ksoftirqd/208:45:382
27094996328,5cyclictest33-21ksoftirqd/208:05:412
27094996324,7cyclictest33-21ksoftirqd/210:00:332
27094996251,6cyclictest33-21ksoftirqd/210:45:202
27094996230,5cyclictest33-21ksoftirqd/209:30:242
27094996229,5cyclictest33-21ksoftirqd/212:24:592
27094996228,4cyclictest33-21ksoftirqd/207:10:212
27094996227,5cyclictest33-21ksoftirqd/207:50:192
27094996131,4cyclictest33-21ksoftirqd/211:00:232
27094996131,4cyclictest33-21ksoftirqd/207:35:022
27094996131,4cyclictest33-21ksoftirqd/207:35:012
27094996124,8cyclictest33-21ksoftirqd/212:25:222
27094996029,4cyclictest33-21ksoftirqd/212:00:272
27094996028,6cyclictest33-21ksoftirqd/211:05:212
27094996028,10cyclictest33-21ksoftirqd/211:50:222
27094996028,10cyclictest33-21ksoftirqd/210:55:272
27094995929,4cyclictest33-21ksoftirqd/212:15:012
27094995926,5cyclictest33-21ksoftirqd/211:55:332
27094995921,10cyclictest33-21ksoftirqd/210:10:272
27094995828,4cyclictest33-21ksoftirqd/208:20:172
27094995828,4cyclictest33-21ksoftirqd/207:15:282
133991580,25ptp4l29062-21/usr/sbin/munin08:25:313
2709499577,10cyclictest33-21ksoftirqd/212:15:182
27094995725,9cyclictest33-21ksoftirqd/210:25:322
27094995725,11cyclictest33-21ksoftirqd/211:10:302
27094995724,5cyclictest33-21ksoftirqd/207:30:342
27094995724,5cyclictest33-21ksoftirqd/207:30:332
27094995723,5cyclictest33-21ksoftirqd/210:20:212
27094995723,5cyclictest33-21ksoftirqd/209:45:202
27094995722,6cyclictest33-21ksoftirqd/210:40:332
27094995626,4cyclictest33-21ksoftirqd/212:06:042
27094995626,4cyclictest33-21ksoftirqd/209:05:372
193642560,2sleep30-21swapper/309:15:173
2709499556,9cyclictest33-21ksoftirqd/211:45:402
27094995526,4cyclictest33-21ksoftirqd/208:29:012
27094995524,6cyclictest33-21ksoftirqd/209:59:592
27094995520,5cyclictest33-21ksoftirqd/209:25:272
2709499547,4cyclictest24572-21latency_hist10:40:002
2648625433,10sleep10-21swapper/107:05:541
2709499536,3cyclictest33-21ksoftirqd/211:15:352
2709499532,10cyclictest33-21ksoftirqd/210:15:342
2709499527,8cyclictest33-21ksoftirqd/210:05:472
2709499524,8cyclictest121rcu_preempt09:35:372
27094995244,4cyclictest33-21ksoftirqd/209:10:052
2709499518,4cyclictest33-21ksoftirqd/208:30:332
2709499514,8cyclictest33-21ksoftirqd/212:30:252
2709499513,9cyclictest33-21ksoftirqd/209:00:002
2709499513,9cyclictest33-21ksoftirqd/209:00:002
2709499510,12cyclictest33-21ksoftirqd/210:30:282
2709499504,7cyclictest121rcu_preempt11:20:322
27094995042,5cyclictest33-21ksoftirqd/212:35:042
27094995039,3cyclictest33-21ksoftirqd/208:15:002
2709499495,9cyclictest33-21ksoftirqd/208:50:202
2709499493,8cyclictest33-21ksoftirqd/207:55:302
2709599484,8cyclictest0-21swapper/310:30:213
2709499484,3cyclictest33-21ksoftirqd/207:20:272
2709599463,9cyclictest0-21swapper/309:00:163
2709599463,8cyclictest0-21swapper/309:30:153
27095994624,19cyclictest41-21ksoftirqd/312:19:133
133991450,1ptp4l401ktimersoftd/309:40:363
55498440,15rtkit-daemon16431-21apache207:16:093
27095994421,20cyclictest41-21ksoftirqd/312:37:593
133991440,1ptp4l401ktimersoftd/311:30:253
133991440,1ptp4l401ktimersoftd/311:30:253
133991440,1ptp4l401ktimersoftd/311:29:123
133991440,1ptp4l401ktimersoftd/308:30:323
133991430,1ptp4l401ktimersoftd/309:25:243
27092994222,4cyclictest9-21ksoftirqd/008:23:070
133991420,1ptp4l401ktimersoftd/311:35:393
133991420,1ptp4l401ktimersoftd/307:54:333
27095994113,6cyclictest32410-21tr12:05:233
27092994121,4cyclictest9-21ksoftirqd/010:35:480
133991410,1ptp4l401ktimersoftd/307:55:283
27095994017,3cyclictest41-21ksoftirqd/310:09:293
27094993934,3cyclictest33-21ksoftirqd/209:25:012
151292390,1sleep00-21swapper/011:25:380
133991390,13ptp4l0-21swapper/309:55:013
133991380,1ptp4l401ktimersoftd/307:27:053
133991380,10ptp4l0-21swapper/308:45:573
27095993731,4cyclictest41-21ksoftirqd/312:25:213
27093993730,4cyclictest25-21ksoftirqd/107:20:581
27093993720,14cyclictest25-21ksoftirqd/108:15:341
133991370,13ptp4l401ktimersoftd/312:04:323
133991370,12ptp4l401ktimersoftd/307:40:273
27093993630,4cyclictest25-21ksoftirqd/109:54:351
227462360,3sleep3391rcuc/308:10:003
133991360,1ptp4l401ktimersoftd/310:01:583
133991360,1ptp4l401ktimersoftd/308:35:323
133991360,13ptp4l0-21swapper/307:49:173
27095993529,4cyclictest41-21ksoftirqd/311:40:203
27095993528,4cyclictest41-21ksoftirqd/312:22:453
27093993529,3cyclictest25-21ksoftirqd/108:49:351
27093993527,4cyclictest25-21ksoftirqd/110:00:101
2709299353,3cyclictest121rcu_preempt08:45:350
27092993528,4cyclictest9-21ksoftirqd/011:57:500
27092993528,4cyclictest9-21ksoftirqd/011:12:570
27092993528,4cyclictest9-21ksoftirqd/008:42:260
27092993528,4cyclictest9-21ksoftirqd/008:26:520
27092993528,4cyclictest9-21ksoftirqd/007:53:200
27092993528,4cyclictest9-21ksoftirqd/007:23:340
133991350,1ptp4l401ktimersoftd/307:10:333
27095993429,3cyclictest41-21ksoftirqd/308:52:143
27093993428,4cyclictest25-21ksoftirqd/110:24:471
27093993428,3cyclictest25-21ksoftirqd/110:42:331
27093993427,4cyclictest25-21ksoftirqd/112:30:511
27093993427,4cyclictest25-21ksoftirqd/111:58:521
27093993427,4cyclictest25-21ksoftirqd/111:53:361
27093993427,4cyclictest25-21ksoftirqd/110:56:351
27093993427,4cyclictest25-21ksoftirqd/110:12:021
27093993427,4cyclictest25-21ksoftirqd/109:00:251
27093993426,5cyclictest25-21ksoftirqd/107:26:031
27093993426,4cyclictest25-21ksoftirqd/110:28:581
2709399341,31cyclictest0-21swapper/108:25:331
27092993427,4cyclictest9-21ksoftirqd/012:11:140
27092993427,4cyclictest9-21ksoftirqd/010:49:090
27092993427,4cyclictest9-21ksoftirqd/008:59:400
27092993427,4cyclictest9-21ksoftirqd/008:59:400
27092993426,5cyclictest9-21ksoftirqd/009:40:520
27092993426,5cyclictest9-21ksoftirqd/009:29:390
133991340,12ptp4l0-21swapper/310:11:273
27095993327,4cyclictest41-21ksoftirqd/307:24:323
27095993327,3cyclictest41-21ksoftirqd/309:47:003
27095993326,4cyclictest41-21ksoftirqd/312:11:593
27095993326,4cyclictest41-21ksoftirqd/310:25:553
27095993326,4cyclictest41-21ksoftirqd/309:51:103
27093993327,4cyclictest25-21ksoftirqd/110:15:241
27093993327,3cyclictest25-21ksoftirqd/110:54:131
27093993326,4cyclictest25-21ksoftirqd/112:38:471
27093993326,4cyclictest25-21ksoftirqd/111:13:461
27093993326,4cyclictest25-21ksoftirqd/110:09:361
27093993326,4cyclictest25-21ksoftirqd/108:59:081
27093993326,4cyclictest25-21ksoftirqd/108:59:071
27093993326,4cyclictest25-21ksoftirqd/107:44:011
27093993326,4cyclictest25-21ksoftirqd/107:16:321
27093993325,5cyclictest25-21ksoftirqd/111:19:211
27093993325,5cyclictest25-21ksoftirqd/109:59:071
27093993324,6cyclictest25-21ksoftirqd/108:06:091
27093993324,5cyclictest25-21ksoftirqd/109:49:031
27092993327,4cyclictest9-21ksoftirqd/012:25:160
27092993327,3cyclictest9-21ksoftirqd/011:03:320
27092993326,5cyclictest9-21ksoftirqd/011:54:200
27092993326,4cyclictest9-21ksoftirqd/010:12:350
27092993326,4cyclictest9-21ksoftirqd/007:32:280
27092993326,4cyclictest9-21ksoftirqd/007:32:280
27092993326,4cyclictest9-21ksoftirqd/007:17:140
27092993325,5cyclictest9-21ksoftirqd/007:28:140
1338913326,3phc2sys41-21ksoftirqd/309:35:213
27095993226,4cyclictest41-21ksoftirqd/311:10:343
27095993226,3cyclictest41-21ksoftirqd/309:13:533
27095993226,3cyclictest41-21ksoftirqd/307:37:543
27095993226,3cyclictest41-21ksoftirqd/307:37:543
27095993225,4cyclictest41-21ksoftirqd/310:37:593
27095993225,4cyclictest41-21ksoftirqd/307:32:573
27095993225,4cyclictest41-21ksoftirqd/307:32:563
27095993224,5cyclictest41-21ksoftirqd/310:55:533
27093993226,4cyclictest25-21ksoftirqd/109:15:101
27093993225,4cyclictest25-21ksoftirqd/112:28:201
27093993225,4cyclictest25-21ksoftirqd/110:46:591
27093993224,5cyclictest25-21ksoftirqd/112:07:051
27093993224,5cyclictest25-21ksoftirqd/108:54:281
27092993226,4cyclictest9-21ksoftirqd/012:17:130
27092993226,3cyclictest9-21ksoftirqd/009:11:500
27092993226,3cyclictest9-21ksoftirqd/007:47:350
27092993225,4cyclictest9-21ksoftirqd/012:24:270
27092993225,4cyclictest9-21ksoftirqd/011:44:530
27092993225,4cyclictest9-21ksoftirqd/010:56:350
27092993225,4cyclictest9-21ksoftirqd/010:05:160
27092993225,4cyclictest9-21ksoftirqd/009:06:470
133991320,1ptp4l401ktimersoftd/311:05:553
27095993124,4cyclictest41-21ksoftirqd/312:33:103
27095993124,4cyclictest41-21ksoftirqd/311:45:453
27095993124,4cyclictest41-21ksoftirqd/308:04:233
27095993123,5cyclictest41-21ksoftirqd/308:12:013
27095993123,5cyclictest16240-21sed10:20:293
27093993125,4cyclictest25-21ksoftirqd/112:24:541
27093993125,4cyclictest25-21ksoftirqd/107:32:091
27093993125,4cyclictest25-21ksoftirqd/107:32:091
27093993124,4cyclictest25-21ksoftirqd/111:40:571
27093993123,5cyclictest25-21ksoftirqd/111:20:081
27092993126,3cyclictest9-21ksoftirqd/010:41:370
27092993125,3cyclictest9-21ksoftirqd/009:31:560
27092993125,3cyclictest9-21ksoftirqd/008:08:260
27092993124,4cyclictest9-21ksoftirqd/011:21:040
27092993124,4cyclictest9-21ksoftirqd/008:36:230
27092993124,4cyclictest9-21ksoftirqd/007:59:340
27092993123,5cyclictest9-21ksoftirqd/010:31:070
27095993024,3cyclictest41-21ksoftirqd/311:24:083
27093993024,3cyclictest25-21ksoftirqd/107:53:251
27093993023,4cyclictest25-21ksoftirqd/111:49:001
27093993023,4cyclictest25-21ksoftirqd/110:39:301
27093993023,4cyclictest25-21ksoftirqd/109:34:591
27093993022,5cyclictest25-21ksoftirqd/111:34:141
27093993022,5cyclictest25-21ksoftirqd/111:34:131
27093993022,5cyclictest25-21ksoftirqd/111:02:421
27092993024,4cyclictest9-21ksoftirqd/012:05:450
27092993024,4cyclictest9-21ksoftirqd/009:01:120
27092993023,4cyclictest9-21ksoftirqd/009:53:080
27092993023,4cyclictest9-21ksoftirqd/008:51:530
27092993022,5cyclictest9-21ksoftirqd/008:15:110
27092993017,10cyclictest27161-21cat07:10:150
133991300,1ptp4l401ktimersoftd/309:21:053
133991300,1ptp4l401ktimersoftd/308:40:213
27095992924,3cyclictest41-21ksoftirqd/309:06:423
27095992923,4cyclictest41-21ksoftirqd/310:45:353
27095992922,4cyclictest41-21ksoftirqd/311:56:563
27095992922,4cyclictest41-21ksoftirqd/311:52:563
27093992923,4cyclictest25-21ksoftirqd/108:39:421
27093992923,3cyclictest25-21ksoftirqd/107:13:241
27093992922,4cyclictest25-21ksoftirqd/109:27:081
27093992922,4cyclictest25-21ksoftirqd/108:33:411
27093992922,4cyclictest25-21ksoftirqd/107:58:351
27093992921,5cyclictest25-21ksoftirqd/112:03:171
27092992922,4cyclictest9-21ksoftirqd/011:35:580
27092992922,4cyclictest9-21ksoftirqd/010:50:170
27092992922,4cyclictest9-21ksoftirqd/009:17:110
27095992822,3cyclictest41-21ksoftirqd/310:44:033
27093992822,4cyclictest25-21ksoftirqd/111:39:061
27093992822,3cyclictest25-21ksoftirqd/112:12:291
27093992821,4cyclictest25-21ksoftirqd/112:15:151
27093992821,4cyclictest25-21ksoftirqd/111:26:521
27093992821,4cyclictest25-21ksoftirqd/110:30:181
27093992820,5cyclictest25-21ksoftirqd/107:46:381
27093992812,14cyclictest0-21swapper/108:20:321
27092992822,4cyclictest9-21ksoftirqd/008:00:510
27092992822,3cyclictest9-21ksoftirqd/008:30:430
27092992822,3cyclictest9-21ksoftirqd/007:40:150
27092992821,4cyclictest9-21ksoftirqd/011:32:170
27092992821,4cyclictest9-21ksoftirqd/011:32:160
27092992821,4cyclictest9-21ksoftirqd/010:21:170
27092992821,4cyclictest9-21ksoftirqd/009:36:450
133991280,1ptp4l401ktimersoftd/310:15:543
133991280,12ptp4l401ktimersoftd/311:18:333
27095992720,4cyclictest41-21ksoftirqd/308:19:443
27093992720,4cyclictest25-21ksoftirqd/109:38:481
27093992720,4cyclictest25-21ksoftirqd/109:14:231
2709299279,5cyclictest9-21ksoftirqd/010:25:400
27092992720,4cyclictest9-21ksoftirqd/009:46:570
27092992720,4cyclictest9-21ksoftirqd/008:13:390
27095992619,4cyclictest41-21ksoftirqd/310:52:293
27093992619,4cyclictest25-21ksoftirqd/107:39:111
27093992619,4cyclictest25-21ksoftirqd/107:39:111
27093992618,5cyclictest25-21ksoftirqd/108:14:091
27092992621,3cyclictest9-21ksoftirqd/012:36:000
27092992620,3cyclictest9-21ksoftirqd/011:47:210
27092992620,3cyclictest9-21ksoftirqd/009:21:050
27092992620,3cyclictest9-21ksoftirqd/007:35:030
27092992620,3cyclictest9-21ksoftirqd/007:35:030
27092992619,4cyclictest9-21ksoftirqd/009:55:150
27093992519,3cyclictest25-21ksoftirqd/109:22:101
27093992517,5cyclictest31393-21apt09:44:591
133991250,1ptp4l401ktimersoftd/308:55:413
133991250,1ptp4l401ktimersoftd/308:55:413
133991250,1ptp4l401ktimersoftd/308:20:353
27093992417,4cyclictest25-21ksoftirqd/108:40:501
2709399241,19cyclictest0-21swapper/108:00:321
27093992317,3cyclictest25-21ksoftirqd/109:09:101
27092992318,3cyclictest9-21ksoftirqd/010:03:480
27092992317,3cyclictest9-21ksoftirqd/012:03:180
27092992316,4cyclictest9-21ksoftirqd/011:06:430
2709299221,8cyclictest13657-21cat12:35:000
27092992215,4cyclictest9-21ksoftirqd/010:17:280
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional