You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-06-07 - 21:30

x86 Intel Atom E3845 @1910 MHz, Linux 4.19.37-rt19 (Profile)

Latency plot of system in rack #d, slot #3
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Up99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 792 highest latencies:
System rackdslot3.osadl.org (updated Wed Jun 07, 2023 12:45:57)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3098621690,8sleep0250699cyclictest10:30:010
25099911948,8cyclictest41-21ksoftirqd/309:40:213
8255911010,3ptp4l401ktimersoftd/307:09:383
21042766,13sleep00-21swapper/007:09:150
2508997433,14cyclictest33-21ksoftirqd/210:13:362
2508997332,11cyclictest33-21ksoftirqd/208:40:352
18932735,12sleep10-21swapper/107:06:401
2509997231,10cyclictest41-21ksoftirqd/310:15:353
2509997220,10cyclictest41-21ksoftirqd/312:25:373
18632725,12sleep20-21swapper/207:06:152
2508997135,4cyclictest33-21ksoftirqd/211:45:012
825591700,2ptp4l401ktimersoftd/309:04:013
825591700,2ptp4l401ktimersoftd/309:04:013
2506996931,5cyclictest9-21ksoftirqd/009:25:380
2509996832,4cyclictest41-21ksoftirqd/309:30:233
825591670,1ptp4l401ktimersoftd/310:58:593
825591670,1ptp4l401ktimersoftd/310:14:433
825591670,1ptp4l401ktimersoftd/307:11:093
58042670,4sleep3250999cyclictest10:45:013
2508996718,5cyclictest33-21ksoftirqd/210:15:312
2506996734,3cyclictest9-21ksoftirqd/008:50:240
246582670,2sleep20-21swapper/209:05:212
825591660,2ptp4l401ktimersoftd/310:00:383
825591660,1ptp4l401ktimersoftd/312:31:523
825591660,1ptp4l401ktimersoftd/312:31:523
320422660,4sleep232044-21irqrtprio10:30:262
2509996628,10cyclictest41-21ksoftirqd/309:56:123
2508996651,4cyclictest33-21ksoftirqd/208:40:012
2508996619,11cyclictest33-21ksoftirqd/207:11:382
825591650,1ptp4l401ktimersoftd/310:49:073
2509996528,9cyclictest41-21ksoftirqd/312:06:303
825591640,1ptp4l401ktimersoftd/308:50:253
2509996428,7cyclictest41-21ksoftirqd/309:45:183
2508996420,11cyclictest33-21ksoftirqd/212:01:512
198442640,6sleep01131-21lldpd08:55:240
825591630,1ptp4l401ktimersoftd/311:33:053
825591630,1ptp4l401ktimersoftd/311:07:503
825591630,1ptp4l401ktimersoftd/310:21:513
2509996328,9cyclictest41-21ksoftirqd/308:22:383
2509996328,10cyclictest41-21ksoftirqd/310:53:503
2508996319,9cyclictest33-21ksoftirqd/209:25:282
2508996318,10cyclictest33-21ksoftirqd/209:40:402
2509996234,4cyclictest41-21ksoftirqd/307:40:283
2508996234,7cyclictest33-21ksoftirqd/209:10:312
2508996229,8cyclictest33-21ksoftirqd/210:50:232
2508996223,4cyclictest121rcu_preempt08:51:542
2508996217,5cyclictest33-21ksoftirqd/210:45:322
88112610,2sleep00-21swapper/009:40:330
2509996135,5cyclictest41-21ksoftirqd/311:35:293
2509996128,5cyclictest41-21ksoftirqd/311:12:203
2509996128,5cyclictest41-21ksoftirqd/311:12:203
2509996116,5cyclictest41-21ksoftirqd/309:07:163
2508996134,10cyclictest33-21ksoftirqd/212:15:362
2508996125,3cyclictest121rcu_preempt08:45:272
2508996121,7cyclictest33-21ksoftirqd/211:00:332
2508996116,4cyclictest33-21ksoftirqd/209:24:402
2509996028,9cyclictest41-21ksoftirqd/311:59:133
2509996025,5cyclictest41-21ksoftirqd/310:37:443
2509996024,10cyclictest41-21ksoftirqd/308:56:053
2509996023,4cyclictest41-21ksoftirqd/307:19:143
2508996040,5cyclictest33-21ksoftirqd/207:25:232
2506996019,7cyclictest9-21ksoftirqd/009:20:180
2509995941,5cyclictest41-21ksoftirqd/307:25:163
2509995935,4cyclictest41-21ksoftirqd/309:37:033
2509995927,3cyclictest41-21ksoftirqd/310:09:103
2509995924,6cyclictest41-21ksoftirqd/308:16:053
2508995926,4cyclictest33-21ksoftirqd/212:24:362
2508995925,4cyclictest121rcu_preempt10:35:232
2508995924,5cyclictest33-21ksoftirqd/207:53:332
2508995922,4cyclictest121rcu_preempt07:35:292
2508995919,9cyclictest33-21ksoftirqd/210:20:252
2508995919,9cyclictest33-21ksoftirqd/209:00:202
2508995919,9cyclictest33-21ksoftirqd/209:00:202
2508995918,5cyclictest33-21ksoftirqd/209:55:432
2508995917,4cyclictest33-21ksoftirqd/212:30:162
2508995917,4cyclictest33-21ksoftirqd/212:30:162
199782590,3sleep11131-21lldpd10:05:201
2509995825,8cyclictest41-21ksoftirqd/307:22:403
2509995818,15cyclictest41-21ksoftirqd/309:51:123
2508995819,7cyclictest33-21ksoftirqd/208:00:242
2508995814,4cyclictest33-21ksoftirqd/210:25:302
250699588,6cyclictest1131-21lldpd07:30:190
188072580,4sleep0101ktimersoftd/010:00:410
2509995749,4cyclictest41-21ksoftirqd/307:55:163
2509995735,4cyclictest41-21ksoftirqd/308:45:273
2509995723,5cyclictest41-21ksoftirqd/311:23:563
2509995721,8cyclictest41-21ksoftirqd/311:00:493
2508995735,5cyclictest33-21ksoftirqd/207:55:352
2508995733,9cyclictest33-21ksoftirqd/209:51:042
2508995723,4cyclictest121rcu_preempt12:05:252
2508995723,14cyclictest33-21ksoftirqd/211:50:312
2508995721,9cyclictest33-21ksoftirqd/207:40:382
2508995720,6cyclictest33-21ksoftirqd/209:35:242
2508995715,10cyclictest33-21ksoftirqd/208:13:382
2509995639,4cyclictest41-21ksoftirqd/311:55:013
2509995637,5cyclictest41-21ksoftirqd/312:35:253
2509995637,5cyclictest41-21ksoftirqd/312:35:253
2509995626,5cyclictest41-21ksoftirqd/309:20:343
2509995626,3cyclictest41-21ksoftirqd/307:39:463
2509995618,12cyclictest41-21ksoftirqd/308:02:203
2508995646,4cyclictest33-21ksoftirqd/211:45:222
2508995635,4cyclictest33-21ksoftirqd/211:20:002
2508995633,5cyclictest33-21ksoftirqd/211:05:332
2508995627,4cyclictest33-21ksoftirqd/211:10:322
2508995627,4cyclictest33-21ksoftirqd/211:10:322
2509995540,3cyclictest41-21ksoftirqd/309:15:393
2509995537,7cyclictest41-21ksoftirqd/308:40:373
2509995536,4cyclictest41-21ksoftirqd/312:17:303
2509995536,4cyclictest41-21ksoftirqd/310:25:303
2509995535,5cyclictest41-21ksoftirqd/311:15:283
2509995535,5cyclictest41-21ksoftirqd/308:30:323
2509995534,4cyclictest41-21ksoftirqd/312:10:223
2509995519,7cyclictest41-21ksoftirqd/312:20:013
2508995537,5cyclictest33-21ksoftirqd/207:15:392
2508995536,4cyclictest33-21ksoftirqd/208:20:002
2508995517,4cyclictest121rcu_preempt10:55:212
250799554,8cyclictest25-21ksoftirqd/110:57:071
250699551,11cyclictest9-21ksoftirqd/011:04:420
2509995436,4cyclictest41-21ksoftirqd/308:35:263
2509995436,4cyclictest41-21ksoftirqd/308:05:313
2509995435,5cyclictest41-21ksoftirqd/311:40:213
2509995435,5cyclictest41-21ksoftirqd/308:10:393
2509995435,5cyclictest41-21ksoftirqd/307:45:253
2509995434,5cyclictest41-21ksoftirqd/309:25:353
2509995434,4cyclictest41-21ksoftirqd/309:10:193
2509995433,5cyclictest41-21ksoftirqd/307:50:323
2509995433,4cyclictest41-21ksoftirqd/311:25:263
2509995433,4cyclictest41-21ksoftirqd/310:30:183
2506995435,8cyclictest9-21ksoftirqd/007:55:360
2506995434,9cyclictest9-21ksoftirqd/007:20:400
2506995432,5cyclictest9-21ksoftirqd/007:50:400
2509995335,4cyclictest41-21ksoftirqd/311:45:233
2509995324,3cyclictest41-21ksoftirqd/312:04:153
2508995337,4cyclictest33-21ksoftirqd/211:35:272
2508995335,4cyclictest33-21ksoftirqd/211:25:372
2508995335,4cyclictest33-21ksoftirqd/207:45:252
2508995334,5cyclictest33-21ksoftirqd/208:59:202
2508995334,4cyclictest33-21ksoftirqd/207:30:362
2508995333,4cyclictest33-21ksoftirqd/212:10:382
2508995332,5cyclictest33-21ksoftirqd/210:01:402
2508995329,12cyclictest33-21ksoftirqd/209:30:312
2508995322,4cyclictest33-21ksoftirqd/212:37:022
2508995322,4cyclictest33-21ksoftirqd/212:37:022
2508995319,5cyclictest33-21ksoftirqd/210:05:292
2508995314,4cyclictest33-21ksoftirqd/208:25:302
250799530,10cyclictest13534-21if_enp1s011:00:261
250699537,7cyclictest1131-21lldpd12:24:060
2506995338,4cyclictest9-21ksoftirqd/011:21:370
2509995234,5cyclictest41-21ksoftirqd/308:25:333
2509995234,5cyclictest41-21ksoftirqd/307:30:163
2508995235,4cyclictest33-21ksoftirqd/212:26:072
2508995230,5cyclictest33-21ksoftirqd/211:30:302
2508995217,4cyclictest121rcu_preempt08:31:542
2508995215,5cyclictest33-21ksoftirqd/211:21:042
2508995213,8cyclictest33-21ksoftirqd/209:16:362
2508995212,9cyclictest33-21ksoftirqd/211:57:522
250799524,9cyclictest569-21in:imuxsock10:00:341
250899514,3cyclictest121rcu_preempt07:20:212
2508995135,4cyclictest33-21ksoftirqd/208:25:012
2508995134,4cyclictest33-21ksoftirqd/208:05:142
250799512,4cyclictest1131-21lldpd10:25:001
250799511,4cyclictest1123-21snmpd09:55:241
250699517,5cyclictest9-21ksoftirqd/010:30:280
2506995133,5cyclictest9-21ksoftirqd/011:42:250
2506995132,5cyclictest9-21ksoftirqd/008:07:060
2506995131,5cyclictest9-21ksoftirqd/012:16:210
2506995130,5cyclictest9-21ksoftirqd/007:25:280
2508995033,6cyclictest33-21ksoftirqd/210:40:412
2506995033,5cyclictest9-21ksoftirqd/012:35:220
2506995033,5cyclictest9-21ksoftirqd/012:35:210
2506995032,5cyclictest9-21ksoftirqd/011:39:400
2506995030,5cyclictest9-21ksoftirqd/010:15:310
2506995029,5cyclictest9-21ksoftirqd/008:04:240
250699502,4cyclictest121rcu_preempt11:15:310
2506995014,11cyclictest9-21ksoftirqd/011:52:470
250699500,3cyclictest121rcu_preempt08:30:010
2508994916,4cyclictest121rcu_preempt09:46:282
250799491,5cyclictest2919-21python11:45:381
250799491,4cyclictest121rcu_preempt12:05:381
2506994942,4cyclictest9-21ksoftirqd/010:15:010
2506994933,4cyclictest9-21ksoftirqd/011:15:010
2506994933,4cyclictest9-21ksoftirqd/011:15:000
2506994931,5cyclictest9-21ksoftirqd/008:35:180
2506994930,5cyclictest9-21ksoftirqd/007:43:520
2506994928,5cyclictest9-21ksoftirqd/010:35:300
2506994927,10cyclictest9-21ksoftirqd/012:10:240
2506994918,5cyclictest9-21ksoftirqd/009:10:220
2506994831,5cyclictest9-21ksoftirqd/009:15:360
2506994831,4cyclictest9-21ksoftirqd/012:06:540
2506994829,5cyclictest9-21ksoftirqd/008:40:180
2506994828,5cyclictest9-21ksoftirqd/007:37:400
2506994827,5cyclictest9-21ksoftirqd/011:30:300
2506994827,5cyclictest9-21ksoftirqd/007:15:380
250699480,3cyclictest121rcu_preempt12:00:240
250799472,3cyclictest121rcu_preempt07:45:301
250799470,4cyclictest1131-21lldpd10:50:011
2506994730,5cyclictest9-21ksoftirqd/007:10:150
2506994728,5cyclictest9-21ksoftirqd/008:30:220
2506994727,5cyclictest9-21ksoftirqd/010:45:250
2506994727,5cyclictest9-21ksoftirqd/009:55:200
2506994727,5cyclictest9-21ksoftirqd/009:35:400
2506994727,5cyclictest9-21ksoftirqd/007:45:390
250699471,5cyclictest9-21ksoftirqd/011:45:460
250799465,3cyclictest121rcu_preempt10:15:461
250799464,10cyclictest121rcu_preempt09:40:441
250799463,9cyclictest1123-21snmpd08:30:291
2507994639,4cyclictest25-21ksoftirqd/111:30:321
250799463,3cyclictest121rcu_preempt07:30:001
250799461,4cyclictest9201-21meminfo10:50:311
2506994626,5cyclictest9-21ksoftirqd/010:40:250
2506994626,5cyclictest9-21ksoftirqd/010:05:210
2506994626,5cyclictest9-21ksoftirqd/009:54:110
250699461,3cyclictest121rcu_preempt10:50:360
250699461,11cyclictest1123-21snmpd10:55:220
250799455,6cyclictest121rcu_preempt11:10:391
250799455,6cyclictest121rcu_preempt11:10:391
250799454,4cyclictest25-21ksoftirqd/111:25:381
250799454,3cyclictest121rcu_preempt11:20:311
250799454,3cyclictest121rcu_preempt08:01:461
250799451,4cyclictest1123-21snmpd10:10:181
250799451,3cyclictest121rcu_preempt12:10:231
2506994529,5cyclictest9-21ksoftirqd/009:00:190
2506994529,5cyclictest9-21ksoftirqd/009:00:190
2506994528,5cyclictest9-21ksoftirqd/010:25:010
2506994527,5cyclictest9-21ksoftirqd/012:30:200
2506994527,5cyclictest9-21ksoftirqd/012:30:200
2506994526,5cyclictest9-21ksoftirqd/011:59:100
2506994525,5cyclictest9-21ksoftirqd/011:05:380
2506994522,5cyclictest9-21ksoftirqd/009:05:280
250799444,6cyclictest121rcu_preempt08:20:181
2507994439,3cyclictest25-21ksoftirqd/107:20:011
2507994437,4cyclictest25-21ksoftirqd/108:45:001
250799443,3cyclictest121rcu_preempt12:35:171
250799443,3cyclictest121rcu_preempt12:35:171
250799443,3cyclictest121rcu_preempt09:25:201
2507994432,3cyclictest25-21ksoftirqd/108:10:221
2507994431,4cyclictest25-21ksoftirqd/107:35:231
250799442,4cyclictest121rcu_preempt08:10:011
250799441,3cyclictest121rcu_preempt09:45:221
2506994437,4cyclictest9-21ksoftirqd/009:35:010
2506994426,4cyclictest9-21ksoftirqd/009:45:340
2506994423,5cyclictest9-21ksoftirqd/011:25:250
2506994415,5cyclictest9-21ksoftirqd/008:10:230
2506994414,5cyclictest9-21ksoftirqd/012:25:270
2507994338,3cyclictest25-21ksoftirqd/110:35:001
250799433,3cyclictest121rcu_preempt08:17:331
250799432,4cyclictest1131-21lldpd07:40:521
250799432,3cyclictest121rcu_preempt07:20:201
2506994325,5cyclictest9-21ksoftirqd/008:48:290
250799423,3cyclictest121rcu_preempt07:50:391
2507994227,4cyclictest25-21ksoftirqd/108:30:001
2507994221,7cyclictest25-21ksoftirqd/109:00:221
2507994221,7cyclictest25-21ksoftirqd/109:00:221
250799421,4cyclictest1131-21lldpd08:35:351
2506994236,3cyclictest9-21ksoftirqd/008:25:000
2507994136,3cyclictest25-21ksoftirqd/108:55:001
2507994134,4cyclictest25-21ksoftirqd/112:15:231
2507994126,4cyclictest25-21ksoftirqd/109:21:071
250799411,8cyclictest1131-21lldpd11:55:211
250799411,3cyclictest0-21swapper/111:40:311
250799404,6cyclictest1131-21lldpd08:45:191
2507994034,3cyclictest25-21ksoftirqd/112:20:011
250799401,8cyclictest2505-21cyclictest11:05:351
2507993934,3cyclictest25-21ksoftirqd/110:37:321
2507993921,5cyclictest25-21ksoftirqd/107:55:171
2507993920,5cyclictest25-21ksoftirqd/109:14:431
250799391,7cyclictest206-21systemd-journal12:00:311
250799391,7cyclictest0-21swapper/108:55:301
250799390,3cyclictest25-21ksoftirqd/110:25:371
250699390,9cyclictest9-21ksoftirqd/008:15:180
250799388,3cyclictest121rcu_preempt09:09:261
250799385,4cyclictest1131-21lldpd12:27:021
2507993831,4cyclictest25-21ksoftirqd/110:40:301
250799381,7cyclictest1123-21snmpd07:10:281
250799380,7cyclictest121rcu_preempt11:15:211
250799380,3cyclictest121rcu_preempt11:50:271
2507993730,4cyclictest25-21ksoftirqd/109:15:231
2507993719,4cyclictest25-21ksoftirqd/109:50:561
250799371,4cyclictest1123-21snmpd07:35:001
250799370,7cyclictest1131-21lldpd09:30:221
250799370,4cyclictest1123-21snmpd12:30:151
250799370,4cyclictest1123-21snmpd12:30:151
2507993621,4cyclictest25-21ksoftirqd/109:35:361
2507993522,4cyclictest25-21ksoftirqd/111:35:291
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional