You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-12-09 - 17:06

x86 Intel Atom E3845 @1915 MHz, Linux 4.19.37-rt20 (Profile)

Latency plot of system in rack #b, slot #6
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot6.osadl.org (updated Wed Dec 07, 2022 12:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3211121820,5sleep12028799cyclictest13:25:141
1463221770,3sleep02028099cyclictest13:41:070
1085721580,5sleep12028799cyclictest13:36:461
91191980,4ptp4l401ktimersoftd/311:02:463
245352820,2sleep30-21swapper/316:05:213
91191760,10ptp4l401ktimersoftd/315:52:283
91191740,1ptp4l401ktimersoftd/316:29:033
91191700,1ptp4l401ktimersoftd/314:10:513
91191670,1ptp4l401ktimersoftd/316:15:143
325512660,5sleep332556-21irqrtprio12:36:383
103142650,2sleep30-21swapper/312:56:493
103142650,2sleep30-21swapper/312:56:493
326382640,3sleep00-21swapper/016:12:490
181492620,14sleep125-21ksoftirqd/112:06:361
91191590,1ptp4l401ktimersoftd/313:27:363
91191580,1ptp4l401ktimersoftd/315:23:323
184892580,1sleep10-21swapper/115:58:131
91191570,1ptp4l401ktimersoftd/314:18:573
91191570,1ptp4l401ktimersoftd/314:18:573
20291995510,17cyclictest28340-21ssh15:36:012
58222540,2sleep20-21swapper/216:18:542
2029199538,17cyclictest20044-21awk15:26:492
91191520,1ptp4l401ktimersoftd/316:23:183
2029199508,20cyclictest11082-21apt-get15:51:382
2029199508,17cyclictest17886-21fschecks_time12:06:342
2029199508,17cyclictest14116-21/usr/sbin/munin13:06:372
2029199499,10cyclictest1973-21cstates12:41:372
2029199499,10cyclictest0-21swapper/213:32:042
2029199498,17cyclictest13357-21latency_hist13:06:192
91191480,1ptp4l401ktimersoftd/314:37:393
91191480,11ptp4l401ktimersoftd/313:53:443
2029199486,12cyclictest21312-21awk13:46:472
20280994810,10cyclictest9-21ksoftirqd/011:51:410
2029199479,33cyclictest24978-21fschecks_count12:21:392
2029199477,18cyclictest365-21sed12:36:392
2029199477,13cyclictest2520-21/usr/sbin/munin14:01:382
2029199477,13cyclictest14957-21cat13:41:212
2029199476,13cyclictest10478-21taskset15:51:052
2029199476,12cyclictest17082-21sed16:31:252
2029199475,12cyclictest26972-21acpi12:26:302
20280994716,10cyclictest9-21ksoftirqd/013:46:210
2029199466,12cyclictest5338-21perl15:11:452
2029199455,12cyclictest31609-21irqrtprio11:26:372
2029199455,12cyclictest10197-21/usr/sbin/munin11:51:412
2028099455,9cyclictest18851-21ssh13:11:380
20280994515,4cyclictest9-21ksoftirqd/015:01:220
2029199446,12cyclictest20798-21ssh13:12:302
2029199446,12cyclictest11113-21sed13:01:192
2029199446,12cyclictest11113-21sed13:01:192
2029199445,17cyclictest27751-21ptp4l-jitter14:26:482
2029199445,12cyclictest30509-21cat13:56:412
2029199445,12cyclictest13289-21diskmemload15:02:452
2029199445,11cyclictest10902-21strings15:16:482
2028099444,9cyclictest9-21ksoftirqd/012:11:350
188812440,6sleep2311rcuc/211:01:382
20280994332,3cyclictest9-21ksoftirqd/013:53:070
2029199427,10cyclictest0-21swapper/215:01:052
2028099425,8cyclictest24332-21ssh15:31:440
2028099424,9cyclictest9-21ksoftirqd/014:36:420
20280994212,10cyclictest9-21ksoftirqd/014:04:160
91191410,1ptp4l401ktimersoftd/314:55:343
91191410,1ptp4l401ktimersoftd/312:06:463
2029199415,11cyclictest13289-21diskmemload16:23:092
2028099415,9cyclictest9-21ksoftirqd/014:26:480
2028099415,9cyclictest9-21ksoftirqd/012:21:190
2028099415,8cyclictest9-21ksoftirqd/015:41:420
2028099415,8cyclictest9-21ksoftirqd/015:41:420
2028099414,9cyclictest9-21ksoftirqd/015:21:220
20280994130,3cyclictest9-21ksoftirqd/011:06:320
20280994130,3cyclictest9-21ksoftirqd/011:06:320
20280994116,4cyclictest9-21ksoftirqd/012:33:480
20280994113,8cyclictest9-21ksoftirqd/015:36:470
20280994113,8cyclictest9-21ksoftirqd/015:36:470
20280994113,11cyclictest9-21ksoftirqd/014:21:480
91191400,1ptp4l401ktimersoftd/315:12:353
91191400,1ptp4l401ktimersoftd/314:28:413
91191400,1ptp4l401ktimersoftd/313:58:003
2029199409,11cyclictest15357-21ssh14:47:282
2029199406,14cyclictest2870-21munin-run11:36:172
2028099405,5cyclictest25233-21grep11:11:500
2028099404,8cyclictest13289-21diskmemload14:31:440
2028099403,9cyclictest9-21ksoftirqd/012:41:510
2028099403,8cyclictest9-21ksoftirqd/012:56:440
2028099403,8cyclictest9-21ksoftirqd/012:56:440
20280994024,3cyclictest9-21ksoftirqd/013:16:360
20280994016,5cyclictest9-21ksoftirqd/013:04:350
20280994016,10cyclictest9-21ksoftirqd/013:06:370
20280994015,6cyclictest9-21ksoftirqd/012:26:410
91191390,1ptp4l401ktimersoftd/315:34:403
91191390,1ptp4l401ktimersoftd/315:29:553
91191390,0ptp4l401ktimersoftd/314:22:233
2029199397,18cyclictest0-21swapper/213:16:432
20291993911,11cyclictest30535-21switchtime15:36:582
20291993911,11cyclictest30535-21switchtime15:36:582
2028099395,5cyclictest9-21ksoftirqd/016:26:450
2028099394,7cyclictest20325-21ssh15:26:500
2028099393,9cyclictest9-21ksoftirqd/011:16:470
2028099393,9cyclictest9-21ksoftirqd/011:16:470
2028099392,9cyclictest9-21ksoftirqd/012:46:500
20280993928,3cyclictest9-21ksoftirqd/013:24:160
20280993924,3cyclictest9-21ksoftirqd/016:31:380
20280993923,3cyclictest9-21ksoftirqd/014:55:340
20280993919,10cyclictest9-21ksoftirqd/013:47:220
20280993916,8cyclictest9-21ksoftirqd/011:36:380
20280993916,10cyclictest9-21ksoftirqd/016:19:270
20280993916,10cyclictest9-21ksoftirqd/014:45:250
20280993915,7cyclictest9-21ksoftirqd/015:13:590
20280993915,4cyclictest9-21ksoftirqd/014:00:120
20280993913,5cyclictest9-21ksoftirqd/015:53:090
20280993913,4cyclictest9-21ksoftirqd/013:28:050
20280993912,11cyclictest9-21ksoftirqd/015:21:420
91191380,3ptp4l25921-21kworker/3:015:37:273
91191380,3ptp4l25921-21kworker/3:015:37:273
91191380,1ptp4l401ktimersoftd/316:07:233
91191380,1ptp4l401ktimersoftd/316:01:153
91191380,1ptp4l401ktimersoftd/313:45:333
91191380,1ptp4l401ktimersoftd/313:19:283
91191380,1ptp4l401ktimersoftd/313:13:393
91191380,1ptp4l401ktimersoftd/311:58:363
2029199382,11cyclictest3707-21rm15:10:032
20291993821,10cyclictest30357-21apt-get16:11:462
2028099387,6cyclictest9-21ksoftirqd/016:01:120
2028099386,7cyclictest121rcu_preempt16:01:530
2028099385,8cyclictest9-21ksoftirqd/012:21:450
2028099385,4cyclictest2860-21turbostat11:36:170
2028099384,7cyclictest9-21ksoftirqd/011:26:300
20280993827,3cyclictest9-21ksoftirqd/012:36:380
2028099381,9cyclictest9-21ksoftirqd/014:11:440
2028099381,9cyclictest9-21ksoftirqd/014:11:440
2028099381,9cyclictest9-21ksoftirqd/012:51:430
20280993815,9cyclictest9-21ksoftirqd/015:04:030
20280993815,5cyclictest9-21ksoftirqd/014:17:050
20280993815,5cyclictest9-21ksoftirqd/014:17:050
20280993813,10cyclictest9-21ksoftirqd/011:46:370
166732380,14sleep341-21ksoftirqd/313:08:503
91191370,1ptp4l401ktimersoftd/316:34:383
91191370,1ptp4l401ktimersoftd/315:45:583
91191370,1ptp4l401ktimersoftd/315:45:583
91191370,1ptp4l401ktimersoftd/314:57:133
91191370,1ptp4l401ktimersoftd/314:46:303
91191370,1ptp4l401ktimersoftd/313:33:283
91191370,1ptp4l401ktimersoftd/311:15:563
91191370,0ptp4l401ktimersoftd/313:48:203
2029199379,13cyclictest21292-21wc14:21:212
2029199379,13cyclictest21292-21wc14:21:212
2028099374,4cyclictest9-21ksoftirqd/013:36:200
2028099374,12cyclictest9-21ksoftirqd/011:56:410
2028099372,8cyclictest9-21ksoftirqd/011:41:370
2028099372,5cyclictest9-21ksoftirqd/012:01:440
20280993725,3cyclictest9-21ksoftirqd/014:47:460
20280993724,3cyclictest9-21ksoftirqd/015:48:250
20280993714,9cyclictest9-21ksoftirqd/016:10:160
20280993714,4cyclictest9-21ksoftirqd/015:11:200
20280993712,7cyclictest9-21ksoftirqd/016:21:580
91191360,1ptp4l401ktimersoftd/316:16:363
91191360,1ptp4l401ktimersoftd/314:41:523
91191360,1ptp4l401ktimersoftd/314:11:403
91191360,1ptp4l401ktimersoftd/314:11:403
91191360,0ptp4l401ktimersoftd/314:34:343
2029199366,12cyclictest0-21swapper/213:26:462
2028099362,11cyclictest7537-21tail14:06:410
20291993518,10cyclictest18027-21apt-get14:51:382
20280993510,4cyclictest9-21ksoftirqd/012:06:310
2028099350,10cyclictest9-21ksoftirqd/011:21:330
91191340,1ptp4l401ktimersoftd/315:19:573
91191340,1ptp4l401ktimersoftd/315:03:393
91191340,1ptp4l401ktimersoftd/311:36:423
2029199348,10cyclictest31542-21diskstats14:31:422
2029199347,17cyclictest6962-21/usr/sbin/munin14:06:392
20287993411,4cyclictest25-21ksoftirqd/116:31:471
91191330,1ptp4l401ktimersoftd/315:51:013
91191330,1ptp4l401ktimersoftd/315:08:283
91191330,1ptp4l401ktimersoftd/313:25:253
91191330,1ptp4l401ktimersoftd/311:26:483
91191330,10ptp4l401ktimersoftd/313:38:203
2029199337,10cyclictest899-21runrttasks11:56:192
2029199336,11cyclictest0-21swapper/216:31:482
2029199335,12cyclictest0-21swapper/211:11:182
2029199335,11cyclictest0-21swapper/212:46:452
20287993316,7cyclictest25-21ksoftirqd/114:11:581
20287993316,7cyclictest25-21ksoftirqd/114:11:581
20287993310,4cyclictest25-21ksoftirqd/112:46:501
91191320,1ptp4l401ktimersoftd/313:02:013
91191320,1ptp4l401ktimersoftd/312:01:583
91191320,1ptp4l401ktimersoftd/311:49:293
20291993219,7cyclictest28371-21apt-get13:21:332
20291993219,6cyclictest7917-21apt-get11:46:352
2028799322,3cyclictest121rcu_preempt16:16:241
2028799320,3cyclictest121rcu_preempt11:56:371
91191310,1ptp4l401ktimersoftd/312:23:413
91191310,0ptp4l401ktimersoftd/311:44:193
2029199314,10cyclictest7187-21ssh14:39:562
20291993119,7cyclictest28340-21apt-get11:21:302
2028799319,4cyclictest25-21ksoftirqd/114:56:471
2028799319,4cyclictest25-21ksoftirqd/111:46:421
2028799318,3cyclictest25-21ksoftirqd/113:06:401
20287993122,3cyclictest25-21ksoftirqd/116:31:251
20287993120,9cyclictest25-21ksoftirqd/114:46:491
20287993116,5cyclictest25-21ksoftirqd/113:11:371
1994123110,9sleep10-21swapper/111:03:121
91191300,1ptp4l401ktimersoftd/311:24:233
91191300,0ptp4l401ktimersoftd/312:12:323
2029199304,11cyclictest0-21swapper/212:51:502
2028799306,3cyclictest25-21ksoftirqd/113:51:481
2028799302,2cyclictest121rcu_preempt14:31:211
2028799301,4cyclictest1324-21ssh13:26:421
2029199296,20cyclictest0-21swapper/215:41:412
2029199296,20cyclictest0-21swapper/215:41:412
2029199292,16cyclictest20765-21rm16:01:402
20291992917,6cyclictest8408-21apt-get14:41:332
20291992914,11cyclictest0-21swapper/212:11:302
20291992910,16cyclictest11939-21munin-node14:11:342
20291992910,16cyclictest11939-21munin-node14:11:342
2028799297,4cyclictest25-21ksoftirqd/114:01:201
2028799297,3cyclictest25-21ksoftirqd/112:01:461
2028799296,6cyclictest25-21ksoftirqd/111:26:371
2028799295,6cyclictest25-21ksoftirqd/114:51:431
2028799295,4cyclictest25-21ksoftirqd/113:01:511
2028799294,4cyclictest25-21ksoftirqd/111:21:351
2028799293,9cyclictest25-21ksoftirqd/114:16:431
2028799293,9cyclictest25-21ksoftirqd/114:16:431
2028799293,3cyclictest121rcu_preempt13:16:511
20287992918,4cyclictest25-21ksoftirqd/112:51:411
20287992914,4cyclictest25-21ksoftirqd/115:21:221
2028799291,3cyclictest15157-21apt-get13:41:321
2028799291,3cyclictest121rcu_preempt16:16:401
2028799291,3cyclictest121rcu_preempt13:31:421
2029199284,10cyclictest840-21lldpd16:00:182
20291992813,11cyclictest0-21swapper/212:31:342
2028799289,4cyclictest25-21ksoftirqd/113:46:491
2028799286,3cyclictest25-21ksoftirqd/115:06:581
2028799285,4cyclictest25-21ksoftirqd/112:31:401
2028799285,3cyclictest25-21ksoftirqd/116:01:461
2028799284,7cyclictest25-21ksoftirqd/115:51:231
2028799282,3cyclictest22903-21kworker/1:215:51:481
20287992823,3cyclictest25-21ksoftirqd/112:11:311
20287992823,3cyclictest25-21ksoftirqd/111:36:401
2028799281,7cyclictest7612-21munin-run11:46:171
20287992815,4cyclictest25-21ksoftirqd/115:31:441
20287992814,4cyclictest25-21ksoftirqd/112:41:531
20287992814,4cyclictest25-21ksoftirqd/111:56:181
2028799281,3cyclictest30883-21users15:37:011
2028799281,3cyclictest30883-21users15:37:011
186302285,10sleep00-21swapper/011:01:370
20300992721,3cyclictest41-21ksoftirqd/312:51:343
20291992715,6cyclictest15186-21apt-get12:01:352
20291992710,14cyclictest22094-21grep14:21:432
2028799276,3cyclictest25-21ksoftirqd/111:06:401
2028799276,3cyclictest25-21ksoftirqd/111:06:401
2028799275,4cyclictest25-21ksoftirqd/114:21:431
2028799275,4cyclictest25-21ksoftirqd/111:16:381
2028799275,4cyclictest25-21ksoftirqd/111:16:381
2028799275,3cyclictest121rcu_preempt15:06:221
20287992722,3cyclictest46550irq/101-ens3-Tx16:24:371
20287992722,3cyclictest25-21ksoftirqd/114:36:221
20287992721,3cyclictest25-21ksoftirqd/111:31:341
20287992714,4cyclictest25-21ksoftirqd/115:41:421
20287992714,4cyclictest25-21ksoftirqd/115:41:421
20287992713,4cyclictest25-21ksoftirqd/112:36:481
20287992711,4cyclictest25-21ksoftirqd/115:12:051
91191260,0ptp4l401ktimersoftd/312:35:013
2029199264,8cyclictest13289-21diskmemload15:21:462
20291992612,11cyclictest3672-21ls11:36:372
20291992612,10cyclictest26588-21runrttasks16:06:472
2028799264,5cyclictest121rcu_preempt13:01:191
2028799264,5cyclictest121rcu_preempt13:01:191
20287992621,3cyclictest25-21ksoftirqd/115:26:471
20287992612,4cyclictest25-21ksoftirqd/114:31:441
20287992612,4cyclictest25-21ksoftirqd/112:21:191
2028799260,6cyclictest121rcu_preempt14:06:371
91191250,1ptp4l401ktimersoftd/314:01:573
91191250,1ptp4l401ktimersoftd/312:46:573
91191250,1ptp4l401ktimersoftd/312:45:523
91191250,1ptp4l401ktimersoftd/312:16:503
91191250,1ptp4l401ktimersoftd/311:51:333
91191250,1ptp4l401ktimersoftd/311:16:363
91191250,1ptp4l401ktimersoftd/311:16:363
20291992514,8cyclictest5900-21grep11:41:342
20291992513,7cyclictest22792-21latency_hist11:11:172
20291992513,7cyclictest22792-21latency_hist11:11:172
20291992512,9cyclictest22814-21chrt12:16:452
20291992512,9cyclictest16338-21/usr/sbin/munin13:41:432
2028799259,7cyclictest25-21ksoftirqd/114:01:481
2028799254,3cyclictest25-21ksoftirqd/111:11:181
20287992521,3cyclictest25-21ksoftirqd/112:21:421
20287992520,3cyclictest25-21ksoftirqd/116:06:561
20287992519,3cyclictest25-21ksoftirqd/115:21:371
20287992519,3cyclictest25-21ksoftirqd/114:41:551
20287992512,3cyclictest25-21ksoftirqd/112:26:411
91191240,1ptp4l401ktimersoftd/312:26:333
91191240,1ptp4l401ktimersoftd/311:31:393
91191240,1ptp4l401ktimersoftd/311:06:323
91191240,1ptp4l401ktimersoftd/311:06:323
20291992413,8cyclictest26254-21/usr/sbin/munin11:16:402
20291992413,8cyclictest26254-21/usr/sbin/munin11:16:402
20291992311,8cyclictest25275-21/usr/sbin/munin13:51:382
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional