You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-02-07 - 13:13

x86 Intel Atom E3845 @1915 MHz, Linux 4.19.37-rt20 (Profile)

Latency plot of system in rack #b, slot #6
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rackbslot6.osadl.org (updated Tue Feb 07, 2023 00:43:32)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
805921690,5sleep12467199cyclictest03:54:021
2836921500,5sleep32468399cyclictest04:15:403
2836921500,5sleep32468399cyclictest04:15:403
911911050,4ptp4l401ktimersoftd/323:38:133
227412770,4sleep00-21swapper/023:35:230
24671996855,6cyclictest25-21ksoftirqd/103:25:271
40592670,2sleep20-21swapper/203:49:492
63672660,2sleep30-21swapper/303:50:473
34422620,2sleep20-21swapper/205:01:572
24671996147,7cyclictest25-21ksoftirqd/103:10:281
283472580,2sleep00-21swapper/000:50:430
219162570,2sleep20-21swapper/200:40:252
313902550,2sleep30-21swapper/304:56:203
278772550,1sleep20-21swapper/204:52:542
24671995342,5cyclictest25-21ksoftirqd/101:40:311
24671995240,5cyclictest25-21ksoftirqd/104:30:331
266212500,3sleep125-21ksoftirqd/102:25:321
24671995037,7cyclictest25-21ksoftirqd/102:45:261
171792500,2sleep30-21swapper/302:50:433
24671994937,6cyclictest25-21ksoftirqd/103:30:281
2468399484,16cyclictest11549-21sed03:20:443
2468399481,18cyclictest19882-21snmp_o5powerctl01:40:463
2467599471,26cyclictest12207-21ssh03:21:262
2467599471,25cyclictest7624-21snmp_o5powerctl01:15:402
24671994738,4cyclictest25-21ksoftirqd/123:55:251
24671994737,4cyclictest25-21ksoftirqd/104:20:301
24671994736,6cyclictest25-21ksoftirqd/100:10:241
24671994734,6cyclictest25-21ksoftirqd/104:40:361
42152460,1sleep20-21swapper/202:00:352
2468399460,19cyclictest302-21awk23:55:263
2467599461,32cyclictest13546-21apt-get04:00:292
24671994636,4cyclictest25-21ksoftirqd/103:00:271
24671994434,4cyclictest25-21ksoftirqd/103:15:271
2441724425,8sleep20-21swapper/223:38:172
2468399430,17cyclictest2546-21sh04:24:023
2467599430,26cyclictest0-21swapper/205:10:162
91191420,1ptp4l401ktimersoftd/302:10:033
91191420,1ptp4l401ktimersoftd/301:56:543
2467599421,25cyclictest7903-21ssh04:30:172
2467599412,32cyclictest25810-21apt-get02:25:252
2467599411,36cyclictest24761-21grep00:45:302
2467599411,24cyclictest789-21ssh04:20:472
2467599410,26cyclictest32506-21ssh03:09:042
24671994130,5cyclictest25-21ksoftirqd/101:30:281
91191400,1ptp4l401ktimersoftd/304:30:543
2467599402,33cyclictest2358-21cut02:35:322
2467599402,33cyclictest2358-21cut02:35:322
2467599401,23cyclictest27309-21/usr/sbin/munin04:15:322
2467599401,23cyclictest27309-21/usr/sbin/munin04:15:322
2467599400,2cyclictest0-21swapper/201:53:432
2467599400,19cyclictest28925-21irqrtprio03:40:372
91191390,1ptp4l401ktimersoftd/302:03:063
91191390,1ptp4l401ktimersoftd/301:50:563
91191390,1ptp4l401ktimersoftd/301:32:393
91191390,0ptp4l401ktimersoftd/303:34:233
265222390,3sleep2311rcuc/201:49:312
91191380,1ptp4l401ktimersoftd/304:51:443
91191380,1ptp4l401ktimersoftd/304:47:393
91191380,1ptp4l401ktimersoftd/303:05:513
91191380,1ptp4l401ktimersoftd/303:03:443
91191380,1ptp4l401ktimersoftd/302:33:023
91191380,1ptp4l401ktimersoftd/302:33:023
91191380,1ptp4l401ktimersoftd/302:13:383
91191380,1ptp4l401ktimersoftd/301:15:363
91191380,0ptp4l401ktimersoftd/303:47:033
91191380,0ptp4l401ktimersoftd/302:37:103
91191380,0ptp4l401ktimersoftd/302:37:103
91191380,0ptp4l401ktimersoftd/302:20:533
2467599381,34cyclictest20685-21sed03:30:462
2467599381,32cyclictest23538-21apt-get03:35:312
2467599381,32cyclictest12912-21grep02:10:302
91191370,1ptp4l401ktimersoftd/304:11:233
91191370,1ptp4l401ktimersoftd/304:11:233
91191370,1ptp4l401ktimersoftd/304:00:083
91191370,1ptp4l401ktimersoftd/302:58:073
91191370,1ptp4l401ktimersoftd/302:27:243
91191370,1ptp4l401ktimersoftd/301:25:333
91191370,1ptp4l401ktimersoftd/300:04:413
91191370,0ptp4l401ktimersoftd/303:43:233
91191370,0ptp4l401ktimersoftd/303:26:373
91191370,0ptp4l401ktimersoftd/300:37:583
2468399370,14cyclictest15073-21switchtime04:00:453
2467599372,28cyclictest31756-21apt-get01:55:322
2467599371,32cyclictest30661-21/usr/sbin/munin04:55:462
24671993731,3cyclictest25-21ksoftirqd/102:30:281
24671993731,3cyclictest25-21ksoftirqd/102:30:281
24671993726,4cyclictest25-21ksoftirqd/101:00:321
24671993714,4cyclictest25-21ksoftirqd/100:30:431
91191360,1ptp4l401ktimersoftd/304:39:473
91191360,1ptp4l401ktimersoftd/303:40:053
91191360,1ptp4l401ktimersoftd/303:15:043
91191360,1ptp4l401ktimersoftd/301:46:183
91191360,0ptp4l401ktimersoftd/305:01:323
91191360,0ptp4l401ktimersoftd/301:12:193
91191360,0ptp4l401ktimersoftd/300:08:393
2467599363,27cyclictest24925-21apt-get23:40:212
2467599363,26cyclictest20125-21apt-get02:55:262
2467599362,28cyclictest12331-21apt-get04:35:362
24675993616,17cyclictest18334-21sh03:29:452
2467599361,20cyclictest10374-21ptp4l03:55:412
2467599360,32cyclictest16164-21ssh02:50:322
91191350,1ptp4l401ktimersoftd/304:08:093
91191350,1ptp4l401ktimersoftd/304:08:093
91191350,1ptp4l401ktimersoftd/302:47:513
91191350,0ptp4l401ktimersoftd/304:25:583
2467599353,27cyclictest8162-21apt-get02:05:332
2467599353,25cyclictest18076-21apt-get04:05:312
2467599353,25cyclictest18076-21apt-get04:05:312
2467599353,24cyclictest8651-21apt-get01:20:282
2467599352,25cyclictest29073-21apt-get00:55:232
24675993514,18cyclictest7468-21sh03:52:512
2467599351,30cyclictest5689-21tr03:15:292
2467599351,26cyclictest19573-21apt-get00:35:292
24675993512,20cyclictest17423-21irqrtprio04:40:432
24675993510,21cyclictest28818-21cut23:45:322
2467599350,1cyclictest0-21swapper/201:41:232
24671993525,5cyclictest25-21ksoftirqd/102:35:311
24671993525,5cyclictest25-21ksoftirqd/102:35:311
240962353,18sleep122436-21/usr/sbin/munin23:35:421
2467599349,22cyclictest8195-21sed02:40:452
2467599345,23cyclictest0-21swapper/201:05:302
2467599345,23cyclictest0-21swapper/201:05:302
2467599342,24cyclictest22588-21apt-get04:10:282
2467599342,24cyclictest22588-21apt-get04:10:282
24675993415,16cyclictest6383-21sh04:27:452
24675993415,16cyclictest30173-21/usr/sbin/munin02:30:292
24675993415,16cyclictest30173-21/usr/sbin/munin02:30:292
24675993414,17cyclictest18160-21sh02:15:382
2467599341,30cyclictest17945-21sed00:30:322
2467599341,26cyclictest3915-21apt-get01:10:262
2467199349,6cyclictest121rcu_preempt04:00:281
2467599332,27cyclictest12663-21fschecks_time00:20:282
24675993313,17cyclictest26652-21sh03:01:122
2467599330,29cyclictest21686-21cut02:20:282
24671993323,5cyclictest25-21ksoftirqd/100:00:251
24671993312,3cyclictest25-21ksoftirqd/105:05:361
91191320,1ptp4l401ktimersoftd/300:15:263
77782320,2sleep00-21swapper/004:30:150
2468399320,20cyclictest17312-21sed01:35:493
2467599327,22cyclictest11515-21fschecks_time02:45:302
24675993214,15cyclictest10945-21/usr/sbin/munin01:25:412
24675993211,18cyclictest21069-21grep04:45:342
2467599320,2cyclictest0-21swapper/200:05:372
2467599320,29cyclictest15152-21grep00:25:352
2467199321,3cyclictest121rcu_preempt01:00:101
24665993226,3cyclictest13068-21cron01:30:100
2467599311,27cyclictest8747-21sshd00:10:372
2467599311,26cyclictest26808-21tr00:50:262
24675993110,18cyclictest14247-21sed01:30:362
2467599310,2cyclictest0-21swapper/223:55:322
2467599310,2cyclictest0-21swapper/201:04:582
2467199315,3cyclictest121rcu_preempt03:40:491
24671993121,4cyclictest25-21ksoftirqd/100:25:131
2467599300,2cyclictest0-21swapper/201:40:112
2467599300,26cyclictest9828-21chrt00:15:112
2468399291,18cyclictest6347-21ssh03:15:353
2468399291,17cyclictest30421-21snmp_o5powerctl00:55:403
2468399290,14cyclictest10064-21sed01:20:443
2467599299,16cyclictest2301-21phc2sys-jitter03:10:382
2467599291,24cyclictest3898-21cut00:00:362
2467199294,3cyclictest121rcu_preempt03:55:311
24671992919,8cyclictest25-21ksoftirqd/105:05:161
24671992918,9cyclictest25-21ksoftirqd/104:15:301
24671992918,9cyclictest25-21ksoftirqd/104:15:301
24671992916,3cyclictest25-21ksoftirqd/101:45:371
158562290,2sleep00-21swapper/002:50:280
2467599281,23cyclictest31740-21grep23:50:362
24671992822,4cyclictest25-21ksoftirqd/101:05:361
24671992822,4cyclictest25-21ksoftirqd/101:05:361
24671992822,3cyclictest46550irq/101-ens3-Tx04:05:381
24671992822,3cyclictest46550irq/101-ens3-Tx04:05:381
24671992814,4cyclictest25-21ksoftirqd/102:40:441
2467199281,3cyclictest121rcu_preempt03:10:131
2467199280,3cyclictest121rcu_preempt01:50:411
24665992821,4cyclictest212-21systemd-journal00:40:090
2468399270,12cyclictest24886-21awk00:45:313
2467199279,9cyclictest25-21ksoftirqd/100:35:331
2467199274,3cyclictest121rcu_preempt04:50:451
24671992722,3cyclictest25-21ksoftirqd/104:50:151
24671992721,3cyclictest25-21ksoftirqd/103:35:361
24671992721,3cyclictest25-21ksoftirqd/101:20:401
24671992720,4cyclictest25-21ksoftirqd/104:55:361
24671992720,4cyclictest25-21ksoftirqd/100:05:341
24671992716,9cyclictest25-21ksoftirqd/102:05:541
24665992720,4cyclictest23936-21grep03:35:360
24665992716,6cyclictest101ktimersoftd/002:40:290
2466599271,11cyclictest9096-21irqstats02:05:420
91191260,1ptp4l401ktimersoftd/323:45:263
91191260,1ptp4l401ktimersoftd/305:05:203
91191260,1ptp4l401ktimersoftd/302:15:273
91191260,0ptp4l401ktimersoftd/300:25:123
2467199265,4cyclictest25-21ksoftirqd/102:00:291
24671992621,3cyclictest25-21ksoftirqd/104:10:371
24671992621,3cyclictest25-21ksoftirqd/104:10:371
24671992620,3cyclictest25-21ksoftirqd/100:45:431
2467199261,3cyclictest121rcu_preempt23:40:251
24671992613,6cyclictest25-21ksoftirqd/100:45:091
24671992612,6cyclictest25-21ksoftirqd/101:35:361
2467199260,3cyclictest121rcu_preempt04:25:311
91191250,1ptp4l401ktimersoftd/323:40:213
91191250,1ptp4l401ktimersoftd/301:05:313
91191250,1ptp4l401ktimersoftd/301:05:313
91191250,1ptp4l401ktimersoftd/300:30:233
91191250,1ptp4l401ktimersoftd/300:20:273
2467199254,8cyclictest25-21ksoftirqd/123:45:111
2467199253,4cyclictest25-21ksoftirqd/102:50:401
2467199253,3cyclictest25-21ksoftirqd/102:10:341
24671992520,3cyclictest25-21ksoftirqd/103:00:121
24671992520,3cyclictest25-21ksoftirqd/101:55:361
24671992519,3cyclictest25-21ksoftirqd/101:30:101
24671992513,5cyclictest25-21ksoftirqd/103:50:131
2467199250,3cyclictest121rcu_preempt04:40:151
91191240,1ptp4l401ktimersoftd/323:50:253
91191240,1ptp4l401ktimersoftd/304:41:383
91191240,1ptp4l401ktimersoftd/302:40:443
91191240,1ptp4l401ktimersoftd/301:00:103
91191240,1ptp4l401ktimersoftd/300:40:323
91191240,1ptp4l401ktimersoftd/300:10:273
24683992414,7cyclictest28115-21cat00:50:413
24671992419,3cyclictest25-21ksoftirqd/103:20:291
24671992419,3cyclictest25-21ksoftirqd/102:20:431
24671992419,3cyclictest25-21ksoftirqd/102:15:281
24671992419,3cyclictest25-21ksoftirqd/101:15:301
24671992419,3cyclictest25-21ksoftirqd/101:10:381
24671992419,3cyclictest25-21ksoftirqd/100:20:281
24665992418,3cyclictest16546-21cron02:15:110
24665992416,5cyclictest31410-21needreboot23:50:290
24671992318,3cyclictest25-21ksoftirqd/123:50:271
24671992318,3cyclictest25-21ksoftirqd/100:50:261
24671992316,4cyclictest25-21ksoftirqd/100:16:191
24665992316,4cyclictest9-21ksoftirqd/001:07:010
24665992316,4cyclictest9-21ksoftirqd/001:07:010
2466599230,4cyclictest2265-21/usr/sbin/munin03:10:370
2466599230,21cyclictest0-21swapper/001:01:000
24665992216,3cyclictest9-21ksoftirqd/000:14:120
2466599221,5cyclictest19534-21snmp_o5powerctl04:05:460
2466599221,5cyclictest19534-21snmp_o5powerctl04:05:460
24665992215,4cyclictest9-21ksoftirqd/023:43:250
24665992215,4cyclictest2766-21ssh02:35:370
24665992215,4cyclictest2766-21ssh02:35:370
24665992215,4cyclictest25951-21grep02:25:260
24665992214,5cyclictest15008-21snmp_o5powerctl04:00:450
2466599219,3cyclictest22832-21acpi01:45:320
24665992115,3cyclictest9-21ksoftirqd/001:33:050
24665992115,3cyclictest9-21ksoftirqd/001:22:270
24665992115,3cyclictest9-21ksoftirqd/000:15:230
24665992115,3cyclictest28995-21/usr/sbin/munin00:55:230
24665992115,3cyclictest16172-21grep01:35:340
24665992114,4cyclictest9-21ksoftirqd/000:48:220
24665992114,4cyclictest31686-21sh03:44:290
2466599211,3cyclictest30533-21df_inode02:30:310
2466599211,3cyclictest30533-21df_inode02:30:310
24665992113,5cyclictest25600-21latency03:00:350
2466599211,17cyclictest24029-21snmp_o5powerctl04:10:450
2466599211,17cyclictest24029-21snmp_o5powerctl04:10:450
2466599211,17cyclictest23634-21users00:40:440
2466599210,4cyclictest22804-21snmp_o5powerctl02:20:380
2466599210,3cyclictest1724-21threads23:55:430
2466599203,5cyclictest18447-21apt-get01:40:300
24665992014,3cyclictest9-21ksoftirqd/003:16:170
24665992014,3cyclictest9-21ksoftirqd/000:20:580
24665992014,3cyclictest899-21runrttasks03:25:510
24665992014,3cyclictest3916-21sendmail_mailqu00:00:360
24665992014,3cyclictest29283-21sh01:51:040
24665992014,3cyclictest28977-21/usr/sbin/munin03:05:490
24665992013,4cyclictest8006-21ssh03:53:560
24665992013,4cyclictest5074-21apt-config05:05:160
24665992013,4cyclictest32706-21wc03:45:290
24665992013,4cyclictest27246-21which23:45:090
24665992013,4cyclictest2500-21ptp4l05:00:500
24665992013,4cyclictest21859-21awk02:55:440
24665992013,4cyclictest14262-21wc03:25:140
24665992013,4cyclictest10292-21sh04:31:420
24665992012,5cyclictest10279-21grep03:55:400
2466599200,4cyclictest5745-21chrt01:12:190
2466599200,3cyclictest17836-21diskmemload04:59:440
2466599200,18cyclictest0-21swapper/002:19:020
2466599191,4cyclictest26187-21memory04:50:410
24665991913,3cyclictest9-21ksoftirqd/000:07:220
24665991913,3cyclictest17836-21diskmemload04:48:400
24665991913,3cyclictest17836-21diskmemload04:39:400
24665991913,3cyclictest17395-21expr00:30:250
24665991913,3cyclictest12765-21cat02:45:440
24665991912,4cyclictest7241-21sed01:15:360
24665991912,4cyclictest32061-21fschecks_count01:55:350
24665991912,4cyclictest17836-21diskmemload04:15:370
24665991912,4cyclictest17836-21diskmemload04:15:370
24665991912,4cyclictest15263-21grep00:25:360
2466599191,16cyclictest0-21swapper/002:01:180
2466599191,11cyclictest31521-21ssh04:20:290
2466599190,17cyclictest0-21swapper/004:41:280
24665991812,3cyclictest17836-21diskmemload03:30:200
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional