You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-08-08 - 18:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

ARM Broadcom BCM2835 @1400 MHz, Linux 4.18.16-rt9 (Profile)

Latency plot of shadow in rack #7, slot #3
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rack7slot3s.osadl.org (updated Mon Aug 08, 2022 12:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
292202970,2sleep30-21swapper/311:29:323
19682850,3sleep30-21swapper/310:14:423
171062810,4sleep1459999cyclictest09:52:121
4598998027,15cyclictest0-21swapper/008:00:270
459899797,57cyclictest11-21rcu_preempt08:15:220
459899797,57cyclictest11-21rcu_preempt08:15:220
4598997910,31cyclictest0-21swapper/011:25:240
4600997610,14cyclictest8150irq/56-dwc_otg_11:55:262
4598997611,15cyclictest0-21swapper/010:35:290
181132760,6sleep1459999cyclictest09:12:461
4598997528,12cyclictest1944-21runrttasks10:55:260
4598997431,7cyclictest0-21swapper/010:25:290
459899733,29cyclictest0-21swapper/008:20:230
460099722,6cyclictest25041-21cut12:05:122
460099722,6cyclictest25041-21cut12:05:122
459899721,41cyclictest0-21swapper/010:15:290
460199695,5cyclictest10631-21ssh11:05:383
460199681,21cyclictest10302-21ssh10:25:113
459899684,11cyclictest13-21rcuc/012:15:250
460099673,5cyclictest8150irq/56-dwc_otg_09:15:102
4600996719,8cyclictest17779-21latency_hist11:15:122
40572663,11sleep20-21swapper/207:06:382
459899643,27cyclictest8078-21http08:50:110
4601996314,11cyclictest5901-21sh11:00:243
460099639,13cyclictest0-21swapper/209:17:352
460099631,24cyclictest10304-21ls10:25:112
459899639,34cyclictest9874-21unixbench_singl12:25:420
459899637,18cyclictest0-21swapper/012:30:260
4598996334,15cyclictest0-21swapper/012:39:100
459899621,23cyclictest0-21swapper/007:51:030
460199619,46cyclictest8150irq/56-dwc_otg_12:10:273
4598996137,15cyclictest23313-21latency_hist09:20:110
4598996118,21cyclictest24613-21grep08:05:320
459899611,33cyclictest7574-21tail07:15:570
459899604,13cyclictest0-21swapper/008:40:260
459899603,54cyclictest13-21rcuc/011:15:110
459899603,36cyclictest12569-21ls09:00:340
459899603,36cyclictest12569-21ls09:00:340
4598996032,6cyclictest30969-21/usr/sbin/munin10:50:370
459899602,33cyclictest8365-21head09:40:460
4598996021,3cyclictest0-21swapper/007:10:230
459899591,14cyclictest27317-21ls10:05:350
210722590,1sleep10-21swapper/111:19:111
459899589,45cyclictest23175-21aten_rbpower_cu10:00:310
459899586,30cyclictest0-21swapper/010:40:280
459899586,25cyclictest0-21swapper/007:30:250
4598995812,3cyclictest0-21swapper/007:25:560
4598995730,8cyclictest26492-21unixbench-2d12:05:470
459899571,34cyclictest2182-21unixbench_multi08:30:430
459899562,28cyclictest0-21swapper/009:30:300
4598995610,7cyclictest0-21swapper/010:45:260
4598995527,14cyclictest147-21systemd-journal10:23:430
4598995525,8cyclictest17603-21fschecks_time07:45:320
459899552,31cyclictest0-21swapper/011:10:110
4598995522,18cyclictest0-21swapper/011:25:120
459899549,34cyclictest0-21swapper/010:35:120
459899549,34cyclictest0-21swapper/010:35:120
459899545,18cyclictest8150irq/56-dwc_otg_11:15:170
459899544,27cyclictest8644-21grep08:50:300
4598995438,7cyclictest8050irq/56-dwc_otg_12:10:430
459899543,48cyclictest14511-21/usr/sbin/munin09:05:390
459899543,28cyclictest9-21ksoftirqd/012:00:110
4598995427,4cyclictest8150irq/56-dwc_otg_09:50:100
459899541,31cyclictest22583-21grep12:00:450
459899541,31cyclictest22583-21grep12:00:450
460199537,26cyclictest0-21swapper/308:55:253
460099533,7cyclictest8150irq/56-dwc_otg_11:35:282
459899539,19cyclictest0-21swapper/007:25:100
4598995326,12cyclictest8150irq/56-dwc_otg_08:25:400
459899531,24cyclictest0-21swapper/009:25:360
4598995312,13cyclictest0-21swapper/011:00:250
459899526,24cyclictest19014-21/usr/sbin/munin09:55:340
459899526,24cyclictest19014-21/usr/sbin/munin09:55:340
459899525,29cyclictest0-21swapper/009:50:300
4598995233,3cyclictest0-21swapper/008:40:110
459899523,32cyclictest0-21swapper/007:40:400
459899521,37cyclictest16967-21ssh11:55:110
459899521,29cyclictest31361-21/usr/sbin/munin10:10:360
459899521,25cyclictest0-21swapper/008:55:340
459899521,20cyclictest0-21swapper/008:00:110
82622510,2sleep00-21swapper/012:25:110
82622510,2sleep00-21swapper/012:25:110
459999514,6cyclictest2208150ssh11:20:301
4598995123,4cyclictest8050irq/56-dwc_otg_11:40:330
4598995116,18cyclictest0-21swapper/009:10:300
459899511,37cyclictest0-21swapper/011:47:500
4598995110,4cyclictest27310-21timerwakeupswit08:10:420
4598995014,29cyclictest8150irq/56-dwc_otg_11:30:260
459899501,33cyclictest0-21swapper/009:35:310
459899491,34cyclictest147-21systemd-journal09:23:320
27892493,36sleep00-21swapper/007:05:270
460099487,28cyclictest0-21swapper/211:15:282
459999485,29cyclictest7950irq/56-dwc_otg08:20:211
459899485,17cyclictest4423-21ssh11:38:580
4598994820,14cyclictest1-21systemd07:37:220
460099463,30cyclictest9963-21ls11:45:352
460099461,29cyclictest0-21swapper/207:20:282
460199457,26cyclictest10583-21ssh10:25:293
460199457,24cyclictest0-21swapper/311:35:303
460099454,33cyclictest8050irq/56-dwc_otg_11:55:112
460199447,24cyclictest0-21swapper/311:15:283
4601994414,5cyclictest1388-21fschecks_time08:30:313
460099444,3cyclictest8050irq/56-dwc_otg_08:45:252
460199437,25cyclictest0-21swapper/309:00:273
460199437,25cyclictest0-21swapper/309:00:273
4601994315,17cyclictest0-21swapper/310:15:463
460099437,24cyclictest0-21swapper/212:15:282
460099436,16cyclictest8050irq/56-dwc_otg_09:20:452
4600994319,6cyclictest23457-21processes08:00:412
4599994314,6cyclictest23-21ksoftirqd/112:05:111
4599994314,6cyclictest23-21ksoftirqd/112:05:111
460199427,25cyclictest0-21swapper/311:50:293
4601994218,5cyclictest2338-21sh12:16:153
460199421,5cyclictest8150irq/56-dwc_otg_11:45:263
460099427,5cyclictest8050irq/56-dwc_otg_11:05:102
460099426,23cyclictest0-21swapper/210:25:292
4600994215,6cyclictest19344-21fschecks_count09:55:352
4600994215,6cyclictest19344-21fschecks_count09:55:352
4600994212,13cyclictest25468-21cpu12:05:332
459999422,33cyclictest0-21swapper/111:02:551
306122421,5sleep130620-21sh12:10:511
460199418,23cyclictest0-21swapper/312:30:283
460199411,25cyclictest0-21swapper/307:20:283
459999415,23cyclictest0-21swapper/107:25:251
459999414,4cyclictest0-21swapper/112:06:161
459999414,24cyclictest8150irq/56-dwc_otg_08:10:281
4599994123,11cyclictest4870-21sh11:39:511
459999412,22cyclictest8050irq/56-dwc_otg_08:10:111
4599994114,5cyclictest0-21swapper/107:50:491
242392410,2sleep2301ktimersoftd/210:00:472
460199408,23cyclictest8150irq/56-dwc_otg_11:40:263
460199404,5cyclictest8150irq/56-dwc_otg_09:55:293
460199404,5cyclictest8150irq/56-dwc_otg_09:55:293
4601994016,4cyclictest29695-21latency_hist08:20:113
4601994016,4cyclictest29695-21latency_hist08:20:113
460099403,25cyclictest25566-21cron11:25:112
4600994018,5cyclictest28957-21/usr/sbin/munin09:25:462
4599994019,4cyclictest4578-21sh12:20:141
4599994019,4cyclictest4578-21sh12:20:141
460199399,10cyclictest15240-21chrt07:40:113
460199397,25cyclictest8150irq/56-dwc_otg_07:40:233
460199396,16cyclictest8150irq/56-dwc_otg_07:55:103
460199393,32cyclictest8150irq/56-dwc_otg_09:10:273
4601993918,10cyclictest0-21swapper/309:35:573
460099399,13cyclictest1008950wget07:25:262
460099393,9cyclictest8050irq/56-dwc_otg_12:25:202
4600993911,16cyclictest0-21swapper/212:30:172
459999397,22cyclictest0-21swapper/111:10:281
459999393,10cyclictest11304-21df_inode09:45:311
4599993919,9cyclictest8078-21switchtime09:40:431
4599993916,4cyclictest30706-21entropy10:50:341
459999391,28cyclictest17450-21ssh12:35:431
460199389,26cyclictest0-21swapper/307:25:253
460199384,5cyclictest0-21swapper/311:59:543
4601993819,4cyclictest27777-21sh10:05:453
4601993818,5cyclictest21671-21sh10:39:223
4601993815,5cyclictest10015-21sh12:25:573
460099387,21cyclictest0-21swapper/208:55:252
4600993810,4cyclictest0-21swapper/210:45:102
459999387,13cyclictest17336-21ssh11:55:281
459999383,19cyclictest0-21swapper/111:34:081
459999383,18cyclictest7009-21latency08:45:341
4599993815,10cyclictest23-21ksoftirqd/110:25:461
459999381,21cyclictest0-21swapper/108:35:271
4599993812,12cyclictest0-21swapper/111:25:241
4599993811,8cyclictest27763-21grep10:45:461
460199377,16cyclictest0-21swapper/309:25:293
460199376,19cyclictest8150irq/56-dwc_otg_10:45:323
4601993718,5cyclictest1907-21/usr/sbin/munin10:55:363
460199371,5cyclictest7950irq/56-dwc_otg10:50:263
460199371,25cyclictest0-21swapper/307:30:323
460199371,24cyclictest0-21swapper/308:00:433
4600993722,8cyclictest8150irq/56-dwc_otg_11:25:262
4600993720,4cyclictest31195-21ntp_kernel_err10:50:402
4600993718,6cyclictest27278-21timerwakeupswit08:10:422
460099371,7cyclictest2861-21grep09:35:112
4600993717,13cyclictest6731-21latency_hist09:40:112
4600993717,11cyclictest6208-21ssh11:40:402
460099371,15cyclictest27252-21ssh10:45:392
459999375,18cyclictest23-21ksoftirqd/110:15:441
459999374,13cyclictest12520-21ssh12:30:261
459999373,7cyclictest8150irq/56-dwc_otg_09:35:281
459999373,3cyclictest7950irq/56-dwc_otg09:20:261
459999373,11cyclictest8050irq/56-dwc_otg_08:30:401
459999372,6cyclictest5404-21/usr/sbin/munin11:40:381
4599993714,5cyclictest24276-21ssh10:41:391
4599993714,12cyclictest23-21ksoftirqd/111:50:101
145962371,30sleep214664-21ls07:35:562
460199363,6cyclictest8050irq/56-dwc_otg_12:00:303
460199363,6cyclictest8050irq/56-dwc_otg_12:00:303
460199362,6cyclictest8050irq/56-dwc_otg_09:50:323
460199361,8cyclictest25531-21ssh12:05:343
4601993616,4cyclictest252482sleep310:43:343
460099369,8cyclictest31-21ksoftirqd/209:40:382
460099369,13cyclictest8150irq/56-dwc_otg_10:30:332
460099369,13cyclictest8150irq/56-dwc_otg_10:30:332
460099367,18cyclictest27161-21aten_rbpower_cu10:05:332
460099363,26cyclictest8150irq/56-dwc_otg_07:50:242
460099362,10cyclictest8050irq/56-dwc_otg_08:40:442
4600993616,5cyclictest1141-21runrttasks07:17:262
460099361,5cyclictest7950irq/56-dwc_otg12:35:302
460099361,24cyclictest0-21swapper/211:06:112
459999369,14cyclictest0-21swapper/109:59:311
459999369,14cyclictest0-21swapper/109:59:311
459999366,4cyclictest8050irq/56-dwc_otg_10:38:141
459999365,23cyclictest8150irq/56-dwc_otg_08:55:251
4599993629,4cyclictest23127-21apt-get10:00:311
4599993621,3cyclictest8150irq/56-dwc_otg_10:31:271
4599993621,3cyclictest8150irq/56-dwc_otg_10:31:271
4599993616,9cyclictest15382-21diskmemload11:50:501
4599993615,5cyclictest31821-21taskset09:30:311
460199351,9cyclictest0-21swapper/309:47:473
4601993519,5cyclictest9432-21sh09:42:453
4601993517,4cyclictest17444-21/usr/sbin/munin12:35:433
4601993515,12cyclictest25728-21sh09:22:133
460199351,14cyclictest0-21swapper/310:00:463
460099357,23cyclictest0-21swapper/208:50:162
460099353,6cyclictest18455-21grep07:45:502
460099353,5cyclictest8150irq/56-dwc_otg_09:50:322
460099351,18cyclictest0-21swapper/211:30:412
4600993511,13cyclictest0-21swapper/210:14:132
4600993510,5cyclictest0-21swapper/212:21:232
4600993510,5cyclictest0-21swapper/212:21:232
459999353,5cyclictest8050irq/56-dwc_otg_09:00:311
459999353,5cyclictest8050irq/56-dwc_otg_09:00:311
4599993512,5cyclictest0-21swapper/110:05:341
4599993512,4cyclictest11295-21seq11:05:531
75072340,5sleep3460199cyclictest12:23:413
75072340,5sleep3460199cyclictest12:23:413
460199347,21cyclictest0-21swapper/308:50:163
460199346,4cyclictest0-21swapper/311:12:403
460199342,14cyclictest8150irq/56-dwc_otg_08:05:473
460199341,4cyclictest0-21swapper/308:00:003
460199341,16cyclictest30715-21ssh11:30:403
460199341,16cyclictest0-21swapper/309:30:403
460099349,6cyclictest8050irq/56-dwc_otg_12:10:432
460099345,6cyclictest2171-21/usr/sbin/munin10:55:302
460099344,5cyclictest7950irq/56-dwc_otg08:20:252
460099342,6cyclictest18886-21fschecks_time10:35:372
460099341,7cyclictest29155-21grep08:15:392
460099341,7cyclictest29155-21grep08:15:392
460099341,6cyclictest12034-21sort09:45:432
4600993414,6cyclictest147-21systemd-journal08:28:212
460099341,3cyclictest0-21swapper/210:20:112
459999345,16cyclictest23-21ksoftirqd/108:00:411
459999341,9cyclictest1804-21ssh10:55:121
4599993417,5cyclictest31207-21aten_rbpower_vo10:10:331
4599993414,5cyclictest6805-21cat10:20:321
4599993410,7cyclictest22124-21runrttasks09:17:551
460199334,4cyclictest8150irq/56-dwc_otg_08:45:113
460199333,23cyclictest8150irq/56-dwc_otg_09:05:273
460199331,8cyclictest23201-21grep11:20:473
460199331,7cyclictest19765-21ssh09:15:283
460099331,6cyclictest12004-21/usr/sbin/munin09:00:402
460099331,6cyclictest12004-21/usr/sbin/munin09:00:402
460099331,5cyclictest4448-21idleruntime-cro08:40:102
460099331,13cyclictest8150irq/56-dwc_otg_07:40:362
459999334,6cyclictest7950irq/56-dwc_otg12:15:321
459999334,16cyclictest7950irq/56-dwc_otg07:55:241
459999333,4cyclictest0-21swapper/108:50:241
4599993312,6cyclictest0-21swapper/108:15:221
4599993312,6cyclictest0-21swapper/108:15:221
177532330,6sleep1221ktimersoftd/107:45:361
53432320,5sleep1459999cyclictest07:10:451
460199327,14cyclictest8150irq/56-dwc_otg_07:10:453
460199327,13cyclictest8150irq/56-dwc_otg_08:45:343
460199322,5cyclictest14588-21taskset10:30:323
460199322,5cyclictest14588-21taskset10:30:323
460199321,5cyclictest4432-21cron08:40:103
4601993213,4cyclictest17916-21netstat07:45:393
460099322,5cyclictest0-21swapper/208:30:312
459999329,16cyclictest8150irq/56-dwc_otg_07:20:261
459999325,12cyclictest8150irq/56-dwc_otg_07:30:281
459999322,7cyclictest5715-21timerwakeupswit08:40:431
4599993211,6cyclictest8150irq/56-dwc_otg_09:05:331
460199319,4cyclictest32033-21/usr/sbin/munin08:25:293
460199312,5cyclictest30027-21aten_rbpower_vo08:20:253
460099315,15cyclictest8150irq/56-dwc_otg_07:10:342
460099312,6cyclictest0-21swapper/208:07:242
4600993112,5cyclictest14949-21taskset09:05:452
459999318,14cyclictest23-21ksoftirqd/108:25:511
459999313,6cyclictest8050irq/56-dwc_otg_07:35:331
459999313,3cyclictest7950irq/56-dwc_otg07:15:251
459999311,4cyclictest0-21swapper/112:25:301
4599993110,11cyclictest0-21swapper/109:25:431
460199306,13cyclictest8150irq/56-dwc_otg_08:10:363
460199302,8cyclictest0-21swapper/307:15:283
459999303,6cyclictest15487-21cut07:40:271
460099291,3cyclictest8050irq/56-dwc_otg_07:30:232
460099291,3cyclictest7950irq/56-dwc_otg07:55:372
27922273,13sleep30-21swapper/307:05:303
27852274,12sleep10-21swapper/107:05:241
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional