You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-05-20 - 06:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

ARM Broadcom BCM2835 @1400 MHz, Linux 4.18.16-rt9 (Profile)

Latency plot of shadow in rack #7, slot #3
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Fri May 20, 2022 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1443925140,5sleep30-21swapper/319:05:513
76982980,2sleep10-21swapper/122:45:551
141802920,2sleep10-21swapper/120:35:441
108202910,4sleep20-21swapper/222:50:302
15590998743,14cyclictest0-21swapper/022:20:360
1559099866,50cyclictest0-21swapper/019:35:420
1559099866,50cyclictest0-21swapper/019:35:420
309632840,2sleep20-21swapper/200:36:022
1559099846,62cyclictest0-21swapper/000:30:430
1559099846,62cyclictest0-21swapper/000:30:430
1559099770,59cyclictest4789-21dpkg20:10:300
15591997610,21cyclictest0-21swapper/122:50:351
1559099756,5cyclictest0-21swapper/020:30:410
153872750,2sleep20-21swapper/200:15:572
15590997410,45cyclictest17934-21fschecks_time20:45:470
15590997410,45cyclictest17934-21fschecks_time20:45:470
1559099729,37cyclictest22823-21apt-get21:45:400
1559099707,30cyclictest2958-21ssh23:20:420
1559099705,36cyclictest0-21swapper/000:15:430
15590997021,28cyclictest0-21swapper/020:50:410
1559199693,8cyclictest7850irq/56-dwc_otg23:30:391
1559099698,51cyclictest0-21swapper/000:22:330
1559099690,51cyclictest833-21seq23:57:190
128322680,2sleep112831-21ssh21:32:481
1559099675,44cyclictest16137-21uniq20:40:460
1559099673,52cyclictest0-21swapper/019:10:300
1559099659,43cyclictest16965-21sh21:37:460
1559099658,40cyclictest27658-21grep21:50:520
1559099656,23cyclictest0-21swapper/020:00:420
15590996529,20cyclictest0-21swapper/019:20:420
111342650,3sleep231-21ksoftirqd/200:10:502
1559099643,48cyclictest6750-21/usr/sbin/munin20:15:500
1559099638,37cyclictest5584-21ls20:10:500
15590996321,9cyclictest0-21swapper/021:00:410
1559099628,40cyclictest0-21swapper/022:44:430
1559099625,43cyclictest9829-21ssh00:09:510
1559099624,44cyclictest0-21swapper/021:10:300
1559099624,26cyclictest0-21swapper/000:10:390
1559099621,36cyclictest0-21swapper/022:55:400
152422620,2sleep215239-21ssh23:35:432
152422620,2sleep215239-21ssh23:35:432
15590996142,8cyclictest0-21swapper/019:25:550
15590996129,6cyclictest0-21swapper/000:35:410
15590996110,46cyclictest25360-21ssh00:30:050
1559099609,41cyclictest0-21swapper/022:06:340
15590996029,5cyclictest0-21swapper/019:50:430
15590996019,20cyclictest15991-21sh22:15:570
15590996019,20cyclictest15991-21sh22:15:570
15590996010,19cyclictest8050irq/56-dwc_otg_22:10:360
1559099599,28cyclictest23250-21seq23:45:440
1559099592,46cyclictest0-21swapper/022:37:120
1559099592,42cyclictest0-21swapper/022:29:400
1559099591,39cyclictest19452-21fschecks_count23:00:470
240332580,2sleep30-21swapper/321:45:583
1559099587,35cyclictest0-21swapper/020:21:190
1559099587,29cyclictest18930-21aten_rbpower_po21:40:500
1559099585,45cyclictest0-21swapper/019:15:380
1559099584,21cyclictest0-21swapper/021:55:410
1559099583,40cyclictest28023-21unixbench_singl21:10:590
1559099582,40cyclictest0-21swapper/021:21:330
1559099581,48cyclictest6970-21grep22:45:440
1559299573,5cyclictest7850irq/56-dwc_otg22:10:422
1559099576,42cyclictest25200-21ssh23:07:140
1559099575,32cyclictest26396-21diskmemload00:02:130
1559099571,38cyclictest0-21swapper/019:55:470
1559099568,39cyclictest0-21swapper/021:17:210
1559099565,37cyclictest0-21swapper/023:12:380
1559099563,44cyclictest18461-21ssh23:39:480
1559099563,44cyclictest18461-21ssh23:39:480
71802550,21sleep01559099cyclictest21:25:500
1559299555,8cyclictest7850irq/56-dwc_otg21:30:422
1559099554,3cyclictest0-21swapper/023:30:420
15590995513,38cyclictest101ktimersoftd/023:50:520
1559099551,31cyclictest27591-21grep19:40:570
1559099549,34cyclictest29124-21seq22:32:090
1559099547,29cyclictest11646-21taskset20:26:430
1559099543,35cyclictest886-21ssh23:17:200
1559099540,40cyclictest0-21swapper/019:47:040
151322540,2sleep10-21swapper/123:35:421
151322540,2sleep10-21swapper/123:35:421
1559099539,30cyclictest22482-21ssh23:44:470
1559099537,35cyclictest0-21swapper/022:52:200
1559099537,24cyclictest0-21swapper/021:30:520
1559299525,18cyclictest7950irq/56-dwc_otg_22:20:362
15592995216,27cyclictest8050irq/56-dwc_otg_22:55:402
1559099522,38cyclictest5503-21sh22:04:000
1559099520,41cyclictest7009-21aten_rbpower_cu23:25:420
1559299516,27cyclictest22825-21ssh21:45:402
1559299514,8cyclictest7850irq/56-dwc_otg23:50:412
1559099517,30cyclictest0-21swapper/019:31:220
1559099514,32cyclictest0-21swapper/020:55:430
1559199493,13cyclictest7850irq/56-dwc_otg00:15:401
1559399488,25cyclictest0-21swapper/321:00:423
1559199487,27cyclictest0-21swapper/121:00:421
1559199477,25cyclictest0-21swapper/120:15:421
1559099478,29cyclictest0-21swapper/020:38:030
201302460,14sleep229-21rcuc/223:40:552
1559299462,7cyclictest0-21swapper/200:25:482
1559199467,25cyclictest0-21swapper/119:45:431
1559199467,23cyclictest0-21swapper/123:25:421
1559199462,3cyclictest0-21swapper/121:35:431
152832463,35sleep00-21swapper/019:10:020
19562450,4sleep231-21ksoftirqd/221:20:172
1559399457,24cyclictest0-21swapper/319:45:433
1559399455,4cyclictest0-21swapper/322:15:433
1559399455,4cyclictest0-21swapper/322:15:433
1559299447,21cyclictest8050irq/56-dwc_otg_00:05:412
1559199447,27cyclictest0-21swapper/121:40:491
1559199447,25cyclictest0-21swapper/121:45:431
15591994420,5cyclictest23556-21fschecks_time23:05:461
6042430,1sleep10-21swapper/122:36:571
1559199437,26cyclictest0-21swapper/120:40:421
1559199437,24cyclictest0-21swapper/120:50:431
1559199437,22cyclictest0-21swapper/123:40:421
1559399424,23cyclictest2865-21ssh00:00:423
1559299423,9cyclictest6478-21ssh21:25:402
1559399411,24cyclictest0-21swapper/320:40:423
1559299417,25cyclictest0-21swapper/221:05:432
1559299416,20cyclictest7950irq/56-dwc_otg_22:25:442
1559199416,25cyclictest0-21swapper/122:10:351
1559199412,12cyclictest0-21swapper/119:40:411
15593994010,5cyclictest7850irq/56-dwc_otg21:35:303
1559299404,6cyclictest0-21swapper/200:00:412
1559299403,12cyclictest0-21swapper/200:30:432
1559299403,12cyclictest0-21swapper/200:30:432
15591994021,4cyclictest675950wget00:05:441
48362390,1sleep10-21swapper/122:42:161
256332391,30sleep10-21swapper/121:05:561
1559299399,8cyclictest0-21swapper/221:40:462
1559299394,5cyclictest8050irq/56-dwc_otg_21:50:402
1559199399,6cyclictest2505-21cron00:00:291
1559199395,22cyclictest6504-21ssh21:25:421
1559199394,6cyclictest7850irq/56-dwc_otg19:20:521
1559199393,26cyclictest0-21swapper/122:00:421
15591993920,5cyclictest6354-21sh00:05:291
221742380,1sleep10-21swapper/122:24:211
15593993819,5cyclictest7950irq/56-dwc_otg_23:20:413
1559299383,11cyclictest7950irq/56-dwc_otg_23:56:542
1559199389,8cyclictest7850irq/56-dwc_otg21:20:421
1559199383,22cyclictest24325-21timerwakeupswit23:45:581
1559199381,27cyclictest10673-21cut20:25:461
1559199381,18cyclictest24561-21latency_hist19:35:301
1559399372,4cyclictest0-21swapper/323:35:413
1559399372,4cyclictest0-21swapper/323:35:413
1559299374,6cyclictest7950irq/56-dwc_otg_19:50:582
15592993718,12cyclictest29-21rcuc/219:55:422
1559199378,17cyclictest7850irq/56-dwc_otg22:20:301
1559199378,17cyclictest7850irq/56-dwc_otg22:20:301
1559199371,25cyclictest0-21swapper/119:15:431
1559199371,24cyclictest2233-21ls20:00:561
89742360,1sleep30-21swapper/322:47:253
1559399369,8cyclictest25969-21unixbench_singl21:05:583
1559399363,3cyclictest0-21swapper/319:15:423
15593993625,3cyclictest0-21swapper/323:50:573
1559399361,16cyclictest31459-21irqstats21:55:493
1559299368,15cyclictest8050irq/56-dwc_otg_21:12:102
1559299367,24cyclictest0-21swapper/220:50:432
1559299365,20cyclictest14636-21ssh21:35:362
15592993616,5cyclictest24788-21sh23:06:282
15592993615,14cyclictest31-21ksoftirqd/219:20:572
1559299361,23cyclictest7734-21timerwakeupswit22:05:572
1559199368,15cyclictest7850irq/56-dwc_otg21:55:511
15591993616,10cyclictest10466-21latency_hist22:10:301
15591993614,5cyclictest26363-21latency00:30:501
15591993614,5cyclictest26363-21latency00:30:501
1559399359,4cyclictest0-21swapper/300:25:253
1559399359,14cyclictest8050irq/56-dwc_otg_22:35:573
1559399352,20cyclictest0-21swapper/320:53:173
15593993514,5cyclictest0-21swapper/323:10:393
1559399351,30cyclictest26928-21ls22:30:303
1559299357,24cyclictest0-21swapper/220:55:422
1559299357,24cyclictest0-21swapper/220:10:422
1559299354,21cyclictest7950irq/56-dwc_otg_23:50:302
1559299351,24cyclictest0-21swapper/220:15:422
1559199354,6cyclictest7850irq/56-dwc_otg00:40:301
15591993515,6cyclictest30132-21sh23:53:551
15591993513,4cyclictest26368-1kworker/u9:023:20:561
15591993512,11cyclictest31809-21cat23:15:471
1559399346,20cyclictest19167-21ssh23:00:433
15593993415,17cyclictest1448450wget00:15:443
15593993414,13cyclictest11100-21ssh22:10:463
1559299348,4cyclictest8050irq/56-dwc_otg_23:20:432
1559299348,17cyclictest8050irq/56-dwc_otg_22:50:412
1559299347,23cyclictest0-21swapper/220:05:432
1559299345,23cyclictest31323-21ssh23:15:412
1559299345,22cyclictest29-21rcuc/221:20:522
1559199349,11cyclictest2128-21latency_hist21:20:301
1559199341,7cyclictest0-21swapper/100:25:471
15591993415,6cyclictest30451-21latency_hist21:15:301
1559399337,5cyclictest22510-21basename21:45:293
1559399336,13cyclictest8050irq/56-dwc_otg_21:50:443
1559399332,7cyclictest14058-21sh00:15:303
1559399332,4cyclictest0-21swapper/300:38:073
1559399332,25cyclictest8050irq/56-dwc_otg_20:20:413
15593993312,5cyclictest125-21systemd-journal20:26:063
1559399331,20cyclictest1738-21ls20:00:483
15593993310,12cyclictest0-21swapper/321:14:523
1559299338,14cyclictest301ktimersoftd/220:32:162
1559299336,6cyclictest7850irq/56-dwc_otg23:10:452
1559299333,9cyclictest8050irq/56-dwc_otg_23:30:412
1559299333,4cyclictest7850irq/56-dwc_otg22:00:562
1559299332,7cyclictest32423-21tail22:35:572
1559299332,14cyclictest7950irq/56-dwc_otg_22:15:562
1559299332,14cyclictest7950irq/56-dwc_otg_22:15:562
1559199336,5cyclictest7850irq/56-dwc_otg00:10:441
15591993315,13cyclictest0-21swapper/120:05:431
202392320,8sleep21559299cyclictest23:00:572
1559399329,11cyclictest8050irq/56-dwc_otg_22:40:433
1559399325,13cyclictest7950irq/56-dwc_otg_23:32:053
1559399325,12cyclictest7950irq/56-dwc_otg_23:06:373
1559399324,18cyclictest0-21swapper/323:42:243
1559399323,3cyclictest0-21swapper/321:27:563
15593993226,4cyclictest1008-21runrttasks19:40:503
15593993224,5cyclictest16744-21grep19:10:573
1559399322,19cyclictest0-21swapper/319:39:273
1559399322,19cyclictest0-21swapper/319:39:273
1559399322,13cyclictest7950irq/56-dwc_otg_22:05:513
1559399321,7cyclictest7445-21if_eth023:25:473
1559399321,7cyclictest22200-21aten_rbpower_vo00:25:513
15593993212,5cyclictest32299-21if_eth019:55:483
15593993212,13cyclictest16481-21kworker/3:221:39:323
1559299327,14cyclictest29-21rcuc/219:25:502
1559299325,5cyclictest7950irq/56-dwc_otg_19:35:422
1559299325,5cyclictest7950irq/56-dwc_otg_19:35:422
1559299322,14cyclictest8050irq/56-dwc_otg_20:20:422
1559299322,14cyclictest7950irq/56-dwc_otg_19:10:472
15592993210,12cyclictest301ktimersoftd/219:15:522
1559199323,17cyclictest31668-21ls19:55:291
1559199322,24cyclictest8050irq/56-dwc_otg_22:25:431
15591993215,6cyclictest18209-21ssh00:20:401
1559199321,4cyclictest7850irq/56-dwc_otg22:30:421
1559399317,14cyclictest0-21swapper/322:20:353
1559399315,6cyclictest8050irq/56-dwc_otg_23:15:433
1559399315,13cyclictest8050irq/56-dwc_otg_00:35:303
1559399315,13cyclictest8050irq/56-dwc_otg_00:35:303
1559399314,5cyclictest7850irq/56-dwc_otg22:55:403
1559399313,5cyclictest8050irq/56-dwc_otg_22:50:573
1559399311,6cyclictest2797150ssh22:30:543
1559399311,5cyclictest22852-21apt-get19:30:423
1559399311,4cyclictest7950irq/56-dwc_otg_19:50:403
1559299319,3cyclictest0-21swapper/223:25:582
1559299314,5cyclictest7950irq/56-dwc_otg_20:45:472
1559299314,5cyclictest7950irq/56-dwc_otg_20:45:472
1559299314,13cyclictest8050irq/56-dwc_otg_00:25:302
1559299312,22cyclictest8050irq/56-dwc_otg_19:30:412
1559299312,13cyclictest0-21swapper/219:40:432
15592993119,4cyclictest7950irq/56-dwc_otg_22:40:572
1559199319,10cyclictest0-21swapper/123:02:541
1559199316,14cyclictest0-21swapper/120:10:361
1559199314,13cyclictest26362-21latency_hist19:40:301
1559199314,13cyclictest26362-21latency_hist19:40:301
1559199313,19cyclictest28110-21grep21:50:581
1559199311,6cyclictest7950irq/56-dwc_otg_19:25:571
15591993116,4cyclictest31042-21latency_hist23:15:291
15591993112,5cyclictest21963-21/usr/sbin/munin20:55:531
15591993111,6cyclictest16358-21unixbench_singl22:55:581
1559399309,5cyclictest7950irq/56-dwc_otg_20:35:493
1559399303,15cyclictest13112-21switchtime20:30:553
1559399302,9cyclictest31965-21unixbench-2d21:15:583
1559399301,6cyclictest39-21ksoftirqd/322:00:443
1559399301,15cyclictest7950irq/56-dwc_otg_00:05:553
15593993010,4cyclictest0-21swapper/321:21:243
1559299306,13cyclictest31-21ksoftirqd/220:35:402
1559299306,11cyclictest8050irq/56-dwc_otg_20:25:572
1559299304,7cyclictest7850irq/56-dwc_otg20:40:502
1559299302,13cyclictest7950irq/56-dwc_otg_21:56:302
15592993011,9cyclictest301ktimersoftd/220:05:302
1559199308,4cyclictest7850irq/56-dwc_otg19:10:501
1559399294,5cyclictest8050irq/56-dwc_otg_20:10:463
1559399293,7cyclictest39-21ksoftirqd/320:15:403
15593992917,5cyclictest3039-21/usr/sbin/munin20:05:573
15593992910,11cyclictest1118-21rm23:57:503
1559299294,4cyclictest7850irq/56-dwc_otg22:30:562
1559299293,12cyclictest7950irq/56-dwc_otg_19:45:442
1559299291,17cyclictest31-21ksoftirqd/221:05:302
152892297,14sleep10-21swapper/119:10:071
1559399288,3cyclictest0-21swapper/320:55:503
1559399283,6cyclictest0-21swapper/323:46:393
1559199282,6cyclictest8571-21/usr/sbin/munin20:20:391
1559199282,6cyclictest8050irq/56-dwc_otg_19:55:471
1559399273,12cyclictest8050irq/56-dwc_otg_19:25:423
1559399271,6cyclictest8050irq/56-dwc_otg_20:50:303
1559399271,6cyclictest8050irq/56-dwc_otg_20:50:303
1559199272,5cyclictest0-21swapper/120:30:421
1559199264,6cyclictest0-21swapper/120:46:541
1559199264,6cyclictest0-21swapper/120:46:541
150652263,14sleep20-21swapper/219:07:132
1559399253,4cyclictest8050irq/56-dwc_otg_19:25:303
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional