You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-08-13 - 02:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

ARM Broadcom BCM2835 @1400 MHz, Linux 4.18.16-rt9 (Profile)

Latency plot of shadow in rack #7, slot #3
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Fri Aug 12, 2022 12:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3041922800,2sleep00-21swapper/007:05:270
2141521830,4sleep13217299cyclictest11:30:391
32171991132,64cyclictest0-21swapper/011:10:320
2749821080,3sleep2301ktimersoftd/212:19:282
3217199902,60cyclictest5660-21chrt12:31:550
3217199892,65cyclictest0-21swapper/012:21:480
3217199891,66cyclictest7610-21fschecks_time09:00:310
3217199881,59cyclictest26334-21/usr/sbin/munin09:30:340
32171998615,49cyclictest0-21swapper/007:48:110
32171998615,49cyclictest0-21swapper/007:48:110
32171998543,21cyclictest8150irq/56-dwc_otg_10:35:250
32171998517,50cyclictest20186-21ssh11:29:070
32171998423,47cyclictest7101-21/usr/sbin/munin10:30:270
3217199841,73cyclictest0-21swapper/008:41:220
214492840,3sleep0101ktimersoftd/010:48:570
32171998145,25cyclictest7950irq/56-dwc_otg11:00:520
3217199804,33cyclictest0-21swapper/008:15:240
255802800,2sleep20-21swapper/210:54:272
32171997923,36cyclictest8150irq/56-dwc_otg_07:50:310
32171997923,36cyclictest17563-21sshd07:58:250
32171997920,50cyclictest0-21swapper/010:25:420
3217199791,58cyclictest29366-21taskset08:30:330
32171997821,36cyclictest12144-21cron07:45:130
3217199778,56cyclictest0-21swapper/011:46:250
3217199778,56cyclictest0-21swapper/011:46:250
3217199776,46cyclictest0-21swapper/012:35:380
32171997718,35cyclictest0-21swapper/011:51:420
3217199771,57cyclictest29410-21runrttasks10:59:250
3217199762,15cyclictest0-21swapper/010:15:430
32171997617,34cyclictest15496-21sh12:04:210
32171997518,47cyclictest0-21swapper/010:20:400
32171997516,43cyclictest23893-21ssh10:09:450
32171997410,4cyclictest16469-21runrttasks10:00:130
317872740,1sleep20-21swapper/211:01:242
3217199729,49cyclictest24794-21sed11:35:130
3217199727,52cyclictest10669-21diskmemload12:13:400
32171997221,36cyclictest0-21swapper/010:52:170
32171997216,33cyclictest19960-21latency_hist12:10:130
32171997216,33cyclictest19960-21latency_hist12:10:130
32171997212,46cyclictest0-21swapper/009:49:120
32171997212,33cyclictest6324-21unixbench_singl07:25:420
3217199721,15cyclictest352-21latency12:25:380
3217199721,15cyclictest352-21latency12:25:380
3217399716,12cyclictest0-21swapper/209:15:322
3217199715,46cyclictest10669-21diskmemload11:55:460
32171997133,7cyclictest8050irq/56-dwc_otg_09:10:240
32171997117,40cyclictest0-21swapper/012:19:180
3217199711,50cyclictest0-21swapper/009:36:020
3217199711,50cyclictest0-21swapper/009:36:020
3217199711,16cyclictest0-21swapper/011:44:140
32173997018,4cyclictest30726-21sh10:18:102
32171997019,40cyclictest7950irq/56-dwc_otg08:25:120
32171997010,51cyclictest0-21swapper/011:18:270
3217199698,48cyclictest16716-21sh09:16:560
32171996918,36cyclictest0-21swapper/011:36:420
3217199691,63cyclictest3044-21grep08:45:420
3217199691,55cyclictest25054-21latency10:10:370
3217199691,50cyclictest0-21swapper/008:27:510
7062680,1sleep30-21swapper/309:39:143
7062680,1sleep30-21swapper/309:39:143
32171996817,38cyclictest0-21swapper/009:44:000
3217199681,55cyclictest0-21swapper/008:57:450
3217199681,54cyclictest0-21swapper/011:09:040
3217199677,4cyclictest32186-21/usr/sbin/munin07:10:280
3217199671,15cyclictest17601-21ssh10:43:460
316022671,15sleep20-21swapper/207:06:272
189412670,5sleep33217499cyclictest12:08:183
189412670,5sleep33217499cyclictest12:08:183
38092660,2sleep00-21swapper/007:20:360
3217199662,51cyclictest0-21swapper/009:51:480
32171996618,39cyclictest0-21swapper/011:21:160
32171996616,3cyclictest0-21swapper/010:04:520
32171996616,3cyclictest0-21swapper/010:04:520
3217199661,16cyclictest0-21swapper/007:15:390
3217199661,16cyclictest0-21swapper/007:15:390
3217199661,15cyclictest0-21swapper/008:55:010
3217199651,32cyclictest9935-21unixbench_singl07:35:410
3217199651,16cyclictest24822-21ssh09:27:510
3217299641,10cyclictest8150irq/56-dwc_otg_11:20:281
3217199641,38cyclictest18249-21fschecks_count08:00:310
3217199631,49cyclictest0-21swapper/009:22:230
32171996310,42cyclictest0-21swapper/008:11:390
3217199622,32cyclictest9130-21aten_rbpower_cu09:05:260
32171996210,37cyclictest0-21swapper/008:06:210
32171996210,37cyclictest0-21swapper/008:06:210
314372620,4sleep30-21swapper/307:05:423
3217299605,7cyclictest5387-21ssh09:45:271
32172995824,4cyclictest8150irq/56-dwc_otg_07:49:021
32172995824,4cyclictest8150irq/56-dwc_otg_07:49:021
32171995813,35cyclictest0-21swapper/008:39:020
32171995812,41cyclictest7028-21aten_rbpower_cu07:30:290
3217299574,6cyclictest7950irq/56-dwc_otg10:05:281
3217399566,6cyclictest8150irq/56-dwc_otg_11:10:292
3217399556,15cyclictest0-21swapper/210:35:192
3217299515,22cyclictest23-21ksoftirqd/110:35:281
3217299514,37cyclictest8150irq/56-dwc_otg_07:30:261
3217299505,15cyclictest0-21swapper/108:25:241
180192490,34sleep316806-21/usr/sbin/munin10:00:483
180192490,34sleep316806-21/usr/sbin/munin10:00:483
3217399477,25cyclictest0-21swapper/208:05:272
3217399477,25cyclictest0-21swapper/208:05:272
3217299477,8cyclictest0-21swapper/107:20:291
3217299477,23cyclictest0-21swapper/108:30:271
32172994714,17cyclictest0-21swapper/111:13:151
3217399467,25cyclictest0-21swapper/208:55:262
3217299467,4cyclictest0-21swapper/112:30:291
3217499458,24cyclictest0-21swapper/311:05:193
3217499444,6cyclictest30250-21ssh11:00:353
3217399447,23cyclictest0-21swapper/207:10:272
3217399443,26cyclictest0-21swapper/209:49:392
3217299446,25cyclictest0-21swapper/108:00:271
3217499432,20cyclictest87902sleep307:35:253
3217399438,23cyclictest0-21swapper/208:25:272
3217399433,34cyclictest8150irq/56-dwc_otg_08:40:252
3217399431,9cyclictest31-21ksoftirqd/211:47:522
3217399431,9cyclictest31-21ksoftirqd/211:47:522
32172994320,5cyclictest22256-21/usr/sbin/munin08:10:351
3217299431,4cyclictest14886-21apt-get10:40:261
250502430,20sleep18150irq/56-dwc_otg_10:10:371
3217499427,22cyclictest0-21swapper/309:57:563
3217499427,20cyclictest0-21swapper/311:50:273
32174994222,4cyclictest8647-21sh11:55:363
3217399427,24cyclictest0-21swapper/210:40:272
32173994211,14cyclictest8150irq/56-dwc_otg_09:10:302
3217299428,16cyclictest0-21swapper/112:13:291
3217299424,13cyclictest0-21swapper/110:56:151
279092420,1sleep10-21swapper/109:31:181
32174994129,9cyclictest27412-21ssh12:19:183
32174994122,12cyclictest11286-21ssh11:16:483
32174994115,4cyclictest18697-21basename10:45:283
3217499411,2cyclictest0-21swapper/308:25:243
3217399416,21cyclictest0-21swapper/207:55:182
3217399413,12cyclictest0-21swapper/209:40:262
32172994128,10cyclictest0-21swapper/112:25:471
32172994128,10cyclictest0-21swapper/112:25:471
3217299412,28cyclictest0-21swapper/111:40:171
32172994122,5cyclictest1141-21runrttasks07:25:371
32172994117,14cyclictest0-21swapper/110:30:281
3217499408,9cyclictest20084-21fschecks_time08:05:313
3217499408,9cyclictest20084-21fschecks_time08:05:313
3217499407,21cyclictest0-21swapper/310:40:193
32173994011,14cyclictest0-21swapper/211:10:132
3217299404,5cyclictest7950irq/56-dwc_otg09:25:271
32172994012,13cyclictest0-21swapper/111:39:531
32172994010,5cyclictest8050irq/56-dwc_otg_09:20:301
3217499397,25cyclictest0-21swapper/309:05:263
3217499396,21cyclictest0-21swapper/307:40:263
3217499395,6cyclictest8050irq/56-dwc_otg_12:20:313
3217499394,23cyclictest12460-21aten_rbpower_en07:45:263
3217499394,23cyclictest12460-21aten_rbpower_en07:45:263
3217499391,7cyclictest8150irq/56-dwc_otg_09:50:333
32174993910,4cyclictest0-21swapper/311:35:283
3217399392,19cyclictest3002-21ssh10:25:132
32173993914,6cyclictest1141-21runrttasks10:06:272
32173993914,5cyclictest10588-21/usr/sbin/munin07:40:412
3217299398,5cyclictest7950irq/56-dwc_otg11:05:261
3217299398,4cyclictest8150irq/56-dwc_otg_12:35:331
3217299398,22cyclictest0-21swapper/111:55:341
3217299392,7cyclictest10223-21unixbench_multi09:05:451
32172993911,4cyclictest21-21rcuc/109:55:191
3217499385,5cyclictest8050irq/56-dwc_otg_09:10:273
3217499384,25cyclictest0-21swapper/310:51:053
3217499382,6cyclictest8150irq/56-dwc_otg_10:55:243
3217499382,16cyclictest0-21swapper/312:25:323
3217499382,16cyclictest0-21swapper/312:25:323
3217399387,23cyclictest0-21swapper/207:25:272
3217399386,22cyclictest0-21swapper/212:05:332
3217399386,22cyclictest0-21swapper/212:05:332
3217399384,6cyclictest27487-21tail10:55:412
3217399384,19cyclictest8150irq/56-dwc_otg_09:30:412
32173993816,5cyclictest7950irq/56-dwc_otg12:24:052
3217399381,10cyclictest23766-21unixbench_singl09:25:462
32172993830,6cyclictest8150irq/56-dwc_otg_09:15:321
32172993824,5cyclictest15952-21munin-plugin-st12:05:131
32172993811,14cyclictest7950irq/56-dwc_otg12:20:441
32172993810,15cyclictest0-21swapper/111:54:361
32174993720,4cyclictest26399-21aten_rbpower_cu09:30:343
32174993714,5cyclictest22076-21sh12:11:403
3217399373,4cyclictest1149-21snmpd09:39:232
3217399373,4cyclictest1149-21snmpd09:39:232
32173993716,4cyclictest1141-21runrttasks11:21:262
3217299378,11cyclictest0-21swapper/110:50:251
3217299377,24cyclictest0-21swapper/109:00:271
3217299376,4cyclictest7950irq/56-dwc_otg11:05:131
3217299376,26cyclictest0-21swapper/109:35:271
3217299376,26cyclictest0-21swapper/109:35:271
3217299372,4cyclictest8150irq/56-dwc_otg_09:50:351
3217499368,7cyclictest7950irq/56-dwc_otg07:55:273
32174993620,4cyclictest5621-21sh09:45:323
32174993611,7cyclictest39-21ksoftirqd/312:39:073
3217499361,10cyclictest0-21swapper/310:10:293
3217399367,16cyclictest8050irq/56-dwc_otg_09:20:512
3217399362,6cyclictest8150irq/56-dwc_otg_09:55:342
32173993617,5cyclictest1141-21runrttasks12:12:422
32173993616,5cyclictest24675-21latency_hist11:35:132
32173993615,4cyclictest29-21rcuc/210:00:242
32173993615,4cyclictest29-21rcuc/210:00:242
32173993610,8cyclictest8150irq/56-dwc_otg_12:00:472
3217299367,13cyclictest8150irq/56-dwc_otg_12:05:451
3217299367,13cyclictest8150irq/56-dwc_otg_12:05:451
3217299365,7cyclictest7950irq/56-dwc_otg08:35:431
3217299365,21cyclictest17212-21ssh11:25:281
32172993615,5cyclictest147-21systemd-journal07:59:041
3217299361,20cyclictest0-21swapper/107:40:261
3217299361,11cyclictest11604-21wc09:10:361
247252360,5sleep13217299cyclictest12:15:331
3217499359,6cyclictest7950irq/56-dwc_otg11:30:513
3217499353,7cyclictest25404-21/usr/sbin/munin08:20:303
3217499352,4cyclictest8150irq/56-dwc_otg_11:20:263
3217499352,30cyclictest8150irq/56-dwc_otg_09:25:273
32174993518,13cyclictest39-21ksoftirqd/310:35:283
32174993511,13cyclictest0-21swapper/310:25:413
3217399355,6cyclictest8050irq/56-dwc_otg_11:55:352
3217399354,19cyclictest6490-21ssh11:52:172
3217399353,5cyclictest7950irq/56-dwc_otg08:35:342
3217399351,16cyclictest8150irq/56-dwc_otg_08:55:132
3217299358,15cyclictest0-21swapper/107:15:261
3217299358,15cyclictest0-21swapper/107:15:261
3217299357,9cyclictest0-21swapper/108:15:271
3217299355,6cyclictest8150irq/56-dwc_otg_08:20:251
3217299353,9cyclictest8150irq/56-dwc_otg_10:15:261
32172993510,6cyclictest0-21swapper/110:20:391
3217499349,14cyclictest0-21swapper/312:02:023
3217499342,6cyclictest8150irq/56-dwc_otg_08:40:323
32174993425,5cyclictest2197-21fschecks_time08:45:313
3217499341,7cyclictest8150irq/56-dwc_otg_09:15:343
3217499341,7cyclictest8050irq/56-dwc_otg_07:15:303
3217499341,7cyclictest8050irq/56-dwc_otg_07:15:303
3217499341,24cyclictest6778-21ls07:30:133
3217499341,16cyclictest8150irq/56-dwc_otg_11:10:513
32174993410,6cyclictest7950irq/56-dwc_otg10:20:323
32174993410,13cyclictest0-21swapper/311:49:283
32174993410,13cyclictest0-21swapper/311:49:283
3217399348,7cyclictest8050irq/56-dwc_otg_12:35:342
3217399348,5cyclictest0-21swapper/209:50:442
3217399346,15cyclictest8050irq/56-dwc_otg_09:00:372
3217399346,14cyclictest8150irq/56-dwc_otg_10:45:372
3217399343,6cyclictest8150irq/56-dwc_otg_12:25:472
3217399343,6cyclictest8150irq/56-dwc_otg_12:25:472
3217399343,15cyclictest8150irq/56-dwc_otg_08:10:412
3217399342,7cyclictest10221-21ssh11:15:392
3217399342,5cyclictest7950irq/56-dwc_otg11:42:202
3217399342,18cyclictest7950irq/56-dwc_otg11:25:422
3217399341,14cyclictest0-21swapper/212:30:322
3217299349,4cyclictest0-21swapper/110:47:061
3217299347,4cyclictest0-21swapper/110:25:431
3217299345,6cyclictest7950irq/56-dwc_otg09:40:301
32172993416,6cyclictest11154-1kworker/u9:211:18:141
32172993415,13cyclictest23-21ksoftirqd/107:10:251
3217299341,27cyclictest0-21swapper/110:00:301
3217299341,27cyclictest0-21swapper/110:00:301
32172993411,12cyclictest0-21swapper/111:45:241
32172993411,12cyclictest0-21swapper/111:45:241
3217499335,7cyclictest8150irq/56-dwc_otg_08:35:413
3217499333,21cyclictest31806-21ssh10:20:133
32174993316,11cyclictest39-21ksoftirqd/311:25:273
3217499331,16cyclictest0-21swapper/310:32:283
32174993311,5cyclictest32510-21tune2fs07:10:313
32174993311,4cyclictest0-21swapper/309:40:273
3217399337,6cyclictest8150irq/56-dwc_otg_10:10:422
3217399335,7cyclictest7950irq/56-dwc_otg11:35:462
3217399334,5cyclictest3706-21grep07:20:342
3217399333,5cyclictest7475-21tune2fs10:30:342
3217399332,11cyclictest8150irq/56-dwc_otg_08:50:142
3217399331,21cyclictest3627-21cut10:25:342
32172993312,10cyclictest1658-21taskset08:45:131
3217499327,5cyclictest7950irq/56-dwc_otg12:30:363
3217499324,25cyclictest8150irq/56-dwc_otg_10:05:283
32174993222,7cyclictest4945-21idleruntime-cro07:25:123
3217499321,3cyclictest0-21swapper/309:00:123
3217399326,5cyclictest8150irq/56-dwc_otg_07:49:012
3217399326,5cyclictest8150irq/56-dwc_otg_07:49:012
3217399324,4cyclictest7950irq/56-dwc_otg07:50:352
32172993212,6cyclictest8823-21aten_rbpower_en07:35:261
317432323,18sleep10-21swapper/107:08:121
3217499318,4cyclictest0-21swapper/309:20:313
3217499316,14cyclictest0-21swapper/308:00:273
3217499314,13cyclictest8150irq/56-dwc_otg_07:50:423
3217499311,7cyclictest7950irq/56-dwc_otg11:40:283
3217399315,6cyclictest8150irq/56-dwc_otg_07:30:332
3217399312,9cyclictest9085-21/usr/sbin/munin09:05:342
3217399312,7cyclictest1545-21/usr/sbin/munin07:15:312
3217399312,7cyclictest1545-21/usr/sbin/munin07:15:312
3217399312,18cyclictest0-21swapper/208:19:062
32173993111,5cyclictest0-21swapper/208:25:132
3217299317,6cyclictest23-21ksoftirqd/108:05:311
3217299317,6cyclictest23-21ksoftirqd/108:05:311
3217299314,4cyclictest7950irq/56-dwc_otg07:50:351
32172993122,7cyclictest6629-21grep08:55:421
3217499306,15cyclictest0-21swapper/308:30:273
3217499303,7cyclictest0-21swapper/308:15:253
3217499303,16cyclictest7914-21timerandwakeup07:30:423
3217499301,24cyclictest22085-21latency08:10:333
3217399306,15cyclictest0-21swapper/208:30:192
3217399305,6cyclictest4750irq/17-3f00b88007:35:302
3217399301,13cyclictest8150irq/56-dwc_otg_08:00:292
3217299301,16cyclictest0-21swapper/108:50:411
3217499292,12cyclictest8150irq/56-dwc_otg_08:50:423
3217299291,5cyclictest8150irq/56-dwc_otg_08:45:261
3217499271,13cyclictest8239-21timerandwakeup09:00:413
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional