You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-08-13 - 01:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

ARM Freescale i.MX6 Quad @996 MHz, Linux 5.15.26-xenomai (Profile)

Latency plot of shadow in rack #b, slot #7
Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s (updated Fri Aug 12, 2022 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
36782146420,1424sleep316194-1kworker/3:0H12:32:363
36782146420,1424sleep316194-1kworker/3:0H12:32:363
1139271716,685sleep31140-21sshd10:38:083
469925669,543sleep34737-21ntpq09:07:463
21491254722,507sleep321551-21/usr/sbin/munin08:17:443
979234010,314sleep31041-21/usr/sbin/munin12:22:513
979234010,314sleep31041-21/usr/sbin/munin12:22:513
123412325124,184sleep112345-21cron11:17:181
29992232318,289sleep230071-21/usr/sbin/munin07:02:472
8444231016,277sleep08466-21/usr/sbin/munin11:02:470
13436230810,284sleep113473-21sh09:37:191
311392291126,144sleep131271-21grep08:47:501
304502274122,30sleep10-21swapper/112:12:481
31088226217,231sleep231095-21diskstats10:32:382
18024222112,193sleep10-21swapper/108:07:371
893221920,185sleep2967-21systemd-run08:57:192
19042220018,162sleep019130-21tune2fs11:37:390
10247219214,159sleep110393-21grep11:07:571
10247219214,159sleep110393-21grep11:07:571
6647218813,156sleep1369-21snmpd10:57:481
15932218810,164sleep016011-21fschecks_time11:27:460
5325218522,40sleep10-21swapper/107:27:331
22128217122,22sleep10-21swapper/111:47:371
22128217122,22sleep10-21swapper/111:47:371
20511216879,77sleep120522-21grep11:42:411
20511216879,77sleep120522-21grep11:42:411
5453216437,112sleep15530-21grep10:52:471
32752216218,123sleep1315-21fschecks_count10:37:381
28342216011,134sleep128243-21/usr/sbin/munin12:07:391
28342216011,134sleep128243-21/usr/sbin/munin12:07:391
28260215912,135sleep128273-21awk08:37:531
28260215912,135sleep128273-21awk08:37:531
13179215814,112sleep10-21swapper/107:52:311
13179215814,112sleep10-21swapper/107:52:311
21693215416,116sleep121707-21fschecks_count10:02:421
226972151106,21sleep10-21swapper/108:22:351
226972151106,21sleep10-21swapper/108:22:351
21598215020,113sleep10-21swapper/108:17:461
160162148101,26sleep10-21swapper/111:27:471
21112214719,34sleep00-21swapper/011:42:560
21112214719,34sleep00-21swapper/011:42:560
205542145103,23sleep10-21swapper/109:57:531
205542145103,23sleep10-21swapper/109:57:531
1501121459,112sleep115058-21systemd-run09:42:191
10960214583,50sleep110992-21ntpq09:27:541
2723214312,23sleep10-21swapper/112:27:541
17336214310,41sleep117377-21awk09:47:521
15570214110,42sleep115589-21tr09:42:441
32534213518,35sleep00-21swapper/008:52:450
9911213412,99sleep10-21swapper/107:42:221
4248213212,16sleep10-21swapper/112:32:501
4248213212,16sleep10-21swapper/112:32:501
7162213013,101sleep10-21swapper/107:32:411
20953212817,88sleep10-21swapper/108:17:221
276521277,104sleep10-21swapper/109:02:311
25786212711,102sleep10-21swapper/110:14:081
3066621246,11sleep10-21swapper/110:31:441
27876212480,35sleep1369-21snmpd12:06:021
27876212480,35sleep1369-21snmpd12:06:021
27125212419,22sleep10-21swapper/108:37:201
26418212428,14sleep10-21swapper/110:17:391
23646212311,100sleep0369-21snmpd08:23:220
23646212311,100sleep0369-21snmpd08:23:220
20262212211,90sleep10-21swapper/111:42:201
10535212030,64sleep210555-21apt-get09:27:392
26636211915,84sleep326667-21/usr/sbin/munin10:17:453
335321185,10sleep10-21swapper/107:20:181
2554321179,84sleep10-21swapper/111:57:461
2554321179,84sleep10-21swapper/111:57:461
13989211712,25sleep014081-21users07:52:510
13989211712,25sleep014081-21users07:52:510
32745211513,87sleep032751-21diskstats10:37:360
2368421158,17sleep10-21swapper/110:07:501
1972921155,96sleep10-21swapper/109:57:171
2791121148,17sleep10-21swapper/110:22:361
2209211417,16sleep10-21swapper/110:42:461
2209211417,16sleep10-21swapper/110:42:461
3211221135,71sleep10-21swapper/107:09:201
3206221138,21sleep131735-21apt-get08:52:301
23238211317,79sleep323251-21fschecks_count10:07:403
15546211310,89sleep10-21swapper/111:27:191
1110211310,82sleep10-21swapper/112:22:531
1110211310,82sleep10-21swapper/112:22:531
1751221125,10sleep10-21swapper/108:05:311
19029211111,87sleep219037-21df11:37:372
3396210914,62sleep10-21swapper/110:47:391
3396210914,62sleep10-21swapper/110:47:391
2360321099,16sleep10-21swapper/111:52:421
2360321099,16sleep10-21swapper/111:52:421
1022521099,89sleep1369-21snmpd09:26:531
17981210810,83sleep318058-21grep11:32:503
2481210778,18sleep12519-21modprobe09:02:191
2139721070,7sleep00-21swapper/010:02:190
2139721070,7sleep00-21swapper/010:02:190
8216210621,64sleep08236-21latency_hist07:37:200
746321068,13sleep10-21swapper/109:17:371
24748210619,70sleep224423-21apt-get10:12:282
18975210613,77sleep319051-21sendmail_mailtr09:52:513
1311210617,73sleep01412-21fschecks_time08:57:420
30814210539,46sleep330855-21latency_hist07:07:203
379721036,13sleep10-21swapper/107:22:321
10259210318,68sleep210283-21awk09:27:182
2423821026,14sleep10-21swapper/108:27:311
1480221029,13sleep10-21swapper/107:57:241
1266321017,77sleep10-21swapper/111:17:341
1266321017,77sleep10-21swapper/111:17:341
598321007,75sleep10-21swapper/109:12:481
27630210012,65sleep327340-21/usr/sbin/munin08:37:423
27630210012,65sleep327340-21/usr/sbin/munin08:37:423
21040210014,69sleep321110-21sensors11:42:553
21040210014,69sleep321110-21sensors11:42:553
2142429918,61sleep221490-21wc08:17:432
78829812,71sleep3841-21interrupts07:12:413
315422985,53sleep10-21swapper/110:32:491
3025729858,16sleep10-21swapper/108:47:211
283412989,74sleep228342-21192.168.115.60-08:38:462
283412989,74sleep228342-21192.168.115.60-08:38:462
16472985,10sleep10-21swapper/107:14:451
120512987,75sleep10-21swapper/107:47:431
1193729819,55sleep00-21swapper/009:32:210
113462985,10sleep10-21swapper/107:47:181
1133129814,28sleep311332-21sshd07:47:083
900429721,45sleep39001-21apt-get09:22:353
43322977,14sleep10-21swapper/109:07:361
314012978,72sleep10-21swapper/112:17:321
314012978,72sleep10-21swapper/112:17:321
2704429721,58sleep327065-21apt08:37:183
183312975,10sleep10-21swapper/111:33:241
89202955,11sleep10-21swapper/111:03:131
440129517,53sleep20-21swapper/207:22:492
2330329517,64sleep223351-21unin-run11:52:182
2330329517,64sleep223351-21unin-run11:52:182
1855129511,69sleep018663-21cat08:07:480
1491429512,69sleep214921-21df_inode07:57:372
2835629424,51sleep328435-21fschecks_time12:07:413
2835629424,51sleep328435-21fschecks_time12:07:413
1005629419,57sleep29734-21apt-get07:42:272
974829221,53sleep39795-21munin-run07:42:203
2005528921,39sleep320050-21apt-get09:57:393
2005528921,39sleep320050-21apt-get09:57:393
3184828822,45sleep031739-21latency_hist08:52:210
1961528816,57sleep319732-21timerwakeupswit11:37:503
492828720,43sleep04949-21apt12:37:170
492828720,43sleep04949-21apt12:37:170
2538428718,51sleep025461-21timerandwakeup10:12:500
2673028613,57sleep326783-21ls12:02:413
2673028613,57sleep326783-21ls12:02:413
1480628614,54sleep014470-21apt-get07:57:270
1856628519,50sleep218648-21fschecks_time09:52:422
261292840,7sleep00-21swapper/008:32:420
622128120,42sleep26133-21latency_hist10:57:202
300922810,7sleep20-21swapper/212:12:402
2165428121,26sleep00-21swapper/008:17:490
1333628112,54sleep213462-21/usr/sbin/munin11:17:512
1333628112,54sleep213462-21/usr/sbin/munin11:17:512
981328021,41sleep09909-21latency_hist07:42:210
3208127922,28sleep30-21swapper/312:17:513
3208127922,28sleep30-21swapper/312:17:513
28702787,12sleep00-21swapper/010:45:240
28702787,12sleep00-21swapper/010:45:240
1765027821,37sleep317664-21modprobe08:07:193
2323027714,48sleep223237-21diskstats10:07:392
205027712,47sleep32120-21/usr/sbin/munin12:27:393
1511027718,38sleep315158-21latency07:57:423
1008427612,52sleep00-21swapper/009:25:020
971927525,35sleep29734-21apt-get07:42:192
237827521,27sleep20-21swapper/212:27:472
1645027515,30sleep30-21swapper/311:27:573
1598027522,30sleep216118-21grep09:42:532
1235327522,30sleep20-21swapper/211:17:192
1968927419,42sleep319764-21fschecks_time08:12:403
1122727414,47sleep011295-21basename11:12:400
433727318,25sleep20-21swapper/212:32:512
433727318,25sleep20-21swapper/212:32:512
324727312,40sleep23305-21wc09:02:482
2733927212,45sleep027356-21apt08:37:310
2733927212,45sleep027356-21apt08:37:310
1408227213,33sleep014216-21ls11:22:210
1408227213,33sleep014216-21ls11:22:210
1140027217,24sleep00-21swapper/007:47:220
2892627110,46sleep328928-21sshd10:24:153
795127021,26sleep20-21swapper/209:17:542
660127021,33sleep36612-21modprobe07:32:193
653727015,39sleep06646-21perl10:57:470
2589827014,38sleep026047-21sed11:57:530
2589827014,38sleep026047-21sed11:57:530
192627015,39sleep30-21swapper/308:57:533
2648226919,30sleep30-21swapper/312:02:203
2648226919,30sleep30-21swapper/312:02:203
725526823,22sleep20-21swapper/207:32:422
3232726825,24sleep30-21swapper/310:37:193
1938526821,29sleep219214-21apt-get08:12:222
1491326811,26sleep30-21swapper/311:22:533
1398426817,32sleep313937-21apt-get11:22:203
1398426817,32sleep313937-21apt-get11:22:203
646126714,25sleep00-21swapper/009:13:000
32212678,44sleep00-21swapper/009:02:460
300122675,11sleep20-21swapper/208:45:202
1720926713,32sleep30-21swapper/309:47:503
1153726719,22sleep30-21swapper/311:12:483
993526617,25sleep20-21swapper/211:07:502
993526617,25sleep20-21swapper/211:07:502
746726618,21sleep00-21swapper/009:17:400
2181526617,22sleep30-21swapper/310:02:443
2083426621,29sleep220899-21modprobe08:17:192
1778126618,21sleep20-21swapper/211:32:452
263062658,12sleep20-21swapper/212:00:422
263062658,12sleep20-21swapper/212:00:422
184626519,22sleep30-21swapper/310:42:403
184626519,22sleep30-21swapper/310:42:403
164722658,17sleep20-21swapper/208:02:372
58426429,19sleep20-21swapper/212:22:422
58426429,19sleep20-21swapper/212:22:422
1351526414,21sleep30-21swapper/309:37:213
769426318,20sleep30-21swapper/311:02:183
3070226310,12sleep30-21swapper/312:12:503
272026317,32sleep02756-21wc07:17:550
1919926316,24sleep30-21swapper/308:12:193
1649226325,22sleep00-21swapper/008:02:400
153572638,13sleep00-21swapper/011:24:510
1402626320,21sleep00-21swapper/009:37:460
962526224,21sleep00-21swapper/011:07:440
962526224,21sleep00-21swapper/011:07:440
493926224,18sleep30-21swapper/310:52:373
409326223,21sleep30-21swapper/309:07:213
3149026213,19sleep20-21swapper/207:07:472
2526626210,43sleep00-21swapper/008:29:160
2262626213,31sleep022719-21/usr/sbin/munin11:47:480
2262626213,31sleep022719-21/usr/sbin/munin11:47:480
2091426220,18sleep00-21swapper/008:17:210
1694126210,39sleep017000-21cpuspeed09:47:430
3148226114,30sleep031541-21/usr/sbin/munin10:32:480
3063026116,27sleep30-21swapper/308:47:393
2890926114,30sleep328979-21/usr/sbin/munin08:42:353
2689926116,30sleep226969-21/usr/sbin/munin10:17:532
240426126,19sleep20-21swapper/210:42:512
240426126,19sleep20-21swapper/210:42:512
1421926017,23sleep20-21swapper/209:37:522
1294526019,21sleep30-21swapper/307:52:213
665725915,18sleep20-21swapper/209:13:022
460525915,26sleep24698-21needreboot09:07:452
430525910,14sleep00-21swapper/009:07:300
389425912,29sleep03983-21/usr/sbin/munin07:22:390
3075525915,30sleep030791-21latency_hist10:32:200
3053925921,22sleep20-21swapper/208:47:382
290782594,17sleep00-21swapper/010:26:100
2384225914,27sleep323871-21irqstats11:52:483
2384225914,27sleep323871-21irqstats11:52:483
2335225919,20sleep30-21swapper/311:52:193
2335225919,20sleep30-21swapper/311:52:193
1363725915,19sleep30-21swapper/307:52:453
1363725915,19sleep30-21swapper/307:52:453
1201125914,17sleep00-21swapper/007:47:420
3239325819,21sleep30-21swapper/307:12:213
3030025816,17sleep10-21swapper/107:02:501
2988025818,19sleep00-21swapper/012:12:350
2672525815,27sleep026720-21apt-get12:02:380
2672525815,27sleep026720-21apt-get12:02:380
2269325816,22sleep20-21swapper/208:22:322
2269325816,22sleep20-21swapper/208:22:322
2062625817,26sleep220705-21cat09:57:542
2062625817,26sleep220705-21cat09:57:542
1484425824,21sleep20-21swapper/211:22:522
91442575,16sleep30-21swapper/311:06:053
3249025715,20sleep30-21swapper/308:52:443
3030225715,25sleep230355-21users10:27:542
260382579,10sleep30-21swapper/310:17:133
192525714,24sleep21955-21munin-run07:17:202
2995325613,17sleep00-21swapper/007:02:450
2985925613,24sleep329906-21ntp_states10:27:473
2555225616,18sleep20-21swapper/208:32:202
253025613,27sleep32576-21sed07:17:503
1186525612,23sleep30-21swapper/309:32:203
853425510,26sleep28596-21/usr/sbin/munin11:02:492
605525512,24sleep06194-21/usr/sbin/munin07:27:520
534725515,23sleep25344-21apt-get07:27:342
379925513,25sleep33794-21apt-get07:22:343
283325516,23sleep22919-21sed07:17:572
222082559,31sleep022282-21find10:02:530
1738325511,26sleep017377-21apt-get11:32:340
1530025515,25sleep015296-21apt-get09:42:340
338025416,22sleep33389-21df_inode10:47:373
338025416,22sleep33389-21df_inode10:47:373
3187625412,26sleep00-21swapper/012:17:470
3187625412,26sleep00-21swapper/012:17:470
3140225415,23sleep231398-21apt-get12:17:332
3140225415,23sleep231398-21apt-get12:17:332
2586325415,25sleep225872-21/usr/sbin/munin08:32:372
254562546,10sleep30-21swapper/308:31:453
2387325417,18sleep00-21swapper/011:52:490
2387325417,18sleep00-21swapper/011:52:490
688525313,24sleep36882-21apt-get07:32:313
643925313,25sleep26434-21apt-get10:57:412
2352025311,28sleep023565-21awk10:07:450
133692536,12sleep20-21swapper/209:36:272
2826525214,21sleep028261-21apt-get12:07:340
2826525214,21sleep028261-21apt-get12:07:340
1318025213,23sleep213176-21apt-get07:52:322
1318025213,23sleep213176-21apt-get07:52:322
1007225212,19sleep30-21swapper/311:07:553
1007225212,19sleep30-21swapper/311:07:553
2912625110,23sleep029175-21irqstats08:42:400
2791225111,19sleep20-21swapper/210:22:372
2277025113,17sleep30-21swapper/308:22:373
2277025113,17sleep30-21swapper/308:22:373
166525111,15sleep20-21swapper/208:57:482
754525012,23sleep37550-21diskstats09:17:433
58425012,19sleep20-21swapper/210:37:432
461625015,15sleep20-21swapper/210:52:202
461625015,15sleep20-21swapper/210:52:202
2135025014,19sleep221408-21users11:42:582
2135025014,19sleep221408-21users11:42:582
1905625013,19sleep00-21swapper/009:52:520
1701625012,23sleep217022-21diskstats09:47:452
124172508,15sleep00-21swapper/009:32:410
46124912,14sleep00-21swapper/007:12:260
3173124913,18sleep00-21swapper/007:07:540
2681324912,18sleep20-21swapper/212:02:442
2681324912,18sleep20-21swapper/212:02:442
2491524911,23sleep224925-21munin-run11:57:192
2491524911,23sleep224925-21munin-run11:57:192
1383624911,23sleep313842-21diskstats09:37:413
1171324913,18sleep20-21swapper/207:47:362
185424812,16sleep00-21swapper/012:27:200
2634324711,18sleep00-21swapper/010:17:340
162852478,31sleep30-21swapper/309:43:513
1626024711,21sleep216296-21ntp_states11:27:522
490424610,14sleep00-21swapper/010:52:340
33912469,19sleep00-21swapper/010:47:380
33912469,19sleep00-21swapper/010:47:380
2171124613,15sleep20-21swapper/210:02:432
2826724511,17sleep20-21swapper/212:07:362
2826724511,17sleep20-21swapper/212:07:362
610724412,17sleep30-21swapper/309:12:503
4952448,19sleep00-21swapper/012:22:360
4952448,19sleep00-21swapper/012:22:360
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional