You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-05-28 - 15:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

ARM Freescale i.MX6 Quad @996 MHz, Linux 5.15.26-xenomai (Profile)

Latency plot of shadow in rack #b, slot #7
Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s (updated Sun May 28, 2023 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2000526316,613sleep020011-21ntpq11:20:450
20012244512,418sleep120018-21ntpq11:20:461
22306242113,393sleep022372-21grep08:00:420
19812241616,384sleep319822-21/usr/sbin/munin11:20:403
2696823886,370sleep026975-21ntpq08:15:410
308223369,244sleep03124-21awk12:10:400
5462293126,25sleep00-21swapper/008:35:400
28155226313,110sleep028243-21fschecks_time08:20:330
3139222341,165sleep03195-21sendmail_mailqu07:00:370
3139222341,165sleep03195-21sendmail_mailqu07:00:370
23130222324,165sleep023136-21ntpq11:30:420
20437222315,186sleep020452-21latency_hist09:40:090
1860422069,183sleep218678-21find11:15:462
668222035,12sleep00-21swapper/008:55:300
5092219752,86sleep05131-21proc_pri07:05:420
2507219420,152sleep22620-21timerwakeupswit08:40:552
30354218727,30sleep030493-21unixbench_singl08:25:580
4715218520,25sleep00-21swapper/008:50:090
2429218411,159sleep02495-21latency_hist12:10:080
2429218411,159sleep02495-21latency_hist12:10:080
25567218029,96sleep025591-21ls09:55:360
14486218014,148sleep014515-21munin-plugin-st11:05:060
3167921779,149sleep00-21swapper/010:15:310
28637217414,144sleep00-21swapper/011:50:090
28650217216,16sleep00-21swapper/010:05:370
746421719,147sleep07493-21awk07:15:070
3238621679,147sleep032420-21ntpq12:00:390
3238621679,147sleep032420-21ntpq12:00:390
5565216620,24sleep00-21swapper/012:20:090
7514216226,72sleep27601-21latency_hist07:15:092
26227216276,22sleep025736-21/usr/sbin/munin11:40:440
3145421589,116sleep031495-21ls08:30:390
1846921589,134sleep018531-21uname11:15:440
2463921568,37sleep024686-21sendmail08:10:070
2463921568,37sleep024686-21sendmail08:10:070
2513721545,16sleep00-21swapper/011:35:550
1682421545,18sleep00-21swapper/007:44:430
1682421545,18sleep00-21swapper/007:44:430
2995721505,11sleep00-21swapper/011:52:470
16126215011,99sleep016129-21cron11:10:070
1083621507,110sleep0369-21snmpd09:07:220
17752214912,69sleep00-21swapper/007:45:400
10849214829,56sleep00-21swapper/007:25:250
1035821465,11sleep00-21swapper/007:22:340
379921456,11sleep00-21swapper/010:30:210
379921456,11sleep00-21swapper/010:30:210
1638921459,103sleep016416-21awk09:25:390
31732214421,98sleep231750-21grep12:00:082
1265821435,12sleep00-21swapper/010:56:300
1265821435,12sleep00-21swapper/010:56:300
1165321419,42sleep00-21swapper/009:10:410
30192214015,109sleep330157-21latency_hist11:55:083
26264213926,79sleep026323-21date08:15:080
26264213926,79sleep026323-21date08:15:080
1465721399,114sleep014815-21sed07:35:470
1465721399,114sleep014815-21sed07:35:470
1070921397,13sleep20-21swapper/209:05:502
857721389,89sleep08607-21munin-run12:30:070
224121388,42sleep00-21swapper/008:40:490
21586213811,93sleep021631-21latency_hist08:00:100
880121365,98sleep00-21swapper/007:17:250
8246213326,17sleep00-21swapper/009:00:270
16832213013,103sleep116833-21sshd07:44:481
16832213013,103sleep116833-21sshd07:44:481
1333721288,86sleep013376-21ntpq09:15:420
3229621270,8sleep20-21swapper/212:00:362
3229621270,8sleep20-21swapper/212:00:362
651121255,87sleep00-21swapper/010:38:370
651121255,87sleep00-21swapper/010:38:370
2790721255,11sleep00-21swapper/010:01:500
2790721255,11sleep00-21swapper/010:01:500
2348421239,27sleep023488-21cron09:50:060
5427212111,84sleep00-21swapper/008:50:460
1867921215,82sleep00-21swapper/009:32:580
1538321210,9sleep30-21swapper/307:40:093
1538321210,9sleep30-21swapper/307:40:093
815821205,12sleep00-21swapper/010:44:500
19820211921,71sleep319957-21grep09:35:443
7910211818,80sleep17872-21apt-get09:00:081
2622211621,75sleep32269-21apt-get07:00:163
2622211621,75sleep32269-21apt-get07:00:163
2244211627,68sleep32323-21sh07:00:073
21628211420,67sleep00-21swapper/009:41:060
20689211422,64sleep120683-21apt-get09:40:261
23465211227,64sleep223508-21fschecks_count08:05:332
23465211227,64sleep223508-21fschecks_count08:05:332
2003821128,89sleep020187-21latency_hist07:55:090
578121118,16sleep00-21swapper/012:20:290
2804211110,84sleep32890-21/usr/sbin/munin12:10:353
650211020,72sleep1660-21netstat08:35:411
5080210816,75sleep15098-21basename10:35:061
5080210816,75sleep15098-21basename10:35:061
22666210812,17sleep20-21swapper/211:30:282
1702821080,7sleep30-21swapper/307:45:103
1702821080,7sleep30-21swapper/307:45:103
960210723,69sleep0972-21cat06:55:200
14509210714,20sleep214579-21sensors07:35:442
14509210714,20sleep214579-21sensors07:35:442
7789210516,73sleep37803-21awk12:25:413
2481721055,89sleep00-21swapper/009:51:520
10513210517,75sleep110518-21sh10:50:451
1546821045,88sleep00-21swapper/009:21:520
1385210422,61sleep31535-21unixbench_multi10:20:553
893021030,9sleep10-21swapper/109:00:441
16425210224,15sleep00-21swapper/011:10:230
6309210115,28sleep36391-21latency_hist08:55:093
319221015,69sleep00-21swapper/010:26:090
3126521015,85sleep00-21swapper/010:14:420
1050521016,62sleep00-21swapper/010:50:440
9492996,30sleep00-21swapper/010:20:440
1391529915,69sleep214039-21sed11:00:482
136712995,65sleep00-21swapper/007:35:020
135629925,46sleep10-21swapper/106:55:341
431629822,61sleep34344-21cut12:15:323
1314729819,54sleep312402-21/usr/sbin/munin07:30:533
1719329721,20sleep20-21swapper/211:10:452
799729617,66sleep28094-21cat09:00:092
54852960,12sleep30-21swapper/310:35:403
54852960,12sleep30-21swapper/310:35:403
309852956,31sleep00-21swapper/011:55:420
908129353,26sleep09160-21cat10:45:430
908129353,26sleep09160-21cat10:45:430
3004729313,63sleep230048-21cpu10:10:282
832229211,64sleep38329-21/usr/sbin/munin09:00:303
304132928,12sleep30-21swapper/310:10:373
219482905,29sleep00-21swapper/011:25:500
1333029015,58sleep113385-21fschecks_time11:00:351
313522895,12sleep10-21swapper/111:55:471
2155828916,55sleep30-21swapper/311:25:443
1884628916,57sleep218867-21modprobe09:35:062
3235628819,50sleep331963-21/usr/sbin/munin12:00:383
3235628819,50sleep331963-21/usr/sbin/munin12:00:383
1133528817,54sleep311388-21grep07:25:403
548028711,21sleep20-21swapper/210:35:402
548028711,21sleep20-21swapper/210:35:402
2665728714,54sleep126720-21idleruntime08:15:341
69628611,31sleep10-21swapper/110:20:321
307752867,69sleep10-21swapper/108:29:021
1319828518,48sleep113287-21load09:15:391
84482840,8sleep20-21swapper/209:00:322
3248528324,39sleep332603-21latency_hist08:35:103
155652835,11sleep20-21swapper/207:40:172
155652835,11sleep20-21swapper/207:40:172
1164728118,44sleep211323-21apt-get10:55:142
1164728118,44sleep211323-21apt-get10:55:142
2700428024,26sleep10-21swapper/110:00:331
2700428024,26sleep10-21swapper/110:00:331
374927912,34sleep20-21swapper/208:45:352
984427814,14sleep20-21swapper/210:50:102
980127822,37sleep19745-21latency_hist10:50:091
980127822,37sleep19745-21latency_hist10:50:091
2907227816,40sleep328441-21/usr/sbin/munin10:05:483
2546627813,15sleep30-21swapper/309:55:343
1653727830,30sleep10-21swapper/111:10:321
149627814,50sleep31542-21grep12:06:033
149627814,50sleep31542-21grep12:06:033
3061127720,44sleep130676-21cat10:10:431
278427714,27sleep30-21swapper/310:25:433
645727619,38sleep36569-21sed12:20:493
3186127515,15sleep30-21swapper/310:15:343
760727417,40sleep27754-21unixbench_multi10:40:452
3185227429,27sleep10-21swapper/108:30:481
2930827420,34sleep129338-21latency_hist08:25:071
2708427416,40sleep127233-21sed11:45:091
2324627421,27sleep30-21swapper/311:30:453
1243127418,41sleep212481-21cpuspeed07:30:312
1136627424,31sleep111378-21apt-get09:10:301
390927311,48sleep33914-21diskstats10:30:323
390927311,48sleep33914-21diskstats10:30:323
340127327,28sleep10-21swapper/110:28:481
231027325,33sleep32346-21grep08:40:523
1827027319,30sleep10-21swapper/111:15:371
1465827321,32sleep314500-21apt-get11:05:093
735227214,27sleep30-21swapper/308:55:463
530327221,36sleep35428-21cat07:05:453
1844627219,30sleep10-21swapper/109:30:491
231427113,43sleep22321-21df_inode10:25:302
2260027110,45sleep122627-21netstat09:45:351
1734827115,38sleep117458-21latency_hist09:30:081
434427013,43sleep24415-21users07:05:152
2515827011,46sleep225209-21if_eth008:10:342
2515827011,46sleep225209-21if_eth008:10:342
2512427014,43sleep225200-21sed09:55:092
1913327017,28sleep10-21swapper/107:50:381
335226918,35sleep23468-21sed12:10:462
1367726924,27sleep113703-21munin-run07:35:071
289722689,12sleep30-21swapper/308:20:483
2869826812,40sleep128734-21awk10:05:381
257532688,43sleep20-21swapper/211:40:252
181782688,49sleep10-21swapper/107:47:151
586826714,39sleep25899-21tune2fs12:20:362
2904926718,28sleep20-21swapper/208:21:482
120526722,25sleep10-21swapper/112:05:571
120526722,25sleep10-21swapper/112:05:571
1941026619,27sleep20-21swapper/209:35:352
1533326612,26sleep30-21swapper/311:05:463
148212668,41sleep10-21swapper/111:05:111
81726512,38sleep3888-21sensors_temp08:35:473
794026516,23sleep10-21swapper/107:15:321
2445126510,24sleep30-21swapper/311:35:343
2153226526,23sleep10-21swapper/108:00:071
1967326516,25sleep10-21swapper/109:35:421
1762326513,25sleep20-21swapper/207:45:382
154592657,10sleep30-21swapper/309:21:463
1338626520,23sleep20-21swapper/209:15:442
1156026518,22sleep20-21swapper/207:25:432
979226413,21sleep30-21swapper/307:20:433
3199526414,26sleep20-21swapper/210:15:372
2089526420,26sleep320935-21if_eth009:40:323
759826324,19sleep20-21swapper/212:25:362
669326312,22sleep30-21swapper/307:10:423
443926310,11sleep10-21swapper/112:15:341
2896326315,31sleep128969-21/usr/sbin/munin11:50:341
2728526310,34sleep30-21swapper/311:45:273
1770226321,28sleep317721-21modprobe11:15:063
84026219,22sleep10-21swapper/112:05:081
84026219,22sleep10-21swapper/112:05:081
622026215,24sleep10-21swapper/108:55:071
388526211,35sleep33959-21cat08:45:403
36002625,10sleep20-21swapper/207:01:032
36002625,10sleep20-21swapper/207:01:032
1577326213,25sleep20-21swapper/209:25:092
1189826221,19sleep30-21swapper/309:10:483
945226124,21sleep20-21swapper/207:20:342
928226116,20sleep10-21swapper/107:20:251
1505026123,16sleep10-21swapper/109:20:441
1112626119,23sleep20-21swapper/210:52:482
1016226118,25sleep30-21swapper/309:05:393
784226019,22sleep30-21swapper/307:15:303
2508126025,19sleep10-21swapper/108:10:331
2508126025,19sleep10-21swapper/108:10:331
2237426017,27sleep122437-21/usr/sbin/munin08:00:431
2075326017,26sleep320768-21/usr/sbin/munin07:55:493
1940026013,31sleep319521-21timerwakeupswit07:50:443
1726826018,17sleep30-21swapper/309:30:063
1532626017,28sleep215332-21/usr/sbin/munin11:05:452
1175226018,27sleep211788-21ntpq09:10:432
900825915,30sleep39079-21sensors_temp10:45:423
900825915,30sleep39079-21sensors_temp10:45:423
570925911,33sleep15710-21sshd07:07:491
2551325919,21sleep10-21swapper/111:40:081
2254725913,16sleep30-21swapper/308:00:453
2243825916,26sleep20-21swapper/208:00:442
2234125914,24sleep122374-21munin-run11:30:071
1298925922,18sleep30-21swapper/311:00:083
1298925922,18sleep30-21swapper/311:00:083
3196125815,29sleep131994-21/usr/sbin/munin10:15:361
290022588,13sleep20-21swapper/210:05:472
2418225813,31sleep124206-21netstat09:50:401
2287625815,19sleep10-21swapper/111:30:351
682525713,21sleep10-21swapper/107:10:441
586225713,16sleep10-21swapper/112:20:351
313925714,19sleep10-21swapper/108:45:101
2392725717,23sleep10-21swapper/108:05:441
2392725717,23sleep10-21swapper/108:05:441
2338125713,25sleep323448-21df08:05:303
2338125713,25sleep323448-21df08:05:303
233652578,11sleep20-21swapper/209:48:352
1870025717,24sleep218717-21apt07:50:222
1134825715,21sleep311435-21gpgv10:55:073
53382565,10sleep10-21swapper/110:35:141
53382565,10sleep10-21swapper/110:35:141
2708525620,18sleep20-21swapper/210:00:342
2708525620,18sleep20-21swapper/210:00:342
2114925620,16sleep20-21swapper/209:40:392
775825518,17sleep30-21swapper/310:40:463
692725512,26sleep16921-21apt-get10:40:271
625025521,17sleep20-21swapper/208:55:082
36622559,35sleep10-21swapper/107:01:511
36622559,35sleep10-21swapper/107:01:511
2855725514,25sleep228586-21munin-run11:50:072
2423825515,22sleep324267-21ntp_states09:50:423
1522525518,17sleep10-21swapper/107:40:081
1522525518,17sleep10-21swapper/107:40:081
1093525518,19sleep10-21swapper/107:25:301
3118125413,24sleep231178-21apt-get08:30:292
3093325414,25sleep230980-21latency_hist08:30:082
2079525421,20sleep20-21swapper/211:25:072
186812549,28sleep318809-21unixbench_multi11:15:473
14222546,37sleep20-21swapper/208:38:452
1300425416,18sleep30-21swapper/309:15:333
1170225412,26sleep111686-21apt-get10:55:381
1170225412,26sleep111686-21apt-get10:55:381
2672425313,24sleep226777-21irqstats08:15:352
2670825315,23sleep126855-21latency_hist10:00:091
1768425317,18sleep30-21swapper/309:30:353
1006325319,19sleep10-21swapper/109:05:371
47482525,38sleep20-21swapper/210:30:512
47482525,38sleep20-21swapper/210:30:512
3064725213,24sleep330413-21/usr/sbin/munin11:55:333
2970025213,21sleep329702-21diskstats08:25:413
2705525216,19sleep30-21swapper/308:15:443
2700925215,21sleep327051-21true11:45:073
2223225212,23sleep322228-21apt-get09:45:253
2109125212,23sleep221088-21apt-get11:25:272
2059225212,23sleep220658-21memory07:55:442
20392527,34sleep20-21swapper/212:06:182
20392527,34sleep20-21swapper/212:06:182
3014325121,16sleep20-21swapper/211:55:072
2594525118,19sleep20-21swapper/209:55:462
1714625112,23sleep317142-21apt-get07:45:233
1602025114,20sleep216014-21apt-get09:25:292
105625113,22sleep21065-21df_inode06:55:262
82325012,22sleep2907-21perl10:20:422
2741925018,15sleep30-21swapper/310:00:433
2741925018,15sleep30-21swapper/310:00:433
268225011,22sleep12675-21apt-get12:10:291
2749424911,24sleep127284-21/usr/sbin/munin11:45:371
854424811,23sleep28552-21df_inode10:45:292
829224813,19sleep28363-21sensors07:15:412
466024815,17sleep20-21swapper/212:15:392
384024816,15sleep10-21swapper/108:45:381
77422467,15sleep10-21swapper/112:25:391
242612458,15sleep20-21swapper/211:35:292
2621424313,14sleep30-21swapper/308:15:073
2621424313,14sleep30-21swapper/308:15:073
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional