You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-23 - 05:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  

ARM Freescale i.MX6 Quad @996 MHz, Linux 5.15.26-xenomai (Profile)

Latency plot of shadow in rack #b, slot #7
Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s (updated Thu Mar 23, 2023 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13455285282,643sleep113468-21modprobe21:04:041
13455285282,643sleep113468-21modprobe21:04:041
20394264210,622sleep120422-21ntpq21:24:431
265782429116,210sleep126657-21timerandwakeup19:59:381
75972341150,160sleep10-21swapper/120:44:251
303312337144,170sleep130335-21cron20:14:041
2318230313,212sleep12440-21wc22:09:381
97112297127,28sleep19838-21latency_hist00:19:061
10206228921,246sleep210348-21sed22:34:392
313422286114,117sleep131484-21grep20:14:451
26997227873,35sleep127133-21unixbench_multi21:44:391
14612223934,189sleep114647-21basename19:24:041
3906223615,206sleep13926-21kernelversion23:59:351
3906223615,206sleep13926-21kernelversion23:59:351
10730222810,18sleep10-21swapper/119:09:341
12245222517,185sleep012392-21unixbench_singl00:24:420
25065222263,126sleep10-21swapper/123:24:261
1642922148,194sleep116450-21dpkg19:29:171
26046220744,22sleep10-21swapper/121:44:061
14837220314,162sleep114993-21unixbench_multi22:49:411
144162203159,24sleep10-21swapper/121:04:361
2092220019,167sleep32134-21ntpq22:09:323
2604219720,152sleep10-21swapper/120:29:061
3643219012,164sleep33651-21needreboot22:14:333
541521889,167sleep1201-21jbd2/mmcblk2p2-20:35:451
19416218312,149sleep119513-21latency_hist19:39:061
19416218312,149sleep119513-21latency_hist19:39:061
23941218117,136sleep10-21swapper/121:34:451
28194217914,151sleep128231-21cpu23:34:271
27150217916,17sleep10-21swapper/123:29:431
6586217610,140sleep18052-21cyclictest22:24:331
15993216513,19sleep10-21swapper/121:09:451
15993216513,19sleep10-21swapper/121:09:451
5313216414,135sleep35373-21sendmail_mailst22:19:383
22812216113,130sleep10-21swapper/119:49:161
2441215123,105sleep22581-21unixbench_singl22:09:392
9573215018,118sleep19574-21munin-run19:09:061
9573215018,118sleep19574-21munin-run19:09:061
28252215017,97sleep128384-21head20:04:391
18239214912,123sleep118388-21cat21:19:061
1811421497,122sleep00-21swapper/019:34:250
1811421497,122sleep00-21swapper/019:34:250
24500214492,24sleep10-21swapper/119:54:251
24021214094,24sleep10-21swapper/123:19:371
178942139103,18sleep10-21swapper/122:59:421
178942139103,18sleep10-21swapper/122:59:421
16653213926,47sleep10-21swapper/122:54:501
14381213928,94sleep014401-21apt00:34:030
14381213928,94sleep014401-21apt00:34:030
11897213515,84sleep111933-21ntpq00:24:351
7221213212,75sleep10-21swapper/120:44:041
7221213212,75sleep10-21swapper/120:44:041
2323921329,85sleep123253-21munin-run23:19:051
2950221319,83sleep129592-21perl21:54:281
28063213011,103sleep328130-21awk20:04:373
18925212510,81sleep118982-21grep23:04:311
7485212128,17sleep10-21swapper/100:09:411
14665211814,83sleep314751-21grep19:24:063
762821170,9sleep20-21swapper/220:44:252
3199211466,36sleep13202-21systemd23:58:421
29088211425,13sleep10-21swapper/121:54:031
28280211425,68sleep228360-21tune2fs23:34:322
22885211415,83sleep222808-21apt-get21:34:052
22885211415,83sleep222808-21apt-get21:34:052
1111221146,65sleep10-21swapper/122:39:311
5538211373,24sleep15579-21latency00:04:361
340211219,81sleep1427-21cat20:19:371
1703211014,67sleep30-21swapper/320:24:363
9263210914,14sleep10-21swapper/122:34:061
9263210914,14sleep10-21swapper/122:34:061
5334210814,78sleep05342-21df00:04:270
31588210814,45sleep10-21swapper/121:59:411
22465210810,18sleep10-21swapper/119:49:041
3018321077,15sleep10-21swapper/123:39:321
3018321077,15sleep10-21swapper/123:39:321
10739210712,81sleep210826-21fschecks_time20:54:322
3569210613,16sleep10-21swapper/120:29:391
16398210613,69sleep215715-21/usr/sbin/munin22:54:472
13090210611,81sleep212780-21apt-get00:29:132
13090210611,81sleep212780-21apt-get00:29:132
1016321065,89sleep00-21swapper/020:51:590
12848210510,84sleep112582-21/usr/sbin/munin22:44:321
12848210510,84sleep112582-21/usr/sbin/munin22:44:321
32586210412,17sleep10-21swapper/123:49:051
984210312,50sleep10-21swapper/123:49:381
2139121030,7sleep00-21swapper/021:29:060
9507210212,72sleep10-21swapper/122:34:221
6759210134,33sleep30-21swapper/318:59:233
2941210010,75sleep03038-21cut20:29:260
24836210018,68sleep024916-21grep21:39:330
933729929,49sleep39401-21cat20:49:283
2203029911,74sleep10-21swapper/121:29:361
2203029911,74sleep10-21swapper/121:29:361
2042429912,22sleep220456-21ntp_states21:24:442
130732997,19sleep112780-21apt-get00:29:071
130732997,19sleep112780-21apt-get00:29:071
425429811,72sleep34374-21cat23:59:453
425429811,72sleep34374-21cat23:59:453
372429819,58sleep23804-21tune2fs23:59:322
372429819,58sleep23804-21tune2fs23:59:322
1547929816,12sleep10-21swapper/119:24:371
845629715,68sleep28495-21idleruntime19:04:342
845629715,68sleep28495-21idleruntime19:04:342
2449929714,67sleep224463-21latency_hist21:39:062
2729729624,42sleep327436-21tail23:29:453
1162929610,64sleep10-21swapper/119:14:071
1040529610,12sleep10-21swapper/100:19:341
1506829515,66sleep215164-21latency_hist21:09:062
12832946,10sleep10-21swapper/122:07:321
196522937,11sleep10-21swapper/121:23:301
196222939,11sleep10-21swapper/119:39:111
57412925,11sleep10-21swapper/122:19:441
440229249,10sleep10-21swapper/122:17:221
2348929225,40sleep323508-21apt-get23:19:193
2172529227,35sleep30-21swapper/323:14:063
1736229216,61sleep317371-21diskstats22:59:283
1736229216,61sleep317371-21diskstats22:59:283
178582915,11sleep10-21swapper/121:15:231
117292915,12sleep10-21swapper/120:56:451
316722905,71sleep00-21swapper/020:16:010
101542895,11sleep10-21swapper/120:51:521
224728819,53sleep02317-21wc22:09:370
207552886,11sleep10-21swapper/123:09:361
1652728824,48sleep316536-21/usr/sbin/munin19:29:233
40828711,59sleep225871-21kworker/2:022:04:312
1248828725,39sleep312628-21unixbench_multi19:14:373
680628617,31sleep30-21swapper/300:09:233
526428416,32sleep20-21swapper/200:04:252
2972928424,38sleep029724-21apt-get23:39:190
2972928424,38sleep029724-21apt-get23:39:190
1705928319,46sleep017141-21grep19:29:360
970328113,51sleep39754-21if_eth022:34:283
3113928121,25sleep20-21swapper/221:59:302
310772817,20sleep30-21swapper/321:59:273
2284028119,37sleep222832-21apt-get19:49:212
1968828110,58sleep219689-21sshd21:24:002
171032806,58sleep217149-21if_eth021:14:312
171032806,58sleep217149-21if_eth021:14:312
106428016,48sleep21187-21unixbench_multi23:49:392
713827927,33sleep07180-21wc18:59:320
1844127816,48sleep318470-21cpu21:19:243
3107627713,45sleep331113-21ntp_states20:14:383
1589227710,53sleep0372-21sshd19:25:370
1275127714,46sleep212812-21apt-config00:29:042
726727625,20sleep20-21swapper/218:59:342
29232759,47sleep20-21swapper/220:29:242
2496127510,48sleep025026-21swap19:54:360
2267427519,40sleep0232-21systemd-journal19:49:070
2670227420,24sleep20-21swapper/223:29:322
2258227417,43sleep322670-21latency_hist19:49:063
2250527419,27sleep20-21swapper/219:49:052
729027322,32sleep20-21swapper/200:09:382
2911127312,43sleep329137-21df20:09:213
1015827317,37sleep310208-21/usr/sbin/munin00:19:283
437727215,34sleep04505-21grep23:59:460
437727215,34sleep04505-21grep23:59:460
286622729,45sleep028700-21/usr/sbin/munin23:34:430
1859127215,41sleep018721-21cat23:04:050
1859127215,41sleep018721-21cat23:04:050
74827134,18sleep312-21rcu_sched23:49:323
1125727113,39sleep011342-21latency_hist00:24:060
1081927115,39sleep210899-21awk19:09:352
862227030,17sleep20-21swapper/222:29:472
862227030,17sleep20-21swapper/222:29:472
234826913,39sleep22384-21ntp_kernel_pll_23:54:352
1540926913,32sleep00-21swapper/022:54:040
814926814,22sleep30-21swapper/322:29:353
814926814,22sleep30-21swapper/322:29:353
2948926818,20sleep30-21swapper/323:39:063
2015726811,19sleep20-21swapper/219:39:442
798526719,29sleep18006-21munin-run19:04:051
3220226721,28sleep20-21swapper/220:19:222
2623926716,28sleep00-21swapper/019:59:290
2484326715,22sleep20-21swapper/223:24:062
2205326719,27sleep00-21swapper/023:14:270
2199726712,24sleep00-21swapper/021:29:350
2199726712,24sleep00-21swapper/021:29:350
2064326712,40sleep220656-21irqstats23:09:332
1783726712,42sleep0369-21snmpd21:15:050
1783726712,42sleep0369-21snmpd21:15:050
162112676,18sleep20-21swapper/219:29:062
2129426615,34sleep321386-21latency_hist21:29:063
877726521,27sleep38786-21modprobe20:49:033
491126515,36sleep04936-21grep22:19:270
2793726518,26sleep00-21swapper/021:49:300
1501726513,30sleep015047-21munin-run21:09:030
1281326522,22sleep30-21swapper/300:29:053
1065126512,38sleep010646-21apt-get20:54:250
53442648,19sleep00-21swapper/020:34:510
2915126412,37sleep229204-21fschecks_count20:09:242
2572926415,26sleep20-21swapper/219:59:072
2013826418,24sleep30-21swapper/323:09:053
1376926420,22sleep00-21swapper/019:19:390
1154326416,25sleep30-21swapper/322:39:453
1040526417,23sleep20-21swapper/220:54:072
320002638,13sleep20-21swapper/223:44:462
1082826312,35sleep310883-21cat20:54:333
1068026310,38sleep310681-21ntp_states19:09:323
849326213,25sleep20-21swapper/200:14:302
468826219,23sleep20-21swapper/220:34:282
3259926215,33sleep332600-21cpu22:04:233
2625826211,22sleep30-21swapper/321:44:163
187826218,24sleep00-21swapper/020:24:410
858226122,21sleep30-21swapper/300:14:313
643626120,20sleep20-21swapper/220:39:412
643626120,20sleep20-21swapper/220:39:412
637826118,27sleep36370-21apt-get22:24:273
388826112,22sleep20-21swapper/222:14:402
3106326113,33sleep331010-21latency_hist23:44:063
3106326113,33sleep331010-21latency_hist23:44:063
2813226116,27sleep00-21swapper/020:04:380
2770126114,21sleep20-21swapper/220:04:282
2489726115,29sleep324959-21/usr/sbin/munin19:54:353
2476826114,24sleep00-21swapper/023:24:040
2284126122,19sleep30-21swapper/319:49:223
2196626115,29sleep321962-21apt-get23:14:223
1499726017,22sleep20-21swapper/222:49:422
3145025918,20sleep00-21swapper/021:59:360
2843825925,16sleep30-21swapper/321:49:413
1531525916,26sleep215312-21apt-get21:09:262
1531525916,26sleep215312-21apt-get21:09:262
847525815,17sleep00-21swapper/022:29:450
847525815,17sleep00-21swapper/022:29:450
594525818,24sleep36068-21sed00:04:463
319525814,19sleep00-21swapper/023:58:410
2599425816,26sleep30-21swapper/321:44:043
2294925813,19sleep30-21swapper/321:34:063
2294925813,19sleep30-21swapper/321:34:063
1925025820,21sleep00-21swapper/023:04:380
151462585,11sleep30-21swapper/322:50:443
2574725717,22sleep20-21swapper/223:24:432
9132568,27sleep00-21swapper/023:49:370
717425619,18sleep00-21swapper/000:09:320
556725615,24sleep25682-21/usr/sbin/munin22:19:412
3260825613,16sleep00-21swapper/020:19:320
3253525613,17sleep30-21swapper/323:49:033
3082325615,20sleep30-21swapper/321:59:063
3041025617,18sleep20-21swapper/223:39:372
3041025617,18sleep20-21swapper/223:39:372
2664925618,25sleep026655-21needreboot21:44:290
2095725622,18sleep30-21swapper/319:44:053
1610825614,26sleep316139-21/usr/sbin/munin22:54:393
1476225621,20sleep30-21swapper/321:05:513
1119825616,22sleep20-21swapper/200:23:592
604825515,24sleep36054-21df_inode20:39:303
604825515,24sleep36054-21df_inode20:39:303
3195925520,19sleep20-21swapper/220:19:062
2915725515,24sleep229166-21munin-run21:54:052
257162557,16sleep20-21swapper/221:40:322
223442559,33sleep222382-21/usr/sbin/munin23:14:332
1929925516,24sleep219300-21ntp_states23:04:402
1928425516,21sleep30-21swapper/319:39:043
1928425516,21sleep30-21swapper/319:39:043
980125411,16sleep00-21swapper/019:09:090
854425419,15sleep00-21swapper/019:04:370
854425419,15sleep00-21swapper/019:04:370
1784025413,26sleep217984-21cat19:34:072
1081625413,18sleep00-21swapper/022:39:060
848125313,24sleep08487-21diskstats00:14:280
835025315,22sleep38356-21diskstats19:04:313
835025315,22sleep38356-21diskstats19:04:313
413925313,24sleep34177-21munin-run20:34:053
2596625313,23sleep225964-21apt-get19:59:192
2530725313,23sleep025336-21irqstats23:24:330
1236425312,18sleep00-21swapper/022:44:060
1185825310,17sleep00-21swapper/019:14:220
1175025314,21sleep20-21swapper/222:39:482
9512528,15sleep00-21swapper/022:04:410
769725213,16sleep20-21swapper/222:29:062
751225211,25sleep07505-21apt-get20:44:200
571925213,17sleep30-21swapper/320:39:063
3073825210,25sleep230696-21apt-get21:59:052
2682625210,17sleep00-21swapper/023:29:340
235292529,16sleep00-21swapper/021:34:360
214812525,36sleep00-21swapper/023:11:570
1979125217,15sleep00-21swapper/019:39:340
1843825215,18sleep00-21swapper/021:19:210
1572125217,16sleep30-21swapper/321:09:393
1572125217,16sleep30-21swapper/321:09:393
1258325213,23sleep312601-21apt-get22:44:223
1258325213,23sleep312601-21apt-get22:44:223
1251425212,16sleep00-21swapper/020:59:340
1251425212,16sleep00-21swapper/020:59:340
1229025218,19sleep30-21swapper/320:59:293
1229025218,19sleep30-21swapper/320:59:293
120772529,16sleep20-21swapper/219:14:282
619525111,16sleep00-21swapper/022:24:070
458625115,17sleep00-21swapper/022:19:060
2631525110,16sleep30-21swapper/323:29:043
2628325111,23sleep226279-21apt-get21:44:192
1814625111,17sleep30-21swapper/321:19:043
1814625111,17sleep30-21swapper/321:19:043
1581425110,15sleep00-21swapper/022:54:320
2906425011,22sleep029059-21apt-get20:09:180
2677925011,14sleep30-21swapper/319:59:403
1297625018,19sleep00-21swapper/022:44:350
1297625018,19sleep00-21swapper/022:44:350
1287725014,21sleep212956-21cat22:44:332
1287725014,21sleep212956-21cat22:44:332
1621424913,15sleep30-21swapper/319:29:073
1464824910,23sleep214664-21munin-run19:24:052
1302424913,21sleep213108-21users20:59:452
1302424913,21sleep213108-21users20:59:452
6862485,31sleep30-21swapper/320:20:003
1312724813,18sleep30-21swapper/300:29:223
1312724813,18sleep30-21swapper/300:29:223
3136924715,17sleep00-21swapper/023:44:320
238524712,20sleep32420-21ntp_states23:54:363
1797924712,13sleep20-21swapper/222:59:432
1797924712,13sleep20-21swapper/222:59:432
652824615,17sleep00-21swapper/020:39:430
652824615,17sleep00-21swapper/020:39:430
2364024611,18sleep30-21swapper/321:34:393
1849224512,18sleep20-21swapper/219:34:352
1849224512,18sleep20-21swapper/219:34:352
294112449,18sleep00-21swapper/021:54:230
213232448,19sleep00-21swapper/019:44:270
167124415,14sleep20-21swapper/220:24:352
1589624410,15sleep00-21swapper/021:09:440
1589624410,15sleep00-21swapper/021:09:440
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional