You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-19 - 04:09

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Fri Apr 19, 2024 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21450247714,444sleep021584-21unixbench_singl22:12:110
2941242421,384sleep23032-21/usr/sbin/munin19:32:182
18975234131,29sleep00-21swapper/023:47:050
16593233833,192sleep016596-21cron20:16:380
21620231412,284sleep121667-40ogrotate23:56:391
21620231412,284sleep121667-40ogrotate23:56:391
21265229520,258sleep221287-21awk20:31:392
13625228916,253sleep213756-21grep21:47:102
13175228717,253sleep213267-21/usr/sbin/munin23:27:242
30895228112,253sleep330939-21unin-run21:01:403
7315225610,233sleep07366-21sort19:46:540
30457225544,144sleep00-21swapper/000:22:270
7518225111,228sleep00-21swapper/023:12:000
7518225111,228sleep00-21swapper/023:12:000
23509225013,178sleep00-21swapper/000:02:030
217742243105,27sleep021805-21sed23:56:420
217742243105,27sleep021805-21sed23:56:420
9652224299,22sleep00-21swapper/021:36:420
9652224299,22sleep00-21swapper/021:36:420
2879222915,197sleep02618-21/usr/sbin/munin19:32:160
27941222014,189sleep328016-21/usr/sbin/munin22:32:133
11498221626,172sleep0373-21sshd19:59:020
8039221527,171sleep18116-21timerandwakeup23:12:131
8039221527,171sleep18116-21timerandwakeup23:12:131
29731221542,121sleep029867-21latency_hist00:21:420
28694221595,76sleep028697-21cron22:36:390
19880219966,83sleep020020-21unixbench_multi22:07:150
3095621969,146sleep031000-21ntp_offset22:42:040
5759218050,24sleep00-21swapper/021:22:120
5759218050,24sleep00-21swapper/021:22:120
28423216727,125sleep028475-21vmstat19:11:450
9933216417,129sleep39965-21fschecks_count21:36:573
3144216229,119sleep03169-21awk22:57:020
27025216122,41sleep327184-21latency_hist22:31:413
31609215937,67sleep00-21swapper/021:02:050
1498215752,64sleep00-21swapper/000:32:120
2173214944,49sleep00-21swapper/021:11:590
2173214944,49sleep00-21swapper/021:11:590
1176214710,27sleep00-21swapper/022:51:490
31831214315,109sleep00-21swapper/019:21:550
2543321437,99sleep025582-21sed22:26:410
15843214110,116sleep115446-21/usr/sbin/munin23:37:051
11494213956,60sleep00-21swapper/021:41:590
1810221376,118sleep00-21swapper/023:44:270
155602136104,19sleep015607-21fschecks_count23:37:000
9969213339,62sleep00-21swapper/021:36:580
885721318,82sleep00-21swapper/019:51:510
9611213024,85sleep39650-21latency_hist21:36:413
9611213024,85sleep39650-21latency_hist21:36:413
2804721299,84sleep028065-21kernelversion20:52:000
1762212933,57sleep00-21swapper/021:11:380
30943212710,101sleep031062-21apt-config21:01:410
13023212710,65sleep00-21swapper/021:46:560
5531212460,48sleep00-21swapper/019:41:410
31363212325,80sleep130878-21apt-get21:01:501
19769212320,87sleep319905-21date20:26:433
2363021229,87sleep00-21swapper/022:21:400
26423212045,46sleep026482-21date19:06:400
1136212044,60sleep01220-21cat19:27:080
1136212044,60sleep01220-21cat19:27:080
10194212020,83sleep110224-21latency_hist19:56:401
3606211910,94sleep13625-21apt-get21:16:481
7142211728,65sleep37166-21modprobe23:11:383
2275121176,12sleep00-21swapper/020:35:530
13540211715,82sleep113624-21awk21:47:091
1297221175,10sleep00-21swapper/020:03:060
7470211623,73sleep17588-21sed21:27:101
24107211614,82sleep224075-21apt-get22:22:072
2615521155,11sleep00-21swapper/000:09:420
32195211421,70sleep332275-21fschecks_time22:46:593
30739211220,70sleep130876-21head00:22:321
29201211215,81sleep129307-21date20:56:411
2026211233,41sleep30-21swapper/321:11:503
2026211233,41sleep30-21swapper/321:11:503
26607211139,21sleep00-21swapper/000:11:570
11961211121,75sleep312053-21latency_hist23:26:413
2559721105,10sleep00-21swapper/020:42:190
1491721105,11sleep00-21swapper/020:10:040
1491721105,11sleep00-21swapper/020:10:040
1348921106,70sleep0869-21sendmail-mta23:30:050
23131210915,81sleep123166-21grep20:36:551
12061210810,41sleep012263-21sed23:26:420
9293210712,44sleep00-21swapper/023:17:050
9293210712,44sleep00-21swapper/023:17:050
685621065,89sleep00-21swapper/019:44:280
28228210613,13sleep20-21swapper/222:32:172
1397521067,53sleep00-21swapper/023:31:550
4326210529,21sleep04328-21apt-get23:01:500
25781210432,54sleep225940-21unixbench_multi19:02:112
26404210314,64sleep126543-21grep22:27:151
23595210349,16sleep00-21swapper/020:37:100
20452210349,38sleep00-21swapper/020:27:040
19714210345,19sleep019747-21munin-run20:26:400
29106210144,18sleep029193-21perl22:36:590
2601021019,57sleep026035-21awk22:27:060
1496021018,50sleep00-21swapper/021:52:090
11870210115,69sleep211958-21apt-config23:26:402
6818210021,56sleep00-21swapper/021:26:570
28007210015,63sleep128133-21grep19:11:421
28007210015,63sleep128133-21grep19:11:421
430829914,64sleep14321-21fschecks_count19:36:581
2919429914,22sleep10-21swapper/122:37:001
2749529833,25sleep00-21swapper/020:51:390
135429826,43sleep31492-21grep00:32:113
1027529815,65sleep110276-21cron23:21:381
1027529815,65sleep110276-21cron23:21:381
2910229615,61sleep328601-21/usr/sbin/munin19:12:083
277082965,80sleep00-21swapper/019:08:380
277082965,80sleep00-21swapper/019:08:380
1667229516,63sleep216729-21latency_hist20:16:402
315232949,62sleep00-21swapper/019:21:420
146902940,6sleep30-21swapper/323:32:093
8632939,69sleep30-21swapper/319:26:583
8632939,69sleep30-21swapper/319:26:583
9612917,15sleep20-21swapper/221:07:082
67972917,52sleep0869-21sendmail-mta23:07:150
315212918,19sleep00-21swapper/000:26:510
226029116,60sleep22453-21latency_hist19:31:422
226029116,60sleep22453-21latency_hist19:31:422
1789429013,65sleep117965-21latency22:02:031
643928921,48sleep26461-21munin-run21:26:382
643928921,48sleep26461-21munin-run21:26:382
132628818,57sleep11335-21open_files19:27:131
132628818,57sleep11335-21open_files19:27:131
1103728810,62sleep111090-21processes23:22:061
719428735,33sleep20-21swapper/223:11:422
1324828715,55sleep213266-21munin-plugin-st20:06:392
303328616,36sleep30-21swapper/319:32:193
288152866,13sleep20-21swapper/219:11:592
2240828619,49sleep222477-21fschecks_time22:16:582
208552866,62sleep30-21swapper/323:52:333
208552866,62sleep30-21swapper/323:52:333
2648328516,54sleep126640-21latency_hist19:06:411
101028417,28sleep10-21swapper/121:07:111
3147028323,38sleep131484-21modprobe19:21:391
2258328225,36sleep00-21swapper/023:57:350
1799328222,32sleep30-21swapper/322:02:053
337928018,37sleep30-21swapper/322:57:093
1398028022,39sleep114014-21fschecks_count23:31:561
305842790,10sleep30-21swapper/319:17:043
1031227921,44sleep110325-21/usr/sbin/munin21:37:071
2377427820,33sleep30-21swapper/300:02:143
2182427814,50sleep221882-21sed20:32:032
448427733,30sleep00-21swapper/021:17:120
176352779,54sleep017775-21unixbench_singl20:17:150
3210227620,13sleep00-21swapper/022:46:520
2085027611,52sleep220854-21ntpq23:52:322
2085027611,52sleep220854-21ntpq23:52:322
180262767,13sleep00-21swapper/022:02:060
588227523,23sleep10-21swapper/119:41:591
1962727512,30sleep20-21swapper/222:07:092
940627417,24sleep20-21swapper/219:52:092
627027415,27sleep30-21swapper/319:42:093
567927418,27sleep30-21swapper/321:22:113
567927418,27sleep30-21swapper/321:22:113
29352740,7sleep10-21swapper/119:32:161
1944227423,25sleep10-21swapper/122:07:041
1651127410,13sleep20-21swapper/221:57:072
1651127410,13sleep20-21swapper/221:57:072
1647227424,28sleep10-21swapper/121:57:061
1647227424,28sleep10-21swapper/121:57:061
454227318,30sleep30-21swapper/323:02:013
2666227318,36sleep226680-21apt-get19:06:502
2666227318,36sleep226680-21apt-get19:06:502
2635827325,28sleep326419-21systemd-run19:06:393
2539627324,25sleep30-21swapper/322:26:403
2397527311,48sleep11166-21snmpd00:02:211
1461227321,33sleep314695-21tune2fs21:51:593
2221027211,28sleep30-21swapper/320:32:123
153722720,8sleep10-21swapper/121:52:131
3136527118,27sleep20-21swapper/221:01:512
2860827114,42sleep128644-21ntp_states00:17:061
1723227131,29sleep00-21swapper/022:00:200
1723227131,29sleep00-21swapper/022:00:200
1449327117,25sleep20-21swapper/221:51:542
386827010,47sleep10-21swapper/122:59:251
2919927010,45sleep329200-21sshd00:18:303
2356927016,36sleep30-21swapper/322:21:393
1738427022,30sleep30-21swapper/322:01:403
1738427022,30sleep30-21swapper/322:01:403
2485826915,23sleep10-21swapper/122:22:301
169032697,15sleep10-21swapper/120:16:521
1508826914,26sleep30-21swapper/320:11:413
1508826914,26sleep30-21swapper/320:11:413
1271026914,39sleep212841-21latency_hist21:46:412
2372026819,32sleep123862-21apt-config22:21:411
1893326817,25sleep20-21swapper/222:06:402
1827726821,24sleep20-21swapper/223:46:392
1659826817,35sleep116671-21systemd20:16:391
128926818,28sleep10-21swapper/122:51:581
554726719,35sleep15560-21grep21:22:051
554726719,35sleep15560-21grep21:22:051
2757926718,33sleep227729-21latency_hist20:51:422
91426627,23sleep20-21swapper/222:51:392
729226617,36sleep37360-21sed21:27:083
3116426611,42sleep231268-21timerandwakeup22:42:102
3109626619,23sleep10-21swapper/122:42:091
1587526614,34sleep215910-21ntp_kernel_pll_23:37:062
1148226615,32sleep311492-21/usr/sbin/munin21:41:583
495126522,29sleep25055-21ls23:02:132
2701626520,28sleep10-21swapper/100:12:221
2040826518,27sleep30-21swapper/320:27:033
1978826518,22sleep30-21swapper/322:07:143
869726418,29sleep28724-21munin-run23:16:382
869726418,29sleep28724-21munin-run23:16:382
777726420,27sleep27814-21ps19:47:082
459226418,26sleep30-21swapper/319:37:043
312692647,12sleep30-21swapper/322:42:113
3107226418,30sleep131279-21latency_hist21:01:421
2872526421,24sleep1288-21dbus-daemon22:36:401
2694926420,25sleep20-21swapper/222:31:402
2684026417,31sleep126934-21sed20:47:081
2071926414,26sleep20-21swapper/222:11:532
1327326421,23sleep30-21swapper/320:06:403
321812636,15sleep10-21swapper/122:46:571
3202626325,22sleep10-21swapper/119:22:001
2904126318,25sleep329108-21http20:56:393
2504726315,18sleep10-21swapper/100:06:531
2206626317,20sleep10-21swapper/122:16:411
1902726316,28sleep20-21swapper/223:47:072
1865326323,20sleep10-21swapper/123:46:581
1207726310,16sleep10-21swapper/120:01:581
615026215,21sleep10-21swapper/123:07:001
37726213,35sleep1490-21latency_hist00:31:401
2847726223,21sleep10-21swapper/119:11:461
2715926211,20sleep20-21swapper/200:12:272
2687426217,24sleep30-21swapper/319:07:003
2687426217,24sleep30-21swapper/319:07:003
2598026212,35sleep225981-21sshd00:07:302
1862126213,35sleep218680-21idleruntime20:21:572
15722625,13sleep30-21swapper/322:52:043
82752615,37sleep10-21swapper/121:31:541
82752615,37sleep10-21swapper/121:31:541
61826126,19sleep20-21swapper/219:26:412
2567226116,20sleep30-21swapper/300:07:193
2539126118,26sleep325502-21sed20:42:143
2930826021,16sleep20-21swapper/220:56:422
2358726010,30sleep20-21swapper/200:02:092
2211426022,23sleep30-21swapper/323:57:223
1739126017,26sleep10-21swapper/123:42:051
1718626012,30sleep217279-21memory20:17:052
123526019,19sleep20-21swapper/200:32:102
104382608,26sleep30-21swapper/319:56:553
89325913,29sleep1987-21memory00:32:011
339225919,24sleep10-21swapper/121:16:401
339225919,24sleep10-21swapper/121:16:401
304372597,43sleep20-21swapper/200:22:162
1853825919,23sleep10-21swapper/120:21:561
1401925911,36sleep314021-21sshd21:49:173
862325818,20sleep30-21swapper/319:51:423
42152586,14sleep20-21swapper/219:36:512
2955525819,22sleep30-21swapper/322:37:103
2529525821,19sleep20-21swapper/220:42:132
2325925820,23sleep20-21swapper/200:01:412
2136625816,22sleep30-21swapper/322:12:103
1893725815,26sleep318970-21/usr/sbin/munin23:47:043
1729025821,19sleep30-21swapper/323:42:033
1523625821,21sleep215277-21date23:36:412
77702576,34sleep10-21swapper/119:47:071
2707925710,30sleep327085-21ntp_states00:12:243
2333925717,18sleep30-21swapper/320:37:013
1236525715,27sleep312337-21apt-get23:27:053
1153725711,31sleep111539-21sshd19:59:311
400025614,27sleep24027-21awk21:17:012
613725515,26sleep26168-21/usr/sbin/munin19:42:042
460725520,16sleep10-21swapper/123:02:041
3170425514,21sleep30-21swapper/300:26:583
2901025515,23sleep229006-21apt-get22:36:532
117725522,16sleep30-21swapper/321:07:133
3223125418,21sleep331732-21/usr/sbin/munin19:22:063
3139125415,24sleep331385-21apt-get21:01:523
1043725412,25sleep210432-21apt-get19:56:542
93422537,34sleep20-21swapper/221:34:142
93422537,34sleep20-21swapper/221:34:142
909025314,21sleep29099-21df_inode23:16:592
909025314,21sleep29099-21df_inode23:16:592
721525318,19sleep20-21swapper/221:27:072
3085625315,18sleep20-21swapper/219:17:122
256152538,10sleep10-21swapper/120:42:321
2000525312,26sleep120072-21df20:26:531
1380125315,23sleep113850-21cpuspeed20:07:111
1380125315,23sleep113850-21cpuspeed20:07:111
100222536,37sleep30-21swapper/323:18:253
100222536,37sleep30-21swapper/323:18:253
3243525214,22sleep20-21swapper/200:27:132
3008525215,22sleep30-21swapper/300:22:093
2824325213,20sleep20-21swapper/220:52:062
1673125115,17sleep30-21swapper/320:16:413
1540525116,20sleep30-21swapper/320:11:573
1231625111,23sleep111837-21apt-get23:26:471
982225017,15sleep20-21swapper/221:36:482
262202509,23sleep326217-21apt-get20:46:503
2316725012,23sleep223202-21/usr/sbin/munin20:36:562
125012505,34sleep10-21swapper/121:44:381
2951324915,19sleep20-21swapper/200:21:402
274924920,16sleep20-21swapper/222:56:482
165824912,14sleep20-21swapper/222:52:072
947424813,18sleep30-21swapper/319:52:103
2155324812,19sleep10-21swapper/120:31:541
205024811,21sleep22044-21apt-get21:11:532
205024811,21sleep22044-21apt-get21:11:532
1385124813,16sleep20-21swapper/220:07:122
1385124813,16sleep20-21swapper/220:07:122
1591124710,14sleep30-21swapper/323:37:073
76792469,14sleep30-21swapper/323:12:033
76792469,14sleep30-21swapper/323:12:033
226642439,19sleep10-21swapper/123:57:361
37012406,17sleep30-21swapper/321:16:543
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional