You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-16 - 13:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Mon Jul 15, 2024 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
29827255356,418sleep329872-21switchtime08:42:523
662025356,497sleep31166-21snmpd07:32:283
6445242726,368sleep31166-21snmpd10:54:353
1445224106,392sleep314462-21unin-run07:56:353
10706238339,214sleep310709-21cron09:26:343
734236517,284sleep3789-21fschecks_time10:36:583
1459123417,298sleep314597-21ntpq09:37:123
4299232340,215sleep34333-21basename07:26:343
4299232340,215sleep34333-21basename07:26:343
2609523140,7sleep00-21swapper/008:32:050
368228316,253sleep31166-21snmpd08:52:293
27176225838,201sleep30-21swapper/310:17:023
786522579,49sleep30-21swapper/309:16:443
26459223737,131sleep30-21swapper/311:57:033
24443223033,135sleep324509-21unixbench_multi10:07:173
12672222644,134sleep312750-21fschecks_time09:31:593
7545222039,138sleep37720-21latency_hist09:16:363
21890221031,161sleep321997-21latency_hist10:01:363
4009220114,169sleep34093-21latency10:47:043
4009220114,169sleep34093-21latency10:47:043
5236219314,161sleep15369-21awk09:07:131
5236219314,161sleep15369-21awk09:07:131
27268217713,149sleep227274-21grep06:56:492
19936217727,130sleep119948-21fschecks_count11:36:541
19936217727,130sleep119948-21fschecks_count11:36:541
848421739,149sleep08511-21df11:01:510
2915217348,86sleep30-21swapper/307:21:373
18290216819,62sleep30-21swapper/309:49:153
11681216515,129sleep10-21swapper/107:47:041
11429216411,34sleep30-21swapper/309:27:103
14700216338,18sleep30-21swapper/311:21:373
14700216338,18sleep30-21swapper/311:21:373
15423216251,65sleep315486-21sendmail_mailst11:22:063
7174216014,130sleep27204-21bash07:33:032
2268216077,45sleep32342-21timerandwakeup12:22:103
5114215916,126sleep35152-21switchtime09:07:113
5114215916,126sleep35152-21switchtime09:07:113
1164321587,33sleep30-21swapper/307:46:503
2880721527,11sleep10-21swapper/112:02:301
29726215112,44sleep30-21swapper/310:26:363
2676215189,46sleep32702-21modprobe09:01:343
20936215017,112sleep30-21swapper/308:16:513
1041214918,53sleep01059-21awk10:37:070
30950214829,37sleep30-21swapper/307:07:073
22634214633,74sleep322709-21grep10:02:123
22634214633,74sleep322709-21grep10:02:123
31994214510,32sleep30-21swapper/310:32:053
4534214410,82sleep34604-21munin-run12:31:353
30269214331,47sleep330346-21diskstats07:06:393
30269214331,47sleep330346-21diskstats07:06:393
3690214214,115sleep23711-21processes12:27:102
25638214214,111sleep325693-21latency_hist08:31:373
16715213731,71sleep316219-21apt-get11:26:463
31146213627,36sleep30-21swapper/308:47:043
23740213613,99sleep20-21swapper/208:26:352
11939213312,54sleep312011-21fschecks_time11:11:563
16538213215,58sleep30-21swapper/308:02:093
9005213126,63sleep39033-21proc_pri07:37:413
1898321306,114sleep00-21swapper/008:10:300
32500212816,74sleep30-21swapper/307:12:043
2865921277,109sleep328676-21munin-run08:41:343
22654212734,45sleep30-21swapper/311:46:333
22654212734,45sleep30-21swapper/311:46:333
16733212623,83sleep016219-21apt-get11:26:590
2898212411,75sleep33030-21unixbench_multi10:42:113
1948212213,86sleep12087-21latency_hist10:41:361
14862212123,84sleep014906-21fschecks_time07:57:050
31221212026,74sleep031222-21cron10:31:330
350211919,87sleep0372-21ntpq07:12:130
29397211912,25sleep328643-21cyclictest12:06:443
19845211934,62sleep30-21swapper/311:36:483
19845211934,62sleep30-21swapper/311:36:483
10199211919,82sleep110210-21sort11:06:451
16972211615,84sleep117025-21latency_hist09:46:351
29676211318,74sleep129708-21munin-run10:26:341
7716211026,44sleep00-21swapper/007:33:260
17301211025,57sleep217337-21uniq09:46:522
10602210822,42sleep30-21swapper/311:07:033
3494210611,79sleep13636-21latency_hist10:46:371
27427210635,53sleep027451-21cut06:57:030
24611210630,42sleep324619-21grep11:51:543
18377210619,40sleep30-21swapper/311:31:513
30741210510,27sleep20-21swapper/208:46:512
20118210526,38sleep30-21swapper/308:12:223
19649210517,68sleep218996-21/usr/sbin/munin09:52:182
7875210418,34sleep00-21swapper/009:16:450
28153210319,64sleep128271-21grep08:37:201
23797210342,42sleep30-21swapper/308:26:363
10638210215,70sleep010678-21switchtime07:42:100
10638210215,70sleep010678-21switchtime07:42:100
6892210116,67sleep16863-21/usr/sbin/munin10:56:511
5157210117,70sleep05234-21timerandwakeup09:07:120
5157210117,70sleep05234-21timerandwakeup09:07:120
28115210112,73sleep028231-21latency_hist10:21:360
5286210015,69sleep05304-21apt-get10:51:490
41229934,45sleep30-21swapper/312:17:023
971929611,47sleep39765-21apt-config11:06:343
697829520,52sleep27017-21cut10:56:562
539729418,62sleep15432-21expr10:51:581
211229416,60sleep10-21swapper/108:57:161
1798129428,45sleep218010-21munin-plugin-st11:31:342
716429311,38sleep30-21swapper/310:57:013
61942937,11sleep10-21swapper/109:11:541
1856929317,60sleep118610-21sort11:31:571
1030229318,32sleep30-21swapper/307:42:013
1030229318,32sleep30-21swapper/307:42:013
3169529112,62sleep30-21swapper/312:12:133
283229016,58sleep22914-21cut07:21:362
459628815,56sleep14650-21sort07:26:521
229452889,36sleep30-21swapper/311:46:493
229452889,36sleep30-21swapper/311:46:493
1502428825,48sleep215051-21fschecks_count11:21:522
995728714,56sleep010006-21sendmail_mailst09:22:150
520828719,47sleep30-21swapper/307:27:193
447028716,58sleep04528-21cat07:26:390
447028716,58sleep04528-21cat07:26:390
1915328719,46sleep119221-21fschecks_time09:52:001
620128616,48sleep26260-21fschecks_time09:11:552
970028426,37sleep29714-21apt11:06:332
278792837,35sleep30-21swapper/312:02:003
178412837,31sleep30-21swapper/308:06:493
2923628210,58sleep210000-1kworker/2:1H08:42:262
665128125,27sleep20-21swapper/210:56:362
3034028126,37sleep30-21swapper/310:27:043
189952816,40sleep319019-21modprobe09:51:463
1836628119,45sleep10-21swapper/108:07:201
584828026,36sleep25850-21sshd07:31:342
2583828017,32sleep00-21swapper/011:56:350
1802428030,31sleep217772-21/usr/sbin/munin08:07:082
3076927927,32sleep20-21swapper/212:11:362
3069127920,43sleep030710-21ls12:11:340
2701727918,51sleep30-21swapper/308:35:293
2566327919,43sleep00-21swapper/010:12:070
1685827918,50sleep30-21swapper/309:45:443
164827912,53sleep01649-21cpu12:21:470
34482788,31sleep30-21swapper/309:02:103
2534827825,13sleep30-21swapper/310:11:583
2441427818,39sleep024469-21tune2fs08:27:140
2075027813,49sleep020757-21users11:37:130
2075027813,49sleep020757-21users11:37:130
1462727816,24sleep20-21swapper/209:37:152
85627716,26sleep00-21swapper/012:17:150
201927717,45sleep12022-21/usr/sbin/munin12:22:001
1632227623,28sleep10-21swapper/111:26:361
112927614,47sleep21153-21processes10:37:132
430927519,31sleep14454-21latency_hist09:06:361
2795427519,39sleep228015-21sensors_temp08:37:172
1783227514,42sleep017185-21/usr/sbin/munin09:47:100
3092427415,28sleep00-21swapper/008:46:570
3016227423,30sleep130176-21munin-node07:06:371
3016227423,30sleep130176-21munin-node07:06:371
2883227412,46sleep028845-21/usr/sbin/munin10:22:080
2553927421,29sleep10-21swapper/110:12:041
3071427330,21sleep00-21swapper/007:06:480
230827321,38sleep02389-21fschecks_time10:41:560
2783427220,33sleep027349-21apt-get12:01:440
240792728,44sleep00-21swapper/010:07:060
2121227212,42sleep121231-21sed08:17:051
1382727228,26sleep10-21swapper/107:52:211
689127119,29sleep00-21swapper/010:56:500
2628227116,35sleep10-21swapper/108:32:111
125427113,26sleep20-21swapper/207:16:352
112727121,26sleep00-21swapper/008:56:340
1133327012,44sleep111308-21latency_hist07:46:361
1133327012,44sleep111308-21latency_hist07:46:361
420126918,28sleep20-21swapper/210:47:112
420126918,28sleep20-21swapper/210:47:112
39826920,24sleep20-21swapper/210:36:362
183702697,14sleep20-21swapper/211:31:502
487826826,20sleep00-21swapper/007:27:080
3231626822,29sleep20-21swapper/212:16:362
2793426825,21sleep30-21swapper/306:57:433
2636126834,18sleep20-21swapper/208:32:172
177026824,22sleep20-21swapper/212:21:532
1575126818,34sleep015812-21fschecks_time09:41:540
662126718,32sleep06686-21mailstats09:12:090
2870226719,32sleep028785-21apt-config08:41:350
2432726715,28sleep10-21swapper/111:51:361
2267226723,28sleep122713-21/usr/sbin/munin08:22:061
1624626715,32sleep00-21swapper/011:26:350
1600926712,17sleep30-21swapper/308:01:363
1156726710,43sleep20-21swapper/207:46:452
1095026722,22sleep20-21swapper/211:07:142
301522669,12sleep20-21swapper/210:26:592
2648526618,33sleep026515-21ntpq11:57:040
2115826620,23sleep20-21swapper/211:41:352
2115826620,23sleep20-21swapper/211:41:352
1755226615,36sleep217530-21latency_hist08:06:362
2327826511,21sleep00-21swapper/008:22:250
1695226514,37sleep117021-21wc08:02:231
1376426512,34sleep213791-21/usr/sbin/munin11:17:042
1376426512,34sleep213791-21/usr/sbin/munin11:17:042
1233126516,26sleep00-21swapper/009:31:360
37526413,31sleep2410-21grep12:17:012
2710826422,22sleep10-21swapper/110:17:001
2522626417,22sleep10-21swapper/111:52:291
220012644,12sleep00-21swapper/010:01:370
220012644,12sleep00-21swapper/010:01:370
2077726415,27sleep00-21swapper/009:57:000
2069426419,31sleep020751-21latency_hist08:16:350
856326315,31sleep18640-21fschecks_time11:01:561
3271326316,23sleep20-21swapper/208:52:042
2843726319,21sleep20-21swapper/210:21:542
2270526320,25sleep10-21swapper/111:46:351
2270526320,25sleep10-21swapper/111:46:351
223632637,45sleep00-21swapper/011:42:500
223632637,45sleep00-21swapper/011:42:500
2095826319,22sleep10-21swapper/109:57:051
2733626218,20sleep10-21swapper/112:01:331
258012626,13sleep20-21swapper/210:12:132
1866926217,23sleep00-21swapper/011:32:000
1855826220,22sleep10-21swapper/109:51:361
1076026221,26sleep210862-21latency_hist09:26:372
920926119,18sleep20-21swapper/209:21:362
344226114,23sleep20-21swapper/210:46:342
2447226112,34sleep124508-21sort08:27:151
2427926120,21sleep00-21swapper/011:51:350
2427926120,21sleep00-21swapper/011:51:350
992026016,30sleep19923-21ntpq09:22:121
3065626019,19sleep10-21swapper/110:27:141
2735726017,25sleep227423-21/usr/sbin/munin10:17:102
1953826015,27sleep119601-21/usr/sbin/munin08:12:001
1902926014,25sleep00-21swapper/009:51:510
1260026014,24sleep10-21swapper/111:12:141
3046525919,23sleep00-21swapper/007:06:440
298325917,26sleep23011-21grep09:01:532
2607825918,25sleep226133-21sort11:56:502
2436825915,27sleep20-21swapper/210:07:162
221425920,24sleep22331-21uptime08:57:172
1225225920,23sleep20-21swapper/209:31:342
1032725924,20sleep00-21swapper/011:06:560
552625811,33sleep25528-21sshd09:08:152
552625811,33sleep25528-21sshd09:08:152
356525812,21sleep00-21swapper/009:02:150
319725817,25sleep03193-21apt-get12:26:480
3096125813,18sleep10-21swapper/108:46:581
2964525820,18sleep00-21swapper/008:42:450
2885825814,26sleep028852-21apt-get07:01:510
2745225820,18sleep10-21swapper/106:57:041
211925814,19sleep10-21swapper/107:17:191
1566325816,25sleep215656-21apt-get09:41:472
127592587,38sleep00-21swapper/011:12:170
3071425716,26sleep130768-21munin-run12:11:351
2219825712,32sleep222241-21unin-run08:21:342
169625719,19sleep00-21swapper/008:57:020
1673525717,25sleep016736-21ntp_states08:02:180
1071725720,18sleep20-21swapper/207:42:122
1071725720,18sleep20-21swapper/207:42:122
786325612,26sleep17379-21apt-get09:16:421
419425616,26sleep14200-21ntp_states10:47:101
419425616,26sleep14200-21ntp_states10:47:101
332825611,31sleep23382-21sed07:22:092
332825611,31sleep23382-21sed07:22:092
2917825611,23sleep10-21swapper/108:41:451
2064225616,24sleep220672-21munin-run08:16:332
1295725615,26sleep20-21swapper/209:32:062
3053825514,23sleep130611-21memory07:06:451
255182555,15sleep20-21swapper/208:31:282
2301825513,24sleep223025-21df_inode11:46:522
2301825513,24sleep223025-21df_inode11:46:522
1389525518,17sleep00-21swapper/009:36:360
1148925513,21sleep00-21swapper/007:46:390
3255925413,24sleep132552-21apt-get12:16:521
3231325411,25sleep132306-21apt-get08:51:501
3162425418,19sleep00-21swapper/010:31:540
2505325413,18sleep20-21swapper/211:52:252
2367725414,23sleep123669-21apt-get10:06:511
151502544,18sleep00-21swapper/009:38:380
1470525411,25sleep114699-21apt-get07:56:491
1439425414,18sleep10-21swapper/109:37:021
1304025418,23sleep112543-21/usr/sbin/munin09:32:091
793125322,16sleep10-21swapper/107:35:171
41762535,15sleep10-21swapper/107:24:591
41762535,15sleep10-21swapper/107:24:591
373525314,24sleep03729-21apt-get10:46:520
373525314,24sleep03729-21apt-get10:46:520
3137625311,28sleep130980-21/usr/sbin/munin12:12:031
938225212,23sleep29434-21users07:37:482
473225212,16sleep10-21swapper/112:31:371
2139425212,24sleep221388-21apt-get11:41:472
2139425212,24sleep221388-21apt-get11:41:472
149525215,22sleep01488-21apt-get07:16:500
847825120,16sleep10-21swapper/107:36:501
3163225116,19sleep10-21swapper/110:31:551
2253625119,19sleep10-21swapper/110:02:061
2253625119,19sleep10-21swapper/110:02:061
3145525013,22sleep031466-21ntp_states12:12:060
2131625011,23sleep021358-21ntpq08:17:080
855424913,20sleep08558-21df_inode07:36:530
3246824910,23sleep132475-21df_inode07:11:541
289392496,33sleep20-21swapper/212:04:052
2331724914,19sleep10-21swapper/111:47:031
2261524913,19sleep20-21swapper/210:02:112
2261524913,19sleep20-21swapper/210:02:112
2085224913,20sleep220859-21irqstats09:57:022
2934424813,20sleep229350-21ntp_states07:02:262
2934424813,20sleep229350-21ntp_states07:02:262
1338124812,21sleep113384-21apt-get11:16:461
1338124812,21sleep113384-21apt-get11:16:461
1478624713,18sleep214790-21diskstats07:56:542
3207824613,17sleep20-21swapper/207:11:342
1674024611,23sleep116693-21/usr/sbin/munin11:27:041
136042469,22sleep213620-21ntp_states07:52:142
1347124611,19sleep00-21swapper/011:16:530
1347124611,19sleep00-21swapper/011:16:530
134072459,20sleep013434-21irqstats07:52:080
1215124514,16sleep10-21swapper/109:30:151
1314624411,19sleep20-21swapper/211:16:352
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional