You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-06-15 - 12:55

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sat Jun 15, 2024 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
252332157111,1543sleep125226-21apt-get23:13:381
17135280317,768sleep117144-21/usr/sbin/munin19:14:031
11039269918,664sleep311075-21/usr/sbin/munin22:28:413
15634266010,638sleep014149-21kworker/0:200:25:590
14652265452,584sleep214687-21fschecks_count00:23:422
565126050,21sleep212486-21cobalt_printf20:28:392
980326036,585sleep29830-21ntpq22:23:512
786324420,8sleep10-21swapper/122:18:361
31412243818,407sleep215276-1kworker/2:2H23:33:262
6504233414,306sleep16540-21unin-run23:58:211
2373023188,300sleep130577-1kworker/1:0H23:08:411
24972226966,181sleep2538-21cron19:48:212
9584224656,40sleep29607-21modprobe00:08:202
9584224656,40sleep29607-21modprobe00:08:202
26159224473,16sleep20-21swapper/219:48:572
190602242109,34sleep218732-21/usr/sbin/munin22:53:482
21859224031,17sleep20-21swapper/223:03:372
21859224031,17sleep20-21swapper/223:03:372
1492622160,15sleep212-21rcu_sched22:38:562
2099220431,150sleep20-21swapper/221:58:552
2099220431,150sleep20-21swapper/221:58:552
26578220139,23sleep20-21swapper/223:18:242
20234220143,55sleep20-21swapper/221:13:432
14182219914,170sleep214265-21switchtime20:53:562
611219836,21sleep20-21swapper/221:53:542
611219836,21sleep20-21swapper/221:53:542
2610219539,22sleep20-21swapper/220:18:422
13096219217,156sleep20-21swapper/200:18:382
1293218914,164sleep21325-21ntpq20:13:492
30659218212,51sleep230682-21modprobe21:48:212
21990218111,19sleep20-21swapper/219:33:512
13287218118,138sleep20-21swapper/219:03:542
370721797,12sleep00-21swapper/023:48:390
1274621797,161sleep212762-21sort22:33:442
1528621709,148sleep215400-21sed19:08:552
364221669,17sleep20-21swapper/223:48:372
4706216430,75sleep24832-21/usr/sbin/munin20:24:032
18235216042,56sleep20-21swapper/219:18:522
16313215917,123sleep216333-21fschecks_count19:13:432
16938215812,125sleep20-21swapper/222:48:222
2318321569,40sleep20-21swapper/221:23:372
11856215513,128sleep111876-21ntp_kernel_pll_00:13:481
11856215513,128sleep111876-21ntp_kernel_pll_00:13:481
30406215410,102sleep20-21swapper/223:28:542
12235215219,99sleep20-21swapper/220:48:432
12235215219,99sleep20-21swapper/220:48:432
27202215116,100sleep227207-21grep19:53:462
13999215118,116sleep314020-21ntp_kernel_pll_20:53:493
1742721500,10sleep00-21swapper/000:33:210
1857214410,73sleep22005-21date23:43:242
2973521427,102sleep21166-21snmpd20:01:012
31814213620,75sleep20-21swapper/220:08:362
2156821358,68sleep221617-21systemd-run23:03:212
31035213394,18sleep20-21swapper/221:48:412
16948213210,107sleep216945-21apt-get21:03:332
9936213018,49sleep29965-21munin-run20:43:222
22180213014,50sleep20-21swapper/221:18:522
3585212815,54sleep23608-21ntp_kernel_pll_22:03:522
3585212815,54sleep23608-21ntp_kernel_pll_22:03:522
29918212422,79sleep029937-21modprobe20:03:210
8389212312,48sleep20-21swapper/220:38:212
2551212319,84sleep10-21swapper/120:18:411
12795212319,80sleep312939-21latency_hist00:18:223
12795212319,80sleep312939-21latency_hist00:18:223
10608212313,46sleep20-21swapper/220:43:582
10608212313,46sleep20-21swapper/220:43:582
9500212218,86sleep00-21swapper/022:23:450
25318212211,88sleep20-21swapper/221:28:582
23694212112,89sleep123157-21/usr/sbin/munin21:23:561
1964421188,99sleep21166-21snmpd21:11:452
11678211811,93sleep211069-21/usr/sbin/munin18:58:542
11678211811,93sleep211069-21/usr/sbin/munin18:58:542
4726211636,51sleep14720-21apt-get22:08:351
534421159,44sleep20-21swapper/222:08:572
3020621158,39sleep20-21swapper/220:03:352
3020621158,39sleep20-21swapper/220:03:352
27614211511,64sleep20-21swapper/221:38:232
1619321129,88sleep116229-21/usr/sbin/munin00:28:381
15533211218,28sleep20-21swapper/220:58:532
15005211213,80sleep115011-21/usr/sbin/munin19:08:451
25352211113,82sleep125419-21fschecks_time19:48:401
23943211120,66sleep024069-21unixbench_singl21:23:590
19182211013,81sleep019277-21timerwakeupswit21:08:570
8304210914,27sleep10-21swapper/100:03:341
8304210914,27sleep10-21swapper/100:03:341
29162210913,74sleep20-21swapper/221:43:242
16230210819,30sleep20-21swapper/200:28:392
1276621088,62sleep212794-21date00:18:212
1276621088,62sleep212794-21date00:18:212
11220210812,80sleep011264-21wc00:13:230
11220210812,80sleep011264-21wc00:13:230
677821078,36sleep20-21swapper/223:58:342
677821078,36sleep20-21swapper/223:58:342
2128021079,37sleep20-21swapper/219:33:222
6200210614,77sleep06314-21timerwakeupswit20:28:540
32282210618,64sleep332226-21latency_hist21:53:223
10912210618,30sleep20-21swapper/200:10:262
10912210618,30sleep20-21swapper/200:10:262
2917421057,35sleep20-21swapper/223:23:572
1146210518,71sleep31163-21cat23:38:523
17579210430,62sleep217604-21ntpq22:48:472
280332996,32sleep20-21swapper/223:23:202
72622985,28sleep20-21swapper/222:14:262
577129717,16sleep20-21swapper/223:53:542
3249929713,59sleep10-21swapper/121:53:331
3249929713,59sleep10-21swapper/121:53:331
139329713,70sleep31505-21cat21:58:233
139329713,70sleep31505-21cat21:58:233
2586629612,66sleep325938-21/usr/sbin/munin19:48:543
721629514,66sleep17239-21cut20:33:391
2423429529,48sleep26000-21kworker/u8:323:09:032
2704529416,61sleep026541-21apt-get19:53:320
2008829414,26sleep00-21swapper/021:13:370
113252946,29sleep20-21swapper/222:28:482
267529311,63sleep20-21swapper/223:43:562
267529311,63sleep20-21swapper/223:43:562
224442938,15sleep30-21swapper/319:34:003
193392936,30sleep20-21swapper/219:24:092
2510529216,59sleep324705-21/usr/sbin/munin21:28:513
1884429213,60sleep118851-21/usr/sbin/munin22:53:431
1692529020,51sleep016432-21apt-get21:03:310
883728918,34sleep20-21swapper/222:19:472
168892895,14sleep00-21swapper/022:48:160
1738728810,62sleep317408-21sed21:04:113
1146428623,45sleep011495-21/usr/sbin/munin18:58:480
1146428623,45sleep011495-21/usr/sbin/munin18:58:480
215252857,17sleep10-21swapper/119:33:331
681328316,51sleep36875-21modprobe20:33:213
3205828314,48sleep132129-21threads23:33:581
1241028320,43sleep312456-21latency_hist19:03:223
1241028320,43sleep312456-21latency_hist19:03:223
1009528219,41sleep00-21swapper/020:43:240
2868928113,49sleep328700-21netstat23:23:463
1928628119,31sleep10-21swapper/121:08:581
2544027913,49sleep025489-21if_eth023:13:450
2231027916,28sleep00-21swapper/021:18:540
91527820,29sleep10-21swapper/120:13:361
364927814,44sleep13159-21/usr/sbin/munin22:03:551
364927814,44sleep13159-21/usr/sbin/munin22:03:551
3061127812,46sleep130731-21/usr/sbin/munin23:28:571
2808627815,39sleep00-21swapper/023:23:220
1322127811,54sleep01166-21snmpd20:52:330
1322127811,54sleep01166-21snmpd20:52:330
783027717,37sleep30-21swapper/322:18:313
2311527718,27sleep20-21swapper/219:39:242
361427614,49sleep03615-21sshd20:21:590
2857027614,28sleep00-21swapper/023:23:430
282422765,11sleep30-21swapper/319:57:023
2180027613,45sleep021886-21sed19:33:450
3256227513,27sleep00-21swapper/020:08:550
1408327513,31sleep30-21swapper/322:38:343
1045427520,34sleep310452-21apt-get20:43:313
1045427520,34sleep310452-21apt-get20:43:313
2948427416,29sleep10-21swapper/121:43:391
2161827412,45sleep321672-21latency_hist23:03:223
222527316,39sleep32187-21latency_hist20:18:233
157112738,53sleep01166-21snmpd19:10:510
1299827312,43sleep313002-21ntpq22:33:533
2966927210,47sleep129820-21latency_hist23:28:241
190727221,21sleep30-21swapper/321:58:483
190727221,21sleep30-21swapper/321:58:483
1339327213,40sleep313528-21/usr/sbin/munin19:03:553
914927116,32sleep10-21swapper/120:38:521
431927117,40sleep14393-21grep23:48:561
2990727016,24sleep00-21swapper/023:28:350
253452709,12sleep00-21swapper/019:48:390
2403327017,39sleep00-21swapper/023:08:560
1281327011,12sleep10-21swapper/120:48:581
1281327011,12sleep10-21swapper/120:48:581
736826913,30sleep00-21swapper/020:33:420
217926920,31sleep32190-21/usr/sbin/munin23:43:413
217926920,31sleep32190-21/usr/sbin/munin23:43:413
1717026917,34sleep116926-21/usr/sbin/munin21:04:051
677726811,27sleep10-21swapper/123:58:331
677726811,27sleep10-21swapper/123:58:331
1804126813,39sleep018088-21users22:48:570
1359926826,23sleep10-21swapper/100:18:531
398626718,32sleep13988-21/usr/sbin/munin20:23:341
3071126712,17sleep00-21swapper/021:48:230
2667626715,21sleep30-21swapper/321:33:523
340226614,26sleep00-21swapper/023:48:230
340226614,26sleep00-21swapper/023:48:230
294126615,24sleep10-21swapper/122:03:221
294126615,24sleep10-21swapper/122:03:221
1631526617,23sleep30-21swapper/300:28:403
1573026619,21sleep10-21swapper/122:43:451
761926517,32sleep37718-21latency_hist22:18:233
2368126512,37sleep123702-21dpkg19:43:271
2361426517,28sleep323131-21apt-get23:08:313
2324826519,22sleep30-21swapper/323:08:233
2324826519,22sleep30-21swapper/323:08:233
176442659,12sleep10-21swapper/122:48:501
1408626514,23sleep10-21swapper/122:38:361
1328126518,30sleep313337-21sendmail-msp20:53:203
1328126518,30sleep313337-21sendmail-msp20:53:203
1126526520,26sleep10-21swapper/100:13:241
79422647,14sleep00-21swapper/022:18:400
457126414,29sleep04631-21/usr/sbin/munin20:24:010
3122526413,21sleep00-21swapper/021:48:470
3068626412,38sleep330709-21munin-run21:48:223
272612648,45sleep11166-21snmpd21:34:301
498526322,19sleep30-21swapper/323:53:223
246272635,17sleep30-21swapper/319:43:583
2089026320,21sleep10-21swapper/122:58:561
123692638,13sleep00-21swapper/000:13:590
123692638,13sleep00-21swapper/000:13:590
563826221,21sleep10-21swapper/120:28:381
448826217,32sleep04545-21latency_hist22:08:220
448826217,32sleep04545-21latency_hist22:08:220
2190226222,23sleep10-21swapper/121:18:431
2181426219,22sleep30-21swapper/321:18:413
1581926214,30sleep315833-21apt00:28:203
1393926210,36sleep113967-21munin-run19:08:211
912026115,24sleep00-21swapper/000:04:020
912026115,24sleep00-21swapper/000:04:020
2987026121,19sleep30-21swapper/323:28:343
2533126126,15sleep30-21swapper/323:13:443
2483026118,27sleep00-21swapper/021:28:440
1858326122,21sleep30-21swapper/321:08:393
155462616,11sleep10-21swapper/100:24:511
129472615,12sleep10-21swapper/122:33:511
67162606,11sleep10-21swapper/122:13:481
2910926016,27sleep329123-21idleruntime-cro21:43:213
2347526017,24sleep10-21swapper/119:43:231
2045726015,27sleep320480-21/usr/sbin/munin21:13:483
1300526013,32sleep013007-21ntp_states22:33:550
1046726017,24sleep10-21swapper/120:43:371
1046726017,24sleep10-21swapper/120:43:371
607325918,25sleep06214-21cut22:13:230
2909225917,22sleep10-21swapper/119:58:501
274812596,10sleep00-21swapper/021:37:160
226022599,24sleep10-21swapper/123:03:571
226022599,24sleep10-21swapper/123:03:571
2206925919,16sleep00-21swapper/023:03:430
2206925919,16sleep00-21swapper/023:03:430
1837925912,28sleep10-21swapper/119:18:551
1000425922,20sleep30-21swapper/320:43:233
94222585,11sleep30-21swapper/300:06:153
94222585,11sleep30-21swapper/300:06:153
53725814,27sleep1532-21apt-get23:38:341
530325816,25sleep35307-21diskstats23:53:393
450925815,23sleep30-21swapper/320:24:003
328225818,24sleep30-21swapper/322:03:453
328225818,24sleep30-21swapper/322:03:453
987025716,25sleep39864-21apt-get00:08:333
987025716,25sleep39864-21apt-get00:08:333
977925716,22sleep10-21swapper/122:23:501
3812576,11sleep30-21swapper/320:10:393
35872574,39sleep30-21swapper/320:21:383
3142125714,21sleep00-21swapper/023:33:320
306112576,11sleep10-21swapper/120:03:511
306112576,11sleep10-21swapper/120:03:511
25882575,10sleep00-21swapper/021:59:290
25882575,10sleep00-21swapper/021:59:290
2342225716,26sleep323454-21munin-run19:43:213
1491525716,24sleep00-21swapper/020:58:220
1341625717,26sleep313456-21awk00:18:473
676225613,25sleep06774-21apt-get23:58:320
676225613,25sleep06774-21apt-get23:58:320
521825614,24sleep05215-21apt-get23:53:320
2707725613,26sleep127071-21apt-get19:53:371
209122565,40sleep10-21swapper/119:29:301
2076125615,25sleep020767-21ntp_states22:58:510
1153925610,31sleep110928-21/usr/sbin/munin22:28:551
1045725613,26sleep09953-21apt-get20:43:320
1045725613,26sleep09953-21apt-get20:43:320
631025515,24sleep06304-21apt-get22:13:350
1726725514,22sleep30-21swapper/322:48:403
1647125515,23sleep30-21swapper/321:03:223
1622625513,25sleep016220-21apt-get19:13:370
1408525514,25sleep014078-21apt-get22:38:350
10102559,31sleep01100-21perl20:13:430
3152825415,20sleep10-21swapper/121:48:561
2087025423,17sleep30-21swapper/319:28:593
2056225413,18sleep00-21swapper/019:28:520
1875925413,24sleep018752-21apt-get22:53:380
1656425413,23sleep016708-21gpgv21:03:230
1591325415,17sleep10-21swapper/120:59:041
1464025413,24sleep314635-21apt-get19:08:353
1095125412,26sleep010946-21apt-get22:28:330
3175325315,19sleep10-21swapper/120:08:351
3041625318,21sleep30-21swapper/320:03:453
3041625318,21sleep30-21swapper/320:03:453
2598925315,21sleep126012-21munin-run21:33:211
252825311,26sleep10-21swapper/123:43:511
252825311,26sleep10-21swapper/123:43:511
1970625311,16sleep30-21swapper/319:28:223
1966725317,20sleep30-21swapper/322:54:013
1776825314,23sleep017762-21apt-get19:18:340
2011225213,23sleep120159-21df21:13:381
182682525,11sleep30-21swapper/319:18:533
1649925212,19sleep30-21swapper/319:13:483
1345825213,24sleep013469-21ntp_states00:18:480
1276425212,22sleep012859-21perl19:03:400
100525212,21sleep00-21swapper/023:38:490
3058125114,22sleep030610-21awk20:03:500
3058125114,22sleep030610-21awk20:03:500
2825325112,17sleep10-21swapper/121:38:501
2698425112,17sleep10-21swapper/123:18:431
2407725110,17sleep00-21swapper/019:43:420
2319725115,20sleep30-21swapper/321:23:383
1972025113,19sleep00-21swapper/019:28:230
1697425114,21sleep317121-21cat22:48:233
1107025112,20sleep10-21swapper/118:58:321
1107025112,20sleep10-21swapper/118:58:321
749925015,16sleep30-21swapper/320:33:453
500125012,15sleep30-21swapper/322:08:453
3226225013,21sleep332356-21users23:34:003
2677225011,24sleep326769-21apt-get23:18:333
2315825013,24sleep023159-21sshd19:39:590
327452489,15sleep00-21swapper/021:53:440
327452489,15sleep00-21swapper/021:53:440
286302489,13sleep30-21swapper/319:58:363
1496224714,14sleep10-21swapper/120:58:231
996124612,18sleep10-21swapper/100:08:391
983224611,22sleep39864-21ntp_states22:23:523
768724610,21sleep37691-21users23:59:003
768724610,21sleep37691-21users23:59:003
2847324617,15sleep00-21swapper/021:38:580
193112327,12sleep10-21swapper/119:23:481
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional