You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-10-03 - 16:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s (updated Tue Oct 03, 2023 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26222105911,1008sleep22624-21sshd12:02:392
20347270012,672sleep020349-21sshd07:49:480
26786255114,521sleep026831-21/usr/sbin/munin09:52:420
1498625388,516sleep115020-21ntpq10:57:471
1498625388,516sleep115020-21ntpq10:57:471
21305251616,482sleep121363-21/usr/sbin/munin07:52:481
22999246813,437sleep123051-21apt-config09:42:141
761123680,154sleep212-21rcu_sched08:52:442
558023490,209sleep22325-21kworker/u8:308:47:152
32321233210,304sleep032331-21ntpq11:52:480
32321233210,304sleep032331-21ntpq11:52:480
8069226387,164sleep28133-21modprobe10:37:132
18420223516,196sleep218566-21grep11:07:532
734222514,65sleep2849-21latency_hist11:57:142
734222514,65sleep2849-21latency_hist11:57:142
12559222586,76sleep212642-21wc09:07:512
12213222241,160sleep212358-21unixbench_singl10:47:552
18931221816,187sleep319002-21sensors_temp09:27:503
18931221816,187sleep319002-21sensors_temp09:27:503
14864221546,22sleep20-21swapper/210:57:442
14864221546,22sleep20-21swapper/210:57:442
301442212101,29sleep230159-21cron08:22:122
28911220817,73sleep229047-21unixbench_singl09:57:532
28911220817,73sleep229047-21unixbench_singl09:57:532
26287220656,107sleep225758-21/usr/sbin/munin08:07:512
26287220656,107sleep225758-21/usr/sbin/munin08:07:512
14724220417,169sleep20-21swapper/207:32:432
10400220211,89sleep20-21swapper/209:02:342
10030220012,131sleep210092-21fschecks_time10:42:422
3185219450,24sleep23230-21awk08:37:552
13174219218,117sleep213328-21latency_hist12:37:152
27891219022,147sleep20-21swapper/211:37:522
27891219022,147sleep20-21swapper/211:37:522
15095217711,148sleep215017-21latency_hist09:17:152
6213217623,24sleep20-21swapper/207:07:152
23933217418,131sleep20-21swapper/211:27:152
23933217418,131sleep20-21swapper/211:27:152
1717216713,139sleep21836-21cat08:32:542
9245216693,57sleep20-21swapper/212:22:502
26915216210,115sleep226939-21modprobe11:37:122
26915216210,115sleep226939-21modprobe11:37:122
3483216110,136sleep23616-21sed10:22:152
3483216110,136sleep23616-21sed10:22:152
28005216124,34sleep228146-21unixbench_multi08:12:522
23056215915,103sleep223069-21munin-run09:42:152
6401215613,83sleep20-21swapper/208:47:562
14349215627,54sleep214485-21latency_hist10:57:152
721521538,100sleep20-21swapper/212:17:372
10105215010,92sleep2286-21dbus-daemon09:02:132
2421921489,62sleep224303-21cat08:02:422
29710214722,105sleep329714-21find10:02:303
16233214510,92sleep20-21swapper/211:02:402
2307121438,63sleep223131-21find11:22:512
1551621428,121sleep0367-21snmpd10:57:580
1551621428,121sleep0367-21snmpd10:57:580
10625214114,79sleep20-21swapper/207:17:532
10625214114,79sleep20-21swapper/207:17:532
1964121389,54sleep219683-21http09:32:132
1964121389,54sleep219683-21http09:32:132
1162421377,14sleep20-21swapper/207:22:412
1162421377,14sleep20-21swapper/207:22:412
22341213514,77sleep20-21swapper/211:22:142
22341213514,77sleep20-21swapper/211:22:142
24109213479,15sleep20-21swapper/209:42:512
24109213479,15sleep20-21swapper/209:42:512
2905821288,30sleep26241-21cyclictest11:42:452
2905821288,30sleep26241-21cyclictest11:42:452
1968721286,85sleep20-21swapper/211:12:452
1714212810,80sleep21909-21latency_hist10:17:152
562021278,83sleep25616-21apt-get12:12:322
32555212719,83sleep132689-21unixbench_multi11:52:531
32555212719,83sleep132689-21unixbench_multi11:52:531
25245212627,82sleep0227-21systemd-journal08:07:120
15230212622,88sleep315351-21ls10:57:533
15230212622,88sleep315351-21ls10:57:533
6141212438,56sleep06153-21modprobe07:07:130
1910212417,91sleep30-21swapper/310:17:163
1910212417,91sleep30-21swapper/310:17:163
17524212211,96sleep217673-21sed07:42:152
22402212011,97sleep022423-21cpu07:57:360
207521195,11sleep00-21swapper/012:00:030
16652211983,18sleep20-21swapper/209:22:162
13695211923,77sleep313212-21apt-get09:12:243
31615211816,86sleep231648-21latency_hist11:52:132
31615211816,86sleep231648-21latency_hist11:52:132
829211613,87sleep1861-21proc_pri10:12:501
30713211518,75sleep20-21swapper/208:22:452
21465211584,17sleep20-21swapper/209:37:142
2199021147,78sleep20-21swapper/207:56:082
2942421125,14sleep00-21swapper/010:02:030
2942421125,14sleep00-21swapper/010:02:030
20304211112,83sleep020392-21/usr/sbin/munin09:32:400
7595210818,71sleep07645-21ntp_kernel_pll_12:17:470
5357210821,71sleep35378-21sshd07:02:493
1722021085,92sleep00-21swapper/011:05:000
7650210612,75sleep37681-21/usr/sbin/munin08:52:463
5079210412,16sleep24993-21latency_hist10:27:162
2323210312,43sleep20-21swapper/212:02:152
22907210310,16sleep20-21swapper/207:57:502
936421029,82sleep2367-21snmpd07:13:132
8150210218,60sleep08096-21latency_hist10:37:150
29735210223,38sleep20-21swapper/210:02:332
1448210222,63sleep31484-21wc11:57:483
4553210119,58sleep00-21swapper/012:07:500
2516721008,13sleep20-21swapper/209:47:402
1936521008,15sleep20-21swapper/207:47:412
1775221008,53sleep20-21swapper/209:22:532
615629932,46sleep16211-21unin-run07:07:141
1943129916,64sleep119266-21/usr/sbin/munin07:47:441
96092986,22sleep20-21swapper/210:41:492
53242988,56sleep25327-21cron12:12:122
1193629719,61sleep312043-21fschecks_time12:32:403
51329644,42sleep2367-21snmpd08:28:512
177482967,14sleep20-21swapper/207:42:362
177482967,14sleep20-21swapper/207:42:362
2865529521,44sleep10-21swapper/108:17:151
1832729533,43sleep117724-21/usr/sbin/munin07:42:551
1832729533,43sleep117724-21/usr/sbin/munin07:42:551
1309529413,67sleep013143-21awk07:27:360
1837429327,35sleep10-21swapper/109:27:321
1837429327,35sleep10-21swapper/109:27:321
2339129223,22sleep10-21swapper/109:42:181
2339129223,22sleep10-21swapper/109:42:181
2947129120,42sleep329627-21grep11:42:543
2947129120,42sleep329627-21grep11:42:543
1074729121,47sleep310889-21head07:17:543
1074729121,47sleep310889-21head07:17:543
266012908,44sleep20-21swapper/209:52:322
1532529015,55sleep015376-21sort09:17:420
1030328936,13sleep20-21swapper/212:27:382
3140028816,56sleep031488-21/usr/sbin/munin10:07:400
3140028816,56sleep031488-21/usr/sbin/munin10:07:400
392728723,48sleep13943-21cat08:42:141
2390328711,60sleep023935-21awk09:42:450
2390328711,60sleep023935-21awk09:42:450
2331128721,46sleep123460-21grep11:22:541
68892869,42sleep20-21swapper/207:07:482
548328622,34sleep10-21swapper/108:47:151
49602845,12sleep20-21swapper/210:27:042
1888028417,22sleep00-21swapper/007:45:490
1888028417,22sleep00-21swapper/007:45:490
162922848,58sleep00-21swapper/007:37:410
131672849,44sleep20-21swapper/207:27:382
2080528224,31sleep00-21swapper/007:52:310
3272228119,40sleep3400-21unixbench_singl08:27:553
457228013,49sleep34608-21ntpq08:42:443
312482805,41sleep20-21swapper/210:07:262
312482805,41sleep20-21swapper/210:07:262
3041728012,54sleep130428-21sort08:22:321
1731728022,26sleep10-21swapper/109:22:481
2577027919,28sleep30-21swapper/311:32:413
2577027919,28sleep30-21swapper/311:32:413
148672790,7sleep00-21swapper/007:32:440
1319927921,40sleep013223-21munin-run09:12:130
2699627826,34sleep027072-21latency_hist11:37:140
2699627826,34sleep027072-21latency_hist11:37:140
1087927816,44sleep010957-21ls12:27:520
1039027822,39sleep010433-21ntp_kernel_pll_07:17:470
1039027822,39sleep010433-21ntp_kernel_pll_07:17:470
2899227727,29sleep00-21swapper/008:17:420
236522769,13sleep10-21swapper/108:02:131
2313427616,38sleep323201-21/usr/sbin/munin11:22:523
1984027613,27sleep10-21swapper/111:12:481
968527517,37sleep30-21swapper/310:42:143
3115827513,43sleep031302-21/usr/sbin/munin08:22:550
2314527513,41sleep323272-21apt-config09:42:163
212582756,15sleep30-21swapper/309:35:013
1438927523,27sleep30-21swapper/307:32:153
3236727424,29sleep00-21swapper/008:27:480
2142827419,40sleep121463-21readlink09:37:131
156427411,47sleep01646-21mailstats08:32:520
1284727424,23sleep30-21swapper/307:27:153
1284727424,23sleep30-21swapper/307:27:153
1175827421,36sleep011815-21latency10:47:440
613627311,49sleep06185-21ntpq08:47:500
104352738,15sleep10-21swapper/107:17:481
104352738,15sleep10-21swapper/107:17:481
571227227,21sleep30-21swapper/312:12:413
228227218,28sleep10-21swapper/112:02:141
888127123,28sleep10-21swapper/110:37:481
887627116,39sleep18962-21/usr/sbin/munin12:22:401
728227113,42sleep1367-21snmpd10:32:481
23052715,10sleep30-21swapper/308:37:033
1216427112,44sleep112259-21timerandwakeup07:22:571
1216427112,44sleep112259-21timerandwakeup07:22:571
975427010,47sleep09789-21latency_hist07:17:140
736727016,34sleep17257-21/usr/sbin/munin08:52:401
657927018,30sleep00-21swapper/010:32:140
2916927012,43sleep329178-21kernelversion08:17:453
290027017,26sleep30-21swapper/312:02:443
2674127030,24sleep30-21swapper/309:52:413
1076927019,23sleep00-21swapper/009:02:440
906426913,43sleep09112-21cat08:57:430
2419326820,27sleep324312-21timerwakeupswit09:42:523
2419326820,27sleep324312-21timerwakeupswit09:42:523
1005326816,26sleep00-21swapper/012:27:160
776426718,31sleep10-21swapper/107:12:151
546726713,38sleep05482-21munin-run08:47:140
3165026717,24sleep30-21swapper/311:52:143
3165026717,24sleep30-21swapper/311:52:143
177792676,51sleep30-21swapper/309:23:023
136922676,46sleep00-21swapper/009:12:210
1015526719,26sleep30-21swapper/309:02:143
861026613,19sleep30-21swapper/307:12:413
1777426613,24sleep10-21swapper/111:07:401
161826619,28sleep01684-21http10:17:130
1157826614,33sleep111669-21/usr/sbin/munin10:47:411
70102656,16sleep30-21swapper/310:32:423
661726514,25sleep10-21swapper/110:32:151
416826519,27sleep30-21swapper/312:07:413
299292659,13sleep10-21swapper/111:45:551
299292659,13sleep10-21swapper/111:45:551
273426520,23sleep30-21swapper/308:37:443
264626516,30sleep02642-21apt-get08:37:370
2632026515,37sleep326369-21unin-run09:52:133
1221026513,24sleep00-21swapper/012:32:450
700226417,32sleep17154-21latency_hist12:17:151
3079826427,22sleep10-21swapper/111:47:531
3079826427,22sleep10-21swapper/111:47:531
284282645,10sleep30-21swapper/308:15:193
2286426426,16sleep00-21swapper/011:22:440
851226320,19sleep10-21swapper/107:12:391
3096326313,32sleep031079-21/usr/sbin/munin11:47:560
3096326313,32sleep031079-21/usr/sbin/munin11:47:560
2925826318,29sleep029320-21processes11:42:510
2925826318,29sleep029320-21processes11:42:510
1955826315,36sleep019645-21fschecks_time11:12:430
1789626321,24sleep30-21swapper/311:07:423
1041926314,34sleep310478-21/usr/sbin/munin10:42:513
972426225,18sleep00-21swapper/010:42:150
1907426217,28sleep00-21swapper/011:12:150
610726115,22sleep00-21swapper/012:12:500
382126113,32sleep03830-21munin-run12:07:130
308526115,28sleep10-21swapper/112:02:501
2982326116,24sleep00-21swapper/010:02:390
2948926121,22sleep30-21swapper/310:02:143
2948926121,22sleep30-21swapper/310:02:143
281812615,11sleep00-21swapper/011:38:110
281812615,11sleep00-21swapper/011:38:110
1955226114,32sleep319556-21diskstats11:12:423
2827426019,24sleep10-21swapper/109:57:401
2827426019,24sleep10-21swapper/109:57:401
2420426014,31sleep024210-21df_inode11:27:370
2420426014,31sleep024210-21df_inode11:27:370
2215326019,19sleep30-21swapper/307:57:143
1937526016,27sleep319376-21diskstats07:47:423
419525910,29sleep00-21swapper/010:22:500
2761425913,20sleep30-21swapper/311:37:453
2761425913,20sleep30-21swapper/311:37:453
2724725917,25sleep127240-21apt-get11:37:351
2724725917,25sleep127240-21apt-get11:37:351
1668525916,19sleep30-21swapper/307:37:523
1322625919,23sleep10-21swapper/109:12:141
2370325816,26sleep30-21swapper/308:02:153
219425814,24sleep10-21swapper/110:17:381
219425814,24sleep10-21swapper/110:17:381
1815325815,23sleep00-21swapper/009:27:150
2592325714,26sleep125946-21irqstats11:32:431
2592325714,26sleep125946-21irqstats11:32:431
2403525716,17sleep30-21swapper/308:02:393
96722566,12sleep30-21swapper/312:22:553
618725613,26sleep16197-21ntpq08:47:511
573225615,24sleep35729-21apt-get08:47:373
2505125611,26sleep10-21swapper/109:47:311
2105425613,25sleep121048-21apt-get11:17:371
2105425613,25sleep121048-21apt-get11:17:371
1463825612,22sleep10-21swapper/107:32:381
1463225612,18sleep10-21swapper/109:12:541
111822568,23sleep10-21swapper/110:47:131
463125517,20sleep10-21swapper/110:22:551
3208425516,21sleep30-21swapper/311:52:433
3208425516,21sleep30-21swapper/311:52:433
3021025513,31sleep130271-21sendmail_mailqu10:02:491
2707225518,18sleep10-21swapper/108:12:141
2707225518,18sleep10-21swapper/108:12:141
2439925514,20sleep10-21swapper/111:27:421
2439925514,20sleep10-21swapper/111:27:421
1962625512,27sleep00-21swapper/009:32:030
1962625512,27sleep00-21swapper/009:32:030
1796125514,23sleep318018-21irqstats07:42:453
1796125514,23sleep318018-21irqstats07:42:453
1538025516,22sleep10-21swapper/109:17:431
1302525511,28sleep013092-21df10:52:330
1231625517,18sleep30-21swapper/309:07:433
1209325510,23sleep00-21swapper/007:22:560
1209325510,23sleep00-21swapper/007:22:560
693225413,24sleep36938-21ntp_states07:07:493
315212546,37sleep30-21swapper/308:25:043
219325412,24sleep02187-21apt-get10:17:370
219325412,24sleep02187-21apt-get10:17:370
161732545,12sleep30-21swapper/311:02:373
1480825416,18sleep30-21swapper/307:32:443
1040125413,18sleep30-21swapper/309:02:353
1038725412,22sleep10-21swapper/112:27:411
585725311,28sleep35950-21cat10:27:543
381525314,19sleep30-21swapper/310:22:413
3167625318,19sleep10-21swapper/110:07:451
3167625318,19sleep10-21swapper/110:07:451
2574325312,25sleep325268-21apt-get08:07:243
2574325312,25sleep325268-21apt-get08:07:243
2445425312,27sleep124528-21mailstats08:02:491
231825312,23sleep12354-21munin-run08:37:131
2229125315,24sleep022330-21modprobe11:22:120
2229125315,24sleep022330-21modprobe11:22:120
164682535,11sleep30-21swapper/309:21:043
736225213,23sleep07366-21diskstats08:52:390
648025213,16sleep00-21swapper/007:07:380
3241325212,24sleep132442-21ntp_states08:27:491
300262524,37sleep10-21swapper/108:20:421
2637125217,17sleep00-21swapper/009:52:140
1209625214,17sleep10-21swapper/112:32:421
1047725211,27sleep110483-21diskstats09:02:371
721625112,15sleep30-21swapper/312:17:383
319272514,11sleep30-21swapper/310:07:513
319272514,11sleep30-21swapper/310:07:513
283192517,33sleep00-21swapper/008:13:590
2183925117,21sleep021857-21fschecks_count09:37:400
1190125113,14sleep30-21swapper/310:47:473
1034825111,15sleep00-21swapper/010:42:480
537725014,16sleep10-21swapper/112:12:151
2112325012,22sleep321132-21df_inode11:17:393
2112325012,22sleep321132-21df_inode11:17:393
1681625014,22sleep116908-21cat07:37:551
1070225016,20sleep310743-21ntpq12:27:473
885424910,23sleep18848-21apt-get08:57:361
81224912,25sleep3822-21ntp_states10:12:483
2661924912,23sleep126613-21apt-get09:52:351
2413524812,20sleep324202-21df11:27:363
2413524812,20sleep324202-21df11:27:363
2702524710,15sleep00-21swapper/008:12:130
2702524710,15sleep00-21swapper/008:12:130
729524610,14sleep10-21swapper/112:17:401
1744024615,16sleep10-21swapper/111:07:151
1377024414,14sleep30-21swapper/310:52:523
134352447,14sleep10-21swapper/107:27:451
1433823911,13sleep10-21swapper/110:57:141
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional