You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-06-24 - 21:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Mon Jun 24, 2024 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
88725376,520sleep2924-21ntpq08:58:242
17283235876,198sleep217334-21cpuspeed08:08:072
17283235876,198sleep217334-21cpuspeed08:08:072
189223539,200sleep21926-21awk12:27:472
147722318124,124sleep214843-21threads11:23:202
8442229411,246sleep28465-21processes11:03:292
30460225823,223sleep230466-21ntpq12:13:252
30460225823,223sleep230466-21ntpq12:13:252
11071221813,18sleep20-21swapper/211:13:022
508122068,162sleep25105-21processes07:28:202
16930218319,86sleep20-21swapper/209:48:282
304721788,137sleep23054-21wc07:23:032
23507217732,86sleep20-21swapper/211:52:492
1472217437,84sleep20-21swapper/210:43:012
30273216629,97sleep230339-21modprobe10:32:472
30938215341,25sleep20-21swapper/208:51:292
23295215316,121sleep323436-21sed08:27:513
15921215378,38sleep20-21swapper/211:28:092
15921215378,38sleep20-21swapper/211:28:092
5112214629,18sleep20-21swapper/210:53:152
2466821465,12sleep30-21swapper/311:53:313
20993214646,21sleep20-21swapper/210:02:502
25672214516,58sleep20-21swapper/210:17:492
25672214516,58sleep20-21swapper/210:17:492
3700214241,83sleep20-21swapper/209:08:092
17380214228,96sleep217470-21grep11:33:102
2746121409,119sleep227486-21cpu10:23:062
2746121409,119sleep227486-21cpu10:23:062
453521397,10sleep30-21swapper/309:08:353
22118213734,63sleep222176-21fschecks_time08:23:262
11427213728,17sleep20-21swapper/209:32:492
7359213630,92sleep27450-21cat09:18:272
26331213420,46sleep20-21swapper/208:37:482
26975213032,54sleep226825-21/usr/sbin/munin12:03:112
25680213033,80sleep20-21swapper/211:58:182
9474212722,85sleep29480-21cat07:42:592
2551212742,44sleep20-21swapper/209:03:252
2551212742,44sleep20-21swapper/209:03:252
11915212612,72sleep211651-21/usr/sbin/munin09:33:142
10000212528,62sleep210069-21mailstats11:08:262
4626212229,78sleep24538-21latency_hist10:52:512
26670212133,69sleep20-21swapper/208:38:042
2293221208,20sleep20-21swapper/210:08:112
2293221208,20sleep20-21swapper/210:08:112
28303211940,53sleep228336-21grep07:02:472
28303211940,53sleep228336-21grep07:02:472
27317211921,83sleep327362-21sed06:58:123
27317211921,83sleep327362-21sed06:58:123
18102211924,47sleep20-21swapper/209:53:012
15489211735,20sleep215566-21sed08:02:502
6483211638,33sleep26525-21fschecks_count10:58:122
30573211619,55sleep20-21swapper/207:08:002
21934211416,73sleep022059-21grep11:47:500
21934211416,73sleep022059-21grep11:47:500
3229211322,73sleep32746-21apt-get10:47:593
1708921139,88sleep217234-21latency_hist08:07:512
562211231,21sleep2667-21sort10:38:332
2722021129,82sleep20-21swapper/210:22:502
12996211226,70sleep20-21swapper/211:18:192
21533211115,71sleep10-21swapper/111:43:311
21533211115,71sleep10-21swapper/111:43:311
7692210930,49sleep07710-21apt-get07:38:000
32075210921,19sleep20-21swapper/212:18:222
32075210921,19sleep20-21swapper/212:18:222
31093210921,75sleep331140-21ps10:33:293
19462210814,78sleep119488-21ntp_kernel_err11:38:221
29544210674,17sleep20-21swapper/208:47:502
13603210624,22sleep213626-21awk09:38:192
13603210624,22sleep213626-21awk09:38:192
24358210517,43sleep20-21swapper/211:53:252
19204210427,62sleep219288-21perl11:38:152
19449210314,74sleep319459-21needreboot11:38:203
2230321026,12sleep20-21swapper/210:05:252
5380210115,66sleep20-21swapper/209:13:142
3439210122,56sleep13440-21cron12:32:471
2868421017,57sleep228712-21latency12:08:162
2868421017,57sleep228712-21latency12:08:162
3899210017,67sleep13230-21/usr/sbin/munin10:48:341
215752995,10sleep20-21swapper/208:21:402
1309029929,52sleep20-21swapper/207:53:252
1009029919,66sleep39582-21apt-get09:28:013
3261429712,72sleep132652-21idleruntime07:13:291
3136129722,14sleep20-21swapper/208:53:032
211929721,52sleep02177-21fschecks_count09:03:100
211929721,52sleep02177-21fschecks_count09:03:100
182892974,83sleep00-21swapper/011:34:290
663629620,14sleep20-21swapper/207:33:252
159082965,11sleep20-21swapper/209:46:142
521829511,63sleep14812-21/usr/sbin/munin10:53:181
1476929512,27sleep30-21swapper/309:43:043
11722955,10sleep20-21swapper/207:17:222
293982949,68sleep10-21swapper/110:28:221
291772930,8sleep30-21swapper/307:03:243
2877029313,66sleep028793-21latency_hist10:27:480
2877029313,66sleep028793-21latency_hist10:27:480
219182937,52sleep221931-21munin-run11:47:482
219182937,52sleep221931-21munin-run11:47:482
1547429319,54sleep315551-21sed11:27:503
978229216,57sleep19475-21/usr/sbin/munin07:43:181
1777829217,61sleep117844-21systemctl11:33:211
273229115,59sleep02773-21gpgv10:47:480
1581329113,61sleep315815-21/usr/sbin/munin08:03:073
838929018,14sleep20-21swapper/209:23:082
2794328913,62sleep127958-21/usr/sbin/munin10:23:291
2794328913,62sleep127958-21/usr/sbin/munin10:23:291
263542887,72sleep00-21swapper/012:00:060
2061828718,54sleep320643-21basename08:18:103
3008728626,44sleep330158-21fschecks_time12:13:143
3008728626,44sleep330158-21fschecks_time12:13:143
351628514,56sleep33572-21cut12:32:493
2901728517,49sleep00-21swapper/012:08:260
2901728517,49sleep00-21swapper/012:08:260
286312857,15sleep20-21swapper/207:03:042
786428417,54sleep37915-21fschecks_time07:38:113
2793228420,46sleep00-21swapper/008:42:510
201182848,44sleep220165-21systemd-run08:17:482
317312837,17sleep00-21swapper/010:36:170
2111328315,45sleep320599-21/usr/sbin/munin11:43:253
2111328315,45sleep320599-21/usr/sbin/munin11:43:253
2924128216,27sleep00-21swapper/007:03:260
969928119,38sleep00-21swapper/009:27:500
745128113,54sleep36740-21/usr/sbin/munin09:18:283
3254428127,26sleep00-21swapper/007:13:280
17622816,13sleep20-21swapper/207:18:152
3033228017,44sleep330382-21diskstats07:07:533
517127917,47sleep05183-21apt-get09:13:000
196592790,6sleep00-21swapper/008:13:290
517627818,39sleep05218-21switchtime07:28:220
2908227814,43sleep129190-21grep12:08:271
2908227814,43sleep129190-21grep12:08:271
1894827824,23sleep10-21swapper/108:13:111
178927822,28sleep00-21swapper/009:02:500
318832779,26sleep231918-21apt-config10:37:482
2837627725,27sleep00-21swapper/007:02:490
2837627725,27sleep00-21swapper/007:02:490
2483727724,26sleep10-21swapper/108:32:501
1289827711,52sleep112902-21irqstats07:53:161
38827617,40sleep1512-21cat12:22:491
38827617,40sleep1512-21cat12:22:491
258827613,46sleep02618-21ntpq12:28:170
171027627,26sleep30-21swapper/307:18:123
2316627513,45sleep023188-21/usr/sbin/munin10:08:180
2316627513,45sleep023188-21/usr/sbin/munin10:08:180
1425427420,40sleep014293-21fschecks_count11:23:060
1177527423,26sleep30-21swapper/309:33:113
566027322,33sleep05734-21grep10:53:340
1270827314,22sleep30-21swapper/307:53:103
1005027313,45sleep310090-21/usr/sbin/munin07:43:283
75427214,41sleep3810-21fschecks_time12:23:113
2910527216,27sleep30-21swapper/310:28:123
1773827220,28sleep017921-21cut09:52:510
749027119,21sleep00-21swapper/009:18:290
3268727114,41sleep332699-21/usr/sbin/munin07:13:313
2824927118,32sleep00-21swapper/008:43:070
2787727112,40sleep127880-21cron08:42:481
2563127121,32sleep125671-21unin-run10:17:481
2563127121,32sleep125671-21unin-run10:17:481
1958127122,20sleep30-21swapper/308:13:293
1210127124,29sleep00-21swapper/009:33:200
198827022,25sleep30-21swapper/310:43:233
1686027015,33sleep10-21swapper/109:48:271
122622708,29sleep20-21swapper/207:52:162
91192698,49sleep29140-21munin-run07:42:502
2725626914,41sleep127021-21/usr/sbin/munin06:58:101
2725626914,41sleep127021-21/usr/sbin/munin06:58:101
2501526915,33sleep01-21systemd11:57:480
23912697,28sleep20-21swapper/212:28:112
2098326914,39sleep120992-21pluginstate10:02:491
194426913,35sleep00-21swapper/012:27:490
1447926927,25sleep30-21swapper/309:42:483
1447926927,25sleep30-21swapper/309:42:483
2894826812,28sleep329012-21mailstats12:08:253
2894826812,28sleep329012-21mailstats12:08:253
2765926819,21sleep10-21swapper/112:03:261
2006226818,30sleep020087-21netstat09:58:160
199026820,24sleep00-21swapper/007:18:250
1792426814,28sleep10-21swapper/109:52:521
521926722,24sleep10-21swapper/107:28:231
44026714,23sleep00-21swapper/010:38:310
3267726715,24sleep10-21swapper/108:57:501
239052676,28sleep20-21swapper/208:28:232
592526618,33sleep35952-21cat07:32:503
2342926618,33sleep023454-21munin-run11:52:470
2182226620,25sleep10-21swapper/110:03:301
2088826621,20sleep10-21swapper/108:18:201
195526615,22sleep10-21swapper/107:18:221
1950726618,20sleep30-21swapper/309:57:503
1237026619,30sleep0288-21dbus-daemon11:17:490
118526617,24sleep10-21swapper/110:42:481
1115426610,27sleep10-21swapper/111:13:051
985926515,35sleep19861-21memory11:08:171
3151326515,34sleep130567-21/usr/sbin/munin10:33:351
1916726518,30sleep019176-21/usr/sbin/munin11:38:130
1861926517,20sleep00-21swapper/008:12:500
1861926517,20sleep00-21swapper/008:12:500
1391426510,39sleep114026-21timerwakeupswit09:38:301
1391426510,39sleep114026-21timerwakeupswit09:38:301
1249526517,31sleep112572-21grep09:33:291
113842650,7sleep30-21swapper/311:13:103
75122647,48sleep00-21swapper/011:01:010
539826413,36sleep35444-21tr10:53:283
3045326413,35sleep130454-21ntp_states12:13:241
3045326413,35sleep130454-21ntp_states12:13:241
2887526416,32sleep329004-21grep08:43:303
253952649,40sleep30-21swapper/311:58:113
2014026418,27sleep320148-21ntp_states09:58:193
1105026414,33sleep111045-21apt-get07:48:031
1040226420,20sleep10-21swapper/109:28:321
794226315,31sleep37935-21apt-get11:03:063
765026321,26sleep10-21swapper/111:02:471
1844426312,28sleep30-21swapper/311:36:303
1765026314,33sleep317726-21latency_hist09:52:493
46226220,22sleep10-21swapper/108:58:021
3000326221,19sleep10-21swapper/108:48:091
2315526211,38sleep323158-21needreboot10:08:173
2315526211,38sleep323158-21needreboot10:08:173
1484026210,13sleep00-21swapper/007:58:320
1458726219,25sleep30-21swapper/307:58:233
828726118,26sleep38304-21/usr/sbin/munin09:23:013
3176826116,23sleep00-21swapper/008:53:210
25422615,11sleep10-21swapper/109:03:241
25422615,11sleep10-21swapper/109:03:241
2432726119,22sleep10-21swapper/111:53:241
2431726114,26sleep00-21swapper/008:28:330
2404426115,30sleep124081-21modprobe10:12:471
2404426115,30sleep124081-21modprobe10:12:471
2349926115,29sleep323502-21apt-get08:28:033
209126112,27sleep10-21swapper/110:43:251
1571226116,27sleep315706-21apt-get11:28:023
1571226116,27sleep315706-21apt-get11:28:023
1476126111,28sleep00-21swapper/009:43:010
861726013,29sleep18683-21memory09:23:151
590426018,27sleep35948-21/usr/sbin/munin09:13:313
336526011,27sleep30-21swapper/309:07:503
336526011,27sleep30-21swapper/309:07:503
3151326015,27sleep031531-21apt-get12:17:590
3151326015,27sleep031531-21apt-get12:17:590
2725726016,32sleep027296-21ntp_offset12:03:170
2206826020,23sleep10-21swapper/108:23:251
1939626016,28sleep019412-21idleruntime-cro09:57:470
1011226015,30sleep010088-21apt-get09:28:110
2562225918,25sleep025623-21cron10:17:470
2562225918,25sleep025623-21cron10:17:470
2530925914,26sleep125303-21apt-get11:58:051
2508025921,23sleep00-21swapper/008:33:050
2253725914,28sleep122540-21/usr/sbin/munin11:48:111
1311125912,20sleep30-21swapper/311:18:253
881725815,24sleep08864-21users11:03:350
67742585,13sleep30-21swapper/310:58:213
617425820,18sleep30-21swapper/307:33:063
3211825821,21sleep00-21swapper/007:12:510
2662325816,26sleep326618-21apt-get08:38:013
2628625814,30sleep326315-21awk10:18:143
2243625816,17sleep30-21swapper/311:48:093
1860725817,24sleep318617-21munin-run08:12:493
1860725817,24sleep318617-21munin-run08:12:493
101425819,20sleep30-21swapper/308:58:293
323225713,17sleep00-21swapper/010:48:000
3049025719,18sleep00-21swapper/008:48:280
3039725719,16sleep10-21swapper/107:07:551
18632576,16sleep10-21swapper/112:27:251
89092565,11sleep10-21swapper/107:42:011
822225617,21sleep10-21swapper/111:03:201
748025615,18sleep00-21swapper/007:37:520
652525614,26sleep16575-21latency_hist09:17:501
2873025611,20sleep10-21swapper/107:03:111
1418325614,25sleep114177-21apt-get07:58:041
131762566,12sleep00-21swapper/007:53:310
1156625612,18sleep00-21swapper/007:48:260
360325517,20sleep13597-21apt-get09:08:001
2838725522,17sleep10-21swapper/108:43:121
2618825517,18sleep10-21swapper/110:18:121
2600325516,22sleep026066-21fschecks_time10:18:070
252202556,11sleep30-21swapper/308:33:123
2178125514,16sleep00-21swapper/010:03:290
2087425520,19sleep00-21swapper/008:18:190
1612925516,23sleep10-21swapper/108:03:171
141582556,37sleep00-21swapper/009:38:370
141582556,37sleep00-21swapper/009:38:370
948925412,26sleep09483-21apt-get11:08:040
3043825417,22sleep30-21swapper/308:48:273
3035325418,18sleep00-21swapper/010:32:490
256992547,14sleep10-21swapper/108:33:311
1728225412,24sleep117276-21apt-get08:08:061
1728225412,24sleep117276-21apt-get08:08:061
1697225415,19sleep00-21swapper/011:32:470
1697225415,19sleep00-21swapper/011:32:470
1616825415,23sleep116174-21ntp_states11:28:161
1616825415,23sleep116174-21ntp_states11:28:161
947225315,23sleep08992-21apt-get07:42:570
808825319,17sleep10-21swapper/109:22:501
3234525312,24sleep331861-21apt-get10:37:573
29112535,12sleep30-21swapper/307:22:523
304625211,24sleep13040-21apt-get07:23:021
1126325216,21sleep30-21swapper/307:48:133
3117525112,16sleep30-21swapper/308:52:513
2748825111,20sleep30-21swapper/310:23:073
2748825111,20sleep30-21swapper/310:23:073
2682625115,19sleep30-21swapper/312:03:003
397825013,19sleep00-21swapper/009:08:150
315052504,16sleep30-21swapper/312:17:543
315052504,16sleep30-21swapper/312:17:543
3035225011,23sleep030366-21memory12:13:190
3035225011,23sleep030366-21memory12:13:190
2133425017,14sleep30-21swapper/310:03:123
2514324818,15sleep30-21swapper/310:13:263
2514324818,15sleep30-21swapper/310:13:263
2234724811,21sleep022350-21memory08:23:320
157342488,14sleep00-21swapper/008:03:040
1262324812,17sleep00-21swapper/011:18:050
1416024711,19sleep114157-21apt-get11:22:591
958724616,14sleep30-21swapper/311:08:113
74282469,15sleep10-21swapper/107:37:491
1567024612,19sleep115692-21users09:43:351
311924510,20sleep03123-21df_inode07:23:050
127052437,13sleep10-21swapper/111:18:101
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional