You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-02-08 - 16:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Sat Feb 08, 2025 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
309426399,614sleep03131-21ntpq08:13:320
13723255612,530sleep313730-21ntp_states10:48:303
2804525198,498sleep228059-21grep09:43:172
17641236610,343sleep217729-21grep07:18:232
9606229311,195sleep20-21swapper/210:33:192
130292288129,25sleep20-21swapper/210:48:042
930322857,228sleep29454-21latency_hist08:38:052
925226112,232sleep1964-21ntpq12:03:361
880422597,205sleep2370-21snmpd12:31:432
25460225723,210sleep125500-21awk09:33:021
13795225011,222sleep213923-21cat07:08:052
13795225011,222sleep213923-21cat07:08:052
2320223813,199sleep22152-21latency_hist08:13:062
3018222147,196sleep2370-21snmpd11:49:122
3018222147,196sleep2370-21snmpd11:49:122
13865221213,95sleep213944-21xargs10:48:332
6483221116,174sleep26568-21fschecks_time10:23:182
6483221116,174sleep26568-21fschecks_time10:23:182
24176219314,62sleep224224-21latency11:28:252
4938218921,147sleep20-21swapper/212:18:022
4938218921,147sleep20-21swapper/212:18:022
1264621839,162sleep212694-21awk08:48:182
17157217916,146sleep217010-21/usr/sbin/munin11:03:182
17157217916,146sleep217010-21/usr/sbin/munin11:03:182
14819217621,23sleep20-21swapper/208:53:312
4965216615,113sleep25072-21fschecks_time08:23:192
16164216613,17sleep20-21swapper/210:58:062
3371216523,18sleep20-21swapper/208:13:382
11699215510,106sleep211811-21timerandwakeup08:43:342
24203215325,68sleep224177-21/usr/sbin/munin09:28:192
1908921449,95sleep219184-21mailstats11:08:312
32530214010,68sleep20-21swapper/209:58:062
23759213911,50sleep20-21swapper/207:38:052
2301421399,95sleep223058-21ntpq09:23:222
29839213412,105sleep329917-21timerandwakeup11:48:453
29839213412,105sleep329917-21timerandwakeup11:48:453
9789213318,93sleep19851-21wc10:33:221
3029421328,47sleep20-21swapper/209:48:462
1147221329,84sleep211546-21latency_hist10:43:042
717021318,89sleep2370-21snmpd12:25:332
28288212631,18sleep20-21swapper/211:43:432
28288212631,18sleep20-21swapper/211:43:432
28475212315,24sleep00-21swapper/007:53:050
20963212321,74sleep20-21swapper/207:28:242
6629212211,48sleep20-21swapper/208:28:242
1283212225,29sleep20-21swapper/210:03:272
1283212225,29sleep20-21swapper/210:03:272
26242212110,68sleep20-21swapper/209:33:322
18554212129,28sleep20-21swapper/209:07:322
18554212129,28sleep20-21swapper/209:07:322
1690821209,67sleep20-21swapper/207:13:392
16500212026,29sleep20-21swapper/210:59:472
3778211922,82sleep33868-21sort12:13:233
3778211922,82sleep33868-21sort12:13:233
9211211810,93sleep29283-21systemd-run10:33:032
14177211813,86sleep314272-21munin-node07:08:113
7628211719,38sleep10-21swapper/112:28:151
277321177,70sleep22822-21ntpq10:08:322
19904211626,28sleep20-21swapper/209:09:582
24459211416,81sleep024498-21wc07:38:320
20365211314,87sleep220374-21munin-run07:28:042
4670211218,70sleep34837-21unixbench_multi10:13:463
177321075,29sleep20-21swapper/212:07:412
1589721069,17sleep20-21swapper/208:58:242
19707210512,73sleep319843-21ls07:23:373
31773210313,48sleep20-21swapper/208:03:212
18849210217,69sleep318560-21/usr/sbin/munin11:08:243
31178210114,73sleep331181-21/usr/sbin/munin11:53:373
31178210114,73sleep331181-21/usr/sbin/munin11:53:373
836210016,69sleep10-21swapper/110:03:141
836210016,69sleep10-21swapper/110:03:141
24367210010,76sleep324375-21awk11:28:303
276522995,30sleep20-21swapper/209:42:442
613229816,64sleep16088-21apt-get12:23:381
609329821,60sleep35590-21apt-get12:23:123
271772978,18sleep227272-21perl07:48:232
2679529717,57sleep00-21swapper/011:38:040
42392969,69sleep10-21swapper/110:13:361
2516329630,29sleep20-21swapper/207:41:572
223372966,30sleep20-21swapper/209:22:422
40912956,28sleep20-21swapper/210:13:212
30122958,54sleep2370-21snmpd12:08:422
30122958,54sleep2370-21snmpd12:08:422
2713029518,59sleep327058-21latency_hist09:38:053
1052129529,30sleep20-21swapper/208:39:262
54129425,49sleep3592-21sort12:03:143
3087129410,47sleep20-21swapper/211:53:202
3087129410,47sleep20-21swapper/211:53:202
3085429422,22sleep10-21swapper/109:53:051
2148029438,44sleep221571-21cat11:18:042
573129332,42sleep20-21swapper/210:18:132
540729330,28sleep20-21swapper/212:20:532
3122629329,31sleep20-21swapper/207:59:492
2676829330,28sleep20-21swapper/207:47:382
1480029314,53sleep20-21swapper/207:08:302
436629229,29sleep20-21swapper/208:20:122
2701529229,30sleep20-21swapper/211:38:182
2701529229,30sleep20-21swapper/211:38:182
2661129225,30sleep20-21swapper/211:36:202
2174429229,28sleep20-21swapper/211:18:532
2153029226,32sleep20-21swapper/209:15:312
38329129,28sleep20-21swapper/210:00:572
3247429129,28sleep20-21swapper/212:00:022
3247429129,28sleep20-21swapper/212:00:022
2327029129,29sleep20-21swapper/211:23:372
1957528916,53sleep119070-21/usr/sbin/munin07:23:351
1212228931,34sleep012157-21munin-run07:03:030
463428819,49sleep04641-21cat08:23:050
2973228826,29sleep20-21swapper/207:55:292
2220928826,36sleep30-21swapper/307:33:063
2763728716,29sleep00-21swapper/007:48:330
2524828716,56sleep125352-21date11:33:041
2437728716,52sleep023883-21/usr/sbin/munin11:28:310
969028512,60sleep09759-21basename08:38:230
212772859,19sleep00-21swapper/009:13:350
68812840,6sleep00-21swapper/008:28:290
3210628415,45sleep032108-21sshd09:54:450
2190228412,60sleep00-21swapper/011:20:520
2935728318,50sleep329424-21date09:48:063
2244728318,32sleep30-21swapper/307:33:183
3079428213,53sleep130793-21apt-get11:53:151
3079428213,53sleep130793-21apt-get11:53:151
3044928215,18sleep00-21swapper/009:48:480
275928116,32sleep10-21swapper/110:08:311
222328120,36sleep10-21swapper/108:13:051
1712328016,21sleep10-21swapper/111:03:171
1712328016,21sleep10-21swapper/111:03:171
1366628011,49sleep10-21swapper/110:48:281
3077527924,35sleep330835-21sendmail-msp09:53:033
820327816,19sleep30-21swapper/312:28:333
2775227817,26sleep00-21swapper/011:43:170
2775227817,26sleep00-21swapper/011:43:170
2241327824,29sleep00-21swapper/009:23:040
1080027819,39sleep310829-21munin-run08:43:023
438627713,48sleep04430-21awk10:13:390
2396727714,16sleep00-21swapper/009:28:050
2180027714,26sleep10-21swapper/107:28:481
1802627716,44sleep17558-1kworker/1:0H09:03:391
1802627716,44sleep17558-1kworker/1:0H09:03:391
1544827715,43sleep015521-21threads10:53:360
2808427612,50sleep128131-21cat11:43:381
2808427612,50sleep128131-21cat11:43:381
2612027621,29sleep10-21swapper/107:43:331
212927616,40sleep32132-21cron08:13:033
2055427619,41sleep320571-21uniq09:13:223
257812759,52sleep10-21swapper/109:33:191
2524927516,43sleep325272-21munin-run07:43:033
1216627524,31sleep112193-21munin-run07:03:041
2834627417,41sleep128361-21cron07:53:021
273292749,54sleep00-21swapper/011:42:200
273292749,54sleep00-21swapper/011:42:200
100632746,53sleep00-21swapper/010:33:290
3203827317,39sleep132054-21/usr/sbin/munin08:03:281
1886727313,44sleep018951-21memory11:08:250
1732827316,41sleep117484-21latency_hist07:18:061
920627217,34sleep19208-21cron10:33:021
828927222,28sleep00-21swapper/012:28:340
2557227217,26sleep30-21swapper/311:33:233
1675427211,24sleep00-21swapper/007:13:360
136342726,20sleep00-21swapper/010:48:270
344327116,40sleep13539-21cut12:13:051
344327116,40sleep13539-21cut12:13:051
2869827119,37sleep128849-21awk09:43:361
3218927018,25sleep00-21swapper/011:58:060
3218927018,25sleep00-21swapper/011:58:060
300427011,15sleep30-21swapper/310:08:413
1449427017,29sleep00-21swapper/007:08:240
1304127032,24sleep312619-21/usr/sbin/munin08:48:273
1246227011,14sleep10-21swapper/108:48:051
212672696,16sleep00-21swapper/011:15:560
1905026910,17sleep10-21swapper/111:08:301
1773326914,22sleep30-21swapper/307:18:243
1007826916,23sleep10-21swapper/108:38:321
699526817,24sleep30-21swapper/308:28:333
495426813,26sleep10-21swapper/108:23:181
2823526824,20sleep00-21swapper/009:43:230
2314626810,14sleep30-21swapper/309:23:483
2243826819,29sleep30-21swapper/311:23:183
2126226815,39sleep021310-21ntpq07:28:380
1117726814,33sleep311065-21/usr/sbin/munin08:43:233
1112826817,29sleep210845-21/usr/sbin/munin06:58:252
3147726718,27sleep00-21swapper/008:03:020
818126620,25sleep30-21swapper/308:33:263
3228026616,24sleep00-21swapper/008:03:350
2235526612,38sleep022425-21df11:23:150
2292426517,33sleep123017-21sed11:23:281
1844026517,20sleep00-21swapper/007:18:370
537326411,29sleep30-21swapper/308:23:283
52392647,17sleep00-21swapper/012:18:450
323312646,46sleep00-21swapper/011:58:140
323312646,46sleep00-21swapper/011:58:140
2837826417,23sleep30-21swapper/311:43:443
2837826417,23sleep30-21swapper/311:43:443
2597226425,22sleep30-21swapper/309:33:253
2177726417,24sleep10-21swapper/109:18:041
2026826421,21sleep30-21swapper/311:13:243
1397826414,23sleep00-21swapper/008:53:050
1168026411,16sleep00-21swapper/010:43:100
1160526419,20sleep30-21swapper/306:58:343
693926322,22sleep00-21swapper/010:23:280
693926322,22sleep00-21swapper/010:23:280
327826311,26sleep10-21swapper/108:13:371
324202637,11sleep10-21swapper/111:59:201
324202637,11sleep10-21swapper/111:59:201
2273226316,24sleep10-21swapper/109:23:171
209826312,37sleep02099-21cron10:08:020
209826312,37sleep02099-21cron10:08:020
1893526318,30sleep10-21swapper/109:08:191
1774626313,35sleep317793-21ntpq09:03:333
1774626313,35sleep317793-21ntpq09:03:333
616126212,34sleep16230-21latency_hist10:23:041
1187126219,23sleep10-21swapper/110:43:191
1142026216,28sleep011434-21apt10:43:020
65712617,12sleep30-21swapper/310:23:193
65712617,12sleep30-21swapper/310:23:193
31912619,40sleep00-21swapper/012:10:500
31912619,40sleep00-21swapper/012:10:500
3114526121,18sleep30-21swapper/309:53:193
2712826118,23sleep30-21swapper/311:39:443
2712826118,23sleep30-21swapper/311:39:443
2585826111,21sleep00-21swapper/009:33:220
2477726019,17sleep00-21swapper/009:28:330
1896826016,29sleep318975-21/usr/sbin/munin09:08:213
1428726016,29sleep114296-21df_inode08:53:181
1289926016,19sleep30-21swapper/307:03:303
1289926016,19sleep30-21swapper/307:03:303
2530425912,31sleep125420-21sed07:43:051
2244325916,26sleep122438-21apt-get07:33:161
1909525918,18sleep00-21swapper/007:23:210
1481725915,27sleep314836-21apt-get10:53:143
1313525921,20sleep00-21swapper/007:03:350
1313525921,20sleep00-21swapper/007:03:350
928425818,25sleep30-21swapper/310:33:043
53825816,25sleep0534-21apt-get12:03:110
2874925824,20sleep10-21swapper/107:53:221
1834925818,18sleep00-21swapper/011:08:050
1834925818,18sleep00-21swapper/011:08:050
1430225818,18sleep30-21swapper/308:53:203
1116225818,26sleep111168-21df_inode08:43:211
375925720,20sleep10-21swapper/112:13:211
375925720,20sleep10-21swapper/112:13:211
255172578,19sleep30-21swapper/307:43:153
180532577,13sleep30-21swapper/311:04:533
180532577,13sleep30-21swapper/311:04:533
1458525713,28sleep014618-21/usr/sbin/munin08:53:250
1197325715,27sleep312032-21latency10:43:213
857425612,24sleep00-21swapper/008:33:350
621525618,23sleep06069-21/usr/sbin/munin12:23:450
3022125614,26sleep130214-21apt-get07:58:191
2998125619,17sleep30-21swapper/307:58:053
2554025614,25sleep025536-21apt-get07:43:160
804325512,23sleep00-21swapper/010:28:240
639425513,25sleep16387-21apt-get10:23:121
639425513,25sleep16387-21apt-get10:23:121
387125510,26sleep03929-21if_eth012:13:240
387125510,26sleep03929-21if_eth012:13:240
326312555,12sleep30-21swapper/309:58:203
2944125516,21sleep10-21swapper/111:48:351
2944125516,21sleep10-21swapper/111:48:351
193462558,31sleep019367-21awk09:08:310
865625415,21sleep10-21swapper/108:33:361
642225412,23sleep16489-21df08:28:191
214825413,26sleep02222-21apt-config08:13:040
1628425416,18sleep00-21swapper/008:58:340
113725413,16sleep30-21swapper/310:03:243
113725413,16sleep30-21swapper/310:03:243
1083325417,21sleep310969-21latency_hist10:38:043
3177425314,21sleep30-21swapper/308:03:223
3062925313,24sleep330675-21ntp_states07:58:293
268925316,18sleep30-21swapper/308:13:193
1599525314,20sleep30-21swapper/307:13:193
1577225313,24sleep315769-21apt-get08:58:163
136972536,35sleep10-21swapper/107:07:561
136972536,35sleep10-21swapper/107:07:561
573225212,22sleep35256-21apt-get10:18:143
486725213,23sleep04862-21apt-get08:23:130
3270525217,20sleep30-21swapper/312:03:023
3270525217,20sleep30-21swapper/312:03:023
2572225214,16sleep10-21swapper/111:33:251
244272526,18sleep30-21swapper/309:28:243
2145425213,24sleep121479-21munin-run11:18:031
218525112,19sleep30-21swapper/312:08:263
218525112,19sleep30-21swapper/312:08:263
168972515,36sleep10-21swapper/109:01:171
464425012,23sleep14793-21date08:23:061
2727325014,21sleep30-21swapper/307:48:243
2211525017,19sleep10-21swapper/111:23:031
1240525013,21sleep012461-21sed08:48:040
1680224913,17sleep00-21swapper/011:03:040
153042498,16sleep10-21swapper/110:53:331
611224811,22sleep06159-21systemd-run10:23:030
286124812,18sleep00-21swapper/010:08:380
2284424812,14sleep00-21swapper/007:33:270
1575224813,15sleep10-21swapper/107:13:051
3134424714,19sleep131380-21irqstats09:53:371
2922224711,19sleep00-21swapper/011:48:170
2922224711,19sleep00-21swapper/011:48:170
2784424712,19sleep00-21swapper/009:43:070
1734524613,16sleep00-21swapper/009:03:220
1734524613,16sleep00-21swapper/009:03:220
570424410,16sleep30-21swapper/312:23:043
1143624411,20sleep111471-21munin-run10:43:031
557424314,15sleep10-21swapper/112:23:021
2236324310,18sleep31-21systemd09:23:033
277192419,18sleep10-21swapper/109:43:031
186402405,25sleep10-21swapper/107:21:001
3269223812,13sleep10-21swapper/109:59:021
2217423811,16sleep10-21swapper/109:20:351
268562376,13sleep10-21swapper/111:38:051
268562376,13sleep10-21swapper/111:38:051
167512379,13sleep30-21swapper/311:03:033
461923610,13sleep30-21swapper/308:23:043
164972366,13sleep10-21swapper/110:59:461
1137023613,15sleep30-21swapper/310:42:243
217512345,11sleep30-21swapper/311:18:583
326172337,14sleep00-21swapper/009:58:090
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional