You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-25 - 21:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Thu Apr 25, 2024 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2930521693414,16900sleep125644-21kworker/u8:007:26:401
2930521693414,16900sleep125644-21kworker/u8:007:26:401
6591288420,846sleep26635-21/usr/sbin/munin07:56:462
17026275616,722sleep217031-21/usr/sbin/munin10:11:332
31303255820,520sleep231309-21/usr/sbin/munin07:31:462
2430252330,480sleep02478-21unin-run09:26:170
27108244014,404sleep027200-21fschecks_time12:26:360
467824120,7sleep20-21swapper/211:16:192
26189235540,260sleep026308-21unixbench_multi12:21:520
71502292117,90sleep00-21swapper/009:41:190
24465228310,240sleep024528-21fschecks_time07:11:370
1494922817,263sleep01166-21snmpd10:04:080
1907922758,231sleep00-21swapper/010:16:480
27886226665,183sleep027961-21sendmail_mailtr09:01:500
826022618,215sleep08302-21awk11:26:430
9255225811,237sleep19005-21kworker/u8:108:06:261
29321222756,65sleep00-21swapper/007:26:510
29321222756,65sleep00-21swapper/007:26:510
24331222712,200sleep124339-21awk07:11:261
2163222517,100sleep00-21swapper/011:06:490
8912221314,156sleep00-21swapper/009:46:280
32094221216,182sleep032140-21awk07:36:160
27975220118,22sleep10-21swapper/110:46:361
23549220136,22sleep00-21swapper/010:31:410
22318220027,79sleep022447-21unixbench_multi10:26:490
15798219929,86sleep00-21swapper/010:06:430
140992198135,18sleep00-21swapper/011:46:180
18357219410,167sleep018459-21timerwakeupswit11:56:480
7828219379,57sleep07887-21fschecks_time08:01:360
14888219053,115sleep015035-21grep08:21:520
24734217817,41sleep00-21swapper/010:36:260
3211821776,160sleep01166-21snmpd10:59:390
28848217526,17sleep00-21swapper/010:47:520
22098217544,105sleep122137-21munin-run07:06:171
3462174100,58sleep00-21swapper/011:01:400
10904217336,86sleep01166-21snmpd11:36:160
29162216819,130sleep329215-21sort10:51:183
26141216517,72sleep00-21swapper/008:56:570
20745216595,19sleep00-21swapper/008:41:340
2761821639,139sleep027728-21cat10:46:190
24027216029,98sleep30-21swapper/308:51:473
10274215728,111sleep310376-21cat11:31:593
21570215539,91sleep221739-21sed07:01:502
2133921539,125sleep021362-21sh10:26:160
484921517,131sleep04863-21grep11:16:340
4405215113,121sleep14343-21latency_hist07:51:191
22869214743,68sleep00-21swapper/012:11:500
1802121459,79sleep00-21swapper/008:31:490
20348214416,113sleep120357-21kernelversion10:21:371
12064214315,110sleep212208-21cat09:56:192
12064214315,110sleep212208-21cat09:56:192
12481213713,16sleep30-21swapper/309:56:363
29616213611,113sleep029667-21cat09:06:510
25239213152,22sleep00-21swapper/012:21:190
6973213027,51sleep07099-21sed11:21:510
6973213027,51sleep07099-21sed11:21:510
29976212814,27sleep130045-21sed10:51:491
1240821288,83sleep012432-21latency_hist08:16:180
5412212622,24sleep30-21swapper/309:34:523
17069212613,38sleep017152-21fschecks_time10:11:350
15977212316,89sleep315484-21apt-get08:26:273
629212114,88sleep0738-21timerwakeupswit07:36:490
5185212110,98sleep05254-21wc07:51:470
15475212134,68sleep00-21swapper/011:46:540
1120021218,70sleep00-21swapper/011:36:280
1120021218,70sleep00-21swapper/011:36:280
974321198,99sleep219378-1kworker/2:1H11:31:382
5600211912,56sleep00-21swapper/009:36:180
29899211910,84sleep029974-21grep10:51:480
26143211917,51sleep00-21swapper/007:16:380
12762211838,36sleep00-21swapper/009:56:460
29194211610,16sleep00-21swapper/012:31:530
10585211633,41sleep00-21swapper/008:11:170
6504211240,33sleep00-21swapper/007:56:440
18505211116,25sleep20-21swapper/210:16:252
29293210921,68sleep229278-21apt-get07:26:332
29293210921,68sleep229278-21apt-get07:26:332
3827210815,24sleep03894-21timerandwakeup11:11:500
22426210732,36sleep00-21swapper/007:06:280
22426210732,36sleep00-21swapper/007:06:280
2221721067,49sleep00-21swapper/008:46:310
18217210618,70sleep118221-21cron10:16:161
10700210617,67sleep310209-21apt-get09:51:283
10700210617,67sleep310209-21apt-get09:51:283
4266210537,39sleep20-21swapper/209:31:322
23777210512,75sleep00-21swapper/008:51:200
2661210334,55sleep00-21swapper/007:44:520
16070210316,66sleep115955-21/usr/sbin/munin08:26:531
12433210218,69sleep112581-21latency_hist08:16:191
11056210025,54sleep210554-21apt-get08:11:282
2209629927,19sleep00-21swapper/012:11:170
2209629927,19sleep00-21swapper/012:11:170
2269529812,28sleep222403-21/usr/sbin/munin07:06:382
2269529812,28sleep222403-21/usr/sbin/munin07:06:382
918929714,56sleep29310-21cat11:31:172
7232975,11sleep00-21swapper/009:19:140
452929714,65sleep14677-21sed11:16:191
2798829714,42sleep00-21swapper/007:21:510
1776629719,51sleep317831-21fschecks_time11:56:343
1678629723,54sleep116918-21sed11:51:481
201072965,11sleep00-21swapper/008:37:510
2992729433,31sleep330015-21timerandwakeup07:27:073
2992729433,31sleep330015-21timerandwakeup07:27:073
2953729414,60sleep229375-21/usr/sbin/munin10:51:372
2434729411,20sleep30-21swapper/307:11:283
2132729430,45sleep321386-21find07:01:473
1339729417,38sleep00-21swapper/008:16:510
1339729417,38sleep00-21swapper/008:16:510
473429113,62sleep14747-21/usr/sbin/munin09:31:511
2518729119,51sleep125217-21cut12:21:161
1805929119,56sleep118089-21ntpq11:56:411
735829013,63sleep27360-21acpi09:41:252
2243129024,50sleep322486-21fschecks_count12:11:373
602228930,32sleep20-21swapper/211:21:162
3020028920,49sleep230215-21ls09:11:172
1984628915,21sleep30-21swapper/310:21:193
1297928813,59sleep213030-21if_eth011:41:392
401028632,13sleep00-21swapper/009:31:180
1023428624,37sleep110263-21munin-run09:51:171
2911428434,30sleep20-21swapper/210:51:172
2271428420,22sleep10-21swapper/112:11:431
172482848,43sleep017278-21munin-run08:31:170
689028219,47sleep36969-21switchtime11:21:503
689028219,47sleep36969-21switchtime11:21:503
1729828223,25sleep20-21swapper/208:31:192
1134828213,56sleep111418-21timerandwakeup09:51:541
1134828213,56sleep111418-21timerandwakeup09:51:541
2673328111,55sleep126775-21systemd-run12:26:171
1361828124,31sleep10-21swapper/110:01:191
2084627919,39sleep020819-21/usr/sbin/munin07:01:320
1526127923,35sleep115381-21sed11:46:511
911927816,43sleep19187-21apt-key11:31:161
774927829,13sleep00-21swapper/009:41:390
163327815,30sleep30-21swapper/311:06:323
436227730,32sleep00-21swapper/007:51:180
1396627712,49sleep114059-21latency_hist08:21:201
1396627712,49sleep114059-21latency_hist08:21:201
2347527617,45sleep123515-21http08:51:171
1253327617,45sleep212670-21latency_hist11:41:192
1253327617,45sleep212670-21latency_hist11:41:192
756827519,37sleep17583-21idleruntime-cro11:26:161
756827519,37sleep17583-21idleruntime-cro11:26:161
506827516,26sleep10-21swapper/107:51:441
1511227523,31sleep20-21swapper/210:06:162
832227416,37sleep38345-21/usr/sbin/munin11:26:463
2444227418,40sleep324478-21sendmail10:36:163
1932327419,29sleep20-21swapper/212:01:342
1932327419,29sleep20-21swapper/212:01:342
1560527418,32sleep315474-21latency_hist08:26:193
1168627419,25sleep30-21swapper/308:12:053
1054127411,19sleep30-21swapper/308:11:163
282127322,25sleep10-21swapper/107:46:191
124227314,22sleep10-21swapper/109:21:331
2866427214,28sleep10-21swapper/112:31:381
2296027219,38sleep222911-21latency_hist10:31:192
2639527115,30sleep20-21swapper/210:41:362
199972718,13sleep00-21swapper/012:01:480
199972718,13sleep00-21swapper/012:01:480
1071527111,37sleep00-21swapper/009:51:290
1071527111,37sleep00-21swapper/009:51:290
58202706,15sleep10-21swapper/109:36:271
2555127014,42sleep225557-21diskstats12:21:332
2351627026,28sleep223534-21munin-run08:51:182
2191927025,25sleep10-21swapper/108:46:161
161002707,14sleep00-21swapper/011:51:270
3261626912,35sleep232683-21/usr/sbin/munin09:16:462
2982626923,30sleep329836-21modprobe12:36:163
2585026919,24sleep10-21swapper/108:56:381
1986626919,28sleep30-21swapper/312:01:473
1986626919,28sleep30-21swapper/312:01:473
3255726821,32sleep132615-21/usr/sbin/munin09:16:451
2204526816,37sleep222075-21munin-run12:11:152
2204526816,37sleep222075-21munin-run12:11:152
1879226818,32sleep318796-21cron08:36:163
1600926819,21sleep20-21swapper/210:06:492
904426722,24sleep10-21swapper/108:06:181
397526721,31sleep34004-21awk09:31:173
2966826718,30sleep129767-21grep09:06:521
276926716,32sleep32785-21ls07:46:173
1276926710,39sleep10-21swapper/111:41:301
317052667,12sleep00-21swapper/009:15:390
317052667,12sleep00-21swapper/009:15:390
3123226623,23sleep30-21swapper/309:11:503
3123226623,23sleep30-21swapper/309:11:503
308242667,15sleep10-21swapper/107:31:291
274826611,39sleep22787-21df09:26:312
2436026611,36sleep224389-21ntp_states12:16:492
2403526616,29sleep20-21swapper/210:31:512
74702656,37sleep30-21swapper/309:41:343
54426519,30sleep2566-21/usr/sbin/munin11:01:472
2784126512,21sleep20-21swapper/207:21:442
259412654,11sleep30-21swapper/310:40:163
2360526519,29sleep323771-21apt-key08:51:193
2091926525,24sleep30-21swapper/308:41:373
1874126515,23sleep10-21swapper/110:16:371
3057526417,27sleep10-21swapper/109:11:361
3057526417,27sleep10-21swapper/109:11:361
2609526418,23sleep10-21swapper/110:41:191
248026419,29sleep22613-21latency_hist09:26:192
2457326420,23sleep20-21swapper/208:52:062
2378526415,33sleep323791-21munin-node07:11:203
2378526415,33sleep323791-21munin-node07:11:203
2293826416,26sleep10-21swapper/110:31:181
1464426424,20sleep30-21swapper/311:46:373
1193626417,30sleep312076-21unixbench_singl11:36:523
1193626417,30sleep312076-21unixbench_singl11:36:523
208682639,20sleep20-21swapper/208:41:362
1925626318,31sleep119326-21basename08:36:351
1745326318,24sleep30-21swapper/310:11:463
1447226316,30sleep114531-21/usr/sbin/munin08:21:401
680426216,28sleep16921-21cat07:56:531
2833426219,27sleep30-21swapper/312:31:193
243722624,47sleep10-21swapper/110:35:221
771626117,27sleep37728-21apt-get08:01:273
3094926122,23sleep20-21swapper/209:11:452
3094926122,23sleep20-21swapper/209:11:452
2680826116,30sleep326957-21sed12:26:193
2628226112,27sleep126293-21acpi10:41:271
2320026115,25sleep10-21swapper/107:06:491
2320026115,25sleep10-21swapper/107:06:491
2067926114,33sleep220755-21/usr/sbin/munin10:21:462
170272615,11sleep20-21swapper/211:51:572
8992606,40sleep10-21swapper/107:37:061
2802626018,27sleep228136-21sort09:01:522
2574626014,19sleep20-21swapper/207:16:192
186372606,12sleep20-21swapper/211:57:182
1583226016,27sleep115855-21ntp_states10:06:441
992725912,27sleep10-21swapper/111:31:491
2817325919,23sleep30-21swapper/307:21:543
2706325917,26sleep226861-21apt-get09:01:192
2451525915,28sleep124572-21/usr/sbin/munin08:52:051
212742594,19sleep30-21swapper/310:25:273
1931825913,29sleep119322-21diskstats12:01:331
1931825913,29sleep119322-21diskstats12:01:331
1010425914,26sleep210202-21users08:06:592
961325818,21sleep30-21swapper/309:46:513
644625818,22sleep30-21swapper/309:36:503
53672588,13sleep20-21swapper/207:51:492
3229425818,18sleep20-21swapper/211:01:182
2169825810,23sleep30-21swapper/310:26:323
1284925813,17sleep30-21swapper/308:16:383
1284925813,17sleep30-21swapper/308:16:383
681725712,26sleep20-21swapper/211:21:492
681725712,26sleep20-21swapper/211:21:492
511325715,20sleep30-21swapper/307:51:463
2664225715,25sleep126770-21unixbench_multi07:16:511
1610425714,26sleep316162-21cpuspeed11:51:303
627325614,25sleep30-21swapper/307:56:393
294725614,27sleep23040-21latency_hist11:11:192
1504525615,24sleep20-21swapper/211:46:482
66825518,17sleep10-21swapper/111:01:501
3238625513,24sleep332404-21apt-get07:36:273
306225513,24sleep33129-21df07:46:333
2870325511,32sleep128818-21latency_hist09:06:191
2578525520,16sleep30-21swapper/308:56:363
2079325514,25sleep120784-21apt-get12:06:341
2079325514,25sleep120784-21apt-get12:06:341
1979025512,22sleep20-21swapper/208:36:482
3253425413,24sleep332529-21apt-get11:01:313
3097625413,24sleep330971-21apt-get10:56:313
271625416,18sleep10-21swapper/109:26:301
2661025414,25sleep326676-21memory10:41:413
2595525414,23sleep325953-21apt-get07:16:283
1405925420,19sleep30-21swapper/311:46:173
1155425414,24sleep111593-21memory11:36:421
1155425414,24sleep111593-21memory11:36:421
820825317,17sleep20-21swapper/208:01:462
2763325321,19sleep20-21swapper/212:26:502
2400925317,20sleep30-21swapper/312:16:383
2258725314,24sleep222614-21irqstats12:11:402
127325320,18sleep20-21swapper/207:41:192
125822535,10sleep20-21swapper/208:16:202
125822535,10sleep20-21swapper/208:16:202
1028125316,20sleep20-21swapper/209:51:182
607525215,20sleep10-21swapper/111:21:191
347125221,15sleep20-21swapper/207:46:442
3166925214,21sleep331742-21users07:31:513
2881525214,15sleep10-21swapper/107:26:201
2758925214,21sleep30-21swapper/309:01:413
2695225213,23sleep226953-21sshd07:17:532
2475625212,24sleep224752-21apt-get10:36:282
208622529,16sleep30-21swapper/312:06:363
208622529,16sleep30-21swapper/312:06:363
202572525,14sleep30-21swapper/308:39:483
1670125217,17sleep30-21swapper/310:11:183
2990825117,17sleep20-21swapper/212:36:192
2745525112,22sleep127479-21ntp_states12:26:451
142042519,16sleep20-21swapper/208:21:332
3243125019,17sleep30-21swapper/309:16:393
1887425012,17sleep20-21swapper/208:36:192
170325012,21sleep11709-21df_inode11:06:341
1446325015,19sleep30-21swapper/310:01:493
709824913,22sleep27126-21munin-run09:41:172
3131724911,23sleep131345-21memory10:56:411
2977224914,17sleep30-21swapper/310:51:423
151324912,21sleep21504-21apt-get07:41:312
775524813,15sleep10-21swapper/109:41:401
2893624811,20sleep328933-21apt-get09:06:293
1753924811,22sleep117605-21df08:31:341
354624710,15sleep30-21swapper/311:11:413
1385524710,20sleep10-21swapper/110:01:311
1320124712,21sleep213213-21sshd09:56:522
484124511,20sleep34847-21diskstats11:16:333
12572429,17sleep31349-21perl09:21:353
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional