You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-02-22 - 05:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s (updated Wed Feb 21, 2024 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22978285710,808sleep222992-21irqstats07:50:302
11114285173,692sleep211145-21install07:15:002
32448252313,496sleep332487-21ntpq08:20:313
24270250311,395sleep224335-21cpuspeed07:55:222
1217247413,444sleep11253-21ntpq10:05:381
16360240411,377sleep216467-21sort09:11:392
16360240411,377sleep216467-21sort09:11:392
25484235716,324sleep025569-21latency_hist11:25:000
16145232813,198sleep216284-21latency_hist10:55:022
96772317113,187sleep29756-21xargs12:15:422
3165122639,242sleep3372-21snmpd08:18:423
3165122639,242sleep3372-21snmpd08:18:423
15604225914,229sleep115745-21awk07:25:411
15604225914,229sleep115745-21awk07:25:411
30951223812,211sleep031018-21awk11:40:390
30951223812,211sleep031018-21awk11:40:390
12507222938,167sleep212634-21unixbench_singl07:15:412
15301222039,105sleep215329-21ntpq07:25:342
15301222039,105sleep215329-21ntpq07:25:342
26896221633,137sleep226922-21fschecks_count09:45:322
32351221512,191sleep232389-21ntp_kernel_pll_11:45:342
2087722009,176sleep221004-21wc07:45:002
13364219816,170sleep313387-21cut10:45:303
27650219514,90sleep327691-21ntp_kernel_err11:30:323
27650219514,90sleep327691-21ntp_kernel_err11:30:323
1840621916,173sleep218439-21ntpq07:35:362
1239221897,166sleep112506-21sed07:15:401
20800218410,157sleep220892-21latency_hist11:10:002
15370217910,155sleep215506-21apt-config09:10:022
15370217910,155sleep215506-21apt-config09:10:022
2694217415,56sleep20-21swapper/210:10:312
9342217326,52sleep20-21swapper/208:50:352
2995221727,114sleep229954-21sshd08:11:542
2995221727,114sleep229954-21sshd08:11:542
21129217114,146sleep221200-21/usr/sbin/munin07:45:252
1884216798,21sleep20-21swapper/208:25:392
5934216639,48sleep25997-21fschecks_time08:40:292
18123216413,81sleep20-21swapper/211:00:272
7515216117,132sleep27554-21processes10:25:382
10864215524,39sleep20-21swapper/208:55:322
747421519,52sleep20-21swapper/212:10:092
25432215121,70sleep20-21swapper/211:24:582
30299214812,78sleep230337-21ntp_kernel_pll_09:55:332
30299214812,78sleep230337-21ntp_kernel_pll_09:55:332
20338214516,115sleep220348-21latency_hist09:25:002
27402214412,40sleep20-21swapper/208:05:242
1875421447,15sleep20-21swapper/209:19:592
14498214315,110sleep214644-21unixbench_singl12:30:412
11164214130,100sleep2372-21snmpd10:37:012
29525213715,102sleep029595-21switchtime08:10:430
29525213715,102sleep029595-21switchtime08:10:430
2402321369,40sleep20-21swapper/209:35:302
12208213637,83sleep20-21swapper/212:25:242
474321358,74sleep24807-21ps08:35:412
12854213446,24sleep20-21swapper/209:00:532
20289212918,93sleep11-21systemd09:24:591
1948212625,90sleep21961-21awk11:50:432
908212514,94sleep0924-21fschecks_count10:05:290
18092212519,84sleep018129-21fschecks_count07:35:290
30072212122,80sleep330093-21modprobe11:39:573
6505211811,84sleep20-21swapper/212:05:452
31447211819,40sleep20-21swapper/210:00:182
16775211829,78sleep20-21swapper/210:55:342
16775211829,78sleep20-21swapper/210:55:342
13759211515,17sleep20-21swapper/207:20:332
11944211510,93sleep2231-21systemd-journal12:24:592
28921211423,34sleep20-21swapper/211:35:282
26778211429,30sleep20-21swapper/211:27:242
21929211313,81sleep322029-21users07:45:423
12241211319,50sleep212320-21mailstats10:40:562
665821126,29sleep20-21swapper/210:23:522
15670211214,81sleep015134-21apt-get09:10:280
15670211214,81sleep015134-21apt-get09:10:280
9088211116,70sleep00-21swapper/008:50:290
15835211110,88sleep215903-21mailstats12:35:372
13312211010,85sleep00-21swapper/009:01:440
9765210923,35sleep20-21swapper/210:33:542
2740121096,16sleep20-21swapper/211:30:272
2740121096,16sleep20-21swapper/211:30:272
22670210812,39sleep20-21swapper/209:30:372
13732210823,58sleep30-21swapper/309:05:023
1352221088,15sleep20-21swapper/210:45:332
26526210710,41sleep20-21swapper/209:44:592
21213210711,82sleep121236-21cut07:45:281
2723210612,34sleep20-21swapper/208:30:202
12325210618,23sleep00-21swapper/007:15:390
1136210619,72sleep01206-21fschecks_time11:50:280
1645021058,16sleep20-21swapper/207:30:182
27864210319,71sleep027871-21awk08:05:380
3112521029,13sleep20-21swapper/211:40:412
3112521029,13sleep20-21swapper/211:40:412
12610210213,68sleep112640-21ntp_kernel_pll_12:25:351
19406210116,66sleep019012-21/usr/sbin/munin09:20:300
7265210023,63sleep37387-21latency_hist12:10:023
27492210018,63sleep327642-21sed09:45:453
13463210014,66sleep113521-21/usr/sbin/munin10:45:321
2715129916,63sleep327160-21latency_hist08:05:013
226742988,37sleep20-21swapper/211:15:252
1698729814,68sleep017105-21timerandwakeup10:55:400
1698729814,68sleep017105-21timerandwakeup10:55:400
10952988,41sleep21097-21irqstats10:05:352
2595229715,60sleep026099-21tail09:40:410
26752967,35sleep20-21swapper/211:55:232
2365429628,35sleep10-21swapper/109:35:171
440029414,64sleep34477-21mailstats10:15:373
320432947,44sleep20-21swapper/208:20:102
2785529423,53sleep327376-21/usr/sbin/munin08:05:373
778429313,64sleep07796-21cut12:10:310
3234929318,60sleep132417-21latency08:20:291
1821529310,66sleep018242-21kernelversion11:00:290
1544329225,50sleep315565-21timerandwakeup10:50:433
1488428919,27sleep30-21swapper/307:25:193
1488428919,27sleep30-21swapper/307:25:193
831628833,45sleep20-21swapper/208:45:512
48292885,46sleep20-21swapper/210:15:442
1198328715,57sleep012028-21munin-plugin-st08:59:580
211632866,67sleep20-21swapper/209:25:412
1040128620,46sleep010453-21latency_hist12:20:000
52802855,46sleep20-21swapper/212:03:022
293362856,28sleep20-21swapper/209:51:372
1798928511,57sleep018065-21grep09:16:060
1661928418,46sleep016638-21latency_hist12:39:590
138528425,25sleep30-21swapper/308:25:283
2333728216,48sleep023439-21cat07:50:400
1226228212,57sleep312323-21/usr/sbin/munin07:15:383
267592816,64sleep20-21swapper/208:00:442
22162816,12sleep10-21swapper/111:53:571
210072815,28sleep20-21swapper/211:10:042
1073628114,53sleep310791-21grep12:20:243
3170028014,49sleep131762-21latency_hist11:45:001
3170028014,49sleep131762-21latency_hist11:45:001
2393528018,41sleep124059-21cat11:20:001
162528020,40sleep31698-21grep11:50:403
1511528014,49sleep315178-21gpgv09:09:593
1511528014,49sleep315178-21gpgv09:09:593
674427911,50sleep3538-21cron10:24:583
384227819,39sleep03867-21munin-run11:59:570
2387427816,29sleep30-21swapper/309:35:273
2418327713,49sleep124228-21grep11:20:211
221432779,39sleep00-21swapper/011:12:570
1550827711,12sleep10-21swapper/112:35:281
1249727715,42sleep112628-21sed10:40:591
1023827717,40sleep310317-21fschecks_time10:35:283
737027623,28sleep00-21swapper/008:45:200
600127612,50sleep10-21swapper/112:05:321
3016727614,43sleep330188-21tr09:55:303
3016727614,43sleep330188-21tr09:55:303
2852627624,35sleep028537-21cron11:34:580
2852627624,35sleep028537-21cron11:34:580
39152759,53sleep03927-21cpu10:15:220
2586527524,28sleep30-21swapper/309:40:403
1675127514,36sleep116428-21/usr/sbin/munin07:30:301
115227514,47sleep11171-21apt-get08:25:181
1139727523,35sleep111416-21idleruntime-cro10:39:581
2934627421,21sleep00-21swapper/011:35:380
2833127412,22sleep00-21swapper/009:50:220
416627325,28sleep10-21swapper/112:00:191
2426927310,47sleep10-21swapper/107:55:211
1396727319,25sleep10-21swapper/109:05:361
1396727319,25sleep10-21swapper/109:05:361
1118027319,35sleep311301-21sed07:15:013
833627218,29sleep00-21swapper/010:30:000
2777227216,40sleep127811-21ntpq08:05:351
1612127218,40sleep116144-21sed10:55:011
955027120,22sleep30-21swapper/307:10:003
3164427119,41sleep20-21swapper/208:18:372
3164427119,41sleep20-21swapper/208:18:372
2959727115,41sleep129696-21/usr/sbin/munin08:10:441
2959727115,41sleep129696-21/usr/sbin/munin08:10:441
1971427117,28sleep30-21swapper/309:20:373
1199327114,41sleep012114-21sed12:25:010
1045627110,45sleep110599-21latency_hist12:20:011
947927024,20sleep10-21swapper/107:09:581
464127013,40sleep34674-21/usr/sbin/munin08:35:343
397327015,41sleep14015-21true08:34:591
2318627014,23sleep10-21swapper/107:50:371
1692727019,32sleep316977-21apt-config09:14:593
1692727019,32sleep316977-21apt-config09:14:593
1465927022,31sleep114709-21wc07:25:001
1257927018,25sleep00-21swapper/012:25:340
1032227015,24sleep00-21swapper/010:35:290
286726911,41sleep32941-21mailstats10:10:413
2324326919,23sleep00-21swapper/011:15:390
1595126921,30sleep10-21swapper/109:11:261
1595126921,30sleep10-21swapper/109:11:261
116992695,17sleep10-21swapper/108:56:201
330926810,45sleep00-21swapper/008:30:380
2096026810,41sleep020995-21ntpq09:25:350
1710726823,21sleep10-21swapper/110:55:411
1710726823,21sleep10-21swapper/110:55:411
1500026821,26sleep30-21swapper/312:34:583
1019626816,36sleep110226-21awk08:54:581
323226720,29sleep33335-21/usr/sbin/munin11:55:363
3225326714,22sleep00-21swapper/011:45:320
2916126720,22sleep30-21swapper/308:10:343
2916126720,22sleep30-21swapper/308:10:343
77272665,11sleep30-21swapper/312:10:303
3270226615,32sleep1389-21grep11:45:411
2883426621,20sleep10-21swapper/109:50:521
2688926617,28sleep10-21swapper/108:02:161
1530026621,21sleep30-21swapper/312:35:223
1229426617,35sleep312382-21tune2fs12:25:293
2701926515,35sleep027059-21/usr/sbin/munin09:45:340
2580326516,29sleep10-21swapper/111:25:251
1925626519,25sleep30-21swapper/311:05:003
1742526514,34sleep117435-21find09:15:181
146702655,12sleep10-21swapper/112:30:441
871126414,27sleep10-21swapper/110:30:251
2419026420,26sleep00-21swapper/009:35:360
1984926414,31sleep019868-21kernelversion07:40:300
1858526419,19sleep10-21swapper/111:00:381
1393626418,23sleep314019-21cat07:20:383
2613926321,22sleep30-21swapper/311:25:363
232062637,11sleep10-21swapper/109:32:421
1180026314,20sleep00-21swapper/010:40:260
993826221,23sleep10-21swapper/107:10:271
876426217,20sleep20-21swapper/207:05:362
308026218,29sleep13164-21grep11:55:341
2894526216,24sleep30-21swapper/309:50:543
2240626213,22sleep30-21swapper/309:30:303
2151526212,36sleep321535-21open_inodes11:10:373
750826114,24sleep10-21swapper/110:25:371
3162526117,27sleep031684-21/usr/sbin/munin10:00:280
2909826121,22sleep10-21swapper/111:35:311
2425626111,33sleep324264-21diskstats11:20:233
1047626120,24sleep00-21swapper/007:10:380
990226012,23sleep00-21swapper/010:35:010
3208826017,27sleep332184-21timerandwakeup10:00:393
133362607,11sleep10-21swapper/109:02:011
50302595,15sleep10-21swapper/110:18:101
461525919,18sleep00-21swapper/012:00:300
3051425919,24sleep10-21swapper/109:55:401
3051425919,24sleep10-21swapper/109:55:401
975825818,25sleep30-21swapper/312:15:433
951125818,21sleep30-21swapper/308:50:403
2630925817,21sleep00-21swapper/008:00:370
2559425818,22sleep00-21swapper/008:00:010
137182588,38sleep00-21swapper/012:30:100
755525719,19sleep30-21swapper/310:25:393
595025714,25sleep06000-21irqstats12:05:310
3203525714,27sleep332037-21entropy11:45:273
277872574,11sleep10-21swapper/109:46:361
2299425717,20sleep30-21swapper/307:50:313
1672125710,34sleep016750-21cut07:30:290
1381525717,19sleep00-21swapper/007:20:350
1111125715,26sleep311144-21/usr/sbin/munin08:55:373
61925614,27sleep3702-21dpkg10:05:073
2703325620,19sleep00-21swapper/011:30:010
2506625614,23sleep325169-21users07:55:403
2183225613,29sleep321872-21modprobe09:29:583
204872565,13sleep30-21swapper/311:06:383
1958125611,31sleep119622-21cpu07:40:231
1935425613,18sleep30-21swapper/307:40:013
763625510,33sleep37664-21irqstats08:45:313
590625516,22sleep10-21swapper/110:20:361
2286925516,21sleep30-21swapper/311:15:303
2159825513,27sleep121661-21/usr/sbin/munin11:10:401
2001125514,23sleep00-21swapper/011:05:420
1977825516,20sleep10-21swapper/111:05:351
1775325513,17sleep30-21swapper/307:35:003
416325415,23sleep34159-21apt-get12:00:173
2788525411,21sleep10-21swapper/111:30:391
2788525411,21sleep10-21swapper/111:30:391
276925411,18sleep012-21rcu_sched10:10:330
245325413,17sleep10-21swapper/110:10:261
2108425416,21sleep121162-21mailstats09:25:401
1072425414,22sleep110730-21df_inode12:20:221
611525318,19sleep10-21swapper/108:40:321
434925311,17sleep00-21swapper/008:35:270
3147225311,25sleep131466-21apt-get10:00:211
3047725310,27sleep330483-21df_inode11:40:263
3047725310,27sleep330483-21df_inode11:40:263
280125313,24sleep32808-21diskstats08:30:253
2610125310,30sleep126162-21users09:40:421
2134025312,24sleep021397-21irqstats07:45:310
1820525313,19sleep30-21swapper/309:16:093
1518325311,25sleep015277-21memory07:25:320
1518325311,25sleep015277-21memory07:25:320
97225212,24sleep0895-21latency_hist08:25:010
739725213,25sleep17464-21/usr/sbin/munin08:45:251
634925214,22sleep00-21swapper/008:40:400
550025214,18sleep00-21swapper/010:20:230
248025212,19sleep00-21swapper/008:30:010
1388125213,22sleep013865-21apt-get09:05:310
1388125213,22sleep013865-21apt-get09:05:310
1374125214,19sleep00-21swapper/010:45:390
1800325111,23sleep118000-21apt-get07:35:221
1495725115,18sleep00-21swapper/010:50:310
631125014,22sleep36343-21proc_pri08:40:393
435725014,19sleep10-21swapper/108:35:281
3095025012,21sleep030956-21ntp_states08:15:360
3095025012,21sleep030956-21ntp_states08:15:360
1200125013,23sleep312078-21sed10:40:333
882824911,14sleep30-21swapper/310:30:273
1997824914,22sleep320019-21ntpq07:40:333
2226324812,18sleep00-21swapper/009:30:270
884624710,16sleep10-21swapper/112:15:011
869124713,16sleep00-21swapper/007:05:330
163712477,19sleep30-21swapper/310:55:233
163712477,19sleep30-21swapper/310:55:233
3182824612,15sleep10-21swapper/108:20:011
3182824612,15sleep10-21swapper/108:20:011
2998724614,17sleep00-21swapper/009:55:270
2998724614,17sleep00-21swapper/009:55:270
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional