You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-10-10 - 21:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Thu Oct 10, 2024 12:43:53)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
224082564612,5613sleep322365-21apt-get07:25:103
224082564612,5613sleep322365-21apt-get07:25:103
28082261683,517sleep028083-21diskstats11:09:510
29691257917,545sleep029724-21/usr/sbin/munin09:29:570
27437257118,538sleep127460-21/usr/sbin/munin07:40:111
1229243221,399sleep21276-21ntpq08:00:022
2099023998,375sleep021028-21ntpq07:19:590
183562358109,189sleep218439-21latency_hist07:14:352
25298223218,124sleep20-21swapper/211:00:002
25298223218,124sleep20-21swapper/211:00:002
16909223034,153sleep217063-21date10:34:362
2235422129,190sleep3371-21snmpd09:06:583
1806220815,18sleep10-21swapper/109:44:561
21900220762,21sleep20-21swapper/207:24:372
14402220427,139sleep214448-21ntpq10:24:592
13274220434,115sleep20-21swapper/208:39:362
285322036,181sleep32861-21ntpq08:05:163
9083220024,134sleep29058-21latency_hist10:09:372
23946219217,101sleep20-21swapper/207:30:022
2959121908,143sleep229649-21sed09:29:552
5227218445,122sleep25287-21fschecks_count11:39:542
24485218263,105sleep224499-21fschecks_count09:14:532
17874218257,88sleep217884-21modprobe08:54:342
10002218111,118sleep210164-21unixbench_multi10:10:052
2554021749,16sleep10-21swapper/107:34:501
6938216821,129sleep37086-21grep10:00:093
31817216765,45sleep20-21swapper/211:19:592
31058216710,120sleep231071-21awk07:50:172
4049215713,127sleep34127-21grep11:35:023
18491215711,107sleep2230-21systemd-journal10:39:372
18491215711,107sleep2230-21systemd-journal10:39:372
18284215371,54sleep20-21swapper/208:54:542
31667215187,18sleep20-21swapper/207:54:452
2482215110,119sleep20-21swapper/208:05:072
30643214714,78sleep229759-21/usr/sbin/munin11:15:272
1507214218,103sleep01639-21tail08:00:080
21531213754,72sleep221575-21latency_hist10:49:342
21531213754,72sleep221575-21latency_hist10:49:342
1136621376,120sleep00-21swapper/012:08:260
1136621376,120sleep00-21swapper/012:08:260
7177213521,92sleep07186-21fschecks_count08:19:550
2630821359,27sleep226313-21diskstats09:19:542
2630821359,27sleep226313-21diskstats09:19:542
28696213423,91sleep028739-21fschecks_count07:45:020
28696213423,91sleep028739-21fschecks_count07:45:020
13510213323,53sleep20-21swapper/208:39:482
7841213018,95sleep07847-21df_inode10:04:500
2627321298,17sleep20-21swapper/211:04:362
2627321298,17sleep20-21swapper/211:04:362
10474212920,89sleep010489-21cut08:29:520
1779212811,103sleep20-21swapper/211:29:362
482321249,102sleep24879-21sendmail-msp11:39:332
15570212311,51sleep215593-21munin-run12:39:332
4301211914,50sleep20-21swapper/208:10:082
8840211820,80sleep38324-21apt-get08:24:513
22904211712,54sleep20-21swapper/207:25:502
22904211712,54sleep20-21swapper/207:25:502
27553211612,81sleep227617-21mailstats07:40:162
17632211612,91sleep117699-21switchtime07:10:061
10878211614,58sleep20-21swapper/208:30:032
2218121159,53sleep20-21swapper/210:49:592
2218121159,53sleep20-21swapper/210:49:592
97221139,65sleep2230-21systemd-journal09:40:542
14370211329,30sleep214518-21cat12:29:362
5749211216,80sleep15851-21sed11:40:051
26447211227,44sleep30-21swapper/311:04:463
26447211227,44sleep30-21swapper/311:04:463
852421118,62sleep28679-21apt-key08:24:372
26909211112,84sleep126197-21/usr/sbin/munin09:20:091
26909211112,84sleep126197-21/usr/sbin/munin09:20:091
29077210911,68sleep10-21swapper/107:45:111
29077210911,68sleep10-21swapper/107:45:111
8320210818,75sleep08389-21fschecks_time11:49:530
12134210849,44sleep2305-21systemd-logind10:19:342
29751210720,66sleep329271-21apt-get11:14:433
13050210712,53sleep20-21swapper/212:19:352
3541210614,77sleep13550-21sh09:50:011
1996621065,43sleep20-21swapper/207:15:452
1003821067,65sleep2371-21snmpd08:28:452
21911210531,20sleep222034-21/usr/sbin/munin09:05:042
2461210413,44sleep20-21swapper/209:45:102
1240210414,22sleep30-21swapper/311:25:053
29012210349,16sleep229058-21ntpq07:45:082
29012210349,16sleep229058-21ntpq07:45:082
23228210313,68sleep20-21swapper/209:10:002
12525210323,17sleep20-21swapper/210:19:502
3930210227,48sleep03919-21apt-get08:09:530
1942210213,75sleep01980-21ntp_kernel_pll_09:44:590
29032210117,56sleep029036-21modprobe09:29:330
15032210110,38sleep215043-21sort08:44:442
25680210013,68sleep125798-21users11:00:071
25680210013,68sleep125798-21users11:00:071
2016210019,17sleep20-21swapper/211:29:482
10131210016,67sleep310133-21cron11:59:333
544429913,69sleep30-21swapper/308:14:453
2578529948,24sleep00-21swapper/009:19:350
363229822,53sleep0502-21kworker/u8:108:09:330
890129624,27sleep10-21swapper/108:25:291
317929611,38sleep20-21swapper/209:49:542
2673629616,57sleep126426-21/usr/sbin/munin11:04:561
2673629616,57sleep126426-21/usr/sbin/munin11:04:561
2576329643,14sleep20-21swapper/207:35:112
2458629611,71sleep024710-21timerwakeupswit07:30:200
958929512,71sleep19649-21latency10:09:561
729329515,18sleep00-21swapper/011:45:050
2297829515,63sleep023049-21sensors_temp07:25:520
2297829515,63sleep023049-21sensors_temp07:25:520
1306329518,60sleep313210-21cat12:19:363
49529410,16sleep30-21swapper/307:59:353
270029219,50sleep32864-21tail11:30:053
2082329213,58sleep320901-21threads10:45:033
2082329213,58sleep320901-21threads10:45:033
1876329217,61sleep118794-21fschecks_count10:39:521
1876329217,61sleep118794-21fschecks_count10:39:521
716329112,58sleep26658-21/usr/sbin/munin11:45:032
3084629111,58sleep00-21swapper/009:34:460
2774529118,11sleep20-21swapper/209:24:422
1592529135,31sleep20-21swapper/207:04:592
3006029036,23sleep10-21swapper/111:15:011
1640829037,37sleep20-21swapper/210:30:122
2021828914,60sleep0372-21sshd07:18:530
1607828935,32sleep10-21swapper/107:05:021
8982878,51sleep10-21swapper/111:24:591
3248228717,50sleep031668-21/usr/sbin/munin07:55:160
954128617,46sleep20-21swapper/211:54:352
907128616,51sleep19137-21/usr/sbin/munin11:50:061
218428618,51sleep32313-21timerwakeupswit09:45:073
1145028518,50sleep111461-21modprobe12:09:321
1145028518,50sleep111461-21modprobe12:09:321
753428416,46sleep20-21swapper/210:04:362
267172849,62sleep026733-21kernelversion11:04:550
267172849,62sleep026733-21kernelversion11:04:550
1214428318,48sleep212233-21systemd-run12:14:342
318172828,57sleep131900-21tune2fs07:54:561
1844028221,41sleep118489-21latency_hist10:39:361
1370128212,57sleep113747-21latency_hist12:24:341
3187728110,58sleep131969-21mailstats11:20:021
3086828133,33sleep20-21swapper/209:34:482
2929428126,43sleep229279-21munin-run11:14:342
1499428125,29sleep10-21swapper/112:34:361
50612808,14sleep20-21swapper/209:54:592
50612808,14sleep20-21swapper/209:54:592
1080928031,32sleep20-21swapper/212:04:332
1080928031,32sleep20-21swapper/212:04:332
2405027917,43sleep324195-21unixbench_singl10:55:043
2405027917,43sleep324195-21unixbench_singl10:55:043
2377327934,30sleep20-21swapper/210:54:592
2377327934,30sleep20-21swapper/210:54:592
2180627914,45sleep121845-21df10:49:501
2180627914,45sleep121845-21df10:49:501
210552799,52sleep20-21swapper/209:04:352
1558227917,30sleep30-21swapper/307:04:523
1372227917,19sleep10-21swapper/108:39:551
364627815,27sleep00-21swapper/011:34:510
2184527717,19sleep10-21swapper/109:05:031
1454927731,11sleep20-21swapper/212:29:402
1428927732,11sleep20-21swapper/212:29:322
1015227716,28sleep00-21swapper/008:29:360
554627613,45sleep05596-21/usr/sbin/munin11:40:000
1331027532,11sleep20-21swapper/212:20:112
356327426,26sleep10-21swapper/111:34:481
1713827421,29sleep00-21swapper/007:09:530
597827323,27sleep30-21swapper/309:59:363
597827323,27sleep30-21swapper/309:59:363
3137127317,24sleep00-21swapper/011:19:490
240752739,47sleep10-21swapper/109:14:031
1862127320,26sleep10-21swapper/108:55:011
1708127316,41sleep317118-21ps08:50:023
1537027314,17sleep30-21swapper/310:29:363
111672736,12sleep20-21swapper/212:05:542
111672736,12sleep20-21swapper/212:05:542
554027215,24sleep10-21swapper/108:14:511
3028127212,45sleep130335-21fschecks_time07:50:041
220227211,16sleep00-21swapper/011:29:540
2020727221,28sleep10-21swapper/109:00:021
546927119,37sleep20-21swapper/208:14:482
2925627120,31sleep129293-21modprobe11:14:331
2734227118,31sleep00-21swapper/007:40:100
1844127116,35sleep30-21swapper/307:14:363
839827017,33sleep10-21swapper/108:24:361
70952700,6sleep00-21swapper/010:00:090
2489027014,38sleep124964-21mailstats09:15:041
1426827021,20sleep30-21swapper/310:24:563
312642698,16sleep10-21swapper/109:34:591
176726910,43sleep11768-21cron08:00:341
97572685,10sleep20-21swapper/211:54:482
52882686,12sleep30-21swapper/311:39:553
3078126814,30sleep00-21swapper/007:50:150
2386526821,24sleep10-21swapper/110:55:021
2386526821,24sleep10-21swapper/110:55:021
1552226820,24sleep30-21swapper/308:45:023
739626712,39sleep17535-21unixbench_singl11:45:061
2312926721,27sleep00-21swapper/009:09:580
1136626714,37sleep111441-21/usr/sbin/munin10:15:051
973026614,35sleep09773-21/usr/sbin/munin10:09:590
501826616,29sleep10-21swapper/109:54:581
501826616,29sleep10-21swapper/109:54:581
483526623,20sleep30-21swapper/308:10:173
265252668,10sleep00-21swapper/007:36:540
322526514,24sleep30-21swapper/309:49:553
2940626517,32sleep329475-21fschecks_time09:29:523
2798126512,14sleep10-21swapper/109:24:531
2068526520,32sleep020691-21ntpq10:45:000
2068526520,32sleep020691-21ntpq10:45:000
2038126515,25sleep10-21swapper/110:44:531
2038126515,25sleep10-21swapper/110:44:531
1985626521,28sleep00-21swapper/008:59:530
1223726514,33sleep312332-21cat12:14:353
1086026517,24sleep00-21swapper/012:04:350
1086026517,24sleep00-21swapper/012:04:350
523226421,30sleep15209-21latency_hist08:14:351
285082647,12sleep30-21swapper/311:10:113
154552647,12sleep00-21swapper/008:44:590
1169826416,23sleep30-21swapper/308:34:363
335126317,21sleep10-21swapper/111:34:361
2573226320,27sleep325764-21munin-run09:19:343
1233826316,30sleep012105-21apt-get12:14:360
1221326313,21sleep00-21swapper/008:34:570
814026221,21sleep30-21swapper/310:04:573
47226219,22sleep10-21swapper/109:40:041
2394726224,21sleep30-21swapper/307:30:033
1947126217,32sleep319509-21/usr/sbin/munin07:15:013
831126122,22sleep30-21swapper/308:24:343
67562616,11sleep10-21swapper/110:00:071
2972526113,22sleep10-21swapper/109:29:581
2241226110,34sleep122365-21apt-get07:25:121
2241226110,34sleep122365-21apt-get07:25:121
2117326114,21sleep10-21swapper/107:20:041
182202617,11sleep30-21swapper/310:37:443
1668526119,19sleep10-21swapper/108:49:521
1221426112,25sleep00-21swapper/010:19:360
1052826118,26sleep30-21swapper/312:00:543
1052826118,26sleep30-21swapper/312:00:543
266532605,44sleep30-21swapper/307:38:353
1998826014,21sleep00-21swapper/010:44:360
1998826014,21sleep00-21swapper/010:44:360
198802606,12sleep30-21swapper/310:43:503
198802606,12sleep30-21swapper/310:43:503
3071225911,27sleep30-21swapper/307:50:143
1686425914,31sleep116906-21latency_hist10:34:351
1164625916,21sleep10-21swapper/108:34:341
665725817,21sleep30-21swapper/311:44:433
2702325825,15sleep30-21swapper/307:39:443
1565125817,24sleep10-21swapper/112:39:361
112372587,16sleep30-21swapper/312:06:483
112372587,16sleep30-21swapper/312:06:483
3253525717,19sleep00-21swapper/009:39:550
2516725717,25sleep325232-21grep10:59:573
2516725717,25sleep325232-21grep10:59:573
2433725712,27sleep10-21swapper/107:30:131
2185925717,18sleep00-21swapper/010:49:530
2185925717,18sleep00-21swapper/010:49:530
109325722,19sleep30-21swapper/307:59:593
823325614,25sleep38229-21apt-get11:49:483
547225614,23sleep05539-21df08:14:500
2555625616,19sleep00-21swapper/011:00:060
2555625616,19sleep00-21swapper/011:00:060
247225614,25sleep12481-21diskstats08:05:061
1446625613,23sleep10-21swapper/110:25:021
1408425610,24sleep00-21swapper/010:24:530
3041325515,20sleep00-21swapper/011:15:250
2622825515,24sleep326216-21apt-get09:19:513
2622825515,24sleep326216-21apt-get09:19:513
1973625516,23sleep319731-21apt-get08:59:483
1731225514,21sleep00-21swapper/010:34:550
164072555,36sleep00-21swapper/007:05:290
1242225514,22sleep30-21swapper/310:19:433
968825417,23sleep30-21swapper/310:09:583
3147825416,17sleep30-21swapper/309:35:053
2139525413,22sleep321492-21users07:20:063
1712225418,19sleep00-21swapper/008:50:030
1549125415,22sleep115514-21ntp_states08:45:001
1359925415,25sleep013601-21sshd12:23:530
1348725412,17sleep00-21swapper/008:39:460
1245625413,17sleep10-21swapper/112:14:371
1193625412,24sleep311932-21apt-get08:34:483
221892539,30sleep022198-21vmstat09:05:060
148812535,38sleep00-21swapper/012:33:540
1257025315,18sleep00-21swapper/010:19:520
117802537,35sleep10-21swapper/112:10:331
1020725317,17sleep10-21swapper/111:59:351
3182625218,16sleep30-21swapper/311:20:003
1927325214,17sleep10-21swapper/107:14:551
1434225213,24sleep1371-21snmpd12:29:351
1434225213,24sleep1371-21snmpd12:29:351
120312529,19sleep10-21swapper/108:34:541
2785725118,17sleep30-21swapper/309:24:513
1251725114,23sleep112524-21df_inode10:19:491
821925020,15sleep10-21swapper/110:04:591
46162509,16sleep00-21swapper/009:54:450
46162509,16sleep00-21swapper/009:54:450
44725011,24sleep3453-21ntp_states09:40:023
2906025013,21sleep329062-21ntp_states07:45:093
2906025013,21sleep329062-21ntp_states07:45:093
2753825014,17sleep00-21swapper/009:24:360
2753825014,17sleep00-21swapper/009:24:360
136492509,18sleep30-21swapper/312:24:323
1351225014,21sleep313506-21apt-get08:39:493
2292824912,21sleep322937-21df_inode09:09:533
1153724914,16sleep30-21swapper/310:15:073
45924812,19sleep00-21swapper/011:24:450
233222479,15sleep00-21swapper/010:54:490
233222479,15sleep00-21swapper/010:54:490
1258424710,22sleep012105-21apt-get12:14:400
42612465,33sleep00-21swapper/009:53:230
1843224613,19sleep00-21swapper/008:54:560
1823124614,17sleep30-21swapper/308:54:513
2234924510,17sleep30-21swapper/310:50:043
2234924510,17sleep30-21swapper/310:50:043
105222439,20sleep110524-21sshd12:00:521
105222439,20sleep110524-21sshd12:00:521
1496824116,13sleep30-21swapper/312:34:343
101392417,14sleep00-21swapper/011:59:340
1563224012,14sleep30-21swapper/312:39:343
1564123910,13sleep00-21swapper/010:29:540
140032396,25sleep00-21swapper/012:25:510
140032396,25sleep00-21swapper/012:25:510
142902366,15sleep30-21swapper/312:29:333
142902366,15sleep30-21swapper/312:29:333
156502337,14sleep00-21swapper/012:39:350
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional