You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-09-20 - 09:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Fri Sep 20, 2024 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
322322283015,2803sleep321005-21kworker/3:121:57:263
322322283015,2803sleep321005-21kworker/3:121:57:263
17032222818,2259sleep212938-21kworker/2:100:34:252
875026678,609sleep2199-21jbd2/mmcblk2p2-00:09:002
875026678,609sleep2199-21jbd2/mmcblk2p2-00:09:002
14084242918,393sleep314129-21/usr/sbin/munin20:59:193
602924049,382sleep26077-21ntpq20:34:312
1859232812,279sleep21873-21grep22:04:152
1010229951,190sleep21072-21latency23:44:172
29778229412,266sleep329823-21unin-run23:33:563
16351227485,173sleep216414-21sshd22:49:262
16351227485,173sleep216414-21sshd22:49:262
4585226820,223sleep14605-21awk22:13:551
13689223656,103sleep20-21swapper/220:59:062
22121222239,168sleep222215-21cut23:09:162
30542221733,163sleep230038-21/usr/sbin/munin23:34:242
28010221751,41sleep20-21swapper/220:03:582
29668221612,98sleep229872-21latency_hist20:08:582
30448221017,178sleep230496-21ntp_offset20:09:272
12632220228,64sleep20-21swapper/219:14:272
18962219566,109sleep20-21swapper/221:14:282
23123219123,150sleep223258-21latency_hist19:48:592
27462218835,128sleep227526-21mailstats23:24:352
3586217517,140sleep33702-21latency_hist20:28:583
3586217517,140sleep33702-21latency_hist20:28:583
799821697,138sleep00-21swapper/022:24:080
76012165103,23sleep27649-21sed00:04:202
76012165103,23sleep27649-21sed00:04:202
25861215721,70sleep226009-21unixbench_multi23:19:302
17346215715,122sleep20-21swapper/222:54:102
29826215615,123sleep029894-21latency_hist23:33:570
17060215622,118sleep017079-21modprobe22:53:560
17060215622,118sleep017079-21modprobe22:53:560
28540215518,120sleep20-21swapper/221:44:252
29122215412,19sleep20-21swapper/223:29:272
21536215210,18sleep20-21swapper/219:43:592
30849215146,93sleep230825-21latency_hist21:53:582
25886215031,64sleep225927-21munin-run21:38:552
25886215031,64sleep225927-21munin-run21:38:552
768421447,13sleep00-21swapper/000:04:220
768421447,13sleep00-21swapper/000:04:220
4084213810,116sleep24098-21grep20:29:182
1201421348,113sleep212064-21fschecks_count00:19:142
32185213311,59sleep20-21swapper/223:39:242
27053212913,61sleep30-21swapper/323:24:233
23528212764,22sleep20-21swapper/219:49:162
10100212710,102sleep310146-21apt-key20:48:573
27248212612,97sleep027317-21latency23:24:290
23684212511,98sleep223793-21sort21:29:262
8595212323,62sleep20-21swapper/222:24:262
1732221217,78sleep217359-21ps19:29:252
10888211715,87sleep310900-21kernelversion20:49:463
2910521160,8sleep00-21swapper/020:04:270
10827211559,38sleep20-21swapper/200:14:262
1531221139,43sleep20-21swapper/219:24:122
14699211332,12sleep20-21swapper/219:19:482
387821119,41sleep20-21swapper/223:53:582
12651211133,62sleep212736-21mailstats20:54:302
26410211011,83sleep325909-21apt-get21:39:043
11575211010,47sleep211705-21latency_hist19:13:582
15073210915,77sleep015124-21ls19:23:580
1502521098,42sleep20-21swapper/200:29:082
26661210719,68sleep326158-21apt-get19:59:063
16337210722,33sleep30-21swapper/300:33:573
13918210513,22sleep013929-21ntp_states00:24:210
2004210319,63sleep02028-21awk23:48:550
19543210328,11sleep20-21swapper/219:37:202
13964210353,36sleep20-21swapper/222:43:562
17081210110,77sleep117108-21systemd-run22:53:571
17081210110,77sleep117108-21systemd-run22:53:571
12601210113,72sleep00-21swapper/020:54:280
6357210040,45sleep26507-21sed23:59:272
26866210013,73sleep00-21swapper/019:59:230
3275429915,67sleep0340-21grep21:59:160
673029719,62sleep210000-21cyclictest22:19:172
3210129714,68sleep032122-21awk23:39:220
2645629719,62sleep026154-21latency_hist19:58:590
2645629719,62sleep026154-21latency_hist19:58:590
1059229717,59sleep110197-21/usr/sbin/munin19:09:211
989529633,40sleep09925-21basename19:08:550
989529633,40sleep09925-21basename19:08:550
2416529611,24sleep20-21swapper/223:14:252
2697829516,62sleep126933-21/usr/sbin/munin23:24:211
381729430,20sleep20-21swapper/222:09:242
871629323,53sleep38735-21fschecks_count19:04:143
871629323,53sleep38735-21fschecks_count19:04:143
2174329231,42sleep10-21swapper/123:08:551
2019429213,66sleep220197-21cron23:03:552
2019429213,66sleep220197-21cron23:03:552
70129121,51sleep20-21swapper/220:19:082
1005929023,50sleep20-21swapper/220:48:562
954528918,21sleep10-21swapper/119:04:291
954528918,21sleep10-21swapper/119:04:291
48402895,11sleep20-21swapper/222:14:002
3224528924,48sleep20-21swapper/220:14:282
2034628918,51sleep020350-21fschecks_count19:39:160
1385728919,51sleep313950-21uniq19:19:163
1189128924,54sleep211901-21munin-run20:53:572
99662888,68sleep210053-21sed19:08:572
99662888,68sleep210053-21sed19:08:572
2647228813,56sleep20-21swapper/221:39:322
249228824,46sleep20-21swapper/220:24:172
249228824,46sleep20-21swapper/220:24:172
2049028824,15sleep20-21swapper/223:04:152
105532888,66sleep00-21swapper/020:49:020
1332428715,51sleep112676-21/usr/sbin/munin22:39:271
654728621,45sleep36603-21tune2fs22:19:143
2286528635,33sleep20-21swapper/221:28:582
1427628639,29sleep20-21swapper/222:44:132
792828526,30sleep30-21swapper/320:39:333
234828513,55sleep12425-21grep22:04:261
172982857,61sleep00-21swapper/019:29:230
1186428517,48sleep312018-21unixbench_singl22:34:293
6172848,22sleep2661-21ntpq21:59:232
2355728412,57sleep323578-21dpkg23:14:063
2018528417,28sleep00-21swapper/021:19:190
2018528417,28sleep00-21swapper/021:19:190
1426228415,53sleep014329-21switchtime20:59:240
1120428438,31sleep20-21swapper/222:34:152
453628310,17sleep20-21swapper/223:54:222
453628310,17sleep20-21swapper/223:54:222
359128318,51sleep33603-21kernelversion22:09:173
2189028318,42sleep021496-21/usr/sbin/munin21:24:200
2189028318,42sleep021496-21/usr/sbin/munin21:24:200
1704328320,13sleep20-21swapper/221:09:152
1345228310,14sleep20-21swapper/200:24:072
2643428224,33sleep126431-21apt-get21:39:061
1849128224,37sleep318525-21fschecks_count19:34:143
1713228212,55sleep117204-21sort00:34:281
159562828,20sleep216118-21grep21:04:272
1323828224,35sleep00-21swapper/000:23:570
1095528217,32sleep00-21swapper/019:09:280
2620428110,26sleep20-21swapper/219:58:572
2620428110,26sleep20-21swapper/219:58:572
993628013,26sleep00-21swapper/022:29:210
224928013,51sleep02268-21apt-get20:24:070
224928013,51sleep02268-21apt-get20:24:070
203128030,31sleep10-21swapper/123:48:561
120672809,54sleep311903-21/usr/sbin/munin00:19:153
953227919,12sleep20-21swapper/222:29:032
72462798,15sleep20-21swapper/220:39:152
319062797,12sleep20-21swapper/221:54:342
319062797,12sleep20-21swapper/221:54:342
384627816,47sleep13877-21cut23:53:571
2502727812,16sleep10-21swapper/121:34:221
2502727812,16sleep10-21swapper/121:34:221
2925227714,41sleep129281-21latency_hist21:48:561
939427522,35sleep19476-21grep20:44:271
312222758,55sleep0374-21snmpd20:12:560
255427517,37sleep32676-21/usr/sbin/munin22:04:283
217892758,16sleep221815-21cut21:24:182
1694527516,33sleep10-21swapper/119:29:161
2981527411,45sleep129909-21memory21:49:201
192112749,13sleep00-21swapper/019:34:280
1868827413,44sleep118821-21sed22:58:571
1278327418,29sleep00-21swapper/022:39:140
2186027316,27sleep00-21swapper/019:44:170
1974327324,29sleep10-21swapper/119:38:571
1182227321,30sleep00-21swapper/019:14:000
2613427217,35sleep126163-21munin-run19:58:561
2613427217,35sleep126163-21munin-run19:58:561
432527117,23sleep10-21swapper/120:29:251
2128427113,37sleep20-21swapper/221:23:572
2128427113,37sleep20-21swapper/221:23:572
221422709,12sleep10-21swapper/119:44:221
1587627017,30sleep10-21swapper/121:04:261
1043927017,24sleep10-21swapper/100:14:161
534626920,30sleep35417-21modprobe23:58:563
534626920,30sleep35417-21modprobe23:58:563
44626928,23sleep10-21swapper/123:43:551
3033126918,25sleep00-21swapper/023:34:180
1486026912,15sleep00-21swapper/022:44:270
1344426921,30sleep313474-21munin-run19:18:563
2076226819,27sleep30-21swapper/323:04:253
2022326813,36sleep120215-21apt-get19:39:091
857926714,36sleep38725-21sed00:08:573
857926714,36sleep38725-21sed00:08:573
3021526714,27sleep10-21swapper/120:09:221
1771326716,34sleep317321-21/usr/sbin/munin22:54:193
1079626719,33sleep010802-21ntp_states00:14:240
624426615,36sleep36397-21latency_hist22:18:583
2878726615,25sleep10-21swapper/120:04:251
264226620,24sleep10-21swapper/120:24:201
264226620,24sleep10-21swapper/120:24:201
221326621,22sleep30-21swapper/323:48:583
1160926617,35sleep311628-21apt00:18:553
484926511,25sleep10-21swapper/122:14:071
2379426520,20sleep30-21swapper/321:29:273
1478526513,22sleep30-21swapper/322:44:263
692426420,29sleep06958-21ps22:19:240
3219426415,26sleep10-21swapper/120:14:271
310792646,10sleep30-21swapper/320:11:103
160872644,18sleep10-21swapper/100:31:251
398726320,23sleep00-21swapper/022:09:260
2834526320,24sleep00-21swapper/021:44:180
2467526318,27sleep10-21swapper/119:53:591
2395126322,23sleep00-21swapper/023:14:190
230162630,6sleep30-21swapper/321:28:583
2185126320,25sleep30-21swapper/319:44:163
1692126316,23sleep00-21swapper/021:09:130
1356026311,38sleep113573-21fschecks_count00:24:141
508226220,27sleep35090-21modprobe20:33:553
1503526217,20sleep10-21swapper/121:03:581
528426115,29sleep00-21swapper/023:58:080
528426115,29sleep00-21swapper/023:58:080
312926118,21sleep00-21swapper/023:49:360
242062616,43sleep00-21swapper/021:32:150
176932615,10sleep00-21swapper/000:37:090
1032026117,29sleep110514-21cat20:48:591
650826018,18sleep30-21swapper/323:59:283
237752609,18sleep00-21swapper/019:49:220
21662606,12sleep00-21swapper/022:04:210
2160726017,27sleep321683-21fschecks_time21:24:153
1916026015,23sleep00-21swapper/021:14:300
1539326022,22sleep00-21swapper/000:29:180
1496326015,24sleep10-21swapper/122:44:281
1092626014,21sleep10-21swapper/122:33:581
1083426016,22sleep30-21swapper/322:33:563
1014926018,26sleep010319-21latency_hist20:48:580
609525919,25sleep10-21swapper/120:34:341
2565325918,23sleep30-21swapper/323:19:273
2525825911,34sleep025262-21diskstats23:19:160
229872595,10sleep10-21swapper/123:10:001
179192596,42sleep30-21swapper/321:11:433
1400725917,24sleep30-21swapper/322:43:573
878525812,20sleep10-21swapper/100:09:121
878525812,20sleep10-21swapper/100:09:121
589625822,21sleep30-21swapper/320:34:283
502525814,24sleep00-21swapper/022:14:190
3003725816,25sleep130056-21apt-get23:34:061
2822325817,21sleep30-21swapper/320:04:073
158425813,27sleep01631-21users20:19:310
801125714,27sleep38006-21apt-get22:24:113
699525716,20sleep00-21swapper/000:03:580
428525713,27sleep04322-21ps20:29:240
3003625712,24sleep30-21swapper/321:49:263
2563125715,24sleep125637-21ntp_states23:19:251
2530225716,18sleep30-21swapper/321:34:283
2530225716,18sleep30-21swapper/321:34:283
1866125715,24sleep318753-21memory21:14:213
1653425716,25sleep316589-21systemd-run21:08:563
1531325721,19sleep30-21swapper/319:24:133
923725614,24sleep09243-21ntp_states00:09:230
923725614,24sleep09243-21ntp_states00:09:230
855325614,25sleep18700-21latency_hist20:43:591
723225615,26sleep17206-21/usr/sbin/munin00:04:111
723225615,26sleep17206-21/usr/sbin/munin00:04:111
3158425614,24sleep031580-21apt-get20:14:090
3109625615,24sleep031089-21apt-get21:54:160
3109625615,24sleep031089-21apt-get21:54:160
3013525612,20sleep30-21swapper/323:34:133
184472565,40sleep00-21swapper/022:56:550
1577425619,18sleep30-21swapper/322:49:063
1577425619,18sleep30-21swapper/322:49:063
1553025614,24sleep015579-21irqstats19:24:180
1244725619,20sleep10-21swapper/119:14:221
956225514,25sleep19557-21apt-get22:29:101
594625514,23sleep16039-21memory23:59:181
2157425519,19sleep30-21swapper/319:44:003
1901125517,18sleep30-21swapper/322:59:163
1901125517,18sleep30-21swapper/322:59:163
1683725512,25sleep116904-21df21:09:101
1588925516,17sleep10-21swapper/119:24:271
851025413,22sleep10-21swapper/122:24:251
543525416,21sleep15554-21cat23:58:581
543525416,21sleep15554-21cat23:58:581
3155425414,26sleep131562-21ntp_states21:54:291
3155425414,26sleep131562-21ntp_states21:54:291
2851425414,24sleep128509-21apt-get23:29:101
2536525414,16sleep30-21swapper/319:54:303
2536525414,16sleep30-21swapper/319:54:303
2041925415,23sleep120432-21ntpq21:19:241
2041925415,23sleep120432-21ntpq21:19:241
2024625414,22sleep10-21swapper/123:03:581
2024625414,22sleep10-21swapper/123:03:581
1796025416,17sleep30-21swapper/300:38:583
1383725410,17sleep10-21swapper/119:19:141
1368325415,25sleep30-21swapper/300:24:163
1030625418,18sleep30-21swapper/319:09:153
686525311,17sleep10-21swapper/122:19:201
2954225311,27sleep029537-21apt-get21:49:110
2307125312,24sleep123065-21apt-get21:29:091
1192725314,25sleep111922-21apt-get00:19:091
96972525,11sleep30-21swapper/300:09:303
96972525,11sleep30-21swapper/300:09:303
724325213,22sleep07240-21apt-get20:39:130
3164025212,24sleep331635-21apt-get23:39:093
2930325218,19sleep30-21swapper/321:48:583
2692325212,22sleep026934-21ntp_states21:39:500
2669425212,23sleep126688-21apt-get19:59:121
2462725210,15sleep00-21swapper/021:34:130
2462725210,15sleep00-21swapper/021:34:130
1725625218,19sleep30-21swapper/319:29:223
1580525214,20sleep00-21swapper/021:04:250
2285525112,24sleep322938-21users23:09:293
1222225112,22sleep312229-21diskstats20:54:183
244862508,31sleep30-21swapper/319:52:403
2203925010,16sleep10-21swapper/121:24:251
2203925010,16sleep10-21swapper/121:24:251
2203425013,21sleep022029-21apt-get23:09:100
1202125013,21sleep012028-21users22:34:300
3158724913,17sleep30-21swapper/320:14:123
253324912,22sleep32525-21apt-get23:49:103
2049324912,21sleep020560-21df23:04:180
136324912,25sleep1374-21snmpd22:02:201
630924810,21sleep06459-21sed20:34:370
197322477,14sleep00-21swapper/022:59:290
197322477,14sleep00-21swapper/022:59:290
1878624711,16sleep10-21swapper/121:14:231
143824714,20sleep3702-21/usr/sbin/munin20:19:303
199582467,15sleep30-21swapper/321:19:103
199582467,15sleep30-21swapper/321:19:103
115724612,19sleep11180-21ntp_states20:19:231
79524311,15sleep30-21swapper/321:59:283
2398524310,14sleep10-21swapper/123:14:201
1260923910,15sleep10-21swapper/120:54:291
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional