You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-24 - 15:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Wed Apr 24, 2024 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
46625799,555sleep0470-21ntpq22:56:500
1755425638,543sleep117596-21ntpq20:26:451
13818242011,393sleep113918-21latency_hist20:16:231
23815241422,379sleep123849-21ps00:11:501
26260229910,249sleep126299-21cat19:11:241
26260229910,249sleep126299-21cat19:11:241
9466225664,177sleep19529-21fschecks_time20:01:421
6932225556,177sleep17061-21grep23:16:541
31283224147,16sleep10-21swapper/119:26:541
27308223743,81sleep10-21swapper/122:41:241
290822328,210sleep13002-21sed19:41:241
30933223016,200sleep330961-21latency_hist00:36:233
17942221962,42sleep118048-21latency_hist22:11:241
1976622118,148sleep119830-21cpuspeed22:16:371
1679522086,192sleep11166-21snmpd20:25:281
30738220710,174sleep10-21swapper/121:06:551
27086220088,70sleep127152-21wc00:21:551
13052220043,110sleep10-21swapper/123:36:571
14288219466,74sleep10-21swapper/123:41:451
5384219132,48sleep15538-21unixbench_multi19:46:571
15811218842,126sleep115900-21unixbench_multi22:01:591
6924218160,105sleep10-21swapper/119:51:571
5821217929,111sleep15856-21grep21:31:421
4909217858,97sleep15045-21unixbench_singl21:27:021
4938217635,126sleep10-21swapper/123:11:451
7907217146,109sleep17956-21cat21:36:551
28258217067,47sleep128367-21cut19:16:241
27636217019,23sleep10-21swapper/122:41:411
31797216980,30sleep131832-21fschecks_count21:11:431
26154216337,86sleep125380-21/usr/sbin/munin20:52:171
20919216328,78sleep120947-21ntpq20:36:541
20919216328,78sleep120947-21ntpq20:36:541
9320215610,107sleep19373-21processes21:41:521
1157921567,116sleep11166-21snmpd21:50:201
3145215115,25sleep03141-21apt-get19:41:350
26267215117,117sleep226312-21latency22:36:452
9456215010,123sleep39530-21find21:41:543
23376214948,84sleep123462-21latency_hist20:46:231
2147214313,16sleep10-21swapper/123:01:561
12415214147,83sleep112444-21ps21:51:501
1349213915,96sleep10-21swapper/119:36:231
3021421389,75sleep10-21swapper/100:31:571
5952213724,38sleep20-21swapper/219:51:222
28261213552,71sleep128304-21/usr/sbin/munin21:01:221
28261213552,71sleep128304-21/usr/sbin/munin21:01:221
4449213413,107sleep34465-21uname21:26:523
20071213312,17sleep10-21swapper/100:01:241
7066213225,82sleep27191-21latency_hist21:36:242
3688213254,54sleep10-21swapper/119:41:561
26246213244,20sleep10-21swapper/100:21:231
14207212620,51sleep10-21swapper/121:56:541
1342821245,105sleep00-21swapper/023:37:320
2558212348,20sleep10-21swapper/121:21:411
19960212212,65sleep119976-21modprobe20:36:221
19960212212,65sleep119976-21modprobe20:36:221
27810212116,78sleep20-21swapper/200:26:242
130521218,74sleep10-21swapper/121:16:521
2231221205,12sleep10-21swapper/122:23:031
20523212012,86sleep320573-21/usr/sbin/munin20:36:403
20523212012,86sleep320573-21/usr/sbin/munin20:36:403
1918521196,45sleep10-21swapper/123:56:511
2391211813,79sleep00-21swapper/019:36:590
31370211615,83sleep231454-21grep22:51:512
31370211615,83sleep231454-21grep22:51:512
23402211514,32sleep10-21swapper/119:01:471
20413211414,82sleep020475-21/usr/sbin/munin00:01:470
11101211411,90sleep211114-21idleruntime23:31:452
7014211219,69sleep27141-21head19:51:582
24385211133,46sleep024420-21munin-run19:06:220
10652211119,16sleep10-21swapper/123:31:241
25055211014,75sleep025144-21fschecks_time19:06:460
25055211014,75sleep025144-21fschecks_time19:06:460
24167211029,24sleep124206-21latency_hist22:31:231
1892721108,66sleep119053-21sed22:11:561
1892721108,66sleep119053-21sed22:11:561
865021088,58sleep10-21swapper/119:56:581
32566210812,78sleep232573-21/usr/sbin/munin19:31:482
18453210814,79sleep318511-21latency22:11:463
18453210814,79sleep318511-21latency22:11:463
3020621075,44sleep10-21swapper/119:23:551
3014421075,11sleep00-21swapper/019:23:100
11564210745,15sleep10-21swapper/120:06:551
11564210745,15sleep10-21swapper/120:06:551
13727210613,77sleep213772-21/usr/sbin/munin23:41:222
1915721056,15sleep30-21swapper/323:56:493
10497210516,71sleep310591-21fschecks_time21:46:433
2959321049,15sleep00-21swapper/000:31:350
31252210322,18sleep10-21swapper/122:51:461
31252210322,18sleep10-21swapper/122:51:461
24686210211,21sleep10-21swapper/122:31:471
22014210221,18sleep10-21swapper/100:06:411
433921019,80sleep24340-21sshd19:45:322
1298210112,72sleep00-21swapper/021:16:510
1699210022,55sleep31790-21fschecks_time19:36:453
857629810,73sleep08611-21munin-run21:41:230
360529810,26sleep13670-21awk23:06:531
2957129819,59sleep20-21swapper/222:47:032
2957129819,59sleep20-21swapper/222:47:032
2475229815,21sleep30-21swapper/319:06:253
2475229815,21sleep30-21swapper/319:06:253
1370729817,68sleep013753-21basename21:56:410
2804429717,18sleep10-21swapper/100:26:351
187032966,13sleep00-21swapper/020:31:360
187032966,13sleep00-21swapper/020:31:360
2670429523,51sleep326468-21/usr/sbin/munin00:21:413
266312959,12sleep00-21swapper/020:55:470
2005229524,54sleep320168-21apt-key20:36:243
2005229524,54sleep320168-21apt-key20:36:243
414629318,55sleep03664-21apt-get21:26:330
2699829222,43sleep227037-21fschecks_count19:11:412
184592929,24sleep118435-40mandb23:56:231
515029115,53sleep05210-21grep23:11:520
2003529116,57sleep020083-21latency22:16:440
1452729140,34sleep10-21swapper/120:16:521
11502915,10sleep10-21swapper/122:59:421
2945628932,32sleep00-21swapper/022:47:010
2945628932,32sleep00-21swapper/022:47:010
217928715,56sleep22259-21cat19:36:572
1625228628,31sleep20-21swapper/223:46:552
1625228628,31sleep20-21swapper/223:46:552
843128510,26sleep10-21swapper/123:21:551
3152828414,56sleep031619-21sed22:51:530
3152828414,56sleep031619-21sed22:51:530
2997628418,42sleep10-21swapper/122:47:141
2997628418,42sleep10-21swapper/122:47:141
2880328416,36sleep228859-21apt-config22:46:222
2895028319,51sleep029060-21sed22:46:240
776928218,50sleep27782-21/usr/sbin/munin21:36:522
629828217,17sleep00-21swapper/023:16:370
534928221,42sleep35486-21grep23:11:553
2541428212,55sleep225399-21apt-get20:51:372
837028113,53sleep08430-21switchtime23:21:540
2262028117,29sleep20-21swapper/222:26:232
350528014,48sleep20-21swapper/221:24:332
2145128010,49sleep20-21swapper/200:05:332
698627922,41sleep00-21swapper/021:36:220
326727915,48sleep03331-21/usr/sbin/munin23:06:430
2855227918,49sleep228619-21mailstats00:26:532
2697127913,47sleep30-21swapper/320:56:353
2697127913,47sleep30-21swapper/320:56:353
160232798,16sleep00-21swapper/020:21:470
2968827817,39sleep229766-21switchtime19:21:502
1867027820,45sleep018674-21find23:56:300
109972787,59sleep111096-21fschecks_time23:31:441
3029227723,29sleep20-21swapper/200:31:582
2177827714,47sleep021804-21munin-run20:41:210
2177827714,47sleep021804-21munin-run20:41:210
1978827711,54sleep21166-21snmpd23:59:022
1740927712,41sleep216604-21/usr/sbin/munin22:06:532
619927529,27sleep30-21swapper/321:31:533
383427515,42sleep33993-21unixbench_multi19:41:583
210027532,28sleep20-21swapper/221:21:222
1753827510,15sleep20-21swapper/223:51:522
856027416,42sleep38572-21apt21:41:223
527627413,14sleep20-21swapper/223:11:542
2881127425,34sleep10-21swapper/121:01:461
2686727416,23sleep00-21swapper/000:21:460
2587127423,31sleep224922-21/usr/sbin/munin00:17:042
1452627414,45sleep014596-21sendmail_mailtr23:41:520
894027322,32sleep20-21swapper/221:41:412
878527324,23sleep30-21swapper/319:57:323
59327320,18sleep30-21swapper/322:56:533
2571327322,31sleep30-21swapper/320:52:073
2395027321,35sleep10-21swapper/120:46:481
2182827320,28sleep20-21swapper/220:41:232
2182827320,28sleep20-21swapper/220:41:232
1714927323,24sleep20-21swapper/220:26:342
1139027319,33sleep211425-21grep20:06:522
1139027319,33sleep211425-21grep20:06:522
1089827310,47sleep310920-21apt-get20:06:323
1089827310,47sleep310920-21apt-get20:06:323
2668327219,26sleep00-21swapper/022:36:550
835727113,31sleep20-21swapper/219:56:552
306122719,48sleep00-21swapper/021:06:500
2886227123,29sleep30-21swapper/322:46:233
287102717,11sleep10-21swapper/119:19:351
2492027125,26sleep20-21swapper/219:06:322
2492027125,26sleep20-21swapper/219:06:322
246522710,6sleep00-21swapper/022:31:450
979727025,27sleep20-21swapper/223:26:482
2782827020,30sleep326885-21/usr/sbin/munin19:11:583
2373927029,23sleep20-21swapper/219:01:562
2358927014,39sleep323600-21/usr/sbin/munin20:46:333
1724627016,37sleep317342-21grep20:26:393
166522705,10sleep10-21swapper/123:49:081
166522705,10sleep10-21swapper/123:49:081
3105226920,29sleep20-21swapper/219:26:472
3043126922,26sleep30-21swapper/321:06:453
2384326925,23sleep30-21swapper/319:01:573
2238126917,25sleep30-21swapper/300:06:513
1779726917,24sleep00-21swapper/023:51:580
621826821,23sleep20-21swapper/223:16:312
3275426819,23sleep30-21swapper/319:31:533
2991526821,28sleep20-21swapper/221:06:232
1726626817,23sleep30-21swapper/323:51:453
1688526824,25sleep30-21swapper/323:51:243
1688526824,25sleep30-21swapper/323:51:243
1315426821,26sleep30-21swapper/320:11:573
42622675,12sleep20-21swapper/221:26:432
2470826718,26sleep20-21swapper/200:16:232
1352326713,26sleep20-21swapper/221:56:352
265726614,36sleep30-21swapper/321:21:433
2243726620,27sleep20-21swapper/220:41:482
1761426618,26sleep00-21swapper/020:26:480
903526514,31sleep08823-21/usr/sbin/munin21:41:430
3077026517,26sleep00-21swapper/019:26:410
2804726515,17sleep30-21swapper/300:26:373
2463226516,28sleep30-21swapper/322:31:453
48826418,22sleep00-21swapper/019:31:580
2608426422,22sleep30-21swapper/322:36:423
2463026420,29sleep024640-21modprobe00:16:210
2155926415,27sleep30-21swapper/300:06:223
1570326418,26sleep00-21swapper/022:01:580
1545626415,23sleep30-21swapper/322:01:493
2936026318,28sleep329318-21apt-get22:46:563
2936026318,28sleep329318-21apt-get22:46:563
2332326310,30sleep323341-21munin-run20:46:213
1641826311,37sleep216557-21latency_hist22:06:242
374726212,22sleep30-21swapper/323:06:553
247782625,39sleep30-21swapper/300:16:243
1328426214,25sleep30-21swapper/321:56:243
93452616,13sleep30-21swapper/323:26:373
624326115,27sleep06239-21apt-get19:51:360
2707226123,18sleep20-21swapper/220:56:422
2707226123,18sleep20-21swapper/220:56:422
2492126119,26sleep00-21swapper/000:16:330
1868626118,24sleep20-21swapper/222:11:532
1868626118,24sleep20-21swapper/222:11:532
1688826117,20sleep30-21swapper/322:06:413
1629026115,28sleep20-21swapper/220:21:542
1469526115,21sleep20-21swapper/223:41:542
1354526111,36sleep013547-21sshd20:13:520
106102615,13sleep20-21swapper/220:06:032
105932617,41sleep00-21swapper/023:31:140
1048726116,28sleep210496-21/usr/sbin/munin21:46:422
788326016,20sleep30-21swapper/323:21:403
622226015,28sleep36241-21sshd23:16:323
51226014,22sleep20-21swapper/222:56:522
2961126018,20sleep00-21swapper/021:03:350
2357426013,24sleep20-21swapper/222:26:562
469525923,22sleep30-21swapper/319:46:383
2889725916,25sleep00-21swapper/019:21:240
278772596,13sleep00-21swapper/019:12:360
270225916,25sleep02755-21memory21:21:440
2192625910,32sleep022013-21fschecks_time00:06:400
164525920,25sleep30-21swapper/323:01:413
1532725911,24sleep30-21swapper/323:46:233
3242125819,20sleep30-21swapper/321:11:583
292382586,43sleep00-21swapper/000:30:300
2884725822,19sleep30-21swapper/319:21:233
2204225816,21sleep00-21swapper/020:41:340
1690325814,21sleep00-21swapper/022:06:420
1097925820,20sleep00-21swapper/023:31:430
1089225815,26sleep310888-21apt-get23:31:383
3207125711,18sleep30-21swapper/319:31:243
1998625717,26sleep220049-21systemd-run20:36:232
1998625717,26sleep220049-21systemd-run20:36:232
1601325717,24sleep016024-21ntp_states23:46:490
1601325717,24sleep016024-21ntp_states23:46:490
852025619,18sleep00-21swapper/019:56:570
316832566,12sleep00-21swapper/021:11:340
285652566,37sleep20-21swapper/219:17:472
2854925615,25sleep328545-21apt-get21:01:363
2389025617,17sleep00-21swapper/020:46:470
2177525620,21sleep20-21swapper/200:06:302
212852565,33sleep00-21swapper/022:21:330
2107225614,27sleep321210-21latency_hist22:21:233
1466225612,26sleep314734-21/usr/sbin/munin20:16:543
600425516,19sleep00-21swapper/019:51:240
320425515,24sleep23205-21diskstats23:06:412
3043425518,18sleep30-21swapper/319:26:243
2726925519,17sleep00-21swapper/020:56:480
2726925519,17sleep00-21swapper/020:56:480
254662558,36sleep20-21swapper/222:33:382
194025515,24sleep21545-21/usr/sbin/munin23:01:492
1473625514,19sleep00-21swapper/020:16:550
1073425514,16sleep00-21swapper/020:06:250
1073425514,16sleep00-21swapper/020:06:250
984525421,18sleep30-21swapper/320:01:523
914125414,15sleep20-21swapper/223:26:242
2286125412,25sleep322855-21apt-get22:26:363
1484325412,19sleep30-21swapper/322:01:243
3217025312,24sleep232171-21ntp_states21:11:532
3146825315,18sleep20-21swapper/221:11:242
3132625314,23sleep331337-21ntp_states22:51:483
3132625314,23sleep331337-21ntp_states22:51:483
84125212,23sleep3836-21apt-get21:16:383
184225211,29sleep01896-21sed23:01:470
1281725212,23sleep212850-21users21:51:552
1194225211,22sleep011938-21apt-get21:51:320
1854425111,16sleep00-21swapper/022:11:470
1854425111,16sleep00-21swapper/022:11:470
1403125113,20sleep20-21swapper/220:16:332
1377325112,24sleep313818-21latency_hist23:41:233
2344325014,20sleep323533-21perl00:11:393
2131125012,22sleep321307-21apt-get22:21:363
1091125014,20sleep00-21swapper/021:46:560
232942499,15sleep00-21swapper/022:26:500
756924712,18sleep30-21swapper/319:56:233
2429724712,15sleep20-21swapper/220:46:572
1997224513,16sleep20-21swapper/222:16:422
1284124412,16sleep20-21swapper/220:11:482
964624211,15sleep00-21swapper/020:01:450
1568124112,14sleep30-21swapper/320:21:383
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional