You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-07-27 - 06:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sat Jul 27, 2024 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5853210278,1004sleep35897-21ntpq22:16:223
20427296678,805sleep320457-21fschecks_count21:21:213
10866252516,492sleep210870-21df00:16:082
3876247826,435sleep03894-21/usr/sbin/munin22:11:020
14878247811,404sleep315013-21head21:01:463
14878247811,404sleep315013-21head21:01:463
839724090,15sleep112-21rcu_sched00:06:251
22892232241,260sleep00-21swapper/023:11:160
24604231414,287sleep024614-21irqstats23:16:130
20411230923,198sleep320441-21modprobe19:40:533
22564230382,203sleep322689-21latency_hist23:10:553
31770227315,113sleep331809-21fschecks_count21:56:153
8343225077,98sleep38289-21latency_hist22:25:553
2927224362,161sleep33001-21threads22:06:243
29089222736,104sleep329106-21fschecks_count23:31:103
19383220312,178sleep319405-21munin-run23:00:533
8953219985,81sleep38979-21ntp_kernel_pll_22:26:243
1771219829,109sleep313199-1kworker/3:2H23:46:253
261821870,65sleep30-21swapper/320:25:553
25344218177,86sleep30-21swapper/321:36:113
9857217430,105sleep39971-21timerwakeupswit20:46:523
13890216811,143sleep313938-21awk00:26:073
13890216811,143sleep313938-21awk00:26:073
20032216611,114sleep320045-21cron21:20:523
20032216611,114sleep320045-21cron21:20:523
8583215963,73sleep38733-21grep19:01:273
1208221548,131sleep312227-21sed19:11:253
30544214550,82sleep330546-21apt-get23:36:033
3023121439,119sleep330300-21fschecks_time21:51:183
28689213920,90sleep228712-21awk23:30:532
3451213529,40sleep33552-21grep23:51:253
12320213514,106sleep112339-21apt-get00:21:041
8714213426,24sleep38673-21apt-get20:45:553
21308213410,108sleep221404-21timerandwakeup19:41:252
8828213011,104sleep09047-21cat20:45:560
1475921307,46sleep30-21swapper/322:46:073
12418212832,18sleep30-21swapper/300:21:103
1297521278,63sleep313041-21date22:40:543
6704212622,82sleep16715-21modprobe22:20:531
23650212623,86sleep023141-21apt-get21:31:030
13577212435,71sleep313644-21systemd-run21:00:533
27833212332,78sleep327855-21ntpq23:26:173
19897212348,14sleep30-21swapper/323:01:133
3972212117,32sleep30-21swapper/320:30:533
14522212110,99sleep314500-21latency_hist22:45:553
24955212010,94sleep125076-21timerandwakeup23:16:221
2226521209,95sleep322311-21grep21:26:183
3247421198,88sleep30-21swapper/323:41:163
23834211718,11sleep30-21swapper/323:13:013
12099211526,18sleep30-21swapper/300:20:543
775321149,90sleep37814-21fschecks_time00:06:123
16247211421,27sleep30-21swapper/319:26:153
15797211312,18sleep30-21swapper/319:25:553
3159821129,44sleep331682-21apt-config20:15:553
16129211228,46sleep30-21swapper/321:06:233
13565211222,63sleep213576-21modprobe21:00:522
28924211134,66sleep328438-21/usr/sbin/munin20:06:223
27381211114,80sleep127457-21/usr/sbin/munin21:41:261
27381211114,80sleep127457-21/usr/sbin/munin21:41:261
27185211012,18sleep00-21swapper/023:25:540
13723211023,67sleep113831-21sed19:16:281
13723211023,67sleep113831-21sed19:16:281
3668210817,69sleep23794-21latency_hist22:10:562
4293210718,32sleep30-21swapper/320:31:093
4293210718,32sleep30-21swapper/320:31:093
17609210710,30sleep30-21swapper/319:31:083
9367210613,76sleep19458-21fschecks_time00:11:101
932210630,64sleep3991-21date20:20:563
956210565,20sleep30-21swapper/322:01:113
28224210517,70sleep1288-21dbus-daemon20:05:551
1654121049,81sleep316560-21apt-get22:51:043
2507021035,12sleep30-21swapper/319:55:433
21222210219,66sleep121286-21df23:06:081
921210115,66sleep2931-21unin-run20:20:552
26745210122,18sleep30-21swapper/321:41:043
26745210122,18sleep30-21swapper/321:41:043
21865210123,56sleep0373-21sshd23:06:230
9724210012,73sleep29762-21munin-node19:06:032
9724210012,73sleep29762-21munin-node19:06:032
9249210020,16sleep30-21swapper/300:11:043
24937210019,68sleep124963-21munin-run21:35:531
1602029914,65sleep216031-21/usr/sbin/munin21:06:172
1120329822,14sleep30-21swapper/319:10:563
624729727,34sleep00-21swapper/018:56:040
2662829617,63sleep026657-21munin-run20:00:530
2390429610,15sleep00-21swapper/019:51:120
976329516,20sleep30-21swapper/319:06:043
3055629512,21sleep130623-21df23:36:091
2157929523,53sleep221580-21cron21:25:522
14162959,61sleep00-21swapper/020:21:170
1355229510,72sleep313588-21switchtime19:16:263
1355229510,72sleep313588-21switchtime19:16:263
2854829423,41sleep20-21swapper/220:06:132
3270429310,68sleep1338-21sed23:41:221
2962829312,63sleep029747-21/usr/sbin/munin23:31:230
223422939,71sleep322413-21fschecks_time19:46:143
1756029320,55sleep30-21swapper/321:11:153
1277729313,65sleep112815-21processes20:56:231
2657529213,63sleep126708-21unixbench_multi23:21:271
2542729225,11sleep30-21swapper/323:18:383
1810129216,63sleep118113-21cat22:56:021
1461829213,13sleep20-21swapper/200:26:262
1461829213,13sleep20-21swapper/200:26:262
397829120,37sleep30-21swapper/322:11:093
3039729115,56sleep230098-21/usr/sbin/munin21:51:212
1106729114,62sleep011071-21/usr/sbin/munin00:16:140
2199829010,48sleep30-21swapper/319:45:543
1601529015,24sleep10-21swapper/122:50:531
30172890,8sleep00-21swapper/020:26:170
2632328925,10sleep30-21swapper/319:56:573
1160928920,13sleep30-21swapper/320:51:283
749628814,60sleep07569-21/usr/sbin/munin22:21:200
296312879,21sleep329662-21munin-run21:50:543
74172866,20sleep37523-21latency_hist00:05:553
271442865,15sleep30-21swapper/323:25:533
1450628616,54sleep114616-21timerandwakeup00:26:251
1450628616,54sleep114616-21timerandwakeup00:26:251
1550828532,33sleep00-21swapper/022:46:280
911628412,53sleep09118-21cron19:05:520
46972847,14sleep30-21swapper/323:56:223
319452847,39sleep20-21swapper/221:56:182
2803128420,11sleep30-21swapper/320:04:033
1409928413,52sleep113584-21apt-get21:01:031
1409928413,52sleep113584-21apt-get21:01:031
2760828310,12sleep00-21swapper/020:01:260
2466528322,28sleep30-21swapper/321:32:243
916628110,56sleep29232-21latency_hist19:05:542
626528111,52sleep10-21swapper/118:56:051
58282810,6sleep10-21swapper/120:36:031
2745828122,39sleep227589-21sed21:41:272
2745828122,39sleep227589-21sed21:41:272
178932810,7sleep20-21swapper/222:55:542
93712809,19sleep00-21swapper/022:26:330
317528013,51sleep13181-21ntp_states20:26:231
2980928023,35sleep129587-21apt-get21:50:561
258728011,52sleep22517-40mandb23:50:562
1078828029,27sleep110913-21sed22:31:271
1078828029,27sleep110913-21sed22:31:271
3255727912,47sleep232578-21/usr/sbin/munin23:41:192
3030427921,40sleep230274-21apt-get20:11:222
2444027918,20sleep20-21swapper/223:16:112
16052799,13sleep10-21swapper/120:21:221
2798027815,39sleep027406-21/usr/sbin/munin23:26:220
1964627715,31sleep00-21swapper/023:01:020
163032778,50sleep00-21swapper/019:26:160
1194227611,50sleep112020-21latency22:36:151
787227524,25sleep20-21swapper/220:41:222
591827517,39sleep25949-21/usr/sbin/munin22:16:252
1160927522,36sleep011654-21if_eth019:11:140
112862756,55sleep30-21swapper/322:34:403
112862756,55sleep30-21swapper/322:34:403
1005427511,48sleep210152-21awk00:11:242
3168727418,30sleep00-21swapper/020:15:560
221492746,14sleep10-21swapper/121:26:161
2004127414,45sleep220069-21nfs4_client23:01:162
1108027415,46sleep00-21swapper/019:10:490
1108027415,46sleep00-21swapper/019:10:490
635027323,21sleep30-21swapper/318:56:113
2317527323,31sleep222779-21/usr/sbin/munin23:11:222
1534527311,29sleep20-21swapper/222:46:262
3226227223,27sleep00-21swapper/021:56:280
2428227210,15sleep20-21swapper/219:51:222
1730527217,30sleep10-21swapper/121:11:091
1453527215,38sleep114562-21df19:21:081
2232827124,27sleep20-21swapper/221:26:212
2123627115,27sleep10-21swapper/119:41:241
1541527117,24sleep00-21swapper/021:05:550
1541527117,24sleep00-21swapper/021:05:550
1513627113,42sleep00-21swapper/019:21:230
1915527017,26sleep20-21swapper/219:36:082
1538927020,23sleep20-21swapper/219:21:252
1341127025,23sleep00-21swapper/019:16:230
1341127025,23sleep00-21swapper/019:16:230
63032697,30sleep30-21swapper/320:36:223
1280526913,39sleep212812-21/usr/sbin/munin00:21:212
1112026918,34sleep211129-21/usr/sbin/munin20:51:192
901426817,23sleep20-21swapper/200:10:552
535126821,28sleep15456-21unixbench_singl23:56:361
1939626819,32sleep118751-21/usr/sbin/munin21:16:341
1939626819,32sleep118751-21/usr/sbin/munin21:16:341
1878026813,31sleep00-21swapper/021:16:090
1878026813,31sleep00-21swapper/021:16:090
3237826711,43sleep032411-21latency20:16:410
1364826713,31sleep00-21swapper/021:00:540
1330226716,37sleep113389-21tune2fs22:41:121
1070426724,22sleep00-21swapper/022:31:260
1070426724,22sleep00-21swapper/022:31:260
922526622,29sleep00-21swapper/000:11:010
587426619,27sleep00-21swapper/000:00:560
501726617,31sleep15150-21/usr/sbin/munin20:31:271
501726617,31sleep15150-21/usr/sbin/munin20:31:271
2956826616,35sleep229628-21sendmail-msp21:50:532
2050326614,36sleep120640-21latency_hist19:40:591
603126520,20sleep00-21swapper/022:16:270
355826510,41sleep03677-21users23:51:260
2636426516,23sleep00-21swapper/021:39:230
1776726511,10sleep10-21swapper/122:54:571
1375626520,27sleep00-21swapper/022:41:230
1107226517,31sleep111164-21/usr/sbin/munin00:16:151
81202645,12sleep20-21swapper/222:24:052
515926416,31sleep25191-21basename22:15:532
3084826415,26sleep00-21swapper/021:51:310
2893926411,38sleep028995-21ps20:06:230
2201026417,26sleep00-21swapper/019:45:550
11712649,41sleep20-21swapper/220:21:072
714426319,21sleep00-21swapper/020:40:550
1782126312,28sleep20-21swapper/219:31:152
1686926324,19sleep00-21swapper/022:51:330
979226217,28sleep20-21swapper/220:46:512
794526220,22sleep10-21swapper/119:01:131
425726220,24sleep10-21swapper/123:55:561
3185926214,25sleep20-21swapper/223:40:542
2407626216,19sleep20-21swapper/221:31:382
232026216,32sleep12387-21df22:06:061
915326118,25sleep10-21swapper/120:46:051
86026110,25sleep20-21swapper/222:01:102
74912616,41sleep20-21swapper/219:00:252
620326121,22sleep20-21swapper/220:36:172
287852617,11sleep00-21swapper/021:46:260
73082608,38sleep00-21swapper/000:05:080
318226019,18sleep20-21swapper/220:26:242
269282607,12sleep20-21swapper/223:23:052
2685826017,24sleep00-21swapper/021:41:130
2685826017,24sleep00-21swapper/021:41:130
2583526019,21sleep10-21swapper/119:56:221
1741526014,18sleep00-21swapper/021:11:120
1367826019,17sleep00-21swapper/000:25:560
1185126017,17sleep20-21swapper/222:36:122
2899725913,24sleep10-21swapper/120:06:241
2804425914,25sleep10-21swapper/123:26:231
2691825921,21sleep20-21swapper/220:01:082
2585725919,19sleep20-21swapper/219:56:232
2265225919,27sleep222679-21ntp_kernel_pll_19:46:212
170772596,10sleep10-21swapper/119:28:071
111122598,11sleep20-21swapper/222:32:252
111122598,11sleep20-21swapper/222:32:252
988525816,19sleep20-21swapper/222:30:552
511525815,21sleep20-21swapper/223:56:332
1882925819,20sleep00-21swapper/019:35:250
1383225811,18sleep20-21swapper/219:16:292
1383225811,18sleep20-21swapper/219:16:292
856125714,26sleep18627-21df22:26:141
3011525712,24sleep10-21swapper/121:51:041
2342425715,18sleep10-21swapper/123:11:291
199382575,40sleep20-21swapper/221:19:382
199382575,40sleep20-21swapper/221:19:382
18032578,38sleep00-21swapper/022:02:540
1565525714,19sleep10-21swapper/121:06:081
1326525711,19sleep20-21swapper/222:41:092
114872575,11sleep20-21swapper/219:11:122
1027525713,27sleep110281-21ntp_states19:06:221
1027525713,27sleep110281-21ntp_states19:06:221
930225620,17sleep00-21swapper/020:46:360
812925612,20sleep00-21swapper/000:06:210
72752565,10sleep20-21swapper/200:04:412
3271625618,18sleep10-21swapper/120:16:501
3055525614,26sleep030546-21apt-get23:36:080
2081925619,18sleep00-21swapper/021:21:420
1669425619,17sleep20-21swapper/219:26:262
155925617,25sleep01570-21grep23:46:170
126352566,39sleep00-21swapper/022:37:070
1073125613,19sleep00-21swapper/020:51:050
390325514,15sleep10-21swapper/122:11:071
199925513,25sleep12098-21users23:46:271
1075925514,25sleep110751-21apt-get20:51:101
2996525415,23sleep130172-21modprobe20:10:561
2565325410,16sleep10-21swapper/121:36:171
2427525414,23sleep124281-21ntp_states19:51:211
2350225416,23sleep123527-21munin-run19:50:531
2286525416,18sleep00-21swapper/019:46:280
2161025412,24sleep221648-21ntpq23:06:172
810625313,19sleep10-21swapper/120:41:251
608725311,17sleep10-21swapper/100:01:051
3209625313,24sleep032091-21apt-get23:41:040
1965525312,23sleep119666-21apt-get23:01:031
1868125310,22sleep20-21swapper/222:56:242
428925214,22sleep04286-21apt-get20:31:060
428925214,22sleep04286-21apt-get20:31:060
1565425212,22sleep015648-21apt-get21:06:070
1410925211,23sleep214096-21apt-get21:01:082
1410925211,23sleep214096-21apt-get21:01:082
2922325114,21sleep229272-21irqstats23:31:132
2329325114,23sleep023503-21latency_hist21:30:550
1983125014,18sleep00-21swapper/019:36:260
1888425015,15sleep00-21swapper/022:56:260
185402506,20sleep20-21swapper/221:15:552
74924911,23sleep1767-21apt-get22:01:011
707424913,16sleep10-21swapper/122:21:091
3189324915,17sleep10-21swapper/121:56:171
2079324911,23sleep120804-21ntp_states21:21:391
1514624914,14sleep00-21swapper/000:30:520
1514624914,14sleep00-21swapper/000:30:520
453424810,14sleep20-21swapper/220:31:162
453424810,14sleep20-21swapper/220:31:162
127024711,22sleep21279-21diskstats23:46:112
2549724512,15sleep20-21swapper/221:36:142
231924414,15sleep00-21swapper/022:06:050
1765424312,14sleep10-21swapper/119:31:101
114012397,15sleep10-21swapper/119:11:061
290762388,14sleep10-21swapper/123:31:081
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional