You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-30 - 12:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Thu Jul 30, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
169252718014,7152sleep315259-21kworker/3:221:25:003
12079267913,653sleep310616-21kworker/3:200:33:593
10658256218,531sleep110706-21ntpq19:23:131
22606251031,424sleep222622-21latency_hist23:27:492
24693239610,331sleep224714-21sshd20:08:002
24693239610,331sleep224714-21sshd20:08:002
1060023899,362sleep210685-21cut22:48:172
5500236064,234sleep25601-21timerandwakeup00:13:192
14793234111,241sleep214880-21latency_hist23:02:502
2896023299,305sleep228996-21ntpq22:03:112
2896023299,305sleep228996-21ntpq22:03:112
26889225212,23sleep20-21swapper/223:38:352
389122399,218sleep23967-21sed00:08:182
467222779,22sleep20-21swapper/222:17:482
24168220422,22sleep20-21swapper/223:32:502
667022020,29sleep24596-21cyclictest19:12:512
25854220223,79sleep20-21swapper/223:37:502
10054219310,90sleep210096-21ntpq21:03:152
26363219117,158sleep026388-21grep20:13:100
449321908,36sleep24580-21timerandwakeup22:28:182
21943218210,159sleep3364-21snmpd21:41:233
30943217918,23sleep10-21swapper/123:53:071
13234217975,29sleep20-21swapper/221:13:162
13234217975,29sleep20-21swapper/221:13:162
6782217618,93sleep26731-21latency_hist22:37:512
1436121759,150sleep214510-21grep19:33:402
17265217317,22sleep20-21swapper/219:43:172
864221708,13sleep00-21swapper/022:43:060
4453216722,119sleep04562-21sed19:07:490
12457216744,95sleep20-21swapper/200:37:492
25163216616,133sleep025304-21uptime23:33:200
15755216124,112sleep015905-21grep23:03:210
597216090,14sleep20-21swapper/223:58:212
597216090,14sleep20-21swapper/223:58:212
595221607,104sleep2364-21snmpd20:49:182
27756216016,68sleep20-21swapper/223:42:592
28331215871,70sleep228489-21latency_hist22:02:502
7086215624,81sleep20-21swapper/200:18:202
22051215011,81sleep222074-21sh21:42:482
23885214764,25sleep20-21swapper/221:48:012
26697214519,104sleep11-21systemd21:57:491
1899021437,11sleep00-21swapper/023:13:180
19325214135,84sleep219339-21fschecks_count21:33:232
5045214049,25sleep20-21swapper/219:08:242
15640214010,78sleep215675-21latency_hist21:22:492
15640214010,78sleep215675-21latency_hist21:22:492
26852213949,33sleep226927-21switchtime20:13:252
13018213811,63sleep20-21swapper/222:57:492
14166213619,92sleep314315-21unixbench_multi22:58:273
16667213162,55sleep20-21swapper/223:08:072
3098212510,101sleep23211-21latency_hist00:07:502
6977212426,79sleep16207-21apt-get19:12:591
31423212414,84sleep00-21swapper/022:12:500
31423212414,84sleep00-21swapper/022:12:500
23023212318,24sleep20-21swapper/220:03:072
18263212348,63sleep218267-21grep19:48:052
29823212151,12sleep20-21swapper/220:23:142
13475211920,79sleep113625-21head21:13:191
13475211920,79sleep113625-21head21:13:191
7113211641,55sleep20-21swapper/220:53:182
29688211529,70sleep20-21swapper/223:48:122
15633211541,53sleep20-21swapper/219:38:162
30115211415,79sleep20-21swapper/222:08:032
30115211415,79sleep20-21swapper/222:08:032
8658211313,77sleep20-21swapper/222:43:082
30958211320,74sleep330998-21fschecks_count23:53:093
31141211214,25sleep20-21swapper/223:53:122
9060211118,75sleep39143-21awk22:43:173
13597211116,76sleep313578-21apt-get19:33:083
2324521104,93sleep00-21swapper/021:43:220
2324521104,93sleep00-21swapper/021:43:220
22699211056,23sleep20-21swapper/220:02:512
22699211056,23sleep20-21swapper/220:02:512
14319211021,74sleep014388-21users22:58:280
3383210934,16sleep20-21swapper/220:43:002
12128210913,78sleep012150-21pluginstate22:53:190
26807210824,63sleep10-21swapper/123:38:341
20268210814,78sleep020270-21ntp_states19:53:160
15961210837,15sleep20-21swapper/221:23:052
3423210713,79sleep33429-21cat00:08:073
3219321069,82sleep332265-21sensors_temp22:13:173
2114221068,84sleep221246-21latency_hist19:57:502
14929210412,78sleep115045-21cut19:37:511
29714210316,68sleep329749-21ntp_kernel_pll_23:48:133
16581210310,16sleep00-21swapper/023:08:010
9110210210,80sleep29157-21ntpq19:18:132
7102210234,14sleep20-21swapper/222:38:072
6222210215,70sleep11-21systemd20:52:481
28128210113,72sleep327834-21/usr/sbin/munin20:18:103
20176210115,70sleep120207-21/usr/sbin/munin19:53:131
32527210018,66sleep332682-21apt-key20:32:523
32527210018,66sleep332682-21apt-key20:32:523
10903210019,59sleep310929-21apt21:07:483
504829917,67sleep05080-21fschecks_count20:48:070
3270229933,13sleep20-21swapper/220:32:552
3270229933,13sleep20-21swapper/220:32:552
193482995,35sleep20-21swapper/223:16:352
2452629713,68sleep124652-21timerwakeupswit21:48:161
1667529716,61sleep316769-21fschecks_time23:08:083
1524429710,72sleep015251-21diskstats19:38:060
2234329517,19sleep20-21swapper/221:43:042
2234329517,19sleep20-21swapper/221:43:042
2199029518,56sleep022123-21sed23:23:190
592029417,61sleep15981-21grep22:33:181
2959029417,64sleep129627-21idleruntime20:23:091
267082945,11sleep30-21swapper/323:38:323
140329422,48sleep31596-21apt-config00:02:503
140329422,48sleep31596-21apt-config00:02:503
347729330,36sleep30-21swapper/319:03:133
2677429320,18sleep20-21swapper/221:57:502
231329311,67sleep02332-21apt-get22:22:590
1321529314,18sleep20-21swapper/219:32:512
84429264,13sleep20-21swapper/222:18:092
486629217,62sleep34877-21acpi00:13:003
1625129212,64sleep016281-21awk21:23:110
319329140,34sleep23210-21fschecks_count19:03:082
216429112,61sleep02192-21nfs4_client00:03:160
2987529016,55sleep129906-21latency_hist22:07:501
2987529016,55sleep129906-21latency_hist22:07:501
1311429030,29sleep10-21swapper/119:32:501
656628919,44sleep36630-21fschecks_time00:18:093
631428916,54sleep06478-21sed19:12:500
397228916,44sleep34059-21fschecks_time22:28:073
87112889,16sleep20-21swapper/220:58:182
2715928817,46sleep026986-21/usr/sbin/munin21:58:080
267692889,62sleep126851-21mailstats20:13:241
2419428726,44sleep31-21systemd20:07:483
2056228612,16sleep20-21swapper/223:18:202
3234128410,16sleep20-21swapper/220:32:512
3234128410,16sleep20-21swapper/220:32:512
123212845,32sleep20-21swapper/221:10:582
410428314,49sleep03314-21/usr/sbin/munin00:08:200
1679928330,26sleep00-21swapper/019:43:060
96692828,14sleep20-21swapper/200:28:092
96692828,14sleep20-21swapper/200:28:092
3075628217,42sleep330907-21latency_hist20:27:513
2530628218,50sleep125319-21vmstat23:33:211
198472820,7sleep10-21swapper/123:18:061
163832817,20sleep216533-21latency_hist23:07:512
84202808,57sleep28458-21ntpq00:23:122
23482807,49sleep20-21swapper/220:38:202
1951928019,45sleep319576-21latency_hist23:17:493
1185328022,35sleep011984-21grep21:08:220
114692800,6sleep30-21swapper/321:08:123
2212327925,29sleep122268-21grep21:42:511
2493927718,28sleep00-21swapper/020:08:220
2493927718,28sleep00-21swapper/020:08:220
2469227718,39sleep124209-21apt-get20:07:591
2469227718,39sleep124209-21apt-get20:07:591
1193227717,38sleep012031-21head00:33:400
799627618,28sleep00-21swapper/000:22:580
322122769,15sleep00-21swapper/023:57:510
27422768,57sleep22783-21ntpq22:23:132
2503927621,39sleep325160-21timerwakeupswit23:33:193
1950127622,40sleep119520-21kernelversion21:33:261
1082927621,32sleep110855-21munin-run00:32:481
1082927621,32sleep110855-21munin-run00:32:481
291572757,19sleep229167-21munin-run20:22:492
2845527520,35sleep128618-21grep23:43:181
2708327411,45sleep127092-21/usr/sbin/munin21:58:051
262192747,14sleep00-21swapper/023:38:000
1302127416,39sleep313058-21latency21:13:093
1302127416,39sleep313058-21latency21:13:093
2317427320,27sleep30-21swapper/321:43:213
2317427320,27sleep30-21swapper/321:43:213
1156527341,17sleep20-21swapper/219:27:502
398927213,42sleep04082-21threads20:43:190
2798227220,26sleep30-21swapper/323:43:083
1853127227,23sleep10-21swapper/119:48:121
113202720,7sleep10-21swapper/121:08:101
349327128,25sleep10-21swapper/120:43:071
2908227121,28sleep00-21swapper/023:47:490
184872710,7sleep00-21swapper/019:48:100
827127015,31sleep00-21swapper/020:58:080
623927026,20sleep10-21swapper/100:17:501
1956927024,23sleep30-21swapper/319:52:503
1793727020,25sleep10-21swapper/123:12:501
112972705,54sleep00-21swapper/022:51:500
996826911,43sleep0364-21snmpd21:03:130
515226919,21sleep00-21swapper/022:32:480
465426918,32sleep14716-21latency_hist00:12:501
2856926915,39sleep028722-21awk20:18:190
2426126916,25sleep30-21swapper/321:48:103
185332695,53sleep00-21swapper/021:30:470
2331626819,35sleep023367-21awk20:03:130
938926724,21sleep00-21swapper/019:18:190
882426716,24sleep30-21swapper/320:58:193
672426717,29sleep36764-21users19:12:533
1086026717,21sleep10-21swapper/122:48:201
456426618,28sleep14782-21latency_hist19:07:511
3116126617,25sleep10-21swapper/120:28:091
3116126617,25sleep10-21swapper/120:28:091
195802668,13sleep00-21swapper/023:17:500
1743226636,14sleep20-21swapper/221:28:022
689426519,23sleep00-21swapper/019:12:580
541726519,19sleep00-21swapper/022:33:000
3092626513,36sleep330934-21/usr/sbin/munin20:27:593
3092626513,36sleep330934-21/usr/sbin/munin20:27:593
2184126521,29sleep321882-21ps19:58:163
2184126521,29sleep321882-21ps19:58:163
209926521,22sleep30-21swapper/322:22:503
1151426523,28sleep011537-21idleruntime-cro19:27:480
1142226519,19sleep30-21swapper/322:52:503
995226419,27sleep310001-21ntpq00:28:143
995226419,27sleep310001-21ntpq00:28:143
709226417,31sleep17101-21/usr/sbin/munin22:38:061
46226411,36sleep0364-21snmpd23:58:190
46226411,36sleep0364-21snmpd23:58:190
204826422,25sleep12064-21ls22:22:481
1698126422,19sleep30-21swapper/319:43:093
1567626419,25sleep30-21swapper/321:22:503
623026320,22sleep30-21swapper/319:12:493
2285726321,22sleep30-21swapper/323:28:023
1897426318,27sleep318714-21apt-get21:32:513
1388826319,27sleep10-21swapper/122:58:211
1096826324,23sleep10-21swapper/121:07:501
627326222,22sleep30-21swapper/320:52:503
3075626223,23sleep10-21swapper/122:08:181
3075626223,23sleep10-21swapper/122:08:181
223272625,10sleep10-21swapper/119:58:461
223272625,10sleep10-21swapper/119:58:461
170822628,12sleep10-21swapper/119:43:121
1638326211,19sleep10-21swapper/121:23:171
1174926212,25sleep30-21swapper/322:53:103
869826112,32sleep18701-21apt-get19:18:001
716826120,24sleep00-21swapper/022:38:090
571326123,17sleep30-21swapper/322:33:123
2976626114,27sleep00-21swapper/020:23:120
2412126118,17sleep00-21swapper/021:48:070
1530526114,25sleep315397-21load23:03:113
1482426121,20sleep30-21swapper/321:18:183
785026013,33sleep37909-21date20:57:513
316926018,19sleep00-21swapper/020:42:500
262722608,37sleep30-21swapper/320:13:013
243426019,20sleep30-21swapper/300:03:233
1870626020,17sleep10-21swapper/123:13:151
1272626017,28sleep00-21swapper/021:12:580
1272626017,28sleep00-21swapper/021:12:580
1257226018,24sleep312702-21/usr/sbin/munin19:28:203
366225915,23sleep30-21swapper/322:27:513
327352596,15sleep10-21swapper/120:33:021
327352596,15sleep10-21swapper/120:33:021
235712596,12sleep10-21swapper/121:47:271
235712596,12sleep10-21swapper/121:47:271
2141025914,29sleep021512-21timerandwakeup21:38:220
1295425920,20sleep10-21swapper/122:57:481
126125915,26sleep11312-21apt-config00:02:481
126125915,26sleep11312-21apt-config00:02:481
622825826,18sleep00-21swapper/000:17:490
229025815,27sleep32314-21ntp_states20:38:173
2158825814,18sleep00-21swapper/019:58:080
2158825814,18sleep00-21swapper/019:58:080
1874525819,19sleep10-21swapper/121:32:491
179525817,24sleep11792-21apt-get00:03:051
1636125818,19sleep30-21swapper/319:42:003
748025715,27sleep37487-21/usr/sbin/munin22:38:163
2899825715,24sleep329017-21ntp_states22:03:123
2899825715,24sleep329017-21ntp_states22:03:123
2285225716,24sleep022848-21apt-get23:27:590
185825717,22sleep11856-21apt-get20:38:031
161825722,19sleep00-21swapper/020:37:490
161825722,19sleep00-21swapper/020:37:490
93252569,17sleep30-21swapper/300:27:493
522925617,27sleep15266-21grep19:08:271
2709825620,18sleep30-21swapper/321:58:073
2538425619,17sleep30-21swapper/320:08:333
2538425619,17sleep30-21swapper/320:08:333
2145725616,20sleep10-21swapper/123:23:081
2004825615,23sleep320140-21users21:33:363
1983325618,18sleep30-21swapper/323:18:053
388325513,25sleep13878-21apt-get22:28:011
3220925515,24sleep032233-21munin-run20:32:490
3220925515,24sleep032233-21munin-run20:32:490
2932225513,25sleep129314-21apt-get23:48:031
2131025513,25sleep321303-21apt-get23:23:023
1974525514,24sleep019837-21mailstats21:33:330
1434025515,25sleep114334-21apt-get21:18:041
1434025515,25sleep114334-21apt-get21:18:041
1358125513,19sleep00-21swapper/019:33:010
883825413,25sleep18932-21memory22:43:111
88225417,18sleep00-21swapper/022:18:110
778625417,16sleep00-21swapper/000:22:500
651425410,16sleep00-21swapper/020:53:040
2563025416,15sleep00-21swapper/021:53:070
2340125415,22sleep123467-21swap23:28:161
1306125414,23sleep013110-21systemd-run19:32:490
1112425413,24sleep111118-21apt-get00:33:041
1000425414,23sleep010028-21ntp_states00:28:150
1000425414,23sleep010028-21ntp_states00:28:150
302592537,35sleep00-21swapper/023:48:420
289302536,10sleep10-21swapper/120:20:221
1516725311,25sleep115159-21apt-get19:38:031
1001725314,16sleep30-21swapper/319:22:513
808625212,24sleep18080-21apt-get20:58:011
340625215,21sleep33403-21apt-get20:43:013
3166225213,23sleep131654-21apt-get22:13:031
3015325212,25sleep330186-21df22:08:043
3015325212,25sleep330186-21df22:08:043
297302525,35sleep00-21swapper/022:06:360
297302525,35sleep00-21swapper/022:06:360
2543625213,24sleep325484-21awk21:53:023
1988625214,22sleep319892-21diskstats19:53:073
2434325116,20sleep124522-21sed20:07:501
70162509,18sleep10-21swapper/120:53:171
54724913,18sleep00-21swapper/022:17:500
33672496,34sleep10-21swapper/122:24:501
1521724914,15sleep10-21swapper/123:03:091
1458724911,21sleep014610-21irqstats21:18:110
1458724911,21sleep014610-21irqstats21:18:110
1055524710,19sleep310624-21memory19:23:113
74382429,19sleep17439-21sshd00:18:591
814524110,15sleep10-21swapper/100:23:071
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional