You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-24 - 10:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Wed Jun 24, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17282330616,3274sleep015679-1kworker/0:1H21:45:270
15346210538,1034sleep329-1kworker/3:0H00:10:223
18256210087,941sleep018291-21ntpq20:55:260
18256210087,941sleep018291-21ntpq20:55:260
21852248815,454sleep221946-21/usr/sbin/munin19:25:012
181582479445,22sleep218160-21sshd19:13:382
22406239529,349sleep122479-21users00:30:361
23937236038,280sleep024118-21apt-key21:15:010
4797233612,309sleep34871-21find23:35:283
3943232953,197sleep03971-21fschecks_count20:10:190
13263232252,218sleep00-21swapper/022:20:330
19788230826,269sleep019853-21latency_hist00:25:010
548922737,254sleep05521-21tune2fs20:15:150
548922737,254sleep05521-21tune2fs20:15:150
807722597,28sleep00-21swapper/020:22:470
807722597,28sleep00-21swapper/020:22:470
20185224643,111sleep020288-21unixbench_singl21:00:310
20185224643,111sleep020288-21unixbench_singl21:00:310
92562240122,81sleep00-21swapper/023:50:240
15694223418,126sleep115762-21latency19:05:211
12889222634,154sleep012949-21sendmail20:39:590
1598522247,201sleep016121-21latency_hist22:30:020
1472222320,190sleep11511-21ntpq23:25:251
6234222032,63sleep00-21swapper/023:40:260
10741221510,152sleep00-21swapper/022:14:590
28301221414,110sleep028409-21latency_hist19:45:020
23910220714,180sleep023913-21cron22:54:580
23910220714,180sleep023913-21cron22:54:580
2609220422,39sleep00-21swapper/020:05:220
21215219811,118sleep021344-21grep19:20:300
7934219746,91sleep00-21swapper/022:05:140
15025218812,159sleep314826-21latency_hist00:10:023
15025218812,159sleep314826-21latency_hist00:10:023
27180218763,69sleep00-21swapper/021:24:590
4279218527,121sleep04346-21fschecks_time23:35:170
2920218436,85sleep00-21swapper/023:30:200
19527218213,154sleep119021-21apt-get22:40:091
20295217845,38sleep00-21swapper/019:20:020
20295217845,38sleep00-21swapper/019:20:020
28622217530,129sleep028704-21cut19:45:180
16496217025,134sleep016555-21if_eth020:50:210
21368216310,90sleep00-21swapper/022:45:210
21368216310,90sleep00-21swapper/022:45:210
499216116,128sleep00-21swapper/021:40:300
2362021619,56sleep00-21swapper/000:35:270
11751215611,128sleep011856-21latency_hist00:00:010
11751215611,128sleep011856-21latency_hist00:00:010
5559215319,113sleep25696-21unixbench_singl21:55:352
6953215241,74sleep07014-21switchtime22:00:290
19519215110,38sleep00-21swapper/022:40:040
17584215118,119sleep317730-21unixbench_multi00:15:313
21939214616,77sleep00-21swapper/000:30:270
302721459,120sleep23063-21ntpq23:30:232
17263214533,54sleep016773-21cyclictest00:15:240
10606214516,111sleep00-21swapper/020:30:300
9481214227,87sleep39476-21apt-get22:10:143
24373214239,86sleep00-21swapper/019:30:280
1720621427,86sleep017260-21if_eth019:10:200
14088214230,75sleep00-21swapper/000:05:350
14088214230,75sleep00-21swapper/000:05:350
15300214121,72sleep00-21swapper/000:10:070
214521406,30sleep00-21swapper/023:27:130
17928213834,56sleep00-21swapper/022:35:190
20530213749,48sleep30-21swapper/319:20:133
25212213234,28sleep00-21swapper/022:56:480
15397213045,39sleep20-21swapper/219:05:102
8625212629,72sleep00-21swapper/020:25:210
15302212623,83sleep214818-21apt-get00:10:092
322921256,29sleep00-21swapper/021:50:070
5813212320,59sleep00-21swapper/021:56:460
29801212315,92sleep129858-21wc21:31:141
9224212115,89sleep19359-21awk20:25:341
22116212118,35sleep00-21swapper/021:08:190
22116212118,35sleep00-21swapper/021:08:190
1312212113,88sleep31471-21latency_hist21:45:013
12389212128,82sleep012426-21ntpq00:00:260
3082421198,98sleep230860-21ntp_kernel_pll_23:15:242
24293211920,83sleep224385-21perl22:55:212
3039921179,24sleep00-21swapper/023:15:100
27679211715,83sleep327194-21apt-get21:25:103
27679211715,83sleep327194-21apt-get21:25:103
14208211618,45sleep00-21swapper/020:41:410
1102321157,16sleep00-21swapper/022:15:110
2073211429,19sleep00-21swapper/020:05:020
22288211315,78sleep321820-21apt-get19:25:103
29247211222,72sleep329276-21/usr/sbin/munin23:10:243
30284211130,38sleep00-21swapper/019:50:190
2578521118,66sleep025829-21ntpq19:35:250
21192210916,79sleep121210-21fschecks_count22:45:181
21192210916,79sleep121210-21fschecks_count22:45:181
18317210921,70sleep118379-21latency_hist19:15:021
23577210850,17sleep00-21swapper/019:30:000
3197921075,30sleep00-21swapper/023:20:070
26706210712,61sleep00-21swapper/023:00:480
25949210713,70sleep125389-21/usr/sbin/munin19:35:301
11998210713,75sleep212026-21nfs4_client20:35:252
19084210611,14sleep20-21swapper/200:20:312
19000210621,67sleep319021-21apt-get22:39:593
16277210616,72sleep216279-21/usr/sbin/munin20:50:112
2864021059,45sleep00-21swapper/023:10:010
23961210522,63sleep324067-21latency_hist22:55:013
23961210522,63sleep324067-21latency_hist22:55:013
25425210413,77sleep125426-21sshd21:17:411
15355210421,63sleep115321-21apt-get00:10:281
29137210326,36sleep00-21swapper/021:30:200
29137210326,36sleep00-21swapper/021:30:200
2548721035,27sleep00-21swapper/021:18:290
17106210222,58sleep317200-21fschecks_time19:10:193
6152210119,65sleep16298-21unixbench_singl20:15:291
6152210119,65sleep16298-21unixbench_singl20:15:291
12556210133,44sleep212313-21apt-get22:20:022
3077921006,85sleep2228-1kworker/2:2H21:35:112
3020210014,67sleep20-21swapper/221:50:012
28303210013,71sleep10-21swapper/123:06:181
21326210015,71sleep221345-21ls00:30:002
2577329915,69sleep225765-21apt23:00:132
1528729823,38sleep00-21swapper/020:45:290
2753429713,25sleep30-21swapper/319:40:293
201092978,54sleep020178-21fschecks_time00:25:180
2577429619,15sleep30-21swapper/323:00:143
3219229515,64sleep00-21swapper/021:40:010
2437429522,54sleep324408-21idleruntime-cro00:39:593
3162929418,14sleep00-21swapper/019:55:120
853229327,41sleep28529-21apt-get20:25:112
2897429323,47sleep229005-21grep21:30:022
2897429323,47sleep229005-21grep21:30:022
2716229321,47sleep20-21swapper/219:40:202
1749929314,60sleep217582-21grep00:15:302
395629031,34sleep20-21swapper/221:50:462
3022029010,21sleep030364-21latency_hist23:15:020
752228913,62sleep37573-21if_eth023:45:223
1166528819,46sleep211692-21munin-run23:59:592
1166528819,46sleep211692-21munin-run23:59:592
1485228715,48sleep114939-21store00:10:001
1485228715,48sleep114939-21store00:10:001
707428518,54sleep27138-21latency_hist23:45:012
2380528518,50sleep223853-21apt-config21:14:592
181428520,15sleep10-21swapper/121:45:321
1461628515,49sleep318329-21munin-node22:25:123
2323928416,45sleep123380-21tail21:10:341
2666928330,33sleep226697-21modprobe19:39:592
1446528221,39sleep3680-21cron20:44:593
1134228214,52sleep111372-21munin-run20:34:591
789128126,41sleep07965-21cat23:45:310
1965928114,54sleep319666-21sh22:40:393
1534128124,38sleep215489-21tail22:25:402
2445528023,30sleep20-21swapper/200:40:022
2413128032,29sleep10-21swapper/119:30:211
198602808,14sleep20-21swapper/221:00:252
198602808,14sleep20-21swapper/221:00:252
1783428014,52sleep217836-21/usr/sbin/munin20:55:122
1783428014,52sleep217836-21/usr/sbin/munin20:55:122
1423527832,20sleep20-21swapper/200:05:372
1423527832,20sleep20-21swapper/200:05:372
2060027721,44sleep2369-21sshd22:42:122
1332327712,46sleep313355-21/usr/sbin/munin22:20:363
741427613,50sleep27447-21ntp_offset20:20:352
741427613,50sleep27447-21ntp_offset20:20:352
459827615,49sleep24642-21ntp_offset23:35:232
2165227614,45sleep221661-21/usr/sbin/munin00:30:202
3144327519,40sleep231357-21latency_hist19:55:022
307702759,54sleep3364-21snmpd21:35:043
3002727521,14sleep00-21swapper/021:31:170
1761527528,25sleep30-21swapper/320:55:013
1111927521,31sleep30-21swapper/323:55:323
1111927521,31sleep30-21swapper/323:55:323
66682749,17sleep20-21swapper/220:19:542
66682749,17sleep20-21swapper/220:19:542
323227427,27sleep33251-21apt-get21:50:103
3206127424,31sleep132065-21cron21:39:581
895627344,16sleep30-21swapper/320:25:283
2709127322,22sleep10-21swapper/123:05:011
2428627313,26sleep10-21swapper/122:55:201
1668227328,29sleep016715-21/usr/sbin/munin19:10:000
585027219,28sleep30-21swapper/323:40:173
545027224,27sleep10-21swapper/123:39:591
3051527223,20sleep20-21swapper/221:34:592
2666027230,25sleep10-21swapper/121:20:321
246722726,49sleep30-21swapper/319:30:313
1812027225,25sleep10-21swapper/120:55:231
1812027225,25sleep10-21swapper/120:55:231
1667627213,27sleep10-21swapper/122:30:311
55127122,26sleep20-21swapper/220:00:032
520327126,22sleep30-21swapper/321:55:273
516927116,29sleep30-21swapper/320:15:023
456827125,27sleep10-21swapper/121:55:011
3042627117,36sleep130493-21df23:15:151
2908127118,34sleep20-21swapper/219:45:292
2302327116,30sleep20-21swapper/222:50:242
2302327116,30sleep20-21swapper/222:50:242
1618827118,38sleep00-21swapper/022:30:140
1207727117,25sleep20-21swapper/200:00:192
1170927126,20sleep30-21swapper/322:15:353
1095227115,28sleep10-21swapper/123:55:301
1095227115,28sleep10-21swapper/123:55:301
2587527014,30sleep30-21swapper/319:35:283
3045026913,39sleep330473-21/usr/sbin/munin19:50:223
288526915,35sleep32954-21/usr/sbin/munin20:05:293
2545626915,33sleep325476-21latency_hist22:59:583
2049126916,36sleep120556-21awk00:25:271
1910926923,28sleep10-21swapper/122:40:011
814426827,22sleep20-21swapper/222:05:202
718526814,37sleep17240-21/usr/sbin/munin20:20:291
718526814,37sleep17240-21/usr/sbin/munin20:20:291
597626816,14sleep30-21swapper/320:15:273
597626816,14sleep30-21swapper/320:15:273
2951026811,41sleep229626-21timerwakeupswit23:10:312
213026816,33sleep32169-21/usr/sbin/munin21:45:383
2006526823,21sleep30-21swapper/321:00:303
2006526823,21sleep30-21swapper/321:00:303
81126723,23sleep30-21swapper/323:24:583
2362726720,21sleep10-21swapper/119:30:011
170926721,28sleep30-21swapper/323:25:313
1484026715,32sleep214900-21/usr/sbin/munin20:45:192
44126621,22sleep10-21swapper/123:20:321
2870526613,31sleep10-21swapper/119:45:191
1687926617,19sleep10-21swapper/120:50:311
1183926618,29sleep30-21swapper/320:35:223
118126621,29sleep11211-21munin-run21:44:591
3257026516,31sleep232639-21switchtime23:20:292
2880426519,26sleep30-21swapper/319:45:223
2730126413,33sleep227393-21apt-config21:25:012
2395026418,20sleep20-21swapper/219:30:182
2333426413,20sleep10-21swapper/122:50:311
2333426413,20sleep10-21swapper/122:50:311
1963926419,26sleep10-21swapper/121:00:201
1963926419,26sleep10-21swapper/121:00:201
1619126415,25sleep30-21swapper/322:30:173
181112638,13sleep10-21swapper/100:19:211
1590726319,22sleep30-21swapper/319:05:273
163626210,25sleep20-21swapper/223:25:302
1213826214,31sleep312202-21idleruntime00:00:203
1940526111,36sleep219411-21/usr/sbin/munin19:15:272
1940526111,36sleep219411-21/usr/sbin/munin19:15:272
126126119,24sleep11339-21grep20:00:381
911826021,23sleep19131-21irqstats23:50:211
425626018,22sleep20-21swapper/220:10:252
267626018,24sleep12705-21/usr/sbin/munin20:05:231
2432326012,18sleep30-21swapper/321:15:123
3237425917,25sleep232371-21apt-get21:40:122
199742596,39sleep10-21swapper/119:17:001
199742596,39sleep10-21swapper/119:17:001
1411225916,25sleep214137-21munin-run22:24:592
1270525911,37sleep1364-21snmpd20:37:381
988325820,23sleep10-21swapper/122:10:241
51025822,16sleep10-21swapper/120:00:021
3051225814,27sleep130536-21ntp_states19:50:241
2179125821,17sleep10-21swapper/119:25:001
1885725813,24sleep20-21swapper/222:38:082
1535925823,22sleep115449-21sed20:45:301
1512225814,30sleep315157-21ntp_kernel_err20:45:243
146422589,23sleep10-21swapper/122:25:141
861325719,24sleep18622-21munin-run23:50:001
705025715,24sleep17073-21munin-run23:45:001
270625715,21sleep20-21swapper/220:05:242
2430525717,21sleep20-21swapper/221:15:112
2338225719,18sleep10-21swapper/100:35:201
2254925714,26sleep322544-21apt-get21:10:163
2047525716,25sleep320477-21ntp_states00:25:253
1919525718,17sleep30-21swapper/321:00:023
1919525718,17sleep30-21swapper/321:00:023
1246625716,19sleep10-21swapper/122:20:011
635325614,25sleep16371-21apt22:00:101
386025620,21sleep10-21swapper/120:10:161
2793125618,17sleep10-21swapper/121:25:321
2793125618,17sleep10-21swapper/121:25:321
2110225615,24sleep321099-21apt-get22:45:123
2110225615,24sleep321099-21apt-get22:45:123
1913325614,24sleep319186-21/usr/sbin/munin19:15:203
1913325614,24sleep319186-21/usr/sbin/munin19:15:203
138692565,14sleep10-21swapper/122:21:511
1008325617,23sleep210160-21grep22:10:292
896925517,16sleep30-21swapper/323:50:193
727225519,17sleep30-21swapper/322:00:323
318325516,14sleep30-21swapper/323:30:283
2733025518,17sleep20-21swapper/223:05:142
2307825516,15sleep20-21swapper/219:25:412
508125415,23sleep15174-21memory21:55:251
2696225413,24sleep126981-21apt-get19:40:101
2166125418,19sleep30-21swapper/321:05:313
2166125418,19sleep30-21swapper/321:05:313
2156825417,19sleep20-21swapper/222:45:272
2156825417,19sleep20-21swapper/222:45:272
1857125415,22sleep318661-21perl00:20:203
1646025415,18sleep20-21swapper/222:30:242
1451725420,19sleep10-21swapper/120:45:011
1017125415,18sleep20-21swapper/220:30:202
768425318,16sleep20-21swapper/223:45:252
302382536,35sleep30-21swapper/321:31:243
2945625317,19sleep10-21swapper/123:10:301
2770125314,16sleep30-21swapper/323:05:233
2615625316,19sleep30-21swapper/321:20:213
74325213,23sleep3746-21apt-get20:00:123
638025215,22sleep26447-21df22:00:152
2203125213,22sleep322037-21ntp_states00:30:303
1847725211,24sleep118472-21apt-get00:20:141
1207025217,20sleep10-21swapper/100:00:181
796725114,21sleep18005-21df22:05:151
278725115,19sleep10-21swapper/123:30:171
2231125120,17sleep10-21swapper/121:10:011
2231125120,17sleep10-21swapper/121:10:011
2106825112,16sleep20-21swapper/221:05:182
2106825112,16sleep20-21swapper/221:05:182
3050525011,22sleep330509-21diskstats23:15:173
120325013,22sleep21219-21ntp_states20:00:352
99432496,34sleep20-21swapper/223:52:442
837024912,16sleep30-21swapper/322:05:253
3169024910,13sleep30-21swapper/319:55:153
745024814,20sleep37460-21ntpq20:20:363
745024814,20sleep37460-21ntpq20:20:363
1008224813,20sleep310078-21apt-get20:30:133
1320624412,15sleep20-21swapper/220:40:142
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional