You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-26 - 19:24

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Wed Aug 26, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
211332102738,10232sleep120958-21kworker/1:112:13:151
263952808410,8062sleep219648-21kworker/2:212:29:142
29271274217,714sleep212196-21kworker/2:210:55:152
11149263317,598sleep211197-21/usr/sbin/munin11:39:372
2370826067,563sleep123749-21ntpq12:19:321
1407525448,520sleep114056-21latency_hist10:09:081
1407525448,520sleep114056-21latency_hist10:09:081
2348825347,519sleep212574-1kworker/2:0H08:54:282
3732240725,348sleep1367-21snmpd11:17:431
28779237915,265sleep128818-21wc10:54:381
2377623177,259sleep123823-21ntpq08:54:351
27848228488,131sleep10-21swapper/109:09:081
25034227315,240sleep325062-21/usr/sbin/munin07:19:083
9277226816,235sleep39279-21sshd09:53:153
3229622617,242sleep132370-21find11:04:431
15354224538,155sleep115425-21fschecks_time11:54:251
1746223214,202sleep21879-21awk11:09:452
31197221712,194sleep131217-21apt-get09:19:171
4788221430,131sleep14912-21ls11:19:371
1577221312,121sleep11586-21modprobe09:29:071
283982211144,52sleep128408-21modprobe07:29:071
283982211144,52sleep128408-21modprobe07:29:071
24048220712,182sleep324097-21ntpq10:39:303
24048220712,182sleep324097-21ntpq10:39:303
14120220645,124sleep114164-21ntpq11:49:331
22365220272,58sleep10-21swapper/107:09:371
10077219611,135sleep10-21swapper/108:14:091
28772218516,149sleep328801-21fschecks_count07:29:253
28772218516,149sleep328801-21fschecks_count07:29:253
28186218313,155sleep228281-21fschecks_time09:09:252
22183217348,65sleep112-21rcu_sched10:34:271
18994217347,55sleep121603-21cyclictest10:24:221
10250216633,49sleep10-21swapper/109:54:381
618121659,56sleep10-21swapper/111:24:341
618121659,56sleep10-21swapper/111:24:341
28434215611,15sleep10-21swapper/112:34:341
18266215636,15sleep10-21swapper/108:39:201
770215311,14sleep00-21swapper/009:24:330
26529215333,42sleep10-21swapper/110:49:091
12417215218,116sleep112026-21/usr/sbin/munin08:19:301
18776214747,59sleep118919-21cat10:24:101
3710214578,52sleep13719-21modprobe07:54:061
31396214315,68sleep10-21swapper/111:04:061
17242214322,49sleep10-21swapper/111:59:301
14919214118,60sleep10-21swapper/110:09:361
14919214118,60sleep10-21swapper/110:09:361
17043213962,21sleep10-21swapper/108:34:321
23883213717,106sleep223952-21switchtime12:19:372
21710213530,84sleep221217-21apt-get12:14:172
27037213311,59sleep127108-21latency10:49:291
2150421310,16sleep0231-21systemd-journal07:09:060
2150421310,16sleep0231-21systemd-journal07:09:060
3585213016,42sleep10-21swapper/109:34:271
2308621269,63sleep123141-21ls12:19:081
3304212416,86sleep23447-21unixbench_singl11:14:372
27597212413,91sleep027631-21/usr/sbin/munin07:25:390
27597212413,91sleep027631-21/usr/sbin/munin07:25:390
2399212462,17sleep10-21swapper/109:29:351
24847212312,95sleep024867-21apt-get12:24:160
20224212347,47sleep120221-21apt-get07:04:201
20224212347,47sleep120221-21apt-get07:04:201
11307212366,16sleep10-21swapper/109:59:271
11307212366,16sleep10-21swapper/109:59:271
29399212230,76sleep129517-21latency_hist09:14:091
25547212114,84sleep20-21swapper/208:59:362
19121211719,76sleep00-21swapper/012:04:380
26743211519,38sleep20-21swapper/210:49:182
7132211410,16sleep30-21swapper/308:04:223
23756211447,50sleep10-21swapper/110:39:241
23756211447,50sleep10-21swapper/110:39:241
10870211447,26sleep110915-21ntp_offset08:14:381
12047211321,68sleep20-21swapper/208:19:192
2471210818,73sleep22549-21/usr/sbin/munin09:29:362
23693210814,78sleep00-21swapper/012:19:310
22678210811,15sleep00-21swapper/010:34:380
21483210836,46sleep321498-21modprobe07:09:063
21483210836,46sleep321498-21modprobe07:09:063
9510210647,19sleep10-21swapper/111:34:351
21244210615,75sleep321245-21sshd08:48:153
1626921057,66sleep10-21swapper/108:32:261
24379210426,26sleep10-21swapper/107:14:371
26122210372,14sleep10-21swapper/107:19:381
23495210313,74sleep023587-21perl08:54:300
31572210245,45sleep131583-21ls07:39:081
21206210243,43sleep121332-21sed10:29:391
21206210243,43sleep121332-21sed10:29:391
16237210117,70sleep316278-21ntp_kernel_pll_10:14:313
2773629915,71sleep027831-21latency_hist12:34:090
2661429915,52sleep20-21swapper/209:04:252
1644929915,65sleep216556-21latency_hist08:34:092
2749729725,52sleep227532-21latency07:25:372
2749729725,52sleep227532-21latency07:25:372
3181429614,66sleep331897-21tune2fs11:04:333
3231629538,16sleep10-21swapper/107:39:371
2963929512,25sleep20-21swapper/209:14:222
2917529539,44sleep129261-21mailstats07:29:351
2917529539,44sleep129261-21mailstats07:29:351
2657929516,24sleep10-21swapper/112:29:251
2518229560,18sleep10-21swapper/108:59:271
2011029539,14sleep10-21swapper/108:45:061
1613429435,12sleep10-21swapper/110:14:291
526029336,14sleep10-21swapper/107:59:071
2395329314,57sleep324071-21timerwakeupswit12:19:383
1756929311,57sleep20-21swapper/210:19:262
759429236,15sleep10-21swapper/111:29:381
629929130,47sleep16356-21sed09:44:081
375129116,29sleep30-21swapper/307:54:083
2014329110,15sleep00-21swapper/008:45:090
1173029122,46sleep211753-21apt08:19:072
103729138,12sleep10-21swapper/111:09:281
2613529014,62sleep21-21systemd12:29:062
2226629012,17sleep20-21swapper/207:09:342
1213529027,25sleep00-21swapper/008:19:250
1193029032,44sleep112001-21latency_hist11:44:081
806128935,13sleep10-21swapper/109:49:171
15442898,16sleep10-21swapper/107:44:371
509028814,56sleep35197-21fschecks_time09:39:263
2534228815,56sleep025369-21awk08:59:300
1402328818,51sleep314041-21awk10:09:063
1402328818,51sleep314041-21awk10:09:063
1070828725,40sleep010773-21fschecks_time11:39:260
944428620,27sleep20-21swapper/211:34:322
721428610,62sleep27218-21entropy08:04:252
1965728610,21sleep1289-21dbus-daemon12:09:071
193872868,18sleep00-21swapper/008:42:220
1271028623,48sleep10-21swapper/111:44:361
748928527,38sleep27483-21apt-get11:29:192
2647628524,42sleep226487-21modprobe10:49:062
250392859,55sleep10-21swapper/112:24:251
2376628521,42sleep223856-21grep10:39:252
2376628521,42sleep223856-21grep10:39:252
222812859,62sleep122407-21cat08:49:381
843928312,57sleep18512-21latency_hist08:09:081
2185728316,51sleep321966-21latency_hist10:34:093
2185728316,51sleep321966-21latency_hist10:34:093
835328224,30sleep00-21swapper/009:49:280
3156128227,35sleep231553-21apt-get11:04:242
2291328212,53sleep323004-21apt-config08:54:083
2048628231,26sleep20-21swapper/207:04:292
2048628231,26sleep20-21swapper/207:04:292
1485228221,49sleep10-21swapper/108:29:071
61628124,30sleep30-21swapper/307:44:073
350428128,24sleep00-21swapper/009:34:260
3149228120,40sleep331555-21sendmail-msp07:39:063
2699928116,47sleep327075-21wc12:29:363
1692728122,35sleep30-21swapper/311:59:243
1017428127,24sleep20-21swapper/208:14:102
784928024,13sleep10-21swapper/109:49:091
55472805,64sleep10-21swapper/107:59:151
1419828012,53sleep314275-21grep08:24:373
2930127924,31sleep20-21swapper/212:39:072
1938827925,30sleep30-21swapper/310:24:323
1225627916,37sleep312319-21fschecks_time11:44:263
2869027816,45sleep028753-21/usr/sbin/munin07:29:220
2869027816,45sleep028753-21/usr/sbin/munin07:29:220
277742787,11sleep00-21swapper/010:50:530
1489627827,30sleep20-21swapper/208:29:082
68442775,61sleep00-21swapper/011:27:350
68442775,61sleep00-21swapper/011:27:350
310262777,14sleep10-21swapper/109:19:091
3050627715,29sleep20-21swapper/207:34:292
2259927711,48sleep322676-21awk10:34:373
1553427710,17sleep00-21swapper/008:29:300
103722770,5sleep20-21swapper/209:54:382
689427620,35sleep36865-21latency_hist08:04:103
400427615,44sleep24071-21df07:54:242
82727527,21sleep30-21swapper/309:24:373
2629127522,25sleep30-21swapper/309:04:093
1314927521,27sleep20-21swapper/210:04:322
1314927521,27sleep20-21swapper/210:04:322
396227421,29sleep20-21swapper/209:34:372
2198027421,31sleep20-21swapper/208:49:312
2701127312,26sleep00-21swapper/009:04:350
2071627325,26sleep00-21swapper/012:09:380
2142027215,32sleep30-21swapper/312:14:093
986827117,30sleep00-21swapper/008:13:150
318327122,21sleep30-21swapper/309:34:093
2899427115,25sleep00-21swapper/010:54:410
19072716,13sleep00-21swapper/009:29:220
1243927113,44sleep212499-21latency11:44:292
1140327111,46sleep211461-21if_eth009:59:282
1140327111,46sleep211461-21if_eth009:59:282
583727017,29sleep25849-21/usr/sbin/munin07:59:282
205172707,16sleep00-21swapper/010:29:170
205172707,16sleep00-21swapper/010:29:170
827026925,20sleep00-21swapper/011:30:140
711626917,36sleep27195-21sort09:44:382
2228026913,39sleep322287-21/usr/sbin/munin12:15:353
1721226920,32sleep017241-21/usr/sbin/munin11:59:290
170112696,10sleep20-21swapper/210:17:372
1477326927,26sleep20-21swapper/210:09:332
1477326927,26sleep20-21swapper/210:09:332
1863526820,31sleep318681-21/usr/sbin/munin08:39:313
1484726812,29sleep00-21swapper/010:09:350
1484726812,29sleep00-21swapper/010:09:350
416226718,31sleep20-21swapper/211:19:222
3253326714,37sleep032680-21sed09:24:090
2674226717,24sleep00-21swapper/007:24:090
1408826717,23sleep00-21swapper/011:49:320
107022675,11sleep30-21swapper/311:39:253
83482666,10sleep00-21swapper/008:08:350
2300626617,19sleep00-21swapper/008:54:090
2187226616,23sleep00-21swapper/012:15:240
1894226615,21sleep20-21swapper/212:04:362
1874726618,26sleep00-21swapper/010:24:090
815226520,24sleep30-21swapper/311:30:133
701726517,27sleep37065-21munin-run11:29:083
701726517,27sleep37065-21munin-run11:29:083
408026518,23sleep00-21swapper/007:54:260
3090426516,25sleep30-21swapper/309:19:073
3055026521,21sleep00-21swapper/009:14:400
285712655,12sleep30-21swapper/309:09:353
691026416,18sleep00-21swapper/009:44:320
233502648,14sleep00-21swapper/007:14:120
1775626411,21sleep20-21swapper/211:59:392
1097226422,25sleep30-21swapper/309:59:083
1092526420,21sleep30-21swapper/308:14:403
993526312,31sleep30-21swapper/308:14:073
261792637,39sleep00-21swapper/010:45:160
2311526314,36sleep023127-21basename07:14:080
210042636,13sleep20-21swapper/210:29:362
210042636,13sleep20-21swapper/210:29:362
3190226212,19sleep00-21swapper/011:04:340
3169526216,30sleep231553-21apt-get11:04:282
196542627,16sleep30-21swapper/312:09:053
904126118,26sleep39036-21apt-get11:34:213
684126116,25sleep00-21swapper/008:04:070
53392618,34sleep30-21swapper/307:59:093
1696326114,31sleep017042-21cat08:34:310
696526015,27sleep27016-21systemd-run11:29:072
696526015,27sleep27016-21systemd-run11:29:072
3013026021,22sleep330127-21apt-get10:59:203
136726013,29sleep20-21swapper/207:44:342
855425918,24sleep28614-21/usr/sbin/munin09:49:352
323502596,10sleep20-21swapper/209:22:252
2131125912,31sleep221312-21cron08:49:072
18182597,13sleep00-21swapper/007:44:400
158925918,27sleep21628-21munin-run09:29:082
1252925917,21sleep30-21swapper/310:04:093
1252925917,21sleep30-21swapper/310:04:093
1217325912,20sleep00-21swapper/011:44:230
100632597,18sleep00-21swapper/011:36:030
507825821,20sleep20-21swapper/209:39:252
75625713,29sleep3930-21apt-config11:09:093
3041325717,20sleep00-21swapper/010:59:370
2823025719,22sleep328243-21irqstats12:34:283
2653125716,20sleep30-21swapper/309:04:223
2495425715,26sleep324958-21diskstats12:24:233
1706325713,28sleep217108-21/usr/sbin/munin08:34:332
1440025722,18sleep30-21swapper/311:49:393
2542625610,30sleep325435-21diskstats07:19:243
2473625616,18sleep30-21swapper/308:59:093
237462568,17sleep00-21swapper/010:39:230
237462568,17sleep00-21swapper/010:39:230
1826725617,25sleep20-21swapper/208:39:212
1526825614,25sleep015262-21apt-get11:54:200
547925516,20sleep00-21swapper/009:39:350
2844925514,18sleep30-21swapper/307:29:093
2844925514,18sleep30-21swapper/307:29:093
2797425513,26sleep027968-21apt-get12:34:210
2583725515,18sleep20-21swapper/207:19:352
205925511,31sleep3367-21snmpd07:47:463
1468125516,19sleep30-21swapper/310:09:303
1468125516,19sleep30-21swapper/310:09:303
1124925513,18sleep00-21swapper/008:14:450
387025413,25sleep33876-21ntp_states09:34:343
25422546,10sleep30-21swapper/311:14:143
2133325412,28sleep221419-21apt-config12:14:082
1586225412,25sleep015856-21apt-get10:14:190
699625313,24sleep37002-21ntp_states09:44:353
2189825311,26sleep321976-21grep07:09:263
130412536,13sleep00-21swapper/010:04:300
130412536,13sleep00-21swapper/010:04:300
471225212,22sleep00-21swapper/011:19:360
2676525214,22sleep326762-21apt-get10:49:193
257525210,15sleep20-21swapper/207:49:272
257225214,22sleep02567-21apt-get11:14:190
245325213,23sleep02448-21apt-get07:49:210
2385225212,20sleep20-21swapper/207:14:262
1772325216,17sleep30-21swapper/308:36:163
1335725213,15sleep30-21swapper/308:24:083
3122025112,23sleep331216-21apt-get09:19:193
307332519,16sleep00-21swapper/007:34:350
1617625112,24sleep316177-21sshd08:31:153
1129625111,26sleep011187-21/usr/sbin/munin09:59:260
1129625111,26sleep011187-21/usr/sbin/munin09:59:260
3000525011,23sleep230149-21latency_hist07:34:092
3000525011,23sleep230149-21latency_hist07:34:092
2966725012,22sleep329756-21apt-config10:59:083
2816725013,23sleep028174-21diskstats09:09:230
2529825013,21sleep325305-21diskstats10:44:263
2343425010,26sleep223543-21cut10:39:082
1380125010,15sleep20-21swapper/211:49:262
1368325013,17sleep00-21swapper/008:24:260
989424912,23sleep09909-21irqstats09:54:290
545424913,21sleep35466-21munin-run11:24:073
93224811,22sleep0512-21apt-get11:09:100
1945624811,19sleep20-21swapper/210:24:352
1658324811,21sleep216604-21munin-run11:59:072
1547824812,20sleep315535-21irqstats11:54:273
3188924611,18sleep231787-21/usr/sbin/munin07:39:262
3180524611,19sleep031807-21apt-get07:39:200
1907024510,17sleep00-21swapper/010:24:250
179332417,14sleep30-21swapper/310:19:353
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional