You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-09 - 22:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Tue Jun 09, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
69822153138,1479sleep06983-21ntp_states12:28:180
69822153138,1479sleep06983-21ntp_states12:28:180
1571329950,121sleep011-21ksoftirqd/009:33:130
1571329950,121sleep011-21ksoftirqd/009:33:130
4407282918,794sleep34416-21/usr/sbin/munin08:58:093
29624273631,690sleep029626-21apt-get11:58:050
2377227240,20sleep00-21swapper/009:58:300
4336260913,578sleep14326-21apt-get08:58:071
306925736,555sleep03104-21ntpq10:33:200
612824128,388sleep017867-1kworker/0:0H09:03:070
19511239310,368sleep019547-21modprobe11:27:520
13492238219,347sleep313493-21sshd09:26:043
3467235917,291sleep03581-21grep08:53:260
4983225610,195sleep05050-21sort12:23:090
21995225140,197sleep022041-21ntpq11:33:170
20207224512,189sleep024024-21kworker/u8:409:47:570
25407223934,138sleep025547-21unixbench_multi11:43:270
1495822260,28sleep114953-21apt-get07:48:061
13685222379,51sleep00-21swapper/009:27:540
8249221556,88sleep00-21swapper/007:27:540
24417221556,55sleep00-21swapper/008:18:120
24417221556,55sleep00-21swapper/008:18:120
1972322080,16sleep20-21swapper/208:03:112
9869220552,133sleep00-21swapper/009:13:250
2659722040,40sleep01601-21cyclictest08:23:240
167022007,44sleep00-21swapper/008:48:210
167022007,44sleep00-21swapper/008:48:210
28150219510,144sleep028286-21unixbench_singl08:28:260
1317621947,149sleep013223-21grep07:43:090
1865721869,158sleep00-21swapper/009:42:560
3253821816,13sleep00-21swapper/008:43:200
2929821808,23sleep00-21swapper/010:13:250
2929821808,23sleep00-21swapper/010:13:250
2314321788,154sleep00-21swapper/011:38:020
2314321788,154sleep00-21swapper/011:38:020
2647621777,159sleep0363-21snmpd10:06:510
2647621777,159sleep0363-21snmpd10:06:510
8672217147,83sleep08816-21latency_hist10:52:550
3099021686,43sleep00-21swapper/010:19:350
105552168105,18sleep00-21swapper/010:58:120
159821668,141sleep01671-21latency_hist12:12:550
1826321649,143sleep018311-21basename07:58:130
4814216022,105sleep30-21swapper/310:38:243
23240216046,22sleep00-21swapper/009:57:540
6530215919,123sleep16536-21cpu12:28:071
6530215919,123sleep16536-21cpu12:28:071
5535215818,124sleep25601-21cat12:23:232
9502215614,127sleep39516-21irqstats09:13:163
18357215624,112sleep218420-21latency07:58:152
1649215616,39sleep00-21swapper/007:07:550
1928215076,23sleep00-21swapper/012:13:120
2673521470,32sleep126739-21unixbench_singl08:23:251
1229621447,83sleep00-21swapper/011:03:170
1229621447,83sleep00-21swapper/011:03:170
2250121438,118sleep022573-21latency_hist08:12:550
5669214012,54sleep05767-21latency_hist09:02:550
447821399,80sleep04588-21wc07:13:250
12424213717,52sleep312002-21/usr/sbin/munin11:03:203
12424213717,52sleep312002-21/usr/sbin/munin11:03:203
26651213211,108sleep226691-21ntpq11:48:182
7787213014,94sleep20-21swapper/212:32:522
7787213014,94sleep20-21swapper/212:32:522
1737221279,106sleep017498-21sed11:18:270
32719212421,79sleep332738-21apt-get12:08:063
19536212325,75sleep119677-21unixbench_multi09:43:251
23122212210,98sleep023167-21ntp_kernel_pll_08:13:190
1826212244,30sleep01941-21timerwakeupswit10:28:480
22931212116,21sleep10-21swapper/111:37:551
826821209,53sleep00-21swapper/009:08:240
826821209,53sleep00-21swapper/009:08:240
22271211915,87sleep322397-21/usr/sbin/munin11:33:253
4007211847,51sleep00-21swapper/012:18:250
29582211815,55sleep00-21swapper/008:33:220
1820211864,14sleep00-21swapper/007:08:040
28220211628,42sleep028263-21awk11:53:210
14666211522,79sleep114807-21sed09:28:241
12240211520,49sleep012360-21cat07:38:250
12029211515,20sleep20-21swapper/211:03:112
12029211515,20sleep20-21swapper/211:03:112
10389211521,34sleep00-21swapper/007:33:160
30231211416,77sleep230263-21sh08:37:532
2933211417,83sleep12946-21kernelversion10:33:171
15721211411,87sleep215707-21apt-get09:33:172
15721211411,87sleep215707-21apt-get09:33:172
6281211117,44sleep00-21swapper/010:43:220
36121116,11sleep00-21swapper/010:25:080
2376210921,67sleep12377-21cron10:32:531
2004421079,18sleep00-21swapper/011:28:080
17238210719,63sleep217381-21grep07:53:262
17238210719,63sleep217381-21grep07:53:262
4345210616,72sleep14219-21/usr/sbin/munin10:38:141
22449210624,60sleep222467-21ls08:12:532
8054210514,77sleep28055-21/usr/sbin/munin12:33:042
8054210514,77sleep28055-21/usr/sbin/munin12:33:042
4522210312,66sleep00-21swapper/010:38:170
1003821035,11sleep00-21swapper/010:56:260
31130210216,33sleep00-21swapper/008:38:230
2810621029,45sleep028139-21munin-run10:12:530
17368210113,49sleep00-21swapper/009:38:110
14497210120,62sleep114591-21gpgv07:47:551
25627210018,67sleep225647-21ls08:22:542
25627210018,67sleep225647-21ls08:22:542
11809210020,63sleep211867-21if_err_eth007:38:142
555629913,72sleep25683-21latency_hist10:42:552
364129912,72sleep33697-21/usr/sbin/munin12:18:153
2655629914,67sleep126568-21kworker/u8:110:07:531
2655629914,67sleep126568-21kworker/u8:110:07:531
3211129819,58sleep332127-21uname10:24:103
91532975,58sleep00-21swapper/012:35:290
91532975,58sleep00-21swapper/012:35:290
2742429628,51sleep227503-21switchtime10:08:222
2319029628,44sleep323210-21apt-get09:57:533
340629519,50sleep13425-21apt-get12:18:051
203212958,41sleep00-21swapper/008:03:260
2014329414,62sleep320205-21sed09:47:563
2310729214,59sleep323120-21/usr/sbin/munin08:13:183
2454029125,34sleep10-21swapper/111:42:561
1601729115,56sleep115689-21/usr/sbin/munin09:34:081
1601729115,56sleep115689-21/usr/sbin/munin09:34:081
705029018,54sleep27078-21idleruntime-cro10:47:522
3271429022,52sleep132715-21http07:03:081
275042905,53sleep00-21swapper/011:52:360
400428914,54sleep34039-21latency_hist10:37:553
160672896,17sleep00-21swapper/011:13:510
154072896,71sleep015446-21ntpq07:49:160
1004728910,62sleep110117-21df07:33:081
315928728,29sleep30-21swapper/307:12:553
1366728720,47sleep213757-21/usr/sbin/munin11:08:112
1366728720,47sleep213757-21/usr/sbin/munin11:08:112
3056528615,57sleep330648-21switchtime10:18:253
589128413,52sleep35985-21fschecks_time10:43:123
1841828410,57sleep118473-21idleruntime11:23:131
41528319,48sleep2485-21cat07:03:132
2839728217,47sleep128486-21switchtime11:53:261
621028121,42sleep00-21swapper/007:18:260
621028121,42sleep00-21swapper/007:18:260
421728120,42sleep23786-21/usr/sbin/munin07:13:192
2316528115,45sleep20-21swapper/211:38:042
2316528115,45sleep20-21swapper/211:38:042
1028828111,57sleep110290-21sshd09:15:031
2590228010,52sleep20-21swapper/210:03:262
2590228010,52sleep20-21swapper/210:03:262
3127827913,52sleep031320-21fschecks_count12:03:100
172827925,27sleep30-21swapper/310:28:473
2788427811,14sleep30-21swapper/310:10:013
95627718,38sleep3927-21latency_hist08:47:553
900227726,25sleep30-21swapper/309:12:553
900227726,25sleep30-21swapper/309:12:553
261327711,49sleep12741-21/usr/sbin/munin12:13:251
1344627714,47sleep313533-21cat07:43:163
132512778,49sleep20-21swapper/207:43:112
130382760,7sleep30-21swapper/307:42:553
2814727521,40sleep128156-21unin-run10:12:541
2490327515,44sleep324980-21switchtime08:18:233
2490327515,44sleep324980-21switchtime08:18:233
2137427514,44sleep121432-21/usr/sbin/munin08:08:111
1975927510,55sleep28998-21kworker/2:209:43:342
1800527526,22sleep30-21swapper/311:22:543
1180027518,10sleep00-21swapper/009:19:100
849927417,26sleep20-21swapper/207:28:092
548727417,21sleep30-21swapper/307:18:123
548727417,21sleep30-21swapper/307:18:123
257727422,32sleep32710-21cut08:52:563
1958927426,22sleep20-21swapper/211:27:542
139272748,48sleep00-21swapper/009:28:070
94482736,55sleep00-21swapper/007:28:560
820427322,35sleep28230-21latency_hist07:27:522
692127313,23sleep10-21swapper/107:23:031
2447027314,31sleep10-21swapper/108:18:131
2447027314,31sleep10-21swapper/108:18:131
147127335,23sleep30-21swapper/308:48:163
76482726,12sleep30-21swapper/312:31:043
76482726,12sleep30-21swapper/312:31:043
560427211,44sleep35728-21timerwakeupswit12:23:243
1200127216,37sleep1363-21snmpd11:03:051
1200127216,37sleep1363-21snmpd11:03:051
559927117,33sleep2225-21systemd-journal09:02:532
3222727110,43sleep232312-21fw_packets08:43:142
2770127120,23sleep30-21swapper/308:28:173
2649127115,39sleep326595-21wc08:23:243
1059627125,29sleep20-21swapper/209:17:552
3016527015,36sleep130223-21/usr/sbin/munin11:58:551
2781927016,38sleep127855-21tr08:28:191
240862700,8sleep20-21swapper/209:58:522
2329627023,25sleep10-21swapper/109:57:551
135732708,46sleep00-21swapper/011:08:050
135732708,46sleep00-21swapper/011:08:050
3121726919,27sleep10-21swapper/108:38:241
2976626921,30sleep329812-21fschecks_count11:58:453
2371626917,32sleep123210-21apt-get09:58:031
1751726919,39sleep00-21swapper/007:54:410
1751726919,39sleep00-21swapper/007:54:410
2046926823,23sleep20-21swapper/209:48:112
1817426813,26sleep30-21swapper/307:58:123
1536526812,41sleep115394-21irqstats11:13:151
1365926816,30sleep113548-21/usr/sbin/munin11:08:101
1365926816,30sleep113548-21/usr/sbin/munin11:08:101
1363326821,22sleep20-21swapper/209:27:522
57626711,39sleep2637-21systemd-run10:27:532
409226718,22sleep10-21swapper/108:57:551
3274526711,28sleep10-21swapper/108:43:251
323252677,10sleep20-21swapper/212:06:152
3095126717,31sleep30-21swapper/308:38:183
3077526716,33sleep130891-21/usr/sbin/munin10:18:271
2943526717,20sleep10-21swapper/108:33:191
2905526715,27sleep20-21swapper/210:13:232
2905526715,27sleep20-21swapper/210:13:232
2848826716,20sleep20-21swapper/211:53:272
1973526718,28sleep319929-21apt-config11:27:553
81026628,22sleep30-21swapper/307:03:233
742326611,39sleep17429-21diskstats10:48:111
2865926619,31sleep128671-21/usr/sbin/munin10:13:141
2865926619,31sleep128671-21/usr/sbin/munin10:13:141
2048226626,19sleep30-21swapper/309:48:123
1570126518,23sleep20-21swapper/211:13:242
1109226520,22sleep20-21swapper/209:18:152
884926414,26sleep30-21swapper/312:33:263
884926414,26sleep30-21swapper/312:33:263
734526414,29sleep27339-21apt-get10:48:082
267026418,29sleep22664-21apt-get10:33:102
251326418,31sleep22569-21sshd08:52:552
251326418,31sleep22569-21sshd08:52:552
1590026419,30sleep315960-21grep09:34:063
1590026419,30sleep315960-21grep09:34:063
1509826416,28sleep315117-21apt-get11:13:053
1447426416,27sleep30-21swapper/309:28:223
1406026423,24sleep30-21swapper/311:08:213
1406026423,24sleep30-21swapper/311:08:213
1073726412,39sleep310797-21latency10:58:153
844026321,20sleep20-21swapper/209:08:262
844026321,20sleep20-21swapper/209:08:262
2752726316,32sleep127571-21systemd-run11:52:531
2282826316,24sleep20-21swapper/208:13:132
1859126327,21sleep30-21swapper/309:42:553
983326221,20sleep10-21swapper/107:32:561
734626211,26sleep30-21swapper/310:48:093
2096026221,22sleep10-21swapper/109:48:221
168526212,21sleep10-21swapper/108:48:221
168526212,21sleep10-21swapper/108:48:221
1398026215,31sleep214004-21df_inode09:28:092
65062618,16sleep20-21swapper/207:20:582
65062618,16sleep20-21swapper/207:20:582
613426118,26sleep26120-21apt-get09:03:092
1728726117,28sleep117280-21apt-get09:38:081
1651126120,25sleep116504-21apt-get07:53:091
1651126120,25sleep116504-21apt-get07:53:091
886526011,19sleep10-21swapper/110:53:041
2602726017,18sleep30-21swapper/311:47:543
257426015,29sleep12715-21sed07:08:251
230012608,43sleep20-21swapper/209:55:262
302525918,23sleep30-21swapper/308:53:163
2722625915,19sleep20-21swapper/208:27:552
2168125923,20sleep20-21swapper/211:33:112
112192599,42sleep30-21swapper/307:36:503
1062725916,23sleep10-21swapper/110:58:131
900025815,27sleep39079-21systemctl07:28:223
3092125813,21sleep20-21swapper/208:38:172
2876525815,23sleep30-21swapper/310:13:163
2876525815,23sleep30-21swapper/310:13:163
2742625817,25sleep227420-21apt-get08:28:072
1803225821,22sleep20-21swapper/209:38:262
610225715,27sleep16120-21apt-get09:03:041
519625719,16sleep30-21swapper/307:17:563
2480625719,23sleep20-21swapper/211:43:132
2385925719,19sleep30-21swapper/309:58:423
191625722,15sleep20-21swapper/207:08:102
1749925713,26sleep117614-21users11:18:281
1656025711,28sleep316584-21df07:53:113
1656025711,28sleep316584-21df07:53:113
1626825717,24sleep30-21swapper/307:52:543
121325714,26sleep21227-21diskstats10:28:342
940325613,18sleep30-21swapper/310:53:223
936125620,15sleep20-21swapper/210:53:212
2022325618,19sleep30-21swapper/308:03:253
184125615,24sleep31837-21apt-get12:13:063
115362565,40sleep20-21swapper/211:00:202
807925512,26sleep18074-21apt-get12:33:081
807925512,26sleep18074-21apt-get12:33:081
3026425516,25sleep330310-21sshd08:37:543
1208425512,17sleep30-21swapper/309:22:533
100425513,26sleep129672-21kworker/u8:310:28:281
357825415,17sleep20-21swapper/212:18:142
275325413,24sleep22747-21apt-get08:53:072
2446825418,19sleep324478-21unin-run11:42:543
2446825418,19sleep324478-21unin-run11:42:543
2000925413,24sleep219526-21apt-get11:28:022
769825313,24sleep17691-21apt-get09:08:091
769825313,24sleep17691-21apt-get09:08:091
709625319,21sleep30-21swapper/307:23:133
612625312,24sleep35614-21apt-get09:03:063
52725313,24sleep1556-21irqstats12:08:161
427825311,27sleep14311-21ps07:13:221
2782425310,28sleep327894-21df11:53:113
2247025318,21sleep322499-21munin-run08:12:543
2003925314,17sleep10-21swapper/111:28:051
1964325314,23sleep119640-21apt-get08:03:061
90025214,22sleep20-21swapper/212:08:252
629025212,16sleep10-21swapper/112:27:551
3118625213,23sleep131183-21apt-get12:03:031
1939725213,17sleep10-21swapper/108:02:541
179182526,34sleep20-21swapper/211:22:172
1870225112,24sleep318711-21ntpq11:23:193
136112517,33sleep10-21swapper/109:27:351
592625011,16sleep10-21swapper/107:18:221
592625011,16sleep10-21swapper/107:18:221
2530425011,15sleep30-21swapper/310:03:103
2530425011,15sleep30-21swapper/310:03:103
2316425014,22sleep123161-21apt-get11:38:031
2316425014,22sleep123161-21apt-get11:38:031
2282125011,22sleep122827-21diskstats08:13:121
1493225017,16sleep20-21swapper/207:48:042
605424913,21sleep16067-21irqstats10:43:151
1162624915,17sleep10-21swapper/107:38:091
1082924913,19sleep110826-21apt-get09:18:061
2651524811,21sleep326539-21irqstats11:48:153
1004224811,20sleep210038-21apt-get07:33:052
3127224711,20sleep331276-21diskstats12:03:093
191224711,22sleep21919-21df_inode12:13:092
3020124611,21sleep230258-21irqstats10:18:162
296322467,19sleep20-21swapper/211:58:072
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional