You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-13 - 21:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sat Jun 13, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20149296438,9623sleep217090-21kworker/2:009:53:192
31360246346,4616sleep231351-21kworker/2:210:32:102
1272263418,600sleep11280-21/usr/sbin/munin10:38:061
1272263418,600sleep11280-21/usr/sbin/munin10:38:061
18988252311,493sleep018962-21latency_hist09:52:410
11915231217,196sleep211927-21apt-key11:12:392
11915231217,196sleep211927-21apt-key11:12:392
5525230711,283sleep21-21systemd07:27:402
4837225166,129sleep24858-21fschecks_count09:08:012
31557224168,161sleep231636-21timerandwakeup08:48:072
20762224017,206sleep020830-21/usr/sbin/munin11:38:090
5349222065,16sleep20-21swapper/212:33:062
31969221118,171sleep032008-21latency10:33:050
28086220941,25sleep20-21swapper/212:02:582
3100220234,35sleep23210-21latency_hist12:27:412
3100220234,35sleep23210-21latency_hist12:27:412
8704219612,115sleep28858-21date07:37:412
2553821968,176sleep225556-21fschecks_count10:12:572
921219016,138sleep20-21swapper/207:12:462
21716218416,149sleep20-21swapper/208:17:592
13116217711,144sleep20-21swapper/209:33:092
13116217711,144sleep20-21swapper/209:33:092
9510216912,141sleep29588-21ls07:38:102
8242216410,139sleep28255-21needreboot09:18:032
2652121609,135sleep226563-21fschecks_count11:57:572
30799215918,120sleep20-21swapper/212:12:392
19899215513,99sleep219902-21cron11:37:382
19899215513,99sleep219902-21cron11:37:382
1399921528,15sleep20-21swapper/211:17:522
20045214930,28sleep20-21swapper/208:12:542
11160214811,30sleep20-21swapper/209:28:002
15521214417,83sleep20-21swapper/207:58:042
9045214110,89sleep29064-21apt-get11:02:502
24006214014,25sleep20-21swapper/211:48:082
11086214039,27sleep20-21swapper/207:43:102
15973213615,35sleep216000-21fschecks_count09:42:582
1904121338,69sleep20-21swapper/208:09:062
16978213327,18sleep20-21swapper/208:03:012
5555213212,99sleep05647-21fschecks_time10:52:590
29901212713,98sleep130054-21cut10:27:411
11300212717,92sleep111427-21sshd11:08:101
11300212717,92sleep111427-21sshd11:08:101
490212562,42sleep2508-21kernelversion12:18:002
31553212312,96sleep031561-21ntp_states12:13:050
30651212362,41sleep230683-21idleruntime-cro08:47:382
18522212021,77sleep00-21swapper/008:08:110
1500212060,45sleep21638-21unixbench_multi10:38:112
1500212060,45sleep21638-21unixbench_multi10:38:112
25127211913,86sleep225214-21latency11:53:002
22490211952,46sleep20-21swapper/210:03:002
22490211952,46sleep20-21swapper/210:03:002
456021185,102sleep226098-21kworker/2:112:32:172
456021185,102sleep226098-21kworker/2:112:32:172
2444121188,94sleep230971-1kworker/2:1H08:27:402
21494211822,77sleep121652-21grep09:58:111
21494211822,77sleep121652-21grep09:58:111
13747211829,16sleep20-21swapper/211:17:392
20831211715,85sleep120957-21timerwakeupswit11:38:101
7322211510,20sleep20-21swapper/210:58:262
30126211524,65sleep030235-21fschecks_time07:02:590
30126211524,65sleep030235-21fschecks_time07:02:590
24146211514,82sleep20-21swapper/210:08:002
6880211416,82sleep17008-21cat10:57:401
381021138,88sleep03889-21grep12:28:070
381021138,88sleep03889-21grep12:28:070
10950211343,54sleep010968-21/usr/sbin/munin09:27:500
9887211114,79sleep19900-21/usr/sbin/munin09:23:061
9188211118,78sleep09236-21latency_hist09:22:410
5436211158,41sleep25440-21find10:52:492
301321118,87sleep20-21swapper/207:18:032
20303211114,76sleep20-21swapper/211:37:582
16859211114,77sleep216864-21latency_hist11:27:412
3169421099,84sleep331772-21find12:13:083
31424210958,36sleep231318-21latency_hist07:07:412
15597210912,80sleep31-21systemd09:42:383
24722210752,43sleep224718-21apt-get08:27:522
18354210718,72sleep318393-21modprobe11:32:383
17069210715,75sleep20-21swapper/211:27:492
26876210410,17sleep20-21swapper/208:33:092
8940210310,13sleep00-21swapper/007:37:550
2915721037,81sleep229298-21sed08:42:422
10649210358,12sleep20-21swapper/209:27:112
31362210132,31sleep10-21swapper/107:07:401
31362210132,31sleep10-21swapper/107:07:401
13245210154,13sleep20-21swapper/207:52:062
6793210045,21sleep20-21swapper/207:29:052
456210049,12sleep20-21swapper/210:37:102
16491210018,65sleep316642-21latency_hist08:02:413
398929945,14sleep20-21swapper/210:47:562
2244829922,50sleep122487-21fschecks_count10:02:591
2244829922,50sleep122487-21fschecks_count10:02:591
146912999,70sleep20-21swapper/209:39:222
62672988,16sleep20-21swapper/209:12:502
297902985,61sleep20-21swapper/210:26:572
90222979,12sleep30-21swapper/307:37:583
733629713,68sleep07356-21apt-get07:32:510
2672229722,50sleep3225-21systemd-journal10:17:383
212712979,77sleep221336-21ps09:58:082
212712979,77sleep221336-21ps09:58:082
35272966,77sleep20-21swapper/210:46:232
2994329615,64sleep130016-21switchtime08:43:091
3108629410,68sleep20-21swapper/212:12:512
176642948,17sleep20-21swapper/209:47:502
1740229312,64sleep117582-21apt-config09:47:411
2215329211,64sleep322303-21cut10:02:403
2215329211,64sleep322303-21cut10:02:403
100229212,57sleep31038-21idleruntime10:38:003
100229212,57sleep31038-21idleruntime10:38:003
671929124,45sleep36765-21systemctl10:57:383
3040429110,65sleep030477-21sed10:28:000
2566129114,60sleep025714-21/usr/sbin/munin10:12:590
948628951,19sleep00-21swapper/011:03:120
728328920,37sleep17269-21apt-get10:57:561
255982890,13sleep30-21swapper/311:53:093
1588728927,37sleep115881-21apt-get09:42:531
428028814,57sleep04309-21/usr/sbin/munin10:48:020
290522889,13sleep00-21swapper/010:23:080
230632887,14sleep20-21swapper/211:47:392
219728820,51sleep02304-21latency_hist10:42:410
219728820,51sleep02304-21latency_hist10:42:410
1445828813,60sleep014465-21sh11:18:060
332728720,18sleep30-21swapper/312:27:543
332728720,18sleep30-21swapper/312:27:543
1258528519,50sleep312591-21fschecks_count11:13:373
735828412,59sleep07408-21cpuspeed10:58:520
1679528319,46sleep316845-21fschecks_count08:02:583
26112826,10sleep20-21swapper/208:59:012
3016128121,40sleep030285-21cat12:08:090
223062810,7sleep00-21swapper/010:02:410
223062810,7sleep00-21swapper/010:02:410
725028010,53sleep30-21swapper/310:57:503
3219427920,44sleep132263-21switchtime07:08:091
2803427920,41sleep327827-21/usr/sbin/munin08:38:023
1600227920,38sleep316073-21fschecks_time09:42:593
2263127815,48sleep022662-21latency10:03:020
2263127815,48sleep022662-21latency10:03:020
214227813,30sleep30-21swapper/312:23:013
214227813,30sleep30-21swapper/312:23:013
105527818,28sleep00-21swapper/007:12:480
41852775,11sleep20-21swapper/209:04:222
279932776,10sleep20-21swapper/210:18:422
1998027721,32sleep10-21swapper/111:37:411
1998027721,32sleep10-21swapper/111:37:411
9692765,10sleep20-21swapper/208:53:162
411027613,51sleep14169-21if_eth010:47:591
345327612,51sleep03505-21latency09:03:030
2056527614,45sleep020700-21sed09:57:410
3236827522,25sleep10-21swapper/112:17:381
27802758,55sleep00-21swapper/012:24:200
27802758,55sleep00-21swapper/012:24:200
3201327420,22sleep00-21swapper/007:08:040
2922427418,15sleep30-21swapper/310:23:113
250727416,29sleep10-21swapper/107:17:411
1582027421,29sleep10-21swapper/111:23:041
147512747,56sleep20-21swapper/207:56:352
1264427421,28sleep00-21swapper/007:48:090
2838927320,32sleep00-21swapper/008:38:110
1970527318,38sleep119769-21/usr/sbin/munin09:53:061
130627322,36sleep11344-21awk08:57:391
2976427220,29sleep00-21swapper/008:43:040
2540327215,31sleep10-21swapper/111:53:081
1878227214,41sleep318835-21/usr/sbin/munin08:08:593
825727114,39sleep38302-21ntp_kernel_pll_09:18:043
50927116,39sleep3591-21/usr/sbin/munin12:18:013
496027113,43sleep15026-21fschecks_time12:32:571
319092719,14sleep30-21swapper/310:33:043
2326027118,33sleep10-21swapper/108:22:581
2183327110,15sleep30-21swapper/311:42:563
1751127116,32sleep016683-21/usr/sbin/munin08:03:110
1274927117,22sleep10-21swapper/107:48:101
600427020,33sleep36005-21cron09:12:393
309627020,20sleep00-21swapper/010:43:100
1949527025,27sleep319568-21sed09:53:003
2770226923,25sleep30-21swapper/312:02:383
14492699,45sleep01450-21sshd12:22:100
1339826916,34sleep012381-21/usr/sbin/munin11:14:060
106202697,12sleep00-21swapper/009:26:490
1024326920,29sleep10-21swapper/107:42:411
1197526813,41sleep312116-21apt-config11:12:403
1197526813,41sleep312116-21apt-config11:12:403
780826710,43sleep37819-21ntpq07:33:063
307826725,26sleep10-21swapper/112:27:401
307826725,26sleep10-21swapper/112:27:401
138626715,34sleep31475-21ls08:57:413
1273226715,36sleep112782-21if_eth009:33:001
1273226715,36sleep112782-21if_eth009:33:001
3074226620,29sleep330888-21unixbench_singl07:03:103
3074226620,29sleep330888-21unixbench_singl07:03:103
269682669,13sleep30-21swapper/308:33:103
2504126620,26sleep00-21swapper/011:52:580
199426613,13sleep10-21swapper/108:58:031
1426926616,32sleep314323-21sed11:18:013
666126515,35sleep36707-21ntp_kernel_pll_09:13:033
3216126518,25sleep10-21swapper/110:33:101
3084126518,23sleep30-21swapper/312:12:403
3020726514,36sleep330214-21df_inode10:27:553
2223726518,31sleep10-21swapper/108:18:101
1736026517,27sleep10-21swapper/111:28:001
1133326513,37sleep011386-21users07:43:120
491826415,23sleep10-21swapper/107:23:111
444926410,12sleep30-21swapper/307:23:003
2457126415,34sleep0225-21systemd-journal11:52:380
2293826411,22sleep00-21swapper/008:22:410
1357926416,26sleep00-21swapper/007:52:530
331726324,20sleep10-21swapper/107:18:101
316382636,12sleep30-21swapper/308:48:083
2439326312,34sleep124459-21cat10:08:071
2236826325,17sleep00-21swapper/011:43:090
2152526311,26sleep30-21swapper/311:42:403
906626212,27sleep30-21swapper/311:02:513
627626211,36sleep16313-21ps07:28:071
605526212,37sleep16131-21sed09:12:411
548026214,30sleep05584-21timerandwakeup12:33:080
2724926214,29sleep127302-21/usr/sbin/munin10:18:001
251082626,11sleep00-21swapper/010:11:540
2451626218,21sleep30-21swapper/310:08:093
1725326216,20sleep00-21swapper/009:47:400
1484026216,20sleep10-21swapper/109:39:261
851926115,22sleep10-21swapper/109:18:101
482326118,19sleep00-21swapper/009:07:590
2826326113,32sleep128357-21memory12:03:011
2707526112,27sleep027172-21users08:33:110
2604326114,19sleep10-21swapper/110:13:081
1107326113,25sleep10-21swapper/109:27:591
395926010,41sleep00-21swapper/007:22:290
3208226016,30sleep331597-21/usr/sbin/munin07:08:073
3059626016,28sleep130111-21/usr/sbin/munin10:28:051
2798926021,23sleep327986-21apt-get12:02:513
2771626017,22sleep30-21swapper/310:18:103
2276326011,26sleep30-21swapper/310:03:053
2276326011,26sleep30-21swapper/310:03:053
178302608,11sleep30-21swapper/309:48:033
1573326018,22sleep30-21swapper/311:23:013
3015025915,19sleep30-21swapper/308:43:113
2377625918,26sleep323841-21awk11:48:053
2200025919,25sleep30-21swapper/308:18:043
1911025915,26sleep319173-21/usr/sbin/munin11:33:073
1911025915,26sleep319173-21/usr/sbin/munin11:33:073
180122597,38sleep00-21swapper/011:28:150
135232595,12sleep00-21swapper/009:34:160
135232595,12sleep00-21swapper/009:34:160
3094525812,25sleep031009-21cpuspeed08:47:520
2930325812,23sleep00-21swapper/012:07:400
2846125820,20sleep10-21swapper/108:38:121
281522586,42sleep00-21swapper/010:20:460
2448625812,26sleep30-21swapper/308:27:413
223925816,18sleep30-21swapper/308:58:093
2176425815,26sleep121758-21apt-get11:42:541
1491125814,23sleep00-21swapper/009:39:290
1056925818,18sleep30-21swapper/307:42:593
1042725813,28sleep110570-21cat11:07:411
103225814,25sleep31053-21ntp_states07:12:473
581425714,26sleep35810-21apt-get07:27:533
2499725714,30sleep125071-21sed08:28:031
2330925715,27sleep1363-21snmpd11:47:511
1800825715,25sleep118038-21munin-run08:07:401
1576825713,28sleep115843-21wc07:58:111
1320325616,24sleep313331-21unixbench_multi09:33:103
1320325616,24sleep313331-21unixbench_multi09:33:103
1232425615,25sleep312474-21cat09:32:413
1073725619,21sleep30-21swapper/309:27:413
782825520,20sleep10-21swapper/107:33:081
637725516,24sleep16384-21forks09:12:571
27292555,16sleep00-21swapper/009:00:320
2670425517,21sleep126745-21ls11:58:001
1224525517,22sleep312298-21irqstats07:48:003
608725413,17sleep00-21swapper/007:28:020
3003225414,23sleep230028-21apt-get07:02:532
3003225414,23sleep230028-21apt-get07:02:532
273472547,36sleep10-21swapper/108:35:181
2643425414,24sleep026429-21apt-get11:57:510
2047525414,24sleep020498-21ntp_states08:13:040
1491725410,21sleep30-21swapper/307:57:413
148925415,25sleep11536-21systemd-run12:22:391
1101125415,23sleep111085-21sensors_temp07:43:091
1088625414,22sleep00-21swapper/011:08:010
1088625414,22sleep00-21swapper/011:08:010
449725313,24sleep14506-21munin-run09:07:401
325725316,20sleep30-21swapper/309:02:583
236625312,24sleep32362-21apt-get10:42:533
162652535,40sleep00-21swapper/008:00:400
1312525315,17sleep10-21swapper/111:14:031
629125215,21sleep06286-21apt-get09:12:520
418525213,22sleep34278-21memory10:48:013
358325216,22sleep13302-21/usr/sbin/munin12:28:001
358325216,22sleep13302-21/usr/sbin/munin12:28:001
322225215,22sleep13270-21users10:43:111
2488625220,18sleep30-21swapper/308:28:013
1552525211,15sleep00-21swapper/011:22:500
2469925116,18sleep00-21swapper/008:27:500
1845925113,24sleep117985-21apt-get08:07:441
426225013,24sleep04257-21apt-get07:22:530
1446625013,16sleep10-21swapper/111:18:071
1367525018,16sleep10-21swapper/107:52:581
48624913,21sleep0497-21ntp_states08:53:050
356224812,18sleep30-21swapper/307:18:123
3101324811,23sleep131020-21df_inode08:47:531
2361924812,21sleep023630-21ntp_states08:23:050
2031524710,21sleep120411-21/usr/sbin/munin08:13:011
2131024615,17sleep30-21swapper/308:17:393
2990024412,18sleep10-21swapper/112:08:021
62032428,17sleep30-21swapper/312:37:393
1997024210,19sleep019979-21/usr/sbin/munin11:37:400
1997024210,19sleep019979-21/usr/sbin/munin11:37:400
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional