You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-27 - 03:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sat Jun 27, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
242352112716,1098sleep324141-21kworker/3:021:42:203
21062282819,792sleep221073-21/usr/sbin/munin23:15:092
640268618,648sleep3688-21/usr/sbin/munin22:10:133
847526156,596sleep18511-21ntpq22:35:111
847526156,596sleep18511-21ntpq22:35:111
7689259237,509sleep17727-21ntpq00:15:281
3102240013,284sleep1225-21systemd-journal22:19:201
3102240013,284sleep1225-21systemd-journal22:19:201
23496232440,250sleep1364-21snmpd23:22:511
23496232440,250sleep1364-21snmpd23:22:511
16384223944,176sleep116450-21sendmail_mailqu23:00:181
23674222518,187sleep123707-21idleruntime-cro19:59:481
2410122148,194sleep124148-21ls23:25:091
4442220428,98sleep14537-21cut00:05:181
2442722046,161sleep124466-21modprobe21:44:481
9687219314,117sleep10-21swapper/122:40:031
1535219021,64sleep10-21swapper/122:14:471
19335218910,61sleep10-21swapper/123:10:071
913221879,117sleep19161-21kernelversion00:20:081
11556218114,151sleep211622-21sort19:20:042
21863217311,125sleep121943-21latency21:35:111
6303217234,43sleep15456-21/usr/sbin/munin00:10:191
1515216316,27sleep10-21swapper/120:30:161
1515216316,27sleep10-21swapper/120:30:161
977621629,114sleep19879-21timerwakeupswit20:55:191
20628216116,124sleep00-21swapper/019:49:490
20628216116,124sleep00-21swapper/019:49:490
21306216049,91sleep121378-21mailstats23:15:161
14328215814,126sleep114303-21/usr/sbin/munin22:55:011
14328215814,126sleep114303-21/usr/sbin/munin22:55:011
27731214921,100sleep30-21swapper/321:54:513
11967214933,95sleep111989-21munin-plugin-st21:04:481
25311214613,116sleep325288-21latency_hist20:04:503
4955214512,21sleep10-21swapper/120:40:171
9586214413,24sleep10-21swapper/119:14:541
15949214315,110sleep215959-21unin-run19:34:492
30083214119,28sleep10-21swapper/120:19:501
28638213819,21sleep10-21swapper/121:55:251
21746213825,36sleep30-21swapper/321:35:093
12692213824,93sleep212832-21grep00:30:192
1523721376,12sleep30-21swapper/322:55:193
1523721376,12sleep30-21swapper/322:55:193
10999213714,104sleep011068-21memory21:00:100
32383213120,39sleep132522-21unixbench_multi23:50:201
28607213117,27sleep10-21swapper/123:40:031
2508421297,109sleep125105-21netstat21:45:121
12620212844,26sleep10-21swapper/121:05:121
12620212844,26sleep10-21swapper/121:05:121
3856212646,60sleep10-21swapper/122:20:131
3856212646,60sleep10-21swapper/122:20:131
9202212515,93sleep09208-21diskstats20:55:060
17709212424,79sleep10-21swapper/123:05:061
17709212424,79sleep10-21swapper/123:05:061
6632212211,90sleep10-21swapper/120:45:181
25941212242,61sleep10-21swapper/123:30:141
30439212120,58sleep17800-21cyclictest20:19:581
30439212120,58sleep17800-21cyclictest20:19:581
17690212120,84sleep217193-21apt-get23:04:592
17690212120,84sleep217193-21apt-get23:04:592
10993212019,88sleep311035-21latency_hist22:44:493
2561721199,19sleep10-21swapper/120:05:041
5510211816,17sleep10-21swapper/122:25:171
5510211816,17sleep10-21swapper/122:25:171
17361211813,18sleep00-21swapper/021:20:130
31755211411,19sleep10-21swapper/122:05:191
18039211422,78sleep318063-21cut19:40:113
5467211314,77sleep05509-21wc22:25:160
5467211314,77sleep05509-21wc22:25:160
23992211323,60sleep023986-21apt-get20:00:030
7653211238,48sleep10-21swapper/120:50:061
29059211219,37sleep029065-21ntp_states23:40:140
27392211210,90sleep127404-21grep23:35:101
20017211210,67sleep120031-21acpi21:29:581
29000211115,82sleep229034-21ntp_kernel_pll_23:40:122
2126211126,70sleep12120-40mandb23:59:491
2126211126,70sleep12120-40mandb23:59:491
17029211112,17sleep10-21swapper/121:20:061
2092210813,74sleep21797-21/usr/sbin/munin22:15:092
2092210813,74sleep21797-21/usr/sbin/munin22:15:092
12739210815,75sleep212801-21/usr/sbin/munin21:05:172
12739210815,75sleep212801-21/usr/sbin/munin21:05:172
11231210811,69sleep20-21swapper/221:00:172
1958921069,28sleep119678-21cat19:45:101
1958921069,28sleep119678-21cat19:45:101
1814121065,13sleep10-21swapper/119:40:131
1348421058,15sleep10-21swapper/100:34:581
10126210528,58sleep010143-21modprobe00:24:480
79921036,52sleep00-21swapper/020:29:300
18258210212,75sleep218389-21cat21:24:512
10177210212,75sleep210251-21cat00:24:502
3016021018,48sleep10-21swapper/122:00:181
8687210018,59sleep28843-21grep19:10:192
768329939,44sleep07726-21idleruntime-cro19:09:480
244522999,22sleep124486-21ps20:00:161
3146429815,66sleep331598-21cut23:49:503
1441229827,22sleep30-21swapper/319:29:503
1215929816,66sleep012255-21cat19:20:180
55129616,64sleep2625-21latency_hist23:54:502
273762965,14sleep10-21swapper/121:52:551
984229413,65sleep09929-21sed19:14:570
290629426,19sleep10-21swapper/120:35:081
264929422,17sleep10-21swapper/100:00:101
727929313,62sleep329917-21kworker/u8:120:49:483
243529315,61sleep32548-21apt-config20:34:503
243529315,61sleep32548-21apt-config20:34:503
787629216,63sleep27962-21switchtime00:15:332
649929218,54sleep10-21swapper/119:05:051
172632929,65sleep217264-21sshd19:37:202
162632928,44sleep10-21swapper/119:35:051
1512629214,49sleep20-21swapper/221:14:502
1512629214,49sleep20-21swapper/221:14:502
1353329212,58sleep10-21swapper/119:25:161
82732919,65sleep10-21swapper/119:10:091
2752229024,16sleep10-21swapper/120:10:131
146262907,44sleep10-21swapper/119:30:001
302272896,45sleep10-21swapper/123:45:041
1901428917,60sleep018985-21latency_hist23:09:500
1901428917,60sleep018985-21latency_hist23:09:500
211672887,12sleep10-21swapper/119:50:111
211672887,12sleep10-21swapper/119:50:111
1113628823,43sleep011273-21unixbench_singl00:25:200
1002228814,60sleep310036-21vmstat20:55:213
1617628713,59sleep216254-21latency23:00:112
10032877,13sleep00-21swapper/023:55:080
10032877,13sleep00-21swapper/023:55:080
2147128616,31sleep00-21swapper/019:50:180
2147128616,31sleep00-21swapper/019:50:180
111472858,40sleep10-21swapper/122:44:511
290352840,7sleep30-21swapper/323:40:123
1904328423,28sleep30-21swapper/321:25:203
1904328423,28sleep30-21swapper/321:25:203
118828414,55sleep01194-21diskstats20:30:070
118828414,55sleep01194-21diskstats20:30:070
2633028315,16sleep00-21swapper/021:50:040
455828214,49sleep30-21swapper/320:40:073
447628214,29sleep00-21swapper/020:40:040
149352826,40sleep10-21swapper/121:12:591
149352826,40sleep10-21swapper/121:12:591
35128114,53sleep2429-21fschecks_time22:10:082
188082810,11sleep00-21swapper/021:25:120
188082810,11sleep00-21swapper/021:25:120
1456828117,46sleep014674-21grep21:10:200
1456828117,46sleep014674-21grep21:10:200
128092817,40sleep10-21swapper/119:24:431
6272805,41sleep10-21swapper/120:27:211
2312428013,52sleep223265-21unixbench_singl19:55:202
1663928028,32sleep016782-21grep23:00:210
1400227914,49sleep014003-21sshd22:54:190
3172827820,38sleep231755-21modprobe20:24:482
3172827820,38sleep231755-21modprobe20:24:482
194742787,38sleep10-21swapper/121:26:111
194742787,38sleep10-21swapper/121:26:111
164032788,38sleep10-21swapper/121:16:511
128822787,39sleep10-21swapper/100:30:261
107852787,49sleep10-21swapper/100:25:131
102682780,6sleep00-21swapper/022:40:170
2059527719,44sleep020666-21expr21:30:180
138252775,40sleep10-21swapper/122:52:021
882627614,27sleep08973-21sed22:35:180
882627614,27sleep08973-21sed22:35:180
76572765,40sleep10-21swapper/122:32:481
76572765,40sleep10-21swapper/122:32:481
1249327628,31sleep00-21swapper/022:49:470
2602227514,14sleep20-21swapper/221:49:492
1071327518,35sleep310862-21sshd19:15:203
50027418,27sleep00-21swapper/023:54:480
40662747,11sleep20-21swapper/220:38:372
305002749,47sleep30-21swapper/320:20:253
305002749,47sleep30-21swapper/320:20:253
1827827413,48sleep218372-21cut19:40:182
787127313,47sleep07846-21latency_hist22:34:490
787127313,47sleep07846-21latency_hist22:34:490
684027320,26sleep30-21swapper/300:14:493
415727323,29sleep04217-21awk20:39:480
305727315,31sleep23200-21grep00:00:192
1318827313,43sleep213195-21/usr/sbin/munin19:25:082
313827223,31sleep03163-21munin-run22:19:470
313827223,31sleep03163-21munin-run22:19:470
2400427216,38sleep224072-21users21:40:212
354127117,25sleep30-21swapper/322:20:063
354127117,25sleep30-21swapper/322:20:063
2680427124,30sleep026824-21modprobe20:09:480
2465527125,24sleep00-21swapper/023:25:200
1508027117,39sleep315102-21ntp_states19:30:143
317012707,46sleep30-21swapper/323:50:023
2840526919,30sleep20-21swapper/220:14:502
2499126917,21sleep00-21swapper/021:45:110
2615926817,22sleep20-21swapper/220:05:182
1971226818,34sleep219753-21grep23:10:162
1130326817,27sleep30-21swapper/321:00:183
784626715,35sleep07934-21cat20:50:090
726526719,30sleep36755-21apt-get00:15:013
599626715,33sleep26014-21/usr/sbin/munin20:44:582
1897826726,23sleep20-21swapper/221:25:192
1897826726,23sleep20-21swapper/221:25:192
1599326726,21sleep30-21swapper/323:00:083
1437326713,25sleep20-21swapper/222:55:022
1437326713,25sleep20-21swapper/222:55:022
1411126721,28sleep314195-21wc00:35:163
3043326614,36sleep029935-21apt-get20:19:530
3043326614,36sleep029935-21apt-get20:19:530
2938926617,34sleep229430-21latency_hist21:59:512
2727726612,23sleep00-21swapper/020:10:080
2529126627,20sleep30-21swapper/321:45:183
2207026620,30sleep022097-21munin-plugin-st23:19:470
1671026622,20sleep20-21swapper/221:19:502
1233126619,29sleep312370-21ntpq00:30:123
716226524,21sleep20-21swapper/219:05:182
369626518,29sleep00-21swapper/000:04:490
2295826518,24sleep00-21swapper/021:39:500
2126826519,26sleep30-21swapper/319:50:133
2126826519,26sleep30-21swapper/319:50:133
2050526514,22sleep20-21swapper/221:30:162
1484726519,29sleep014789-21latency_hist00:39:500
971026412,36sleep29773-21grep19:14:552
2941226419,23sleep30-21swapper/320:15:203
289962645,12sleep00-21swapper/021:55:450
2323326416,30sleep323286-21users23:20:193
2323326416,30sleep323286-21users23:20:193
1931926413,35sleep319326-21df_inode23:10:053
1476726417,30sleep214803-21awk00:39:482
131672645,11sleep30-21swapper/321:05:223
131672645,11sleep30-21swapper/321:05:223
559426316,26sleep20-21swapper/222:25:182
559426316,26sleep20-21swapper/222:25:182
545526315,22sleep00-21swapper/000:09:580
393826317,29sleep34004-21sort00:05:043
2885326312,22sleep30-21swapper/321:55:283
2595226316,25sleep20-21swapper/223:30:152
3033426215,20sleep00-21swapper/023:45:070
2454126221,24sleep30-21swapper/323:25:193
218526220,21sleep30-21swapper/322:15:103
218526220,21sleep30-21swapper/322:15:103
214026220,21sleep20-21swapper/223:59:502
214026220,21sleep20-21swapper/223:59:502
1940626223,19sleep20-21swapper/219:45:072
1940626223,19sleep20-21swapper/219:45:072
119552626,12sleep30-21swapper/319:20:133
82126114,20sleep20-21swapper/220:29:482
726726116,29sleep07261-21apt-get00:15:020
2247026119,27sleep222502-21fschecks_count23:20:052
2247026119,27sleep222502-21fschecks_count23:20:052
1530926117,24sleep00-21swapper/019:30:190
1447026116,30sleep014609-21latency_hist19:29:510
1150026121,20sleep30-21swapper/322:45:103
949226019,26sleep29520-21/usr/sbin/munin20:55:122
871226026,19sleep30-21swapper/322:35:173
871226026,19sleep30-21swapper/322:35:173
525426017,27sleep05397-21latency_hist00:09:500
345926012,12sleep00-21swapper/022:20:030
345926012,12sleep00-21swapper/022:20:030
310592600,8sleep20-21swapper/222:04:502
2628726015,26sleep30-21swapper/320:05:193
393725918,24sleep23931-21apt-get00:05:032
3033525917,17sleep30-21swapper/322:00:203
2434225915,27sleep224387-21/usr/sbin/munin23:25:142
2052625918,26sleep30-21swapper/321:30:173
1432725915,26sleep014321-21apt-get22:55:000
1432725915,26sleep014321-21apt-get22:55:000
1304925914,31sleep213133-21cat22:50:092
1254425915,21sleep20-21swapper/222:49:492
2891425811,22sleep20-21swapper/220:15:112
160302588,31sleep00-21swapper/019:34:510
898725716,27sleep39073-21fw_packets00:20:063
701125716,22sleep20-21swapper/222:30:152
701125716,22sleep20-21swapper/222:30:152
447325713,27sleep24469-21apt-get20:40:022
2672025721,20sleep20-21swapper/221:50:142
264352576,11sleep30-21swapper/323:30:523
1823125717,18sleep00-21swapper/019:40:160
1769625717,24sleep317693-21apt-get23:05:003
1769625717,24sleep317693-21apt-get23:05:003
1381725717,23sleep313812-21apt-get21:10:023
1381725717,23sleep313812-21apt-get21:10:023
692025616,22sleep30-21swapper/322:30:123
692025616,22sleep30-21swapper/322:30:123
324832568,32sleep032493-21munin-run22:09:500
2755625615,26sleep327631-21sensors_temp23:35:173
2706825615,25sleep327087-21apt-get20:09:593
2702525615,26sleep227022-21apt-get23:34:592
2148125612,21sleep30-21swapper/323:15:183
1665125615,25sleep316657-21ntp_states19:35:153
799425516,22sleep38002-21apt-get19:09:593
290925514,24sleep32887-21apt-get20:35:103
2375325516,17sleep30-21swapper/319:59:503
160925516,22sleep30-21swapper/323:55:193
160925516,22sleep30-21swapper/323:55:193
390325419,18sleep20-21swapper/222:20:142
390325419,18sleep20-21swapper/222:20:142
2748325412,17sleep00-21swapper/023:35:130
221092547,16sleep20-21swapper/219:54:362
221092547,16sleep20-21swapper/219:54:362
2159425411,23sleep20-21swapper/221:35:042
130982546,36sleep00-21swapper/000:33:160
1275525413,28sleep312776-21dpkg22:49:583
1223125413,25sleep012250-21apt-get21:04:590
1223125413,25sleep012250-21apt-get21:04:590
936825312,26sleep29439-21systemd-run22:39:482
936825312,26sleep29439-21systemd-run22:39:482
527725314,21sleep35370-21memory22:25:113
527725314,21sleep35370-21memory22:25:113
3119025315,20sleep331253-21df22:05:043
2999325314,22sleep030019-21/usr/sbin/munin20:19:490
2679125317,20sleep00-21swapper/023:34:490
1951725316,17sleep00-21swapper/023:10:100
926925212,24sleep01-21systemd19:14:490
689525213,23sleep36396-21/usr/sbin/munin19:05:153
677525218,20sleep20-21swapper/200:14:482
602025213,23sleep06016-21apt-get20:45:000
3118225212,24sleep231188-21grep22:05:032
237525212,24sleep32372-21apt-get23:59:593
237425212,26sleep21-21systemd20:34:492
237425212,26sleep21-21systemd20:34:492
2362325213,24sleep023629-21ntp_states21:40:150
2281925213,23sleep022868-21ntp_states19:55:140
1534825212,24sleep015360-21apt-get21:15:000
33325112,23sleep0343-21df_inode22:10:060
2974825112,23sleep029835-21fw_packets22:00:090
1980225114,23sleep30-21swapper/319:45:163
1980225114,23sleep30-21swapper/319:45:163
115832515,10sleep30-21swapper/300:28:383
86225013,22sleep3871-21munin-run20:29:493
756925010,17sleep20-21swapper/220:50:032
3145325014,21sleep231463-21munin-run23:49:492
1536625010,20sleep30-21swapper/321:15:033
799124913,17sleep30-21swapper/320:50:123
2406024812,15sleep20-21swapper/220:00:052
2238324711,18sleep00-21swapper/023:19:590
2238324711,18sleep00-21swapper/023:19:590
114792464,12sleep20-21swapper/200:27:202
104682409,14sleep20-21swapper/222:40:202
1731423711,11sleep30-21swapper/321:20:123
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional