You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-24 - 13:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Fri Apr 24, 2026 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
12826211496,1132sleep014129-1kworker/0:0H22:55:590
12826211496,1132sleep014129-1kworker/0:0H22:55:590
3246263214,600sleep33239-21apt-get22:26:023
3205126069,462sleep232113-21sendmail-msp22:15:472
2039524628,436sleep220536-21latency_hist23:20:492
737241979,320sleep2803-21fschecks_time00:01:082
737241979,320sleep2803-21fschecks_time00:01:082
2082423908,370sleep220862-21fschecks_count19:56:052
29363238056,302sleep213441-21snmpd22:06:142
29363238056,302sleep213441-21snmpd22:06:142
2119923228,300sleep221286-21cat23:21:182
29050229712,194sleep229178-21timerwakeupswit23:46:172
135342277128,132sleep213561-21fschecks_count21:16:062
627522510,42sleep24165-21cyclictest22:35:492
22379223944,128sleep20-21swapper/220:01:062
28056223270,133sleep20-21swapper/222:01:172
3855222846,89sleep23999-21latency_hist20:45:492
3855222846,89sleep23999-21latency_hist20:45:492
27206221936,92sleep227182-21latency_hist22:00:482
27206221936,92sleep227182-21latency_hist22:00:482
11392220316,170sleep211438-21ntpq22:51:132
5850219027,141sleep25938-21latency_hist19:10:502
19432218222,62sleep219380-21latency_hist21:35:492
720321808,44sleep20-21swapper/220:56:032
6464217910,157sleep26481-21fschecks_count22:36:062
3715217514,45sleep23848-21latency_hist00:10:492
22169217512,17sleep20-21swapper/223:26:022
143021728,52sleep21459-21unin-run22:20:482
680521717,68sleep20-21swapper/200:20:462
1309221718,51sleep20-21swapper/222:56:582
1309221718,51sleep20-21swapper/222:56:582
2589216911,143sleep21674-21/usr/sbin/munin22:21:202
22480216414,136sleep322524-21unin-run21:45:473
22480216414,136sleep322524-21unin-run21:45:473
27267216338,108sleep20-21swapper/220:16:042
2393921638,16sleep00-21swapper/020:06:090
2393921638,16sleep00-21swapper/020:06:090
30025216111,133sleep20-21swapper/223:51:062
19360216032,113sleep219139-21/usr/sbin/munin19:51:082
765721586,36sleep20-21swapper/219:14:422
691215130,109sleep2732-21munin-run20:35:472
14393214913,116sleep20-21swapper/223:01:022
14393214913,116sleep20-21swapper/223:01:022
8842214415,106sleep18907-21uniq21:01:061
2407214411,116sleep12394-21apt-get00:06:031
654214021,99sleep1688-21awk20:35:461
20978214041,54sleep221078-21latency_hist21:40:492
20978214041,54sleep221078-21latency_hist21:40:492
14786214016,108sleep014869-21tr19:36:160
14786214016,108sleep014869-21tr19:36:160
122372140105,18sleep20-21swapper/221:11:132
122372140105,18sleep20-21swapper/221:11:132
24972213822,97sleep124977-21modprobe23:35:461
1551821385,33sleep20-21swapper/219:40:302
1551821385,33sleep20-21swapper/219:40:302
7834213445,24sleep20-21swapper/200:21:192
7834213445,24sleep20-21swapper/200:21:192
17088213445,77sleep217155-21dpkg19:45:472
16949213415,103sleep216987-21ntpq21:26:152
16949213415,103sleep216987-21ntpq21:26:152
16028213310,102sleep216081-21fschecks_count23:06:072
21214213112,23sleep20-21swapper/221:41:022
21214213112,23sleep20-21swapper/221:41:022
31607213011,107sleep231621-21grep23:56:072
1121321299,99sleep20-21swapper/221:06:212
9610212616,43sleep20-21swapper/219:21:042
14040212417,25sleep10-21swapper/119:35:491
10674212431,50sleep210648-21/usr/sbin/munin00:31:042
412212216,88sleep00-21swapper/000:00:490
4113211945,25sleep24266-21gpgv19:05:492
28760211922,15sleep20-21swapper/220:21:012
1757821196,15sleep20-21swapper/219:45:512
12964211942,32sleep20-21swapper/219:31:092
3520211721,34sleep33604-21users19:01:183
25510211720,12sleep20-21swapper/223:36:082
25510211720,12sleep20-21swapper/223:36:082
24382211725,35sleep20-21swapper/220:06:192
24382211725,35sleep20-21swapper/220:06:192
10770211760,16sleep20-21swapper/222:50:482
913221158,19sleep20-21swapper/200:26:082
25987211417,77sleep026094-21cut21:56:060
25987211417,77sleep026094-21cut21:56:060
18060211449,19sleep20-21swapper/223:11:162
18060211449,19sleep20-21swapper/223:11:162
14721211413,32sleep114732-21ntp_states19:36:131
14721211413,32sleep114732-21ntp_states19:36:131
4598211321,35sleep20-21swapper/222:30:492
2053821130,7sleep30-21swapper/323:20:493
15612211322,14sleep20-21swapper/221:21:192
15612211322,14sleep20-21swapper/221:21:192
12837211313,84sleep112821-21apt-get22:56:041
12837211313,84sleep112821-21apt-get22:56:041
27506211217,74sleep027187-21/usr/sbin/munin20:16:150
10496211212,87sleep013441-21snmpd22:47:470
603821119,92sleep00-21swapper/022:35:280
29935211110,17sleep10-21swapper/123:51:011
1176821118,25sleep211890-21cat19:26:172
1176821118,25sleep211890-21cat19:26:172
918321108,87sleep29231-21systemd-run22:45:472
24744211022,57sleep20-21swapper/221:51:152
16546211012,80sleep116619-21/usr/sbin/munin23:06:181
4530210816,14sleep20-21swapper/220:46:132
4757210715,76sleep34802-21if_eth019:06:273
25376210717,71sleep325381-21/usr/sbin/munin23:36:053
25376210717,71sleep325381-21/usr/sbin/munin23:36:053
224821079,86sleep21-21systemd20:40:482
224821079,86sleep21-21systemd20:40:482
3149121057,24sleep213441-21snmpd20:29:502
20322210413,72sleep020442-21/usr/sbin/munin21:36:190
20322210413,72sleep020442-21/usr/sbin/munin21:36:190
14476210122,64sleep114500-21fschecks_count23:01:051
14476210122,64sleep114500-21fschecks_count23:01:051
5227210029,32sleep25230-21cron00:15:462
5227210029,32sleep25230-21cron00:15:462
3962210040,40sleep03989-21apt-get19:05:470
1818529919,56sleep118220-21sort21:31:071
1541729913,73sleep315418-21sshd19:39:143
1541729913,73sleep315418-21sshd19:39:143
2884529811,68sleep328738-21/usr/sbin/munin20:21:063
1925829821,59sleep10-21swapper/119:51:071
1410229820,62sleep314152-21unin-run23:00:473
1410229820,62sleep314152-21unin-run23:00:473
467029721,56sleep14818-21grep00:11:161
467029721,56sleep14818-21grep00:11:161
2439829413,16sleep30-21swapper/321:51:083
805729311,30sleep20-21swapper/219:15:592
805729311,30sleep20-21swapper/219:15:592
730129313,64sleep36820-21apt-get00:20:563
730129313,64sleep36820-21apt-get00:20:563
686729314,64sleep06938-21mailstats22:36:160
1194629323,19sleep00-21swapper/021:11:070
1194629323,19sleep00-21swapper/021:11:070
992429228,41sleep39930-21ntp_states22:46:133
1489729217,60sleep314980-21mailstats23:01:153
1489729217,60sleep314980-21mailstats23:01:153
583129016,56sleep05916-21memory00:16:120
51482909,68sleep05150-21sshd20:48:210
3006629019,24sleep10-21swapper/120:25:491
263592905,28sleep20-21swapper/220:12:052
238452907,32sleep20-21swapper/221:48:332
238452907,32sleep20-21swapper/221:48:332
2681728930,32sleep30-21swapper/320:15:493
98332886,30sleep20-21swapper/221:03:262
951028811,57sleep19544-21/usr/sbin/munin00:26:231
2809828818,48sleep328122-21modprobe23:45:463
1758728820,48sleep017102-21apt-get19:45:570
1227828822,44sleep012301-21munin-run22:55:470
1135528819,45sleep111493-21grep00:31:201
66452875,29sleep20-21swapper/220:52:442
2936028716,56sleep129362-21expr22:06:131
2936028716,56sleep129362-21expr22:06:131
279592875,28sleep20-21swapper/223:43:592
246882875,28sleep20-21swapper/223:32:082
203592868,55sleep00-21swapper/023:20:470
1788628616,54sleep117903-21uname19:46:111
2274228516,35sleep00-21swapper/021:46:000
2274228516,35sleep00-21swapper/021:46:000
2099928518,52sleep121012-21kernelversion19:56:081
2707728414,54sleep027110-21/usr/sbin/munin23:41:060
216432836,55sleep10-21swapper/121:41:131
216432836,55sleep10-21swapper/121:41:131
187728223,40sleep11901-21munin-run00:05:471
187728223,40sleep11901-21munin-run00:05:471
1715628220,39sleep317258-21dpkg19:45:483
167328221,29sleep10-21swapper/122:20:591
873928122,37sleep38879-21tail19:16:163
873928122,37sleep38879-21tail19:16:163
811028116,44sleep17305-21/usr/sbin/munin00:21:221
811028116,44sleep17305-21/usr/sbin/munin00:21:221
446428111,52sleep03989-21apt-get19:05:510
2289128113,44sleep323029-21unixbench_singl23:26:193
1745828118,49sleep317463-21find23:10:573
1745828118,49sleep317463-21find23:10:573
1913828013,16sleep00-21swapper/019:50:580
2529227912,53sleep025318-21cpu23:36:020
2529227912,53sleep025318-21cpu23:36:020
2193027918,30sleep10-21swapper/123:25:491
143992796,60sleep00-21swapper/021:16:480
878227821,27sleep00-21swapper/000:25:490
878227821,27sleep00-21swapper/000:25:490
322102770,17sleep032374-21grep22:15:490
133827721,34sleep00-21swapper/020:36:100
133827721,34sleep00-21swapper/020:36:100
1156727719,27sleep30-21swapper/322:51:183
1038227722,28sleep00-21swapper/000:30:460
2972927619,29sleep10-21swapper/123:50:491
2870127610,48sleep30-21swapper/322:05:463
1619427616,27sleep10-21swapper/119:41:101
2920427527,30sleep328715-21apt-get22:05:583
2920427527,30sleep328715-21apt-get22:05:583
717627419,35sleep17184-21acpi20:55:581
37327423,29sleep30-21swapper/300:00:483
3144727413,24sleep10-21swapper/122:11:221
2748627412,26sleep30-21swapper/320:16:143
923227326,29sleep30-21swapper/322:45:483
799927315,27sleep00-21swapper/022:41:050
58727329,26sleep0608-21apt00:00:570
58727329,26sleep0608-21apt00:00:570
1006027312,46sleep010062-21ntp_states19:21:140
2977827214,24sleep00-21swapper/022:06:240
2977827214,24sleep00-21swapper/022:06:240
1935227216,40sleep019416-21sendmail-msp21:35:460
1107527215,26sleep10-21swapper/121:06:201
594027130,21sleep30-21swapper/319:10:513
2139927119,23sleep00-21swapper/021:41:080
2139927119,23sleep00-21swapper/021:41:080
1892627123,22sleep30-21swapper/319:50:493
616127014,24sleep00-21swapper/020:51:160
1321227019,26sleep10-21swapper/121:15:481
1321227019,26sleep10-21swapper/121:15:481
237326916,36sleep31892-21apt-get00:05:573
1498326913,35sleep015071-21awk23:01:160
1498326913,35sleep015071-21awk23:01:160
1123926924,28sleep30-21swapper/319:26:063
1123926924,28sleep30-21swapper/319:26:063
933326810,42sleep09421-21switchtime21:01:180
2547526820,22sleep10-21swapper/123:36:071
2547526820,22sleep10-21swapper/123:36:071
705126710,39sleep10-21swapper/119:11:171
401226719,30sleep14112-21gpgv19:05:481
1976826715,34sleep119874-21tune2fs21:36:081
1976826715,34sleep119874-21tune2fs21:36:081
1545826711,13sleep30-21swapper/321:21:163
1545826711,13sleep30-21swapper/321:21:163
1170026618,30sleep111767-21/usr/sbin/munin19:26:161
1170026618,30sleep111767-21/usr/sbin/munin19:26:161
1131526617,35sleep011373-21/usr/sbin/munin19:26:070
1131526617,35sleep011373-21/usr/sbin/munin19:26:070
1094826620,22sleep00-21swapper/021:06:180
3257826520,28sleep00-21swapper/022:15:570
2539126520,25sleep10-21swapper/120:11:031
2452126518,25sleep10-21swapper/121:51:101
2323526517,22sleep10-21swapper/121:46:181
2323526517,22sleep10-21swapper/121:46:181
134982656,44sleep00-21swapper/019:31:200
90002646,11sleep00-21swapper/019:16:420
90002646,11sleep00-21swapper/019:16:420
2420026417,31sleep324262-21/usr/sbin/munin23:31:163
228226421,21sleep30-21swapper/320:40:493
228226421,21sleep30-21swapper/320:40:493
2024526426,20sleep30-21swapper/321:36:183
2024526426,20sleep30-21swapper/321:36:183
141026422,21sleep10-21swapper/122:20:461
1396126418,26sleep314024-21sendmail-msp19:35:473
136726424,21sleep10-21swapper/120:36:111
136726424,21sleep10-21swapper/120:36:111
688626324,18sleep30-21swapper/320:55:473
538026318,18sleep10-21swapper/120:50:491
3202726319,21sleep10-21swapper/123:56:181
2761326316,26sleep00-21swapper/022:01:070
263542635,11sleep10-21swapper/121:56:151
263542635,11sleep10-21swapper/121:56:151
2597326315,29sleep30-21swapper/321:56:053
2597326315,29sleep30-21swapper/321:56:053
252226319,26sleep10-21swapper/120:41:031
252226319,26sleep10-21swapper/120:41:031
2445326311,24sleep30-21swapper/320:06:203
2445326311,24sleep30-21swapper/320:06:203
589526213,29sleep10-21swapper/120:51:091
3057426213,27sleep00-21swapper/020:26:080
302726219,24sleep30-21swapper/320:41:183
302726219,24sleep30-21swapper/320:41:183
2718826219,25sleep127210-21apt-get20:15:591
1748726219,25sleep00-21swapper/023:11:020
1748726219,25sleep00-21swapper/023:11:020
1722126215,21sleep00-21swapper/021:26:220
1722126215,21sleep00-21swapper/021:26:220
1313126221,24sleep30-21swapper/321:15:463
1313126221,24sleep30-21swapper/321:15:463
614126112,25sleep10-21swapper/122:35:491
2432326112,17sleep10-21swapper/120:06:181
2432326112,17sleep10-21swapper/120:06:181
2108626116,21sleep00-21swapper/023:21:160
2037326116,19sleep10-21swapper/123:20:481
1849026121,20sleep30-21swapper/321:31:133
941026023,21sleep30-21swapper/300:26:213
3185326017,28sleep031848-21apt-get20:31:010
314022606,20sleep10-21swapper/120:28:401
3051426016,25sleep30-21swapper/320:26:073
295526015,26sleep0294-21rs:main0
2692326014,21sleep10-21swapper/123:41:031
931825915,22sleep10-21swapper/119:20:461
931825915,22sleep10-21swapper/119:20:461
80725916,27sleep3865-21if_eth000:01:093
80725916,27sleep3865-21if_eth000:01:093
371425910,14sleep00-21swapper/022:26:150
2256225920,19sleep10-21swapper/120:01:091
2124325915,21sleep00-21swapper/019:56:150
1726425919,16sleep00-21swapper/019:45:490
160152599,21sleep00-21swapper/023:06:050
746725816,23sleep00-21swapper/020:56:090
48872586,12sleep30-21swapper/322:31:063
3257625818,25sleep332070-21apt-get22:15:563
3047225815,24sleep00-21swapper/023:51:160
1724625817,28sleep317285-21sed23:10:483
1499525811,31sleep10-21swapper/121:21:011
1499525811,31sleep10-21swapper/121:21:011
2912825710,27sleep029151-21netstat20:21:110
2888625717,16sleep00-21swapper/023:46:150
2882325714,25sleep128910-21gpgv22:05:481
2509725712,20sleep00-21swapper/021:51:210
2276925719,22sleep30-21swapper/320:01:153
211062577,35sleep30-21swapper/319:56:103
966025623,17sleep10-21swapper/122:46:071
573125617,16sleep10-21swapper/100:16:091
418225615,24sleep34226-21irqstats00:11:063
418225615,24sleep34226-21irqstats00:11:063
2311725612,33sleep013441-21snmpd23:26:490
770825515,27sleep37780-21mailstats20:56:163
422825517,16sleep00-21swapper/000:11:070
422825517,16sleep00-21swapper/000:11:070
2875625512,28sleep328781-21awk23:46:103
2605025524,15sleep00-21swapper/020:11:180
2251325511,15sleep00-21swapper/020:01:080
1471425520,18sleep30-21swapper/321:20:473
119932558,36sleep10-21swapper/122:52:091
846825413,19sleep10-21swapper/122:41:181
648225418,21sleep36561-21/usr/sbin/munin22:36:073
50292545,21sleep30-21swapper/320:46:513
1029025411,21sleep10-21swapper/119:21:191
33782536,36sleep00-21swapper/000:07:080
2216825313,23sleep122163-21apt-get23:26:011
2128725316,17sleep30-21swapper/323:21:193
1654925313,23sleep316542-21apt-get21:26:043
1654925313,23sleep316542-21apt-get21:26:043
554725212,23sleep35539-21apt-get00:16:033
35502529,33sleep00-21swapper/020:42:520
35502529,33sleep00-21swapper/020:42:520
2379525212,22sleep023800-21diskstats23:31:050
1167225213,18sleep30-21swapper/321:10:503
882825112,22sleep38835-21df_inode21:01:043
37742519,27sleep13840-21grep22:26:161
3211725111,20sleep30-21swapper/322:15:483
2995225112,20sleep330008-21df23:51:033
177225113,22sleep31781-21diskstats22:21:053
583425014,20sleep10-21swapper/119:10:491
1499425015,17sleep00-21swapper/021:21:000
1499425015,17sleep00-21swapper/021:21:000
144472506,35sleep10-21swapper/121:17:261
3192324912,22sleep331932-21df_inode20:31:043
1768324913,22sleep10-21swapper/121:29:131
1768324913,22sleep10-21swapper/121:29:131
1091724911,16sleep30-21swapper/319:25:493
195792489,15sleep30-21swapper/319:51:143
1748324814,18sleep10-21swapper/123:10:591
1748324814,18sleep10-21swapper/123:10:591
694024717,17sleep10-21swapper/122:36:171
687124712,16sleep00-21swapper/019:11:120
1107324717,14sleep30-21swapper/300:31:133
2321524511,14sleep30-21swapper/321:46:163
2321524511,14sleep30-21swapper/321:46:163
106722399,16sleep00-21swapper/000:31:020
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional