You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-08 - 13:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Fri May 08, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17315214189,1398sleep215715-21kworker/2:123:05:582
17315214189,1398sleep215715-21kworker/2:123:05:582
1080029677,925sleep210844-21ntpq19:20:162
19836296314,937sleep115626-21kworker/1:221:32:541
10686240819,372sleep310663-21latency_hist21:04:513
32103233686,197sleep232114-21modprobe23:54:482
32103233686,197sleep232114-21modprobe23:54:482
3670228212,245sleep23758-21xargs00:05:182
31385228213,207sleep231529-21grep20:25:212
31385228213,207sleep231529-21grep20:25:212
22300228236,230sleep2309-21rs:main2
12957226615,173sleep213152-21cat22:54:522
2232422630,15sleep3225-21systemd-journal23:24:483
13690222798,115sleep213195-21/usr/sbin/munin22:55:212
22552222014,183sleep222569-21modprobe19:59:492
12583221241,112sleep212695-21timerandwakeup19:25:212
30372220432,114sleep230404-21modprobe20:24:482
1200220127,25sleep20-21swapper/223:59:482
9050220053,107sleep20-21swapper/220:59:492
9050220053,107sleep20-21swapper/220:59:492
23109219015,158sleep023110-21cron21:44:480
2006021816,161sleep220134-21find23:15:192
2036021809,157sleep020383-21fschecks_count21:35:080
2187217434,98sleep22279-21tune2fs22:20:072
8209216919,133sleep228359-21kworker/u8:319:14:512
30512216410,139sleep230685-21cat23:49:512
30512216410,139sleep230685-21cat23:49:512
24973216116,28sleep20-21swapper/220:05:182
28680215815,127sleep328547-21latency_hist23:44:513
10373215788,50sleep20-21swapper/222:45:192
26087215618,27sleep20-21swapper/223:35:132
31792215387,53sleep231817-21awk22:10:142
5927215124,109sleep26059-21cat19:05:182
2184521490,68sleep00-21swapper/019:55:180
1643521499,12sleep30-21swapper/323:05:073
1643521499,12sleep30-21swapper/323:05:073
23302214856,77sleep223448-21unixbench_singl23:25:202
23159214528,17sleep20-21swapper/221:44:502
554721445,12sleep30-21swapper/300:10:233
13529214422,45sleep20-21swapper/219:30:052
27482214313,49sleep227612-21unixbench_multi21:56:092
27358214218,30sleep00-21swapper/023:40:080
2605421428,12sleep10-21swapper/120:10:061
32750214032,16sleep20-21swapper/220:30:302
2066421405,11sleep20-21swapper/219:50:282
15620213917,62sleep20-21swapper/219:35:182
5940213634,20sleep20-21swapper/200:14:512
23113213516,102sleep1225-21systemd-journal21:44:491
2976421340,6sleep20-21swapper/222:04:522
1774321329,112sleep3363-21snmpd19:42:423
60321309,47sleep2684-21tune2fs22:15:112
19507212931,66sleep20-21swapper/219:49:522
13297212921,63sleep10-21swapper/122:55:071
324921245,12sleep30-21swapper/320:40:083
4951212312,84sleep20-21swapper/220:45:112
4951212312,84sleep20-21swapper/220:45:112
27800212315,86sleep027866-21wc20:15:140
21919212321,84sleep122045-21ls19:55:201
5883211815,44sleep20-21swapper/222:30:252
5883211815,44sleep20-21swapper/222:30:252
7880211717,80sleep37940-21/usr/sbin/munin20:55:103
7880211717,80sleep37940-21/usr/sbin/munin20:55:103
410621178,74sleep2363-21snmpd20:41:122
26587211732,74sleep226692-21timerandwakeup20:10:192
203021159,93sleep20-21swapper/220:35:182
9454211412,78sleep00-21swapper/021:00:070
9038211213,85sleep29067-21awk00:24:482
11778211222,70sleep20-21swapper/200:30:212
499211110,85sleep1550-21processes20:30:371
2278211013,71sleep12425-21unixbench_singl20:35:211
12460211012,31sleep20-21swapper/221:09:582
881821099,19sleep20-21swapper/219:15:032
8252210822,57sleep38440-21latency_hist19:14:523
7544210719,72sleep07674-21sed19:10:210
17257210727,20sleep20-21swapper/221:25:082
7221210611,45sleep27255-21ps22:35:182
12248210614,80sleep212313-21date21:09:502
2380221058,21sleep223846-21ntpq21:45:152
2027121057,15sleep20-21swapper/221:35:012
14012210412,77sleep20-21swapper/221:14:592
23447210313,74sleep20-21swapper/220:00:182
27899210238,53sleep227943-21ntpq20:15:162
12405210213,16sleep20-21swapper/200:35:012
244821018,25sleep20-21swapper/200:00:292
16082210113,35sleep20-21swapper/221:20:162
579021006,74sleep00-21swapper/019:05:160
1379421008,19sleep00-21swapper/022:55:230
49042998,45sleep20-21swapper/222:29:442
49042998,45sleep20-21swapper/222:29:442
306742998,15sleep20-21swapper/222:05:222
2441129920,57sleep124431-21uname23:30:121
966529716,59sleep20-21swapper/222:44:512
2161229719,55sleep321737-21cat21:39:513
179062978,36sleep20-21swapper/223:10:022
179062978,36sleep20-21swapper/223:10:022
281652969,41sleep228178-21awk23:40:222
957629511,32sleep20-21swapper/221:00:092
284229524,49sleep32929-21latency_hist00:04:503
1991729515,64sleep319918-21ntp_states23:15:163
132842940,5sleep30-21swapper/321:10:193
102552946,11sleep10-21swapper/122:45:141
650929219,50sleep36481-21latency_hist22:34:503
650929219,50sleep36481-21latency_hist22:34:503
2717029215,62sleep327201-21nfs_client21:56:023
1704729214,64sleep117090-21ntpq19:40:141
248202918,11sleep20-21swapper/223:30:212
188042917,34sleep20-21swapper/221:30:092
1504929125,41sleep015068-21apt-get19:35:000
2612228912,64sleep3363-21snmpd23:35:143
249502896,13sleep20-21swapper/221:50:032
206422895,35sleep10-21swapper/119:50:231
169082897,30sleep20-21swapper/219:40:112
2606628820,53sleep326122-21tune2fs20:10:083
73452877,30sleep20-21swapper/200:18:222
3014128715,21sleep00-21swapper/022:05:110
2451728720,43sleep024580-21fschecks_time20:05:080
2319928715,48sleep123301-21/usr/sbin/munin23:25:191
957728622,47sleep39720-21grep19:15:243
830828617,53sleep38317-21acpi22:39:593
2620728628,37sleep326230-21munin-run21:54:493
1194428624,32sleep30-21swapper/322:50:213
102482865,10sleep20-21swapper/200:25:292
2944428510,61sleep029493-21ntpq23:45:210
2944428510,61sleep029493-21ntpq23:45:210
473228412,49sleep04818-21fschecks_time00:10:080
157672849,33sleep20-21swapper/223:00:582
72782835,27sleep20-21swapper/220:51:582
72782835,27sleep20-21swapper/220:51:582
552428327,27sleep10-21swapper/119:05:091
3085228315,51sleep030829-21apt-get23:50:090
3085228315,51sleep030829-21apt-get23:50:090
2504028314,52sleep025091-21fschecks_count21:50:090
291412829,55sleep10-21swapper/123:45:141
291412829,55sleep10-21swapper/123:45:141
1992528215,30sleep00-21swapper/023:15:170
1371628227,28sleep30-21swapper/322:55:223
2952728116,46sleep329026-21/usr/sbin/munin23:45:243
2952728116,46sleep329026-21/usr/sbin/munin23:45:243
2003428113,53sleep320098-21cat21:34:503
606228027,23sleep30-21swapper/319:05:193
2831628021,43sleep128318-21cpu22:00:041
692227917,44sleep07065-21sed00:15:210
667127925,21sleep10-21swapper/100:15:181
279227923,26sleep10-21swapper/100:04:481
2194427925,23sleep30-21swapper/321:40:113
1293927915,46sleep013013-21mailstats00:35:200
2073627818,29sleep10-21swapper/121:35:171
2763527720,37sleep127674-21ntpq23:40:131
1790727725,25sleep00-21swapper/019:44:490
2532227611,50sleep125356-21ntp_kernel_err21:50:141
247027624,30sleep30-21swapper/322:20:133
2333527619,27sleep00-21swapper/020:00:150
200727617,20sleep00-21swapper/000:00:190
1218827626,27sleep30-21swapper/300:34:503
1101527621,31sleep10-21swapper/121:05:091
52072759,23sleep10-21swapper/122:30:081
52072759,23sleep10-21swapper/122:30:081
3202527519,25sleep00-21swapper/020:29:510
3202527519,25sleep00-21swapper/020:29:510
3196227523,30sleep331988-21munin-run20:29:503
3196227523,30sleep331988-21munin-run20:29:503
2723727519,35sleep027264-21munin-run20:14:490
2354027517,25sleep30-21swapper/320:00:193
1806427414,40sleep117884-21/usr/sbin/munin23:10:091
1806427414,40sleep117884-21/usr/sbin/munin23:10:091
909227313,14sleep10-21swapper/100:24:511
363427323,24sleep00-21swapper/020:40:170
3045627315,42sleep130604-21latency_hist20:24:521
1477127321,26sleep30-21swapper/322:59:583
94142727,12sleep10-21swapper/100:25:071
633027218,29sleep00-21swapper/020:50:100
633027218,29sleep00-21swapper/020:50:100
2390327215,26sleep00-21swapper/023:29:500
1328727221,25sleep00-21swapper/019:29:510
1283527220,24sleep10-21swapper/122:54:511
373227120,21sleep00-21swapper/022:25:060
373227120,21sleep00-21swapper/022:25:060
3087127120,30sleep30-21swapper/323:50:253
3087127120,30sleep30-21swapper/323:50:253
242182719,16sleep00-21swapper/023:30:070
1269627114,40sleep312840-21unixbench_singl19:25:223
237027018,31sleep10-21swapper/122:20:101
1693627021,21sleep00-21swapper/021:24:500
1616127016,21sleep00-21swapper/023:04:510
1274827023,25sleep00-21swapper/022:54:500
1070327013,27sleep00-21swapper/019:20:140
936926917,27sleep10-21swapper/121:00:041
643226917,36sleep16482-21if_eth020:50:111
643226917,36sleep16482-21if_eth020:50:111
3259926923,31sleep30-21swapper/320:30:273
151692696,43sleep10-21swapper/123:00:121
924626814,40sleep09292-21ntp_offset19:15:170
801526818,30sleep08046-21modprobe22:39:480
448226818,26sleep10-21swapper/122:25:191
448226818,26sleep10-21swapper/122:25:191
188232686,39sleep00-21swapper/019:45:220
963726714,40sleep19663-21latency_hist22:44:501
953626722,28sleep39586-21/usr/sbin/munin00:25:093
58402678,47sleep00-21swapper/020:48:330
58402678,47sleep00-21swapper/020:48:330
1525726712,12sleep10-21swapper/119:35:091
1218326715,34sleep012187-21cron21:09:480
376026616,30sleep33882-21/usr/sbin/munin00:05:193
3118126614,37sleep331277-21cut22:09:513
1986026621,27sleep10-21swapper/123:15:141
70826528,21sleep00-21swapper/020:30:400
2470126517,26sleep10-21swapper/121:49:501
2336826514,28sleep10-21swapper/120:00:171
2148926513,38sleep121499-21ntp_states23:20:151
1637026515,27sleep30-21swapper/321:20:213
3127326413,31sleep131384-21sort20:25:201
3127326413,31sleep131384-21sort20:25:201
1718126416,18sleep00-21swapper/021:25:060
56972635,33sleep10-21swapper/100:12:191
2569326319,28sleep125743-21/usr/sbin/munin23:35:041
2384826313,35sleep323858-21ntpq21:45:163
833326214,31sleep38390-21/usr/sbin/munin00:20:213
699226217,17sleep10-21swapper/119:10:091
286092629,39sleep328611-21sshd20:17:273
279942627,43sleep00-21swapper/021:59:460
2177726216,32sleep321842-21awk19:55:183
2155826210,28sleep00-21swapper/023:20:180
16232626,12sleep10-21swapper/122:17:481
680426115,23sleep30-21swapper/300:15:203
40552619,40sleep10-21swapper/100:06:091
3245626116,28sleep032458-21sshd22:11:580
2930126115,28sleep329092-21/usr/sbin/munin20:20:093
2799126124,18sleep10-21swapper/120:15:191
2469926122,22sleep30-21swapper/320:05:113
2001726116,26sleep30-21swapper/319:50:133
1831526111,38sleep3363-21snmpd21:28:163
1570626118,23sleep10-21swapper/121:20:071
98362605,14sleep10-21swapper/119:16:271
2879226021,24sleep328823-21processes22:00:183
27202605,17sleep00-21swapper/020:38:390
1881726014,31sleep018873-21tune2fs21:30:110
1430926019,18sleep10-21swapper/121:15:101
70422590,6sleep10-21swapper/122:35:121
1568725924,18sleep30-21swapper/319:35:193
120732597,40sleep00-21swapper/000:33:590
937025813,21sleep00-21swapper/022:41:580
598625813,26sleep10-21swapper/120:49:511
598625813,26sleep10-21swapper/120:49:511
1450625816,29sleep314565-21processes21:15:163
468225717,24sleep34675-21apt-get20:45:043
468225717,24sleep34675-21apt-get20:45:043
3041925720,20sleep00-21swapper/020:24:510
2340125716,24sleep023394-21apt-get21:45:050
1706925721,20sleep10-21swapper/123:05:211
1706925721,20sleep10-21swapper/123:05:211
1288025722,18sleep10-21swapper/100:35:171
1044525715,26sleep310529-21sed22:45:203
839225617,18sleep00-21swapper/000:20:220
3237125611,28sleep332373-21/usr/sbin/munin23:55:013
3155625615,18sleep10-21swapper/123:50:431
3155625615,18sleep10-21swapper/123:50:431
2996025623,18sleep10-21swapper/122:05:081
267842567,27sleep10-21swapper/121:55:391
1398125616,24sleep113987-21ntp_states19:30:161
1134125612,20sleep10-21swapper/100:30:161
2569225515,24sleep025686-21apt-get23:35:030
1983525515,18sleep00-21swapper/019:50:100
1696625516,23sleep017068-21sed23:05:200
1696625516,23sleep017068-21sed23:05:200
155125514,25sleep31544-21apt-get20:35:033
997425416,18sleep00-21swapper/022:45:090
623225413,24sleep36225-21apt-get20:50:053
623225413,24sleep36225-21apt-get20:50:053
2461125415,25sleep30-21swapper/323:30:183
1798925414,24sleep317993-21diskstats23:10:073
1798925414,24sleep317993-21diskstats23:10:073
15582545,37sleep00-21swapper/022:16:580
1352625414,24sleep013521-21apt-get19:30:030
715525312,29sleep37191-21sed22:35:153
364425312,24sleep33637-21apt-get22:25:013
364425312,24sleep33637-21apt-get22:25:013
1299225313,21sleep00-21swapper/021:10:170
31725212,22sleep3472-21modprobe22:14:513
2916825211,23sleep029221-21users22:00:230
217525218,18sleep00-21swapper/022:20:050
1819725213,24sleep318193-21apt-get19:45:043
1790925215,23sleep017902-21apt-get23:10:040
1790925215,23sleep017902-21apt-get23:10:040
1534925213,23sleep015385-21munin-run21:19:490
129125212,24sleep01429-21sed23:59:500
1248525212,24sleep112479-21apt-get21:10:011
932825111,23sleep09323-21apt-get00:25:020
3239525113,23sleep132392-21apt-get23:55:031
247992518,11sleep10-21swapper/120:05:131
901225012,24sleep19013-21sendmail-mta20:59:191
901225012,24sleep19013-21sendmail-mta20:59:191
1069825011,18sleep30-21swapper/319:20:133
388624910,17sleep00-21swapper/000:05:200
3139724915,20sleep10-21swapper/122:10:011
1197324912,22sleep111968-21apt-get19:25:031
1894124815,14sleep10-21swapper/119:45:231
135922485,34sleep30-21swapper/300:38:343
3181824716,17sleep30-21swapper/322:10:153
2389224711,21sleep323901-21munin-run23:29:493
1764224713,20sleep117682-21ps21:25:201
1175424712,15sleep10-21swapper/119:24:511
32072467,15sleep10-21swapper/120:40:061
527924410,20sleep05288-21df_inode22:30:110
527924410,20sleep05288-21df_inode22:30:110
1141224411,13sleep30-21swapper/321:05:193
85972429,15sleep00-21swapper/020:55:230
85972429,15sleep00-21swapper/020:55:230
1371424110,14sleep30-21swapper/319:30:103
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional