You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-11 - 14:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Fri Sep 11, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
240322335522,3295sleep024018-21kworker/u8:300:23:270
1691262211,589sleep04996-1kworker/0:0H23:13:200
2049024636,447sleep1367-21snmpd22:30:201
21103244410,420sleep321112-21ntpq00:13:373
31637238313,354sleep131776-21unixbench_singl19:38:431
31637238313,354sleep131776-21unixbench_singl19:38:431
28785234518,267sleep128845-21find21:13:421
12933233933,293sleep112960-21modprobe22:08:111
748723218,294sleep17526-21http20:08:111
24843231324,274sleep124941-21unixbench_multi00:24:331
20825230614,274sleep31-21systemd19:08:123
25668229556,214sleep125730-21fschecks_time00:28:291
14859229232,198sleep115012-21unixbench_multi23:53:441
13996228093,127sleep10-21swapper/122:09:121
13996228093,127sleep10-21swapper/122:09:121
2141226823,227sleep32151-21/usr/sbin/munin23:14:203
4619226118,228sleep34704-21latency_hist23:23:133
29641225917,227sleep229651-21unin-run21:18:122
8428225816,226sleep08493-21awk23:33:440
1893022420,6sleep30-21swapper/320:43:263
28721224133,187sleep128776-21sendmail22:58:111
28721224133,187sleep128776-21sendmail22:58:111
10303223416,155sleep110438-21grep20:13:491
22712223148,83sleep10-21swapper/100:18:401
22700223039,171sleep122841-21unixbench_multi20:53:431
22700223039,171sleep122841-21unixbench_multi20:53:431
2980322249,200sleep129874-21mailstats22:58:521
567322218,202sleep15738-21awk21:43:391
27754222118,58sleep10-21swapper/119:28:211
32744222044,81sleep10-21swapper/121:28:121
32744222044,81sleep10-21swapper/121:28:121
17523221715,190sleep117533-21acpi00:03:211
32522221629,174sleep132533-21/usr/sbin/munin19:43:281
31441220911,181sleep331515-21find23:03:423
7713220631,79sleep10-21swapper/123:33:121
449219818,165sleep3456-21ntp_states19:43:383
3779219723,43sleep13918-21unixbench_multi19:53:421
9720219279,96sleep19870-21cp21:58:131
30840217915,55sleep10-21swapper/123:03:281
17269217577,22sleep10-21swapper/122:18:451
21810217412,146sleep021952-21sed20:53:140
15960217034,84sleep10-21swapper/120:33:321
7984216914,139sleep37504-21apt-get20:08:223
10210216182,64sleep10-21swapper/123:38:421
1312215965,28sleep10-21swapper/123:13:131
26471215754,87sleep126580-21sed21:08:131
191021500,19sleep120907-21cyclictest21:33:121
15502214948,88sleep115549-21systemd-run20:33:111
1547214314,113sleep01543-21apt-get19:48:230
24613213953,74sleep124678-21mailstats22:43:411
10276213822,92sleep010280-21grep21:58:410
2870213757,56sleep10-21swapper/121:33:461
28015213521,97sleep228030-21idleruntime-cro21:13:112
2132721358,85sleep10-21swapper/120:48:461
11436213448,45sleep111565-21latency_hist22:03:141
2864213049,44sleep13012-21latency_hist19:53:131
4945212742,42sleep10-21swapper/119:58:341
3340212618,90sleep2314-21rs:main2
12713212645,69sleep112775-21fschecks_time23:48:311
12387212622,82sleep212533-21sed22:03:442
12495212432,71sleep112635-21latency_hist20:23:141
12495212432,71sleep112635-21latency_hist20:23:141
11444212418,53sleep10-21swapper/123:43:341
16118212348,40sleep10-21swapper/122:18:121
9291212011,94sleep19406-21latency_hist23:38:131
1579021206,12sleep00-21swapper/020:33:270
28174211834,21sleep10-21swapper/100:35:281
29807211718,79sleep229871-21sendmail_mailqu19:33:392
29807211718,79sleep229871-21sendmail_mailqu19:33:392
2744821177,86sleep10-21swapper/121:08:421
4568211426,73sleep14596-21idleruntime-cro23:23:111
15691211419,74sleep315454-21apt-get23:58:143
31142211317,73sleep330880-21/usr/sbin/munin19:38:333
31142211317,73sleep330880-21/usr/sbin/munin19:38:333
20337211316,41sleep10-21swapper/100:12:541
20337211316,41sleep10-21swapper/100:12:541
2879221115,54sleep00-21swapper/019:29:580
1369211118,69sleep20-21swapper/219:48:132
17626211018,75sleep017629-21df_inode00:03:280
583210813,76sleep0608-21/usr/sbin/munin21:28:280
12693210826,19sleep10-21swapper/120:23:261
28405210720,67sleep028506-21fschecks_time21:13:330
26200210519,22sleep10-21swapper/122:48:421
21469210527,17sleep10-21swapper/122:33:391
5888210316,62sleep06035-21grep21:43:420
25650210114,26sleep325659-21df_inode00:28:273
20170210119,68sleep220194-21idleruntime-cro20:48:112
15449210113,74sleep015525-21apt-config23:58:110
931629949,38sleep120907-21cyclictest20:13:121
3092529912,73sleep030950-21fschecks_count23:03:310
2228729915,67sleep022329-21latency20:53:340
2228729915,67sleep022329-21latency20:53:340
684429817,63sleep06881-21fschecks_count21:48:310
649929820,51sleep36587-21fschecks_time23:28:343
616629622,60sleep26141-21latency_hist23:28:122
2515129510,70sleep025220-21df21:03:270
2291029516,58sleep222469-21munin-node19:13:202
2202729528,39sleep20-21swapper/200:18:122
65229423,48sleep0797-21grep23:08:440
2082029437,36sleep220824-21cron19:08:112
1945229413,68sleep019467-21cpu22:28:250
2668129313,66sleep026692-21sort21:08:200
806329211,44sleep18102-21sed21:53:121
2778229016,60sleep327805-21grep19:28:273
2124829012,62sleep321356-21fschecks_time19:08:323
1766329010,67sleep017710-21latency_hist22:23:120
3118928919,50sleep231210-21latency_hist21:23:122
3118928919,50sleep231210-21latency_hist21:23:122
237382897,72sleep1367-21snmpd22:41:511
236752899,65sleep223679-21diskstats20:58:292
2979228821,52sleep10-21swapper/119:33:381
2979228821,52sleep10-21swapper/119:33:381
485528725,36sleep34850-21apt-get23:23:253
914228612,60sleep29144-21sshd20:11:282
470128618,27sleep10-21swapper/121:40:241
2683928614,59sleep226885-21sh00:33:112
2731028515,30sleep10-21swapper/119:25:511
1726028518,42sleep217275-21apt00:03:102
804528417,52sleep38106-21fschecks_time23:33:353
2839428425,31sleep20-21swapper/219:28:432
54528213,45sleep30-21swapper/323:08:433
256602827,21sleep1367-21snmpd19:19:331
231262829,13sleep10-21swapper/119:13:271
1705328219,32sleep10-21swapper/100:00:301
497228120,29sleep30-21swapper/321:43:133
2538628122,42sleep225414-21munin-run22:48:102
2359128124,15sleep10-21swapper/120:58:241
61552808,16sleep16165-21munin-run23:28:111
21982807,62sleep12235-21ntp_states23:14:221
1489028024,39sleep214990-21timerwakeupswit20:28:442
1477528019,20sleep20-21swapper/222:13:292
2246327925,37sleep022508-21latency_hist19:13:140
1559227911,52sleep315733-21cat20:33:133
2258227823,25sleep30-21swapper/322:38:263
223327817,36sleep32376-21grep19:48:433
1891127823,13sleep10-21swapper/120:43:251
987827724,30sleep20-21swapper/221:58:142
214082776,11sleep10-21swapper/119:08:341
209662775,11sleep10-21swapper/119:08:141
1743527720,35sleep017441-21df_inode20:38:280
3015227611,48sleep10-21swapper/121:18:351
3015227611,48sleep10-21swapper/121:18:351
2368027624,31sleep30-21swapper/320:58:303
2815027519,38sleep228304-21sed22:53:442
2815027519,38sleep228304-21sed22:53:442
2598427523,24sleep20-21swapper/219:23:132
2233127526,25sleep00-21swapper/000:18:310
1134427512,49sleep311400-21latency23:43:323
363227415,42sleep23653-21dpkg21:38:232
306682749,38sleep00-21swapper/019:38:140
306682749,38sleep00-21swapper/019:38:140
2690527416,31sleep20-21swapper/219:23:422
217032747,11sleep10-21swapper/100:15:211
1947327411,50sleep019077-21/usr/sbin/munin00:08:370
1947327411,50sleep019077-21/usr/sbin/munin00:08:370
178082748,15sleep10-21swapper/120:38:371
1712627414,39sleep317254-21/usr/sbin/munin22:18:433
810327224,32sleep28245-21latency_hist21:53:132
435727220,30sleep24413-21awk19:58:112
2475227220,29sleep30-21swapper/322:43:433
190362726,10sleep10-21swapper/122:26:321
532027120,35sleep05358-21grep23:23:380
3038827121,27sleep00-21swapper/021:18:420
3038827121,27sleep00-21swapper/021:18:420
1957427120,26sleep00-21swapper/019:03:370
1323027113,25sleep30-21swapper/320:23:403
837227013,41sleep08461-21/usr/sbin/munin20:09:360
116092709,16sleep20-21swapper/223:43:392
114027012,24sleep20-21swapper/221:28:422
68326912,37sleep0811-21unixbench_multi19:43:430
339626914,37sleep03439-21fschecks_count23:18:290
2350126913,40sleep323539-21http00:23:103
1245426913,43sleep012423-21latency_hist20:23:130
1245426913,43sleep012423-21latency_hist20:23:130
1083126919,33sleep310861-21modprobe20:18:113
838826817,29sleep20-21swapper/221:53:302
2925026813,21sleep00-21swapper/022:58:220
272772687,12sleep30-21swapper/300:33:323
1952426815,23sleep20-21swapper/222:28:272
1116126819,26sleep00-21swapper/023:43:290
752726720,30sleep27548-21munin-run20:08:122
2857726716,35sleep328637-21latency21:13:363
1955826721,26sleep30-21swapper/300:08:403
1955826721,26sleep30-21swapper/300:08:403
2662726618,27sleep00-21swapper/019:23:360
234992666,48sleep00-21swapper/022:38:470
2154126613,41sleep30-21swapper/320:50:463
1736626616,23sleep30-21swapper/320:38:263
1306426610,29sleep20-21swapper/220:23:352
127962668,44sleep312797-21sshd22:06:273
363026520,25sleep33700-21/usr/sbin/munin19:53:403
2721426526,21sleep30-21swapper/322:53:123
1798826516,25sleep20-21swapper/222:23:312
1712926520,26sleep20-21swapper/220:38:132
537826419,25sleep20-21swapper/219:58:432
312122640,7sleep30-21swapper/321:23:133
312122640,7sleep30-21swapper/321:23:133
2929326413,31sleep329270-21apt-get22:58:333
956026317,27sleep39628-21http21:58:113
508426312,36sleep05130-21ntp_states19:58:370
443726318,19sleep00-21swapper/019:58:130
3152426320,19sleep30-21swapper/321:23:343
3152426320,19sleep30-21swapper/321:23:343
266032637,12sleep00-21swapper/000:30:100
1603826318,23sleep30-21swapper/322:18:103
1083626314,33sleep310902-21date23:43:123
105072636,44sleep00-21swapper/020:14:080
971726219,21sleep20-21swapper/223:38:302
2783426211,36sleep327440-21/usr/sbin/munin22:53:363
2783426211,36sleep327440-21/usr/sbin/munin22:53:363
2640526220,19sleep30-21swapper/321:08:113
2346026215,22sleep00-21swapper/019:13:340
208326218,25sleep32076-21apt-get21:33:273
1277926221,20sleep20-21swapper/223:48:322
2759126115,20sleep20-21swapper/200:33:392
2723426111,26sleep00-21swapper/022:53:130
2545126123,20sleep00-21swapper/022:48:120
2389126111,27sleep20-21swapper/222:43:132
1804026121,20sleep20-21swapper/220:38:422
1731126118,20sleep00-21swapper/000:03:120
1400826122,19sleep00-21swapper/020:28:140
756626020,18sleep30-21swapper/320:08:133
65182600,7sleep20-21swapper/221:48:122
2803126018,26sleep328064-21munin-run21:13:123
256572606,15sleep30-21swapper/319:19:313
1633026015,20sleep20-21swapper/222:18:212
1524326011,25sleep00-21swapper/022:13:430
472425919,23sleep34748-21df19:58:283
3201525921,20sleep20-21swapper/221:23:452
3201525921,20sleep20-21swapper/221:23:452
2407725910,21sleep20-21swapper/200:23:582
2376425920,20sleep00-21swapper/020:58:310
2079525918,18sleep30-21swapper/322:33:133
182025914,22sleep20-21swapper/219:48:342
3087925813,29sleep030897-21apt19:38:220
3087925813,29sleep030897-21apt19:38:220
3238525713,26sleep232378-21apt-get23:08:252
2335525721,19sleep20-21swapper/220:58:122
2335525721,19sleep20-21swapper/220:58:122
2192425716,21sleep00-21swapper/000:18:100
1596525719,24sleep315454-21apt-get23:58:223
1560825719,23sleep215690-21apt-config23:58:132
976925618,16sleep20-21swapper/220:13:382
903025614,22sleep00-21swapper/021:53:440
745425611,31sleep0367-21snmpd23:30:290
3083625615,25sleep230832-21apt-get23:03:252
2161425621,17sleep00-21swapper/022:33:420
2121025619,19sleep20-21swapper/222:33:322
2073225614,24sleep220791-21/usr/sbin/munin20:48:352
1861325615,20sleep30-21swapper/322:23:443
171425616,25sleep21709-21apt-get23:13:222
1714025615,16sleep30-21swapper/320:38:143
342025515,24sleep03475-21sed21:38:130
3221225517,18sleep20-21swapper/219:43:122
3221225517,18sleep20-21swapper/219:43:122
2720425511,17sleep00-21swapper/000:33:290
2371125518,20sleep30-21swapper/319:13:413
1811125515,19sleep00-21swapper/022:23:330
1755025512,19sleep20-21swapper/200:03:262
1474225518,22sleep014820-21mailstats20:28:420
143882555,11sleep00-21swapper/022:11:570
143882555,11sleep00-21swapper/022:11:570
1307525512,24sleep013082-21ntp_states23:48:380
680525417,18sleep00-21swapper/020:03:420
2592425418,19sleep20-21swapper/222:48:302
2123525415,18sleep20-21swapper/219:08:312
1455925415,16sleep30-21swapper/322:13:143
1455925415,16sleep30-21swapper/322:13:143
1452825415,24sleep214558-21date22:13:132
1452825415,24sleep214558-21date22:13:132
1194125411,27sleep011955-21/usr/sbin/munin22:03:330
565825316,22sleep35659-21ntp_states21:43:373
2224025314,23sleep222236-21apt-get00:18:252
1977025315,23sleep019816-21users20:43:450
952625211,25sleep09520-21apt-get23:38:240
684125212,24sleep36842-21diskstats21:48:303
417125213,22sleep00-21swapper/021:38:410
3273425215,22sleep032743-21munin-run21:28:110
3273425215,22sleep032743-21munin-run21:28:110
292912526,19sleep20-21swapper/222:58:322
2748925211,24sleep327520-21munin-run19:28:113
99525111,25sleep31002-21/usr/sbin/munin21:28:393
2932625112,23sleep329322-21apt-get19:33:243
2932625112,23sleep329322-21apt-get19:33:243
2799925113,23sleep028068-21sensors_temp22:53:420
2799925113,23sleep028068-21sensors_temp22:53:420
2468625115,21sleep00-21swapper/019:18:270
2433125111,24sleep324369-21irqstats00:24:233
1114925111,16sleep20-21swapper/220:18:262
1114925111,16sleep20-21swapper/220:18:262
2100525011,23sleep321001-21apt-get22:33:253
1269925012,16sleep30-21swapper/323:48:293
371124913,17sleep20-21swapper/223:18:352
1011924815,14sleep30-21swapper/321:58:273
1918424712,20sleep219188-21diskstats00:08:312
1918424712,20sleep219188-21diskstats00:08:312
841224610,21sleep28414-21ntp_states23:33:422
667924513,17sleep20-21swapper/223:28:372
2514824510,21sleep225144-21apt-get21:03:252
1851724510,19sleep318518-21sshd00:04:283
1121724211,15sleep30-21swapper/320:18:273
1121724211,15sleep30-21swapper/320:18:273
220142388,15sleep30-21swapper/320:53:253
220142388,15sleep30-21swapper/320:53:253
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional