You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-29 - 20:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Mon Jun 29, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20781256786,5660sleep116228-21kworker/1:112:05:581
18040275717,724sleep018048-21/usr/sbin/munin08:34:510
14635263014,601sleep114696-21find10:05:091
587260811,484sleep1736-21tail09:20:111
25812252716,499sleep125858-21latency_hist12:24:401
2095325107,490sleep120928-21latency_hist08:44:411
2095325107,490sleep120928-21latency_hist08:44:411
531324058,346sleep15329-21modprobe07:54:381
3054324049,380sleep130684-21sed09:14:411
2627023836,361sleep026313-21/usr/sbin/munin09:00:030
7728234812,316sleep27794-21/usr/sbin/munin08:00:082
6286233529,200sleep10-21swapper/111:20:071
13290228513,254sleep213338-21unin-run11:44:392
8944227518,198sleep18959-21cron09:49:381
26224226473,171sleep126263-21/usr/sbin/munin10:40:181
3293222815,194sleep13444-21unixbench_singl07:45:581
19601221825,177sleep119753-21date12:04:411
4198221314,188sleep124766-21munin-node11:14:501
3720220820,131sleep13867-21grep09:30:121
3720220820,131sleep13867-21grep09:30:121
15648220122,92sleep115778-21unixbench_multi08:25:111
629221910,11sleep112-21rcu_sched09:39:561
25000218913,131sleep125090-21fschecks_time07:14:451
18958218510,73sleep119010-21latency_hist10:19:421
18958218510,73sleep119010-21latency_hist10:19:421
13203218420,59sleep10-21swapper/108:19:421
13420217238,121sleep113459-21cpu08:19:541
10546216918,137sleep210629-21latency_hist09:54:402
11033216810,123sleep111154-21unixbench_singl08:10:131
1714121639,120sleep117179-21grep11:55:081
9222216231,117sleep19289-21run-parts08:05:081
1390821598,74sleep1364-21snmpd11:45:031
11650215910,132sleep111757-21timerandwakeup09:55:241
15913215376,56sleep10-21swapper/110:09:491
23897214938,95sleep10-21swapper/107:10:081
26314214536,94sleep10-21swapper/109:00:041
839821446,89sleep1364-21snmpd11:26:501
839821446,89sleep1364-21snmpd11:26:501
30173214427,107sleep1364-21snmpd12:36:351
31364214048,80sleep131429-21cat09:15:101
12492214043,77sleep10-21swapper/111:40:061
23820213674,20sleep124033-21latency_hist10:34:411
23080213412,40sleep122881-21/usr/sbin/munin12:14:591
29237213316,96sleep329314-21fschecks_time12:35:003
2779921339,103sleep00-21swapper/007:20:130
10216213016,90sleep00-21swapper/011:34:400
3809212955,39sleep03798-21/usr/sbin/munin07:49:400
27807212814,99sleep10-21swapper/110:45:061
27807212814,99sleep10-21swapper/110:45:061
31552212612,98sleep331664-21unixbench_singl09:15:123
10495212629,84sleep110543-21apt-config09:54:391
32129212312,98sleep032241-21cat09:19:410
22187212018,21sleep20-21swapper/212:10:112
22617211938,58sleep10-21swapper/112:14:381
18163211925,20sleep118249-21fschecks_time08:35:001
30654211638,23sleep10-21swapper/107:30:111
27353211616,19sleep10-21swapper/107:20:061
19615211612,18sleep00-21swapper/008:39:480
19615211612,18sleep00-21swapper/008:39:480
1710211610,87sleep10-21swapper/111:05:091
1710211610,87sleep10-21swapper/111:05:091
15495211610,92sleep015534-21ntpq11:50:020
4847211514,88sleep24853-21cat07:50:562
25921211512,88sleep325941-21sh08:59:533
15609211419,17sleep10-21swapper/111:50:071
347211213,80sleep00-21swapper/009:20:050
12877211116,79sleep312917-21latency10:00:063
12877211116,79sleep312917-21latency10:00:063
23083211022,64sleep0225-21systemd-journal07:09:390
26706210923,70sleep026757-21latency_hist07:19:400
6405210812,83sleep36488-21sed09:39:593
26582210517,67sleep025684-21/usr/sbin/munin10:40:250
1378521058,86sleep1364-21snmpd10:03:351
1378521058,86sleep1364-21snmpd10:03:351
15674210415,74sleep215747-21wc11:50:082
5605210326,62sleep15629-21grep07:54:541
29696210314,71sleep029719-21/usr/sbin/munin09:10:080
2509210215,69sleep32492-21apt-get07:44:553
73821015,46sleep10-21swapper/111:03:241
73821015,46sleep10-21swapper/111:03:241
22422210016,66sleep222396-21apt-get10:29:572
20555210010,54sleep10-21swapper/110:24:431
20555210010,54sleep10-21swapper/110:24:431
204621004,11sleep20-21swapper/211:05:182
204621004,11sleep20-21swapper/211:05:182
1631729912,71sleep316326-21needreboot10:10:113
1542829918,66sleep315523-21apt-key10:09:393
2768129811,64sleep327570-21/usr/sbin/munin12:29:553
2223329824,42sleep10-21swapper/108:46:511
2223329824,42sleep10-21swapper/108:46:511
3163529723,43sleep10-21swapper/110:58:471
3134529716,16sleep20-21swapper/207:34:402
2991029722,42sleep10-21swapper/110:51:281
294429660,17sleep10-21swapper/111:10:021
2859429622,41sleep10-21swapper/112:31:141
2736129621,15sleep10-21swapper/112:29:411
2474529618,51sleep224766-21munin-node07:14:422
243129621,42sleep10-21swapper/109:25:561
243129621,42sleep10-21swapper/109:25:561
2341129620,45sleep323406-21apt-get07:09:543
2193429620,57sleep022062-21grep08:45:130
2193429620,57sleep022062-21grep08:45:130
587629523,50sleep05947-21latency07:55:020
1834229513,67sleep018352-21forks12:00:000
173829524,39sleep10-21swapper/107:41:341
3269429415,13sleep00-21swapper/011:00:080
3269429415,13sleep00-21swapper/011:00:080
3044329411,17sleep00-21swapper/010:54:550
289592936,54sleep128973-21awk07:25:061
289592936,54sleep128973-21awk07:25:061
195252938,48sleep10-21swapper/110:20:151
195252938,48sleep10-21swapper/110:20:151
2870129221,44sleep10-21swapper/109:06:421
2645529212,65sleep326577-21cat10:40:243
2434829215,55sleep224372-21fschecks_count10:35:062
81962915,11sleep10-21swapper/108:01:251
56022918,17sleep30-21swapper/307:54:523
160172900,9sleep012-21rcu_sched10:09:550
117529010,67sleep21219-21systemd-run09:24:402
2359828921,41sleep023759-21unixbench_multi12:15:100
2217128912,61sleep322173-21gpgv10:29:423
2217128912,61sleep322173-21gpgv10:29:423
1884228913,64sleep318888-21wc10:19:403
1884228913,64sleep318888-21wc10:19:403
307928816,52sleep02651-21/usr/sbin/munin11:10:050
251542886,11sleep10-21swapper/110:35:291
2240428816,25sleep00-21swapper/010:29:510
936828721,44sleep39485-21grep08:05:103
316012877,46sleep10-21swapper/107:34:551
2885928728,36sleep228879-21modprobe12:34:382
975128612,58sleep39813-21awk09:50:093
2727228616,47sleep227462-21sed09:04:412
100262867,47sleep10-21swapper/111:32:511
707128524,40sleep07107-21basename11:24:380
237912846,43sleep10-21swapper/108:51:521
237912846,43sleep10-21swapper/108:51:521
253922816,43sleep10-21swapper/108:57:331
1648828111,54sleep216507-21apt-get08:29:482
104282815,43sleep10-21swapper/111:34:491
175042806,43sleep10-21swapper/108:31:081
1688328017,45sleep316916-21/usr/sbin/munin08:30:013
162028010,53sleep21633-21awk07:40:142
1055028013,51sleep310647-21fschecks_time11:34:593
88812789,32sleep30-21swapper/311:29:503
83627813,48sleep2851-21modprobe11:04:382
83627813,48sleep2851-21modprobe11:04:382
56062785,41sleep10-21swapper/109:36:541
464627821,26sleep30-21swapper/309:34:583
2767327714,17sleep00-21swapper/009:04:550
242827714,46sleep32465-21latency_hist11:09:393
242827714,46sleep32465-21latency_hist11:09:393
2266727725,23sleep30-21swapper/312:14:403
1862627716,41sleep218665-21ntpq12:00:062
2716327614,17sleep00-21swapper/010:44:410
2716327614,17sleep00-21swapper/010:44:410
267227610,49sleep30-21swapper/311:09:523
2505627615,34sleep30-21swapper/312:20:103
2315527619,43sleep223235-21cut07:09:412
2179627629,29sleep30-21swapper/307:04:503
182727613,47sleep01856-21/usr/sbin/munin09:25:020
182727613,47sleep01856-21/usr/sbin/munin09:25:020
2903227518,39sleep329097-21grep07:25:083
2903227518,39sleep329097-21grep07:25:083
207127519,42sleep32187-21sed09:25:093
207127519,42sleep32187-21sed09:25:093
1694527526,25sleep30-21swapper/311:55:023
1541327516,44sleep215425-21sh10:09:382
2891127417,25sleep00-21swapper/012:34:400
1488427421,28sleep20-21swapper/211:49:402
865827324,26sleep00-21swapper/011:29:390
865827324,26sleep00-21swapper/011:29:390
2871327319,26sleep00-21swapper/010:49:420
2555327320,34sleep025583-21grep08:59:380
1399627314,26sleep00-21swapper/008:20:090
12722730,7sleep30-21swapper/307:40:103
899527217,28sleep30-21swapper/309:49:403
39427221,32sleep0422-21munin-run07:39:390
3020427215,43sleep230264-21latency_hist10:54:402
2892927217,34sleep328935-21modprobe09:09:393
1704927212,41sleep017119-21/usr/sbin/munin08:30:060
1415927221,24sleep00-21swapper/011:45:100
1085127219,20sleep00-21swapper/011:35:040
2127627117,40sleep221300-21fschecks_count08:44:582
2127627117,40sleep221300-21fschecks_count08:44:582
1037527117,22sleep00-21swapper/008:10:000
2273827015,25sleep00-21swapper/008:49:530
2273827015,25sleep00-21swapper/008:49:530
1392427014,39sleep313994-21/usr/sbin/munin08:20:083
1229027020,27sleep30-21swapper/311:40:003
1192727019,24sleep20-21swapper/208:14:582
1104227021,24sleep20-21swapper/209:55:082
613626916,25sleep00-21swapper/009:39:520
224542696,51sleep00-21swapper/012:12:310
1241626920,18sleep30-21swapper/309:59:423
2918326815,23sleep30-21swapper/310:50:053
2024626824,26sleep30-21swapper/308:40:073
2024626824,26sleep30-21swapper/308:40:073
1086626822,23sleep30-21swapper/308:10:113
950526717,19sleep00-21swapper/009:50:010
540026717,21sleep00-21swapper/007:54:410
2962326715,28sleep00-21swapper/012:35:090
2438526723,19sleep30-21swapper/308:55:013
135026712,34sleep00-21swapper/007:40:120
852826618,23sleep30-21swapper/308:04:423
276226618,25sleep20-21swapper/211:09:592
2229026621,24sleep00-21swapper/010:29:430
2229026621,24sleep00-21swapper/010:29:430
782326512,31sleep30-21swapper/311:25:063
782326512,31sleep30-21swapper/311:25:063
2004726518,26sleep20-21swapper/208:40:022
2004726518,26sleep20-21swapper/208:40:022
1974026516,27sleep20-21swapper/210:20:202
1974026516,27sleep20-21swapper/210:20:202
105402659,13sleep20-21swapper/211:34:582
578126417,26sleep30-21swapper/311:19:493
46222649,13sleep30-21swapper/311:15:053
405326420,21sleep30-21swapper/307:49:523
2317826416,34sleep323238-21latency12:15:013
1735626418,28sleep00-21swapper/011:55:110
1310326414,32sleep313173-21systemd-run08:19:403
2472126318,28sleep324784-21/usr/sbin/munin10:35:233
235152630,8sleep00-21swapper/007:09:580
1492326319,28sleep315074-21latency_hist11:49:413
3135626217,19sleep30-21swapper/307:34:413
2804326219,20sleep20-21swapper/209:05:052
2586126221,19sleep20-21swapper/212:24:412
2563726211,33sleep325731-21memory07:15:033
2563326220,23sleep20-21swapper/208:59:402
1462126212,20sleep00-21swapper/010:05:080
712126116,24sleep20-21swapper/211:24:412
448526117,22sleep00-21swapper/011:15:020
2405626114,23sleep30-21swapper/308:54:403
2405626114,23sleep30-21swapper/308:54:403
2034226116,29sleep020355-21idleruntime-cro10:24:380
2034226116,29sleep020355-21idleruntime-cro10:24:380
576526017,27sleep25825-21/usr/sbin/munin07:55:002
2865526021,17sleep30-21swapper/310:49:413
2865526021,17sleep30-21swapper/310:49:413
2592126014,23sleep20-21swapper/207:15:102
1955726014,19sleep30-21swapper/310:20:173
1955726014,19sleep30-21swapper/310:20:173
2297925914,20sleep20-21swapper/212:14:562
791425817,18sleep00-21swapper/008:00:100
328325824,16sleep00-21swapper/009:30:030
328325824,16sleep00-21swapper/009:30:030
2276025819,17sleep30-21swapper/310:30:553
1720425815,18sleep20-21swapper/210:14:392
1504125821,20sleep20-21swapper/208:24:562
45572577,18sleep00-21swapper/009:34:510
325162574,15sleep30-21swapper/307:35:183
2931125713,25sleep229317-21diskstats09:09:582
258972579,32sleep225900-21/usr/sbin/munin10:40:112
2000125721,20sleep30-21swapper/312:04:593
1904725717,17sleep20-21swapper/210:19:512
1904725717,17sleep20-21swapper/210:19:512
1004725717,18sleep20-21swapper/208:09:412
556925619,16sleep20-21swapper/211:19:402
455925615,24sleep24552-21apt-get09:34:532
2942025613,27sleep229461-21ls12:35:032
264525613,27sleep22649-21diskstats07:45:392
1980725614,26sleep019801-21apt-get12:04:520
1509925610,19sleep30-21swapper/311:49:493
1505425614,20sleep00-21swapper/008:24:580
122125618,18sleep30-21swapper/309:24:413
3178325515,18sleep20-21swapper/210:59:412
193572556,16sleep30-21swapper/312:02:333
1273025517,18sleep00-21swapper/010:00:030
1273025517,18sleep00-21swapper/010:00:030
1268525515,24sleep212692-21diskstats10:00:012
1268525515,24sleep212692-21diskstats10:00:012
325025411,25sleep33281-21irqstats09:30:023
325025411,25sleep33281-21irqstats09:30:023
3246125413,23sleep332472-21ntp_states11:00:023
3246125413,23sleep332472-21ntp_states11:00:023
3105425411,16sleep20-21swapper/209:15:032
3045225415,22sleep00-21swapper/007:30:060
2931825412,20sleep30-21swapper/309:09:593
2755525417,23sleep20-21swapper/207:20:112
2605025415,23sleep226042-21apt-get12:24:532
2477525413,23sleep024869-21memory12:20:030
1825525418,16sleep20-21swapper/208:35:012
143432546,36sleep20-21swapper/208:20:192
133625414,25sleep01393-21irqstats11:05:000
133625414,25sleep01393-21irqstats11:05:000
1100825412,26sleep010512-21apt-get09:54:500
768625312,24sleep07680-21apt-get09:44:530
739625315,22sleep27424-21munin-run09:44:382
3052025312,26sleep330527-21/usr/sbin/munin10:54:583
2328125314,23sleep223352-21sensors08:50:072
2328125314,23sleep223352-21sensors08:50:072
2189225315,19sleep10-21swapper/107:04:561
1992925315,20sleep20-21swapper/212:04:582
1035625314,21sleep20-21swapper/208:09:582
890625212,24sleep28900-21apt-get11:29:532
803825217,19sleep20-21swapper/211:25:092
803825217,19sleep20-21swapper/211:25:092
3208125212,18sleep20-21swapper/207:35:092
2651225219,18sleep30-21swapper/312:25:063
2062625114,22sleep220628-21apt-get10:24:522
2062625114,22sleep220628-21apt-get10:24:522
87382509,17sleep00-21swapper/008:04:550
2771525014,18sleep20-21swapper/210:45:032
2771525014,18sleep20-21swapper/210:45:032
2130225010,16sleep30-21swapper/308:44:593
2130225010,16sleep30-21swapper/308:44:593
1831625012,15sleep30-21swapper/308:35:023
2725124913,21sleep327318-21memory07:20:043
2720624910,24sleep027207-21sshd12:28:180
2465624811,21sleep024693-21ntp_states10:35:200
132782484,35sleep00-21swapper/011:44:290
1816424713,15sleep00-21swapper/010:15:170
1816424713,15sleep00-21swapper/010:15:170
145824710,21sleep21454-21apt-get09:24:522
145824710,21sleep21454-21apt-get09:24:522
2881324612,18sleep00-21swapper/007:25:010
2881324612,18sleep00-21swapper/007:25:010
1208324511,19sleep012105-21irqstats08:15:000
319424411,19sleep22985-21/usr/sbin/munin09:30:012
319424411,19sleep22985-21/usr/sbin/munin09:30:012
1627924410,14sleep30-21swapper/308:29:413
896224211,14sleep20-21swapper/209:49:392
287222407,15sleep20-21swapper/207:24:592
287222407,15sleep20-21swapper/207:24:592
167532389,15sleep20-21swapper/211:54:572
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional