You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-16 - 23:17

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Thu Apr 16, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25622266438,6622sleep16418-21kworker/1:208:32:371
25622266438,6622sleep16418-21kworker/1:208:32:371
318392123226,1168sleep331877-21http12:21:133
2971026966,655sleep329751-21ntpq08:46:403
29147255245,482sleep129174-21cron07:06:131
2281625026,462sleep313441-21snmpd10:08:063
15453246618,431sleep115481-21/usr/sbin/munin08:01:391
3441244515,414sleep03527-21apt-config10:51:160
20613243724,396sleep220622-21/usr/sbin/munin10:01:392
3491242513,394sleep23539-21ntpq09:06:382
22244234346,248sleep322374-21unixbench_multi08:21:563
13607233642,280sleep313631-21fschecks_count07:56:323
25982230912,281sleep025926-21/usr/sbin/munin08:36:150
25982230912,281sleep025926-21/usr/sbin/munin08:36:150
14676229233,166sleep314797-21cat09:41:453
1307922697,224sleep313169-21sort09:36:443
8605223447,134sleep30-21swapper/311:06:333
5174222531,149sleep35288-21latency_hist10:56:163
5174222531,149sleep35288-21latency_hist10:56:163
62522219128,73sleep30-21swapper/309:16:293
20623221622,141sleep320660-21ntpq10:01:403
32135220844,110sleep332178-21run-parts08:56:133
5961220753,117sleep36023-21uname07:31:443
1496422057,104sleep30-21swapper/311:26:253
5814220051,93sleep30-21swapper/310:56:403
28738219711,171sleep228769-21latency_hist12:11:142
28738219711,171sleep228769-21latency_hist12:11:142
7314219642,92sleep30-21swapper/307:36:363
7314219642,92sleep30-21swapper/307:36:363
13813218931,61sleep313838-21awk11:21:403
21934218612,160sleep021970-21ntpq08:21:490
20974218527,119sleep321118-21latency_hist11:46:163
5294218414,155sleep05441-21latency_hist07:31:160
10110218248,93sleep31-21systemd07:46:143
23956217920,139sleep224052-21timerandwakeup10:11:442
17860217029,127sleep317996-21grep09:51:463
15161217020,21sleep30-21swapper/308:01:333
10653216810,101sleep310704-21latency_hist09:31:153
27199216714,117sleep327260-21latency_hist12:06:153
30049216512,134sleep230189-21unixbench_singl08:46:472
7586216351,53sleep30-21swapper/311:01:453
11275216129,43sleep30-21swapper/309:31:393
8412215917,68sleep18520-21wc09:21:441
2461721598,99sleep324635-21modprobe10:16:133
2501521589,109sleep30-21swapper/308:31:383
2501521589,109sleep30-21swapper/308:31:383
11454215618,123sleep011515-21switchtime09:31:440
138621530,27sleep21390-21unixbench_singl10:41:482
30041215228,89sleep329805-21/usr/sbin/munin10:31:363
4231215140,18sleep30-21swapper/307:26:513
18172214812,96sleep330836-21munin-node08:11:263
18172214812,96sleep330836-21munin-node08:11:263
10154214622,106sleep210219-21fschecks_time11:11:332
3010321457,120sleep00-21swapper/010:31:370
278221420,18sleep029255-21cobalt_printf12:31:150
278221420,18sleep029255-21cobalt_printf12:31:150
26307213928,83sleep226364-21tune2fs08:36:332
18580213941,40sleep30-21swapper/311:36:433
12168213919,102sleep012218-21sshd07:51:360
28237213414,105sleep227758-21apt-get10:26:222
23397213332,49sleep30-21swapper/311:51:453
19161213013,100sleep119224-21cut09:56:411
3349212834,53sleep33381-21munin-run10:51:153
13411212810,42sleep10-21swapper/111:21:261
32001212722,54sleep30-21swapper/307:11:443
13042212624,51sleep313181-21gpgv11:21:153
13042212624,51sleep313181-21gpgv11:21:153
18375212517,37sleep218438-21sendmail-msp09:56:132
25340212432,28sleep30-21swapper/311:57:513
17099212425,83sleep30-21swapper/308:06:423
17099212425,83sleep30-21swapper/308:06:423
10486212418,89sleep110520-21cut07:46:321
10486212418,89sleep110520-21cut07:46:321
29679212319,84sleep029824-21unixbench_multi12:11:450
21381212315,95sleep121430-21idleruntime11:46:341
22860212219,84sleep20-21swapper/208:26:162
11951211722,47sleep30-21swapper/307:51:313
6520211620,80sleep26535-21kernelversion09:16:362
4092211619,62sleep30-21swapper/312:33:533
4092211619,62sleep30-21swapper/312:33:533
32360211622,54sleep30-21swapper/312:21:253
32360211622,54sleep30-21swapper/312:21:253
29219211544,25sleep329301-21date07:06:153
16779211542,31sleep30-21swapper/311:31:383
2947421149,92sleep029526-21sort07:06:280
233421140,7sleep20-21swapper/207:21:322
591211231,68sleep3593-21apt-get07:16:253
27972211221,68sleep027862-21/usr/sbin/munin07:01:320
3082821119,86sleep330951-21sed07:11:163
28256211111,83sleep228328-21mailstats08:41:422
1779211129,44sleep31-21systemd10:46:143
4186211014,19sleep00-21swapper/010:51:520
4186211014,19sleep00-21swapper/010:51:520
27521211010,89sleep327592-21cut12:06:313
27521211010,89sleep327592-21cut12:06:313
23702211027,67sleep323766-21cat08:26:453
23702211027,67sleep323766-21cat08:26:453
44521098,14sleep30-21swapper/308:56:383
27520210913,87sleep00-21swapper/010:23:210
16869210920,71sleep016892-21latency_hist09:51:150
21480210810,84sleep321542-21wc11:46:363
2909521077,61sleep329248-21wc08:46:163
2868210710,40sleep30-21swapper/309:06:153
23110210720,27sleep20-21swapper/210:11:162
25506210648,38sleep30-21swapper/310:16:423
25506210648,38sleep30-21swapper/310:16:423
18961210611,82sleep319016-21latency09:56:353
10662210619,72sleep010674-21/usr/sbin/munin07:46:350
10662210619,72sleep010674-21/usr/sbin/munin07:46:350
5934210424,62sleep05961-21munin-run09:16:130
29538210011,73sleep029573-21idleruntime-cro10:31:130
2823821009,43sleep30-21swapper/310:26:233
2701921008,55sleep327089-21sensors_temp10:21:433
19757210013,67sleep119847-21/usr/sbin/munin11:41:331
13730210021,55sleep013744-21idleruntime-cro09:41:130
1371210019,59sleep11293-21latency_hist09:01:161
1866429911,70sleep018683-21apt-get09:56:240
558729831,43sleep25677-21grep07:31:352
49629618,59sleep10-21swapper/110:41:311
27662959,42sleep30-21swapper/312:31:153
27662959,42sleep30-21swapper/312:31:153
2353129511,70sleep023560-21idleruntime10:11:340
2302929322,52sleep2292-21in:imuxsock11:51:362
266912929,40sleep30-21swapper/308:36:433
2662829213,56sleep026195-21/usr/sbin/munin08:36:400
1525229214,64sleep215269-21kernelversion11:26:362
3187629125,47sleep00-21swapper/010:36:420
1984629111,66sleep019849-21entropy08:16:320
1340729113,63sleep212925-21apt-get11:21:232
31142909,35sleep30-21swapper/309:06:273
3095029012,65sleep030961-21netstat12:16:380
3235928922,45sleep231855-21apt-get12:21:242
3235928922,45sleep231855-21apt-get12:21:242
210762899,58sleep221139-21apt-key08:21:152
1205628820,28sleep00-21swapper/011:16:400
1205628820,28sleep00-21swapper/011:16:400
934728717,34sleep10-21swapper/109:26:281
3185528716,57sleep231864-21uname10:36:402
80272860,8sleep00-21swapper/009:21:330
2992528619,45sleep129981-21fschecks_time10:31:341
1760328617,50sleep017665-21/usr/sbin/munin09:51:430
971728515,42sleep29752-21ntp_kernel_err09:26:372
729628541,25sleep07292-21apt-get07:36:250
729628541,25sleep07292-21apt-get07:36:250
65332855,68sleep20-21swapper/210:59:222
2388528510,59sleep123955-21/usr/sbin/munin10:11:431
602628415,41sleep00-21swapper/007:31:450
427228419,42sleep04332-21modprobe12:36:120
427228419,42sleep04332-21modprobe12:36:120
393228415,51sleep23439-21apt-get07:26:252
191428422,38sleep11883-21latency_hist07:21:151
726428316,49sleep16770-21apt-get07:36:181
726428316,49sleep16770-21apt-get07:36:181
3140628327,44sleep331439-21cpu10:36:293
25162839,53sleep10-21swapper/107:21:361
828028212,23sleep30-21swapper/309:21:423
3266028217,44sleep232717-21/usr/sbin/munin08:56:332
3210128218,31sleep00-21swapper/007:11:450
488528024,18sleep30-21swapper/309:11:363
1043828030,39sleep310474-21ntpq11:11:383
869227919,19sleep00-21swapper/011:06:340
386327926,35sleep14004-21grep09:06:461
3036127926,30sleep20-21swapper/212:16:152
2991527936,10sleep30-21swapper/312:11:523
1148027928,39sleep313441-21snmpd07:48:563
1148027928,39sleep313441-21snmpd07:48:563
346427820,26sleep00-21swapper/007:26:150
1863227826,27sleep10-21swapper/108:11:401
1863227826,27sleep10-21swapper/108:11:401
2442227724,24sleep20-21swapper/208:31:162
2442227724,24sleep20-21swapper/208:31:162
249072767,15sleep10-21swapper/110:16:241
249072767,15sleep10-21swapper/110:16:241
905727518,30sleep20-21swapper/209:26:132
337027518,26sleep20-21swapper/212:31:382
337027518,26sleep20-21swapper/212:31:382
2918127530,24sleep20-21swapper/207:06:142
1556327519,40sleep30-21swapper/309:46:253
1164627519,26sleep20-21swapper/209:31:462
1162927513,46sleep211648-21apt-get11:16:262
1162927513,46sleep211648-21apt-get11:16:262
961827421,31sleep00-21swapper/009:26:350
3268727421,28sleep00-21swapper/012:22:150
3268727421,28sleep00-21swapper/012:22:150
3119527325,25sleep20-21swapper/210:36:152
2472927220,31sleep224619-21/usr/sbin/munin08:31:332
2472927220,31sleep224619-21/usr/sbin/munin08:31:332
1143327223,19sleep00-21swapper/011:16:160
313627128,11sleep30-21swapper/307:22:433
2439627120,27sleep10-21swapper/108:31:151
2439627120,27sleep10-21swapper/108:31:151
211942719,49sleep321250-21dpkg08:21:163
2101727117,30sleep10-21swapper/108:21:141
1779827121,33sleep217843-21apt11:36:132
231627020,28sleep10-21swapper/110:46:361
1483827016,34sleep20-21swapper/208:01:152
136352700,8sleep00-21swapper/007:56:320
609826917,26sleep10-21swapper/107:31:461
2087426910,45sleep020875-21sshd11:45:360
1535926915,36sleep015384-21/usr/sbin/munin11:26:380
925426812,41sleep09297-21/usr/sbin/munin07:41:380
40732680,9sleep212-21rcu_sched10:51:492
40732680,9sleep212-21rcu_sched10:51:492
366626813,34sleep23318-21apt-get10:51:182
2491226819,21sleep10-21swapper/111:56:441
2413526819,24sleep10-21swapper/111:56:151
2409426817,26sleep00-21swapper/011:56:140
1558726822,21sleep20-21swapper/208:01:442
727226722,26sleep27289-21apt07:36:232
727226722,26sleep27289-21apt07:36:232
3114226721,26sleep10-21swapper/108:51:351
2794226713,39sleep128025-21tr12:06:411
2794226713,39sleep128025-21tr12:06:411
1480526714,30sleep00-21swapper/009:41:460
953026614,38sleep29654-21cat07:41:452
902326612,24sleep10-21swapper/111:06:441
40862666,10sleep00-21swapper/009:07:290
3179526622,23sleep10-21swapper/107:11:381
3096026613,13sleep20-21swapper/208:51:322
1727926619,26sleep117417-21sed11:31:481
136532666,10sleep10-21swapper/109:40:131
57026517,23sleep20-21swapper/207:16:242
443026518,28sleep14490-21/usr/sbin/munin07:26:571
3192126512,31sleep20-21swapper/207:11:432
3155926520,27sleep10-21swapper/112:17:361
2984426510,30sleep10-21swapper/107:06:371
2932726513,36sleep129320-21apt-get08:46:301
2786526514,22sleep10-21swapper/108:41:331
1787926516,26sleep00-21swapper/011:36:150
43322646,12sleep10-21swapper/110:51:581
43322646,12sleep10-21swapper/110:51:581
2714826419,28sleep21-21systemd12:06:142
1793726414,24sleep00-21swapper/008:11:150
1793726414,24sleep00-21swapper/008:11:150
163726410,38sleep11528-21/usr/sbin/munin09:01:321
676226318,25sleep10-21swapper/109:16:431
250042630,7sleep20-21swapper/211:56:442
183726320,22sleep20-21swapper/212:26:372
183726320,22sleep20-21swapper/212:26:372
1662726318,28sleep116621-21apt-get08:06:271
1662726318,28sleep116621-21apt-get08:06:271
2841426214,32sleep028536-21sed08:41:440
2376926213,24sleep00-21swapper/008:26:460
2376926213,24sleep00-21swapper/008:26:460
2150926210,37sleep121553-21systemd-run10:06:141
140232620,7sleep10-21swapper/107:56:411
1240426219,29sleep112410-21/usr/sbin/munin07:51:411
67926127,18sleep00-21swapper/008:56:430
1826926121,16sleep20-21swapper/208:11:332
1826926121,16sleep20-21swapper/208:11:332
1731326115,29sleep20-21swapper/208:06:452
1731326115,29sleep20-21swapper/208:06:452
116152618,10sleep20-21swapper/207:50:412
116152618,10sleep20-21swapper/207:50:412
601226019,18sleep20-21swapper/209:16:152
2998126013,20sleep20-21swapper/207:06:432
2823926017,17sleep00-21swapper/012:06:440
2823926017,17sleep00-21swapper/012:06:440
162726013,22sleep00-21swapper/009:01:310
770925914,26sleep00-21swapper/011:01:460
2162325917,24sleep221665-21ntp_states11:46:392
155325917,20sleep10-21swapper/112:26:321
155325917,20sleep10-21swapper/112:26:321
3138425816,20sleep00-21swapper/008:51:420
2646525815,19sleep00-21swapper/012:01:430
2520425819,20sleep00-21swapper/010:16:350
2520425819,20sleep00-21swapper/010:16:350
2433325814,27sleep024328-21apt-get11:56:260
1165425814,26sleep111648-21apt-get11:16:291
1165425814,26sleep111648-21apt-get11:16:291
60525716,24sleep0593-21apt-get07:16:300
535225720,22sleep10-21swapper/109:11:461
2186925717,24sleep121878-21df_inode10:06:301
2156225715,24sleep121534-21apt-get08:21:381
1952425716,21sleep00-21swapper/008:16:160
1952425716,21sleep00-21swapper/008:16:160
1766625715,20sleep10-21swapper/109:51:441
1459425719,22sleep20-21swapper/209:41:442
1203525720,16sleep20-21swapper/207:51:342
2774325612,26sleep127782-21http10:26:131
2747625614,27sleep127515-21modprobe08:41:131
2532225619,22sleep20-21swapper/210:16:372
2532225619,22sleep20-21swapper/210:16:372
223925621,19sleep20-21swapper/209:01:462
217382566,12sleep00-21swapper/010:06:170
1689325615,20sleep10-21swapper/109:51:161
1409125616,25sleep114098-21df_inode09:41:301
1364725617,16sleep00-21swapper/011:21:370
1300825614,27sleep213078-21sensors_temp09:36:432
977725514,22sleep29804-21munin-run11:11:132
828125517,16sleep10-21swapper/111:06:151
2930025516,20sleep00-21swapper/008:46:250
2880725517,17sleep10-21swapper/110:26:541
2778425519,22sleep227811-21munin-run10:26:142
2702025515,23sleep10-21swapper/112:04:351
1945725520,17sleep00-21swapper/011:41:160
1820225513,18sleep20-21swapper/211:36:332
1547925518,24sleep115549-21tr11:26:431
49525413,25sleep0486-21apt-get10:41:300
328725418,21sleep10-21swapper/112:31:371
328725418,21sleep10-21swapper/112:31:371
3057925413,25sleep230575-21apt-get12:16:272
2913525412,21sleep10-21swapper/112:11:341
1022125415,17sleep10-21swapper/107:46:161
696725313,23sleep26963-21apt-get11:01:272
2032825315,14sleep00-21swapper/010:01:320
1995825312,25sleep220005-21systemd-run10:01:142
549525212,24sleep05499-21diskstats10:56:330
1577325213,21sleep20-21swapper/209:46:322
662325012,23sleep06655-21awk09:16:380
2129825016,16sleep00-21swapper/011:46:330
1151625012,23sleep111643-21timerwakeupswit09:31:451
587424915,21sleep10-21swapper/110:56:421
3271624917,16sleep10-21swapper/110:41:151
1535924912,22sleep015452-21memory08:01:380
197102487,17sleep20-21swapper/211:41:302
1052324812,20sleep110529-21ntp_states11:11:401
3270024710,20sleep1324-21memory12:22:161
3270024710,20sleep1324-21memory12:22:161
181162448,18sleep10-21swapper/111:36:281
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional