You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-09 - 16:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Fri Jan 09, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
309802391314,3886sleep121109-21kworker/1:208:59:421
454827764,741sleep228313-1kworker/2:0H11:00:242
3251026686,649sleep027735-1kworker/0:3H10:45:160
3251026686,649sleep027735-1kworker/0:3H10:45:160
155192531496,22sleep315521-21sshd09:52:413
2837823736,357sleep027735-1kworker/0:3H08:51:410
28303236564,244sleep228314-21find12:14:582
26952233431,33sleep10-21swapper/107:04:511
17556232978,235sleep017549-21apt-get10:00:010
31393230286,146sleep231411-21fschecks_count09:00:072
31393230286,146sleep231411-21fschecks_count09:00:072
24945229188,184sleep21-21systemd12:04:482
22251225413,224sleep322315-21/usr/sbin/munin11:55:083
15879223964,20sleep20-21swapper/208:10:122
15879223964,20sleep20-21swapper/208:10:122
22785222624,163sleep20-21swapper/210:15:182
22785222624,163sleep20-21swapper/210:15:182
54221999,176sleep2667-21cat07:20:172
23641219913,169sleep123678-21ntpq08:35:111
85821985,183sleep02624-1kworker/0:0H12:30:300
9466219212,162sleep29469-21cron09:34:482
95082185132,39sleep29526-21kernelversion07:50:092
5837218116,148sleep25840-21cron07:39:482
5754218128,132sleep20-21swapper/209:20:182
13562218126,136sleep20-21swapper/211:29:472
28938217917,115sleep229045-21grep07:10:082
28938217917,115sleep229045-21grep07:10:082
9746217516,93sleep20-21swapper/211:15:182
18582217017,132sleep218681-21latency_hist11:44:492
26600216615,134sleep226630-21sshd10:29:472
26534216218,127sleep20-21swapper/212:09:492
30015216120,80sleep20-21swapper/212:20:062
12155215628,57sleep20-21swapper/207:59:502
1901921525,135sleep02624-1kworker/0:0H11:45:120
1456215111,83sleep21524-21readlink12:32:102
27120214815,65sleep20-21swapper/210:29:592
27101214820,109sleep126627-21apt-get10:29:581
2028021487,13sleep00-21swapper/008:25:080
11305214816,114sleep011313-21/usr/sbin/munin09:39:590
15111214715,119sleep2314-21rs:main2
31479214327,52sleep20-21swapper/212:25:032
15186214313,60sleep228634-21cyclictest08:09:482
10240214112,75sleep210326-21mailstats09:35:162
3607213710,50sleep20-21swapper/209:15:052
3962213315,101sleep24008-21sendmail10:59:472
3034121339,68sleep230393-21df07:14:522
3034121339,68sleep230393-21df07:14:522
26981213113,17sleep20-21swapper/208:45:172
1902621306,54sleep20-21swapper/211:45:182
25610212913,69sleep10-21swapper/110:25:101
1568321276,113sleep115576-1kworker/1:2H11:35:321
1702521248,52sleep20-21swapper/208:15:012
16062212318,89sleep116102-21fschecks_count09:55:061
7380211910,97sleep27416-21latency_hist11:09:492
32627211919,83sleep032759-21latency_hist09:04:500
32627211919,83sleep032759-21latency_hist09:04:500
5371211811,90sleep14491-21/usr/sbin/munin11:02:131
19419211817,73sleep00-21swapper/008:20:200
3278211778,19sleep20-21swapper/209:14:492
32149211615,82sleep132239-21/usr/sbin/munin10:45:091
32149211615,82sleep132239-21/usr/sbin/munin10:45:091
2425521169,92sleep00-21swapper/010:20:160
2425521169,92sleep00-21swapper/010:20:160
23337211527,69sleep323419-21fschecks_time08:35:053
8661211314,15sleep10-21swapper/109:30:151
20929211321,19sleep20-21swapper/211:50:112
28085211248,52sleep228108-21sed12:14:502
1963211212,82sleep12025-21/usr/sbin/munin07:25:171
11542211011,36sleep20-21swapper/209:40:092
32022210915,77sleep331537-21apt-get10:44:593
32022210915,77sleep331537-21apt-get10:44:593
18822210955,41sleep218825-21cron10:04:482
6239210811,83sleep16275-21/usr/sbin/munin07:40:071
28132210838,11sleep20-21swapper/208:49:542
9625210612,74sleep30-21swapper/311:15:153
10896210618,70sleep110997-21fschecks_time07:55:091
32053210544,31sleep20-21swapper/210:45:022
32053210544,31sleep20-21swapper/210:45:022
15768210511,77sleep015846-21/usr/sbin/munin08:10:100
15768210511,77sleep015846-21/usr/sbin/munin08:10:100
31645210413,79sleep222-21ksoftirqd/210:44:492
31645210413,79sleep222-21ksoftirqd/210:44:492
2977210410,55sleep20-21swapper/212:35:172
26545210420,25sleep30-21swapper/312:09:503
28109210315,72sleep328255-21wc12:14:513
13336210321,66sleep213370-21ps09:45:142
5836210217,62sleep00-21swapper/011:04:500
12372210142,46sleep20-21swapper/208:00:032
770021008,51sleep20-21swapper/207:45:012
7339210014,63sleep07471-21tail09:25:210
96029813,66sleep31018-21mailstats09:05:173
1829729814,64sleep118368-21true08:19:481
1491229812,69sleep014973-21sendmail_mailst09:50:160
1121729615,22sleep228634-21cyclictest11:20:152
398429511,67sleep33997-21/usr/sbin/munin09:15:143
1955529533,11sleep20-21swapper/208:20:462
58429415,67sleep2637-21basename09:05:082
479329424,51sleep04836-21run-parts09:19:480
421129449,24sleep10-21swapper/110:59:501
3034329456,16sleep20-21swapper/208:55:192
1644429410,20sleep20-21swapper/209:55:152
1510029411,71sleep315101-21sshd11:34:413
406929317,47sleep04208-21cut10:59:490
1466729220,47sleep014818-21head08:05:180
62762918,71sleep26359-21perl07:40:082
363029115,56sleep03634-21cron12:39:470
2961129020,50sleep329635-21munin-run12:19:473
2042329012,22sleep220445-21tr08:25:102
148429013,62sleep01485-21cpu07:25:040
797028914,59sleep17937-21latency_hist09:29:501
2086928912,35sleep20-21swapper/210:10:092
511228813,51sleep35085-21/usr/sbin/munin09:20:033
2880428816,52sleep328140-21/usr/sbin/munin08:52:213
2509528814,60sleep325119-21cut08:40:093
2216728813,63sleep222201-21ps08:30:152
403828719,12sleep20-21swapper/207:31:362
2733328720,49sleep027337-21diskstats07:05:060
2489828719,32sleep20-21swapper/208:40:042
2081728613,58sleep320935-21timerwakeupswit08:25:193
243628520,11sleep20-21swapper/207:25:552
2230028513,53sleep322388-21fschecks_time10:15:063
2230028513,53sleep322388-21fschecks_time10:15:063
1910828512,13sleep20-21swapper/210:05:002
778128419,12sleep20-21swapper/209:28:052
3106228427,31sleep30-21swapper/307:15:103
3106228427,31sleep30-21swapper/307:15:103
3046928420,50sleep030570-21cat07:14:540
3046928420,50sleep030570-21cat07:14:540
2483828420,11sleep20-21swapper/210:22:042
2483828420,11sleep20-21swapper/210:22:042
2433028415,12sleep20-21swapper/208:37:302
2302328419,12sleep20-21swapper/211:55:282
1538428419,12sleep20-21swapper/209:50:552
13922849,63sleep21131-21/usr/sbin/munin10:50:112
855128315,12sleep20-21swapper/211:10:342
2529628311,57sleep125342-21processes08:40:161
1902228311,14sleep30-21swapper/311:45:153
901128220,32sleep30-21swapper/307:49:503
299128210,55sleep12993-21sshd09:11:411
2969128229,30sleep10-21swapper/112:19:491
2960528116,11sleep20-21swapper/210:35:252
155582818,14sleep20-21swapper/211:34:522
1364028025,23sleep30-21swapper/308:04:483
167627923,39sleep11693-21modprobe09:09:471
3046527815,46sleep330610-21unixbench_singl08:55:203
1607127822,33sleep30-21swapper/308:10:173
1607127822,33sleep30-21swapper/308:10:173
1537127818,27sleep10-21swapper/111:34:511
2668527719,42sleep026812-21dpkg10:29:490
963327623,39sleep00-21swapper/011:15:160
2491427623,33sleep00-21swapper/008:40:060
2042227621,27sleep30-21swapper/310:09:493
8732756,11sleep00-21swapper/007:20:440
2949727512,51sleep030095-21snmpd12:18:190
121127515,41sleep11243-21munin-run07:24:491
745627415,41sleep37494-21sed07:44:493
3206427424,30sleep10-21swapper/107:19:481
3206427424,30sleep10-21swapper/107:19:481
3022027417,40sleep130238-21munin-node07:14:511
3022027417,40sleep130238-21munin-node07:14:511
2933427420,24sleep30-21swapper/307:10:173
2933427420,24sleep30-21swapper/307:10:173
1847527417,41sleep018503-21munin-run11:44:470
808627314,27sleep10-21swapper/111:10:161
779827321,28sleep00-21swapper/007:45:070
2941527321,26sleep00-21swapper/007:10:180
2941527321,26sleep00-21swapper/007:10:180
271982739,14sleep20-21swapper/207:04:522
2698127325,19sleep10-21swapper/112:10:081
2197027226,23sleep30-21swapper/310:14:503
152562729,49sleep015370-21apt-config11:34:500
1136927216,17sleep10-21swapper/111:20:181
924627113,29sleep10-21swapper/107:50:001
3063027122,24sleep00-21swapper/010:40:110
3063027122,24sleep00-21swapper/010:40:110
175262718,51sleep1196-21jbd2/mmcblk2p2-09:59:541
245327021,26sleep30-21swapper/312:35:063
2210127018,26sleep30-21swapper/308:30:123
1574127024,29sleep10-21swapper/109:54:491
30592695,10sleep00-21swapper/009:12:330
2510426921,23sleep10-21swapper/110:24:491
2510426921,23sleep10-21swapper/110:24:491
168902697,49sleep00-21swapper/009:55:210
1308026919,22sleep30-21swapper/309:45:063
1108626916,39sleep311112-21date09:39:493
3086926832,19sleep330953-21expr10:40:183
3086926832,19sleep330953-21expr10:40:183
2948426812,30sleep10-21swapper/108:54:491
2347826819,31sleep023489-21modprobe10:19:480
2347826819,31sleep023489-21modprobe10:19:480
2301726823,25sleep10-21swapper/108:34:501
615926711,26sleep00-21swapper/011:05:060
31926718,26sleep30-21swapper/312:29:483
2851626721,28sleep328545-21munin-run07:09:493
2344026715,21sleep00-21swapper/011:59:500
1868726720,26sleep30-21swapper/311:44:503
1628826727,23sleep00-21swapper/011:37:210
475126616,33sleep34786-21/usr/sbin/munin07:35:083
2637826619,28sleep326373-21apt-get08:45:023
25812665,13sleep10-21swapper/112:35:081
1419626616,21sleep30-21swapper/308:05:083
1264026613,37sleep312761-21latency_hist09:44:503
51112656,12sleep00-21swapper/007:35:170
39422657,44sleep00-21swapper/007:30:210
2804226520,22sleep00-21swapper/012:14:470
2684526515,30sleep026910-21awk08:45:150
264932657,16sleep10-21swapper/112:09:481
2581826523,24sleep30-21swapper/310:25:163
224726516,22sleep30-21swapper/309:10:103
1294426517,32sleep113070-21sed11:25:181
1077426513,24sleep00-21swapper/007:55:000
1051726517,20sleep10-21swapper/111:19:491
681626412,32sleep16968-21sed11:05:201
351526415,32sleep30-21swapper/307:30:163
2436126414,28sleep10-21swapper/112:00:191
226662646,46sleep00-21swapper/008:30:580
224626410,39sleep32248-21sshd10:52:423
1592326415,30sleep315999-21fschecks_time11:37:113
1918826315,22sleep00-21swapper/010:05:100
1869626318,25sleep30-21swapper/308:20:073
1146326314,32sleep111306-21/usr/sbin/munin09:40:081
448126217,27sleep33981-21apt-get10:59:523
3246626221,26sleep30-21swapper/307:20:063
2146926219,21sleep10-21swapper/108:29:501
2112826215,31sleep121188-21/usr/sbin/munin10:10:161
2539626115,20sleep00-21swapper/012:05:070
1244526111,37sleep012454-21diskstats08:00:050
1130026115,31sleep011368-21/usr/sbin/munin11:20:170
316662607,13sleep00-21swapper/012:25:090
1361426016,19sleep00-21swapper/011:29:490
23742595,11sleep00-21swapper/010:54:200
2066125916,27sleep020655-21apt-get10:10:030
1050825920,20sleep00-21swapper/007:54:480
741725823,18sleep30-21swapper/311:09:503
692825810,25sleep37004-21latency09:25:123
568425813,29sleep15753-21/usr/sbin/munin09:20:171
3004825817,19sleep00-21swapper/010:39:510
1768525818,19sleep10-21swapper/111:40:161
175125814,20sleep30-21swapper/307:25:113
1747225813,22sleep30-21swapper/308:15:143
152525812,33sleep31612-21sed12:32:113
320552578,12sleep10-21swapper/112:25:181
3156025717,17sleep10-21swapper/110:44:481
3156025717,17sleep10-21swapper/110:44:481
2424725714,26sleep324253-21ntp_states10:20:153
2424725714,26sleep324253-21ntp_states10:20:153
2389025715,22sleep30-21swapper/312:00:093
2361825717,19sleep00-21swapper/008:35:100
196232579,34sleep119629-21ntpq11:47:191
1861025715,25sleep118607-21apt-get08:20:001
1137225716,19sleep30-21swapper/307:55:193
1111425719,17sleep00-21swapper/009:39:500
3156925615,23sleep10-21swapper/109:00:101
3156925615,23sleep10-21swapper/109:00:101
2866125620,19sleep30-21swapper/310:34:573
232032565,12sleep10-21swapper/111:57:451
2062625620,16sleep30-21swapper/311:50:043
1440925613,22sleep10-21swapper/108:05:151
1414025610,32sleep014141-21cron09:49:480
68325519,17sleep00-21swapper/009:05:100
434025515,21sleep10-21swapper/107:34:501
401225516,24sleep34058-21munin-run10:59:483
2785425517,21sleep00-21swapper/010:31:470
2367825513,26sleep023670-21apt-get12:00:020
2270825510,27sleep122784-21/usr/sbin/munin10:15:171
2270825510,27sleep122784-21/usr/sbin/munin10:15:171
197232556,11sleep10-21swapper/110:05:231
1359125514,25sleep313612-21munin-run11:29:483
1332925512,26sleep113335-21ntp_states09:45:131
762525414,24sleep07688-21/usr/sbin/munin11:10:030
2972725414,24sleep029723-21apt-get08:55:000
2132125415,20sleep10-21swapper/111:50:181
2100625413,26sleep021012-21ntp_states11:50:130
1362925417,20sleep10-21swapper/108:04:421
1201625413,23sleep312043-21munin-run11:24:483
3199025311,24sleep332129-21tail09:00:213
3199025311,24sleep332129-21tail09:00:213
1816625316,19sleep30-21swapper/310:00:173
984525212,23sleep19937-21perl09:35:071
97625213,23sleep1982-21df_inode12:31:041
846225113,22sleep08478-21irqstats09:30:090
484625116,21sleep14877-21latency07:35:101
115425111,23sleep11151-21apt-get10:50:011
970125013,22sleep39707-21ntp_states07:50:153
841425012,17sleep30-21swapper/309:30:083
770725018,16sleep30-21swapper/311:10:063
271872506,18sleep30-21swapper/310:30:453
820524916,17sleep10-21swapper/107:45:161
550524916,14sleep00-21swapper/009:20:120
3000124914,21sleep030008-21sshd12:20:040
2994624911,17sleep30-21swapper/312:20:033
1770724918,18sleep00-21swapper/008:15:190
1584724913,17sleep110185-21kworker/1:208:10:111
1584724913,17sleep110185-21kworker/1:208:10:111
623124814,20sleep06238-21diskstats07:40:060
483824813,21sleep14871-21munin-run09:19:491
3127924811,21sleep131423-21latency_hist12:24:501
2994624810,15sleep10-21swapper/110:39:481
2764024712,17sleep10-21swapper/108:49:481
172402457,15sleep30-21swapper/309:59:483
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional