You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-20 - 19:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Thu Aug 20, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
211992105815,1026sleep221266-21df07:54:442
30618256710,540sleep230627-21/usr/sbin/munin10:05:022
12154247114,439sleep212193-21ntpq12:34:552
7625243210,408sleep17626-21sshd10:36:221
16172232427,274sleep116327-21grep09:19:591
17183228046,219sleep117250-21idleruntime09:24:481
3211227095,161sleep13239-21grep10:24:271
7348225712,229sleep27489-21awk10:35:022
28642224170,87sleep10-21swapper/109:59:561
182922284,212sleep0451-21kworker/0:212:00:340
182922284,212sleep0451-21kworker/0:212:00:340
26403222441,106sleep10-21swapper/109:54:311
23433222417,184sleep123549-21timerwakeupswit08:00:001
23227221231,119sleep123248-21apt-get09:44:381
18117221013,107sleep10-21swapper/111:09:531
11705219826,84sleep10-21swapper/110:49:511
14791219711,169sleep114834-21fschecks_time10:59:481
18752219621,153sleep10-21swapper/107:44:581
5208218668,30sleep10-21swapper/108:44:591
4431218615,106sleep14471-21ps12:09:551
29277218411,161sleep029370-21cut08:19:500
1323021800,48sleep013231-21unixbench_singl09:10:000
5298217826,135sleep25358-21/usr/sbin/munin07:04:462
27565217792,70sleep127630-21awk11:39:561
19883217780,58sleep16556-21cyclictest07:49:471
30465217238,113sleep130615-21unixbench_singl10:05:011
9156217092,66sleep19235-21awk12:24:571
8451216911,136sleep10-21swapper/110:39:501
3259216717,84sleep10-21swapper/108:39:581
32374216411,90sleep10-21swapper/108:29:521
20136216117,94sleep120149-21cpu09:34:451
20136216117,94sleep120149-21cpu09:34:451
20321216044,105sleep120324-21modprobe11:16:271
11020216069,77sleep10-21swapper/109:04:571
1716215343,94sleep11854-21latency_hist10:19:311
8256214928,106sleep00-21swapper/010:39:450
1804214846,81sleep10-21swapper/112:00:231
1804214846,81sleep10-21swapper/112:00:231
1404221428,122sleep0367-21snmpd10:55:470
27213214016,49sleep027329-21timerwakeupswit09:54:590
31993213720,46sleep232134-21grep10:09:592
13881213642,70sleep10-21swapper/109:14:381
7066213351,17sleep10-21swapper/107:09:491
23777213369,46sleep10-21swapper/111:29:311
18954213245,24sleep118559-21/usr/sbin/munin09:29:521
18954213245,24sleep118559-21/usr/sbin/munin09:29:521
20963212713,100sleep020999-21fschecks_count11:19:450
22664212634,19sleep10-21swapper/111:24:511
15826212635,26sleep10-21swapper/107:35:011
2955212315,92sleep03024-21sensors_temp12:04:580
32356212278,29sleep132488-21timerwakeupswit11:54:581
19268212117,85sleep019413-21unixbench_multi09:29:590
19268212117,85sleep019413-21unixbench_multi09:29:590
16659212120,25sleep10-21swapper/111:04:571
22606211814,81sleep021682-21/usr/sbin/munin09:40:010
22606211814,81sleep021682-21/usr/sbin/munin09:40:010
12220211811,95sleep112259-21wc12:34:581
13671211719,81sleep10-21swapper/109:14:301
14688211413,49sleep10-21swapper/107:34:281
14688211413,49sleep10-21swapper/107:34:281
11953211413,26sleep311962-21df_inode07:24:453
11953211413,26sleep311962-21df_inode07:24:453
10802211412,70sleep00-21swapper/009:04:440
21876211310,76sleep10-21swapper/107:55:001
5026211110,24sleep10-21swapper/110:29:401
556021085,90sleep00-21swapper/008:49:090
17748210816,46sleep30-21swapper/311:09:433
25004210711,82sleep324528-21apt-get09:49:333
27179210614,78sleep127283-21latency_hist08:14:301
250021067,50sleep10-21swapper/112:04:471
20879210610,33sleep10-21swapper/111:19:421
20328210621,61sleep320461-21sed07:49:573
5610210212,75sleep35709-21cut12:14:453
31497210212,40sleep131528-21latency_hist11:54:291
24997210212,75sleep225115-21cat08:05:022
30951210128,18sleep10-21swapper/108:24:521
1193210029,13sleep10-21swapper/108:34:411
35012996,13sleep00-21swapper/010:24:430
464629816,67sleep04755-21cat12:09:580
2323729813,65sleep223279-21wc07:59:572
2887729718,53sleep029024-21grep09:59:590
254329719,20sleep30-21swapper/308:39:313
2476029624,15sleep10-21swapper/108:04:571
45302956,11sleep10-21swapper/110:26:291
2556929510,71sleep10-21swapper/111:34:431
163629513,58sleep31671-21/usr/sbin/munin10:19:293
1525429511,65sleep015361-21latency_hist09:19:300
60302946,13sleep00-21swapper/008:49:500
3219929420,55sleep231713-21/usr/sbin/munin11:54:552
2957529418,62sleep329648-21/usr/sbin/munin08:19:573
269482949,65sleep10-21swapper/109:54:521
1073829411,64sleep310783-21/usr/sbin/munin07:19:533
1011129113,66sleep110149-21fschecks_time10:45:031
894429012,13sleep10-21swapper/107:14:501
786429024,43sleep37912-21sendmail10:39:273
1432629034,39sleep114357-21cron10:59:281
1308629019,47sleep313226-21grep09:10:003
1031929018,39sleep10-21swapper/109:04:291
770228921,16sleep10-21swapper/108:54:521
711228913,54sleep00-21swapper/012:19:430
627828916,18sleep10-21swapper/108:49:551
2876928911,14sleep10-21swapper/108:19:311
2442828910,61sleep224429-21sshd09:48:232
2516428746,31sleep1367-21snmpd11:33:341
83972867,14sleep10-21swapper/112:24:281
2139128626,39sleep321417-21munin-run09:39:273
2139128626,39sleep321417-21munin-run09:39:273
1512628614,55sleep214620-21/usr/sbin/munin10:59:572
1371528623,31sleep30-21swapper/310:54:583
733128511,57sleep06773-21/usr/sbin/munin07:09:560
2835628517,52sleep028378-21modprobe11:44:270
1966028517,52sleep319688-21/usr/sbin/munin11:14:523
1638728520,40sleep216445-21/usr/sbin/munin11:04:502
588628425,25sleep10-21swapper/107:04:571
55152848,14sleep10-21swapper/112:14:391
3065428421,30sleep00-21swapper/011:49:560
2909728420,11sleep10-21swapper/108:19:471
2544528412,57sleep025489-21awk09:49:580
118842849,59sleep10-21swapper/107:24:431
118842849,59sleep10-21swapper/107:24:431
2872628324,34sleep20-21swapper/211:44:452
810628221,37sleep20-21swapper/207:14:302
3019328225,35sleep330188-21apt-get11:49:433
103382829,57sleep10-21swapper/107:19:421
76228118,41sleep2327-21/usr/sbin/munin10:14:552
40882819,58sleep24159-21cat08:44:312
2303728111,56sleep023051-21grep07:59:510
2713727925,28sleep30-21swapper/309:54:583
900727815,48sleep09075-21df08:59:430
570427828,31sleep30-21swapper/307:04:553
171232787,11sleep10-21swapper/107:39:571
151892780,8sleep30-21swapper/310:59:573
1146727822,38sleep311482-21apt12:34:273
1146727822,38sleep311482-21apt12:34:273
106512786,13sleep10-21swapper/112:29:531
106512786,13sleep10-21swapper/112:29:531
945727713,48sleep29464-21ntp_states08:59:542
317622778,56sleep0367-21snmpd08:28:500
217842776,32sleep10-21swapper/109:39:461
217842776,32sleep10-21swapper/109:39:461
3148427610,47sleep00-21swapper/010:09:490
1340627619,33sleep00-21swapper/007:29:390
1340627619,33sleep00-21swapper/007:29:390
3141327518,42sleep331477-21idleruntime10:09:483
2408727512,49sleep324127-21latency_hist08:04:303
1361727514,28sleep213623-21modprobe09:14:272
1015027512,45sleep210210-21/usr/sbin/munin10:45:042
324932746,12sleep10-21swapper/110:14:021
1704027311,43sleep017103-21mailstats07:39:560
648027215,43sleep26560-21grep08:50:002
2457927210,28sleep30-21swapper/311:29:573
159927111,30sleep20-21swapper/208:34:542
635827014,40sleep36507-21latency_hist10:34:313
298232706,9sleep10-21swapper/111:48:241
1076927021,32sleep310288-21apt-get09:04:393
932826914,38sleep29421-21cat07:14:592
2770226916,23sleep30-21swapper/311:39:583
2182926917,37sleep321913-21perl09:39:483
2182926917,37sleep321913-21perl09:39:483
111602696,50sleep20-21swapper/212:30:312
111602696,50sleep20-21swapper/212:30:312
1567526815,24sleep00-21swapper/007:35:000
91026719,22sleep00-21swapper/008:34:280
49426712,43sleep31-21systemd11:58:473
2741926716,32sleep227394-21/usr/sbin/munin08:14:432
2528526721,23sleep00-21swapper/011:34:300
2504326715,32sleep20-21swapper/209:49:442
792426619,30sleep38004-21/usr/sbin/munin08:54:583
2568126617,28sleep30-21swapper/308:09:313
1557426610,43sleep0367-21snmpd11:00:150
630726519,24sleep00-21swapper/010:34:280
1730526510,42sleep317359-21sed09:24:503
1631626520,19sleep20-21swapper/207:39:302
1549326511,39sleep215563-21df09:19:442
1363826514,22sleep20-21swapper/210:54:572
992126431,16sleep00-21swapper/012:29:280
875626416,21sleep00-21swapper/007:14:450
606826413,36sleep26069-21sshd10:31:232
2903826418,26sleep00-21swapper/011:44:520
2373726416,32sleep00-21swapper/011:29:300
2373726416,32sleep00-21swapper/011:29:300
2096126418,30sleep00-21swapper/007:54:300
590926310,13sleep00-21swapper/012:14:510
250726318,24sleep20-21swapper/212:04:482
208026315,28sleep32161-21/usr/sbin/munin10:19:493
1744026311,38sleep217488-21/usr/sbin/munin09:24:532
1506026312,38sleep315085-21fschecks_count07:34:463
942226211,35sleep39549-21timerwakeupswit07:15:003
54482625,11sleep20-21swapper/208:47:422
447426214,18sleep20-21swapper/212:09:562
44726215,29sleep0327-21/usr/sbin/munin10:14:490
352326214,24sleep00-21swapper/008:40:010
3063026212,35sleep230637-21diskstats08:24:442
2944426212,32sleep329542-21users11:44:593
2482426220,26sleep00-21swapper/008:05:000
1716726220,21sleep20-21swapper/207:39:582
866426117,19sleep30-21swapper/310:39:563
2748526118,29sleep227497-21vmstat09:55:012
1885126117,23sleep30-21swapper/309:29:503
1885126117,23sleep30-21swapper/309:29:503
528326019,18sleep00-21swapper/007:04:440
3228926017,25sleep00-21swapper/011:54:570
279026017,23sleep30-21swapper/312:04:533
2617426017,20sleep00-21swapper/011:34:590
2609426012,21sleep30-21swapper/311:34:583
2214026020,19sleep30-21swapper/311:24:283
2030826017,20sleep20-21swapper/209:34:502
2030826017,20sleep20-21swapper/209:34:502
1472926023,20sleep20-21swapper/207:34:292
1472926023,20sleep20-21swapper/207:34:292
1229326016,20sleep00-21swapper/010:50:020
1159426015,27sleep211625-21awk07:24:282
455825913,27sleep34644-21/usr/sbin/munin12:09:573
2595725919,23sleep30-21swapper/308:09:473
2587525914,18sleep00-21swapper/008:09:440
2233125917,22sleep20-21swapper/209:39:592
2233125917,22sleep20-21swapper/209:39:592
2040625917,21sleep00-21swapper/009:34:520
2040625917,21sleep00-21swapper/009:34:520
1655525930,15sleep30-21swapper/307:39:433
1403625920,22sleep30-21swapper/307:29:583
1403625920,22sleep30-21swapper/307:29:583
1243425914,29sleep012557-21sed12:35:010
756225813,25sleep37564-21ntp_states12:19:543
696025817,22sleep30-21swapper/307:09:473
407425815,20sleep30-21swapper/310:24:583
339425820,18sleep30-21swapper/308:40:003
2455525817,23sleep324641-21apt-config09:49:283
1810025811,19sleep20-21swapper/207:44:432
1164525815,18sleep00-21swapper/007:24:300
2145725715,21sleep00-21swapper/007:54:500
1996625717,22sleep20-21swapper/211:14:592
1647025713,27sleep016564-21memory11:04:520
128725712,22sleep00-21swapper/008:34:480
923925517,21sleep28633-21/usr/sbin/munin12:24:582
789125516,19sleep20-21swapper/208:54:572
660625515,23sleep06601-21apt-get10:34:440
391525513,23sleep23951-21ntp_states10:24:532
2778825514,22sleep327819-21/usr/sbin/munin08:14:523
195125513,26sleep01945-21apt-get10:19:420
1941325516,19sleep00-21swapper/007:49:290
1864425513,26sleep218653-21vmstat11:10:022
1775025516,21sleep00-21swapper/011:09:440
1059725520,20sleep00-21swapper/010:45:140
969925415,20sleep20-21swapper/210:44:312
459025421,19sleep30-21swapper/308:44:483
308842548,36sleep30-21swapper/310:08:143
3052725415,22sleep30-21swapper/308:24:373
2717025418,17sleep20-21swapper/211:39:452
2303225410,16sleep20-21swapper/211:25:002
2303225410,16sleep20-21swapper/211:25:002
1849025417,17sleep30-21swapper/307:44:523
3209425314,22sleep232096-21apt-get08:29:412
2948225318,17sleep20-21swapper/210:04:302
2551825314,23sleep225514-21apt-get11:34:402
2253125319,18sleep30-21swapper/307:59:303
151122535,11sleep20-21swapper/209:18:392
1225025317,15sleep20-21swapper/207:24:522
1225025317,15sleep20-21swapper/207:24:522
3027325212,24sleep230277-21diskstats11:49:462
2741725215,19sleep00-21swapper/008:14:410
2611625213,17sleep20-21swapper/208:09:512
1865625214,22sleep218662-21df_inode09:29:452
1865625214,22sleep218662-21df_inode09:29:452
131412525,35sleep00-21swapper/007:29:260
131412525,35sleep00-21swapper/007:29:260
874225111,24sleep38753-21awk12:24:463
718425111,24sleep27190-21df_inode12:19:452
2352325114,16sleep00-21swapper/009:44:490
1021125114,23sleep310259-21irqstats10:45:053
123225013,18sleep20-21swapper/211:59:512
123225013,18sleep20-21swapper/211:59:512
656124914,17sleep30-21swapper/308:50:013
286192499,27sleep328625-21ntp_states09:59:543
2240324914,18sleep30-21swapper/311:24:403
2240324914,18sleep30-21swapper/311:24:403
204272495,35sleep00-21swapper/011:17:430
203824917,18sleep20-21swapper/210:19:482
801024810,20sleep00-21swapper/008:54:590
2907524818,15sleep20-21swapper/208:19:442
2108624817,17sleep20-21swapper/211:19:472
128582487,32sleep30-21swapper/312:37:263
1020824811,22sleep010204-21apt-get12:29:400
1020824811,22sleep010204-21apt-get12:29:400
551624711,21sleep25508-21apt-get12:14:402
2009124711,21sleep220102-21ntp_states07:49:522
136624713,15sleep30-21swapper/311:59:563
136624713,15sleep30-21swapper/311:59:563
1282024715,17sleep30-21swapper/310:54:293
1151524712,20sleep211512-21apt-get10:49:402
168522469,14sleep00-21swapper/009:24:300
1576024612,22sleep015822-21wc09:19:500
139812467,16sleep30-21swapper/309:14:443
107682467,22sleep20-21swapper/209:04:382
3033824514,16sleep00-21swapper/010:05:000
157092459,19sleep315758-21irqstats09:19:493
96324411,19sleep31106-21date08:34:313
897824412,17sleep30-21swapper/308:59:383
2527324416,15sleep20-21swapper/211:34:282
161972448,18sleep30-21swapper/311:04:423
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional