You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-23 - 15:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Thu Apr 23, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1716213468,1305sleep3982-1kworker/3:0H07:56:123
3416280073,658sleep325884-21systemd-journal09:42:153
18918271117,681sleep118920-21sshd07:07:161
10855257117,536sleep010872-21cron11:50:480
2120252919,491sleep02195-21/usr/sbin/munin11:21:170
2120252919,491sleep02195-21/usr/sbin/munin11:21:170
1533824985,453sleep313968-21kworker/3:112:03:153
14968231614,191sleep315030-21fschecks_time10:21:083
17201230419,232sleep30-21swapper/312:10:503
20872230050,193sleep320932-21latency_hist10:40:503
5529225321,216sleep15592-21/usr/sbin/munin09:51:051
30304222539,71sleep30-21swapper/309:26:083
31616222313,189sleep331774-21unixbench_multi11:11:353
31616222313,189sleep331774-21unixbench_multi11:11:353
11607222212,193sleep011628-21awk11:51:170
9681221610,149sleep30-21swapper/311:46:073
27937221113,187sleep227947-21needreboot11:01:122
29391220636,127sleep329455-21latency11:06:103
29391220636,127sleep329455-21latency11:06:103
20360220161,27sleep30-21swapper/307:11:183
23376220012,18sleep00-21swapper/010:46:320
19323219970,63sleep319409-21fschecks_time12:16:073
29251219723,130sleep329252-21cpu07:41:023
9362219212,109sleep30-21swapper/311:45:503
20841218761,22sleep30-21swapper/308:56:123
32108218558,54sleep332230-21latency_hist07:50:513
27386218011,151sleep327401-21modprobe07:35:493
22508217541,51sleep30-21swapper/310:45:513
16480216024,90sleep30-21swapper/310:26:043
16480216024,90sleep30-21swapper/310:26:043
24239215924,58sleep30-21swapper/310:50:513
27865215413,110sleep30-21swapper/307:36:093
20398215328,103sleep320553-21date08:55:523
29433215123,112sleep029482-21basename07:41:070
7732214316,113sleep27854-21sort09:56:192
28430214380,17sleep30-21swapper/309:21:033
19297214127,95sleep219322-21fschecks_count12:16:062
32611213914,27sleep30-21swapper/307:51:113
520121389,114sleep34788-21/usr/sbin/munin08:06:133
22151213821,24sleep00-21swapper/007:16:200
3539213731,94sleep33586-21ntpq11:26:123
3539213731,94sleep33586-21ntpq11:26:123
2519021369,89sleep325228-21latency_hist12:35:493
2519021369,89sleep325228-21latency_hist12:35:493
6286213480,34sleep16085-21apt-get08:10:521
6203213014,101sleep06322-21timerwakeupswit09:51:210
12412213032,53sleep30-21swapper/308:29:153
12412213032,53sleep30-21swapper/308:29:153
29795212816,99sleep129863-21swap07:41:161
20031212719,75sleep10-21swapper/110:36:161
15658212520,83sleep215687-21/usr/sbin/munin08:40:492
1453121247,87sleep00-21swapper/012:01:100
9052212346,64sleep39097-21ntp_offset10:01:133
8753212318,67sleep30-21swapper/308:16:193
22791212235,72sleep322902-21latency_hist07:20:513
15291212057,41sleep30-21swapper/308:36:223
28057211913,93sleep027553-21/usr/sbin/munin11:01:180
9336211723,72sleep19483-21grep10:01:191
8655211513,18sleep20-21swapper/210:01:042
17666211556,41sleep1281-21dbus-daemon07:05:511
8549211414,82sleep18572-21/usr/sbin/munin08:16:131
6757211314,83sleep26836-21/usr/sbin/munin11:36:162
20247211313,82sleep020400-21awk10:36:190
13224211220,40sleep30-21swapper/308:31:133
8020211016,74sleep07535-21apt-get11:40:590
20636211014,77sleep220702-21sort08:56:072
18232210846,46sleep30-21swapper/310:31:093
18232210846,46sleep30-21swapper/310:31:093
748121078,13sleep00-21swapper/009:56:130
26957210754,14sleep30-21swapper/309:16:073
8838210616,72sleep08964-21sort08:16:200
6782210630,35sleep36785-21cron09:55:483
640210620,26sleep30-21swapper/311:16:353
640210620,26sleep30-21swapper/311:16:353
17568210625,63sleep20-21swapper/208:46:062
14679210614,75sleep014835-21latency_hist10:20:520
1287921065,12sleep30-21swapper/310:14:013
10470210631,23sleep30-21swapper/310:06:103
1434221055,12sleep30-21swapper/310:17:323
26016210440,17sleep30-21swapper/310:56:003
11055210419,59sleep211203-21grep08:25:512
11055210419,59sleep211203-21grep08:25:512
20516210318,69sleep120547-21latency_hist12:20:491
17087210319,67sleep117113-21awk12:10:481
14083210219,66sleep214102-21cron08:35:482
14042210221,57sleep114168-21latency_hist12:00:501
24528210139,45sleep30-21swapper/310:51:083
24528210139,45sleep30-21swapper/310:51:083
18851210133,18sleep30-21swapper/308:50:513
2194921009,76sleep122044-21latency_hist09:00:511
16455210013,73sleep316527-21mailstats08:41:183
257822995,48sleep30-21swapper/309:11:183
83662978,35sleep30-21swapper/310:00:483
2985029619,56sleep030002-21sed11:06:190
2985029619,56sleep030002-21sed11:06:190
2583629510,68sleep325839-21cron07:30:483
97302947,37sleep30-21swapper/308:21:033
97302947,37sleep30-21swapper/308:21:033
65752947,75sleep36594-21apt-get08:11:023
2290029419,56sleep222885-21apt-get10:46:062
1635129427,48sleep216405-21sort07:01:042
1066529413,31sleep110707-21ps10:06:161
234062935,51sleep30-21swapper/312:28:433
247052925,46sleep30-21swapper/309:06:543
111442927,68sleep30-21swapper/311:51:033
395529118,57sleep03956-21munin-node09:46:000
1559328917,18sleep10-21swapper/112:05:511
1251428916,50sleep212438-21latency_hist11:55:512
1151628927,41sleep011563-21grep10:10:510
2638628819,43sleep20-21swapper/210:56:112
285028712,36sleep32878-21awk11:25:483
285028712,36sleep32878-21awk11:25:483
2398928510,53sleep20-21swapper/212:31:082
2398928510,53sleep20-21swapper/212:31:082
211272858,64sleep00-21swapper/012:21:130
211272858,64sleep00-21swapper/012:21:130
2523028419,16sleep00-21swapper/012:35:500
2523028419,16sleep00-21swapper/012:35:500
1320228412,54sleep30-21swapper/311:56:163
434828316,17sleep10-21swapper/109:46:131
2709228313,56sleep027093-21sshd10:58:150
2456628315,32sleep10-21swapper/112:31:191
2456628315,32sleep10-21swapper/112:31:191
1343328317,45sleep113310-21/usr/sbin/munin10:16:091
1156528319,16sleep10-21swapper/110:10:521
3146428218,43sleep131538-21/usr/sbin/munin11:11:331
3146428218,43sleep131538-21/usr/sbin/munin11:11:331
2312928218,31sleep00-21swapper/007:21:080
2312928218,31sleep00-21swapper/007:21:080
1309128224,38sleep013128-21latency08:31:100
1012528210,15sleep10-21swapper/108:21:131
1012528210,15sleep10-21swapper/108:21:131
81802818,54sleep20-21swapper/208:16:022
68372819,44sleep30-21swapper/311:36:173
319712818,57sleep332023-21if_eth009:31:093
3091728117,44sleep031039-21/usr/sbin/munin09:26:340
2880728128,23sleep00-21swapper/009:21:130
2673728110,59sleep126813-21timerandwakeup07:31:191
2468128016,45sleep224715-21/usr/sbin/munin10:51:112
2468128016,45sleep224715-21/usr/sbin/munin10:51:112
2084428028,15sleep30-21swapper/312:21:073
2084428028,15sleep30-21swapper/312:21:073
1889728020,37sleep118931-21munin-run12:15:481
1889728020,37sleep118931-21munin-run12:15:481
54952797,14sleep20-21swapper/211:31:202
54952797,14sleep20-21swapper/211:31:202
3151427921,36sleep131662-21grep07:46:211
1949627920,28sleep10-21swapper/107:10:561
381927827,31sleep13957-21grep11:26:181
381927827,31sleep13957-21grep11:26:181
2356727816,48sleep223570-21cron12:30:472
2344627811,52sleep30-21swapper/309:05:493
1162927820,25sleep10-21swapper/111:51:181
1926627712,49sleep019030-21/usr/sbin/munin08:51:080
1871527719,31sleep10-21swapper/110:31:191
1871527719,31sleep10-21swapper/110:31:191
1760827714,46sleep017663-21systemd-run07:05:500
1553927726,33sleep215553-21sh12:05:482
444827611,16sleep00-21swapper/011:30:490
444827611,16sleep00-21swapper/011:30:490
2519727622,27sleep00-21swapper/007:26:210
2008227612,27sleep00-21swapper/007:11:110
1758427616,44sleep017641-21fschecks_count08:46:080
1650727517,45sleep016510-21df_inode10:26:050
1650727517,45sleep016510-21df_inode10:26:050
606927421,38sleep26109-21http08:10:492
46832748,16sleep30-21swapper/311:31:013
46832748,16sleep30-21swapper/311:31:013
1134627420,30sleep00-21swapper/008:26:050
1134627420,30sleep00-21swapper/008:26:050
279482738,53sleep327994-21ntpq11:01:133
2194227313,46sleep121973-21proc_pri07:16:171
1932327324,23sleep10-21swapper/108:51:091
1832727335,19sleep30-21swapper/307:06:133
175862737,15sleep30-21swapper/312:10:583
175862737,15sleep30-21swapper/312:10:583
466027215,41sleep10-21swapper/111:30:591
466027215,41sleep10-21swapper/111:30:591
436927219,41sleep30-21swapper/308:03:513
1619427216,24sleep20-21swapper/210:25:512
1580327221,33sleep015822-21apt-get12:05:580
2257727112,28sleep10-21swapper/112:26:101
1711527125,25sleep20-21swapper/212:10:492
124427124,32sleep21367-21latency_hist07:55:512
1154327111,44sleep111575-21/usr/sbin/munin08:26:101
1154327111,44sleep111575-21/usr/sbin/munin08:26:101
1019327123,27sleep00-21swapper/008:21:170
1019327123,27sleep00-21swapper/008:21:170
882627016,36sleep18955-21users11:41:241
1779227018,32sleep20-21swapper/210:30:522
149327012,45sleep11569-21timerandwakeup09:36:201
137427022,23sleep20-21swapper/211:20:502
137427022,23sleep20-21swapper/211:20:502
3176626914,35sleep231739-21/usr/sbin/munin09:31:042
2898526915,39sleep129045-21switchtime09:21:181
2428626918,10sleep30-21swapper/307:25:473
2428626918,10sleep30-21swapper/307:25:473
2392826912,42sleep023992-21/usr/sbin/munin09:06:060
1846326921,30sleep20-21swapper/210:31:162
1846326921,30sleep20-21swapper/210:31:162
8822687,14sleep30-21swapper/309:36:013
752026816,28sleep10-21swapper/111:40:481
497526816,10sleep30-21swapper/309:47:243
372226815,38sleep23732-21awk08:01:162
2636726821,22sleep20-21swapper/209:13:152
2621726814,15sleep20-21swapper/207:31:072
1651626815,35sleep216611-21perl10:26:072
1447826820,27sleep00-21swapper/008:36:060
1240726816,29sleep012532-21ls10:11:190
456626721,20sleep00-21swapper/008:05:500
2232626719,25sleep00-21swapper/010:45:150
2115826710,43sleep121202-21ntp_offset12:21:141
2115826710,43sleep121202-21ntp_offset12:21:141
1897826721,22sleep00-21swapper/012:15:520
1897826721,22sleep00-21swapper/012:15:520
204226615,19sleep10-21swapper/111:21:141
204226615,19sleep10-21swapper/111:21:141
1895626614,38sleep21-21systemd12:15:492
1895626614,38sleep21-21systemd12:15:492
765326519,26sleep10-21swapper/109:56:181
722526512,38sleep07322-21timerandwakeup08:11:270
418826515,20sleep20-21swapper/209:46:102
3177626522,23sleep00-21swapper/011:11:360
3177626522,23sleep00-21swapper/011:11:360
2116326519,27sleep20-21swapper/207:15:492
1280626521,21sleep20-21swapper/211:56:072
117026520,29sleep11240-21systemd-run07:55:501
115126520,28sleep01168-21cron07:55:490
3184926413,37sleep131862-21fschecks_count09:31:071
252126417,23sleep20-21swapper/209:41:092
2345926412,31sleep223483-21ntp_states07:21:142
2345926412,31sleep223483-21ntp_states07:21:142
186982648,10sleep10-21swapper/108:49:561
1603326417,27sleep10-21swapper/108:41:081
1279726421,23sleep10-21swapper/108:30:591
573426322,18sleep20-21swapper/209:51:112
2589026312,28sleep226014-21latency_hist07:30:512
304426225,20sleep10-21swapper/109:41:201
3021926213,32sleep20-21swapper/209:26:032
2965826212,31sleep229730-21/usr/sbin/munin11:06:172
2965826212,31sleep229730-21/usr/sbin/munin11:06:172
2697526218,26sleep127062-21tune2fs09:16:091
2245926218,28sleep122164-21/usr/sbin/munin09:01:111
2241726214,28sleep222449-21munin-run10:45:502
2226526218,25sleep022271-21diskstats09:01:060
1927426218,28sleep219438-21latency_hist07:10:522
3267926114,19sleep00-21swapper/009:31:220
297126118,21sleep00-21swapper/009:41:190
2587826119,26sleep125889-21munin-run07:30:501
986426010,12sleep20-21swapper/211:46:102
78352606,9sleep10-21swapper/108:15:241
1655626026,18sleep30-21swapper/307:01:093
1071026021,20sleep20-21swapper/210:06:172
996125921,20sleep00-21swapper/010:05:500
98025915,24sleep20-21swapper/209:36:092
771425911,28sleep07922-21sed11:40:510
3138125916,20sleep20-21swapper/211:11:302
3138125916,20sleep20-21swapper/211:11:302
2444225918,25sleep124439-21apt-get10:51:021
2444225918,25sleep124439-21apt-get10:51:021
227425915,19sleep20-21swapper/211:21:192
227425915,19sleep20-21swapper/211:21:192
1488525910,24sleep00-21swapper/010:21:040
1467025915,22sleep10-21swapper/108:36:121
1326725914,26sleep013326-21sensors_temp11:56:170
89932584,45sleep20-21swapper/211:41:462
219742589,12sleep20-21swapper/207:16:182
602625723,19sleep00-21swapper/011:35:500
602625723,19sleep00-21swapper/011:35:500
314025712,23sleep20-21swapper/211:26:032
314025712,23sleep20-21swapper/211:26:032
3104025713,29sleep131112-21unixbench_singl09:26:351
3080225713,28sleep00-21swapper/007:46:030
2770525715,25sleep127699-21apt-get07:36:031
2251825715,24sleep00-21swapper/012:26:090
212325714,23sleep20-21swapper/207:56:272
169825715,27sleep01695-21apt-get07:56:010
1124925716,26sleep211267-21apt08:25:592
1124925716,26sleep211267-21apt08:25:592
52425617,25sleep1548-21proc_pri11:16:331
52425617,25sleep1548-21proc_pri11:16:331
219725620,16sleep00-21swapper/009:40:510
1920925614,25sleep019205-21apt-get12:16:000
1669925621,18sleep10-21swapper/110:26:101
1669925621,18sleep10-21swapper/110:26:101
2907025518,18sleep20-21swapper/209:21:192
2779225515,20sleep20-21swapper/207:36:082
2757725513,25sleep127571-21apt-get11:01:031
254402556,39sleep00-21swapper/010:52:220
254402556,39sleep00-21swapper/010:52:220
481125416,22sleep04806-21apt-get08:06:020
27992546,38sleep10-21swapper/107:58:321
2572525415,22sleep025749-21ntp_states09:11:150
2135925420,18sleep20-21swapper/212:21:192
2135925420,18sleep20-21swapper/212:21:192
1269525414,21sleep20-21swapper/210:11:462
371425315,21sleep10-21swapper/108:01:151
2986425315,16sleep20-21swapper/207:41:172
1798525313,23sleep017979-21apt-get10:31:020
1798525313,23sleep017979-21apt-get10:31:020
1905525213,23sleep219049-21apt-get08:51:022
1620925218,17sleep00-21swapper/008:41:110
10032527,36sleep00-21swapper/011:16:570
10032527,36sleep00-21swapper/011:16:570
2629225112,21sleep126385-21memory10:56:101
2071225114,21sleep020718-21diskstats08:56:090
1818525110,18sleep00-21swapper/007:06:100
1495325115,19sleep10-21swapper/110:21:051
65702505,11sleep20-21swapper/208:11:012
2733725013,22sleep027345-21ntp_states09:16:160
2428925019,18sleep10-21swapper/107:25:491
2428925019,18sleep10-21swapper/107:25:491
2401625012,22sleep224110-21memory09:06:082
2236925013,19sleep10-21swapper/110:45:491
1956125012,23sleep219558-21apt-get10:36:002
317662494,35sleep20-21swapper/207:47:022
2818124914,18sleep20-21swapper/209:20:502
1641424813,17sleep20-21swapper/208:41:172
2911924711,20sleep129117-21apt-get11:05:591
2911924711,20sleep129117-21apt-get11:05:591
2527124715,12sleep10-21swapper/109:11:001
966724611,20sleep19673-21df_inode11:46:051
2812224510,19sleep028159-21ntp_states07:36:140
2292424311,15sleep10-21swapper/110:46:211
222832427,19sleep20-21swapper/209:01:082
1463624111,13sleep20-21swapper/210:20:502
263422407,15sleep00-21swapper/007:31:090
237442398,16sleep10-21swapper/109:05:591
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional