You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-15 - 13:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Wed Jul 15, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
190382529120,5258sleep015699-21kworker/0:019:08:010
254612123617,1202sleep225466-21ntp_states21:09:102
1339426537,608sleep113430-21ntpq22:14:081
24817256419,529sleep224855-21/usr/sbin/munin19:24:022
14180247519,442sleep314221-21ntpq23:59:163
19147246817,437sleep319190-21ntpq20:49:093
166402462424,24sleep016642-21sshd20:40:020
166402462424,24sleep016642-21sshd20:40:020
1727224127,391sleep117316-21ntpq00:09:071
20462233910,315sleep220473-21ntpq00:19:092
3462229394,138sleep13610-21latency_hist21:43:441
10331227996,57sleep10-21swapper/122:04:111
274342269109,61sleep10-21swapper/121:14:341
4622255125,109sleep10-21swapper/119:49:081
13741224554,178sleep113764-21apt-get23:58:541
79882232108,21sleep10-21swapper/120:14:031
79882232108,21sleep10-21swapper/120:14:031
20101222735,117sleep120250-21tail19:09:151
31229221214,184sleep231255-21latency_hist21:28:442
10611221216,172sleep310748-21unixbench_multi22:04:173
921222119,187sleep19223-21uname23:44:091
3106622106,177sleep211610-1kworker/2:1H19:44:002
24540219813,145sleep00-21swapper/022:49:140
30743219016,159sleep030778-21awk21:24:180
1730421807,103sleep1364-21snmpd22:27:151
13669218013,140sleep00-21swapper/022:14:150
3230421787,98sleep132360-21sed19:48:441
1497821759,59sleep10-21swapper/122:19:081
20834217348,102sleep10-21swapper/122:38:581
22384217211,139sleep10-21swapper/122:43:581
485121688,145sleep04874-21fschecks_count20:04:010
308721669,82sleep13121-21ps23:24:111
94021649,59sleep10-21swapper/121:34:081
10671215855,83sleep10-21swapper/123:49:051
32033215433,102sleep132075-21fschecks_count23:14:021
17788214512,21sleep10-21swapper/122:29:011
306214428,101sleep1788-21sendmail-mta21:33:431
18425214154,14sleep10-21swapper/100:14:001
31345213819,16sleep10-21swapper/123:09:421
31345213819,16sleep10-21swapper/123:09:421
9918213723,17sleep10-21swapper/120:19:121
18738213628,92sleep10-21swapper/120:48:551
11367213519,18sleep10-21swapper/120:24:081
15327213438,16sleep10-21swapper/100:03:591
29358213145,26sleep10-21swapper/123:04:131
2568521318,110sleep125737-21cat22:54:041
25293213127,90sleep125367-21cat19:24:131
9377212919,11sleep10-21swapper/122:00:231
726421296,112sleep1364-21snmpd20:09:431
17677212911,102sleep117719-21wc20:44:101
6024212811,101sleep16061-21idleruntime-cro20:08:421
29450212739,12sleep10-21swapper/119:38:591
26153212531,79sleep126200-21idleruntime-cro00:38:421
26104212518,11sleep10-21swapper/121:11:351
26311212424,83sleep326397-21systemd-run21:13:433
275121236,12sleep10-21swapper/119:56:511
22090212310,98sleep122166-21tr19:14:161
21876212235,13sleep10-21swapper/120:58:561
1143921227,13sleep10-21swapper/122:08:471
522021185,12sleep10-21swapper/123:30:571
11970211813,86sleep211972-21sshd20:25:022
11920211822,16sleep10-21swapper/123:53:531
4198211730,16sleep13669-21/usr/sbin/munin21:44:131
4198211730,16sleep13669-21/usr/sbin/munin21:44:131
21452211620,79sleep320689-21apt-get19:13:543
24772211417,25sleep10-21swapper/121:08:451
24772211417,25sleep10-21swapper/121:08:451
1579121149,94sleep3364-21snmpd22:22:453
3990211314,78sleep34114-21/usr/sbin/munin19:59:153
2323321129,81sleep10-21swapper/119:19:001
2323321129,81sleep10-21swapper/119:19:001
15151211217,61sleep30-21swapper/300:03:443
1617021117,19sleep10-21swapper/120:39:141
1617021117,19sleep10-21swapper/120:39:141
1295621118,20sleep10-21swapper/120:29:091
6790211010,18sleep10-21swapper/121:53:571
27800211011,27sleep127819-21proc_pri22:59:411
31659210916,26sleep20-21swapper/223:13:422
31659210916,26sleep20-21swapper/223:13:422
29936210712,75sleep329997-21/usr/sbin/munin19:39:133
27284210721,24sleep30-21swapper/322:58:543
28385210617,75sleep10-21swapper/121:18:581
16583210615,72sleep016614-21munin-run00:08:420
417421045,89sleep10-21swapper/119:59:461
26767210414,69sleep226280-21apt-get21:13:542
23161210428,38sleep00-21swapper/019:18:550
23161210428,38sleep00-21swapper/019:18:550
32019210316,28sleep332026-21df_inode23:14:003
14049210316,71sleep014080-21/usr/sbin/munin23:59:130
15430210112,16sleep10-21swapper/120:38:451
2784921007,76sleep10-21swapper/119:33:541
2662029810,74sleep3364-21snmpd22:57:123
2384529819,62sleep223926-21awk00:29:102
63572979,75sleep10-21swapper/121:51:521
62829719,62sleep2696-21swap19:49:132
673529613,65sleep06771-21ntp_states20:09:100
2217429613,67sleep022219-21ntp_kernel_pll_00:24:080
21412965,80sleep10-21swapper/123:21:121
1508729614,56sleep20-21swapper/200:03:432
2758729522,56sleep227635-21unin-run19:33:432
229082955,78sleep10-21swapper/100:26:431
1913429517,61sleep218739-21/usr/sbin/munin20:49:082
103982958,73sleep010459-21processes22:04:140
2642529416,52sleep10-21swapper/119:29:021
1728529416,61sleep217292-21df_inode20:43:592
734729312,68sleep07427-21sed21:54:130
213702935,11sleep10-21swapper/120:56:011
985429112,60sleep29901-21ntp_offset20:19:092
245332916,12sleep10-21swapper/100:32:451
2373929115,63sleep223815-21sed19:19:142
2373929115,63sleep223815-21sed19:19:142
2345129118,52sleep223457-21grep21:03:572
2345129118,52sleep223457-21grep21:03:572
82642906,12sleep10-21swapper/123:39:571
67662907,11sleep10-21swapper/123:35:531
2697929015,59sleep227187-21sed22:58:452
239022907,11sleep10-21swapper/122:48:511
1971129011,21sleep30-21swapper/322:34:083
1818429022,49sleep318253-21/usr/sbin/munin19:04:133
1409129012,61sleep314092-21/usr/sbin/munin20:33:593
209322895,11sleep10-21swapper/100:19:211
202862895,11sleep10-21swapper/122:35:431
1070628922,51sleep310720-21apt20:23:423
310932887,11sleep10-21swapper/121:27:301
2945228822,46sleep229532-21awk23:04:142
2802028817,54sleep228033-21switchtime23:00:062
2549828714,48sleep324707-21/usr/sbin/munin19:24:153
1988728712,60sleep219951-21ps19:09:122
1010828718,51sleep210123-21apt23:48:422
28942868,15sleep30-21swapper/319:58:433
1895228527,35sleep118955-21dpkg19:07:101
134728521,18sleep00-21swapper/023:19:050
938528420,31sleep30-21swapper/323:44:153
921128412,56sleep39358-21sed20:18:453
921128412,56sleep39358-21sed20:18:453
2688428315,55sleep326968-21timerandwakeup19:29:123
1786628317,27sleep00-21swapper/020:44:130
912928227,33sleep09157-21munin-run20:18:420
912928227,33sleep09157-21munin-run20:18:420
1745128216,46sleep217543-21switchtime00:09:122
386728116,44sleep23988-21timerwakeupswit19:59:142
3264228127,27sleep00-21swapper/019:49:030
1777028122,32sleep20-21swapper/219:04:042
1259528116,29sleep00-21swapper/020:29:000
1160128123,29sleep30-21swapper/320:24:143
41532807,50sleep30-21swapper/321:44:113
41532807,50sleep30-21swapper/321:44:113
698327820,40sleep36984-21cron23:38:423
2064627819,27sleep00-21swapper/022:38:450
1205227823,38sleep012127-21grep22:09:140
3034727717,42sleep330383-21ntp_states21:24:093
2369227713,51sleep223809-21latency_hist22:48:442
3070727618,39sleep030728-21munin-plugin-st19:43:420
2153227620,36sleep021566-21tune2fs19:13:550
3254127520,38sleep032626-21xargs23:14:130
312702759,48sleep031364-21memory19:44:060
2508027523,27sleep30-21swapper/300:34:043
1013327525,28sleep00-21swapper/020:19:150
56882747,16sleep20-21swapper/221:49:102
328827413,48sleep03412-21sed23:24:140
1553427413,44sleep215301-21/usr/sbin/munin00:04:042
2814027316,24sleep00-21swapper/023:00:080
194942737,15sleep30-21swapper/319:09:013
361227221,22sleep00-21swapper/019:59:080
2899927212,43sleep029063-21/usr/sbin/munin23:04:040
2187027224,29sleep00-21swapper/020:58:550
178042728,13sleep00-21swapper/000:09:310
155352727,16sleep00-21swapper/022:19:270
2787427112,23sleep00-21swapper/019:33:570
2392727113,43sleep323990-21switchtime00:29:113
198272717,11sleep00-21swapper/020:51:040
3076927016,31sleep00-21swapper/023:09:080
3076927016,31sleep00-21swapper/023:09:080
2885327014,28sleep30-21swapper/323:03:583
2468327018,33sleep024833-21latency_hist00:33:440
1941827014,26sleep00-21swapper/019:08:580
1345426911,14sleep30-21swapper/322:14:103
1089226920,26sleep30-21swapper/323:49:113
807526814,26sleep20-21swapper/220:14:042
807526814,26sleep20-21swapper/220:14:042
655226815,26sleep00-21swapper/021:53:440
3156226812,23sleep30-21swapper/319:44:143
2216326810,43sleep30-21swapper/300:24:073
1346826818,33sleep013672-21latency_hist23:58:450
495126714,38sleep24953-21cron21:48:422
495126714,38sleep24953-21cron21:48:422
3273726719,30sleep0304-21awk21:33:420
2245626713,38sleep322465-21diskstats22:44:003
2128226718,22sleep30-21swapper/300:23:423
2110526717,23sleep00-21swapper/022:39:050
1322526711,21sleep30-21swapper/320:29:153
1225626713,26sleep00-21swapper/020:28:430
65432660,9sleep30-21swapper/321:53:423
2444326615,26sleep30-21swapper/319:23:433
2444326615,26sleep30-21swapper/319:23:433
2411426619,23sleep20-21swapper/222:49:042
2187526614,31sleep20-21swapper/219:14:092
181182668,29sleep30-21swapper/322:29:073
1441026615,29sleep20-21swapper/222:18:452
3019326513,35sleep230329-21sed19:39:162
2921126519,23sleep30-21swapper/319:38:443
1534826524,25sleep215360-21modprobe20:38:422
1117926512,24sleep20-21swapper/223:49:142
699226413,37sleep07061-21munin-run23:38:430
765726319,21sleep30-21swapper/320:13:453
58482636,11sleep20-21swapper/220:06:252
240926312,35sleep02538-21awk19:54:160
220826314,24sleep20-21swapper/219:54:142
1797126318,21sleep00-21swapper/022:29:040
295502627,43sleep00-21swapper/021:22:320
2316026222,23sleep30-21swapper/321:03:423
2180826215,25sleep30-21swapper/322:40:053
1624126219,27sleep20-21swapper/222:24:022
466126118,28sleep24739-21sshd23:29:122
325172617,13sleep30-21swapper/319:48:543
2736226117,27sleep027433-21sensors_temp21:14:330
2521926116,29sleep025292-21tr19:24:120
2265426016,17sleep00-21swapper/022:44:060
2167626018,20sleep20-21swapper/220:58:442
1337926013,27sleep313464-21apt-config23:58:443
445025915,20sleep00-21swapper/021:44:160
445025915,20sleep00-21swapper/021:44:160
3040125913,17sleep30-21swapper/323:08:593
3040125913,17sleep30-21swapper/323:08:593
2678025919,24sleep026840-21rm22:58:430
26122599,12sleep00-21swapper/021:39:110
152842595,11sleep00-21swapper/020:37:520
860625814,26sleep38635-21irqstats21:59:043
769125819,24sleep30-21swapper/321:54:163
2134525818,25sleep021450-21apt-config00:23:430
196912586,12sleep30-21swapper/300:18:383
1277525815,27sleep212784-21munin-run22:13:442
964025714,25sleep29685-21munin-run22:03:432
3155525716,25sleep331559-21diskstats21:29:013
2229625714,22sleep20-21swapper/220:59:092
2177425720,19sleep20-21swapper/200:23:542
210162575,13sleep20-21swapper/222:39:032
1242925718,24sleep212511-21mailstats23:54:112
2787625613,25sleep227869-21apt-get19:33:592
2701125614,25sleep30-21swapper/321:14:243
238825618,16sleep30-21swapper/323:23:443
1732825613,28sleep317335-21ntpq00:09:093
1624525612,23sleep20-21swapper/220:39:152
1624525612,23sleep20-21swapper/220:39:152
992725515,23sleep29920-21apt-get22:03:592
814825511,25sleep38201-21users23:39:143
572925516,21sleep20-21swapper/223:34:012
270325516,24sleep32709-21diskstats23:24:013
1852425514,25sleep318643-21date20:48:443
885225410,23sleep00-21swapper/023:44:000
530225417,19sleep30-21swapper/320:04:123
2859425419,15sleep30-21swapper/321:19:043
2838025415,23sleep228377-21apt-get21:18:552
273972545,37sleep00-21swapper/019:31:150
2487325416,15sleep20-21swapper/200:33:552
2386825413,16sleep00-21swapper/021:04:070
2386825413,16sleep00-21swapper/021:04:070
2092825418,21sleep021019-21sed20:54:140
1983225416,25sleep219928-21sed22:34:112
186492545,13sleep20-21swapper/222:29:432
1575925413,23sleep315852-21perl20:39:043
1575925413,23sleep315852-21perl20:39:043
1272125412,29sleep312722-21sendmail-mta22:13:333
56625312,24sleep0560-21apt-get21:33:580
425525314,23sleep34259-21diskstats23:29:013
31892535,13sleep20-21swapper/221:40:552
2585625313,22sleep025914-21users21:09:160
170902535,36sleep00-21swapper/022:24:320
1281525316,22sleep212877-21latency20:29:062
1097325316,23sleep011059-21sed23:49:120
836625212,24sleep08360-21apt-get21:58:570
701125213,19sleep20-21swapper/220:09:162
417625212,23sleep04170-21apt-get23:28:580
250602525,35sleep30-21swapper/322:52:043
2345925213,23sleep323522-21df21:03:583
2345925213,23sleep323522-21df21:03:583
393525110,28sleep23996-21latency_hist23:28:442
3147325112,23sleep231469-21apt-get21:28:562
260425112,15sleep30-21swapper/321:39:103
237825111,21sleep22387-21munin-run23:23:432
203942517,16sleep30-21swapper/320:54:013
1482625111,15sleep20-21swapper/222:19:052
2544425013,16sleep00-21swapper/000:34:130
191025017,15sleep20-21swapper/221:38:452
737424912,21sleep07382-21diskstats23:38:590
730024914,18sleep20-21swapper/223:38:572
1870524915,19sleep00-21swapper/000:14:070
1695924912,22sleep317057-21users22:24:153
64424815,17sleep30-21swapper/321:34:013
725524716,14sleep20-21swapper/221:54:112
320782418,15sleep20-21swapper/223:14:032
186312395,14sleep20-21swapper/200:14:052
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional