You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-31 - 01:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Mon Mar 30, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
24142277021,732sleep124170-21/usr/sbin/munin09:23:011
24142277021,732sleep124170-21/usr/sbin/munin09:23:011
25439276012,730sleep325504-21/usr/sbin/munin09:27:353
19277248022,440sleep319284-21/usr/sbin/munin09:07:403
1622123637,341sleep216370-21latency_hist10:42:162
3149229317,258sleep03183-21/usr/sbin/munin08:17:390
31835225167,165sleep231917-21fschecks_time09:47:332
12920224142,123sleep212225-21kworker/u8:107:07:432
12920224142,123sleep212225-21kworker/u8:107:07:432
28829224087,77sleep21-21systemd11:22:132
1205822300,11sleep20-21swapper/212:12:152
27816220824,82sleep227395-21/usr/sbin/munin07:52:412
30748220714,20sleep20-21swapper/211:27:302
20388219423,148sleep120537-21unixbench_singl07:27:451
20388219423,148sleep120537-21unixbench_singl07:27:451
3262321850,12sleep20-21swapper/211:32:382
3262321850,12sleep20-21swapper/211:32:382
10463218518,64sleep20-21swapper/210:22:382
27253217829,59sleep227286-21modprobe11:17:132
3049217431,131sleep23095-21sed08:17:372
27252217342,110sleep20-21swapper/207:52:172
27252217342,110sleep20-21swapper/207:52:172
6239216817,42sleep20-21swapper/208:27:382
14188216519,131sleep214224-21ntp_kernel_pll_12:17:372
14188216519,131sleep214224-21ntp_kernel_pll_12:17:372
19225216340,23sleep20-21swapper/207:25:282
19225216340,23sleep20-21swapper/207:25:282
17815216322,63sleep20-21swapper/209:02:432
20128215714,59sleep20-21swapper/209:12:162
14767215711,59sleep214724-21apt-get07:12:522
595121547,82sleep25971-21dpkg11:52:262
31253215483,18sleep20-21swapper/209:44:342
4175215360,81sleep24212-21cat10:02:352
17158215114,118sleep217303-21grep10:42:442
12558215113,121sleep212568-21acpi08:47:252
19813214915,54sleep20-21swapper/207:27:342
19813214915,54sleep20-21swapper/207:27:342
17865214936,96sleep218038-21cat07:22:172
18214214719,106sleep20-21swapper/212:32:152
9708214666,62sleep20-21swapper/208:38:082
23321214273,23sleep20-21swapper/207:37:432
12006213720,20sleep30-21swapper/310:27:353
17975213540,19sleep20-21swapper/210:47:292
13585213312,101sleep027260-21kworker/u8:012:17:150
1307213317,96sleep20-21swapper/211:37:242
1307213317,96sleep20-21swapper/211:37:242
4715213213,97sleep14753-21/usr/sbin/munin08:22:361
3094821279,106sleep230983-21ntpq08:02:392
2865121265,112sleep00-21swapper/007:56:550
15526212512,92sleep115664-21unixbench_singl07:13:401
2741321248,93sleep20-21swapper/209:32:432
15976212211,97sleep316010-21wc08:57:353
12338212020,30sleep10-21swapper/108:47:151
7069211919,82sleep00-21swapper/010:12:300
7069211919,82sleep00-21swapper/010:12:300
19744211623,76sleep319808-21sendmail-msp12:37:133
29176211434,64sleep20-21swapper/209:38:282
25053211410,79sleep20-21swapper/211:07:442
1275211411,16sleep20-21swapper/208:12:322
4137211328,16sleep24164-21munin-run11:47:132
16640211126,18sleep216649-21munin-run12:27:142
11119211117,74sleep311212-21fschecks_time08:42:373
11230211026,64sleep311308-21latency07:02:363
32309210914,74sleep20-21swapper/208:07:352
31126210915,78sleep331223-21cat08:02:443
2555621096,15sleep00-21swapper/011:10:240
1829721090,12sleep30-21swapper/307:22:453
1829721090,12sleep30-21swapper/307:22:453
1130321088,74sleep20-21swapper/208:42:402
1998221079,85sleep220016-21ps10:52:422
30312210421,65sleep330430-21latency_hist08:02:153
30312210421,65sleep330430-21latency_hist08:02:153
10710210312,75sleep110799-21mailstats10:22:451
2968021029,22sleep229754-21sed11:22:422
24248210217,15sleep20-21swapper/211:07:152
22009210111,71sleep20-21swapper/210:57:472
1894321018,27sleep20-21swapper/212:32:432
16182210116,71sleep116219-21latency_hist10:42:151
15281210115,72sleep00-21swapper/010:37:380
1464121019,21sleep214726-21cat10:37:152
21944210018,62sleep122093-21unixbench_singl07:32:461
17348210021,59sleep017429-21grep09:02:330
377329915,69sleep03924-21latency_hist10:02:160
2899729916,14sleep20-21swapper/207:57:302
2899729916,14sleep20-21swapper/207:57:302
2864429918,66sleep028097-21cron09:37:250
2275129913,69sleep022770-21apt-get07:37:250
1646929913,72sleep316470-21sshd12:25:343
159152997,73sleep20-21swapper/208:57:342
122582998,77sleep212359-21cut10:27:422
1219729926,55sleep20-21swapper/212:12:282
1219729926,55sleep20-21swapper/212:12:282
1096329924,38sleep00-21swapper/007:02:290
2174329850,19sleep20-21swapper/209:17:162
1434929812,58sleep00-21swapper/008:52:350
885729766,18sleep28867-21munin-run12:02:152
816029725,48sleep00-21swapper/008:32:450
90292968,16sleep20-21swapper/210:17:372
3208629615,32sleep132092-21apt-get08:07:261
2809529621,52sleep028118-21idleruntime-cro09:37:130
252872967,18sleep225283-21apt-get09:27:262
188422960,11sleep00-21swapper/010:47:460
1696829621,48sleep116994-21munin-plugin-st09:02:131
1534629620,56sleep314706-21/usr/sbin/munin07:13:383
42492957,74sleep00-21swapper/010:02:370
2742229513,68sleep327470-21awk07:52:303
682729423,45sleep16874-21latency_hist10:12:151
609929313,16sleep20-21swapper/210:07:442
3229829218,52sleep032386-21tune2fs11:32:330
3229829218,52sleep032386-21tune2fs11:32:330
2517529118,56sleep325326-21unixbench_singl11:07:453
1668929117,21sleep00-21swapper/007:17:380
734029018,52sleep37360-21kernelversion10:12:373
734029018,52sleep37360-21kernelversion10:12:373
731329025,32sleep30-21swapper/311:57:153
3127628910,66sleep131392-21sed11:27:421
2192928928,31sleep30-21swapper/309:17:253
2186328912,56sleep021995-21/usr/sbin/munin10:57:450
2584028812,60sleep125908-21awk09:27:451
20272885,11sleep20-21swapper/209:56:172
1083128816,57sleep110863-21fw_packets12:07:351
1642228719,44sleep016567-21sed08:57:440
994628627,25sleep30-21swapper/308:38:133
373428614,57sleep33698-21latency_hist10:02:153
2574928616,18sleep10-21swapper/111:12:151
243222868,62sleep20-21swapper/207:42:282
239832867,11sleep20-21swapper/209:22:582
239832867,11sleep20-21swapper/209:22:582
2236328618,51sleep122435-21sort11:02:141
151782867,10sleep20-21swapper/208:54:312
841428515,57sleep18489-21latency_hist10:17:151
841428515,57sleep18489-21latency_hist10:17:151
81272855,12sleep20-21swapper/210:14:342
81272855,12sleep20-21swapper/210:14:342
230382855,12sleep20-21swapper/209:19:592
101092856,10sleep20-21swapper/212:03:572
56282846,11sleep20-21swapper/211:51:292
85222837,11sleep20-21swapper/208:35:172
41762835,12sleep20-21swapper/208:22:172
36502836,11sleep20-21swapper/210:01:462
2598728311,58sleep026002-21grep11:12:300
2173128319,47sleep321857-21sed10:57:433
1871228311,17sleep00-21swapper/012:32:370
907228215,49sleep39107-21ntp_states10:17:383
238132826,65sleep30-21swapper/311:02:523
1808428219,45sleep117591-21apt-get07:22:241
1808428219,45sleep117591-21apt-get07:22:241
106452820,7sleep00-21swapper/012:07:250
3066428113,15sleep30-21swapper/311:27:273
2289228111,51sleep00-21swapper/011:02:290
224212816,10sleep20-21swapper/207:36:462
1659828121,40sleep016628-21grep12:27:120
1562528121,29sleep10-21swapper/112:22:371
2522028043,23sleep125229-21vmstat07:42:471
217728022,43sleep32217-21latency_hist09:57:153
1456228014,48sleep014600-21munin-run10:37:130
3101327922,42sleep131045-21proc_pri08:02:421
2909027918,37sleep329091-21/usr/sbin/munin11:22:233
2710927912,55sleep113441-21snmpd11:15:221
245927929,32sleep31-21systemd08:17:133
1367227919,30sleep10-21swapper/110:32:381
444027819,30sleep10-21swapper/110:02:421
803127730,23sleep30-21swapper/308:32:443
52227714,25sleep10-21swapper/111:32:461
52227714,25sleep10-21swapper/111:32:461
2494827714,49sleep30-21swapper/309:26:343
2494827714,49sleep30-21swapper/309:26:343
1824427711,51sleep118275-21grep10:47:361
419027623,30sleep00-21swapper/011:47:150
2085027617,21sleep00-21swapper/009:12:420
1389227616,44sleep013854-21latency_hist08:52:150
782027514,42sleep37864-21latency11:57:363
2444727515,41sleep024298-21/usr/sbin/munin07:42:340
2085827515,38sleep320799-21latency_hist10:57:153
1303127514,47sleep113043-21ntpq08:47:401
84227412,30sleep10-21swapper/109:52:281
69922745,59sleep00-21swapper/011:54:050
2779027423,31sleep10-21swapper/107:52:401
2422927312,46sleep013441-21snmpd09:23:040
2422927312,46sleep013441-21snmpd09:23:040
2350227324,25sleep10-21swapper/111:02:461
1715827316,22sleep30-21swapper/312:27:363
3028227215,42sleep330288-21diskstats09:42:333
1753227216,23sleep30-21swapper/309:02:363
1697427228,31sleep017028-21fschecks_count12:27:330
2338527118,28sleep30-21swapper/307:37:443
1388627120,27sleep10-21swapper/112:17:321
1388627120,27sleep10-21swapper/112:17:321
1387527122,24sleep00-21swapper/012:17:310
1387527122,24sleep00-21swapper/012:17:310
288427024,24sleep10-21swapper/111:42:311
2334227015,40sleep123428-21apt-key09:22:161
1773127019,35sleep117860-21latency_hist07:22:161
453226918,24sleep30-21swapper/310:02:443
302872695,52sleep10-21swapper/111:26:041
2988326917,32sleep030015-21/usr/sbin/munin11:22:440
2614326911,43sleep326194-21latency07:47:383
2614326911,43sleep326194-21latency07:47:383
2293726913,38sleep122997-21idleruntime07:37:341
1383926922,29sleep313890-21unin-run08:52:143
573526813,28sleep30-21swapper/311:52:153
2692326817,34sleep30-21swapper/309:32:323
2536626811,41sleep025373-21df_inode09:27:320
1807226816,35sleep018083-21sort07:22:230
1807226816,35sleep018083-21sort07:22:230
831426726,22sleep10-21swapper/108:32:461
1402526714,31sleep314203-21cat07:12:163
1402526714,31sleep314203-21cat07:12:163
956926623,24sleep10-21swapper/112:02:421
61326612,39sleep0614-21sshd08:08:340
2096226624,17sleep30-21swapper/307:32:153
2096226624,17sleep30-21swapper/307:32:153
201172669,42sleep10-21swapper/109:12:151
1281926615,27sleep00-21swapper/007:07:410
1281926615,27sleep00-21swapper/007:07:410
1183026621,20sleep00-21swapper/010:27:320
1073426615,37sleep310736-21diskstats12:07:333
1053826614,32sleep110562-21munin-run08:42:151
299082657,13sleep00-21swapper/007:57:480
299082657,13sleep00-21swapper/007:57:480
1725826520,27sleep117292-21/usr/sbin/munin12:27:381
1470526517,29sleep014724-21apt-get07:12:250
408826421,21sleep10-21swapper/108:22:161
2732626421,20sleep10-21swapper/109:32:421
2170726416,24sleep10-21swapper/109:17:151
175102648,12sleep10-21swapper/110:44:571
1545526421,25sleep315411-21/usr/sbin/munin08:57:153
59882638,13sleep00-21swapper/010:07:420
2253826315,32sleep322641-21latency_hist07:37:163
187602635,12sleep00-21swapper/009:07:170
18342635,13sleep00-21swapper/011:37:420
18342635,13sleep00-21swapper/011:37:420
1421426311,34sleep114218-21diskstats08:52:321
1227526325,18sleep20-21swapper/207:07:152
1153926319,19sleep00-21swapper/010:27:160
597626215,28sleep15970-21apt-get11:52:291
56362625,11sleep00-21swapper/011:51:350
440626214,31sleep34416-21/usr/sbin/munin08:22:303
2908026217,30sleep129097-21uniq07:57:331
2908026217,30sleep129097-21uniq07:57:331
222026215,21sleep00-21swapper/009:57:160
1247626221,20sleep10-21swapper/112:12:361
1247626221,20sleep10-21swapper/112:12:361
964226114,29sleep09670-21/usr/sbin/munin08:38:060
611626114,32sleep06145-21irqstats08:27:360
2633526111,18sleep00-21swapper/007:47:430
2633526111,18sleep00-21swapper/007:47:430
2165626114,18sleep00-21swapper/007:32:410
1965326116,27sleep019647-21apt-get07:27:280
1965326116,27sleep019647-21apt-get07:27:280
881626018,22sleep00-21swapper/012:02:130
801826023,19sleep10-21swapper/111:57:421
3067426020,17sleep10-21swapper/109:42:431
3049726018,21sleep00-21swapper/009:42:380
298942596,11sleep10-21swapper/109:41:581
96512586,42sleep00-21swapper/010:18:520
429825814,18sleep00-21swapper/008:22:230
286425815,18sleep30-21swapper/308:17:343
1487825814,18sleep30-21swapper/310:37:283
968125714,18sleep30-21swapper/312:02:443
865125719,17sleep10-21swapper/110:17:281
560525714,27sleep35617-21/usr/sbin/munin10:07:323
2823925720,16sleep00-21swapper/011:17:430
118825716,25sleep01184-21apt-get08:12:260
977625618,21sleep00-21swapper/012:02:450
480125611,28sleep14832-21/usr/sbin/munin11:47:401
1219525615,24sleep012190-21apt-get12:12:260
3216925513,17sleep00-21swapper/009:47:390
2625325512,22sleep30-21swapper/311:12:383
2624325521,18sleep10-21swapper/107:47:401
2624325521,18sleep10-21swapper/107:47:401
2036825520,17sleep30-21swapper/309:12:293
1225525517,23sleep312274-21df_inode12:12:303
1225525517,23sleep312274-21df_inode12:12:303
587025415,23sleep35867-21apt-get08:27:273
58222544,38sleep10-21swapper/108:27:171
3202225411,19sleep10-21swapper/109:47:361
2214025418,22sleep00-21swapper/009:17:340
1982525421,13sleep10-21swapper/112:37:151
1809625415,20sleep30-21swapper/310:47:343
1150225417,17sleep30-21swapper/310:27:153
3192025313,14sleep30-21swapper/309:47:343
2887225316,18sleep30-21swapper/311:22:143
2755225313,22sleep127564-21apt-get11:17:241
1684125313,23sleep016878-21ntp_states10:42:380
1422625314,23sleep314245-21ntp_states12:17:383
1422625314,23sleep314245-21ntp_states12:17:383
133425312,25sleep31401-21df11:37:293
133425312,25sleep31401-21df11:37:293
1103425310,18sleep00-21swapper/008:42:340
330725112,21sleep33336-21ntp_states11:42:423
315392519,30sleep00-21swapper/008:03:460
1730625114,15sleep30-21swapper/310:42:453
1300825110,18sleep10-21swapper/107:07:461
1300825110,18sleep10-21swapper/107:07:461
3229425014,21sleep332296-21diskstats11:32:323
3229425014,21sleep332296-21diskstats11:32:323
241425013,21sleep12411-21apt-get09:57:251
1256725018,16sleep10-21swapper/110:27:451
1095225011,21sleep110955-21apt-get07:02:261
246232498,15sleep30-21swapper/307:42:373
175762498,17sleep30-21swapper/307:22:143
133224911,22sleep11327-21apt-get11:37:271
133224911,22sleep11327-21apt-get11:37:271
2918524814,15sleep30-21swapper/309:38:293
1774624812,21sleep11-21systemd09:02:421
254024713,15sleep10-21swapper/108:17:151
1960324615,15sleep10-21swapper/110:52:331
3107824413,16sleep00-21swapper/011:27:370
1951824411,16sleep00-21swapper/010:52:270
325462439,17sleep30-21swapper/308:07:403
1338024010,15sleep30-21swapper/310:32:323
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional