You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-14 - 01:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Sun Sep 13, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19017255396,5521sleep217689-21kworker/2:209:34:282
271142166319,1631sleep227115-21ntp_states08:18:282
2670428908,869sleep226740-21ntpq11:43:252
2670428908,869sleep226740-21ntpq11:43:252
24014256312,533sleep024100-21users11:33:380
22252231245,80sleep20-21swapper/208:03:262
23395230314,213sleep222693-21/usr/sbin/munin09:48:372
3537225944,202sleep23629-21tune2fs12:13:422
22399225113,224sleep122416-21ntpq08:03:291
15853223635,176sleep20-21swapper/207:43:232
1265223678,109sleep20-21swapper/210:23:222
19239221840,77sleep20-21swapper/207:53:252
26023220830,142sleep2231-21systemd-journal11:43:012
32629219823,151sleep332723-21fschecks_time12:03:213
8543219715,109sleep28586-21fschecks_count09:03:232
3427219739,80sleep20-21swapper/210:28:352
13088219219,79sleep20-21swapper/207:33:302
2092321858,125sleep221033-21latency_hist09:43:042
23325218417,27sleep00-21swapper/008:08:040
8857217521,134sleep38991-21unixbench_singl12:28:373
3191521697,109sleep2367-21snmpd11:58:482
980521669,79sleep29850-21latency07:23:242
28439216585,65sleep228510-21switchtime11:48:342
15028216472,26sleep20-21swapper/209:23:222
7642215752,90sleep27776-21grep08:58:342
727521569,134sleep37319-21ntpq08:58:273
24528215010,124sleep124646-21latency_hist11:38:031
18202214611,121sleep218217-21vmstat07:48:362
18202214611,121sleep218217-21vmstat07:48:362
32069214454,71sleep232090-21cpu10:18:172
32069214454,71sleep232090-21cpu10:18:172
21691214469,20sleep2312-21in:imuxsock08:03:022
11568214315,19sleep20-21swapper/209:13:042
27949213615,21sleep30-21swapper/311:48:233
25561213519,87sleep20-21swapper/208:13:322
9341213313,104sleep39489-21latency_hist07:23:053
29968213247,49sleep229986-21apt08:28:132
29968213247,49sleep229986-21apt08:28:132
23506213231,89sleep223522-21acpi08:08:152
14484213235,39sleep20-21swapper/211:03:322
7250213052,58sleep27395-21unixbench_multi12:23:352
6670213010,77sleep20-21swapper/207:13:232
8389212945,43sleep20-21swapper/212:28:282
8077212948,46sleep28103-21fschecks_count07:18:242
27145212741,16sleep20-21swapper/210:03:032
27145212741,16sleep20-21swapper/210:03:032
2162821278,108sleep221140-21/usr/sbin/munin09:43:332
2162821278,108sleep221140-21/usr/sbin/munin09:43:332
23195212643,47sleep223184-21apt-get11:33:202
28698212520,20sleep20-21swapper/210:08:042
28698212520,20sleep20-21swapper/210:08:042
2851212516,80sleep20-21swapper/208:43:322
18524212529,24sleep20-21swapper/211:18:142
18524212529,24sleep20-21swapper/211:18:142
3535212317,88sleep10-21swapper/112:13:411
1479721235,109sleep00-21swapper/011:03:460
14575212316,54sleep20-21swapper/207:38:272
1354212312,98sleep31406-21if_err_eth010:23:233
608021228,101sleep2231-21systemd-journal12:20:282
8219212115,92sleep08354-21cat09:03:040
28716212139,19sleep212-21rcu_sched08:23:332
11029212118,86sleep011162-21latency_hist12:38:030
2895212036,31sleep22930-21modprobe12:13:012
3253721198,82sleep20-21swapper/212:03:162
16105211938,19sleep20-21swapper/211:08:372
12071211837,14sleep20-21swapper/210:58:032
9797211517,77sleep39921-21latency_hist09:08:043
30700211417,82sleep330791-21expr08:29:073
30700211417,82sleep330791-21expr08:29:073
17418211458,15sleep20-21swapper/211:13:292
17418211458,15sleep20-21swapper/211:13:292
16675211219,71sleep316698-21modprobe11:13:023
8164211022,69sleep18183-21ls09:03:011
2538821105,12sleep20-21swapper/209:56:252
27387210913,29sleep00-21swapper/010:03:170
27387210913,29sleep00-21swapper/010:03:170
2658121088,14sleep00-21swapper/011:43:230
2658121088,14sleep00-21swapper/011:43:230
13765210811,79sleep013801-21ntpq09:18:270
561421069,43sleep25742-21ls10:38:042
22627210629,19sleep20-21swapper/211:29:282
22627210629,19sleep20-21swapper/211:29:282
4885210420,64sleep34988-21/usr/sbin/munin10:33:333
5322210328,20sleep25316-21apt-get08:53:172
31752210311,77sleep331822-21systemd-run10:18:023
10445210217,65sleep110469-21modprobe10:53:011
10445210217,65sleep110469-21modprobe10:53:011
2192210114,75sleep12254-21switchtime12:08:321
3201521008,76sleep30-21swapper/308:33:483
28671210016,70sleep028675-21ntpq08:23:310
26343210026,55sleep026344-21cron08:18:020
10920210010,31sleep210960-21awk12:38:012
3205529625,15sleep20-21swapper/208:33:512
162429615,64sleep11634-21ntp_states10:23:291
963629520,51sleep19637-21/usr/sbin/munin12:33:111
2174429521,23sleep00-21swapper/008:03:040
666329417,63sleep16815-21latency_hist08:58:041
585529415,64sleep15877-21cpu10:38:191
3119529434,16sleep20-21swapper/210:13:332
18162906,64sleep00-21swapper/012:08:230
91952898,45sleep29187-21apt-get10:48:222
91952898,45sleep29187-21apt-get10:48:222
2559428911,63sleep025724-21sort09:58:040
971328819,54sleep19754-21idleruntime-cro09:08:021
110852875,42sleep20-21swapper/207:28:172
3066528625,41sleep030676-21modprobe11:58:010
646428526,38sleep06547-21timerandwakeup10:38:340
307442859,64sleep230804-21latency_hist11:58:032
2999728516,34sleep10-21swapper/108:28:161
2999728516,34sleep10-21swapper/108:28:161
19772857,63sleep22028-21ls08:43:032
1339428518,11sleep20-21swapper/209:18:172
632728411,54sleep20-21swapper/210:38:322
478228424,41sleep04903-21latency_hist12:18:040
2932428412,55sleep129372-21ntpq10:08:311
2607428417,48sleep026161-21wc11:43:030
2140628311,40sleep20-21swapper/211:28:032
1263428316,36sleep30-21swapper/307:33:193
1221128311,56sleep112257-21ntp_offset09:13:271
767328220,47sleep17727-21sendmail07:18:021
2241728212,50sleep322418-21cron09:48:023
2241728212,50sleep322418-21cron09:48:023
1970328226,23sleep00-21swapper/009:38:220
1049528238,26sleep310582-21grep10:53:033
1049528238,26sleep310582-21grep10:53:033
76402817,33sleep20-21swapper/210:43:172
76402817,33sleep20-21swapper/210:43:172
38612810,6sleep30-21swapper/308:48:203
164972819,53sleep20-21swapper/209:28:222
3125928025,30sleep00-21swapper/008:33:040
3125928025,30sleep00-21swapper/008:33:040
45342798,32sleep20-21swapper/207:08:222
131552798,58sleep213196-21latency_hist09:18:042
509427814,49sleep35101-21diskstats12:18:193
457227812,54sleep3367-21snmpd12:16:213
37662789,17sleep20-21swapper/208:48:162
1882127815,17sleep10-21swapper/111:18:251
1882127815,17sleep10-21swapper/111:18:251
1774627817,46sleep117777-21munin-plugin-st09:33:021
2543027614,38sleep125053-21/usr/sbin/munin08:13:261
711027521,34sleep37131-21munin-run10:43:013
2311727513,45sleep023155-21ntpq09:48:310
3214327418,28sleep00-21swapper/010:18:190
3214327418,28sleep00-21swapper/010:18:190
2562427428,30sleep325697-21switchtime08:13:333
1151627410,46sleep011538-21basename09:13:020
2955727316,28sleep30-21swapper/310:08:373
2931127314,27sleep00-21swapper/010:08:300
172427316,39sleep31814-21fschecks_time12:08:223
170942730,11sleep017074-21mailstats09:28:350
1508327315,43sleep315084-21sshd11:07:293
128622736,28sleep20-21swapper/210:58:312
1088027320,30sleep30-21swapper/307:28:063
942627221,23sleep10-21swapper/112:33:031
436127213,34sleep04418-21fschecks_count10:33:210
2719727211,46sleep127267-21sensors08:18:311
2247927215,30sleep10-21swapper/111:28:341
2247927215,30sleep10-21swapper/111:28:341
2009527216,36sleep120153-21sendmail_mailqu09:38:311
1700227213,46sleep317090-21grep09:28:353
683427111,17sleep10-21swapper/112:23:261
2796027126,24sleep00-21swapper/011:48:240
2022427116,24sleep30-21swapper/309:38:333
1420827111,30sleep10-21swapper/107:38:171
606127019,32sleep35776-21munin-node07:13:083
2583027024,29sleep325831-21sshd09:58:153
2583027024,29sleep325831-21sshd09:58:153
2195727010,17sleep30-21swapper/308:03:153
1551927015,34sleep00-21swapper/007:43:050
1166227012,38sleep011760-21/usr/sbin/munin07:28:310
924926913,42sleep19295-21systemctl07:23:031
2497526913,24sleep10-21swapper/111:38:241
1717226917,28sleep00-21swapper/011:13:230
1717226917,28sleep00-21swapper/011:13:230
1227826924,26sleep00-21swapper/009:13:300
1143326913,43sleep1367-21snmpd09:11:581
454126819,27sleep30-21swapper/307:08:233
3185626816,23sleep10-21swapper/110:18:041
3182426816,27sleep00-21swapper/010:18:030
291842687,46sleep20-21swapper/210:08:282
2628526812,25sleep10-21swapper/111:43:121
2628526812,25sleep10-21swapper/111:43:121
1441426829,21sleep10-21swapper/111:03:311
13202687,49sleep10-21swapper/112:07:391
1538326713,37sleep115397-21sshd09:23:291
5832666,12sleep00-21swapper/012:03:300
367826613,30sleep23770-21users07:03:342
351626610,41sleep13526-21munin-run08:48:031
3158926621,25sleep30-21swapper/311:58:333
2959926622,28sleep129683-21apt-config08:28:041
1850826610,13sleep30-21swapper/309:33:323
1470026620,26sleep114750-21grep09:23:041
1319726617,32sleep313337-21sed09:18:053
3135226512,38sleep130960-21/usr/sbin/munin11:58:261
2950126513,33sleep10-21swapper/111:53:221
2018926518,20sleep10-21swapper/107:58:041
192926515,22sleep10-21swapper/108:43:021
1811226514,34sleep018118-21df_inode09:33:210
1534026517,23sleep00-21swapper/009:23:280
1112826520,28sleep10-21swapper/110:53:251
501826410,26sleep00-21swapper/012:18:160
2483826420,21sleep30-21swapper/308:13:043
185112646,10sleep00-21swapper/007:52:260
185112646,10sleep00-21swapper/007:52:260
1082826419,26sleep00-21swapper/010:53:200
893926312,36sleep38949-21munin-run10:48:023
893926312,36sleep38949-21munin-run10:48:023
617126316,29sleep06214-21munin-node07:13:090
2444826319,21sleep30-21swapper/311:38:013
406526218,18sleep30-21swapper/310:33:043
3252426219,25sleep10-21swapper/110:18:281
3252426219,25sleep10-21swapper/110:18:281
3144626212,22sleep00-21swapper/011:58:300
267472629,40sleep326783-21/usr/sbin/munin08:18:213
163432626,44sleep00-21swapper/011:08:430
133092626,45sleep00-21swapper/010:59:300
464626114,28sleep14680-21/usr/sbin/munin10:33:271
2506626120,24sleep325133-21memory11:38:263
2426026110,22sleep30-21swapper/309:53:193
2009826118,20sleep10-21swapper/111:23:171
1830726121,24sleep10-21swapper/109:33:261
65812605,13sleep30-21swapper/308:58:023
2829326012,29sleep328308-21cut08:23:223
2784226014,26sleep327848-21ntp_states10:03:283
2784226014,26sleep327848-21ntp_states10:03:283
248026015,28sleep02559-21cat10:28:040
1942126014,29sleep319492-21switchtime07:53:303
1897026018,22sleep00-21swapper/011:18:280
1897026018,22sleep00-21swapper/011:18:280
928525911,23sleep00-21swapper/010:48:280
928525911,23sleep00-21swapper/010:48:280
852825915,24sleep00-21swapper/009:03:200
538525915,27sleep15412-21awk12:18:251
419725913,21sleep00-21swapper/007:08:040
33925917,24sleep1368-21munin-run08:38:021
1708925915,18sleep30-21swapper/307:48:053
168352595,10sleep10-21swapper/107:45:441
1137925915,28sleep310707-21/usr/sbin/munin10:53:323
1096625912,32sleep311027-21apt-config12:38:023
810125815,18sleep00-21swapper/010:43:350
810125815,18sleep00-21swapper/010:43:350
782525817,20sleep00-21swapper/012:28:010
419725811,18sleep10-21swapper/108:48:281
332925817,28sleep10-21swapper/110:28:341
2271925816,26sleep322786-21df09:48:223
1949725814,21sleep00-21swapper/007:53:310
184212585,11sleep30-21swapper/307:51:163
184212585,11sleep30-21swapper/307:51:163
1730525816,23sleep117372-21df07:48:191
1730525816,23sleep117372-21df07:48:191
990525711,33sleep09937-21awk07:23:260
665125723,16sleep00-21swapper/008:58:030
38152574,11sleep10-21swapper/107:04:011
2948225717,23sleep329489-21diskstats11:53:203
39562564,42sleep00-21swapper/010:32:450
3209325614,26sleep332140-21df10:18:183
3209325614,26sleep332140-21df10:18:183
287025614,17sleep30-21swapper/307:03:193
252682564,13sleep10-21swapper/109:54:501
2210725610,30sleep022114-21open_files11:28:290
2210725610,30sleep022114-21open_files11:28:290
2116425614,25sleep021159-21apt-get09:43:180
2116425614,25sleep021159-21apt-get09:43:180
1830325615,27sleep30-21swapper/311:18:023
1830325615,27sleep30-21swapper/311:18:023
157425615,25sleep31577-21/usr/sbin/munin08:38:363
2403825516,19sleep00-21swapper/008:08:330
1397125516,22sleep00-21swapper/007:38:040
1386225516,21sleep10-21swapper/107:37:411
121882556,15sleep10-21swapper/107:31:031
1025425515,24sleep39733-21apt-get09:08:163
582525414,23sleep15776-21munin-node07:13:061
376425413,25sleep03760-21apt-get08:48:140
2762125417,18sleep30-21swapper/311:48:033
2762125417,18sleep30-21swapper/311:48:033
221325414,24sleep02209-21apt-get08:43:140
203025417,20sleep30-21swapper/308:43:043
244772535,35sleep10-21swapper/108:09:021
2405025312,18sleep10-21swapper/109:53:041
2225225314,26sleep322351-21cat11:28:323
2225225314,26sleep322351-21cat11:28:323
18222535,37sleep00-21swapper/008:41:400
90532525,38sleep00-21swapper/007:20:320
833425213,22sleep18470-21awk10:43:401
833425213,22sleep18470-21awk10:43:401
703625217,17sleep30-21swapper/312:23:323
457325210,14sleep00-21swapper/007:08:240
2582825212,24sleep125825-21apt-get09:58:131
2582825212,24sleep125825-21apt-get09:58:131
1696425211,24sleep116959-21apt-get11:13:161
1696425211,24sleep116959-21apt-get11:13:161
1672625214,22sleep116811-21latency_hist11:13:041
953525113,22sleep19530-21apt-get07:23:151
899525113,19sleep00-21swapper/012:28:380
3154425114,21sleep031529-21apt-get08:33:160
2792125116,18sleep00-21swapper/008:23:020
2178125112,25sleep121906-21cat09:43:361
2178125112,25sleep121906-21cat09:43:361
2135525111,26sleep021356-21cron11:28:010
153132515,10sleep30-21swapper/307:41:063
1531025117,19sleep314917-21/usr/sbin/munin09:23:273
211772504,14sleep30-21swapper/311:25:423
1361825013,21sleep113627-21munin-run11:03:021
1127825016,21sleep30-21swapper/307:28:223
2788424913,16sleep10-21swapper/110:03:301
2788424913,16sleep10-21swapper/110:03:301
2629424912,14sleep00-21swapper/009:58:290
2629424912,14sleep00-21swapper/009:58:290
2836024814,15sleep10-21swapper/111:48:331
1940324811,21sleep319455-21users11:18:353
1940324811,21sleep319455-21users11:18:353
144042475,11sleep30-21swapper/309:19:513
2171024612,17sleep30-21swapper/308:03:033
2043124412,16sleep10-21swapper/107:58:201
127682444,13sleep30-21swapper/310:58:283
1575524211,18sleep00-21swapper/007:43:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional