You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-28 - 20:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sun Dec 28, 2025 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
5937282766,8255sleep029379-21kworker/0:108:44:290
251052509619,5064sleep213378-21kworker/2:111:29:292
7409239734,342sleep17486-21apt-config07:09:071
2203239768,277sleep01533-21/usr/sbin/munin10:14:370
14112233813,80sleep013941-21/usr/sbin/munin10:54:260
2416622576,202sleep0369-21snmpd09:43:490
25704224311,54sleep00-21swapper/008:04:110
175722223138,66sleep017614-21processes11:04:480
23018221833,146sleep023082-21find07:54:380
12870221614,184sleep012888-21fschecks_count12:34:280
7306221033,151sleep37449-21grep12:14:403
10610219945,20sleep00-21swapper/008:59:270
22122219315,63sleep00-21swapper/011:19:340
22122219315,63sleep00-21swapper/011:19:340
27926219215,122sleep00-21swapper/008:09:360
20597219248,91sleep020148-21/usr/sbin/munin11:14:450
8459218366,103sleep08605-21unixbench_multi10:34:400
31215216919,58sleep00-21swapper/011:49:260
7838216629,85sleep07842-21cron12:19:050
7490216538,84sleep27625-21latency_hist07:09:082
594521625,146sleep00-21swapper/012:09:450
594521625,146sleep00-21swapper/012:09:450
12172216013,131sleep112287-21sort07:20:021
1710215816,127sleep31790-21tr08:29:423
30649215525,29sleep20-21swapper/208:19:272
2029021559,135sleep020324-21ps09:29:350
15623215431,71sleep015637-21grep10:59:270
6612215131,81sleep06650-21awk10:29:310
27796215111,87sleep027799-21apt-get09:54:190
27796215111,87sleep027799-21apt-get09:54:190
8000215022,110sleep00-21swapper/007:09:510
18095214828,64sleep018133-21ls07:39:270
3208821477,130sleep0369-21snmpd10:07:550
12622214717,116sleep312681-21if_eth010:49:293
12622214717,116sleep312681-21if_eth010:49:293
3137214422,65sleep00-21swapper/010:19:220
3267621378,78sleep032732-21sshd10:09:280
13008213776,42sleep00-21swapper/010:49:380
13008213776,42sleep00-21swapper/010:49:380
10307213755,19sleep00-21swapper/012:24:390
31033213620,102sleep331034-21cpu11:49:203
9977213415,105sleep110102-21sed10:39:381
24019212931,77sleep224166-21grep11:24:442
8129212711,54sleep00-21swapper/012:19:180
2484521268,78sleep00-21swapper/009:44:290
1936212516,94sleep21982-21ntpq10:14:312
29959212335,23sleep00-21swapper/008:15:290
29959212335,23sleep00-21swapper/008:15:290
25572212215,88sleep225380-21apt-get08:04:082
18454211756,42sleep00-21swapper/009:24:260
58021166,29sleep00-21swapper/008:24:540
9066211415,79sleep39152-21fw_packets08:54:253
5046211415,83sleep34651-21/usr/sbin/munin10:24:303
15533211317,33sleep30-21swapper/309:14:323
22295211118,72sleep322332-21latency_hist07:54:083
30772211047,40sleep00-21swapper/008:19:290
1250321100,41sleep20-21swapper/209:04:332
1250321100,41sleep20-21swapper/209:04:332
7953210918,73sleep27417-21apt-get07:09:212
26298210925,50sleep10-21swapper/109:49:471
16849210918,75sleep116926-21latency_hist11:04:081
4675210852,39sleep00-21swapper/010:24:180
1570221085,30sleep00-21swapper/007:32:410
1570221085,30sleep00-21swapper/007:32:410
7391210727,59sleep07407-21modprobe07:09:060
24114210618,16sleep10-21swapper/107:59:271
21341210630,37sleep30-21swapper/309:34:193
10297210622,67sleep110335-21ntpq07:14:371
2931121056,50sleep029204-21cron11:44:090
1963021045,30sleep00-21swapper/011:10:450
1963021045,30sleep00-21swapper/011:10:450
12841210411,78sleep312917-21cut07:24:123
2437221036,28sleep00-21swapper/011:26:480
19225210312,66sleep10-21swapper/111:09:501
19225210312,66sleep10-21swapper/111:09:501
17363210211,15sleep20-21swapper/211:04:342
25801210118,63sleep225943-21dpkg09:49:082
10644210114,31sleep010770-21latency_hist10:44:090
4579210036,41sleep00-21swapper/008:39:280
4579210036,41sleep00-21swapper/008:39:280
253142999,74sleep025374-21find11:29:350
1759029911,72sleep317645-21munin-plugin-st07:39:063
1352129919,67sleep213609-21grep07:24:352
90152988,74sleep00-21swapper/008:54:220
684229817,65sleep26920-21sed12:14:312
1423229830,46sleep00-21swapper/009:09:370
1423229830,46sleep00-21swapper/009:09:370
1254329834,16sleep00-21swapper/012:34:080
1254329834,16sleep00-21swapper/012:34:080
1353029725,42sleep113671-21grep12:34:411
1146429710,72sleep011559-21threads10:44:370
1426829632,44sleep114292-21modprobe07:29:061
1208329643,15sleep012171-21cat07:20:010
255329521,53sleep12649-21timerandwakeup11:59:401
255329521,53sleep12649-21timerandwakeup11:59:401
2727429416,59sleep227329-21wc08:09:092
2727429416,59sleep227329-21wc08:09:092
553229318,52sleep35610-21mailstats12:09:363
553229318,52sleep35610-21mailstats12:09:363
1641429330,15sleep00-21swapper/007:34:260
1255829319,57sleep112615-21sendmail_mailst09:04:371
1255829319,57sleep112615-21sendmail_mailst09:04:371
220429226,54sleep02222-21grep11:59:310
220429226,54sleep02222-21grep11:59:310
842529129,29sleep30-21swapper/312:19:293
690729124,42sleep37055-21grep10:29:383
1528229129,37sleep20-21swapper/209:14:272
2614828934,15sleep00-21swapper/009:49:100
2137028930,13sleep00-21swapper/009:34:240
2962528832,13sleep00-21swapper/009:59:310
275242887,12sleep10-21swapper/108:09:251
2232528819,56sleep222397-21sed11:19:402
2232528819,56sleep222397-21sed11:19:402
3270328720,13sleep00-21swapper/011:54:100
2040128711,63sleep220494-21sed09:29:372
50728616,48sleep1576-21fschecks_time11:55:041
404728634,13sleep00-21swapper/008:39:080
2789528628,28sleep10-21swapper/109:54:281
2789528628,28sleep10-21swapper/109:54:281
2586228632,14sleep00-21swapper/008:04:150
2586228632,14sleep00-21swapper/008:04:150
265328521,42sleep22797-21unixbench_singl11:59:412
265328521,42sleep22797-21unixbench_singl11:59:412
126922850,7sleep30-21swapper/309:04:383
126922850,7sleep30-21swapper/309:04:383
1070028520,46sleep00-21swapper/007:14:440
674128422,38sleep36115-21/usr/sbin/munin07:04:383
2663628422,29sleep10-21swapper/111:34:291
1516228421,16sleep00-21swapper/009:14:210
711628315,31sleep10-21swapper/112:14:381
388228310,50sleep00-21swapper/012:04:320
388228310,50sleep00-21swapper/012:04:320
130162838,18sleep00-21swapper/007:24:210
828628229,28sleep10-21swapper/107:10:121
2852328229,41sleep028598-21grep11:39:360
1694028221,31sleep20-21swapper/207:34:572
1682328120,46sleep00-21swapper/009:19:260
1682328120,46sleep00-21swapper/009:19:260
1398928117,43sleep214034-21awk09:09:302
1398928117,43sleep214034-21awk09:09:302
1007128112,49sleep29677-21/usr/sbin/munin12:24:332
398828025,39sleep34039-21sendmail08:39:073
3167528012,52sleep231801-21sed11:49:362
2638428020,15sleep30-21swapper/309:49:533
2070228021,37sleep220725-21munin-run07:49:062
1836828026,26sleep30-21swapper/309:24:253
16492809,16sleep00-21swapper/008:29:390
166227919,46sleep11669-21ntpq08:29:401
1168527910,17sleep20-21swapper/210:44:392
83252786,24sleep00-21swapper/008:49:460
2787627812,28sleep30-21swapper/309:54:263
2787627812,28sleep30-21swapper/309:54:263
2735527810,21sleep00-21swapper/011:35:290
2589527819,42sleep322255-21kworker/u8:108:04:223
2589527819,42sleep322255-21kworker/u8:108:04:223
2386727825,23sleep30-21swapper/307:59:093
210782788,56sleep00-21swapper/007:49:250
955127722,36sleep19634-21wc08:54:351
3063527721,39sleep230667-21munin-run10:04:062
2756727724,26sleep20-21swapper/209:54:082
136142779,54sleep313615-21sshd09:09:193
136142779,54sleep313615-21sshd09:09:193
2075627611,52sleep020849-21cat07:49:080
485427516,32sleep30-21swapper/308:39:353
485427516,32sleep30-21swapper/308:39:353
36027512,49sleep3339-21apt-get11:54:293
356727524,30sleep20-21swapper/212:04:262
356727524,30sleep20-21swapper/212:04:262
27992756,15sleep30-21swapper/311:59:423
27992756,15sleep30-21swapper/311:59:423
1696227523,22sleep30-21swapper/311:04:103
1182727517,24sleep10-21swapper/109:04:081
98932747,14sleep20-21swapper/207:14:212
1598227417,36sleep316179-21cat07:34:093
1598227417,36sleep316179-21cat07:34:093
2613827317,28sleep10-21swapper/111:34:081
2454427319,28sleep30-21swapper/311:29:053
934327220,38sleep29374-21grep10:39:222
34727217,36sleep2339-21apt-get11:54:202
3255327224,27sleep20-21swapper/210:09:262
2960827216,23sleep20-21swapper/208:14:392
2960827216,23sleep20-21swapper/208:14:392
2008027222,34sleep320125-21sed11:14:123
2008027222,34sleep320125-21sed11:14:123
1328527211,45sleep213344-21mailstats12:34:382
1108727219,27sleep30-21swapper/310:44:283
594627123,23sleep20-21swapper/208:44:312
2251327123,24sleep20-21swapper/207:54:192
1655427113,42sleep116615-21/usr/sbin/munin07:34:481
1040327117,30sleep10-21swapper/112:24:401
406927016,32sleep14176-21latency_hist08:39:091
364527012,29sleep20-21swapper/210:19:362
2792327028,26sleep127935-21cat11:39:171
2763027023,21sleep20-21swapper/211:39:062
1447227017,34sleep114565-21mailstats10:54:351
135127012,44sleep21389-21fschecks_time08:29:332
3186826910,43sleep332003-21latency_hist08:24:083
1581326923,27sleep115834-21/usr/sbin/munin07:34:071
1581326923,27sleep115834-21/usr/sbin/munin07:34:071
3133226814,24sleep10-21swapper/110:04:331
2948726813,31sleep229481-21apt-get11:44:192
95426718,25sleep20-21swapper/208:29:082
923926714,35sleep29312-21memory08:54:282
473526715,34sleep14759-21latency_hist12:09:051
473526715,34sleep14759-21latency_hist12:09:051
2827726723,27sleep20-21swapper/209:54:372
2827726723,27sleep20-21swapper/209:54:372
1860726724,22sleep20-21swapper/211:09:182
1860726724,22sleep20-21swapper/211:09:182
1673626721,29sleep316732-21apt-get09:19:213
1673626721,29sleep316732-21apt-get09:19:213
1216526710,17sleep00-21swapper/009:04:280
1216526710,17sleep00-21swapper/009:04:280
1010526720,21sleep30-21swapper/312:24:343
874726619,26sleep30-21swapper/307:10:233
815726620,18sleep20-21swapper/208:49:402
622826618,33sleep36268-21ntp_kernel_err08:44:363
505026620,31sleep15188-21sed08:39:371
505026620,31sleep15188-21sed08:39:371
2985226614,30sleep329352-21/usr/sbin/munin09:59:393
292226616,34sleep33071-21latency_hist10:19:083
2117026618,24sleep10-21swapper/109:34:091
2091126624,19sleep30-21swapper/311:14:523
1288926617,33sleep212926-21ps10:49:362
1288926617,33sleep212926-21ps10:49:362
830326513,26sleep30-21swapper/308:49:413
73592658,17sleep10-21swapper/110:32:351
548226518,28sleep10-21swapper/112:09:331
548226518,28sleep10-21swapper/112:09:331
442826511,14sleep10-21swapper/110:24:071
3219826520,24sleep10-21swapper/108:24:271
2961826518,25sleep30-21swapper/311:44:243
210302659,19sleep30-21swapper/309:33:303
1945426516,18sleep30-21swapper/307:44:233
1553626519,27sleep215533-21apt-get10:59:212
1218226513,37sleep312185-21sshd12:30:033
1218226513,37sleep312185-21sshd12:30:033
3181226417,28sleep1287-21dbus-daemon08:24:061
1958526427,20sleep10-21swapper/109:29:081
1720426415,25sleep10-21swapper/109:19:351
1720426415,25sleep10-21swapper/109:19:351
873026334,18sleep28797-21switchtime12:19:362
777226315,27sleep30-21swapper/310:34:233
3138926322,24sleep331390-21ntp_states10:04:353
2975626325,19sleep10-21swapper/111:44:261
2576126319,28sleep125796-21munin-run09:49:071
1486826318,30sleep114902-21munin-run09:14:061
1486826318,30sleep114902-21munin-run09:14:061
2925026219,22sleep30-21swapper/311:44:073
2831726211,30sleep20-21swapper/211:39:302
2187726218,20sleep10-21swapper/109:34:371
1818126218,21sleep20-21swapper/207:39:292
1679826222,24sleep316842-21systemd-run11:04:063
1405126219,25sleep30-21swapper/310:54:253
1102526215,25sleep20-21swapper/208:59:372
945926113,18sleep10-21swapper/112:24:071
85252617,11sleep10-21swapper/108:52:251
646726113,28sleep10-21swapper/108:44:421
2898526110,14sleep30-21swapper/308:14:233
2898526110,14sleep30-21swapper/308:14:233
226032617,11sleep20-21swapper/209:38:542
193002618,11sleep20-21swapper/209:26:022
1925126114,31sleep319353-21cat07:44:113
1328326125,21sleep30-21swapper/307:24:283
1115526117,28sleep311187-21munin-run07:19:073
937526017,18sleep30-21swapper/310:39:233
385126011,14sleep30-21swapper/312:04:313
385126011,14sleep30-21swapper/312:04:313
1716926014,29sleep117178-21diskstats11:04:291
2447525915,26sleep324470-21apt-get09:44:203
2046025917,21sleep10-21swapper/111:14:421
2024125919,25sleep220248-21df_inode11:14:232
1543425919,18sleep10-21swapper/109:14:301
1167725914,29sleep211696-21ntp_states12:29:332
1167725914,29sleep211696-21ntp_states12:29:332
2418925820,23sleep324197-21diskstats07:59:293
2378725816,27sleep123810-21munin-run07:59:071
667625714,25sleep26684-21ntp_states10:29:332
284682575,45sleep20-21swapper/208:11:112
1587625720,17sleep20-21swapper/207:34:082
1587625720,17sleep20-21swapper/207:34:082
465025618,17sleep10-21swapper/110:24:151
39722566,10sleep20-21swapper/208:38:532
3024225616,22sleep10-21swapper/108:19:061
3024225616,22sleep10-21swapper/108:19:061
2950125615,22sleep10-21swapper/109:59:281
2136525618,22sleep221406-21grep07:49:312
1213425615,23sleep10-21swapper/110:49:061
467925517,21sleep24723-21tr08:39:302
467925517,21sleep24723-21tr08:39:302
270502555,10sleep10-21swapper/108:07:511
270502555,10sleep10-21swapper/108:07:511
2293625515,24sleep223003-21df09:39:232
2239825512,27sleep322520-21sed11:19:413
2239825512,27sleep322520-21sed11:19:413
582925410,18sleep20-21swapper/212:09:392
582925410,18sleep20-21swapper/212:09:392
340325415,23sleep33434-21cut10:19:293
244032547,34sleep10-21swapper/111:27:141
2097025411,16sleep10-21swapper/107:49:171
164825414,23sleep11738-21perl10:14:261
1322025414,24sleep313239-21ntp_states12:34:353
114782547,35sleep20-21swapper/207:19:382
1051825412,25sleep310513-21apt-get08:59:223
228192534,10sleep10-21swapper/111:21:411
228192534,10sleep10-21swapper/111:21:411
2002925319,19sleep120096-21switchtime07:44:371
1396525311,26sleep213961-21apt-get10:54:202
38425215,23sleep3410-21netstat10:09:323
3153425112,17sleep20-21swapper/210:04:382
2492225120,13sleep30-21swapper/311:29:263
3208525011,23sleep132086-21sshd11:51:281
2781025013,16sleep30-21swapper/308:09:313
2768525014,21sleep30-21swapper/311:39:073
2293525013,21sleep122929-21apt-get09:39:221
1673424911,15sleep20-21swapper/209:19:202
1673424911,15sleep20-21swapper/209:19:202
126992499,16sleep10-21swapper/110:49:311
126992499,16sleep10-21swapper/110:49:311
777124810,21sleep27768-21apt-get10:34:222
467124811,21sleep24669-21apt-get10:24:162
777524711,21sleep17842-21df10:34:251
3273324716,16sleep10-21swapper/110:09:291
1163924716,16sleep10-21swapper/112:29:321
1163924716,16sleep10-21swapper/112:29:321
190972465,31sleep10-21swapper/107:43:161
3210524410,17sleep30-21swapper/308:24:213
2314624011,17sleep322910-21/usr/sbin/munin09:39:283
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional