You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-16 - 19:31

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Mon Mar 16, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28757222799,2236sleep128561-21kworker/1:108:17:241
28757222799,2236sleep128561-21kworker/1:108:17:241
11848217887,1764sleep011850-21ntp_states10:48:290
2721924787,406sleep12403-1kworker/1:0H11:38:181
10702238910,364sleep110751-21awk12:28:251
133112363151,184sleep16442-21cyclictest07:28:171
207272327166,139sleep10-21swapper/109:33:311
1712023070,7sleep00-21swapper/007:38:330
6768228491,17sleep10-21swapper/107:08:221
6807227512,198sleep16879-21fschecks_time10:33:201
6807227512,198sleep16879-21fschecks_time10:33:201
11396226779,26sleep111406-21cpu10:48:181
23161224867,35sleep16442-21cyclictest07:58:261
25766224370,106sleep10-21swapper/111:34:021
8253224184,88sleep10-21swapper/108:53:301
6118223264,96sleep16204-21cut12:13:311
10087221718,144sleep110227-21latency_hist12:28:051
24121221613,185sleep123825-21/usr/sbin/munin11:28:241
15404221515,183sleep115548-21latency_hist09:18:051
8558221029,28sleep10-21swapper/107:13:101
32062219826,150sleep332185-21wc10:08:463
17534219810,170sleep117139-21/usr/sbin/munin09:23:261
2194219312,47sleep12326-21timerwakeupswit08:33:351
16687218912,25sleep10-21swapper/111:03:331
2154218614,80sleep10-21swapper/110:18:221
981621817,162sleep13586-21kworker/u8:010:43:091
21930218115,152sleep022036-21tune2fs09:38:230
5238217718,136sleep10-21swapper/110:28:191
5238217718,136sleep10-21swapper/110:28:191
793217513,148sleep1939-21cut11:58:041
31612217520,76sleep10-21swapper/110:08:361
13794217466,24sleep10-21swapper/109:13:041
21927217378,26sleep10-21swapper/107:53:331
11284217313,140sleep00-21swapper/012:29:130
11301217222,68sleep10-21swapper/109:03:271
5995217116,113sleep10-21swapper/108:48:051
6513216713,76sleep10-21swapper/108:48:251
6513216713,76sleep10-21swapper/108:48:251
16150216712,80sleep10-21swapper/109:18:331
19801216311,21sleep10-21swapper/111:13:301
22320215612,105sleep122392-21mailstats09:38:321
1582621569,65sleep1281-21dbus-daemon11:03:041
10682215410,82sleep110716-21ps07:18:321
25030215016,73sleep125137-21fschecks_time09:48:251
1769121476,13sleep30-21swapper/311:08:193
5154214116,103sleep05218-21ps08:43:320
8488213915,21sleep00-21swapper/007:13:090
3335213963,54sleep10-21swapper/108:38:241
20164213712,64sleep120200-21ntpq07:48:281
20164213712,64sleep120200-21ntpq07:48:281
98121368,63sleep10-21swapper/111:58:171
18303213512,85sleep118440-21unixbench_multi11:08:331
23824213430,77sleep223843-21apt-get11:28:132
28419213331,42sleep128455-21ntpq09:58:271
18880213222,92sleep10-21swapper/109:28:231
23555213155,66sleep113441-21snmpd11:27:561
14281213120,92sleep214405-21cat10:58:042
29787212911,59sleep129843-21if_err_eth010:03:231
16551212920,96sleep316613-21sendmail_mailqu11:03:313
28743212719,16sleep10-21swapper/111:43:141
28743212719,16sleep10-21swapper/111:43:141
15344212610,96sleep315414-21mailstats07:33:323
16114212531,26sleep10-21swapper/107:38:021
31693212311,14sleep10-21swapper/108:23:381
31945212210,95sleep232025-21fschecks_time11:53:212
25632212114,95sleep325628-21apt-get11:33:153
18686212134,43sleep00-21swapper/009:28:130
17354211767,32sleep10-21swapper/107:38:391
9846211619,78sleep19877-21processes08:58:311
6412211545,37sleep30-21swapper/307:08:043
6412211545,37sleep30-21swapper/307:08:043
24689211456,40sleep124741-21latency_hist09:48:051
1692211120,74sleep21821-21sed11:58:342
18980211017,71sleep318998-21uname09:28:253
3138210930,46sleep33133-21apt-get08:38:173
13728210814,17sleep30-21swapper/307:28:273
32028210715,76sleep332077-21idleruntime11:53:223
31428210611,79sleep231508-21tr08:23:352
27732210650,44sleep127767-21munin-run09:58:021
3082210517,68sleep33150-21/usr/sbin/munin12:03:323
2677210414,74sleep02779-21sed10:18:330
13011210310,76sleep10-21swapper/110:53:191
32533210156,21sleep10-21swapper/108:28:181
1599021017,12sleep00-21swapper/009:18:280
15373210111,73sleep015402-21latency_hist09:18:040
1883529910,77sleep213441-21snmpd11:12:262
1630329820,54sleep30-21swapper/309:18:353
1362429718,51sleep313773-21tail10:53:333
131029714,66sleep21323-21latency_hist08:33:042
47729520,36sleep10-21swapper/110:13:171
2933829422,55sleep228842-21apt-get08:18:142
1486029410,69sleep214910-21cpuspeed07:33:192
1241729455,19sleep10-21swapper/107:23:411
1241729455,19sleep10-21swapper/107:23:411
1089829419,51sleep010917-21apt-get09:03:140
1460229227,39sleep214658-21tune2fs10:58:212
305792917,39sleep10-21swapper/111:48:231
2094129112,61sleep021094-21awk09:33:340
1858929157,15sleep10-21swapper/107:43:281
1858929157,15sleep10-21swapper/107:43:281
52552906,35sleep10-21swapper/112:11:531
290429021,38sleep10-21swapper/112:03:251
2415629013,60sleep224204-21/usr/sbin/munin08:03:032
218682905,32sleep10-21swapper/111:21:091
1253829010,64sleep012659-21cat12:33:330
853628910,41sleep18510-21latency_hist12:23:041
2491328911,35sleep10-21swapper/108:03:311
632128713,56sleep26384-21fschecks_time08:48:222
632128713,56sleep26384-21fschecks_time08:48:222
408928714,15sleep10-21swapper/110:23:311
266562879,56sleep10-21swapper/108:08:361
1985128740,31sleep319861-21/usr/sbin/munin07:48:223
1985128740,31sleep319861-21/usr/sbin/munin07:48:223
135482876,69sleep00-21swapper/009:10:290
945328614,58sleep09531-21grep08:58:210
390928611,60sleep04037-21/usr/sbin/munin08:38:350
3100028616,54sleep231221-21date10:08:052
2495428529,31sleep20-21swapper/208:03:322
3002328418,48sleep030037-21apt11:48:020
3002328418,48sleep030037-21apt11:48:020
776428313,48sleep07765-21/usr/sbin/munin08:53:130
878128124,28sleep00-21swapper/010:38:320
878128124,28sleep00-21swapper/010:38:320
294832817,13sleep10-21swapper/108:18:491
1849628112,53sleep018588-21cut07:43:260
1849628112,53sleep018588-21cut07:43:260
1223728112,13sleep10-21swapper/112:33:261
1269428017,45sleep212680-21latency_hist10:53:042
785927813,51sleep27867-21df08:53:192
67172787,14sleep20-21swapper/210:33:132
67172787,14sleep20-21swapper/210:33:132
2609327821,23sleep20-21swapper/208:08:242
281827727,32sleep02844-21munin-run08:38:020
836627627,31sleep30-21swapper/308:53:323
54627618,31sleep30-21swapper/310:13:193
313552767,21sleep20-21swapper/210:08:092
2622627615,45sleep326256-21sed09:53:043
2576827624,27sleep30-21swapper/308:08:053
2352527616,45sleep323591-21idleruntime09:43:233
430327523,25sleep00-21swapper/010:23:340
2006327533,27sleep30-21swapper/311:13:323
3011327419,27sleep30-21swapper/311:48:053
3011327419,27sleep30-21swapper/311:48:053
2406127411,41sleep224188-21unixbench_multi09:43:342
81162737,48sleep10-21swapper/110:38:041
81162737,48sleep10-21swapper/110:38:041
798827320,34sleep38091-21latency_hist07:13:043
604827319,27sleep20-21swapper/208:48:062
157932738,17sleep00-21swapper/011:03:030
1147127316,38sleep211544-21switchtime09:03:322
839727213,39sleep38492-21fschecks_time10:38:233
839727213,39sleep38492-21fschecks_time10:38:233
289827223,28sleep20-21swapper/208:38:042
229927217,39sleep02273-21latency_hist12:03:040
921527128,23sleep39225-21ntp_states12:23:273
645227122,21sleep00-21swapper/007:08:050
645227122,21sleep00-21swapper/007:08:050
2813527115,23sleep00-21swapper/009:58:220
2390427111,43sleep023982-21mailstats09:43:320
74227014,41sleep0790-21cat11:58:030
477527017,26sleep30-21swapper/308:43:223
1972927021,30sleep00-21swapper/011:13:290
927726911,37sleep29361-21mailstats12:23:302
835726919,30sleep38484-21munin-node07:13:083
308182696,15sleep00-21swapper/010:07:260
2674026916,26sleep30-21swapper/309:53:243
264462695,41sleep00-21swapper/009:53:130
2206526915,35sleep222145-21latency_hist11:23:042
132526910,43sleep31475-21sed08:33:053
760326818,26sleep30-21swapper/310:33:343
760326818,26sleep30-21swapper/310:33:343
492826824,23sleep20-21swapper/210:28:042
3005926815,37sleep330065-21/usr/sbin/munin10:03:303
1179826816,36sleep311843-21/usr/sbin/munin10:48:283
1115726820,27sleep00-21swapper/010:48:050
417126720,20sleep00-21swapper/012:08:210
2937726710,42sleep029378-21sshd08:18:240
1986426719,26sleep00-21swapper/007:48:230
1986426719,26sleep00-21swapper/007:48:230
1058726718,25sleep20-21swapper/212:28:222
772026614,24sleep00-21swapper/012:18:310
449826615,33sleep24640-21cut08:43:052
390426619,26sleep00-21swapper/012:08:050
2629326620,29sleep026371-21grep11:34:330
1633026623,20sleep00-21swapper/007:38:050
1578426616,33sleep315791-21modprobe11:03:023
1307526612,27sleep00-21swapper/010:53:220
1080026622,19sleep30-21swapper/307:18:343
521926525,23sleep20-21swapper/207:03:232
521926525,23sleep20-21swapper/207:03:232
2425626515,25sleep00-21swapper/008:03:050
1162126516,32sleep311635-21wc12:33:043
1071826522,20sleep30-21swapper/309:03:053
885626415,34sleep08886-21fschecks_count12:23:190
560726411,37sleep05626-21apt-get12:13:140
2511026412,39sleep325157-21apt-config11:33:033
2294726420,22sleep30-21swapper/311:23:343
1697826414,26sleep00-21swapper/011:03:360
1622626414,35sleep216302-21/usr/sbin/munin09:18:342
727926313,26sleep00-21swapper/007:08:330
3227526316,25sleep00-21swapper/008:28:050
183126316,22sleep00-21swapper/010:18:040
1321026321,18sleep30-21swapper/312:38:043
797726215,28sleep27987-21sshd07:13:032
3155526214,23sleep00-21swapper/010:08:350
26322626,10sleep30-21swapper/308:35:393
539526111,36sleep35537-21cat12:13:053
1213526115,32sleep212144-21needreboot07:23:332
1213526115,32sleep212144-21needreboot07:23:332
827526022,20sleep20-21swapper/210:38:132
827526022,20sleep20-21swapper/210:38:132
255742606,41sleep00-21swapper/008:06:240
226712609,33sleep022695-21netstat11:23:270
1724926015,30sleep017305-21tune2fs09:23:210
1451726013,26sleep00-21swapper/007:32:240
1245926018,23sleep20-21swapper/209:08:132
1029526016,27sleep210301-21diskstats07:18:202
2754325911,25sleep00-21swapper/008:13:190
2754325911,25sleep00-21swapper/008:13:190
2239425920,21sleep20-21swapper/209:38:332
1976625917,25sleep219760-21apt-get07:48:172
1976625917,25sleep219760-21apt-get07:48:172
1234325915,25sleep012416-21/usr/sbin/munin07:23:390
1234325915,25sleep012416-21/usr/sbin/munin07:23:390
104125913,33sleep31056-21df_inode11:58:193
648725815,18sleep20-21swapper/210:33:052
648725815,18sleep20-21swapper/210:33:052
2337425811,21sleep30-21swapper/307:58:323
2168525818,20sleep30-21swapper/307:53:273
1733225816,25sleep217358-21munin-run11:08:022
108022587,15sleep30-21swapper/310:44:083
690725720,23sleep36932-21munin-run12:18:013
3222325719,20sleep30-21swapper/308:28:043
3039625717,19sleep20-21swapper/211:48:202
2899325714,26sleep229136-21apt-config08:18:062
2899325714,26sleep229136-21apt-config08:18:062
207882577,12sleep20-21swapper/209:33:322
1952825713,21sleep20-21swapper/207:48:052
1952825713,21sleep20-21swapper/207:48:052
1025125720,22sleep210293-21awk10:43:312
806225618,16sleep00-21swapper/010:38:030
806225618,16sleep00-21swapper/010:38:030
370025613,29sleep33704-21diskstats10:23:203
2936525618,17sleep30-21swapper/310:03:053
2083925620,20sleep30-21swapper/311:18:213
587625514,22sleep30-21swapper/310:28:333
587625514,22sleep30-21swapper/310:28:333
2657425517,22sleep00-21swapper/008:08:340
2569525512,25sleep225628-21apt-get11:33:592
1764625516,18sleep20-21swapper/207:39:252
1667625517,20sleep30-21swapper/307:38:213
1154625519,19sleep30-21swapper/309:03:333
94625411,22sleep20-21swapper/210:13:302
886625411,17sleep20-21swapper/207:13:242
590025413,16sleep10-21swapper/107:03:341
590025413,16sleep10-21swapper/107:03:341
292025414,24sleep22972-21users10:18:352
2876325413,25sleep228760-21apt-get11:43:152
2876325413,25sleep228760-21apt-get11:43:152
2730525413,21sleep227195-21/usr/sbin/munin11:38:232
2247625418,18sleep30-21swapper/309:38:343
1762125413,25sleep017616-21apt-get11:08:160
1756825419,16sleep20-21swapper/209:23:272
1249125414,23sleep312558-21df09:08:223
293402536,16sleep30-21swapper/308:18:153
2808425314,23sleep328121-21users11:38:363
261625314,24sleep32675-21sensors_temp10:18:323
2538725312,23sleep025393-21ntp_states09:48:320
1893825317,18sleep20-21swapper/209:28:242
1847225316,18sleep30-21swapper/309:28:043
1822025312,23sleep318215-21apt-get07:43:173
1822025312,23sleep318215-21apt-get07:43:173
1214525312,19sleep30-21swapper/307:23:343
1214525312,19sleep30-21swapper/307:23:343
638525216,18sleep30-21swapper/308:48:233
638525216,18sleep30-21swapper/308:48:233
312842525,36sleep00-21swapper/011:49:190
2804725213,23sleep228043-21apt-get09:58:152
2025825212,23sleep320253-21apt-get09:33:173
1530025214,23sleep315371-21true09:18:033
2852925114,22sleep028641-21latency_hist11:43:050
2778725112,18sleep30-21swapper/308:13:263
2778725112,18sleep30-21swapper/308:13:263
2365325113,16sleep20-21swapper/211:28:052
2173225112,24sleep021768-21ntp_states07:53:280
253325012,21sleep22529-21apt-get12:03:142
2131525017,17sleep20-21swapper/207:53:182
1925325011,22sleep219345-21perl11:13:192
1487925011,21sleep214931-21users09:13:342
1411425010,23sleep214212-21users07:28:342
1068225017,16sleep20-21swapper/209:03:042
80224912,21sleep0912-21users08:28:330
2474324912,16sleep20-21swapper/209:48:062
2392324913,20sleep023932-21diskstats11:28:190
2201324918,17sleep00-21swapper/011:23:020
720624810,20sleep37218-21apt-get12:18:133
623324811,21sleep06228-21apt-get08:48:160
623324811,21sleep06228-21apt-get08:48:160
1193624813,17sleep20-21swapper/212:33:192
312992465,30sleep30-21swapper/311:49:303
276392466,31sleep20-21swapper/209:56:502
2599424612,20sleep325991-21apt-get08:08:173
521524510,18sleep00-21swapper/010:28:180
521524510,18sleep00-21swapper/010:28:180
1089624511,14sleep00-21swapper/007:18:350
789224411,16sleep20-21swapper/212:18:332
58732449,20sleep25903-21irqstats12:13:242
429324411,13sleep20-21swapper/212:08:232
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional