You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-16 - 15:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sat May 16, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3106127876,769sleep323087-21kworker/3:209:49:133
1772270129,653sleep11785-21irqstats08:14:431
14465254125,506sleep3363-21snmpd12:20:263
27089248811,462sleep127130-21ntpq09:34:461
9558248720,452sleep09608-21unin-run10:24:190
10049238311,330sleep310051-21apt-get10:24:303
4638235712,289sleep34723-21switchtime11:49:493
4638235712,289sleep34723-21switchtime11:49:493
929722797,222sleep39362-21awk12:04:473
25287227426,214sleep3363-21snmpd11:13:403
11443226224,196sleep310845-21/usr/sbin/munin08:44:493
5410224163,128sleep35467-21latency10:09:403
5410224163,128sleep35467-21latency10:09:403
28473222649,161sleep328498-21awk11:24:183
28473222649,161sleep328498-21awk11:24:183
4914222547,166sleep34959-21latency08:24:423
4914222547,166sleep34959-21latency08:24:423
1714222014,187sleep3225-21systemd-journal09:59:193
4408221974,125sleep34374-21latency_hist08:24:223
4408221974,125sleep34374-21latency_hist08:24:223
27530221923,182sleep027578-21latency07:54:430
20077221724,175sleep320214-21unixbench_multi10:54:513
1499522170,15sleep312637-21cyclictest10:39:393
9532217729,90sleep39556-21cron10:24:183
31966217140,96sleep332055-21timerandwakeup09:49:503
31919216636,87sleep331973-21apt-key08:09:193
31919216636,87sleep331973-21apt-key08:09:193
12618215913,130sleep112558-21latency_hist07:09:211
12618215913,130sleep112558-21latency_hist07:09:211
25467215511,123sleep30-21swapper/307:49:213
25467215511,123sleep30-21swapper/307:49:213
2390521559,110sleep323911-21systemd-run07:44:203
2390521559,110sleep323911-21systemd-run07:44:203
13039215545,62sleep30-21swapper/307:09:433
27349215330,100sleep30-21swapper/309:34:523
1988821539,106sleep319935-21ntpq07:29:463
14000215211,124sleep314144-21latency_hist08:54:223
1067215121,19sleep10-21swapper/111:39:381
32535214814,89sleep332630-21fschecks_time08:09:563
16818214167,18sleep30-21swapper/312:29:303
1331921409,92sleep313466-21latency_hist12:19:213
28930213915,30sleep029024-21grep07:59:390
1547321389,117sleep315532-21sendmail-msp08:59:193
18877213641,73sleep30-21swapper/312:34:473
20533213518,18sleep30-21swapper/309:14:373
2691213240,38sleep30-21swapper/309:59:513
8425213129,39sleep30-21swapper/308:34:523
1292721296,14sleep30-21swapper/310:34:203
20974212810,55sleep30-21swapper/310:59:353
1782421288,84sleep317844-21apt-get10:49:303
7252212723,46sleep30-21swapper/311:59:313
22906212728,44sleep30-21swapper/311:04:443
17936212725,46sleep312637-21cyclictest07:24:333
2922421248,101sleep328738-21/usr/sbin/munin11:24:463
2922421248,101sleep328738-21/usr/sbin/munin11:24:463
18660212450,25sleep30-21swapper/309:09:203
28917212224,39sleep30-21swapper/307:59:383
21510212115,18sleep00-21swapper/010:59:480
21140211824,72sleep121285-21sed09:14:511
32434211716,84sleep232406-21apt-get08:09:472
1901921175,102sleep00-21swapper/010:53:100
2847321169,46sleep30-21swapper/309:39:403
6168211552,46sleep30-21swapper/308:29:303
6168211552,46sleep30-21swapper/308:29:303
19306211443,59sleep319341-21ntpq09:09:443
26450211220,73sleep10-21swapper/109:34:211
17696211226,74sleep317728-21latency_hist07:24:213
17696211226,74sleep317728-21latency_hist07:24:213
15666211132,63sleep315774-21grep12:24:503
8350211014,75sleep28247-21/usr/sbin/munin10:19:372
23969210915,79sleep024013-21ntpq09:24:430
7141210741,50sleep30-21swapper/310:14:453
23828210615,75sleep123862-21irqstats09:24:401
22800210614,70sleep122892-21/usr/sbin/munin11:04:421
13139210627,27sleep10-21swapper/107:09:451
10632210522,21sleep30-21swapper/308:44:203
6684210312,74sleep06692-21ntp_states08:30:160
6684210312,74sleep06692-21ntp_states08:30:160
27488210316,67sleep027580-21/usr/sbin/munin11:19:430
27488210316,67sleep027580-21/usr/sbin/munin11:19:430
24533210317,65sleep00-21swapper/007:44:450
24533210317,65sleep00-21swapper/007:44:450
24396210311,77sleep124432-21uname07:44:421
24396210311,77sleep124432-21uname07:44:421
564321025,50sleep30-21swapper/311:54:223
564321025,50sleep30-21swapper/311:54:223
10092210128,45sleep210051-21apt-get10:24:542
6283210016,68sleep26370-21sort11:54:492
6283210016,68sleep26370-21sort11:54:492
798929918,61sleep18132-21sed11:59:491
285629935,18sleep30-21swapper/308:19:213
2278429927,47sleep022933-21grep09:19:510
174342985,52sleep30-21swapper/310:47:443
174342985,52sleep30-21swapper/310:47:443
115282986,45sleep30-21swapper/312:13:073
72962978,14sleep20-21swapper/211:59:342
216329721,54sleep02227-21fschecks_time09:59:400
1494829740,20sleep315013-21perl07:14:403
890129614,67sleep28993-21tune2fs12:04:382
267202967,41sleep3363-21snmpd11:17:033
267202967,41sleep3363-21snmpd11:17:033
3068829541,16sleep330729-21ntpq11:29:433
3068829541,16sleep330729-21ntpq11:29:433
1239329516,63sleep112455-21sendmail_mailst12:14:471
445029416,60sleep14460-21/usr/sbin/munin11:49:431
445029416,60sleep14460-21/usr/sbin/munin11:49:431
187472948,12sleep10-21swapper/110:49:521
1847929413,63sleep118370-21/usr/sbin/munin12:34:371
1397129420,57sleep113484-21/usr/sbin/munin12:19:471
1288629412,53sleep20-21swapper/210:34:192
2618029316,58sleep225682-21/usr/sbin/munin07:49:482
2618029316,58sleep225682-21/usr/sbin/munin07:49:482
229229112,55sleep30-21swapper/311:44:203
80252898,15sleep10-21swapper/110:19:191
2577528713,60sleep225901-21sed09:29:502
1781928715,27sleep10-21swapper/109:04:501
1697128719,43sleep216978-21fschecks_count12:29:412
588328521,42sleep16032-21grep10:09:511
588328521,42sleep16032-21grep10:09:511
53528518,46sleep2575-21fschecks_count09:54:372
2794528515,48sleep227988-21awk09:39:192
2664628512,62sleep3225-21systemd-journal07:50:133
2664628512,62sleep3225-21systemd-journal07:50:133
1788328512,56sleep217954-21/usr/sbin/munin09:04:512
148628513,54sleep21487-21/usr/sbin/munin08:14:312
3176328423,26sleep20-21swapper/209:49:452
1965228429,39sleep219678-21munin-run12:39:182
1467828416,52sleep014691-21/usr/sbin/munin08:54:470
133232848,62sleep313392-21basename10:34:403
404328318,49sleep30-21swapper/310:04:483
404328318,49sleep30-21swapper/310:04:483
308642838,59sleep030941-21cat11:29:480
308642838,59sleep030941-21cat11:29:480
2570928313,52sleep00-21swapper/007:49:340
2570928313,52sleep00-21swapper/007:49:340
26082829,16sleep30-21swapper/311:44:373
26082829,16sleep30-21swapper/311:44:373
1021528224,23sleep10-21swapper/110:25:011
3017628019,19sleep10-21swapper/108:04:211
2560328018,27sleep00-21swapper/011:14:300
2560328018,27sleep00-21swapper/011:14:300
222902809,56sleep3284-21dbus-daemon07:39:193
1986428015,50sleep019914-21processes10:54:480
282192795,64sleep30-21swapper/307:55:303
2734727925,31sleep10-21swapper/107:54:401
306252787,57sleep226808-21kworker/2:108:04:342
306252787,57sleep226808-21kworker/2:108:04:342
435627725,36sleep04382-21modprobe08:24:190
435627725,36sleep04382-21modprobe08:24:190
274002770,9sleep20-21swapper/207:54:402
3111327612,18sleep20-21swapper/209:49:202
2745627618,34sleep00-21swapper/009:34:530
176382765,11sleep00-21swapper/007:24:140
176382765,11sleep00-21swapper/007:24:140
1501527614,47sleep00-21swapper/007:14:410
318602757,55sleep331921-21cpuspeed11:34:353
2325827518,11sleep30-21swapper/309:23:453
1686127515,10sleep30-21swapper/309:01:453
1555427522,22sleep10-21swapper/108:59:211
141242756,13sleep00-21swapper/010:35:180
959327416,29sleep00-21swapper/008:39:410
515227413,43sleep25245-21mailstats08:24:492
515227413,43sleep25245-21mailstats08:24:492
2590227418,12sleep30-21swapper/309:29:513
2471127418,10sleep30-21swapper/309:27:333
2015927413,18sleep00-21swapper/009:14:130
1947527419,39sleep219482-21diskstats10:54:372
1639327422,33sleep116480-21tune2fs10:44:391
1639327422,33sleep116480-21tune2fs10:44:391
1396927422,28sleep20-21swapper/208:54:212
857727313,41sleep08647-21/usr/sbin/munin08:34:530
2016727323,26sleep10-21swapper/109:14:191
1701627324,28sleep10-21swapper/107:19:511
1701627324,28sleep10-21swapper/107:19:511
2241327225,25sleep10-21swapper/109:19:441
180772726,14sleep20-21swapper/210:49:412
27332718,50sleep00-21swapper/008:18:450
219802715,55sleep20-21swapper/207:35:182
701027019,25sleep00-21swapper/011:59:190
701027019,25sleep00-21swapper/011:59:190
370527012,42sleep13807-21sed08:19:511
370527012,42sleep13807-21sed08:19:511
201892706,14sleep20-21swapper/207:29:532
66126913,43sleep0721-21/usr/sbin/munin09:54:390
485026913,41sleep14883-21basename10:09:181
485026913,41sleep14883-21basename10:09:181
100232698,47sleep00-21swapper/010:24:230
644726826,20sleep20-21swapper/210:14:202
644726826,20sleep20-21swapper/210:14:202
439426814,23sleep00-21swapper/010:04:530
439426814,23sleep00-21swapper/010:04:530
3077526815,17sleep00-21swapper/008:05:000
3077526815,17sleep00-21swapper/008:05:000
2581226814,23sleep10-21swapper/111:14:391
2581226814,23sleep10-21swapper/111:14:391
1459226814,29sleep10-21swapper/107:14:261
1378126818,25sleep00-21swapper/012:19:420
1344626820,29sleep113450-21/usr/sbin/munin10:34:421
1326926820,27sleep20-21swapper/212:19:202
1259226824,28sleep012617-21latency_hist07:09:200
1259226824,28sleep012617-21latency_hist07:09:200
1058026818,22sleep10-21swapper/108:44:181
980726719,26sleep20-21swapper/208:39:472
539926719,34sleep05440-21munin-run11:54:190
539926719,34sleep05440-21munin-run11:54:190
2917526714,23sleep00-21swapper/009:39:540
2158026724,22sleep10-21swapper/110:59:491
2074926717,36sleep020791-21munin-run07:34:190
1706026716,19sleep00-21swapper/012:29:430
1639126718,27sleep10-21swapper/112:29:201
1523926718,26sleep20-21swapper/210:39:482
670226613,28sleep20-21swapper/208:30:182
670226613,28sleep20-21swapper/208:30:182
473226620,32sleep04787-21fschecks_count08:24:390
473226620,32sleep04787-21fschecks_count08:24:390
2279626617,27sleep20-21swapper/207:39:382
2279626617,27sleep20-21swapper/207:39:382
2270226621,22sleep10-21swapper/107:39:371
2270226621,22sleep10-21swapper/107:39:371
1616226615,31sleep00-21swapper/008:59:440
1392626614,36sleep013955-21idleruntime-cro08:54:190
1215226519,30sleep112223-21sensors_temp10:29:491
1048526520,28sleep110566-21tune2fs12:09:381
2480326412,27sleep20-21swapper/207:44:512
2480326412,27sleep20-21swapper/207:44:512
1919226415,32sleep019216-21munin-run07:29:190
162962647,11sleep10-21swapper/108:59:491
1157626418,27sleep311643-21memory07:04:433
1157626418,27sleep311643-21memory07:04:433
1043326413,35sleep210439-21df_inode12:09:352
261862635,11sleep00-21swapper/009:31:340
2252926317,26sleep222580-21cpuspeed11:04:352
214052636,12sleep10-21swapper/107:34:451
1743926318,25sleep017477-21grep09:04:410
1514626318,31sleep015234-21tune2fs12:24:390
1481626312,36sleep214820-21/usr/sbin/munin12:24:202
3145626214,34sleep1363-21snmpd11:32:521
3145626214,34sleep1363-21snmpd11:32:521
1924326218,28sleep219356-21cat07:29:212
1815926224,23sleep118135-21latency_hist12:34:211
179426220,19sleep10-21swapper/109:59:211
1012726216,23sleep00-21swapper/012:09:210
250526111,36sleep02524-21apt-get11:44:300
250526111,36sleep02524-21apt-get11:44:300
1636926118,27sleep216464-21switchtime08:59:502
1513326114,28sleep20-21swapper/212:24:372
75272609,33sleep00-21swapper/011:59:400
3194426017,26sleep231956-21/usr/sbin/munin11:34:382
250232607,41sleep20-21swapper/211:10:162
2112326012,35sleep021184-21fschecks_time07:34:400
2079226018,18sleep10-21swapper/107:34:201
199952606,12sleep20-21swapper/209:12:062
1540026020,22sleep00-21swapper/010:39:500
1475226016,25sleep20-21swapper/208:54:492
1161426018,25sleep011610-21apt-get10:29:320
961025921,20sleep10-21swapper/110:24:201
416425916,28sleep24259-21switchtime10:04:512
416425916,28sleep24259-21switchtime10:04:512
3261825915,27sleep032644-21munin-run09:54:190
3029525914,18sleep20-21swapper/208:04:222
2876725916,25sleep028836-21df11:24:350
2876725916,25sleep028836-21df11:24:350
257972599,12sleep10-21swapper/107:49:391
257972599,12sleep10-21swapper/107:49:391
253325917,25sleep12600-21df11:44:351
253325917,25sleep12600-21df11:44:351
1678525918,22sleep20-21swapper/210:44:482
1678525918,22sleep20-21swapper/210:44:482
1317525916,22sleep10-21swapper/108:49:521
1172925916,25sleep211735-21ntp_states07:04:462
1172925916,25sleep211735-21ntp_states07:04:462
3234525819,18sleep00-21swapper/011:34:490
2335325817,24sleep20-21swapper/209:24:212
2306225814,14sleep00-21swapper/007:39:450
2306225814,14sleep00-21swapper/007:39:450
1675625814,26sleep016762-21ntp_states10:44:460
1675625814,26sleep016762-21ntp_states10:44:460
157952589,10sleep20-21swapper/207:15:522
1475425815,23sleep10-21swapper/110:39:341
1095625822,19sleep00-21swapper/008:44:380
900225716,25sleep29027-21munin-run08:39:192
595525718,20sleep10-21swapper/108:29:201
595525718,20sleep10-21swapper/108:29:201
260125717,24sleep22607-21df_inode11:44:362
260125717,24sleep22607-21df_inode11:44:362
174282576,40sleep20-21swapper/207:21:292
174282576,40sleep20-21swapper/207:21:292
1213025714,16sleep10-21swapper/108:49:191
943825626,15sleep10-21swapper/112:04:501
3058825616,20sleep00-21swapper/009:44:540
3010125617,17sleep10-21swapper/111:29:201
268442567,16sleep20-21swapper/211:18:402
268442567,16sleep20-21swapper/211:18:402
2483625616,16sleep00-21swapper/011:09:500
2045925613,25sleep020453-21apt-get09:14:340
1973025617,23sleep019824-21sed12:39:200
98125513,25sleep0975-21apt-get11:39:330
514125513,25sleep05134-21apt-get10:09:330
514125513,25sleep05134-21apt-get10:09:330
335025513,17sleep10-21swapper/110:04:211
608125419,16sleep10-21swapper/111:54:441
608125419,16sleep10-21swapper/111:54:441
30425414,18sleep10-21swapper/111:34:541
2721725416,21sleep127285-21df11:19:351
2721725416,21sleep127285-21df11:19:351
2407525412,24sleep124070-21apt-get11:09:301
2308225413,20sleep00-21swapper/011:04:490
2179425426,15sleep20-21swapper/209:19:202
2105825422,18sleep20-21swapper/210:59:382
1417225417,20sleep00-21swapper/007:14:210
105525411,30sleep00-21swapper/008:12:110
827225314,18sleep10-21swapper/108:34:501
3238425313,23sleep131904-21apt-get08:09:301
2907025313,21sleep229162-21users09:39:522
2386325313,25sleep223566-21/usr/sbin/munin09:24:412
1199925314,20sleep00-21swapper/012:14:380
714525212,25sleep07149-21ntp_states10:14:460
3209425218,16sleep10-21swapper/108:09:211
3209425218,16sleep10-21swapper/108:09:211
3028625210,22sleep230305-21apt-get11:29:302
3028625210,22sleep230305-21apt-get11:29:302
2988925216,22sleep129895-21df_inode09:44:391
2937525217,18sleep20-21swapper/211:24:492
2937525217,18sleep20-21swapper/211:24:492
1946825213,24sleep119474-21df_inode10:54:361
106362529,27sleep010729-21memory12:09:410
30872514,36sleep20-21swapper/210:01:362
2876125112,22sleep128756-21apt-get11:24:321
2894225012,23sleep128237-21/usr/sbin/munin09:39:511
1793325011,21sleep117931-21apt-get07:24:311
2728624915,15sleep20-21swapper/211:19:362
2728624915,15sleep20-21swapper/211:19:362
1411924913,15sleep20-21swapper/207:14:192
692724812,18sleep20-21swapper/210:14:402
3135224812,21sleep131354-21apt-get09:49:311
1345124810,21sleep213543-21memory10:34:432
108624811,22sleep21188-21perl11:39:392
446124611,17sleep20-21swapper/211:49:442
446124611,17sleep20-21swapper/211:49:442
109402468,15sleep20-21swapper/208:44:362
61922427,18sleep10-21swapper/108:29:331
61922427,18sleep10-21swapper/108:29:331
309224011,16sleep20-21swapper/208:19:322
309224011,16sleep20-21swapper/208:19:322
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional