You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-26 - 23:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Fri Dec 26, 2025 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
91792333620,3298sleep226396-21kworker/u8:007:29:252
80042227421,2236sleep221784-21kworker/u8:110:49:312
80042227421,2236sleep221784-21kworker/u8:110:49:312
18856210628,1037sleep0197-21jbd2/mmcblk2p2-09:44:150
23185280915,776sleep023327-21head09:55:090
17725246420,431sleep117761-21ntpq11:19:381
4300237820,338sleep14440-21grep08:54:451
1227231719,279sleep31229-21ntp_states07:04:593
1227231719,279sleep31229-21ntp_states07:04:593
5225231514,257sleep05283-21fschecks_count08:59:330
4278226875,142sleep04439-21latency_hist10:39:150
4278226875,142sleep04439-21latency_hist10:39:150
21779226480,129sleep00-21swapper/008:09:130
26811224218,208sleep126818-21/usr/sbin/munin08:24:321
2474422428,222sleep3369-21snmpd11:43:473
26028224137,177sleep00-21swapper/008:19:470
31595223934,190sleep00-21swapper/012:04:320
28960220822,124sleep029089-21unixbench_multi08:29:440
6135218716,153sleep26198-21latency07:19:362
8571218562,63sleep0372-21sshd12:34:320
8571218562,63sleep0372-21sshd12:34:320
24258218110,156sleep024385-21timerwakeupswit08:14:460
8899216636,114sleep09013-21timerwakeupswit09:09:420
9340216416,131sleep39407-21fschecks_time07:30:153
11221216242,81sleep011302-21cut07:34:440
11865215737,74sleep011991-21grep10:59:470
552215211,129sleep0556-21entropy08:44:300
8720214912,119sleep08819-21apt-config07:29:150
8720214912,119sleep08819-21apt-config07:29:150
6162214855,60sleep00-21swapper/012:24:470
26045214533,34sleep026127-21mailstats10:04:420
26045214533,34sleep026127-21mailstats10:04:420
27287214412,114sleep326804-21apt-get10:09:223
2761214214,110sleep12843-21grep12:14:421
1701221418,120sleep017075-21sendmail-msp11:19:130
3632213917,108sleep23690-21latency_hist07:14:152
3632213917,108sleep23690-21latency_hist07:14:152
329021395,45sleep00-21swapper/007:10:250
329021395,45sleep00-21swapper/007:10:250
2733221396,12sleep00-21swapper/011:49:500
27459213810,114sleep027488-21grep10:09:430
2396621389,114sleep00-21swapper/009:59:250
29056213557,65sleep029069-21awk11:54:460
15964213327,27sleep00-21swapper/009:34:250
15964213327,27sleep00-21swapper/009:34:250
29067213213,103sleep128586-21apt-get10:14:201
4366213013,96sleep24482-21timerwakeupswit12:19:442
19463212820,91sleep319615-21grep08:00:113
851621278,63sleep00-21swapper/010:50:260
851621278,63sleep00-21swapper/010:50:260
27882212615,99sleep00-21swapper/008:27:510
23898212617,37sleep023503-21/usr/sbin/munin11:39:400
27164212517,32sleep227227-21/usr/sbin/munin11:49:482
1498421255,109sleep00-21swapper/007:47:450
1339121246,49sleep00-21swapper/009:24:440
919221237,101sleep00-21swapper/007:29:310
10519212327,72sleep310654-21grep09:14:463
9835212142,57sleep00-21swapper/009:14:310
961721199,77sleep00-21swapper/010:54:290
544021195,46sleep00-21swapper/007:17:480
1585621188,99sleep015891-21ps07:49:420
7568211629,26sleep00-21swapper/012:29:480
18582211611,62sleep018630-21unin-run11:24:130
13357211621,73sleep313507-21unixbench_multi11:04:423
25269211515,85sleep325320-21cat11:44:323
6665211423,73sleep00-21swapper/009:04:250
6665211423,73sleep00-21swapper/009:04:250
6028211448,24sleep00-21swapper/007:19:340
4805211417,84sleep24891-21mailstats07:14:432
18036211420,76sleep118099-21sendmail_mailqu09:39:441
971211219,78sleep00-21swapper/010:28:330
30454211249,18sleep00-21swapper/008:34:450
1860221110,7sleep20-21swapper/207:59:162
6031211016,33sleep00-21swapper/010:44:160
10796211028,58sleep210798-21cron10:59:122
4224210943,23sleep04300-21mailstats12:19:420
31295210915,78sleep131371-21/usr/sbin/munin10:19:461
31295210915,78sleep131371-21/usr/sbin/munin10:19:461
603210820,15sleep00-21swapper/012:09:280
2178321080,38sleep121782-21cron08:09:131
20496210859,36sleep00-21swapper/011:29:310
19239210827,18sleep00-21swapper/008:00:080
29405210724,60sleep229426-21sh11:59:132
26699210710,81sleep326704-21/usr/sbin/munin11:49:373
13509210717,72sleep00-21swapper/011:04:430
685210621,61sleep3701-21fschecks_count12:09:313
619321069,85sleep06205-21acpi10:44:240
31935210631,19sleep00-21swapper/010:24:130
31935210631,19sleep00-21swapper/010:24:130
201721047,52sleep00-21swapper/008:49:270
5555210312,71sleep25649-21/usr/sbin/munin12:24:372
20170210342,14sleep00-21swapper/011:29:140
14526210321,57sleep314663-21unixbench_multi07:44:453
6326210112,75sleep35840-21/usr/sbin/munin07:19:413
29826210162,21sleep229430-21apt-get11:59:172
11363210125,23sleep00-21swapper/009:19:270
466429914,68sleep34138-21apt-get10:39:263
59929822,44sleep1596-21apt-get12:09:251
2999529823,16sleep00-21swapper/011:59:470
2052329830,43sleep320515-21apt-get08:04:273
930629726,39sleep10-21swapper/110:54:141
930629726,39sleep10-21swapper/110:54:141
2294429614,67sleep123019-21mailstats09:55:061
1444929626,58sleep014497-21awk09:29:290
385629423,53sleep33926-21uniq12:19:333
3191729319,53sleep331957-21ntp_kernel_pll_12:04:393
14982937,71sleep01551-21basename10:29:340
256429212,58sleep00-21swapper/012:14:370
2203029237,16sleep00-21swapper/011:34:310
2203029237,16sleep00-21swapper/011:34:310
2956029120,14sleep00-21swapper/010:14:560
1811229148,11sleep00-21swapper/007:58:230
2700728912,61sleep227065-21latency08:24:382
2204428823,39sleep222110-21fschecks_time11:34:332
2204428823,39sleep222110-21fschecks_time11:34:332
785428712,62sleep27864-21ntp_states07:24:412
785428712,62sleep27864-21ntp_states07:24:412
1723728721,45sleep317238-21cron09:39:133
1723728721,45sleep317238-21cron09:39:133
284128617,51sleep12920-21grep07:09:581
284128617,51sleep12920-21grep07:09:581
2306428626,31sleep30-21swapper/309:55:083
1709128615,56sleep217198-21latency_hist11:19:152
117628629,36sleep21226-21grep07:04:582
117628629,36sleep21226-21grep07:04:582
2694228525,38sleep127141-21dpkg10:09:151
2694228525,38sleep127141-21dpkg10:09:151
1293328520,44sleep212988-21latency11:04:332
716828425,27sleep10-21swapper/107:24:151
261702846,11sleep00-21swapper/011:47:150
1183228421,47sleep111890-21processes09:19:411
3122628321,46sleep131249-21modprobe12:04:121
200328334,33sleep12035-21munin-run07:09:131
200328334,33sleep12035-21munin-run07:09:131
355128125,36sleep33700-21unixbench_multi10:34:463
355128125,36sleep33700-21unixbench_multi10:34:463
323912817,64sleep0369-21snmpd08:40:590
3044528127,27sleep30-21swapper/310:19:153
1174028118,46sleep311834-21timerandwakeup10:59:463
863028015,27sleep10-21swapper/112:34:331
863028015,27sleep10-21swapper/112:34:331
282582808,52sleep30-21swapper/311:54:333
225552807,21sleep022617-21mailstats08:09:410
2064028020,15sleep00-21swapper/009:49:280
1302828014,45sleep313092-21fschecks_time09:24:353
1163427914,49sleep311687-21cat09:19:343
2514727817,47sleep125172-21grep11:44:301
1918027811,56sleep019212-21tune2fs09:44:310
152672789,57sleep011-21ksoftirqd/011:11:380
124002787,57sleep012497-21fschecks_time07:39:330
77072777,15sleep30-21swapper/307:24:383
77072777,15sleep30-21swapper/307:24:383
46527611,51sleep20-21swapper/208:44:242
3037627617,42sleep330433-21switchtime08:34:443
1967127614,49sleep319738-21switchtime09:44:423
153872767,56sleep20-21swapper/207:49:282
448327524,18sleep00-21swapper/008:54:480
2426127529,22sleep30-21swapper/309:59:363
2214427514,44sleep122151-21diskstats08:09:301
3263827320,25sleep30-21swapper/308:44:133
211727319,24sleep20-21swapper/208:49:332
859527218,22sleep30-21swapper/310:50:293
859527218,22sleep30-21swapper/310:50:293
504227221,26sleep10-21swapper/112:24:151
46927216,42sleep1485-21cpu08:44:271
411927218,32sleep24142-21munin-run10:39:132
411927218,32sleep24142-21munin-run10:39:132
3137227216,40sleep231507-21unixbench_multi10:19:472
3137227216,40sleep231507-21unixbench_multi10:19:472
1796027214,40sleep318075-21/usr/sbin/munin11:19:443
647527118,23sleep20-21swapper/209:04:152
3139027130,23sleep10-21swapper/108:39:311
1750327127,23sleep10-21swapper/107:54:441
969427021,20sleep20-21swapper/212:39:152
969427021,20sleep20-21swapper/212:39:152
949027013,26sleep20-21swapper/209:14:122
1782927017,38sleep317904-21cat09:39:373
1545827013,24sleep10-21swapper/107:49:311
3062826913,33sleep30-21swapper/312:00:153
1949326919,24sleep20-21swapper/209:44:372
177062698,12sleep30-21swapper/307:54:463
1547326919,21sleep30-21swapper/307:49:333
642226816,33sleep36475-21if_eth010:44:353
3020026819,30sleep230246-21ntpq08:34:392
2881126814,35sleep20-21swapper/208:29:422
2734626828,19sleep30-21swapper/308:24:473
140026816,38sleep21402-21entropy10:29:322
3219526724,24sleep20-21swapper/210:24:232
221182678,42sleep30-21swapper/311:34:343
221182678,42sleep30-21swapper/311:34:343
1904026721,31sleep219087-21cat08:00:022
1870526720,28sleep118197-21apt-get07:59:251
1044326719,31sleep110466-21modprobe07:34:131
722526614,36sleep17243-21/usr/sbin/munin12:29:411
314626622,25sleep3228-21systemd-journal08:52:333
2410226625,18sleep20-21swapper/208:14:442
1731726619,27sleep10-21swapper/109:39:151
1731726619,27sleep10-21swapper/109:39:151
994626521,24sleep10-21swapper/110:54:391
764326520,24sleep10-21swapper/107:24:361
764326520,24sleep10-21swapper/107:24:361
559126510,17sleep30-21swapper/308:59:403
2780526518,23sleep20-21swapper/210:09:532
131726518,28sleep30-21swapper/310:29:293
966626424,20sleep30-21swapper/310:54:333
824126421,23sleep10-21swapper/109:09:271
2418626416,21sleep30-21swapper/308:14:453
2193726419,25sleep222023-21apt-config09:54:142
212826427,25sleep02180-21grep07:09:160
212826427,25sleep02180-21grep07:09:160
2071326414,26sleep30-21swapper/309:49:313
1955826414,36sleep119595-21ps09:44:401
1472026414,29sleep30-21swapper/309:29:363
102392648,43sleep210245-21ntp_states09:14:412
747226321,25sleep17494-21munin-run10:49:131
488926324,22sleep20-21swapper/208:59:142
455926318,28sleep34601-21latency07:14:363
380526317,31sleep33839-21irqstats08:54:353
1352126326,17sleep20-21swapper/209:24:462
1245626316,20sleep10-21swapper/111:04:161
709026216,16sleep20-21swapper/209:04:392
709026216,16sleep20-21swapper/209:04:392
695226218,24sleep30-21swapper/309:04:363
695226218,24sleep30-21swapper/309:04:363
481026212,25sleep20-21swapper/210:40:062
2529726218,23sleep10-21swapper/110:04:151
2452526215,29sleep224596-21/usr/sbin/munin09:59:432
2216226215,28sleep322251-21fschecks_time08:09:323
2021726219,23sleep10-21swapper/111:29:151
1737526222,23sleep20-21swapper/209:39:162
1554726218,20sleep30-21swapper/311:14:153
3018626120,19sleep10-21swapper/108:34:381
2710026120,25sleep10-21swapper/111:49:471
2407526123,20sleep10-21swapper/108:14:431
2378326114,18sleep20-21swapper/209:59:152
2131726120,19sleep10-21swapper/109:49:451
1294626115,19sleep30-21swapper/307:39:443
537826015,19sleep30-21swapper/312:24:343
2962526020,18sleep10-21swapper/111:59:161
1205626015,33sleep212078-21latency_hist07:39:152
800325924,20sleep10-21swapper/109:09:151
800325924,20sleep10-21swapper/109:09:151
35225917,18sleep20-21swapper/212:09:142
287902599,32sleep128993-21sed10:14:161
262425920,22sleep10-21swapper/110:34:151
2115325911,33sleep221255-21timerandwakeup08:04:592
1073525911,18sleep30-21swapper/307:34:313
812525817,22sleep30-21swapper/312:34:153
593125817,22sleep10-21swapper/108:59:461
3268725811,28sleep332726-21wc10:24:403
3230225815,22sleep10-21swapper/110:24:301
3151825816,25sleep131510-21apt-get12:04:291
2206825816,26sleep222062-21apt-get08:09:272
1942925821,20sleep30-21swapper/311:24:453
154222585,13sleep10-21swapper/109:30:351
634525718,20sleep20-21swapper/210:44:342
3269125714,18sleep20-21swapper/208:44:162
2078225721,22sleep20-21swapper/211:29:372
1863225710,30sleep218726-21latency_hist11:24:162
831625615,24sleep38322-21diskstats09:09:293
679925615,23sleep26817-21apt-get12:29:252
475725617,19sleep10-21swapper/107:14:421
308825611,31sleep13137-21irqstats10:34:351
308825611,31sleep13137-21irqstats10:34:351
2562325614,28sleep225659-21/usr/sbin/munin10:04:322
2562325614,28sleep225659-21/usr/sbin/munin10:04:322
2543125611,26sleep225460-21/usr/sbin/munin11:44:352
2527825615,17sleep30-21swapper/308:19:343
224125615,24sleep22316-21fschecks_time12:14:312
2177625616,21sleep10-21swapper/111:34:161
2177625616,21sleep10-21swapper/111:34:161
1546325617,25sleep115509-21systemctl11:14:131
1360725616,18sleep20-21swapper/207:44:152
1290025615,25sleep112932-21irqstats11:04:321
1189625617,19sleep20-21swapper/209:19:422
3068525513,25sleep330680-21apt-get10:19:273
3068525513,25sleep330680-21apt-get10:19:273
2874525514,21sleep10-21swapper/108:29:411
2827625513,25sleep328271-21apt-get08:29:273
2379125511,27sleep223803-21/usr/sbin/munin11:39:382
1478825515,17sleep20-21swapper/211:09:422
1278925520,21sleep112863-21mailstats07:39:421
583625418,16sleep10-21swapper/112:24:441
2590825412,22sleep30-21swapper/310:04:373
2590825412,22sleep30-21swapper/310:04:373
2579625418,16sleep20-21swapper/208:19:452
252525419,22sleep30-21swapper/312:14:363
1130325417,17sleep10-21swapper/107:34:451
1111925413,23sleep211113-21apt-get10:59:282
234125312,21sleep30-21swapper/307:09:353
234125312,21sleep30-21swapper/307:09:353
210425312,24sleep12116-21awk08:49:321
1574925314,16sleep20-21swapper/209:34:152
978325211,25sleep19778-21apt-get09:14:281
2401925218,17sleep10-21swapper/111:39:451
2041025212,24sleep320406-21apt-get11:29:253
191025213,15sleep20-21swapper/212:14:152
2919725112,23sleep229204-21df_inode10:14:462
1621025114,21sleep116216-21ntp_states11:14:421
1392125113,22sleep113924-21df_inode07:44:311
2560825011,23sleep125644-21ntp_states08:19:401
1574925012,23sleep315746-21apt-get11:14:273
1886824913,19sleep10-21swapper/111:24:271
1384424912,20sleep213838-21apt-get07:44:272
529724813,15sleep10-21swapper/110:40:171
5412479,22sleep3550-21diskstats08:44:293
1290624711,22sleep112973-21df09:24:291
882824514,14sleep30-21swapper/312:34:393
882824514,14sleep30-21swapper/312:34:393
190572429,14sleep20-21swapper/211:24:362
159872407,16sleep20-21swapper/209:34:272
159872407,16sleep20-21swapper/209:34:272
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional