You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-12 - 01:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Fri Sep 11, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
260322374465,37421sleep13599-21kworker/u8:311:03:541
27854269216,657sleep027916-21/usr/sbin/munin07:43:380
29660245213,397sleep129800-21awk07:48:401
2089236313,337sleep12113-21latency_hist09:48:101
2089236313,337sleep12113-21latency_hist09:48:101
256232336119,199sleep125689-21idleruntime09:18:301
25799228811,163sleep10-21swapper/111:03:131
79732286115,96sleep18053-21fschecks_time08:23:291
79732286115,96sleep18053-21fschecks_time08:23:291
89342273125,37sleep18542-21/usr/sbin/munin10:08:371
2800122699,246sleep228005-21ntpq11:08:352
4152226121,225sleep14216-21latency09:53:311
4152226121,225sleep14216-21latency09:53:311
1936122616,244sleep113860-1kworker/1:0H08:58:391
12283224425,199sleep112407-21latency_hist08:38:111
23878223059,150sleep123901-21modprobe10:58:081
32590222937,22sleep10-21swapper/111:23:331
10738222613,197sleep210742-21diskstats11:58:272
17013222467,56sleep117134-21sed10:33:411
17013222467,56sleep117134-21sed10:33:411
27159221315,179sleep227301-21latency_hist07:43:122
71892211173,23sleep07191-21sshd11:47:260
23200220820,155sleep10-21swapper/107:28:411
12902220647,82sleep112943-21ntpq08:38:361
30759220513,166sleep10-21swapper/111:18:281
21508220326,105sleep321651-21unixbench_singl09:03:423
24977220113,45sleep125121-21unixbench_singl07:33:461
20034220018,154sleep10-21swapper/107:18:401
2253219425,69sleep12361-21cat08:03:511
55221908,169sleep0619-21latency_hist09:43:120
55221908,169sleep0619-21latency_hist09:43:120
28680218614,150sleep128766-21latency_hist07:48:121
9876218415,149sleep19936-21sendmail_mailqu08:28:381
19283217837,59sleep119362-21date10:43:101
19283217837,59sleep119362-21date10:43:101
7905217714,142sleep17510-21/usr/sbin/munin11:48:321
4484217417,133sleep10-21swapper/108:13:111
17745216713,136sleep317824-21threads12:18:393
496216511,135sleep1527-21munin-run09:43:091
496216511,135sleep1527-21munin-run09:43:091
31372216521,124sleep10-21swapper/109:38:081
1097216567,47sleep11140-21gpgv08:03:101
16826216417,130sleep116829-21cron12:18:081
907921628,68sleep10-21swapper/111:53:211
2720721609,130sleep10-21swapper/109:23:301
15828216010,138sleep326043-21kworker/u8:307:08:243
15828216010,138sleep326043-21kworker/u8:307:08:243
31722215912,25sleep131789-21modprobe07:58:091
19091215915,108sleep119129-21ntp_kernel_pll_12:23:341
18383215513,32sleep118462-21xargs07:13:391
23141215414,125sleep123279-21latency_hist12:38:121
10378215414,26sleep10-21swapper/110:13:301
29881214813,98sleep130020-21sed09:33:121
29881214813,98sleep130020-21sed09:33:121
31208214322,92sleep231350-21grep07:53:422
13005214112,112sleep113134-21latency_hist10:23:111
24259213511,18sleep10-21swapper/109:13:331
27811213415,19sleep10-21swapper/111:08:301
444121338,16sleep10-21swapper/111:38:231
21551213399,15sleep10-21swapper/112:33:111
2091521338,17sleep10-21swapper/107:23:241
1840421337,55sleep11-21systemd12:23:091
17545213118,20sleep20-21swapper/212:18:342
4204213010,49sleep14203-21pluginstate11:38:101
4204213010,49sleep14203-21pluginstate11:38:101
1406021308,109sleep114062-21apt-get08:43:201
676521278,40sleep10-21swapper/108:18:371
29328212716,39sleep10-21swapper/111:13:311
3239621256,107sleep212872-1kworker/2:2H07:58:512
2228221257,40sleep1367-21snmpd10:52:311
1771321258,36sleep10-21swapper/108:53:341
1065621259,14sleep10-21swapper/111:58:221
545921238,37sleep10-21swapper/109:58:241
698821227,35sleep10-21swapper/110:03:201
9155212013,89sleep29166-21sshd11:53:262
1630621197,37sleep10-21swapper/108:48:581
598721177,35sleep10-21swapper/111:43:231
1904211765,38sleep12000-21sed11:28:461
1438021175,42sleep10-21swapper/110:26:341
2104121167,31sleep10-21swapper/109:03:321
10727211617,79sleep210874-21latency_hist08:33:122
27754211468,33sleep027879-21unixbench_singl09:23:420
574321125,95sleep10-21swapper/108:15:091
30436211219,67sleep030522-21grep11:18:100
2048121116,31sleep10-21swapper/110:44:191
1319521105,29sleep10-21swapper/112:04:361
2579221085,27sleep10-21swapper/107:38:141
11625210820,74sleep311761-21unixbench_multi08:33:413
2352321075,27sleep10-21swapper/109:12:081
1592521066,28sleep10-21swapper/110:31:271
1592521066,28sleep10-21swapper/110:31:271
1580021065,28sleep10-21swapper/107:08:181
1580021065,28sleep10-21swapper/107:08:181
1400721066,31sleep10-21swapper/112:08:271
346421045,27sleep10-21swapper/109:51:291
1229210419,65sleep31404-21latency_hist08:03:123
2707210311,78sleep02717-21grep09:48:370
14502210022,62sleep214518-21idleruntime-cro10:28:092
1973829818,66sleep00-21swapper/012:25:260
1137329819,49sleep311517-21unixbench_singl11:58:403
2982729717,65sleep229853-21latency_hist09:33:092
2982729717,65sleep229853-21latency_hist09:33:092
434429617,62sleep23857-21/usr/sbin/munin09:53:362
434429617,62sleep23857-21/usr/sbin/munin09:53:362
865329520,56sleep08669-21fschecks_count10:08:320
677329414,59sleep20-21swapper/208:18:382
324562929,62sleep20-21swapper/211:23:292
2879829226,34sleep30-21swapper/311:13:093
211429126,27sleep20-21swapper/209:48:112
211429126,27sleep20-21swapper/209:48:112
894629021,54sleep28995-21awk10:08:382
76112887,22sleep30-21swapper/311:48:263
2115928516,46sleep321248-21fschecks_time10:48:293
1550228530,33sleep015514-21modprobe07:08:090
491128411,60sleep0367-21snmpd09:55:120
491128411,60sleep0367-21snmpd09:55:120
2591228317,50sleep325993-21grep07:38:293
2560028314,52sleep325685-21apt-config11:03:113
1019228320,43sleep210284-21fschecks_time10:13:272
606828222,37sleep06155-21latency_hist08:18:110
412428218,43sleep34164-21awk11:38:083
412428218,43sleep34164-21awk11:38:083
76328114,52sleep0780-21apt09:43:200
76328114,52sleep0780-21apt09:43:200
382628116,50sleep23951-21ls08:08:402
3104627913,45sleep331091-21/usr/sbin/munin11:18:343
2282227915,46sleep222883-21grep09:08:372
1377927917,34sleep20-21swapper/208:43:092
546127713,48sleep35509-21awk09:58:263
312727717,44sleep23146-21uname11:33:312
312727717,44sleep23146-21uname11:33:312
2067727710,52sleep020786-21latency_hist07:23:110
1034227719,41sleep010382-21munin-run11:58:080
598827617,36sleep26023-21apt08:18:092
2596027623,34sleep325978-21apt-get11:03:193
2442427617,43sleep224494-21sensors_temp09:13:382
2286327615,44sleep022897-21irqstats10:53:350
1713827623,24sleep20-21swapper/207:13:122
1713827623,24sleep20-21swapper/207:13:122
3271027514,43sleep032736-21/usr/sbin/munin07:59:130
2259527514,42sleep222614-21apt-get10:53:212
1806127526,30sleep30-21swapper/310:38:283
1806127526,30sleep30-21swapper/310:38:283
1478827518,41sleep214857-21awk07:03:392
2955227417,29sleep30-21swapper/311:13:373
291062749,53sleep329176-21sensors_temp09:28:413
291062749,53sleep329176-21sensors_temp09:28:413
2602527413,28sleep025978-21apt-get11:03:490
197482747,17sleep30-21swapper/312:25:343
1713527419,24sleep20-21swapper/210:33:422
1713527419,24sleep20-21swapper/210:33:422
1705527413,49sleep00-21swapper/008:52:160
1065527417,42sleep310687-21awk08:33:083
2550727319,37sleep225598-21apt-config11:03:102
3208627214,41sleep232120-21/usr/sbin/munin09:38:342
3208627214,41sleep232120-21/usr/sbin/munin09:38:342
3230227119,19sleep30-21swapper/311:23:263
3057827118,24sleep20-21swapper/209:33:372
2226427121,32sleep322322-21/usr/sbin/munin12:33:373
2157627115,39sleep221710-21date12:33:122
2082927114,35sleep00-21swapper/010:48:100
425127026,23sleep30-21swapper/309:53:333
425127026,23sleep30-21swapper/309:53:333
2709627016,38sleep227107-21diskstats09:23:272
1531227020,27sleep00-21swapper/012:13:110
1443127015,30sleep20-21swapper/208:43:332
1436527017,18sleep10-21swapper/107:03:301
1047727017,38sleep310522-21ntp_kernel_pll_10:13:323
916726915,24sleep30-21swapper/308:28:123
916726915,24sleep30-21swapper/308:28:123
286002696,17sleep00-21swapper/011:10:350
2400926910,43sleep024157-21apt-config07:33:120
1690526920,23sleep30-21swapper/312:18:103
3185326820,29sleep331979-21apt-config07:58:113
3046326816,33sleep030458-21apt-get07:53:240
2307926818,26sleep00-21swapper/007:28:400
935426716,29sleep30-21swapper/311:53:313
643426718,25sleep30-21swapper/311:43:343
39012676,13sleep00-21swapper/011:35:190
39012676,13sleep00-21swapper/011:35:190
2811126714,30sleep30-21swapper/307:43:413
2023226714,36sleep220371-21unixbench_singl10:43:402
89362666,12sleep00-21swapper/008:25:550
89362666,12sleep00-21swapper/008:25:550
3104326614,28sleep20-21swapper/211:18:332
2917626620,23sleep20-21swapper/211:13:282
2568726615,24sleep00-21swapper/011:03:120
1527126621,27sleep00-21swapper/010:28:390
1527126621,27sleep00-21swapper/010:28:390
32792658,12sleep20-21swapper/209:49:082
1298626511,38sleep013006-21/usr/sbin/munin08:38:390
805526414,28sleep20-21swapper/208:23:302
805526414,28sleep20-21swapper/208:23:302
232742646,12sleep00-21swapper/009:08:560
1823726421,23sleep20-21swapper/210:38:312
1823726421,23sleep20-21swapper/210:38:312
1430726413,20sleep00-21swapper/008:43:310
834926314,27sleep00-21swapper/010:08:120
802226319,23sleep20-21swapper/211:48:372
2691926315,30sleep226933-21/usr/sbin/munin11:04:112
1989226316,28sleep319937-21awk10:43:333
176242639,28sleep30-21swapper/308:53:323
1599026320,25sleep316075-21grep08:48:523
131332636,11sleep20-21swapper/212:03:482
1230826315,27sleep20-21swapper/210:18:402
522826214,32sleep35318-21grep08:13:383
2385126217,29sleep324004-21dpkg07:33:113
1924726214,32sleep219320-21switchtime12:23:392
1885426211,22sleep20-21swapper/208:58:112
1403226223,17sleep30-21swapper/307:03:123
1347126218,26sleep31-21systemd12:08:093
86272617,16sleep00-21swapper/011:51:020
681926118,19sleep30-21swapper/308:18:393
2884326116,19sleep00-21swapper/009:28:340
2884326116,19sleep00-21swapper/009:28:340
88152608,20sleep30-21swapper/310:08:353
3026026012,32sleep030405-21latency_hist07:53:110
257726013,29sleep02590-21apt11:33:090
227662605,11sleep20-21swapper/212:34:332
181292607,14sleep00-21swapper/012:19:370
1808226018,21sleep20-21swapper/207:13:322
173542606,15sleep00-21swapper/010:34:530
173542606,15sleep00-21swapper/010:34:530
690925912,33sleep06984-21find08:18:400
530125910,32sleep05347-21users11:38:420
447225915,31sleep04454-21/usr/sbin/munin08:13:100
3030025915,26sleep330358-21/usr/sbin/munin09:33:303
220042595,12sleep00-21swapper/010:48:590
2062525917,26sleep220663-21modprobe07:23:092
1564125913,28sleep015653-21/usr/sbin/munin12:13:270
1412125913,31sleep20-21swapper/212:08:532
23182586,11sleep30-21swapper/311:29:493
2078325816,24sleep220777-21apt-get09:03:242
2049325815,20sleep30-21swapper/309:03:093
1455425818,17sleep00-21swapper/010:28:110
1401125816,25sleep314002-21apt-get12:08:293
132925817,25sleep21320-21apt-get11:28:302
120672588,13sleep00-21swapper/010:18:340
627125710,18sleep00-21swapper/009:58:430
3201225716,22sleep00-21swapper/011:23:110
2673425714,22sleep00-21swapper/009:23:090
2415925717,18sleep30-21swapper/309:13:313
2085325714,27sleep020859-21df_inode09:03:260
1720725713,18sleep30-21swapper/307:13:133
1249625717,20sleep30-21swapper/308:38:213
1241925715,24sleep30-21swapper/310:18:413
111592576,40sleep00-21swapper/010:15:230
573725614,27sleep35748-21munin-run11:43:093
253292564,19sleep20-21swapper/207:35:532
2397725615,20sleep00-21swapper/009:13:280
1930525617,23sleep219313-21diskstats08:58:282
1645625620,21sleep20-21swapper/212:13:412
1538525614,23sleep30-21swapper/308:48:113
140725619,16sleep00-21swapper/008:03:130
754125515,24sleep37616-21mailstats10:03:393
724725513,26sleep27295-21systemd-run11:48:092
2246025513,24sleep222457-21apt-get07:28:222
210142559,32sleep220890-21/usr/sbin/munin07:23:292
182925512,18sleep30-21swapper/308:03:403
2547425414,25sleep325469-21apt-get09:18:243
2140025413,25sleep321478-21mailstats07:23:393
1764125411,27sleep017725-21switchtime07:13:180
1515525412,28sleep315164-21/usr/sbin/munin10:28:343
1515525412,28sleep315164-21/usr/sbin/munin10:28:343
1292625416,18sleep00-21swapper/012:03:420
3019025314,24sleep030199-21diskstats09:33:270
2312825312,23sleep00-21swapper/012:38:100
1808325313,24sleep018183-21perl10:38:290
1808325313,24sleep018183-21perl10:38:290
159225314,22sleep21122-21apt-get08:03:192
1589625312,24sleep215853-21apt-get08:48:462
1479325313,24sleep214786-21apt-get10:28:252
1479325313,24sleep214786-21apt-get10:28:252
986125210,16sleep00-21swapper/008:28:370
719625215,17sleep20-21swapper/210:03:292
476825214,23sleep24811-21memory11:38:322
3017625215,19sleep2231-21systemd-journal07:53:092
2915825212,17sleep30-21swapper/307:48:303
2091425212,23sleep020909-21apt-get07:23:230
2082625218,19sleep30-21swapper/312:28:453
205432529,19sleep00-21swapper/009:03:100
133525217,17sleep30-21swapper/309:43:403
133525217,17sleep30-21swapper/309:43:403
1331425215,22sleep313320-21df_inode10:23:253
1094425214,23sleep210938-21apt-get08:33:242
554325114,21sleep25547-21diskstats09:58:292
354025115,18sleep30-21swapper/308:08:333
3221925111,26sleep032230-21grep11:23:230
279625012,23sleep32802-21ntp_states09:48:403
2763625013,22sleep327751-21sed09:23:413
2678525010,16sleep20-21swapper/209:23:112
2025125011,20sleep00-21swapper/012:28:290
1591725014,20sleep00-21swapper/007:08:290
1591725014,20sleep00-21swapper/007:08:290
3194824917,18sleep30-21swapper/309:38:313
3194824917,18sleep30-21swapper/309:38:313
2782424913,21sleep227830-21ntp_states07:43:362
1845724916,18sleep00-21swapper/008:55:270
258422488,15sleep00-21swapper/007:38:250
2421824812,22sleep323737-21apt-get07:33:203
2242024811,23sleep322427-21diskstats09:08:263
1065524811,21sleep010651-21apt-get11:58:210
1131524712,20sleep010920-21/usr/sbin/munin08:33:340
194372468,15sleep30-21swapper/307:18:263
234812457,15sleep30-21swapper/310:53:463
223282457,14sleep20-21swapper/210:53:092
164172418,16sleep30-21swapper/310:33:273
164172418,16sleep30-21swapper/310:33:273
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional