You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-06 - 12:30

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Mon Jul 06, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
28191261288,6108sleep31293-21kworker/3:220:46:453
2032253523,493sleep32108-21/usr/sbin/munin19:24:443
19076242415,393sleep119159-21apt-config23:44:161
28909241737,352sleep228999-21fschecks_time00:14:342
298423878,367sleep23031-21cat21:09:372
298423878,367sleep23031-21cat21:09:372
17163233114,302sleep117171-21ntpq21:54:421
2412228718,239sleep02449-21awk21:09:150
2412228718,239sleep02449-21awk21:09:150
14196227114,206sleep21-21systemd20:04:162
14196227114,206sleep21-21systemd20:04:162
2069224148,174sleep22208-21unixbench_singl00:29:532
17633224157,153sleep20-21swapper/220:14:342
31539223611,22sleep20-21swapper/222:39:332
9567222889,24sleep20-21swapper/219:49:382
23826222521,184sleep123860-21idleruntime-cro23:59:151
23826222521,184sleep123860-21idleruntime-cro23:59:151
18370222511,198sleep218450-21tune2fs21:59:362
2814222226,203sleep228168-21df22:29:312
13886220819,147sleep213960-21latency21:44:382
21144220210,28sleep221199-21fschecks_time23:49:352
5777219777,22sleep20-21swapper/222:59:442
227752188118,45sleep20-21swapper/222:14:172
809521798,156sleep28251-21date23:09:172
718221719,150sleep1364-21snmpd19:40:411
32356216910,15sleep30-21swapper/300:24:533
32356216910,15sleep30-21swapper/300:24:533
13372216893,54sleep20-21swapper/223:24:392
15761216391,53sleep20-21swapper/220:09:172
1796321618,55sleep20-21swapper/221:59:152
23861215968,78sleep223892-40mandb23:59:162
23861215968,78sleep223892-40mandb23:59:162
20390215612,128sleep220415-21awk20:24:152
15133215340,100sleep215148-21acpi21:49:262
5528215153,75sleep25531-21cron21:19:142
16381214913,121sleep216466-21latency23:34:372
29944214835,99sleep229965-21munin-run20:54:152
765221459,121sleep37665-21irqstats21:24:363
27564214437,91sleep227595-21munin-run22:29:152
31370214327,83sleep2364-21snmpd00:20:322
19443214174,49sleep219462-21apt-get23:44:262
19443214174,49sleep219462-21apt-get23:44:262
2319521409,15sleep20-21swapper/222:14:412
4909213918,103sleep123183-21kworker/u8:322:59:141
12903213919,93sleep30-21swapper/319:59:333
12903213919,93sleep30-21swapper/319:59:333
27132213840,15sleep20-21swapper/220:44:302
666621377,94sleep26701-21ps19:39:452
18243213511,22sleep20-21swapper/223:39:452
10282213218,95sleep00-21swapper/023:14:400
211121308,107sleep00-21swapper/019:24:450
31225212847,65sleep231318-21latency_hist22:39:172
32189212731,65sleep132182-21apt-get00:24:301
32189212731,65sleep132182-21apt-get00:24:301
2619521279,104sleep226305-21cat20:39:462
18838212716,97sleep00-21swapper/021:59:460
32483212521,17sleep20-21swapper/200:24:562
32483212521,17sleep20-21swapper/200:24:562
28423212535,67sleep328526-21latencyplot19:09:173
28423212535,67sleep328526-21latencyplot19:09:173
23477212214,94sleep023533-21latency22:15:000
4466212013,65sleep10-21swapper/121:14:371
3166821195,47sleep20-21swapper/220:58:232
1581221195,10sleep20-21swapper/223:34:082
3739211816,83sleep33773-21fschecks_count22:54:363
896521176,11sleep20-21swapper/219:48:412
896521176,11sleep20-21swapper/219:48:412
4629211712,60sleep20-21swapper/219:34:322
2661421176,11sleep20-21swapper/200:04:492
829121147,91sleep20-21swapper/221:24:482
20264211415,81sleep020402-21sed23:45:250
20264211415,81sleep020402-21sed23:45:250
19794211416,81sleep319863-21sort23:45:153
19794211416,81sleep319863-21sort23:45:153
1223121145,12sleep20-21swapper/219:57:502
1054821137,91sleep210676-21cat23:14:462
21523211225,70sleep10-21swapper/120:24:501
1696821126,15sleep00-21swapper/020:10:320
321421115,48sleep20-21swapper/222:52:212
2859921118,86sleep228742-21latency_hist00:14:182
1196921118,91sleep212047-21mailstats23:19:472
8724211014,78sleep08869-21grep21:29:180
31878211011,78sleep20-21swapper/219:16:152
2365211016,28sleep10-21swapper/122:49:371
552210938,49sleep20-21swapper/222:44:302
28314210813,19sleep00-21swapper/022:29:460
1056721089,84sleep20-21swapper/221:34:332
305721079,17sleep20-21swapper/219:29:272
251121075,91sleep20-21swapper/219:25:402
22674210710,61sleep20-21swapper/220:29:362
20891210744,17sleep20-21swapper/220:24:272
2958210647,47sleep22856-21/usr/sbin/munin00:34:332
138421068,15sleep20-21swapper/221:04:372
138421068,15sleep20-21swapper/221:04:372
630221057,14sleep20-21swapper/221:19:432
32235210540,18sleep20-21swapper/220:59:362
28962210416,71sleep329019-21latency20:49:403
2048421049,56sleep20-21swapper/222:04:482
16094210433,53sleep116185-21/usr/sbin/munin20:09:361
1308721035,50sleep20-21swapper/221:41:152
2893021028,14sleep20-21swapper/219:09:492
2458121028,52sleep20-21swapper/222:19:182
25471210127,52sleep20-21swapper/200:04:162
25471210127,52sleep20-21swapper/200:04:162
1502429911,73sleep014538-21/usr/sbin/munin23:29:420
90112988,14sleep20-21swapper/221:29:322
27252989,76sleep12727-21sshd19:28:251
10992988,74sleep20-21swapper/219:22:252
2419729710,67sleep20-21swapper/220:34:372
1251029714,66sleep012699-21apt-config19:59:170
93122969,12sleep10-21swapper/121:29:391
815029619,60sleep17370-21/usr/sbin/munin21:24:471
2833929628,46sleep228421-21systemctl19:09:162
2833929628,46sleep228421-21systemctl19:09:162
3095029517,62sleep031016-21/usr/sbin/munin00:19:450
37322946,14sleep20-21swapper/222:54:352
2257029410,16sleep00-21swapper/023:54:290
2257029410,16sleep00-21swapper/023:54:290
160502947,74sleep20-21swapper/220:09:332
83322938,14sleep20-21swapper/223:09:292
985129215,56sleep19852-21/usr/sbin/munin23:14:251
490329213,64sleep35026-21cat21:14:473
210529229,39sleep02100-21apt-get22:49:280
1182629112,61sleep311932-21cat21:39:183
278929015,60sleep12798-21diskstats21:09:321
278929015,60sleep12798-21diskstats21:09:321
824328818,47sleep37723-21/usr/sbin/munin19:44:433
824328818,47sleep37723-21/usr/sbin/munin19:44:433
539628822,18sleep10-21swapper/122:59:351
511228812,56sleep35190-21basename19:34:453
2372228815,57sleep123748-21sh20:34:151
18862878,11sleep00-21swapper/000:29:510
2516028620,49sleep024734-21/usr/sbin/munin22:19:400
250528524,29sleep30-21swapper/321:09:183
250528524,29sleep30-21swapper/321:09:183
2762528428,38sleep327732-21unin-run22:29:163
1725728422,45sleep317277-21modprobe20:14:143
662528315,49sleep35823-21/usr/sbin/munin21:19:483
161228327,27sleep10-21swapper/100:29:431
355628219,19sleep30-21swapper/300:34:463
1398328217,53sleep00-21swapper/023:25:300
842028023,36sleep08459-21cut23:09:350
740028013,53sleep17511-21sed23:04:481
2147728021,42sleep121484-21/usr/sbin/munin23:49:421
1587227919,29sleep30-21swapper/323:34:173
2773527812,49sleep127826-21sed20:44:451
272142780,6sleep10-21swapper/119:04:341
272142780,6sleep10-21swapper/119:04:341
2606227816,38sleep025560-21/usr/sbin/munin20:39:430
2314427815,47sleep322662-21apt-get22:14:263
1445627822,32sleep00-21swapper/020:04:300
746127726,30sleep00-21swapper/019:44:150
3009927714,47sleep030263-21sed20:54:170
1138027724,28sleep10-21swapper/121:34:481
3088727613,17sleep30-21swapper/300:19:443
3030527612,48sleep330314-21/usr/sbin/munin22:35:223
2705827630,24sleep00-21swapper/019:04:260
2705827630,24sleep00-21swapper/019:04:260
2464027613,48sleep124765-21ls20:34:481
1845827610,49sleep018587-21awk23:39:470
1691727613,30sleep30-21swapper/321:54:363
2542027514,38sleep025458-21modprobe00:04:140
2542027514,38sleep025458-21modprobe00:04:140
1255827511,48sleep112640-21mailstats21:39:491
929927420,30sleep30-21swapper/319:49:313
660527411,15sleep00-21swapper/023:04:180
2684227421,39sleep326876-21basename20:44:153
1697527424,26sleep00-21swapper/021:54:370
1073427415,37sleep110945-21sed19:54:181
2857527313,46sleep128597-21latency_hist00:14:171
2526727321,34sleep025291-21munin-run20:39:150
1116927218,35sleep111193-21fschecks_count19:54:341
892327110,48sleep39010-21/usr/sbin/munin23:09:463
686827123,29sleep30-21swapper/323:04:333
2809527112,46sleep128141-21grep22:29:301
117127121,29sleep01155-21apt-get00:29:340
117127121,29sleep01155-21apt-get00:29:340
642327014,36sleep16519-21sed21:19:461
3066027014,37sleep130301-21munin-node19:14:241
2701927015,28sleep10-21swapper/100:09:161
205002705,18sleep10-21swapper/123:46:141
205002705,18sleep10-21swapper/123:46:141
1921427013,43sleep319113-21/usr/sbin/munin20:19:353
1881327015,33sleep3225-21systemd-journal20:19:153
1453527014,39sleep314539-21diskstats20:04:333
1408927019,21sleep00-21swapper/021:44:440
1367927015,38sleep113820-21unixbench_singl20:00:311
1367927015,38sleep113820-21unixbench_singl20:00:311
3173426919,31sleep131757-21munin-run20:59:151
2639426916,32sleep026541-21grep00:04:470
191122699,12sleep00-21swapper/020:19:280
3252726816,24sleep30-21swapper/319:19:353
2773626822,23sleep00-21swapper/022:29:170
2233226815,25sleep00-21swapper/023:54:170
3181226715,38sleep331909-21cat20:59:173
2491126717,35sleep30-21swapper/320:34:503
2030426716,28sleep30-21swapper/322:04:453
1647026723,25sleep10-21swapper/121:54:181
128622679,43sleep00-21swapper/019:59:210
128622679,43sleep00-21swapper/019:59:210
1250326722,26sleep00-21swapper/023:21:260
102772678,49sleep00-21swapper/019:50:410
961226616,25sleep00-21swapper/021:29:460
438126612,37sleep34416-21latency_hist19:34:173
2930026615,37sleep329382-21mailstats00:14:433
1855326614,37sleep118637-21cat21:59:391
328126526,23sleep10-21swapper/100:34:401
2598026515,37sleep326006-21cut00:04:383
2207226522,20sleep10-21swapper/122:09:481
2132526515,28sleep00-21swapper/020:24:450
2032826511,30sleep020397-21mailstats22:04:460
1815426519,19sleep10-21swapper/120:14:451
53412647,12sleep00-21swapper/021:16:500
3098326410,41sleep031018-21ps20:55:140
3012726413,22sleep00-21swapper/022:35:190
1493726416,25sleep10-21swapper/120:04:431
1144226416,27sleep311461-21apt-get23:19:273
3266826314,24sleep10-21swapper/120:59:471
3169026316,22sleep00-21swapper/019:14:470
2548426322,19sleep30-21swapper/300:04:173
2548426322,19sleep30-21swapper/300:04:173
2462226311,20sleep10-21swapper/123:59:471
2462226311,20sleep10-21swapper/123:59:471
841726224,19sleep18535-21cat19:44:451
841726224,19sleep18535-21cat19:44:451
179512628,14sleep00-21swapper/020:14:400
52326112,24sleep10-21swapper/119:19:451
3153426118,20sleep30-21swapper/319:14:463
2933326115,26sleep329376-21ntpq19:10:183
2889926117,28sleep128901-21apt-get19:09:271
2889126112,33sleep028898-21diskstats00:14:320
271626116,31sleep32725-21cpu21:09:303
271626116,31sleep32725-21cpu21:09:303
241212615,13sleep10-21swapper/122:15:131
2377426118,19sleep00-21swapper/020:34:180
2093426117,25sleep320993-21df20:24:323
164126120,23sleep10-21swapper/121:04:441
164126120,23sleep10-21swapper/121:04:441
1536926116,21sleep00-21swapper/021:49:370
6872606,13sleep00-21swapper/021:02:000
2784326012,23sleep10-21swapper/122:29:181
203142608,42sleep10-21swapper/120:23:171
1777126018,25sleep317781-21/usr/sbin/munin23:39:343
1552826021,18sleep30-21swapper/321:49:403
1487026014,25sleep30-21swapper/321:49:153
147322605,36sleep10-21swapper/121:47:281
1280826012,32sleep112952-21latency_hist23:24:181
1046526011,20sleep00-21swapper/021:34:260
2689325920,19sleep10-21swapper/120:44:171
207552599,22sleep10-21swapper/122:04:511
15852594,13sleep10-21swapper/122:46:181
964125813,30sleep19744-21latency_hist23:14:171
441925816,21sleep00-21swapper/019:34:180
413625816,25sleep04212-21systemd-run00:39:150
325032586,11sleep30-21swapper/322:41:283
2805925817,24sleep327578-21apt-get22:29:243
2792725816,23sleep30-21swapper/300:09:473
1619725817,19sleep30-21swapper/323:34:343
59142577,32sleep00-21swapper/021:19:330
3192025713,26sleep131671-21apt-get00:24:181
298912575,40sleep10-21swapper/120:53:341
832525615,26sleep08416-21switchtime19:44:440
832525615,26sleep08416-21switchtime19:44:440
72912567,13sleep30-21swapper/319:42:043
63825616,27sleep3639-21sshd21:01:273
3143925616,23sleep031458-21apt-get22:39:270
271125614,26sleep02707-21apt-get21:09:270
271125614,26sleep02707-21apt-get21:09:270
2606925616,23sleep126163-21sed22:24:171
2273325619,21sleep30-21swapper/320:29:373
2256825513,25sleep322561-21apt-get23:54:283
2256825513,25sleep322561-21apt-get23:54:283
1472925516,21sleep10-21swapper/123:29:351
1339325514,18sleep30-21swapper/321:44:193
1301825512,26sleep113012-21apt-get23:24:301
1114725513,24sleep311220-21systemd-run23:19:153
30525415,23sleep1375-21sed22:44:161
2985425415,22sleep129375-21apt-get22:34:231
2139225413,24sleep021384-21apt-get22:09:310
2102025413,24sleep321014-21apt-get23:49:273
1744525318,19sleep10-21swapper/123:39:161
1610525315,22sleep016102-21apt-get23:34:260
894425212,23sleep38940-21apt-get21:29:293
522625212,22sleep35219-21apt-get22:59:293
365725216,15sleep00-21swapper/022:54:330
1241925213,17sleep30-21swapper/319:59:163
130952516,36sleep00-21swapper/021:41:210
62225010,24sleep0631-21df_inode22:44:320
617325012,22sleep06175-21modprobe23:00:140
486925010,16sleep00-21swapper/019:34:380
2887825012,20sleep00-21swapper/019:09:260
2881525011,22sleep128807-21apt-get00:14:291
2477925012,21sleep324833-21df22:19:313
2180725011,23sleep321843-21ntp_states22:09:423
1304425012,23sleep313090-21df23:24:323
69524911,22sleep3766-21fw_packets22:44:353
3247024912,21sleep032513-21df19:19:320
308224912,22sleep13076-21apt-get19:29:301
2805124910,14sleep00-21swapper/000:09:480
588424814,18sleep10-21swapper/119:39:151
221324614,18sleep30-21swapper/300:29:543
22022469,17sleep30-21swapper/322:49:353
271282439,16sleep00-21swapper/020:44:280
182862439,13sleep30-21swapper/321:59:323
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional