You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-01 - 15:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Fri May 01, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
151662682721,6772sleep09144-21kworker/0:010:52:130
87632135312,1327sleep38747-21apt-get08:50:393
18986274426,701sleep219017-21/usr/sbin/munin07:40:442
26269238316,348sleep326271-21sshd08:03:103
54023559,16sleep00-21swapper/010:05:450
13812230752,195sleep013964-21latency_hist07:25:230
31153230511,279sleep031259-21tune2fs10:00:400
7652227464,152sleep07800-21unixbench_multi08:45:520
37742267111,134sleep03919-21latency_hist12:00:230
22367224760,89sleep022409-21fschecks_count11:16:370
20530222136,74sleep020548-21kernelversion07:45:420
16450222014,182sleep016588-21grep10:55:530
5178221829,172sleep05281-21latency_hist08:40:230
24310221318,69sleep00-21swapper/007:55:520
1347221010,183sleep11421-21find08:25:501
10707221011,25sleep00-21swapper/007:15:340
3038220940,107sleep00-21swapper/010:15:220
31653220630,138sleep031701-21awk08:20:360
2802622069,134sleep00-21swapper/008:10:220
8473219510,125sleep07904-21/usr/sbin/munin10:30:490
1429821935,27sleep00-21swapper/012:31:330
11708219314,134sleep00-21swapper/010:40:510
15552219214,164sleep015618-21switchtime09:10:520
23513218519,149sleep323648-21cat11:20:233
8893218222,146sleep38933-21if_eth012:15:403
6801218245,66sleep00-21swapper/012:10:200
18594218074,50sleep018680-21latency_hist11:05:220
878721689,123sleep08833-21grep08:50:570
28663216715,136sleep228134-21apt-get11:35:332
28663216715,136sleep228134-21apt-get11:35:332
22200216514,135sleep28614-1kworker/2:0H11:15:342
7645216444,96sleep00-21swapper/007:05:490
26141216126,92sleep00-21swapper/009:45:370
156632160106,41sleep00-21swapper/007:30:360
39922158103,42sleep00-21swapper/012:00:350
2256321587,93sleep022698-21latency_hist09:35:230
6098215610,131sleep06228-21grep08:40:550
24347215116,76sleep00-21swapper/009:40:360
24347215116,76sleep00-21swapper/009:40:360
14194215129,85sleep014341-21awk09:05:510
25186214923,17sleep00-21swapper/008:00:360
12101214614,25sleep06972-21cyclictest09:00:420
29512214510,76sleep00-21swapper/009:55:430
2986721438,22sleep00-21swapper/008:15:230
2986721438,22sleep00-21swapper/008:15:230
1891214326,85sleep013441-21snmpd11:52:230
15461214311,116sleep214974-21/usr/sbin/munin09:10:502
10816214310,103sleep010930-21latency_hist10:40:220
1404721389,44sleep00-21swapper/007:25:360
1404721389,44sleep00-21swapper/007:25:360
2307921355,49sleep00-21swapper/007:52:200
9262213422,22sleep00-21swapper/012:15:500
8369213461,55sleep08445-21systemd-run12:15:210
20141213216,22sleep020224-21wc11:10:230
1317721317,111sleep013243-21sed10:45:470
28769213018,16sleep00-21swapper/011:35:550
28769213018,16sleep00-21swapper/011:35:550
25669213065,42sleep00-21swapper/011:25:420
16842212711,91sleep00-21swapper/009:15:400
1820721266,28sleep00-21swapper/009:20:370
20354212227,64sleep120375-21apt-get11:10:321
31921211923,29sleep30-21swapper/308:20:433
2910221189,93sleep029298-21latency_hist09:55:230
3967211637,40sleep10-21swapper/112:00:321
28276211610,92sleep028490-21latency_hist11:35:230
28276211610,92sleep028490-21latency_hist11:35:230
26150211630,34sleep30-21swapper/309:45:443
10243211625,69sleep110379-21sed10:35:511
9586211452,49sleep09627-21grep07:10:500
28187211478,24sleep328201-21systemd-run11:35:223
28187211478,24sleep328201-21systemd-run11:35:223
16804211315,84sleep316806-21sshd07:34:473
15356211224,67sleep315442-21grep07:30:233
15356211224,67sleep315442-21grep07:30:233
2038121116,29sleep00-21swapper/011:10:350
1281221118,91sleep1191-21jbd2/mmcblk2p2-12:27:221
5884210910,83sleep15926-21cut08:40:521
3222210910,81sleep00-21swapper/008:31:100
28999210918,72sleep329031-21proc_pri08:11:013
28999210918,72sleep329031-21proc_pri08:11:013
2106521098,15sleep20-21swapper/207:45:522
16038210913,83sleep116048-21ntp_states07:30:451
2439210721,70sleep12489-21cpuspeed11:55:351
22938210629,17sleep023005-21perl09:35:400
19397210516,73sleep319543-21unixbench_multi07:40:533
14151210529,61sleep314202-21sort10:50:353
20080210428,19sleep020122-21ntpq09:25:460
1164521047,85sleep011673-21date09:00:220
22080210315,73sleep122041-21apt-get07:50:561
30683210211,76sleep130688-21sh11:40:491
12311210243,42sleep112325-21sh10:45:201
26942210118,59sleep226105-21/usr/sbin/munin09:46:162
858210022,54sleep1852-21apt-get11:50:341
28942210020,47sleep229012-21gpgv09:55:212
1156629810,17sleep00-21swapper/012:25:230
737929717,55sleep17071-21/usr/sbin/munin12:10:421
45922979,21sleep00-21swapper/010:20:230
1189329722,43sleep10-21swapper/109:00:351
790329623,39sleep20-21swapper/210:30:312
1788829626,37sleep10-21swapper/109:20:221
1301629614,64sleep313094-21mailstats07:21:063
1854229520,56sleep218571-21munin-run11:05:202
1236929512,67sleep011755-21/usr/sbin/munin12:25:510
2002529444,39sleep020027-21munin-run07:45:220
522329220,54sleep25257-21/usr/sbin/munin10:20:452
522329220,54sleep25257-21/usr/sbin/munin10:20:452
2325229214,61sleep323258-21ntp_states09:35:473
857829050,17sleep30-21swapper/307:10:243
2949228929,40sleep129487-21apt-get09:55:321
1753628817,55sleep317583-21ntp_kernel_pll_07:35:443
1753628817,55sleep317583-21ntp_kernel_pll_07:35:443
1812528724,29sleep30-21swapper/311:00:533
2180328618,40sleep321962-21apt-key11:15:223
718328510,55sleep20-21swapper/207:05:352
1060328516,52sleep210633-21/usr/sbin/munin12:20:442
60728412,58sleep3639-21proc_pri10:05:483
58142849,57sleep25903-21load12:05:422
2497228411,57sleep325114-21latency_hist08:00:233
1694828415,50sleep216979-21/usr/sbin/munin09:15:422
1332928213,52sleep313397-21df12:30:353
1330228214,52sleep213303-21/usr/sbin/munin12:30:302
3018828121,46sleep00-21swapper/008:15:390
1876028123,35sleep118877-21wc09:20:501
1842828127,17sleep018460-21munin-run07:40:210
1842828127,17sleep018460-21munin-run07:40:210
2991828015,48sleep329469-21/usr/sbin/munin09:55:543
2414628022,42sleep324183-21sed11:20:443
1257228020,44sleep212703-21grep09:00:522
952127921,42sleep19564-21ntpq07:10:471
512627923,39sleep25127-21cron08:40:212
2442327916,45sleep324430-21/usr/sbin/munin09:40:393
2442327916,45sleep324430-21/usr/sbin/munin09:40:393
1752627917,48sleep217535-21needreboot07:35:432
1752627917,48sleep217535-21needreboot07:35:432
1441127922,28sleep10-21swapper/110:50:411
1214927920,36sleep111991-21latency_hist07:20:241
1031227915,49sleep310329-21/usr/sbin/munin08:55:323
81427824,15sleep00-21swapper/008:25:360
74502786,60sleep00-21swapper/010:27:540
74502786,60sleep00-21swapper/010:27:540
401527816,16sleep20-21swapper/210:15:532
3185427827,30sleep20-21swapper/211:45:392
3080627810,17sleep00-21swapper/011:40:520
289327713,16sleep10-21swapper/108:30:501
2656227713,49sleep31-21systemd11:30:213
196802778,15sleep30-21swapper/309:25:373
1393627720,38sleep113998-21/usr/sbin/munin09:05:481
2683527619,25sleep10-21swapper/109:46:151
1393227626,32sleep314081-21latency_hist10:50:233
1268527615,41sleep112576-21/usr/sbin/munin10:45:361
757927516,43sleep17614-21ntp_kernel_pll_07:05:461
178127529,21sleep30-21swapper/310:10:413
1278927513,47sleep112883-21memory07:21:001
389527428,21sleep10-21swapper/110:15:521
2506427418,41sleep125159-21cut11:25:221
581727316,24sleep20-21swapper/208:40:492
2649627320,34sleep226621-21cat08:05:232
2409527321,25sleep10-21swapper/107:55:491
2319527315,37sleep123239-21awk09:35:451
145627323,26sleep10-21swapper/110:10:221
934327218,30sleep10-21swapper/108:52:231
3058727216,39sleep230655-21systemctl08:15:492
2998527213,43sleep230044-21/usr/sbin/munin09:55:572
2587527221,16sleep20-21swapper/211:25:482
248052726,11sleep00-21swapper/011:22:430
2098627225,22sleep10-21swapper/109:30:241
1853027216,21sleep30-21swapper/309:20:443
999827122,27sleep10-21swapper/112:20:231
82927127,24sleep20-21swapper/208:25:382
637327113,43sleep26428-21cpuspeed10:25:342
637327113,43sleep26428-21cpuspeed10:25:342
57562716,11sleep00-21swapper/010:21:080
57562716,11sleep00-21swapper/010:21:080
4992716,11sleep00-21swapper/011:49:280
3236727113,14sleep20-21swapper/210:05:222
3153027117,25sleep20-21swapper/211:45:222
2421227125,25sleep20-21swapper/211:20:472
2245427120,35sleep322483-21/usr/sbin/munin07:51:103
608227020,32sleep06169-21/usr/sbin/munin07:00:490
608227020,32sleep06169-21/usr/sbin/munin07:00:490
457927022,22sleep30-21swapper/310:20:223
1755527021,29sleep117649-21/usr/sbin/munin11:00:421
1196627020,29sleep20-21swapper/207:20:212
528926918,35sleep15325-21wc10:20:481
528926918,35sleep15325-21wc10:20:481
2998126921,25sleep30-21swapper/311:40:233
2998126921,25sleep30-21swapper/311:40:233
1849626914,31sleep20-21swapper/209:20:432
828526820,27sleep28371-21cat08:50:222
530526821,23sleep30-21swapper/312:05:223
1924526815,25sleep10-21swapper/107:40:511
601926723,26sleep30-21swapper/308:40:543
297126719,22sleep20-21swapper/208:30:522
2799226717,35sleep328012-21apt-get08:10:213
2193326715,34sleep122077-21unixbench_singl09:30:521
762726612,40sleep113441-21snmpd10:30:131
762726612,40sleep113441-21snmpd10:30:131
447926615,27sleep20-21swapper/212:00:492
308126614,25sleep10-21swapper/110:15:231
162702668,13sleep20-21swapper/207:30:502
1329926616,20sleep20-21swapper/207:21:092
1157826619,20sleep20-21swapper/209:00:032
62126517,27sleep10-21swapper/111:50:221
568226524,22sleep30-21swapper/307:00:403
568226524,22sleep30-21swapper/307:00:403
2862326517,26sleep20-21swapper/208:10:522
2862326517,26sleep20-21swapper/208:10:522
1340626519,29sleep113413-21diskstats12:30:371
540226422,18sleep30-21swapper/310:20:503
540226422,18sleep30-21swapper/310:20:503
3043426415,24sleep30-21swapper/311:40:433
2208826415,18sleep20-21swapper/207:51:012
204272648,40sleep220453-21df11:10:372
1634326427,19sleep30-21swapper/310:55:523
1444526414,22sleep20-21swapper/207:25:462
1444526414,22sleep20-21swapper/207:25:462
1097326420,20sleep30-21swapper/307:15:413
915026314,22sleep20-21swapper/212:15:472
605026317,20sleep20-21swapper/207:00:472
605026317,20sleep20-21swapper/207:00:472
3034026316,21sleep30-21swapper/308:15:423
2786526322,22sleep10-21swapper/109:50:391
192382635,11sleep20-21swapper/211:05:442
1537826317,23sleep30-21swapper/309:10:473
872726224,19sleep20-21swapper/208:50:302
3145126210,37sleep10-21swapper/110:00:451
2865826218,26sleep128134-21apt-get11:35:321
2865826218,26sleep128134-21apt-get11:35:321
1590026217,23sleep10-21swapper/110:55:421
3268826117,23sleep20-21swapper/210:05:392
3192226121,17sleep30-21swapper/311:45:403
2264826110,13sleep10-21swapper/111:16:421
928526016,18sleep10-21swapper/110:35:221
685326020,19sleep30-21swapper/312:10:233
454626012,25sleep10-21swapper/108:35:511
2729526021,21sleep10-21swapper/108:05:501
2579526018,22sleep10-21swapper/111:25:471
2484826022,24sleep224923-21mailstats09:40:512
2484826022,24sleep224923-21mailstats09:40:512
196526015,30sleep22035-21cat10:10:442
1704626019,25sleep217176-21sort11:00:232
1345226016,27sleep313447-21apt-get09:05:343
840525911,17sleep30-21swapper/310:30:483
696525917,25sleep36961-21apt-get08:45:353
305652598,34sleep10-21swapper/108:15:481
2076825918,27sleep320842-21mailstats07:45:493
1930625923,19sleep10-21swapper/111:05:471
2565825819,19sleep10-21swapper/109:45:221
2565825819,19sleep10-21swapper/109:45:221
2555325816,23sleep10-21swapper/108:00:461
1549025820,20sleep20-21swapper/210:55:222
2735725717,19sleep20-21swapper/208:05:512
2709825711,32sleep227121-21irqstats11:30:402
2709825711,32sleep227121-21irqstats11:30:402
2324025717,24sleep223251-21ntp_states09:35:462
219025719,25sleep32211-21ls11:55:213
1708625714,26sleep317177-21mailstats09:15:483
1232825719,24sleep212363-21munin-run10:45:212
1033625716,24sleep110333-21apt-get08:55:341
68425617,22sleep2822-21modprobe11:50:232
1967925615,24sleep219672-21apt-get09:25:362
1337125619,16sleep20-21swapper/210:45:492
609925516,17sleep10-21swapper/112:05:491
3150925512,17sleep30-21swapper/310:00:473
209382556,13sleep10-21swapper/107:45:511
1069325520,19sleep10-21swapper/112:20:471
889325412,23sleep28971-21df07:10:272
437125417,22sleep34462-21mailstats08:35:493
2415725411,20sleep20-21swapper/207:55:502
2122025412,25sleep221217-21apt-get09:30:332
1038025415,23sleep210408-21users10:35:522
957925315,22sleep39588-21diskstats10:35:373
3172825313,24sleep231735-21diskstats08:20:382
297262536,10sleep10-21swapper/108:14:351
297262536,10sleep10-21swapper/108:14:351
2829425320,16sleep30-21swapper/309:50:493
1910025315,18sleep30-21swapper/311:05:413
1779925315,18sleep10-21swapper/107:35:501
1779925315,18sleep10-21swapper/107:35:501
1153725322,18sleep311520-21latency_hist12:25:223
374625217,16sleep30-21swapper/312:00:223
3164925213,24sleep131646-21apt-get08:20:331
2724225219,15sleep10-21swapper/111:30:431
2724225219,15sleep10-21swapper/111:30:431
1450725210,29sleep114544-21ps07:25:491
1450725210,29sleep114544-21ps07:25:491
1162725219,17sleep30-21swapper/310:40:493
91252516,18sleep30-21swapper/307:10:363
823125116,22sleep18284-21systemd-run08:50:211
129582515,11sleep20-21swapper/212:29:162
355125010,16sleep30-21swapper/308:35:213
1562025015,16sleep10-21swapper/109:10:531
121025012,16sleep30-21swapper/308:25:483
65782499,18sleep30-21swapper/310:25:393
65782499,18sleep30-21swapper/310:25:393
709324810,22sleep37090-21apt-get12:10:313
566724813,17sleep10-21swapper/107:00:381
566724813,17sleep10-21swapper/107:00:381
2129924812,22sleep321305-21diskstats09:30:383
1309224814,17sleep30-21swapper/312:30:223
1352624711,14sleep20-21swapper/209:05:372
1188524611,20sleep311887-21apt-get09:00:333
263924411,12sleep20-21swapper/211:55:402
72182405,14sleep20-21swapper/212:10:392
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional