You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-18 - 18:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Sat Jul 18, 2026 12:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
19319271611,659sleep229685-21kworker/u8:412:03:412
27479252918,497sleep127472-21apt-get09:03:451
7321230816,238sleep27202-21latency_hist09:43:322
102142291112,55sleep210274-21sendmail_mailqu11:33:582
225552278100,22sleep20-21swapper/208:48:322
11828226812,242sleep311913-21mailstats11:38:593
2716722630,112sleep223248-21cyclictest10:43:592
2716722630,112sleep223248-21cyclictest10:43:592
1164822557,235sleep211665-21grep08:13:312
28334224860,27sleep228383-21latency_hist07:23:322
5689222851,154sleep20-21swapper/209:38:312
16582222446,161sleep216703-21timerwakeupswit11:54:002
1516221717,184sleep11600-21grep07:39:011
18007221613,188sleep118070-21ps11:59:001
18007221613,188sleep118070-21ps11:59:001
32141221210,150sleep232142-21/usr/sbin/munin09:18:402
17941220752,99sleep217989-21grep11:58:572
17941220752,99sleep217989-21grep11:58:572
17041220527,27sleep20-21swapper/210:13:322
227422048,183sleep3364-21snmpd09:25:203
13751220435,108sleep20-21swapper/208:18:522
2809220010,153sleep22832-21grep09:28:462
13377219928,147sleep213517-21unixbench_singl09:59:102
4563219412,166sleep04622-21awk09:33:480
22753219016,161sleep222778-21fschecks_count12:13:512
76962187149,24sleep37698-21sshd11:25:513
27480217844,25sleep20-21swapper/209:03:462
1530121778,154sleep215370-21sort08:23:532
12385217743,22sleep20-21swapper/208:13:592
25524217612,150sleep225529-21entropy07:13:472
27332217432,103sleep20-21swapper/212:28:412
1926121720,7sleep30-21swapper/312:03:333
27321216219,124sleep027462-21unixbench_multi10:44:020
27321216219,124sleep027462-21unixbench_multi10:44:020
14455216231,93sleep214505-21sed10:03:512
3131021598,55sleep20-21swapper/210:58:452
16981215719,42sleep117018-21grep08:28:561
12195215614,26sleep00-21swapper/008:13:530
24784215368,46sleep224863-21mailstats12:19:002
22417215030,29sleep20-21swapper/207:04:192
2876221477,100sleep228853-21sed10:49:002
17833214528,55sleep217850-21modprobe08:33:302
10900214112,109sleep20-21swapper/209:53:492
24864213615,107sleep324938-21find12:19:013
17247213618,30sleep20-21swapper/210:13:362
3595213514,103sleep20-21swapper/211:13:512
30345213547,27sleep20-21swapper/209:13:312
28937213462,20sleep20-21swapper/207:23:522
32437212514,42sleep20-21swapper/207:34:022
24772212425,73sleep124796-21munin-run10:38:291
20881212319,28sleep30-21swapper/312:08:473
3137421198,93sleep20-21swapper/207:33:292
3137421198,93sleep20-21swapper/207:33:292
2099621198,56sleep221006-21munin-run08:43:312
3172921180,7sleep10-21swapper/110:58:551
388121179,94sleep23960-21date07:48:312
8385211612,93sleep28511-21sort09:44:082
28906211510,65sleep20-21swapper/212:33:452
29119211410,16sleep00-21swapper/009:08:530
8311211312,50sleep20-21swapper/211:28:492
9295211214,80sleep39375-21cut08:03:593
28091211217,25sleep00-21swapper/009:04:000
31038211121,51sleep20-21swapper/209:13:562
2525221109,39sleep20-21swapper/208:54:052
2351621098,39sleep20-21swapper/210:33:462
32610210818,76sleep132624-21ntp_states09:18:551
228721088,65sleep22312-21munin-run07:43:312
1382421086,27sleep20-21swapper/211:44:392
31666210727,50sleep131660-21apt-get07:33:441
2460210616,73sleep32539-21awk11:09:003
1178321068,61sleep211827-21processes11:38:582
24136210329,61sleep224258-21sed07:09:032
4117210220,63sleep319796-21kworker/u8:109:33:313
17377210214,63sleep30-21swapper/310:13:533
9266210132,20sleep29292-21proc_pri08:03:582
5818210112,73sleep05888-21idleruntime07:53:500
17216210125,56sleep317240-21munin-run11:58:293
12308210114,41sleep00-21swapper/009:58:320
284621006,82sleep00-21swapper/007:43:530
78422995,30sleep20-21swapper/211:27:442
745629916,61sleep07221-21/usr/sbin/munin07:58:510
2607929940,45sleep226143-21latency12:23:522
25442996,76sleep00-21swapper/011:09:010
2416129927,25sleep20-21swapper/208:53:322
2410929916,70sleep124160-21latency_hist08:53:311
412029829,30sleep20-21swapper/207:48:432
2213129711,73sleep222188-21basename10:28:502
1998329713,26sleep10-21swapper/112:04:571
1438329718,66sleep314399-21awk11:48:403
968329613,67sleep19717-21/usr/sbin/munin09:49:081
968329613,67sleep19717-21/usr/sbin/munin09:49:081
2746729614,59sleep126614-21/usr/sbin/munin10:44:031
2746729614,59sleep126614-21/usr/sbin/munin10:44:031
2739429515,63sleep127440-21/usr/sbin/munin07:18:551
3038529417,59sleep330534-21latency_hist09:13:323
2185029412,65sleep121930-21wc08:43:591
2149629411,38sleep20-21swapper/212:09:022
1216729418,62sleep312182-21kernelversion08:13:523
2600029319,32sleep20-21swapper/208:58:492
228812937,12sleep00-21swapper/008:48:500
378129218,21sleep10-21swapper/111:13:541
66982915,11sleep20-21swapper/207:55:342
372729111,67sleep03729-21sshd09:29:080
2474729124,46sleep124775-21munin-plugin-st07:13:301
198402915,33sleep20-21swapper/210:19:492
1549529118,59sleep315572-21sed10:08:323
13582919,32sleep20-21swapper/209:23:492
1019929113,63sleep110213-21awk11:33:571
2723029012,59sleep327244-21latency_hist09:03:313
2426029022,45sleep324403-21head10:34:033
950028915,20sleep00-21swapper/011:33:310
2601428821,50sleep026032-21fschecks_count08:58:510
2592428815,54sleep025981-21wc07:13:570
2049728622,44sleep320525-21fschecks_count10:23:503
2615128518,45sleep325415-21/usr/sbin/munin07:14:003
1927728518,11sleep20-21swapper/208:37:092
1437628519,11sleep20-21swapper/211:48:352
1138928512,58sleep111447-21cat09:54:011
62228423,40sleep0831-21cat07:38:320
302192840,8sleep10-21swapper/107:28:511
302192840,8sleep10-21swapper/107:28:511
259672840,8sleep00-21swapper/012:23:490
118228419,11sleep20-21swapper/211:04:022
604228313,51sleep16136-21tune2fs09:38:501
55892839,61sleep20-21swapper/211:19:002
48928315,53sleep1547-21cut11:03:491
2517428312,55sleep025198-21fschecks_count10:38:490
2517428312,55sleep025198-21fschecks_count10:38:490
115128318,44sleep01215-21fschecks_time07:38:520
90528213,29sleep30-21swapper/311:03:593
3095528218,10sleep20-21swapper/210:58:002
2690828211,56sleep326945-21grep10:43:523
2690828211,56sleep326945-21grep10:43:523
2622528218,10sleep20-21swapper/210:41:542
2622528218,10sleep20-21swapper/210:41:542
2494228222,34sleep125133-21memory07:13:341
2204328220,30sleep10-21swapper/110:28:491
851028118,11sleep20-21swapper/208:03:292
3113028112,13sleep10-21swapper/109:13:591
671128017,35sleep00-21swapper/011:23:460
313782800,10sleep30-21swapper/307:33:293
313782800,10sleep30-21swapper/307:33:293
248182800,7sleep10-21swapper/108:53:591
2472728027,30sleep10-21swapper/112:18:591
1050928015,11sleep20-21swapper/209:53:172
1050928015,11sleep20-21swapper/209:53:172
530427911,56sleep15359-21awk11:18:511
523927924,26sleep00-21swapper/011:18:500
310327913,28sleep30-21swapper/307:44:003
302127918,11sleep20-21swapper/211:10:562
2244727916,19sleep00-21swapper/012:13:320
1044127926,28sleep30-21swapper/308:08:513
204912786,32sleep20-21swapper/210:23:492
1511827726,24sleep30-21swapper/308:23:503
887127622,33sleep19019-21grep11:29:001
690927618,41sleep16945-21wc11:23:511
638727613,47sleep06393-21ntp_states09:38:570
2896227613,24sleep30-21swapper/307:23:533
1709427623,27sleep30-21swapper/310:13:333
1140027616,46sleep111455-21fschecks_count11:38:491
266227512,46sleep12723-21/usr/sbin/munin07:43:501
1653227518,35sleep116580-21wc11:53:591
477727413,38sleep00-21swapper/011:18:290
2277927416,41sleep322871-21grep12:13:523
1536827413,45sleep115370-21sshd11:49:391
830527224,25sleep10-21swapper/109:44:071
1356827219,35sleep313646-21fschecks_time08:18:493
1411927119,32sleep014150-21munin-plugin-st11:48:290
1016227111,15sleep10-21swapper/108:08:331
2651827016,37sleep326657-21/usr/sbin/munin12:24:013
2090127014,43sleep120983-21mailstats10:24:001
1633927022,22sleep10-21swapper/108:28:321
3206226917,24sleep30-21swapper/307:33:553
3020926913,35sleep030218-21/usr/sbin/munin07:28:510
3020926913,35sleep030218-21/usr/sbin/munin07:28:510
2666926924,24sleep10-21swapper/108:59:051
2426026914,28sleep00-21swapper/012:18:450
2323026914,35sleep323254-21ntp_states08:48:573
1820726919,26sleep318262-21fschecks_count08:33:473
650626819,31sleep36580-21cat09:39:003
1799926815,26sleep00-21swapper/011:58:590
1799926815,26sleep00-21swapper/011:58:590
250912678,13sleep10-21swapper/110:38:461
250912678,13sleep10-21swapper/110:38:461
241626724,22sleep10-21swapper/111:08:581
2220226722,30sleep022292-21latency07:04:130
212432679,15sleep30-21swapper/308:43:443
1295026713,21sleep10-21swapper/111:43:501
897826622,29sleep39002-21modprobe09:48:293
3024426619,26sleep00-21swapper/012:38:320
2388426619,22sleep00-21swapper/007:08:570
1980126614,27sleep00-21swapper/012:04:520
700726520,21sleep10-21swapper/107:58:321
3263326513,35sleep332672-21/usr/sbin/munin09:18:573
2826826518,28sleep30-21swapper/310:48:493
178042657,10sleep00-21swapper/008:33:070
146602658,13sleep00-21swapper/010:03:570
3179626414,23sleep00-21swapper/010:58:590
3132326417,25sleep30-21swapper/309:14:013
2566426422,22sleep30-21swapper/310:39:003
2566426422,22sleep30-21swapper/310:39:003
2087626419,23sleep00-21swapper/010:23:590
990426316,26sleep30-21swapper/309:49:143
990426316,26sleep30-21swapper/309:49:143
934626320,21sleep00-21swapper/009:48:470
934626320,21sleep00-21swapper/009:48:470
2744126312,20sleep10-21swapper/112:28:481
2125926317,24sleep00-21swapper/012:08:560
78526210,42sleep00-21swapper/009:21:050
692426221,27sleep36953-21munin-run07:58:303
485426220,19sleep00-21swapper/007:49:020
2047626216,23sleep10-21swapper/108:39:061
1000626212,36sleep010087-21cat11:33:520
3016326119,27sleep130171-21modprobe12:38:291
2991826119,20sleep00-21swapper/010:53:510
2729326115,19sleep30-21swapper/307:18:533
2432626117,26sleep024345-21apt-get08:53:430
2351426114,25sleep00-21swapper/010:33:440
1880126115,28sleep318847-21apt-config12:03:303
1880126115,28sleep318847-21apt-config12:03:303
126162619,21sleep10-21swapper/109:58:411
772426015,27sleep07718-21apt-get09:43:460
3024026019,25sleep10-21swapper/110:54:001
2908626016,20sleep30-21swapper/312:33:503
2104026013,28sleep00-21swapper/008:43:330
1477726014,24sleep00-21swapper/011:48:530
1264426017,20sleep00-21swapper/011:43:320
308512595,11sleep30-21swapper/310:56:403
2914525919,23sleep00-21swapper/007:23:580
1317625911,35sleep113241-21unin-run08:18:301
326325815,26sleep13269-21ntp_states09:28:571
2330025815,25sleep023425-21latency_hist10:33:320
1348525813,26sleep013555-21/usr/sbin/munin08:18:460
26102579,30sleep02760-21cat09:28:310
2323725718,20sleep10-21swapper/112:14:021
2111825716,19sleep10-21swapper/112:08:531
2017225718,23sleep020243-21latency_hist10:23:310
2776725612,19sleep00-21swapper/007:19:020
244362569,32sleep324437-21diskstats08:53:503
1819125615,19sleep10-21swapper/108:33:451
1811325618,23sleep018116-21apt-get08:33:400
1770225614,18sleep10-21swapper/110:13:591
1715825617,18sleep30-21swapper/308:29:023
1282625619,17sleep30-21swapper/309:58:593
102925613,26sleep31096-21df07:38:473
1011225620,23sleep00-21swapper/008:08:320
50642557,15sleep10-21swapper/107:49:271
462325514,25sleep10-21swapper/109:33:491
40325514,25sleep0398-21apt-get11:03:440
2908825518,18sleep10-21swapper/110:49:031
2735625511,26sleep027351-21apt-get12:28:430
2322025518,23sleep323245-21cpufreq07:08:313
1746925516,18sleep00-21swapper/010:13:540
157082557,12sleep00-21swapper/010:08:410
751025415,23sleep10-21swapper/107:58:521
736825410,17sleep30-21swapper/307:58:503
2988025411,17sleep10-21swapper/107:28:321
2326725417,19sleep30-21swapper/310:33:313
1259625416,23sleep00-21swapper/009:58:400
123425412,23sleep11253-21apt-get09:23:401
607525310,34sleep30-21swapper/311:19:543
439225318,17sleep30-21swapper/307:48:533
3167225314,19sleep30-21swapper/310:58:543
2782525317,22sleep327863-21/usr/sbin/munin12:28:583
2479825317,19sleep30-21swapper/307:13:323
1886025313,24sleep318856-21apt-get10:18:413
167962536,36sleep10-21swapper/110:10:561
973525211,24sleep39731-21apt-get11:33:423
449925214,22sleep34561-21fw_packets09:33:473
2714625211,19sleep10-21swapper/112:28:321
198122529,32sleep10-21swapper/110:19:281
1347925212,23sleep113480-21sshd08:18:431
1035225212,24sleep110347-21apt-get08:08:451
2145925115,20sleep021513-21irqstats08:43:490
1471925115,15sleep10-21swapper/110:03:581
2755125012,23sleep327557-21df_inode09:03:473
839824918,16sleep30-21swapper/311:28:503
3062924911,19sleep00-21swapper/009:13:460
2912924911,23sleep129130-21diskstats09:08:541
1595624912,21sleep015952-21apt-get11:53:420
2100724815,19sleep30-21swapper/308:43:323
2816124710,21sleep028179-21apt-get10:48:420
1082224511,16sleep00-21swapper/008:09:010
1081524514,15sleep30-21swapper/309:53:463
1607824413,16sleep00-21swapper/008:25:480
1309924414,15sleep30-21swapper/311:43:523
35052398,15sleep30-21swapper/311:13:443
226572397,12sleep10-21swapper/107:04:221
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional