You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-13 - 15:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Wed May 13, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
43292391011,3887sleep213943-21kworker/2:210:44:372
96262133212,1304sleep09585-21/usr/sbin/munin09:19:310
96262133212,1304sleep09585-21/usr/sbin/munin09:19:310
2693929388,917sleep226947-21ntpq08:29:582
227628139,791sleep22299-21munin-node07:14:362
227628139,791sleep22299-21munin-node07:14:362
882326108,585sleep28876-21apt-config07:34:302
882326108,585sleep28876-21apt-config07:34:302
24088253527,485sleep324100-21apt-get10:04:413
6594241420,378sleep36682-21/usr/sbin/munin09:09:493
6594241420,378sleep36682-21/usr/sbin/munin09:09:493
1223524068,357sleep212213-21latency_hist07:44:322
20757229819,22sleep20-21swapper/211:35:002
3140226418,226sleep33269-21latency_hist08:59:313
27714225213,137sleep20-21swapper/210:14:572
2643223424,171sleep22323-21gpgv10:39:322
20555223216,200sleep120603-21grep11:34:551
4919222419,145sleep25010-21fschecks_time12:29:512
28742222414,133sleep228892-21unixbench_singl08:34:592
1599221945,160sleep21697-21timerandwakeup10:35:002
23717221712,153sleep223715-21gpgv10:04:312
11840221713,187sleep2225-21systemd-journal11:09:282
15909221646,63sleep20-21swapper/211:19:542
29318221544,83sleep20-21swapper/212:04:322
28653221523,172sleep228799-21unixbench_singl11:59:592
677221354,52sleep20-21swapper/212:14:572
32406220813,134sleep2225-21systemd-journal08:49:292
31535220518,165sleep031616-21grep12:09:590
27295219811,170sleep027332-21fschecks_count10:14:470
4002218920,144sleep34144-21head12:25:003
1886218811,162sleep01937-21fschecks_count08:54:480
2757721865,16sleep00-21swapper/008:32:270
13063217919,132sleep013157-21fschecks_time09:29:490
14915216033,107sleep20-21swapper/209:34:562
1677215949,96sleep2225-21systemd-journal07:11:122
19452215724,76sleep20-21swapper/209:49:502
19452215724,76sleep20-21swapper/209:49:502
3757215646,23sleep20-21swapper/208:59:552
5667215527,104sleep35718-21latency_hist10:49:303
23672215221,114sleep223714-21awk11:44:532
12137214915,114sleep012191-21sort11:09:470
10334214712,80sleep210372-21grep09:19:572
10044214511,103sleep20-21swapper/207:35:192
20826214320,102sleep00-21swapper/009:54:480
2885621406,12sleep10-21swapper/112:00:031
8176213920,100sleep18279-21grep07:30:011
8176213920,100sleep18279-21grep07:30:011
24127213923,103sleep00-21swapper/010:04:540
16178213622,54sleep20-21swapper/209:39:482
21563213312,43sleep20-21swapper/208:14:322
5906213228,56sleep20-21swapper/207:24:412
29191213011,82sleep229229-21ntpq10:19:542
2616821300,12sleep30-21swapper/310:09:573
13391212914,69sleep213405-21apt-get11:14:292
31133212844,20sleep231182-21grep12:09:492
1236121287,86sleep20-21swapper/209:25:272
825821278,98sleep20-21swapper/209:14:402
825821278,98sleep20-21swapper/209:14:402
17138212631,59sleep217141-21apt-get07:59:422
24703212333,57sleep224788-21cat10:05:122
6606212212,61sleep20-21swapper/212:34:402
16171212217,83sleep016289-21timerwakeupswit11:20:000
15281212213,51sleep215302-21awk07:54:292
15281212213,51sleep215302-21awk07:54:292
22984212112,53sleep222987-21cron11:44:292
25433212051,52sleep20-21swapper/211:49:582
27625211914,87sleep027644-21apt11:59:290
13806211846,33sleep213405-21apt-get11:14:332
32165211737,59sleep20-21swapper/210:29:552
28892211714,82sleep129000-21cut10:19:491
20716211716,43sleep120735-21apt-get09:54:411
4745211612,90sleep04757-21kernelversion07:20:070
4117211418,81sleep04204-21cat10:44:310
658421137,70sleep26593-21forks09:09:482
658421137,70sleep26593-21forks09:09:482
23470211247,13sleep20-21swapper/208:19:492
23470211247,13sleep20-21swapper/208:19:492
22225211214,78sleep322301-21/usr/sbin/munin11:39:583
17121211213,81sleep117130-21/usr/sbin/munin11:24:461
6179211022,26sleep20-21swapper/210:49:502
1032211014,77sleep11047-21/usr/sbin/munin07:09:541
13442210927,12sleep20-21swapper/207:45:452
17418210623,58sleep317484-21latency07:59:553
15655210621,64sleep115718-21fschecks_time07:54:481
15655210621,64sleep115718-21fschecks_time07:54:481
2666321057,84sleep00-21swapper/010:11:120
7721210434,54sleep27743-21tr10:54:512
16192210413,76sleep316277-21fschecks_time09:39:493
7329210314,73sleep17421-21latency_hist07:29:331
18703210315,17sleep30-21swapper/308:04:443
10974210313,73sleep111001-21sh07:39:461
18572210227,11sleep20-21swapper/211:29:352
18145210213,73sleep318225-21mailstats09:44:583
14514210214,32sleep114512-21apt-get09:34:421
11893210116,69sleep112035-21cat11:09:311
7456210025,52sleep37449-21apt-get10:54:433
31652210030,59sleep231741-21ls10:29:312
3116421000,29sleep031168-21unixbench_multi10:25:000
28553210013,72sleep328496-21latency_hist10:19:303
18315210012,72sleep00-21swapper/011:29:290
443429918,63sleep33909-21kworker/u8:107:19:413
2612929912,22sleep20-21swapper/210:09:572
1801629916,48sleep20-21swapper/211:25:392
1495829915,25sleep314971-21ntp_states09:34:573
2612929816,62sleep326253-21cat11:54:313
2770529712,70sleep227784-21date11:59:312
2378829612,69sleep123834-21ntp_offset08:19:561
2378829612,69sleep123834-21ntp_offset08:19:561
2094129636,13sleep20-21swapper/209:54:502
2502529423,48sleep125116-21cut08:24:491
221632940,10sleep30-21swapper/309:59:313
388529014,18sleep20-21swapper/212:24:592
30102905,57sleep20-21swapper/208:58:542
1304229022,13sleep20-21swapper/209:29:472
3034728910,25sleep30-21swapper/312:05:013
2034128911,62sleep020362-21fschecks_count08:09:500
200128916,58sleep32052-21cat12:19:513
2180928812,55sleep121827-21fschecks_count11:39:471
1844628818,13sleep20-21swapper/209:45:012
1117028828,30sleep30-21swapper/309:24:313
52128713,16sleep20-21swapper/208:49:532
52128713,16sleep20-21swapper/208:49:532
1597028716,55sleep316015-21/usr/sbin/munin07:54:553
1597028716,55sleep316015-21/usr/sbin/munin07:54:553
1247228719,21sleep10-21swapper/107:44:431
412128526,37sleep14265-21sed09:00:021
3081128513,56sleep130959-21cat12:09:311
1945528412,16sleep00-21swapper/011:30:010
193728417,51sleep01950-21modprobe07:14:290
1603528413,19sleep20-21swapper/207:54:582
1603528413,19sleep20-21swapper/207:54:582
541928312,57sleep25487-21readlink09:04:552
191562838,17sleep20-21swapper/208:04:552
1402128323,31sleep00-21swapper/007:49:430
1402128323,31sleep00-21swapper/007:49:430
2645628219,41sleep326550-21fschecks_time11:54:523
504328118,38sleep15190-21grep10:45:001
2385428114,43sleep323915-21switchtime11:44:583
1617328123,31sleep00-21swapper/007:55:000
1617328123,31sleep00-21swapper/007:55:000
1061128119,40sleep010630-21idleruntime-cro07:39:290
718028018,46sleep07218-21ps12:36:110
303228035,32sleep20-21swapper/212:24:302
2492828033,33sleep20-21swapper/208:24:412
175182809,52sleep10-21swapper/107:59:571
73127925,31sleep00-21swapper/008:49:590
73127925,31sleep00-21swapper/008:49:590
2920827912,53sleep0363-21snmpd08:38:320
2050527925,27sleep00-21swapper/009:54:320
2050527925,27sleep00-21swapper/009:54:320
1791827910,14sleep10-21swapper/109:44:521
1034627914,14sleep20-21swapper/211:04:322
1034627914,14sleep20-21swapper/211:04:322
922227815,45sleep19285-21/usr/sbin/munin10:59:541
922227815,45sleep19285-21/usr/sbin/munin10:59:541
2317127822,36sleep123316-21latency_hist08:19:321
1681427812,52sleep316838-21sed11:24:313
91727714,47sleep31036-21timerwakeupswit08:50:023
91727714,47sleep31036-21timerwakeupswit08:50:023
739627713,45sleep37529-21unixbench_multi12:36:143
480527711,51sleep04817-21uname10:44:550
3200527720,41sleep132039-21ps07:04:581
3181627730,26sleep00-21swapper/007:04:530
2841727714,16sleep30-21swapper/311:59:563
2546327623,32sleep10-21swapper/110:09:351
149827618,38sleep01424-21kworker/u8:012:19:320
3262427527,25sleep00-21swapper/010:30:050
702927431,11sleep20-21swapper/207:26:232
3212627427,24sleep30-21swapper/307:05:003
3066727430,11sleep20-21swapper/208:42:022
2968927318,30sleep30-21swapper/308:39:493
2363427320,29sleep123657-21munin-run10:04:301
1724627320,32sleep017342-21fschecks_time07:59:520
1713627317,39sleep317178-21fschecks_count11:24:483
995127219,27sleep10-21swapper/107:35:181
757927217,40sleep0225-21systemd-journal09:12:120
757927217,40sleep0225-21systemd-journal09:12:120
381327216,36sleep13884-21/usr/sbin/munin12:24:581
2285427229,11sleep20-21swapper/208:16:462
888227116,26sleep30-21swapper/307:34:313
888227116,26sleep30-21swapper/307:34:313
39927120,31sleep2418-21munin-run07:09:312
1996427118,27sleep10-21swapper/108:09:301
879727021,26sleep00-21swapper/007:34:120
879727021,26sleep00-21swapper/007:34:120
1460526920,25sleep00-21swapper/009:34:490
1452126919,28sleep314610-21switchtime11:15:153
284782688,12sleep00-21swapper/011:59:570
2839726819,31sleep328442-21ntpq08:34:523
1187426816,26sleep30-21swapper/309:24:553
544126720,26sleep10-21swapper/112:30:031
454126718,26sleep04567-21munin-run12:29:290
313126714,26sleep10-21swapper/107:14:591
313126714,26sleep10-21swapper/107:14:591
1119126715,21sleep00-21swapper/009:24:320
871226617,32sleep08713-21cron10:59:290
530226619,23sleep10-21swapper/109:04:501
2694826614,33sleep30-21swapper/308:30:003
2306726619,30sleep31-21systemd08:19:303
520926520,24sleep30-21swapper/312:29:563
500926517,29sleep30-21swapper/309:04:443
2957126513,34sleep029580-21/usr/sbin/munin12:04:460
243282656,48sleep00-21swapper/011:46:530
2550426423,21sleep30-21swapper/308:24:593
228982645,48sleep00-21swapper/011:43:220
1391526419,27sleep013909-21apt-get11:14:430
1268026416,19sleep10-21swapper/109:29:291
125752647,14sleep00-21swapper/009:28:110
590926315,34sleep35940-21grep10:49:433
3250926315,35sleep332620-21sed10:30:043
2051526319,27sleep320527-21/usr/sbin/munin08:09:533
1740126317,31sleep017381-21latency_hist09:44:310
1629226321,17sleep10-21swapper/111:20:011
1533326321,23sleep10-21swapper/107:54:321
1533326321,23sleep10-21swapper/107:54:321
1473826321,24sleep10-21swapper/111:15:171
866026220,22sleep10-21swapper/109:14:561
866026220,22sleep10-21swapper/109:14:561
69702625,12sleep00-21swapper/010:51:580
64612620,9sleep112-21rcu_sched10:49:561
272112625,14sleep10-21swapper/110:14:441
1342626215,28sleep313459-21munin-run11:14:303
934526019,25sleep38837-21apt-get07:34:433
2045226016,28sleep120484-21munin-run09:54:291
2045226016,28sleep120484-21munin-run09:54:291
1105626010,12sleep30-21swapper/311:04:573
826825915,27sleep38278-21apt-get09:14:413
826825915,27sleep38278-21apt-get09:14:413
628825918,27sleep36319-21kernelversion07:24:583
3042725910,35sleep330474-21cut10:24:473
29982596,11sleep10-21swapper/108:58:441
2936925914,20sleep30-21swapper/310:19:593
226842597,12sleep00-21swapper/009:59:530
1927625918,26sleep30-21swapper/309:49:463
1927625918,26sleep30-21swapper/309:49:463
1112925919,19sleep10-21swapper/109:24:291
903125817,24sleep38934-21apt-get10:59:443
903125817,24sleep38934-21apt-get10:59:443
647525811,26sleep031260-21kworker/u8:007:25:030
2967025814,24sleep129681-21df_inode08:39:471
2299125823,18sleep30-21swapper/310:00:003
19282586,19sleep00-21swapper/010:35:180
130542584,18sleep10-21swapper/111:10:081
1119825815,21sleep30-21swapper/307:39:523
278225714,24sleep10-21swapper/110:39:391
2023625719,17sleep30-21swapper/311:34:493
1857825713,31sleep318595-21apt11:29:403
1310025718,21sleep00-21swapper/007:44:580
1302625715,22sleep30-21swapper/307:44:573
658525614,20sleep10-21swapper/112:34:391
585825615,25sleep10-21swapper/107:24:401
145425620,21sleep30-21swapper/310:34:573
1343225614,19sleep10-21swapper/109:29:581
1129325616,16sleep00-21swapper/007:39:530
977725519,18sleep00-21swapper/011:00:060
977725519,18sleep00-21swapper/011:00:060
2612025520,18sleep10-21swapper/110:09:561
2228425510,24sleep022348-21sendmail_mailst08:14:580
1399825510,18sleep30-21swapper/307:49:423
1399825510,18sleep30-21swapper/307:49:423
3197025422,16sleep10-21swapper/110:29:501
3114825414,24sleep031143-21apt-get08:44:420
2493125412,25sleep024925-21apt-get08:24:440
2469325421,18sleep00-21swapper/008:24:310
2469325421,18sleep00-21swapper/008:24:310
1895725413,15sleep00-21swapper/009:49:310
1058125414,24sleep010576-21apt-get11:04:420
705425315,17sleep10-21swapper/109:09:591
705425315,17sleep10-21swapper/109:09:591
67525312,25sleep3671-21apt-get07:09:443
59692535,10sleep00-21swapper/009:06:350
362225313,24sleep33650-21irqstats08:59:523
323525312,17sleep30-21swapper/310:39:583
3041225312,23sleep030504-21users08:40:020
2877025312,20sleep00-21swapper/010:19:390
1869925313,23sleep018696-21apt-get08:04:410
1296825313,23sleep312963-21apt-get09:29:443
48725213,23sleep1520-21irqstats08:49:521
48725213,23sleep1520-21irqstats08:49:521
3258825213,23sleep332583-21apt-get12:14:423
2932625217,21sleep129364-21munin-run08:39:301
242925216,19sleep10-21swapper/110:39:311
1700725213,22sleep116067-21/usr/sbin/munin09:40:031
390225112,23sleep03927-21munin-run07:19:300
390225112,23sleep03927-21munin-run07:19:300
33522519,28sleep03371-21apt-get08:59:400
2674825113,21sleep126764-21kernelversion08:29:531
260492514,36sleep00-21swapper/011:54:080
2523625113,23sleep125273-21ntp_states11:49:531
173425113,23sleep01727-21apt-get12:19:440
89792509,17sleep10-21swapper/110:59:421
2373525012,22sleep123772-21ps11:44:561
524824913,21sleep05285-21ntp_states12:29:570
3034024912,21sleep130337-21apt-get10:24:411
210224915,20sleep12139-21sed12:19:531
3263924813,18sleep10-21swapper/112:14:441
314762489,15sleep10-21swapper/112:09:561
2764824812,20sleep127704-21munin-run11:59:301
2014724812,20sleep020144-21apt-get11:34:420
3161524712,16sleep30-21swapper/308:44:583
230024710,20sleep32378-21ntp_states07:14:373
230024710,20sleep32378-21ntp_states07:14:373
2024824710,18sleep10-21swapper/108:09:421
323402469,18sleep30-21swapper/312:14:303
1860224511,16sleep10-21swapper/111:29:421
235072439,14sleep30-21swapper/308:19:503
235072439,14sleep30-21swapper/308:19:503
316002377,15sleep131602-21ntp_states08:44:551
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional