You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-05 - 20:50

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sat Sep 05, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21752293998,9379sleep018517-21kworker/0:109:04:140
21752293998,9379sleep018517-21kworker/0:109:04:140
69402660916,6581sleep323534-21kworker/3:210:02:333
261482150275,1410sleep026291-21apt-config07:38:330
281012149514,1468sleep318004-1kworker/3:1H11:08:493
20206278414,755sleep220216-21/usr/sbin/munin07:18:492
22010264816,615sleep122018-21/usr/sbin/munin12:33:421
4687234512,53sleep00-21swapper/011:38:320
19620234220,298sleep019768-21grep12:24:010
232922298105,175sleep023318-21grep12:38:300
230682276113,31sleep00-21swapper/009:09:030
17909226019,219sleep018022-21perl07:13:340
192772258103,38sleep019418-21unixbench_singl07:14:030
31307224113,202sleep031339-21fschecks_count07:53:510
21317223819,205sleep021385-21latency_hist10:48:310
17370221719,43sleep00-21swapper/012:18:440
5754221677,23sleep00-21swapper/009:58:470
12968221626,178sleep312999-21cut12:03:543
19068220822,169sleep119149-21/usr/sbin/munin12:23:501
30841220530,152sleep030854-21modprobe11:18:300
6183220436,155sleep26224-21systemd-run11:43:312
28675220216,164sleep028806-21timerwakeupswit11:09:060
8614219439,24sleep00-21swapper/010:08:320
15550219325,29sleep00-21swapper/012:13:310
9570219121,142sleep00-21swapper/011:53:460
5500218810,164sleep05570-21/usr/sbin/munin11:39:010
15474218510,161sleep015540-21idleruntime10:28:540
17642217916,18sleep20-21swapper/212:18:542
20675217715,27sleep00-21swapper/012:28:530
925121765,159sleep00-21swapper/008:26:450
21755217316,139sleep021769-21fschecks_count07:23:510
967721699,17sleep00-21swapper/008:28:470
13665216828,79sleep00-21swapper/008:39:040
6362216512,135sleep00-21swapper/008:18:330
308216515,111sleep00-21swapper/007:58:430
2718215815,36sleep02752-21fschecks_count09:48:540
5588215713,126sleep35648-21ls11:39:043
28529215733,109sleep028574-21ntpq07:43:580
28529215733,109sleep028574-21ntpq07:43:580
23950215713,24sleep00-21swapper/007:29:040
11312215722,27sleep00-21swapper/008:33:510
2300821560,8sleep10-21swapper/110:53:321
27312215117,118sleep327342-21cpu09:23:473
7928214616,33sleep08078-21latency_hist08:23:340
23799214623,96sleep224009-21apt-config09:13:332
24148214511,30sleep024167-21apt-get09:13:430
265492144117,16sleep026038-21apt-get07:38:410
1591621436,12sleep30-21swapper/312:13:513
1423921439,121sleep014303-21ps10:23:580
27993214114,110sleep127517-21apt-get11:08:381
23654214122,98sleep00-21swapper/010:54:000
18922214010,73sleep018956-21ps10:38:590
18922214010,73sleep018956-21ps10:38:590
30168213810,112sleep030315-21sed09:33:320
21375213811,111sleep0289-21dbus-daemon07:23:310
1362621376,42sleep00-21swapper/012:04:150
6197213210,109sleep26207-21ntpq09:58:572
1222921319,19sleep016342-21cyclictest10:18:450
11234213096,17sleep00-21swapper/011:58:500
4755212914,78sleep04866-21latency_hist08:13:330
22019212847,69sleep022048-21proc_pri10:49:000
644721278,107sleep0367-21snmpd11:43:410
32456212361,47sleep132443-21/usr/sbin/munin11:23:321
1199021228,50sleep00-21swapper/010:18:330
2718221209,76sleep00-21swapper/011:04:300
1434521188,17sleep00-21swapper/008:43:440
417721177,49sleep00-21swapper/009:53:460
1586621178,47sleep00-21swapper/012:13:480
927721158,55sleep0367-21snmpd11:53:290
1733521157,45sleep00-21swapper/010:33:580
1733521157,45sleep00-21swapper/010:33:580
3122921147,12sleep00-21swapper/011:18:510
31021147,48sleep00-21swapper/011:23:470
15096211424,76sleep215199-21fschecks_time07:03:522
30310211116,79sleep230440-21cat07:49:002
30050211110,45sleep00-21swapper/007:48:540
2504721115,11sleep00-21swapper/010:58:530
1753121117,45sleep00-21swapper/008:53:490
16313211125,65sleep316347-21latency_hist07:08:333
23217211030,51sleep223213-21apt-get07:28:462
1589421109,42sleep00-21swapper/008:48:440
2840821096,12sleep00-21swapper/009:26:260
32373210813,80sleep232411-21/usr/sbin/munin09:38:552
3180421086,42sleep00-21swapper/009:38:330
30375210846,52sleep00-21swapper/009:33:370
1923621086,44sleep00-21swapper/008:58:370
1923621086,44sleep00-21swapper/008:58:370
14812210813,79sleep214850-21processes08:43:582
29841210515,74sleep329872-21/usr/sbin/munin11:13:533
208421057,41sleep00-21swapper/008:03:520
11708210515,74sleep211771-21/usr/sbin/munin08:34:012
92521036,43sleep00-21swapper/009:43:330
31662210220,60sleep131663-21cron09:38:301
305721025,42sleep00-21swapper/011:33:280
305721025,42sleep00-21swapper/011:33:280
20925210116,70sleep121028-21fschecks_time09:03:511
20925210116,70sleep121028-21fschecks_time09:03:511
1351729913,70sleep313656-21unixbench_singl08:39:033
98482986,42sleep00-21swapper/010:10:050
267922965,40sleep00-21swapper/009:20:330
1787929611,72sleep117958-21basename12:19:011
1707129622,57sleep217092-21latency_hist12:18:302
1307129611,68sleep113102-21nfs_client12:03:561
1231529627,46sleep112355-21fschecks_count10:18:501
2140429519,21sleep10-21swapper/112:29:061
1710129514,67sleep317168-21awk07:09:013
706229416,65sleep17074-21latency_hist10:03:321
2707329414,58sleep326553-21/usr/sbin/munin07:39:123
1659329413,62sleep216660-21/usr/sbin/munin07:08:482
198829118,53sleep32082-21fschecks_time08:03:513
788429013,63sleep37923-21cat08:23:333
2126429013,56sleep221266-21cron10:48:292
1377329014,54sleep313768-21apt-get10:23:443
776328910,62sleep27769-21ntpq10:03:582
3122428915,55sleep131294-21/usr/sbin/munin07:53:481
289828917,54sleep30-21swapper/309:48:573
2250728914,60sleep122593-21fschecks_time09:08:511
2479828715,55sleep124864-21grep09:15:421
1502628718,44sleep115183-21tail08:44:011
1444528715,49sleep210905-21kworker/2:212:08:532
323028620,47sleep13377-21grep08:08:331
2798128610,61sleep228102-21timerwakeupswit09:24:022
1141628618,48sleep311124-21/usr/sbin/munin11:58:533
392428536,32sleep10-21swapper/111:34:011
2657728525,36sleep126572-21apt-get07:38:461
300342830,8sleep30-21swapper/307:48:523
551128225,24sleep20-21swapper/209:58:332
392328211,56sleep33982-21sendmail_mailqu08:09:003
2297528214,52sleep123109-21latency_hist07:28:331
2886028119,37sleep128857-21apt-get09:28:451
184282810,7sleep20-21swapper/207:13:392
1635228137,22sleep00-21swapper/007:08:340
2307728017,49sleep223096-21apt-get10:53:422
2220327925,22sleep20-21swapper/207:24:022
219312790,6sleep30-21swapper/307:23:533
2058327917,31sleep30-21swapper/307:18:583
813227821,27sleep20-21swapper/208:23:482
3192788,17sleep10-21swapper/109:39:021
2527627819,44sleep225356-21xargs10:58:592
1277227810,53sleep212779-21df_inode12:03:492
1784327721,43sleep117864-21cron07:13:311
741727619,26sleep30-21swapper/310:03:503
2178427523,24sleep20-21swapper/212:33:312
151227522,31sleep30-21swapper/311:28:303
3040327420,35sleep230400-21apt-get09:33:432
2608527417,34sleep326109-21munin-run07:38:323
2476627424,27sleep20-21swapper/207:33:462
2141727412,42sleep321506-21awk09:04:013
2141727412,42sleep321506-21awk09:04:013
1127727421,28sleep30-21swapper/310:14:043
2337227324,27sleep20-21swapper/212:38:322
1927427327,28sleep20-21swapper/208:58:472
1927427327,28sleep20-21swapper/208:58:472
163292738,15sleep10-21swapper/108:48:581
862927220,28sleep30-21swapper/308:24:013
2702127232,21sleep30-21swapper/309:23:313
192427210,28sleep30-21swapper/311:28:513
192427210,28sleep30-21swapper/311:28:513
1871327221,35sleep218793-21apt-config08:58:312
1340927226,21sleep20-21swapper/208:39:022
560227116,40sleep25699-21timerandwakeup08:13:592
485827117,33sleep14998-21sed09:54:041
2010027122,26sleep30-21swapper/310:43:503
1558327115,41sleep115738-21cat12:13:321
2457727018,35sleep124719-21cat07:33:331
2444127017,32sleep30-21swapper/310:58:313
88526921,26sleep20-21swapper/207:59:022
823526914,39sleep18293-21/usr/sbin/munin11:48:541
2858626914,40sleep228593-21ntpq07:44:002
2858626914,40sleep228593-21ntpq07:44:002
1984726913,38sleep119878-21munin-run07:18:321
161126911,27sleep30-21swapper/308:03:313
1603526914,26sleep30-21swapper/310:29:053
783826820,30sleep17856-21munin-run08:23:311
3268026814,30sleep20-21swapper/211:23:412
2133326825,21sleep20-21swapper/209:04:002
2133326825,21sleep20-21swapper/209:04:002
1753626826,23sleep10-21swapper/108:53:501
154126822,25sleep10-21swapper/109:43:591
1235726815,28sleep20-21swapper/210:18:512
1014126818,21sleep20-21swapper/211:54:002
438426620,22sleep30-21swapper/309:53:543
2872826615,35sleep128808-21awk07:44:031
2872826615,35sleep128808-21awk07:44:031
2762626621,27sleep327794-21gpgv11:08:323
1664026621,29sleep20-21swapper/210:33:322
668226519,20sleep30-21swapper/311:43:523
3162426519,21sleep30-21swapper/311:19:023
250826517,20sleep20-21swapper/211:29:022
250826517,20sleep20-21swapper/211:29:022
3124826418,23sleep20-21swapper/211:18:532
311226422,20sleep10-21swapper/111:33:331
311226422,20sleep10-21swapper/111:33:331
2518426415,22sleep10-21swapper/110:58:581
240802646,11sleep10-21swapper/110:54:501
1848526420,24sleep30-21swapper/307:13:413
1483526413,32sleep314927-21/usr/sbin/munin12:09:023
886026321,17sleep10-21swapper/110:08:491
289392638,18sleep20-21swapper/209:28:502
1904026316,19sleep20-21swapper/210:39:012
1904026316,19sleep20-21swapper/210:39:012
1135726318,19sleep20-21swapper/211:58:522
757026217,24sleep10-21swapper/110:03:531
572326215,29sleep35864-21grep08:14:003
3162326221,22sleep20-21swapper/207:53:572
256532629,12sleep30-21swapper/310:59:403
2447026217,23sleep30-21swapper/307:33:313
1964026224,21sleep30-21swapper/308:59:013
1964026224,21sleep30-21swapper/308:59:013
1820926216,19sleep10-21swapper/110:38:311
1820926216,19sleep10-21swapper/110:38:311
3010726120,17sleep20-21swapper/211:14:002
277812616,13sleep20-21swapper/207:42:472
253672617,12sleep10-21swapper/107:34:021
829426018,21sleep20-21swapper/211:48:552
538126012,32sleep15387-21ntp_states11:38:581
2632326016,20sleep10-21swapper/111:03:501
2213126018,20sleep20-21swapper/210:49:022
1920826018,24sleep30-21swapper/312:23:523
1821926024,17sleep20-21swapper/210:38:322
1821926024,17sleep20-21swapper/210:38:322
933225911,29sleep20-21swapper/210:09:022
773025917,20sleep10-21swapper/111:48:301
371925917,26sleep13768-21latency08:08:541
2058325911,32sleep120675-21cat10:44:001
1287325912,24sleep10-21swapper/108:38:481
128725912,34sleep11288-21sshd11:25:341
1061625915,25sleep20-21swapper/210:13:422
1004125913,29sleep110054-21/usr/sbin/munin08:28:561
84025818,20sleep30-21swapper/309:43:323
353625816,19sleep20-21swapper/208:08:512
2990825818,20sleep10-21swapper/107:48:511
269792586,12sleep20-21swapper/211:04:032
242302586,44sleep20-21swapper/209:14:252
2019825814,27sleep120205-21diskstats07:18:481
1434325817,26sleep314340-21apt-get08:43:433
1098525821,16sleep20-21swapper/208:33:332
1016025815,27sleep310183-21munin-run10:13:313
2932525717,19sleep20-21swapper/211:13:322
2201125710,30sleep322017-21ntp_states10:48:593
1976925717,23sleep119909-21latency_hist10:43:321
1976925717,23sleep119909-21latency_hist10:43:321
1716025714,25sleep217186-21munin-run08:53:312
704325616,26sleep27075-21proc_pri08:19:002
39025615,26sleep1416-21df07:58:481
2943325617,20sleep30-21swapper/307:48:333
2943325617,20sleep30-21swapper/307:48:333
2622425622,19sleep20-21swapper/209:18:582
2180025615,25sleep321953-21date12:33:323
1878825613,28sleep318827-21memory10:38:543
1878825613,28sleep318827-21memory10:38:543
1573925621,18sleep10-21swapper/107:04:031
1061525611,26sleep10-21swapper/110:13:411
848525515,27sleep38493-21awk11:49:013
344625514,20sleep30-21swapper/311:33:513
1246025514,19sleep10-21swapper/112:03:311
109322556,10sleep30-21swapper/308:33:303
362325415,23sleep23717-21memory11:33:542
2953725420,16sleep30-21swapper/309:29:043
158425416,19sleep20-21swapper/209:44:002
1353325417,17sleep30-21swapper/310:23:323
1122525414,23sleep311221-21apt-get08:33:463
1879425318,18sleep30-21swapper/308:58:323
1586125317,16sleep10-21swapper/110:29:031
647525213,21sleep26542-21df11:43:472
2738525213,23sleep127394-21diskstats09:23:491
263525210,27sleep12702-21df09:48:511
2493425212,25sleep325055-21sed09:15:443
238925212,23sleep32457-21latency_hist09:48:333
2021125213,24sleep21-21systemd12:28:312
1927325213,23sleep119268-21apt-get08:58:461
1927325213,23sleep119268-21apt-get08:58:461
190225212,24sleep21896-21apt-get08:03:462
122712527,33sleep10-21swapper/108:35:461
956625113,22sleep19563-21apt-get11:53:431
2707225114,14sleep10-21swapper/109:23:331
1021825115,15sleep30-21swapper/311:54:013
1766625010,14sleep30-21swapper/312:18:553
3182624912,21sleep331902-21awk07:54:023
1589724915,15sleep30-21swapper/308:48:473
1385024912,22sleep213856-21diskstats10:23:472
821524814,17sleep10-21swapper/108:23:511
38152476,32sleep20-21swapper/209:52:352
1430524713,18sleep10-21swapper/110:23:591
167042466,26sleep30-21swapper/310:33:333
167042466,26sleep30-21swapper/310:33:333
3174224511,21sleep331682-21latency_hist09:38:323
590824410,17sleep10-21swapper/108:14:061
3086124414,15sleep10-21swapper/111:18:311
223902438,18sleep30-21swapper/309:08:453
1623924211,16sleep10-21swapper/112:13:571
2210624010,15sleep30-21swapper/312:33:493
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional