You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-10 - 02:41

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Wed Sep 09, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
64642268512,2658sleep030074-21kworker/u8:110:23:390
271152103927,1000sleep327151-21ntpq08:03:413
30064274012,708sleep130201-21/usr/sbin/munin09:53:481
30064274012,708sleep130201-21/usr/sbin/munin09:53:481
13301261627,553sleep313343-21ntpq10:43:413
28838242412,399sleep328843-21cat11:34:053
19239240820,367sleep119314-21fschecks_time11:03:341
28743239914,371sleep128753-21ntpq08:08:441
9307236713,338sleep19316-21unin-run10:33:171
28763235538,304sleep328804-21cut08:08:463
10536234419,307sleep110545-21/usr/sbin/munin12:18:171
11353233815,301sleep011365-21apt-get10:38:290
4418230910,242sleep30-21swapper/310:18:173
22853229318,257sleep122968-21/usr/sbin/munin07:48:491
22853229318,257sleep122968-21/usr/sbin/munin07:48:491
1893422868,263sleep319014-21grep09:18:453
6220224843,79sleep30-21swapper/312:03:413
28060223914,208sleep028200-21cat08:08:190
7559223521,198sleep06969-21apt-get08:43:450
2702022329,209sleep327113-21fschecks_time11:28:363
7668222743,59sleep30-21swapper/308:43:483
4387222511,71sleep34468-21grep11:58:363
22562222214,191sleep222641-21sendmail_mailtr07:48:462
22562222214,191sleep222641-21sendmail_mailtr07:48:462
7974221612,192sleep37988-21acpi10:28:273
26934221643,120sleep327067-21grep09:43:493
2813220951,55sleep30-21swapper/308:28:453
10315220118,166sleep010439-21nin-node07:13:180
19070219420,104sleep319176-21fschecks_time07:38:383
7801219317,90sleep37949-21latency_hist10:28:193
3203421917,172sleep332059-21awk08:18:403
17671218641,21sleep30-21swapper/310:58:353
22849217941,123sleep322825-21latency_hist09:33:193
30642217328,132sleep330721-21latency_hist09:58:183
30642217328,132sleep330721-21latency_hist09:58:183
21115216740,90sleep321195-21xargs07:43:473
7994216631,117sleep37615-21/usr/sbin/munin12:08:383
948216559,50sleep30-21swapper/308:23:363
20971216517,134sleep320989-21kernelversion11:08:413
17763216435,112sleep30-21swapper/307:33:413
16383216326,118sleep30-21swapper/310:53:393
1505921566,130sleep00-21swapper/007:23:490
3981215029,109sleep34051-21systemd-run11:58:163
11344215044,89sleep30-21swapper/310:38:283
31581214920,109sleep130856-21/usr/sbin/munin09:58:491
206921480,14sleep00-21swapper/010:08:440
206921480,14sleep00-21swapper/010:08:440
9878214519,69sleep310027-21tail12:13:473
856214447,54sleep31039-21cmp11:48:193
15987214447,84sleep316064-21switchtime12:33:463
15987214447,84sleep316064-21switchtime12:33:463
1259521440,12sleep10-21swapper/107:18:191
14299214311,77sleep30-21swapper/307:23:313
25585214119,23sleep30-21swapper/307:58:413
29062214012,78sleep329095-21sed09:53:173
1902213716,76sleep31941-21ntpq10:08:403
1902213716,76sleep31941-21ntpq10:08:403
15404213716,77sleep315494-21fschecks_time09:08:393
29971213632,85sleep129973-21cron11:38:151
10232213614,61sleep310261-21munin-run07:13:173
14164213414,101sleep38750-21cyclictest09:03:423
12822213413,43sleep30-21swapper/307:18:353
16309213355,57sleep30-21swapper/307:28:443
14477213330,91sleep314532-21cpuspeed10:48:343
18373213219,95sleep218501-21/usr/sbin/munin10:58:512
23494212643,65sleep30-21swapper/311:18:023
11666212411,57sleep0367-21snmpd08:56:120
1145212111,54sleep31164-21apt-get11:48:273
21073212015,88sleep121104-21/usr/sbin/munin11:08:431
12608211910,58sleep312617-21awk08:58:463
29762211820,80sleep029947-21latency_hist08:13:200
26636211618,84sleep026684-21/usr/sbin/munin11:28:170
26685211516,81sleep126727-21date11:28:181
955421149,13sleep30-21swapper/310:33:323
28255211418,74sleep128424-21sed11:33:181
20228211321,71sleep120252-21munin-run07:43:161
2836321128,87sleep128437-21find09:48:471
10940211254,42sleep310545-21/usr/sbin/munin08:53:413
17364210943,17sleep30-21swapper/309:13:443
554421080,6sleep10-21swapper/108:38:191
30358210844,17sleep30-21swapper/311:38:333
24292210844,50sleep324333-21proc_pri11:18:433
21949210714,74sleep221994-21/usr/sbin/munin09:28:412
5962210618,72sleep26016-21cmp10:23:172
29402210611,81sleep029520-21sed11:34:230
2073210515,76sleep02114-21cut08:28:180
16430210411,76sleep116522-21/usr/sbin/munin07:28:461
17594210310,80sleep0367-21snmpd10:58:280
466021028,40sleep30-21swapper/310:18:293
32704210238,48sleep30-21swapper/310:03:393
5216210139,48sleep35326-21dpkg08:38:183
5216210139,48sleep35326-21dpkg08:38:183
16941210014,64sleep016834-21/usr/sbin/munin09:13:330
2897729817,64sleep229079-21timerwakeupswit08:08:492
259192989,39sleep325960-21systemd-run09:43:163
2130229864,14sleep30-21swapper/309:28:183
2397629714,33sleep30-21swapper/307:53:403
309432967,67sleep30-21swapper/308:14:483
1347729618,58sleep013544-21cat10:43:470
2700929517,56sleep227019-21/usr/sbin/munin11:28:352
2249929532,48sleep30-21swapper/307:48:433
2249929532,48sleep30-21swapper/307:48:433
1127129418,58sleep30-21swapper/307:13:583
703129328,23sleep30-21swapper/308:43:233
2682829312,28sleep10-21swapper/108:03:351
1965729335,44sleep319746-21grep11:03:443
446529115,59sleep04560-21timerandwakeup08:33:470
446529115,59sleep04560-21timerandwakeup08:33:470
1735929115,26sleep20-21swapper/210:58:182
3141629018,51sleep30-21swapper/309:58:473
2994129016,62sleep030063-21cat09:53:470
2994129016,62sleep030063-21cat09:53:470
2125129024,43sleep121283-21grep09:28:161
2527328913,43sleep218416-21kworker/2:009:38:472
2224928915,52sleep222314-21cpuspeed11:13:332
1283628919,50sleep112893-21tune2fs07:18:371
864328729,31sleep30-21swapper/307:08:173
3193228724,40sleep132023-21fschecks_time11:43:361
1088528528,32sleep00-21swapper/010:38:170
738428426,41sleep27454-21df07:03:322
656028411,55sleep26569-21/usr/sbin/munin10:24:022
161728318,48sleep11620-21/usr/sbin/munin10:08:341
161728318,48sleep11620-21/usr/sbin/munin10:08:341
44552829,16sleep00-21swapper/010:18:180
3028928220,29sleep10-21swapper/111:38:311
2442428119,22sleep20-21swapper/209:38:182
2442428119,22sleep20-21swapper/209:38:182
3210728013,54sleep00-21swapper/010:02:470
1975628011,35sleep00-21swapper/009:23:180
165128013,49sleep21688-21ntp_states11:49:272
1453328011,54sleep014647-21timerwakeupswit12:28:500
1415728016,48sleep214163-21/usr/sbin/munin09:03:412
784527923,27sleep10-21swapper/107:03:431
566727921,38sleep05662-21apt-get08:38:270
1715127921,29sleep00-21swapper/007:33:180
1410827917,31sleep10-21swapper/109:03:401
1808527820,37sleep218243-21unixbench_multi07:33:482
386027711,49sleep13909-21/usr/sbin/munin08:33:321
277627723,30sleep20-21swapper/208:28:442
235422779,33sleep323541-21ntp_states09:33:433
235422779,33sleep323541-21ntp_states09:33:433
1531827717,45sleep215319-21cpu09:08:342
3240527614,46sleep232407-21acpi10:03:262
312152764,60sleep00-21swapper/011:38:590
2683327619,22sleep20-21swapper/208:03:362
1493427618,10sleep30-21swapper/312:30:353
1199527618,10sleep30-21swapper/312:22:393
964527518,39sleep19735-21fschecks_time10:33:381
2649527519,11sleep30-21swapper/311:26:303
1384927511,47sleep013914-21/usr/sbin/munin09:03:350
735827414,38sleep27401-21systemd-run12:08:172
408227424,27sleep10-21swapper/111:58:181
878327319,24sleep20-21swapper/208:48:192
515227317,37sleep25214-21unin-run08:38:172
515227317,37sleep25214-21unin-run08:38:172
3006427314,25sleep00-21swapper/008:13:280
2162927315,38sleep021693-21sort09:28:350
1952127315,29sleep20-21swapper/211:03:392
1760827312,47sleep017663-21irqstats07:33:380
1363727344,18sleep313641-21munin-run12:28:173
1044227320,34sleep110501-21munin-node07:13:191
3232227216,23sleep20-21swapper/211:43:452
973627114,44sleep29775-21idleruntime10:33:392
67127113,38sleep0825-21tail10:03:490
577327128,23sleep10-21swapper/112:03:191
297227114,41sleep13097-21timerwakeupswit08:28:471
1006027119,12sleep30-21swapper/308:50:303
1006027119,12sleep30-21swapper/308:50:303
2318427015,27sleep00-21swapper/009:33:360
2318427015,27sleep00-21swapper/009:33:360
1610327012,24sleep00-21swapper/009:08:520
939526915,33sleep28996-21/usr/sbin/munin08:48:432
939526915,33sleep28996-21/usr/sbin/munin08:48:432
2391326913,40sleep123805-21/usr/sbin/munin11:18:321
1909126922,20sleep10-21swapper/109:18:471
1366626817,23sleep20-21swapper/210:43:492
1251226814,25sleep00-21swapper/007:18:190
2528726725,23sleep20-21swapper/207:58:362
23742678,19sleep00-21swapper/011:52:320
1657126719,24sleep10-21swapper/110:53:451
1364526716,29sleep03988-21kworker/u8:212:28:180
1137426713,26sleep20-21swapper/210:38:312
1104726714,38sleep111063-21uname12:18:381
56902669,14sleep00-21swapper/012:03:180
2284726623,24sleep00-21swapper/011:13:470
1341226621,30sleep113448-21munin-run09:03:161
564226512,37sleep10-21swapper/108:38:241
530826518,23sleep00-21swapper/010:18:470
362226511,34sleep00-21swapper/008:33:190
1844926519,27sleep20-21swapper/209:18:322
902426418,28sleep09015-21apt-get08:48:330
902426418,28sleep09015-21apt-get08:48:330
854226412,37sleep18551-21vmstat12:08:481
1456026418,30sleep214565-21/usr/sbin/munin10:48:372
856326324,19sleep20-21swapper/210:28:462
556226318,28sleep20-21swapper/212:03:162
331726319,23sleep10-21swapper/110:13:391
2969626312,37sleep229724-21ntpq09:53:412
2969626312,37sleep229724-21ntpq09:53:412
2564726314,35sleep225701-21sed11:23:382
217342635,11sleep00-21swapper/007:47:430
2008326318,28sleep220093-21/usr/sbin/munin09:23:362
2508926216,31sleep025117-21munin-run11:23:160
1561026218,20sleep20-21swapper/207:28:192
1514026214,35sleep215184-21systemd-run12:33:162
1242226215,26sleep20-21swapper/212:23:352
2655226117,20sleep10-21swapper/108:03:191
2495326117,21sleep10-21swapper/107:58:181
1766426117,23sleep10-21swapper/107:33:391
1555226114,27sleep10-21swapper/109:08:411
1050426117,26sleep210728-21cat07:13:202
3230926019,23sleep20-21swapper/208:18:472
2650826016,23sleep00-21swapper/008:03:180
2388626013,19sleep20-21swapper/207:53:392
1543326015,27sleep015427-21apt-get12:33:310
1543326015,27sleep015427-21apt-get12:33:310
82725918,22sleep10-21swapper/110:03:501
734925916,26sleep17356-21modprobe12:08:161
356825915,18sleep00-21swapper/011:53:490
3083025913,29sleep230942-21sort08:14:472
2712025915,17sleep00-21swapper/011:28:370
2046325917,19sleep00-21swapper/011:08:180
1760025912,25sleep10-21swapper/110:58:331
1489125915,25sleep114929-21ntp_states10:48:441
1113225916,21sleep20-21swapper/207:13:292
1054425915,26sleep110563-21apt-get08:53:271
699125819,25sleep27002-21munin-run08:43:222
656825816,23sleep00-21swapper/012:03:510
2658825813,25sleep00-21swapper/009:43:410
1824925816,26sleep118383-21cat09:18:181
170112586,11sleep00-21swapper/010:54:490
138802585,42sleep00-21swapper/007:21:380
966525711,25sleep00-21swapper/012:13:440
775125714,19sleep10-21swapper/108:43:501
763725712,24sleep00-21swapper/012:08:270
3183625715,26sleep031829-21apt-get11:43:310
3073825714,28sleep130829-21sed08:14:461
2894925715,19sleep20-21swapper/211:34:122
2682325724,18sleep20-21swapper/209:43:482
2520925713,29sleep10-21swapper/109:38:461
24772574,12sleep20-21swapper/210:09:162
24772574,12sleep20-21swapper/210:09:162
2201025715,19sleep00-21swapper/011:13:180
1924525715,26sleep219337-21/usr/sbin/munin07:38:412
763925615,24sleep27633-21apt-get12:08:292
36832566,16sleep20-21swapper/211:54:282
326422565,42sleep00-21swapper/008:19:380
2308525614,26sleep123082-21apt-get09:33:291
2308525614,26sleep123082-21apt-get09:33:291
1897025615,25sleep018965-21apt-get07:38:310
1690825615,27sleep216930-21df09:13:312
141925618,20sleep10-21swapper/111:49:221
3166525515,24sleep131659-21apt-get08:18:291
2383025521,21sleep20-21swapper/211:18:292
2332625519,22sleep023340-21apt07:53:160
2332625519,22sleep023340-21apt07:53:160
2177725514,24sleep121808-21munin-run07:48:171
1448425515,20sleep10-21swapper/107:23:371
113125518,18sleep20-21swapper/208:23:392
919625415,24sleep29192-21apt-get12:13:302
85025419,17sleep10-21swapper/108:23:341
644825413,22sleep15936-21apt-get10:23:321
3173125412,23sleep231784-21users09:58:502
307725411,26sleep23071-21apt-get10:13:322
2645825413,26sleep126481-21irqstats09:43:381
1593325414,17sleep20-21swapper/207:28:352
154272547,36sleep00-21swapper/010:49:240
1275325419,18sleep20-21swapper/208:58:492
1138125416,20sleep10-21swapper/110:38:351
746325312,22sleep07469-21diskstats07:03:340
1066125310,17sleep20-21swapper/208:53:362
512625214,21sleep10-21swapper/110:18:441
2399225212,24sleep024036-21ntpq07:53:410
2070225212,24sleep220699-21apt-get11:08:322
158952529,17sleep10-21swapper/112:33:441
158952529,17sleep10-21swapper/112:33:441
975425113,19sleep10-21swapper/108:48:501
975425113,19sleep10-21swapper/108:48:501
3158925120,17sleep20-21swapper/211:43:172
2821825113,24sleep228225-21ntp_states09:48:442
1591825112,19sleep20-21swapper/212:33:452
1591825112,19sleep20-21swapper/212:33:452
133342515,12sleep10-21swapper/112:24:551
92292508,31sleep0367-21snmpd10:32:480
2596225012,23sleep00-21swapper/009:43:170
2233425010,20sleep10-21swapper/111:13:361
2169525014,17sleep10-21swapper/109:28:361
2006925014,19sleep00-21swapper/009:23:340
138652508,15sleep10-21swapper/112:28:311
1087225010,20sleep20-21swapper/212:18:342
2051524911,22sleep220511-21apt-get07:43:292
1220024912,22sleep012222-21df08:58:350
1274424811,21sleep212742-21apt-get07:18:292
138962475,29sleep10-21swapper/110:44:361
2783524610,15sleep00-21swapper/009:48:340
1208324410,17sleep00-21swapper/012:23:170
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional