You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-24 - 15:08

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Tue Mar 24, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2184253028,5282sleep030076-21kworker/0:110:03:590
43682134910,1324sleep34050-21http10:12:383
19509287313,844sleep019519-21/usr/sbin/munin09:17:530
22886238410,362sleep322911-21awk09:27:583
21539231515,284sleep321752-21latency_hist11:07:373
455822830,16sleep117-21ksoftirqd/110:12:591
5297226381,166sleep35006-21/usr/sbin/munin11:57:563
30287223853,61sleep330352-21latency_hist09:52:363
31082223469,19sleep30-21swapper/309:53:043
27799222213,192sleep227837-21ntp_kernel_err09:42:582
1174722136,13sleep30-21swapper/312:18:033
1174722136,13sleep30-21swapper/312:18:033
19520220616,175sleep119620-21fschecks_time09:17:541
15920220664,123sleep30-21swapper/312:32:493
16657220310,174sleep316684-21netstat09:07:593
3479219520,29sleep30-21swapper/310:08:063
31870219323,95sleep331571-21/usr/sbin/munin08:13:013
31870219323,95sleep331571-21/usr/sbin/munin08:13:013
2355219032,81sleep30-21swapper/311:47:593
2355219032,81sleep30-21swapper/311:47:593
8315218811,155sleep08439-21timerwakeupswit10:23:060
822721809,109sleep38308-21switchtime10:23:053
27916217418,93sleep327980-21sendmail_mailst09:43:033
19979215712,91sleep320056-21switchtime09:18:053
27905215618,115sleep30-21swapper/311:27:363
8224215533,99sleep310261-21cyclictest12:07:583
26354215372,16sleep30-21swapper/311:22:353
26354215372,16sleep30-21swapper/311:22:353
28209214728,100sleep228245-21idleruntime-cro08:02:342
12911214213,116sleep212943-21tune2fs12:22:552
24588214114,98sleep00-21swapper/009:34:040
21386213961,57sleep30-21swapper/307:38:053
14911213663,16sleep30-21swapper/309:02:553
811213416,102sleep0817-21cat08:17:550
4539213413,104sleep04530-21apt-get10:12:510
6619213347,71sleep30-21swapper/308:37:283
11890213353,62sleep312088-21cat07:12:373
13003213039,69sleep30-21swapper/310:38:063
14348212511,23sleep310261-21cyclictest12:27:443
10739212417,88sleep210756-21cut07:07:562
19183212016,86sleep319268-21latency_hist09:17:363
1269821205,12sleep00-21swapper/008:55:330
1269821205,12sleep00-21swapper/008:55:330
6857211821,14sleep30-21swapper/312:02:573
25439211710,91sleep125440-21sshd09:35:591
18332211617,79sleep118480-21grep07:28:381
19801211516,83sleep219848-21grep09:18:002
1403021147,39sleep30-21swapper/312:26:513
8159211117,79sleep28226-21systemctl10:23:042
312211147,11sleep30-21swapper/309:58:053
312211147,11sleep30-21swapper/309:58:053
14444211111,85sleep314489-21awk10:43:073
28497210915,25sleep00-21swapper/008:02:490
27983210915,74sleep028052-21cat09:43:040
3535210718,69sleep03568-21fschecks_count11:52:520
18908210619,30sleep318990-21latency_hist07:32:363
13948210615,33sleep313951-21cron07:17:343
10320210668,20sleep30-21swapper/308:47:573
10320210668,20sleep30-21swapper/308:47:573
5142210325,57sleep15297-21cat08:32:371
22958210317,68sleep223093-21unixbench_multi07:43:092
23821210210,31sleep323869-21awk07:47:503
7433210128,52sleep30-21swapper/310:22:363
11683210019,65sleep311726-21fschecks_count08:52:543
11683210019,65sleep311726-21fschecks_count08:52:543
10146210035,13sleep30-21swapper/312:13:023
980429912,74sleep29869-21cat08:47:362
443529918,22sleep30-21swapper/308:28:033
2703029911,29sleep327060-21grep07:57:553
2597229923,61sleep226048-21find07:53:062
2224729938,15sleep30-21swapper/307:42:453
1647029866,13sleep30-21swapper/307:23:073
1104629811,73sleep211082-21wc10:32:572
1236329725,47sleep011764-21apt-get07:12:430
84422967,12sleep10-21swapper/110:23:071
835229521,53sleep18109-21/usr/sbin/munin12:08:001
2248829513,70sleep322568-21cat11:08:143
2097129423,15sleep30-21swapper/309:22:493
1976629416,18sleep30-21swapper/307:33:043
508129318,49sleep25102-21modprobe08:32:342
2669729313,66sleep026720-21cat07:57:360
2287629312,63sleep122955-21find07:43:081
1650729311,67sleep116508-21sshd10:49:591
3004529213,18sleep30-21swapper/308:07:483
2596929218,59sleep225997-21cut09:37:542
254229241,39sleep32605-21latency08:22:583
1077029213,21sleep010773-21apt-get10:32:470
948329116,62sleep39499-21uname10:27:573
930629112,64sleep09319-21fschecks_count10:27:540
1077829110,17sleep30-21swapper/310:32:503
557929013,56sleep05632-21/usr/sbin/munin08:32:560
3135928913,62sleep131444-21fschecks_time11:37:511
3135928913,62sleep131444-21fschecks_time11:37:511
120428912,15sleep30-21swapper/310:02:493
702228821,13sleep30-21swapper/308:37:523
2545428811,19sleep30-21swapper/307:52:553
72428718,14sleep30-21swapper/308:17:493
2672128724,35sleep10-21swapper/107:57:371
3031328636,32sleep00-21swapper/008:07:570
281472868,66sleep328137-21apt-get11:27:523
3083228530,12sleep30-21swapper/311:36:043
2418228519,42sleep024331-21unixbench_singl11:13:070
2418228519,42sleep024331-21unixbench_singl11:13:070
480528444,26sleep34949-21cat11:57:363
316128419,48sleep13191-21modprobe11:52:331
316128419,48sleep13191-21modprobe11:52:331
1908128410,62sleep219154-21/usr/sbin/munin10:58:032
241622836,12sleep30-21swapper/309:32:383
2353228225,38sleep023544-21cron07:47:350
2251928213,50sleep10-21swapper/109:27:471
2144628217,48sleep221537-21grep11:07:362
1605228214,18sleep00-21swapper/009:07:360
150532828,15sleep30-21swapper/307:18:073
930828114,18sleep20-21swapper/207:03:002
860828112,46sleep08109-21/usr/sbin/munin12:08:070
2707728118,40sleep027139-21grep11:23:050
2392228113,13sleep30-21swapper/311:13:023
2392228113,13sleep30-21swapper/311:13:023
3130528027,34sleep131330-21munin-run08:12:351
793327911,45sleep28079-21cat12:07:362
1989227913,20sleep320012-21date11:02:363
155722797,17sleep315658-21perl10:47:553
1435227916,45sleep214444-21fschecks_time07:17:532
1326027910,18sleep313285-21grep08:57:543
313382787,15sleep30-21swapper/311:37:493
313382787,15sleep30-21swapper/311:37:493
512327719,40sleep05139-21cat08:32:360
1293727720,25sleep10-21swapper/108:57:361
1293727720,25sleep10-21swapper/108:57:361
1022427748,13sleep30-21swapper/307:07:373
96327619,32sleep00-21swapper/011:43:030
2917127619,39sleep20-21swapper/209:47:552
1342727615,41sleep113506-21find12:23:061
1292927615,38sleep213002-21/usr/sbin/munin10:38:052
84942759,49sleep30-21swapper/308:42:483
755127511,51sleep113441-21snmpd12:03:481
162412747,14sleep20-21swapper/207:23:022
1495027413,45sleep215041-21sed12:28:042
3079827314,40sleep030501-21/usr/sbin/munin09:52:570
3037327322,27sleep10-21swapper/111:33:071
160227330,21sleep10-21swapper/110:02:591
14432736,10sleep30-21swapper/311:45:203
1324627311,46sleep113252-21df_inode08:57:521
1434827219,26sleep10-21swapper/107:17:521
2394427117,27sleep20-21swapper/209:32:372
2291327116,29sleep00-21swapper/009:27:590
182112717,10sleep30-21swapper/310:56:593
1095927120,32sleep210961-21cron12:17:342
406527024,20sleep10-21swapper/110:12:361
295622708,12sleep30-21swapper/308:05:063
2639127012,29sleep00-21swapper/009:38:040
251327018,28sleep20-21swapper/210:07:362
1449027014,37sleep014538-21cat09:02:360
268472695,10sleep30-21swapper/309:38:483
2473026921,31sleep224763-21modprobe11:17:332
2473026921,31sleep224763-21modprobe11:17:332
1679226911,44sleep016858-21mailstats09:08:040
1260426916,24sleep00-21swapper/012:22:360
1260426916,24sleep00-21swapper/012:22:360
478426815,37sleep20-21swapper/211:57:352
2744626825,22sleep10-21swapper/107:58:051
2513226816,39sleep225095-21latency_hist07:52:372
1914826819,26sleep00-21swapper/007:32:490
1894426816,29sleep118552-21/usr/sbin/munin10:57:581
1585326824,23sleep00-21swapper/010:48:000
972926711,14sleep20-21swapper/210:28:042
545226722,30sleep15456-21/usr/sbin/munin08:32:531
323702677,13sleep00-21swapper/008:13:100
323702677,13sleep00-21swapper/008:13:100
3129826713,30sleep10-21swapper/109:53:061
3102826719,22sleep20-21swapper/211:37:362
296426720,23sleep00-21swapper/008:23:070
1636626714,37sleep116374-21diskstats09:07:531
1272626717,24sleep00-21swapper/010:37:590
548826612,28sleep00-21swapper/011:58:010
29072668,13sleep00-21swapper/010:07:550
2242026617,28sleep20-21swapper/211:08:122
2230326618,24sleep10-21swapper/111:08:071
1791926624,23sleep00-21swapper/009:12:530
1700026610,14sleep20-21swapper/210:52:512
692026517,31sleep06921-21munin-node08:37:450
59942659,11sleep10-21swapper/111:58:391
2876926516,28sleep10-21swapper/108:02:581
2751726523,20sleep10-21swapper/109:42:531
1910226517,30sleep119159-21awk09:17:341
1824826514,33sleep018329-21cat07:28:370
1015226520,21sleep10-21swapper/107:07:351
96526420,23sleep20-21swapper/210:02:362
96526420,23sleep20-21swapper/210:02:362
3260726417,22sleep20-21swapper/211:42:372
3260726417,22sleep20-21swapper/211:42:372
2876126416,28sleep00-21swapper/011:28:090
2705426413,35sleep227060-21ntp_states11:23:032
1811126417,25sleep20-21swapper/207:28:352
1612626410,23sleep00-21swapper/012:32:540
1474226416,19sleep00-21swapper/012:27:580
1173926412,35sleep111792-21sh07:12:351
612726315,32sleep16120-21apt-get10:17:501
203526313,37sleep12098-21latency_hist08:22:361
1598526317,20sleep20-21swapper/209:07:342
1250226317,23sleep10-21swapper/107:12:441
849726217,23sleep10-21swapper/108:42:511
740726211,31sleep17408-21ntp_states08:38:021
634626225,20sleep20-21swapper/212:02:362
3101326210,13sleep20-21swapper/209:53:032
3041526221,22sleep20-21swapper/208:07:592
1712126216,21sleep00-21swapper/007:27:360
1136126215,27sleep20-21swapper/208:52:362
1136126215,27sleep20-21swapper/208:52:362
218322616,11sleep20-21swapper/207:40:382
187726111,36sleep01899-21dpkg11:47:440
187726111,36sleep01899-21dpkg11:47:440
1825626123,24sleep118273-21cron10:57:341
81826017,24sleep10-21swapper/111:43:001
668226016,25sleep06791-21perl12:02:540
3134626014,30sleep031358-21/usr/sbin/munin11:37:500
3134626014,30sleep031358-21/usr/sbin/munin11:37:500
2907426014,20sleep10-21swapper/109:47:541
1648126020,18sleep10-21swapper/112:33:031
1539526020,18sleep10-21swapper/109:03:051
858825912,23sleep20-21swapper/212:08:052
537225914,21sleep20-21swapper/208:32:502
2630625919,23sleep226353-21systemctl11:22:342
2630625919,23sleep226353-21systemctl11:22:342
2013525913,25sleep10-21swapper/111:02:511
1599725916,17sleep10-21swapper/107:22:571
1012925918,20sleep00-21swapper/008:47:540
1012925918,20sleep00-21swapper/008:47:540
969625813,17sleep00-21swapper/007:03:060
694325816,25sleep26940-21apt-get08:37:472
2943725816,25sleep129449-21munin-run11:32:341
2667225813,26sleep126681-21diskstats11:22:531
2108925817,21sleep00-21swapper/007:37:580
1886125814,22sleep00-21swapper/010:57:570
1450125812,29sleep014535-21/usr/sbin/munin07:17:550
127632586,10sleep20-21swapper/208:56:242
127632586,10sleep20-21swapper/208:56:242
876525714,26sleep08833-21ls08:42:580
3157025719,19sleep20-21swapper/208:12:482
3157025719,19sleep20-21swapper/208:12:482
1451425719,20sleep00-21swapper/010:43:080
1444425715,18sleep10-21swapper/112:27:501
1408525712,28sleep014096-21modprobe12:27:330
1154225716,19sleep10-21swapper/112:17:571
1154225716,19sleep10-21swapper/112:17:571
3452569,18sleep20-21swapper/211:42:492
2826125615,20sleep00-21swapper/008:02:360
2504925618,21sleep125042-21apt-get11:17:491
2504925618,21sleep125042-21apt-get11:17:491
2097425611,24sleep121026-21cpuspeed09:22:511
170622566,18sleep20-21swapper/212:36:032
148722566,10sleep10-21swapper/110:43:501
108632565,11sleep00-21swapper/012:16:190
968125513,27sleep19676-21apt-get12:12:481
73692556,38sleep00-21swapper/010:22:250
3220025519,15sleep00-21swapper/009:57:540
3220025519,15sleep00-21swapper/009:57:540
264425513,25sleep12688-21ntpq08:23:001
2040725516,24sleep120475-21modprobe07:37:341
1270625514,24sleep212670-21apt-get07:13:092
629725420,20sleep20-21swapper/210:17:552
3260425414,19sleep10-21swapper/109:58:031
3260425414,19sleep10-21swapper/109:58:031
2945025415,16sleep20-21swapper/211:32:352
2849925413,25sleep228566-21/usr/sbin/munin08:02:512
2588325413,25sleep125878-21apt-get09:37:491
2097025414,24sleep220967-21apt-get09:22:482
1783225415,22sleep217829-21apt-get09:12:472
121225413,22sleep20-21swapper/208:18:052
1060125413,17sleep00-21swapper/007:07:540
2995925314,20sleep00-21swapper/011:32:580
251292539,19sleep00-21swapper/011:17:520
251292539,19sleep00-21swapper/011:17:520
2386625313,26sleep123908-21ntp_kernel_pll_11:13:001
2386625313,26sleep123908-21ntp_kernel_pll_11:13:001
1957425312,16sleep20-21swapper/207:32:592
1317525311,24sleep213171-21apt-get08:57:492
1001925313,21sleep110124-21users10:28:071
47125215,24sleep1422-21latency_hist08:17:351
47125215,24sleep1422-21latency_hist08:17:351
450925213,24sleep20-21swapper/210:12:452
2175425210,15sleep00-21swapper/011:07:380
2013425212,24sleep020128-21apt-get11:02:500
181412526,36sleep00-21swapper/010:56:030
1629125211,24sleep216285-21apt-get09:07:502
1136225217,17sleep20-21swapper/212:17:542
1136225217,17sleep20-21swapper/212:17:542
111792526,11sleep10-21swapper/110:33:001
381825111,26sleep13426-21/usr/sbin/munin11:52:581
283725113,23sleep22838-21diskstats10:07:532
2381825114,21sleep023814-21apt-get07:47:470
250162505,34sleep10-21swapper/107:51:471
218972505,34sleep00-21swapper/009:23:190
190125011,22sleep21897-21apt-get11:47:462
190125011,22sleep21897-21apt-get11:47:462
2198524912,22sleep122013-21munin-run07:42:351
1284624910,17sleep10-21swapper/110:38:041
1397824811,22sleep213982-21df_inode10:42:542
1573424711,22sleep215818-21memory10:47:582
24922469,14sleep20-21swapper/208:22:572
100482469,23sleep110115-21df08:47:511
100482469,23sleep110115-21df08:47:511
248522459,14sleep20-21swapper/209:34:102
289802439,17sleep00-21swapper/009:47:490
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional