You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-18 - 03:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Tue Aug 18, 2026 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
46762578417,5753sleep33224-21kworker/3:221:05:143
242442275215,2717sleep324228-21apt-get22:09:563
285892115915,1131sleep028630-21ntpq20:40:030
27908292221,885sleep227921-21/usr/sbin/munin00:00:162
17992268413,654sleep218087-21/usr/sbin/munin23:30:072
6958268313,652sleep17018-21/usr/sbin/munin21:14:581
12338261312,589sleep19278-21kworker/1:023:13:141
2671825838,561sleep026763-21unin-run23:59:370
607257320,534sleep1713-21nin-node19:14:391
1193253512,506sleep31223-21munin-node19:14:453
13875251313,480sleep113900-21awk19:54:371
29663242115,385sleep029724-21find22:25:280
274612410200,182sleep027499-21fschecks_count22:19:580
2712234618,310sleep12659-21latency_hist22:44:391
196882331148,156sleep00-21swapper/023:35:140
21312317124,61sleep02280-21unixbench_singl22:40:100
1119122568,209sleep011164-21/usr/sbin/munin23:09:540
1972224622,206sleep1367-21snmpd20:59:391
742222436,109sleep00-21swapper/022:58:150
23074223834,189sleep023200-21timerwakeupswit22:05:080
1072221916,158sleep0289-21dbus-daemon22:39:370
12711221520,179sleep012817-21fschecks_time19:49:560
16270221327,164sleep00-21swapper/023:25:040
22934220413,176sleep222999-21awk22:05:062
9986219913,74sleep010020-21proc_pri19:40:060
6644219638,23sleep00-21swapper/019:30:410
1829121969,169sleep018292-21sshd20:06:150
25870219314,162sleep325872-21ntp_states23:55:043
364821900,8sleep00-21swapper/019:20:060
364821900,8sleep00-21swapper/019:20:060
2822218923,146sleep02945-21timerandwakeup21:00:110
23610218817,156sleep123690-21tune2fs20:24:581
17048218429,98sleep017057-21munin-run20:04:380
2208621839,160sleep022182-21latency_hist23:44:390
15934218211,155sleep015967-21awk21:44:370
3882218128,64sleep03953-21cmp00:29:380
1741621797,68sleep017391-21/usr/sbin/munin23:29:530
5099217115,140sleep05243-21latency_hist21:09:400
16390216615,130sleep00-21swapper/021:45:130
2901321659,140sleep00-21swapper/000:05:060
7951216414,48sleep07989-21fschecks_count23:00:130
32000216010,13sleep30-21swapper/319:10:013
31900216013,129sleep231928-21/usr/sbin/munin20:50:032
3174221589,41sleep00-21swapper/019:09:530
18199215832,43sleep00-21swapper/021:50:020
1465321579,65sleep014733-21mailstats19:55:090
11315215310,131sleep011410-21latency_hist21:29:390
11069214813,119sleep111113-21cpu19:44:521
375214617,114sleep1385-21unin-run20:54:381
2747021468,96sleep00-21swapper/000:00:100
30784214514,39sleep00-21swapper/022:29:560
1250214517,109sleep2561-21apt-get19:14:482
5011214411,99sleep00-21swapper/019:25:040
1959214411,99sleep01971-21latency_hist20:59:380
26780214010,93sleep0367-21snmpd22:16:150
30680213931,42sleep00-21swapper/000:10:070
963321379,93sleep09628-21apt-get23:04:490
2517521375,12sleep10-21swapper/122:10:261
9179213519,91sleep29288-21latency_hist23:04:392
26688213511,83sleep00-21swapper/020:34:540
26688213511,83sleep00-21swapper/020:34:540
19171213016,26sleep20-21swapper/223:34:472
2352421298,36sleep00-21swapper/020:24:530
19517212816,92sleep20-21swapper/221:55:012
2427621257,41sleep024323-21ntpq23:50:040
2427621257,41sleep024323-21ntpq23:50:040
24475212411,96sleep124550-21find23:50:091
24475212411,96sleep124550-21find23:50:091
3252421239,56sleep032607-21sed00:15:440
9923212225,77sleep10-21swapper/119:40:031
26378212224,78sleep126424-21latency_hist20:34:391
26378212224,78sleep126424-21latency_hist20:34:391
9059212116,77sleep30-21swapper/321:20:083
9059212116,77sleep30-21swapper/321:20:083
6872212029,34sleep00-21swapper/021:14:520
4362212017,80sleep34360-21apt-get00:29:493
1992521190,12sleep30-21swapper/323:35:163
3151211818,82sleep13191-21fschecks_count19:19:561
3151211818,82sleep13191-21fschecks_count19:19:561
486821169,51sleep04932-21mailstats00:30:140
2232521158,63sleep00-21swapper/023:44:510
9290211344,35sleep39499-21ls23:04:403
25072211210,42sleep00-21swapper/020:29:540
25072211210,42sleep00-21swapper/020:29:540
796721118,69sleep0201-21jbd2/mmcblk2p2-19:34:530
2994321107,41sleep00-21swapper/020:44:480
24232211012,37sleep00-21swapper/022:09:490
2276621100,8sleep30-21swapper/323:45:013
331221099,66sleep03353-21ntpq22:45:020
2737321099,84sleep127399-21netstat00:00:031
2221521097,87sleep022291-21sed22:04:400
4881210814,35sleep00-21swapper/022:50:030
18895210817,74sleep319070-21latency_hist23:34:403
10186210822,68sleep310303-21sort19:40:093
25630210715,65sleep20-21swapper/223:54:592
1238021078,26sleep012509-21latency_hist19:49:390
20851210617,69sleep320366-21/usr/sbin/munin20:15:073
20851210617,69sleep320366-21/usr/sbin/munin20:15:073
12735210610,52sleep00-21swapper/023:14:510
8431210510,83sleep08451-21apt-get21:19:490
8431210510,83sleep08451-21apt-get21:19:490
22525210510,77sleep00-21swapper/020:20:080
20601210510,37sleep00-21swapper/020:15:000
20601210510,37sleep00-21swapper/020:15:000
31349210319,68sleep231477-21timerwakeupswit22:30:102
1200521039,34sleep00-21swapper/021:30:030
591721017,13sleep00-21swapper/000:34:510
1965721019,12sleep00-21swapper/020:10:110
653221006,38sleep00-21swapper/021:14:000
28463210012,72sleep228556-21memory20:40:012
3192929814,68sleep331965-21ntp_kernel_pll_20:50:043
2225129818,62sleep122344-21memory20:20:011
130972989,54sleep00-21swapper/021:34:510
15572978,49sleep00-21swapper/019:14:580
1443729718,64sleep314570-21sed21:39:393
3189229616,64sleep131389-21apt-get00:14:491
235229514,65sleep22506-21date00:24:392
1753429517,19sleep10-21swapper/121:49:381
145682949,48sleep00-21swapper/023:19:590
3071429327,30sleep00-21swapper/019:05:100
305429214,56sleep22563-21/usr/sbin/munin00:25:072
655128926,41sleep36638-21fschecks_time19:30:403
2099728918,53sleep221069-21find20:15:102
2099728918,53sleep221069-21find20:15:102
514628814,57sleep25265-21timerwakeupswit22:50:092
1745428825,35sleep217456-21sshd21:49:152
229728617,30sleep00-21swapper/000:24:370
148328611,58sleep10-21swapper/119:14:551
69752849,61sleep36976-21sshd00:37:043
518928412,53sleep10-21swapper/119:25:091
2494328315,46sleep225108-21unixbench_multi22:10:232
3248428219,28sleep00-21swapper/020:50:420
1908328229,24sleep10-21swapper/121:54:391
1705828212,54sleep117076-21cat20:04:391
1584428212,53sleep115862-21/usr/sbin/munin23:24:491
221028112,15sleep10-21swapper/120:59:551
3136428011,56sleep21-21systemd00:14:372
647527821,26sleep10-21swapper/122:55:051
1930427816,46sleep319311-21ntpq20:10:063
395427722,36sleep10-21swapper/100:29:391
2396027619,44sleep224043-21fschecks_time23:49:582
2396027619,44sleep224043-21fschecks_time23:49:582
1514227621,25sleep20-21swapper/221:40:062
1476127622,33sleep114803-21fschecks_count21:39:571
3063727527,21sleep30-21swapper/319:05:093
192992755,12sleep00-21swapper/021:54:500
2125327428,26sleep20-21swapper/223:40:052
1432227419,37sleep20-21swapper/223:19:532
1044827422,28sleep30-21swapper/323:05:493
347427317,40sleep23509-21idleruntime-cro21:04:372
2944627320,34sleep129461-21apt-get20:44:371
2268827316,38sleep222837-21sed20:20:112
1738127320,34sleep217421-21fschecks_count20:04:562
930827214,24sleep20-21swapper/219:39:402
794427222,24sleep30-21swapper/319:34:523
384127226,32sleep33870-21munin-run00:29:373
2997527214,39sleep230042-21/usr/sbin/munin20:44:542
1450327223,23sleep30-21swapper/323:19:583
3028727120,29sleep30-21swapper/320:45:323
2245927122,27sleep30-21swapper/320:20:073
1400127019,27sleep114026-21munin-run23:19:361
1164827015,28sleep10-21swapper/121:29:561
430926915,30sleep10-21swapper/121:05:081
369226914,26sleep30-21swapper/322:45:093
2129426917,28sleep10-21swapper/120:15:131
2129426917,28sleep10-21swapper/120:15:131
1095026916,22sleep10-21swapper/123:09:391
603026821,28sleep15756-21apt-get22:54:401
1652726821,22sleep20-21swapper/223:25:102
1557626825,21sleep10-21swapper/123:24:371
317612675,51sleep30-21swapper/322:31:573
2639626717,38sleep126507-21timerandwakeup22:15:071
1128826714,35sleep311337-21irqstats19:44:583
74052668,17sleep10-21swapper/119:31:231
3079726619,31sleep130943-21cat19:05:111
1978526616,33sleep119809-21/usr/sbin/munin21:55:081
658126519,23sleep10-21swapper/121:14:371
459826522,16sleep30-21swapper/322:49:583
151912659,47sleep20-21swapper/219:56:402
2863026420,28sleep228643-21apt-get22:24:372
2856326411,39sleep228632-21df00:04:562
2612526410,39sleep20-21swapper/222:15:002
228526413,22sleep30-21swapper/320:59:573
1721726416,24sleep30-21swapper/323:29:403
3148326318,22sleep20-21swapper/219:09:392
2487226312,24sleep20-21swapper/220:29:392
2197926320,23sleep10-21swapper/122:02:141
2098326317,32sleep221037-21grep21:59:572
1707826319,28sleep217020-21latency_hist20:04:402
116022639,40sleep211627-21/usr/sbin/munin21:29:532
823426219,28sleep18261-21netstat23:00:181
818026222,25sleep18214-21/usr/sbin/munin19:34:591
734726217,27sleep20-21swapper/221:15:072
44326216,26sleep30-21swapper/322:35:083
432226213,22sleep20-21swapper/219:24:382
432226213,22sleep20-21swapper/219:24:382
2956626221,25sleep329620-21munin-run20:44:393
2721826214,26sleep10-21swapper/120:35:081
2721826214,26sleep10-21swapper/120:35:081
1989626215,26sleep30-21swapper/321:55:103
71826114,28sleep2869-21gpgv19:14:402
3192326114,27sleep231889-21apt-get00:15:102
3005826120,22sleep10-21swapper/120:44:571
104126112,28sleep11034-21apt-get00:19:531
348726018,30sleep13563-21sed22:45:071
151526013,25sleep30-21swapper/300:20:083
133692606,12sleep20-21swapper/221:35:022
825725921,17sleep30-21swapper/321:19:403
3019125915,19sleep10-21swapper/100:09:561
2856125912,27sleep128536-21/usr/sbin/munin00:04:551
2678325916,21sleep30-21swapper/320:34:573
2678325916,21sleep30-21swapper/320:34:573
2670025921,20sleep20-21swapper/220:34:562
2670025921,20sleep20-21swapper/220:34:562
1434825915,19sleep30-21swapper/319:55:003
979425814,26sleep29897-21perl23:05:362
591925820,19sleep20-21swapper/200:34:532
303592584,10sleep30-21swapper/322:28:543
2542225814,24sleep125489-21df23:54:541
1924825815,25sleep10-21swapper/120:10:041
191122588,11sleep20-21swapper/220:10:012
1871725812,26sleep118687-21apt-get23:34:381
1306925810,32sleep313075-21ntp_states19:50:033
1008425814,25sleep310089-21diskstats21:24:573
2866825713,26sleep328660-21munin-run22:24:383
278362577,12sleep10-21swapper/122:20:071
2404425714,22sleep315423-21kworker/3:123:49:593
2404425714,22sleep315423-21kworker/3:123:49:593
1144925722,17sleep20-21swapper/223:10:002
568725612,22sleep30-21swapper/321:09:593
559325614,23sleep25686-21memory21:09:582
545925612,29sleep15461-21sshd00:32:251
458825617,16sleep20-21swapper/219:24:502
2727325614,26sleep327276-21irqstats00:00:013
2272225613,19sleep20-21swapper/223:45:012
882425512,26sleep18858-21ntp_kernel_err21:20:021
882425512,26sleep18858-21ntp_kernel_err21:20:021
70725514,16sleep30-21swapper/320:54:573
627225514,27sleep26318-21grep22:54:542
461625515,17sleep30-21swapper/319:24:553
3193125514,23sleep332003-21df00:15:153
3162825515,18sleep10-21swapper/122:30:131
2912925515,21sleep20-21swapper/222:24:492
2270825511,17sleep30-21swapper/322:04:593
1823125520,18sleep20-21swapper/221:50:042
1400125520,17sleep10-21swapper/121:35:131
1333525516,24sleep213415-21wc23:15:052
1126225513,27sleep211293-21munin-run21:29:362
68225413,24sleep2706-21df_inode20:54:562
673125415,24sleep36737-21ntp_states22:55:113
573225416,20sleep10-21swapper/100:34:391
3017825412,24sleep330184-21df_inode00:09:543
264525416,20sleep30-21swapper/322:44:373
2371025419,20sleep10-21swapper/122:09:371
2118725414,24sleep321229-21ntp_states23:40:023
1270625413,26sleep312714-21/usr/sbin/munin23:14:453
1155025414,24sleep311546-21apt-get21:29:503
103722546,12sleep10-21swapper/121:25:031
1009925415,20sleep20-21swapper/219:40:082
306125313,24sleep33057-21apt-get19:19:503
306125313,24sleep33057-21apt-get19:19:503
301825315,23sleep22911-21/usr/sbin/munin22:44:562
2089025314,23sleep320892-21apt-get21:59:503
1729725312,25sleep317364-21df20:04:533
1192325315,15sleep20-21swapper/219:45:092
770725210,29sleep27777-21latency_hist19:34:392
438425219,16sleep20-21swapper/221:05:092
434125210,24sleep20-21swapper/200:29:482
2642725115,17sleep20-21swapper/220:34:402
2352225112,23sleep323518-21apt-get20:24:523
1917025113,22sleep118687-21apt-get23:34:461
1860625118,16sleep10-21swapper/120:09:391
1443625113,22sleep10-21swapper/119:55:021
115125110,24sleep21213-21sed22:39:392
852825010,15sleep20-21swapper/221:19:552
852825010,15sleep20-21swapper/221:19:552
163125013,22sleep21659-21irqstats22:39:592
1261325012,21sleep212609-21apt-get19:49:502
1147025012,21sleep311554-21memory23:10:013
3245624911,24sleep10-21swapper/122:34:581
2626024915,16sleep30-21swapper/322:15:053
170072499,15sleep30-21swapper/320:04:373
2549724811,20sleep325523-21ntp_states20:30:053
2549724811,20sleep325523-21ntp_states20:30:053
2243524811,21sleep122530-21perl23:44:561
1680024812,22sleep316880-21sed21:45:243
1781424710,20sleep317843-21df21:49:533
148862478,13sleep30-21swapper/321:39:593
2912224610,23sleep128643-21apt-get22:24:441
2544824614,19sleep20-21swapper/220:30:042
2086424510,14sleep10-21swapper/123:39:551
285582386,14sleep30-21swapper/300:04:533
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional