You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-17 - 07:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Mon Aug 17, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
77772657211,6544sleep03775-21kworker/0:223:35:080
25239258787,5859sleep119154-21kworker/1:222:46:091
9836252710,500sleep09849-21/usr/sbin/munin23:40:120
11166235111,318sleep111310-21grep23:45:101
14602234821,211sleep114624-21munin-plugin-st22:14:401
14602234821,211sleep114624-21munin-plugin-st22:14:401
26187227813,123sleep126266-21fschecks_time19:25:021
2884221087,35sleep13033-21unixbench_multi21:35:121
1597221018,175sleep11632-21proc_pri23:15:111
25389218512,83sleep124999-21/usr/sbin/munin21:05:051
20764218311,122sleep120811-21grep00:15:091
5448217215,94sleep10-21swapper/121:45:011
2824216911,42sleep10-21swapper/123:19:591
21022216911,112sleep121172-21sed20:50:121
26345216615,106sleep30-21swapper/321:09:433
24751216331,117sleep124809-21latency_hist00:29:421
15273216134,109sleep10-21swapper/122:15:121
15273216134,109sleep10-21swapper/122:15:121
9157216010,40sleep10-21swapper/123:40:011
27061215953,93sleep127086-21awk22:54:401
23469215617,122sleep223463-21apt-get20:59:572
20744215637,99sleep320786-21latency_hist19:09:423
17085215316,22sleep00-21swapper/022:20:120
17085215316,22sleep00-21swapper/022:20:120
19335215253,88sleep119334-21/usr/sbin/munin22:29:421
14429214856,74sleep10-21swapper/120:30:131
893214610,118sleep11015-21latency_hist23:14:421
24306214516,114sleep224316-21systemd19:19:422
22469214524,100sleep120778-21cyclictest22:39:431
27879214214,39sleep127835-21latency_hist00:39:431
8370214019,103sleep0231-21systemd-journal21:54:410
8370214019,103sleep0231-21systemd-journal21:54:410
7405214054,72sleep17556-21date20:09:431
7405214054,72sleep17556-21date20:09:431
11048213912,111sleep111135-21latency20:20:041
1108213611,109sleep01127-21apt-get23:14:540
31318213217,90sleep10-21swapper/121:24:431
25190213138,78sleep10-21swapper/119:20:141
8173213063,40sleep10-21swapper/120:10:111
27200213015,99sleep00-21swapper/021:10:130
25674212915,97sleep125829-21unixbench_multi00:30:151
16156212684,21sleep10-21swapper/100:00:151
1330921265,109sleep00-21swapper/020:26:090
4167212526,18sleep10-21swapper/121:40:051
30564212566,40sleep10-21swapper/119:39:431
32387212415,20sleep10-21swapper/121:25:161
4951212361,23sleep15099-21grep23:25:161
25592212214,93sleep025670-21find00:30:140
4545212016,90sleep24598-21ls19:59:592
4545212016,90sleep24598-21ls19:59:592
32326212013,94sleep132340-21cpu19:45:001
9562211940,63sleep10-21swapper/120:15:081
9562211940,63sleep10-21swapper/120:15:081
19416211941,61sleep10-21swapper/100:10:131
20890211841,16sleep10-21swapper/122:34:421
16211211831,16sleep10-21swapper/120:36:151
7994211617,44sleep18115-21timerwakeupswit23:35:131
12738211516,24sleep10-21swapper/122:05:401
12738211516,24sleep10-21swapper/122:05:401
11801211511,87sleep211770-21latency_hist23:49:432
1074521138,83sleep10-21swapper/122:00:111
26807211217,79sleep326826-21sed00:35:053
23453211116,75sleep123538-21mailstats19:15:091
28363210816,74sleep028360-21apt-get21:14:530
2346821089,17sleep10-21swapper/120:59:561
13830210815,22sleep20-21swapper/222:10:092
13830210815,22sleep20-21swapper/222:10:092
6729210715,70sleep36940-21cut21:49:433
31629210610,25sleep231635-21df_inode21:25:002
409210515,77sleep0460-21latency_hist21:29:420
2040921059,47sleep00-21swapper/020:49:580
13876210534,52sleep313943-21/usr/sbin/munin20:29:593
12529210415,73sleep012682-21head22:05:140
12529210415,73sleep012682-21head22:05:140
4224210025,58sleep0314-21rs:main0
4224210025,58sleep0314-21rs:main0
310432997,77sleep131093-21sed19:40:041
2694029914,25sleep20-21swapper/200:35:082
326812975,12sleep20-21swapper/223:10:162
309462979,75sleep231009-21processes23:05:072
309462979,75sleep231009-21processes23:05:072
1657329713,71sleep116677-21fschecks_time22:20:011
1657329713,71sleep116677-21fschecks_time22:20:011
268829613,69sleep02707-21apt-get23:19:500
2006129618,23sleep30-21swapper/300:14:433
1742429610,73sleep317472-21if_err_eth000:05:183
1625829616,64sleep316196-21latency_hist22:19:433
1625829616,64sleep316196-21latency_hist22:19:433
122142966,11sleep10-21swapper/123:50:021
1092129620,59sleep311072-21sed22:00:133
481529514,61sleep10-21swapper/120:00:061
481529514,61sleep10-21swapper/120:00:061
1884429520,58sleep218940-21uniq00:10:012
1212729514,27sleep00-21swapper/023:50:010
2735229312,15sleep10-21swapper/122:54:571
2005329311,62sleep30-21swapper/320:49:413
2005329311,62sleep30-21swapper/320:49:413
83822929,69sleep10-21swapper/121:54:421
83822929,69sleep10-21swapper/121:54:421
1720529212,14sleep10-21swapper/100:04:561
772929116,61sleep37757-21fschecks_count20:10:013
245562915,11sleep20-21swapper/219:19:582
2946429028,43sleep029527-21latency19:35:020
96812897,72sleep1367-21snmpd21:56:351
42328910,65sleep3509-21switchtime19:45:143
260952880,7sleep20-21swapper/219:24:542
2187028810,14sleep10-21swapper/100:19:531
707128717,21sleep10-21swapper/123:34:401
2004028717,52sleep220052-21modprobe20:49:402
2004028717,52sleep220052-21modprobe20:49:402
1731428713,55sleep20-21swapper/220:39:592
1731428713,55sleep20-21swapper/220:39:592
1723528613,60sleep10-21swapper/120:39:541
1723528613,60sleep10-21swapper/120:39:541
607128511,58sleep36090-21apt-get20:04:523
607128511,58sleep36090-21apt-get20:04:523
265872857,61sleep10-21swapper/121:09:581
1849428513,56sleep01-21systemd20:44:420
1849428513,56sleep01-21systemd20:44:420
123002859,59sleep10-21swapper/120:24:521
824928421,51sleep28334-21timerandwakeup20:10:122
3250028415,18sleep30-21swapper/323:10:123
2481528317,45sleep024531-21/usr/sbin/munin19:20:040
2431828311,51sleep324470-21latency_hist19:19:433
1892428324,31sleep30-21swapper/322:25:163
289302828,58sleep10-21swapper/122:59:571
2320428217,47sleep223333-21ls00:24:432
2536128115,50sleep225390-21nfs4_client00:30:082
148352817,19sleep10-21swapper/123:56:281
107022810,7sleep00-21swapper/022:00:090
985028037,26sleep00-21swapper/020:15:150
985028037,26sleep00-21swapper/020:15:150
214512806,13sleep10-21swapper/119:10:081
2117328022,28sleep20-21swapper/220:50:132
968227918,25sleep30-21swapper/323:40:113
434027914,47sleep34349-21/usr/sbin/munin23:25:023
2824027912,12sleep10-21swapper/119:30:111
373627852,12sleep10-21swapper/119:55:131
373627852,12sleep10-21swapper/119:55:131
307632786,12sleep10-21swapper/123:05:021
307632786,12sleep10-21swapper/123:05:021
2952227815,42sleep029609-21find23:00:120
219662787,56sleep10-21swapper/120:55:001
1363527815,15sleep30-21swapper/320:29:433
63672777,18sleep20-21swapper/220:05:032
63672777,18sleep20-21swapper/220:05:032
430327726,27sleep20-21swapper/219:59:432
430327726,27sleep20-21swapper/219:59:432
3208427717,29sleep00-21swapper/019:44:430
1517927722,23sleep20-21swapper/220:34:422
63522768,18sleep00-21swapper/021:46:410
3045727619,37sleep230502-21awk19:39:402
2897327614,38sleep229131-21latency_hist19:34:442
2929627516,24sleep20-21swapper/223:00:062
2794527511,42sleep228042-21cat21:14:432
2473827511,49sleep324800-21/usr/sbin/munin22:45:103
2083827513,47sleep320851-21sh22:34:403
2671427416,27sleep00-21swapper/019:25:130
234482746,11sleep10-21swapper/122:40:111
1508327417,37sleep215126-40mandb23:59:402
3035927316,30sleep00-21swapper/021:20:060
2804327319,35sleep327863-21latency_hist21:14:443
1956427219,29sleep20-21swapper/219:05:012
188302729,47sleep00-21swapper/000:09:590
712127123,23sleep30-21swapper/323:34:423
3207227118,39sleep332082-21unin-run19:44:423
295172717,10sleep10-21swapper/121:18:051
260027121,25sleep30-21swapper/321:35:063
25672715,10sleep10-21swapper/119:53:161
25672715,10sleep10-21swapper/119:53:161
1940127129,24sleep10-21swapper/119:04:521
18082716,12sleep10-21swapper/121:33:191
1673427115,43sleep316830-21apt-config00:04:413
1085227114,22sleep00-21swapper/020:19:590
2232427017,30sleep322387-21munin-node19:14:423
196872706,52sleep10-21swapper/120:45:151
196872706,52sleep10-21swapper/120:45:151
871526919,32sleep38868-21ls23:39:433
60852698,34sleep20-21swapper/223:30:052
2774926913,43sleep327848-21fschecks_time19:30:003
2663726918,21sleep30-21swapper/319:25:123
2333626911,25sleep00-21swapper/019:15:040
1933626916,22sleep20-21swapper/222:29:432
991726817,34sleep39983-21modprobe21:59:403
869226822,20sleep20-21swapper/223:39:422
3149526824,27sleep20-21swapper/219:40:132
2460526813,39sleep324630-21df_inode19:19:593
2297526818,27sleep00-21swapper/022:40:020
1490726814,33sleep00-21swapper/022:15:070
1490726814,33sleep00-21swapper/022:15:070
1409626826,25sleep30-21swapper/323:55:053
2804126714,37sleep028079-21ntp_kernel_pll_19:30:060
2510726715,32sleep00-21swapper/021:05:000
2216726715,33sleep00-21swapper/000:20:050
181562676,11sleep00-21swapper/000:05:430
170792676,14sleep20-21swapper/200:04:452
473626614,30sleep20-21swapper/223:25:132
2868426619,27sleep20-21swapper/222:59:422
191592665,12sleep00-21swapper/022:28:030
509826513,39sleep05182-21sed20:00:130
509826513,39sleep05182-21sed20:00:130
2556526511,39sleep025715-21latency_hist22:49:430
2344226510,40sleep20-21swapper/200:24:552
228772655,12sleep20-21swapper/220:56:102
2280826514,22sleep30-21swapper/300:20:163
137632657,13sleep30-21swapper/322:10:063
137632657,13sleep30-21swapper/322:10:063
895226411,28sleep00-21swapper/020:14:420
874826423,19sleep00-21swapper/021:55:010
375026418,29sleep30-21swapper/321:39:503
31342646,11sleep20-21swapper/221:35:302
3108726415,21sleep00-21swapper/023:05:090
3108726415,21sleep00-21swapper/023:05:090
2396526424,25sleep223988-21munin-run22:44:402
2390526419,26sleep00-21swapper/000:25:090
2084626415,21sleep30-21swapper/320:50:103
1769026412,34sleep317696-21ntp_states20:40:083
1769026412,34sleep317696-21ntp_states20:40:083
1208626418,21sleep30-21swapper/320:24:423
92626318,25sleep2972-21latency21:30:052
386926315,22sleep00-21swapper/021:40:000
3126626311,39sleep331314-21/usr/sbin/munin21:24:413
1539126221,17sleep00-21swapper/020:34:440
11292626,14sleep20-21swapper/219:49:412
955426115,24sleep20-21swapper/223:40:102
599726117,22sleep00-21swapper/023:30:030
2549826117,20sleep30-21swapper/300:30:133
2009326111,33sleep020060-21latency_hist20:49:420
2009326111,33sleep020060-21latency_hist20:49:420
3252426019,21sleep00-21swapper/023:10:130
269602605,10sleep30-21swapper/322:53:213
233632606,11sleep00-21swapper/020:59:430
2159826020,21sleep00-21swapper/022:35:100
209652605,11sleep20-21swapper/222:34:432
1632226013,32sleep316391-21switchtime20:36:173
1329626015,26sleep30-21swapper/323:54:403
959225912,20sleep20-21swapper/220:15:092
959225912,20sleep20-21swapper/220:15:092
3055025911,29sleep30-21swapper/323:04:563
3055025911,29sleep30-21swapper/323:04:563
2110325915,26sleep321122-21apt-get22:34:523
1667425921,19sleep20-21swapper/200:04:402
1179525916,24sleep30-21swapper/322:04:573
1179525916,24sleep30-21swapper/322:04:573
976925824,19sleep30-21swapper/320:15:143
976925824,19sleep30-21swapper/320:15:143
3259625812,18sleep00-21swapper/019:45:070
275812585,12sleep00-21swapper/000:36:560
2433925819,19sleep20-21swapper/222:45:012
2389725817,23sleep323903-21ntp_states00:25:083
2037625814,28sleep320382-21diskstats00:14:593
1778625818,19sleep20-21swapper/222:24:422
1778625818,19sleep20-21swapper/222:24:422
1606825814,29sleep016155-21/usr/sbin/munin00:00:140
97862575,17sleep20-21swapper/221:57:582
572725717,23sleep20-21swapper/221:45:062
315925716,19sleep30-21swapper/323:20:053
2772325717,22sleep00-21swapper/022:55:080
2744725719,19sleep30-21swapper/322:55:033
238982578,39sleep30-21swapper/322:43:483
190425711,32sleep31983-21grep19:50:113
190425711,32sleep31983-21grep19:50:113
1382825713,17sleep00-21swapper/023:54:580
1074525718,22sleep00-21swapper/023:45:010
74225617,20sleep30-21swapper/321:30:023
2905125619,20sleep20-21swapper/221:15:282
2633425613,25sleep226344-21/usr/sbin/munin21:09:422
176592564,11sleep30-21swapper/322:23:423
176592564,11sleep30-21swapper/322:23:423
1563125612,25sleep00-21swapper/020:34:530
1536525612,18sleep00-21swapper/022:15:150
1536525612,18sleep00-21swapper/022:15:150
133802565,11sleep20-21swapper/220:27:032
304322555,11sleep30-21swapper/319:39:213
2943425517,19sleep30-21swapper/323:00:113
2847425519,20sleep30-21swapper/321:15:123
2029125511,17sleep00-21swapper/022:30:130
196525516,18sleep00-21swapper/021:34:430
140225517,24sleep30-21swapper/323:15:053
1211325514,26sleep212120-21df_inode23:49:592
426725412,24sleep04261-21apt-get23:24:590
2588025414,19sleep00-21swapper/022:49:590
2383725415,19sleep30-21swapper/321:00:073
114602545,10sleep20-21swapper/222:04:072
841825314,20sleep20-21swapper/221:54:432
841825314,20sleep20-21swapper/221:54:432
70812539,25sleep30-21swapper/321:49:513
70812539,25sleep30-21swapper/321:49:513
35425313,24sleep3407-21systemd-run21:29:413
2996125312,24sleep229955-21apt-get21:19:562
292025316,18sleep20-21swapper/223:20:002
264825315,15sleep00-21swapper/021:35:070
1942625312,23sleep019421-21apt-get19:04:550
1566325314,24sleep215650-21apt-get20:34:592
1179425312,24sleep211788-21apt-get22:04:562
1179425312,24sleep211788-21apt-get22:04:562
435225213,25sleep20-21swapper/221:40:102
276625211,18sleep00-21swapper/019:54:440
2322825217,17sleep30-21swapper/320:59:423
2027425211,24sleep020293-21apt-get00:14:520
15082529,16sleep20-21swapper/219:50:022
15082529,16sleep20-21swapper/219:50:022
2590025010,16sleep20-21swapper/222:50:012
139225011,24sleep01396-21find19:49:520
1170625012,24sleep30-21swapper/323:49:073
320624911,20sleep33235-21irqstats19:55:033
320624911,20sleep33235-21irqstats19:55:033
2188724913,19sleep00-21swapper/020:54:550
2151924911,22sleep021598-21systemctl19:10:110
1964624812,22sleep219653-21df_inode22:29:592
492124711,15sleep30-21swapper/320:00:083
492124711,15sleep30-21swapper/320:00:083
2110024711,18sleep20-21swapper/219:10:012
1511524713,17sleep20-21swapper/222:15:092
1511524713,17sleep20-21swapper/222:15:092
135224712,17sleep20-21swapper/223:15:042
107532478,15sleep20-21swapper/220:19:532
2081324211,13sleep20-21swapper/200:15:102
60992407,15sleep00-21swapper/020:04:570
60992407,15sleep00-21swapper/020:04:570
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional