You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-12 - 10:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Fri Jun 12, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3360273621,697sleep03416-21/usr/sbin/munin21:58:080
22606255226,400sleep322655-21basename21:17:443
21193233611,309sleep021267-21find19:28:150
21193233611,309sleep021267-21find19:28:150
27638230431,259sleep227644-21df_inode21:33:012
609229015,250sleep1696-21tune2fs23:33:041
18345228577,94sleep318407-21fschecks_time21:03:223
16730225410,206sleep316773-21ntp_kernel_pll_00:23:223
248502250123,26sleep30-21swapper/321:23:093
13990224552,140sleep314021-21munin-plugin-st22:32:443
1151222840,124sleep30-21swapper/321:52:453
10026222095,28sleep30-21swapper/322:18:303
9688220816,170sleep09774-21latency_hist20:37:460
16001220612,149sleep316094-21fschecks_time19:13:043
21119220312,144sleep321261-21latency_hist21:12:473
21119220312,144sleep321261-21latency_hist21:12:473
13437219078,69sleep313463-21netstat20:48:073
21895218313,132sleep321994-21latency_hist19:32:473
21895218313,132sleep321994-21latency_hist19:32:473
1069218331,134sleep31148-21sort23:33:143
8620218217,146sleep10-21swapper/120:33:071
430921817,124sleep30-21swapper/320:18:143
1012821817,38sleep30-21swapper/300:03:073
1012821817,38sleep30-21swapper/300:03:073
26484217915,146sleep326635-21latency_hist23:12:463
12834217844,19sleep30-21swapper/320:47:473
19646217628,133sleep319741-21cat19:23:143
817321736,133sleep38209-21ntpq22:13:093
11797217113,53sleep30-21swapper/300:08:243
5322216435,113sleep30-21swapper/322:03:163
5322216435,113sleep30-21swapper/322:03:163
2914321627,38sleep30-21swapper/321:37:593
14408215819,112sleep30-21swapper/322:33:043
6892215710,64sleep36982-21sshd20:28:043
31797215588,49sleep30-21swapper/323:28:093
31452215147,59sleep30-21swapper/320:02:563
2908215016,114sleep30-21swapper/321:57:463
2908215016,114sleep30-21swapper/321:57:463
16980214852,25sleep30-21swapper/320:58:133
16980214852,25sleep30-21swapper/320:58:133
23390214715,113sleep323393-21cron19:37:433
23390214715,113sleep323393-21cron19:37:433
665021469,80sleep36686-21ntpq22:08:083
665021469,80sleep36686-21ntpq22:08:083
3462214648,86sleep33528-21date20:17:463
26556214430,60sleep30-21swapper/319:47:453
5352214220,102sleep30-21swapper/323:48:043
20687214180,18sleep30-21swapper/300:37:443
20687214180,18sleep30-21swapper/300:37:443
29895214027,98sleep329902-21basename19:57:553
2451721388,95sleep30-21swapper/319:38:163
26263213712,110sleep30-21swapper/321:28:063
26948213418,100sleep127011-21idleruntime19:48:031
26948213418,100sleep127011-21idleruntime19:48:031
14488213015,89sleep20-21swapper/220:52:472
2867021279,16sleep30-21swapper/323:18:073
2867021279,16sleep30-21swapper/323:18:073
15831212723,16sleep30-21swapper/322:37:573
24924212429,17sleep324934-21munin-run23:07:453
10657212417,89sleep010785-21unixbench_multi20:38:150
18753212313,81sleep10-21swapper/121:03:321
16522212112,39sleep016525-21entropy20:58:020
16522212112,39sleep016525-21entropy20:58:020
16327212014,88sleep115835-21apt-get00:22:561
9293211813,89sleep09495-21apt-key22:17:460
10209211817,15sleep30-21swapper/320:38:053
5337211712,65sleep30-21swapper/320:23:033
11718211714,84sleep00-21swapper/022:23:200
12103211315,81sleep212208-21sort00:08:312
2360821118,87sleep323759-21sed21:18:163
2049321116,33sleep30-21swapper/322:52:593
2049321116,33sleep30-21swapper/322:52:593
186821117,88sleep30-21swapper/320:12:453
2325211011,82sleep22438-21unixbench_singl21:53:252
2325211011,82sleep22438-21unixbench_singl21:53:252
3068421098,22sleep330678-21apt-get21:42:563
29606210823,59sleep029659-21munin-plugin-st19:57:440
29606210823,59sleep029659-21munin-plugin-st19:57:440
9734210715,72sleep09599-21/usr/sbin/munin22:18:230
22353210710,15sleep20-21swapper/219:33:062
22353210710,15sleep20-21swapper/219:33:062
10443210716,77sleep110495-21processes20:38:121
31196210612,77sleep131344-21sed23:27:471
31196210612,77sleep131344-21sed23:27:471
26257210411,80sleep126259-21sshd23:10:301
2680321037,85sleep326696-21/usr/sbin/munin23:13:033
2680321037,85sleep326696-21/usr/sbin/munin23:13:033
32606210227,56sleep232645-21ntpq21:48:082
20193210218,57sleep220235-21ntpq21:08:092
20193210218,57sleep220235-21ntpq21:08:092
3383210126,56sleep13393-21modprobe20:17:441
32063210117,68sleep132179-21timerwakeupswit23:28:151
24316210127,52sleep124471-21grep23:03:151
19317210117,67sleep219354-21ntpq22:48:102
1918329921,14sleep30-21swapper/322:48:073
1894829924,50sleep018942-21apt-get22:48:000
1498229924,14sleep30-21swapper/320:53:363
1303929917,65sleep213118-21fschecks_time00:13:032
115972997,52sleep30-21swapper/322:23:153
2221929813,69sleep022292-21fschecks_time19:33:040
2221929813,69sleep022292-21fschecks_time19:33:040
1788329820,15sleep30-21swapper/300:27:593
114329715,68sleep01212-21sed20:08:140
876729615,60sleep28278-21/usr/sbin/munin23:58:132
43312966,36sleep30-21swapper/322:01:383
2508029611,68sleep125175-21cat21:23:151
247229613,16sleep30-21swapper/323:38:113
292322950,6sleep10-21swapper/119:53:151
292322950,6sleep10-21swapper/119:53:151
2016329517,63sleep120192-21nfs4_client21:08:081
2016329517,63sleep120192-21nfs4_client21:08:081
2578229412,68sleep025792-21latency_hist21:27:460
220642948,48sleep30-21swapper/322:57:563
1904129413,67sleep019045-21grep19:22:580
483329312,59sleep00-21swapper/023:47:080
272552936,11sleep30-21swapper/319:48:093
272552936,11sleep30-21swapper/319:48:093
2131529310,65sleep00-21swapper/021:12:560
133222938,48sleep30-21swapper/300:13:083
67942926,28sleep30-21swapper/323:53:053
186852926,54sleep30-21swapper/319:21:523
46352915,10sleep30-21swapper/323:44:373
183342916,32sleep30-21swapper/322:43:343
95832905,11sleep30-21swapper/320:37:253
287152906,29sleep30-21swapper/321:36:173
1245228934,33sleep00-21swapper/019:03:040
448428821,43sleep24439-21latency_hist22:02:462
3197728816,58sleep032062-21cat23:28:140
2202428818,45sleep122156-21tail21:13:171
96192877,32sleep10-21swapper/122:17:561
2379428716,58sleep223829-21fschecks_count19:38:032
1179028712,20sleep10-21swapper/122:23:211
744928621,44sleep17598-21grep23:53:191
307842865,28sleep30-21swapper/323:23:333
307842865,28sleep30-21swapper/323:23:333
156772865,28sleep30-21swapper/300:21:043
295682856,45sleep30-21swapper/319:57:143
295682856,45sleep30-21swapper/319:57:143
1537728512,21sleep10-21swapper/120:53:551
811328416,50sleep28254-21sed20:32:472
151202846,42sleep30-21swapper/319:11:343
656428311,56sleep26701-21latency_hist20:27:462
1718728321,29sleep00-21swapper/022:42:470
1177728318,51sleep111786-21sed00:08:221
525528215,32sleep00-21swapper/020:23:000
2602028213,29sleep10-21swapper/121:28:001
2184528221,32sleep10-21swapper/119:32:451
2184528221,32sleep10-21swapper/119:32:451
1803628220,31sleep20-21swapper/221:02:482
1797028130,27sleep00-21swapper/000:28:040
1612528119,27sleep20-21swapper/222:38:082
1386128115,54sleep00-21swapper/022:31:030
1564428013,28sleep00-21swapper/019:12:530
155828020,27sleep10-21swapper/121:52:561
155828020,27sleep10-21swapper/121:52:561
1525728012,53sleep115400-21unixbench_singl00:18:151
1521028017,42sleep015243-21munin-plugin-st19:12:440
2827827915,44sleep228286-21/usr/sbin/munin23:17:542
2827827915,44sleep228286-21/usr/sbin/munin23:17:542
1503327915,18sleep20-21swapper/200:18:122
3022027830,21sleep20-21swapper/223:23:072
3022027830,21sleep20-21swapper/223:23:072
247772785,15sleep00-21swapper/023:06:210
1748527820,43sleep117508-21fschecks_count22:43:041
2733027715,30sleep20-21swapper/219:48:132
2733027715,30sleep20-21swapper/219:48:132
258527719,41sleep22623-21/usr/sbin/munin20:13:122
2058027728,26sleep00-21swapper/022:53:040
2058027728,26sleep00-21swapper/022:53:040
111682779,11sleep10-21swapper/100:07:451
111682779,11sleep10-21swapper/100:07:451
993127616,44sleep210025-21memory22:18:292
65027624,28sleep10-21swapper/120:08:021
514327611,44sleep15144-21/usr/sbin/munin23:47:541
1355627613,47sleep013628-21mailstats20:48:120
838427510,29sleep30-21swapper/323:58:023
2662627520,28sleep00-21swapper/021:28:150
1432427514,45sleep114363-21ntpq19:08:091
1362627527,30sleep113656-21munin-run19:07:451
1350027519,25sleep00-21swapper/000:13:130
3184827418,24sleep00-21swapper/020:03:090
2951227411,46sleep029118-21/usr/sbin/munin21:38:080
2765427414,42sleep027697-21fschecks_count21:33:030
1652627422,32sleep116614-21fschecks_time20:58:031
1652627422,32sleep116614-21fschecks_time20:58:031
3044527323,23sleep10-21swapper/121:42:461
1826427315,43sleep218307-21ntpq00:28:102
1740227312,45sleep217465-21cpuspeed22:43:012
104027319,36sleep11064-21munin-run21:52:431
2748427222,25sleep00-21swapper/019:48:150
2748427222,25sleep00-21swapper/019:48:150
1726027216,24sleep20-21swapper/219:17:472
378727115,35sleep03869-21grep20:18:030
1494927124,23sleep20-21swapper/222:33:152
1152227120,22sleep00-21swapper/020:42:590
340127024,20sleep20-21swapper/223:42:462
2280927015,34sleep222960-21unixbench_singl22:58:152
1954927026,29sleep20-21swapper/221:07:452
177702707,49sleep117473-21/usr/sbin/munin19:18:071
3045726913,39sleep230530-21sensors19:58:142
183026926,27sleep20-21swapper/223:37:452
1483426914,27sleep20-21swapper/220:52:552
3096626814,23sleep20-21swapper/221:43:072
2671826823,26sleep226714-21apt-get23:12:572
2671826823,26sleep226714-21apt-get23:12:572
2447326812,17sleep20-21swapper/221:23:002
141802685,50sleep10-21swapper/100:17:061
1340926717,32sleep113554-21sed22:28:181
1295026720,20sleep20-21swapper/219:03:142
907126624,26sleep19095-21apt22:17:431
526326617,21sleep20-21swapper/223:48:032
3048226619,21sleep20-21swapper/221:42:472
2292426619,29sleep122919-21apt-get21:17:581
2099226614,21sleep20-21swapper/219:28:092
2099226614,21sleep20-21swapper/219:28:092
113582669,15sleep20-21swapper/222:23:102
1110226624,25sleep211104-21sshd20:41:032
960826518,32sleep29644-21munin-run20:37:442
2919826515,23sleep00-21swapper/023:18:170
2919826515,23sleep00-21swapper/023:18:170
247192658,45sleep00-21swapper/019:39:340
1749826516,31sleep20-21swapper/219:17:592
1270926520,20sleep00-21swapper/000:12:450
3205526425,19sleep20-21swapper/220:03:152
2507726418,21sleep20-21swapper/219:42:472
249182647,10sleep10-21swapper/119:42:061
162842646,12sleep20-21swapper/200:22:482
1607526418,23sleep10-21swapper/122:38:071
1564926419,21sleep10-21swapper/122:37:471
1500426411,39sleep215006-21sshd19:10:042
125126413,24sleep00-21swapper/021:52:460
125126413,24sleep00-21swapper/021:52:460
2798726318,29sleep028015-21munin-run23:17:440
2798726318,29sleep028015-21munin-run23:17:440
2572626313,25sleep10-21swapper/119:43:151
239372636,45sleep00-21swapper/021:19:340
1940326316,24sleep20-21swapper/200:32:542
1940326316,24sleep20-21swapper/200:32:542
1292426317,32sleep212968-21cat22:28:062
995226211,21sleep00-21swapper/000:03:040
995226211,21sleep00-21swapper/000:03:040
625426215,20sleep20-21swapper/222:07:592
625426215,20sleep20-21swapper/222:07:592
3030126218,18sleep10-21swapper/119:58:091
233726212,21sleep00-21swapper/023:38:080
603726125,19sleep20-21swapper/222:07:472
603726125,19sleep20-21swapper/222:07:472
427826115,23sleep20-21swapper/223:43:152
3199926114,25sleep10-21swapper/121:47:471
3063426118,17sleep00-21swapper/019:58:160
2887926115,21sleep10-21swapper/123:18:141
2887926115,21sleep10-21swapper/123:18:141
1987926124,20sleep00-21swapper/021:08:030
1987926124,20sleep00-21swapper/021:08:030
1926226120,26sleep219308-21sshd19:23:042
910226015,28sleep2284-21dbus-daemon22:17:442
2563726019,20sleep00-21swapper/023:08:110
207226015,22sleep10-21swapper/123:38:011
200452605,45sleep10-21swapper/119:24:291
838825913,31sleep08424-21fschecks_count23:58:030
544925919,18sleep00-21swapper/022:03:170
544925919,18sleep00-21swapper/022:03:170
419425914,29sleep10-21swapper/123:43:141
3116425913,31sleep031195-21cut23:27:460
3116425913,31sleep031195-21cut23:27:460
2233225913,22sleep00-21swapper/022:58:050
1909525915,30sleep119096-21sshd00:32:091
1304425925,18sleep30-21swapper/319:03:153
771625816,25sleep16776-21/usr/sbin/munin20:28:181
3147625817,26sleep131473-21apt-get20:02:581
2542025815,19sleep20-21swapper/223:08:052
93825713,27sleep2966-21/usr/sbin/munin20:08:082
692925722,16sleep10-21swapper/122:08:141
692925722,16sleep10-21swapper/122:08:141
1595425715,27sleep015821-21latency_hist00:22:460
1547725717,20sleep00-21swapper/020:53:580
985625615,24sleep29852-21apt-get00:02:572
985625615,24sleep29852-21apt-get00:02:572
673025616,23sleep26793-21df23:53:042
62552566,11sleep00-21swapper/023:50:320
500925621,17sleep20-21swapper/220:22:462
376925618,21sleep10-21swapper/120:18:001
323525618,20sleep10-21swapper/121:58:051
2677025617,23sleep126792-21df23:13:001
2677025617,23sleep126792-21df23:13:001
2659125614,18sleep00-21swapper/019:47:460
2049025615,25sleep120487-21apt-get22:52:571
2049025615,25sleep120487-21apt-get22:52:571
188032569,33sleep018895-21latency_hist19:22:460
1615325614,23sleep116180-21irqstats19:13:061
756625515,18sleep00-21swapper/022:12:460
756625515,18sleep00-21swapper/022:12:460
1926925515,24sleep119315-21memory22:48:091
1363225523,18sleep10-21swapper/120:48:131
728925416,18sleep00-21swapper/020:28:130
2180925413,17sleep20-21swapper/221:13:142
2757025315,23sleep127637-21df21:33:001
2075525316,22sleep020766-21munin-run00:37:450
2075525316,22sleep020766-21munin-run00:37:450
2434825217,18sleep20-21swapper/221:22:472
2187225214,15sleep20-21swapper/222:57:472
1819825213,22sleep017721-21apt-get21:02:500
1740025214,25sleep017395-21apt-get22:42:590
1282325216,23sleep21-21systemd20:47:462
115125220,17sleep00-21swapper/023:33:150
93412516,33sleep10-21swapper/123:59:501
733825113,17sleep00-21swapper/023:53:180
272125117,17sleep00-21swapper/020:13:150
2072925114,22sleep220763-21/usr/sbin/munin22:53:062
1942825115,20sleep119423-21apt-get00:32:571
1942825115,20sleep119423-21apt-get00:32:571
1677525110,16sleep00-21swapper/000:23:230
1448825116,15sleep00-21swapper/022:33:050
323172509,14sleep00-21swapper/021:48:020
1200625014,22sleep112072-21mailstats20:43:121
565324917,17sleep10-21swapper/120:23:091
3148724910,20sleep031591-21users21:43:170
1661624815,19sleep20-21swapper/220:58:042
601724714,21sleep16035-21latency_hist22:07:461
601724714,21sleep16035-21latency_hist22:07:461
2947724716,17sleep20-21swapper/219:56:042
2947724716,17sleep20-21swapper/219:56:042
229842477,18sleep10-21swapper/119:33:171
229842477,18sleep10-21swapper/119:33:171
268642466,15sleep20-21swapper/221:28:172
2383024513,15sleep20-21swapper/223:03:042
220672448,18sleep10-21swapper/122:57:581
317692407,14sleep20-21swapper/223:28:082
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional