You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-25 - 16:23

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sat Apr 25, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
31806211365,1120sleep09437-1kworker/0:0H09:16:110
340126899,643sleep03465-21cat11:11:060
283024815,461sleep02876-21ntpq09:26:080
283024815,461sleep02876-21ntpq09:26:080
220382468116,334sleep022087-21latency_hist12:10:430
21444246513,401sleep021536-21fschecks_time10:26:020
261012441121,210sleep026245-21grep12:21:130
2309423157,16sleep00-21swapper/007:06:130
2309423157,16sleep00-21swapper/007:06:130
2816323106,293sleep013441-21snmpd12:26:470
2816323106,293sleep013441-21snmpd12:26:470
2041223088,262sleep020554-21latency_hist08:40:450
2041223088,262sleep020554-21latency_hist08:40:450
7808229028,249sleep07850-21latency_hist11:25:440
167512285143,21sleep00-21swapper/010:11:040
3109622786,235sleep031140-21ntpq10:56:070
3109622786,235sleep031140-21ntpq10:56:070
267292247118,22sleep00-21swapper/007:16:030
5318223920,204sleep15326-21uname07:51:061
13031223856,128sleep013051-21idleruntime-cro10:00:420
27673221711,156sleep027780-21fschecks_time10:46:010
17259221628,175sleep017318-21date08:30:430
48062206110,36sleep04903-21latency_hist07:50:440
24979220155,23sleep00-21swapper/010:36:120
9970216827,127sleep010117-21grep09:50:440
26300216637,84sleep00-21swapper/010:41:050
14581216232,74sleep014616-21munin-run10:05:420
14581216232,74sleep014616-21munin-run10:05:420
1789621599,135sleep017931-21ntpq08:31:070
28692215723,115sleep00-21swapper/009:06:050
2553321529,129sleep00-21swapper/008:56:030
25060215113,122sleep025112-21latency_hist08:55:430
3229721478,127sleep313441-21snmpd07:32:343
19286214718,103sleep319298-21fschecks_count12:01:133
13394214710,19sleep00-21swapper/008:16:130
13394214710,19sleep00-21swapper/008:16:130
3221521437,99sleep013441-21snmpd07:31:300
26321214358,72sleep01-21systemd07:15:430
471621429,93sleep00-21swapper/011:15:580
20257214247,56sleep00-21swapper/010:21:090
20257214247,56sleep00-21swapper/010:21:090
479214123,102sleep2637-21unixbench_multi11:01:142
1218521405,13sleep10-21swapper/108:11:171
5922139102,18sleep00-21swapper/007:36:020
1288213752,63sleep00-21swapper/011:05:440
6892213411,86sleep06927-21latency07:56:060
2145021330,36sleep221454-21unixbench_multi08:41:142
2145021330,36sleep221454-21unixbench_multi08:41:142
22922213210,107sleep023060-21unixbench_singl08:46:140
195321327,113sleep01961-21grep07:40:590
17596213015,90sleep317703-21fschecks_time08:31:023
1116521308,36sleep00-21swapper/011:35:530
31231212938,28sleep031317-21systemd-run09:15:420
31231212938,28sleep031317-21systemd-run09:15:420
3098721295,69sleep00-21swapper/007:30:370
2885521290,8sleep30-21swapper/309:06:073
12312212756,50sleep00-21swapper/008:12:500
8248212328,85sleep013441-21snmpd09:44:220
6472212344,24sleep20-21swapper/211:20:532
18656212266,44sleep018673-21munin-run12:00:410
30812211813,20sleep30-21swapper/310:56:023
30812211813,20sleep30-21swapper/310:56:023
8373211722,22sleep20-21swapper/208:01:042
8373211722,22sleep20-21swapper/208:01:042
2822621170,8sleep30-21swapper/310:46:113
8728211329,44sleep08881-21awk11:26:120
1824821138,61sleep00-21swapper/010:15:560
10239211326,52sleep010357-21sed11:31:130
2781121125,73sleep00-21swapper/009:01:450
1920211212,66sleep00-21swapper/009:22:530
866211020,69sleep3373-21/usr/sbin/munin07:36:093
21555211022,29sleep00-21swapper/007:01:050
19555211020,73sleep319624-21sed10:20:443
30213210916,78sleep230271-21/usr/sbin/munin07:26:112
2450521098,63sleep024618-21cat12:16:120
1376321097,68sleep013441-21snmpd11:42:560
5230210833,18sleep01-21systemd09:35:410
5230210833,18sleep01-21systemd09:35:410
19817210818,70sleep119950-21grep08:36:131
19817210818,70sleep119950-21grep08:36:131
29602210714,74sleep329664-21/usr/sbin/munin10:51:093
1123821068,64sleep013441-21snmpd09:52:580
990721058,58sleep00-21swapper/008:06:030
2967321045,67sleep00-21swapper/012:31:190
2967321045,67sleep00-21swapper/012:31:190
28489210416,71sleep228498-21/usr/sbin/munin09:05:592
16414210424,19sleep016423-21munin-run10:10:430
16414210424,19sleep016423-21munin-run10:10:430
14490210429,19sleep014510-21fschecks_count08:21:020
14490210429,19sleep014510-21fschecks_count08:21:020
25227210313,75sleep125271-21grep07:12:081
23634210326,13sleep00-21swapper/012:15:440
23634210326,13sleep00-21swapper/012:15:440
30448210118,69sleep130513-21sendmail_mailst09:11:111
30448210118,69sleep130513-21sendmail_mailst09:11:111
15575210132,53sleep015701-21latency_hist11:50:440
5633210018,65sleep35526-21/usr/sbin/munin09:36:013
29738210012,74sleep129753-21grep07:25:571
25110210021,48sleep225043-21apt-get07:11:412
2949729810,76sleep029559-21latency_hist07:25:440
281812988,78sleep119031-1kworker/1:0H12:27:001
281812988,78sleep119031-1kworker/1:0H12:27:001
1566029815,68sleep325884-21systemd-journal08:25:413
1566029815,68sleep325884-21systemd-journal08:25:413
377029613,70sleep13848-21sed07:46:051
2037229619,65sleep220376-21/usr/sbin/munin08:40:432
2037229619,65sleep220376-21/usr/sbin/munin08:40:432
171522968,27sleep017266-21latency_hist11:55:450
3240329319,54sleep332478-21grep09:16:423
245702937,56sleep20-21swapper/210:36:022
81792928,33sleep00-21swapper/008:00:580
81792928,33sleep00-21swapper/008:00:580
555129211,38sleep00-21swapper/009:35:580
1806329217,21sleep30-21swapper/311:56:123
1994429014,59sleep10-21swapper/110:21:021
1994429014,59sleep10-21swapper/110:21:021
532828919,45sleep25474-21latency_hist09:35:442
532828919,45sleep25474-21latency_hist09:35:442
1332728919,54sleep313392-21awk08:16:123
1332728919,54sleep313392-21awk08:16:123
714928819,47sleep17273-21timerwakeupswit11:21:161
251328828,37sleep22595-21fschecks_time09:26:022
251328828,37sleep22595-21fschecks_time09:26:022
1065428817,55sleep310679-21proc_pri09:51:113
847828712,59sleep28607-21modprobe09:45:452
1331928718,53sleep113404-21grep11:41:121
1656128618,49sleep215929-21/usr/sbin/munin08:26:132
2494428518,45sleep324975-21ps10:36:113
1018028512,57sleep210187-21/usr/sbin/munin08:06:092
827528426,31sleep10-21swapper/108:01:031
827528426,31sleep10-21swapper/108:01:031
358728315,41sleep20-21swapper/207:46:022
253422830,7sleep30-21swapper/308:55:573
204372837,65sleep013441-21snmpd12:05:380
17222837,63sleep024206-21munin-node11:06:170
17222837,63sleep024206-21munin-node11:06:170
1261528320,50sleep312684-21sed08:15:433
1451128210,51sleep114600-21grep08:21:031
1451128210,51sleep114600-21grep08:21:031
2936328114,17sleep20-21swapper/212:31:132
2936328114,17sleep20-21swapper/212:31:132
2418028118,42sleep224206-21munin-node07:10:442
2418028118,42sleep224206-21munin-node07:10:442
237922806,52sleep30-21swapper/310:31:143
2198828019,46sleep222023-21awk12:10:412
1402928019,28sleep20-21swapper/211:45:452
313712799,57sleep131376-21/usr/sbin/munin07:31:021
206232798,33sleep00-21swapper/008:40:570
206232798,33sleep00-21swapper/008:40:570
2140327824,23sleep20-21swapper/207:01:032
2336327716,25sleep20-21swapper/210:31:092
1707127725,33sleep117097-21munin-run11:55:411
858027620,29sleep20-21swapper/211:26:102
3972769,46sleep20-21swapper/207:35:562
2511427617,33sleep10-21swapper/108:55:441
2399627612,45sleep324055-21/usr/sbin/munin08:51:063
2158827610,46sleep10-21swapper/107:01:061
1618227616,27sleep30-21swapper/311:51:073
2888427517,27sleep20-21swapper/210:50:442
2456427522,30sleep10-21swapper/110:36:011
1464327521,36sleep1281-21dbus-daemon10:05:431
1464327521,36sleep1281-21dbus-daemon10:05:431
75792747,10sleep20-21swapper/207:56:442
659827411,44sleep16617-21apt-get07:55:541
301982746,11sleep00-21swapper/010:53:200
2681027415,33sleep20-21swapper/212:25:432
1426627418,43sleep314303-21grep11:45:583
2275327326,28sleep222829-21wc08:46:122
611727224,12sleep00-21swapper/007:53:190
2315127222,26sleep10-21swapper/107:06:141
2315127222,26sleep10-21swapper/107:06:141
2110627221,29sleep20-21swapper/210:25:442
2110627221,29sleep20-21swapper/210:25:442
885827116,39sleep38915-21/usr/sbin/munin09:46:073
671427113,42sleep36764-21if_eth011:21:063
439427122,31sleep34425-21idleruntime-cro11:15:413
1285927114,27sleep30-21swapper/311:41:013
1251727113,42sleep212661-21latency_hist11:40:442
2312727023,32sleep00-21swapper/010:31:030
1453627022,25sleep20-21swapper/211:46:052
1049427020,25sleep20-21swapper/211:31:152
527026918,25sleep30-21swapper/311:16:133
3104926920,23sleep30-21swapper/307:30:443
2819126919,29sleep228183-21apt-get07:20:582
2797126921,23sleep10-21swapper/110:46:061
2694426917,34sleep226949-21/usr/sbin/munin09:01:022
1957526913,39sleep119598-21/usr/sbin/munin12:01:231
1872626918,18sleep10-21swapper/112:00:421
935426822,20sleep30-21swapper/311:30:433
560126812,37sleep15625-21df_inode09:35:591
3271826810,41sleep3311-21/usr/sbin/munin11:01:113
2693826824,23sleep30-21swapper/312:25:443
2219826816,31sleep321263-21/usr/sbin/munin07:01:163
706926723,22sleep10-21swapper/109:40:511
691226717,29sleep27054-21latency_hist09:40:442
443326720,26sleep10-21swapper/109:31:101
443326720,26sleep10-21swapper/109:31:101
373326717,28sleep30-21swapper/309:30:443
373326717,28sleep30-21swapper/309:30:443
2557226714,34sleep125647-21latency08:56:041
1731926720,29sleep10-21swapper/108:30:441
1630926712,32sleep315929-21/usr/sbin/munin08:26:053
150842678,25sleep20-21swapper/210:05:532
150842678,25sleep20-21swapper/210:05:532
1485826720,30sleep314596-21apt-get10:05:453
1485826720,30sleep314596-21apt-get10:05:453
873426624,20sleep10-21swapper/109:46:051
620626617,30sleep25986-21apt-get11:20:452
1841626618,26sleep10-21swapper/110:16:011
2956026514,25sleep10-21swapper/107:25:451
2720326517,26sleep30-21swapper/312:25:533
2720326517,26sleep30-21swapper/312:25:533
2317226526,22sleep10-21swapper/110:31:041
157632657,13sleep10-21swapper/110:06:571
157632657,13sleep10-21swapper/110:06:571
1247326520,25sleep10-21swapper/111:40:431
3222626410,38sleep232260-21/usr/sbin/munin09:16:372
2718526415,22sleep20-21swapper/207:16:132
2660026418,23sleep30-21swapper/310:41:123
2424526420,24sleep30-21swapper/307:10:453
2424526420,24sleep30-21swapper/307:10:453
222326418,24sleep30-21swapper/307:41:063
2073526411,27sleep30-21swapper/312:05:573
611326318,31sleep10-21swapper/111:20:441
3196026319,24sleep20-21swapper/211:00:412
3196026319,24sleep20-21swapper/211:00:412
3141226321,19sleep20-21swapper/209:15:442
3141226321,19sleep20-21swapper/209:15:442
2939226319,21sleep10-21swapper/110:51:031
2387326322,21sleep10-21swapper/112:15:571
2134526315,29sleep221341-21apt-get10:25:562
1630526317,25sleep30-21swapper/310:09:493
1630526317,25sleep30-21swapper/310:09:493
1483926325,20sleep10-21swapper/111:46:121
972526214,19sleep30-21swapper/308:05:583
669626212,36sleep36705-21diskstats07:56:003
2686126220,20sleep30-21swapper/309:00:593
2517126217,20sleep20-21swapper/212:20:432
2237826216,29sleep122428-21/usr/sbin/munin08:46:031
187126222,17sleep30-21swapper/311:06:203
187126222,17sleep30-21swapper/311:06:203
1792926217,26sleep117974-21modprobe10:15:411
141326218,21sleep10-21swapper/111:05:451
141326218,21sleep10-21swapper/111:05:451
1125526218,29sleep311326-21fschecks_time11:36:003
111432625,12sleep10-21swapper/109:51:461
87226016,24sleep30-21swapper/309:20:543
841526018,24sleep10-21swapper/111:26:051
289026020,17sleep20-21swapper/211:10:432
289026020,17sleep20-21swapper/211:10:432
2878626019,18sleep10-21swapper/109:06:061
2732726014,28sleep20-21swapper/210:45:432
2718126016,26sleep226693-21apt-get12:25:522
2718126016,26sleep226693-21apt-get12:25:522
2343526016,25sleep323467-21awk08:50:413
2268826019,18sleep10-21swapper/110:30:441
1002426012,31sleep310048-21ntp_states11:31:083
437625911,33sleep34424-21ntp_states09:31:083
437625911,33sleep34424-21ntp_states09:31:083
2948425919,23sleep329495-21unin-run07:25:433
2873325911,31sleep30-21swapper/312:30:543
2873325911,31sleep30-21swapper/312:30:543
2680925919,19sleep10-21swapper/107:16:041
232325919,20sleep10-21swapper/107:41:081
2130425915,19sleep10-21swapper/108:41:141
2130425915,19sleep10-21swapper/108:41:141
2081425912,31sleep120818-21diskstats12:05:591
2062125917,25sleep320618-21apt-get08:40:563
2062125917,25sleep320618-21apt-get08:40:563
1746925915,27sleep217473-21diskstats11:55:592
1332325910,23sleep20-21swapper/210:00:562
1332325910,23sleep20-21swapper/210:00:562
1256025920,20sleep20-21swapper/208:15:422
643225813,23sleep30-21swapper/307:55:443
398625814,25sleep24033-21users11:11:162
3023625816,24sleep330325-21/usr/sbin/munin09:11:053
3023625816,24sleep330325-21/usr/sbin/munin09:11:053
2789425813,31sleep327938-21systemd-run07:20:423
2286225822,21sleep10-21swapper/112:11:131
2286225822,21sleep10-21swapper/112:11:131
2143125819,21sleep30-21swapper/310:26:013
1721825814,21sleep30-21swapper/310:11:153
1661125815,24sleep216657-21/usr/sbin/munin10:10:532
1590025815,28sleep215915-21fschecks_count11:51:022
2742425717,20sleep10-21swapper/109:01:131
39625614,25sleep1391-21apt-get07:35:551
3129825611,31sleep131373-21find10:56:121
3129825611,31sleep131373-21find10:56:121
2672125617,24sleep126750-21munin-run12:25:421
232292566,38sleep20-21swapper/212:11:382
232292566,38sleep20-21swapper/212:11:382
2040125619,18sleep30-21swapper/310:21:123
2040125619,18sleep30-21swapper/310:21:123
1907425615,24sleep319070-21apt-get08:35:553
1907425615,24sleep319070-21apt-get08:35:553
1807425623,15sleep10-21swapper/108:31:121
1185325617,19sleep10-21swapper/109:56:021
107322565,9sleep10-21swapper/111:34:091
3126725513,21sleep231279-21apt-get07:30:552
2912225516,17sleep20-21swapper/210:50:562
1990025522,19sleep20-21swapper/212:01:402
504825413,25sleep25042-21apt-get07:50:592
260932545,36sleep30-21swapper/307:12:473
2237725415,17sleep30-21swapper/312:11:023
2237725415,17sleep30-21swapper/312:11:023
154325420,17sleep20-21swapper/209:21:182
147972547,19sleep20-21swapper/208:21:082
147972547,19sleep20-21swapper/208:21:082
1317125415,15sleep20-21swapper/208:16:072
1317125415,15sleep20-21swapper/208:16:072
554825312,24sleep25544-21apt-get09:35:562
447325315,15sleep10-21swapper/111:15:431
2530325321,16sleep20-21swapper/208:55:572
1331225314,24sleep113310-21ntp_states08:16:101
1331225314,24sleep113310-21ntp_states08:16:101
2602925213,22sleep126096-21df10:40:571
2394525214,23sleep323954-21/usr/sbin/munin12:15:593
1750025212,23sleep217493-21apt-get08:30:572
1308225214,21sleep20-21swapper/210:00:442
2929125112,23sleep129362-21sensors_temp12:31:121
2929125112,23sleep129362-21sensors_temp12:31:121
269625110,16sleep10-21swapper/109:26:051
269625110,16sleep10-21swapper/109:26:051
105125110,19sleep10-21swapper/111:05:411
1026225112,21sleep110332-21sensors_temp08:06:121
366425017,17sleep30-21swapper/307:46:033
23692508,16sleep20-21swapper/207:41:092
131525012,22sleep11337-21ntp_states09:21:131
789524913,19sleep37994-21users09:41:143
1183724912,20sleep311846-21df_inode09:56:003
875224815,16sleep30-21swapper/308:01:143
875224815,16sleep30-21swapper/308:01:143
2394624811,22sleep223995-21irqstats08:51:052
194062476,30sleep20-21swapper/210:19:262
318242448,18sleep10-21swapper/109:16:241
101942397,16sleep20-21swapper/209:50:582
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional