You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-10-20 - 00:34

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Mon Oct 19, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
13572232719,2296sleep11121-21kworker/1:110:40:171
13572232719,2296sleep11121-21kworker/1:110:40:171
2029427506,689sleep020335-21ntpq09:56:180
2029427506,689sleep020335-21ntpq09:56:180
16559265916,627sleep216575-21/usr/sbin/munin08:01:162
11363248553,405sleep011378-21latency_hist07:45:550
11363248553,405sleep011378-21latency_hist07:45:550
16374236512,283sleep016517-21unixbench_multi11:26:240
10737236310,339sleep110775-21unin-run11:10:541
18027235588,250sleep018030-21cron09:50:530
18027235588,250sleep018030-21cron09:50:530
368423506,334sleep0367-21snmpd12:30:290
31925233217,296sleep032068-21grep07:06:250
1932122679,243sleep019401-21tr11:36:210
1932122679,243sleep019401-21tr11:36:210
10802261119,72sleep00-21swapper/007:11:180
12697224136,182sleep012786-21fschecks_time11:16:140
21444223714,21sleep00-21swapper/008:16:240
21444223714,21sleep00-21swapper/008:16:240
6896223531,103sleep00-21swapper/009:12:530
8604222640,168sleep08698-21fschecks_time07:36:140
8604222640,168sleep08698-21fschecks_time07:36:140
1471622235,12sleep00-21swapper/009:38:050
4801221711,186sleep34859-21/usr/sbin/munin09:06:223
21852221411,30sleep021898-21ntp_offset10:01:190
21852221411,30sleep021898-21ntp_offset10:01:190
29860220916,143sleep029896-21ntp_kernel_pll_10:26:180
24380220718,69sleep00-21swapper/010:10:560
16219220511,180sleep216293-21mailstats11:26:222
14613220511,177sleep014646-21ps11:21:230
2857220314,169sleep02947-21fschecks_time09:01:140
4879218716,158sleep24967-21fschecks_time10:51:122
24967218712,160sleep224976-21vmstat08:26:282
24967218712,160sleep224976-21vmstat08:26:282
11851217715,22sleep20-21swapper/207:46:172
4593217011,22sleep10-21swapper/109:06:161
446021677,47sleep00-21swapper/012:31:220
15473216617,128sleep115618-21wc11:25:561
14933216219,123sleep214950-21awk09:40:532
2973321618,62sleep029879-21uptime08:41:280
2973321618,62sleep029879-21uptime08:41:280
966021577,135sleep09784-21ls09:21:250
24717215513,122sleep124767-21sendmail11:55:531
25245215435,102sleep025264-21apt-get11:56:040
5045215217,109sleep25182-21unixbench_singl09:06:252
2950214951,64sleep02986-21awk10:45:530
245321488,19sleep00-21swapper/007:16:150
7026214624,20sleep00-21swapper/007:31:130
15421214611,118sleep315576-21unixbench_multi07:56:283
15421214611,118sleep315576-21unixbench_multi07:56:283
128821465,109sleep00-21swapper/008:56:140
3102421437,99sleep00-21swapper/012:13:330
621214250,26sleep00-21swapper/012:20:560
7677214026,101sleep07715-21date11:00:550
28348214033,25sleep028372-21ntp_states10:21:220
24945214036,66sleep025131-21apt-config11:55:560
32594213830,22sleep032588-21apt-get10:36:080
32594213830,22sleep032588-21apt-get10:36:080
20734213876,22sleep020868-21latency_hist08:15:550
1072621389,29sleep0231-21systemd-journal11:10:530
32277213523,89sleep232282-21modprobe10:35:532
32277213523,89sleep232282-21modprobe10:35:532
27396213519,92sleep126890-21apt-get08:36:041
27396213519,92sleep126890-21apt-get08:36:041
25169213527,90sleep00-21swapper/010:11:320
26933213231,21sleep026970-21munin-run08:35:550
26343212914,97sleep00-21swapper/010:16:050
21422212641,43sleep20-21swapper/210:01:042
21422212641,43sleep20-21swapper/210:01:042
1449821229,24sleep014612-21latency_hist07:55:560
2054621219,97sleep020609-21if_err_eth011:41:140
1228521208,100sleep012354-21systemd-run11:15:540
24339211811,67sleep011-21ksoftirqd/008:26:170
24339211811,67sleep011-21ksoftirqd/008:26:170
31019211713,86sleep331091-21df10:31:103
31019211713,86sleep331091-21df10:31:103
7893211615,87sleep17901-21/usr/sbin/munin11:01:041
7515211620,74sleep37589-21switchtime07:31:243
545021168,93sleep15574-21sed10:51:231
28022211526,62sleep00-21swapper/008:36:350
28022211526,62sleep00-21swapper/008:36:350
1721421148,54sleep00-21swapper/011:31:040
1721421148,54sleep00-21swapper/011:31:040
2822021139,59sleep00-21swapper/012:05:580
1832211353,41sleep00-21swapper/012:21:570
8753211113,81sleep28787-21/usr/sbin/munin07:36:162
8753211113,81sleep28787-21/usr/sbin/munin07:36:162
3980211115,38sleep10-21swapper/112:31:061
12684211116,80sleep312695-21awk11:16:133
23081210926,65sleep123098-21fschecks_count10:06:121
12919210948,23sleep013031-21latency_hist07:50:560
12714210720,66sleep212818-21wc09:31:252
15692210618,74sleep315754-21uname09:41:233
15692210618,74sleep315754-21uname09:41:233
1014210312,77sleep01165-21latency_hist08:55:570
22292210211,33sleep022333-21ntpq11:46:190
22292210211,33sleep022333-21ntpq11:46:190
21714210113,72sleep221858-21date11:45:562
3101621008,52sleep00-21swapper/010:31:070
3101621008,52sleep00-21swapper/010:31:070
15906210025,51sleep20-21swapper/209:41:262
15906210025,51sleep20-21swapper/209:41:262
923229913,67sleep09339-21latency_hist11:05:560
2990329934,37sleep329922-21apt-get12:11:053
32362979,53sleep00-21swapper/010:46:050
72229621,58sleep30-21swapper/310:36:273
72229621,58sleep30-21swapper/310:36:273
318762969,27sleep032032-21latency_hist08:50:570
310062969,65sleep00-21swapper/007:05:570
283802968,49sleep00-21swapper/012:06:060
195772967,51sleep00-21swapper/009:55:540
195772967,51sleep00-21swapper/009:55:540
38412956,13sleep00-21swapper/007:21:070
105282956,49sleep00-21swapper/009:26:060
190992948,49sleep00-21swapper/008:10:550
349929314,62sleep13511-21kernelversion10:46:141
2773129311,68sleep227743-21vmstat12:01:272
181122937,11sleep30-21swapper/308:06:173
221329210,66sleep22246-21latency_hist12:25:542
147422926,48sleep00-21swapper/007:56:130
147422926,48sleep00-21swapper/007:56:130
486429019,50sleep00-21swapper/009:06:230
314129015,51sleep13185-21ntpq09:01:191
76528911,64sleep1767-21sshd08:54:171
224922886,45sleep00-21swapper/008:21:040
224922886,45sleep00-21swapper/008:21:040
1653428825,18sleep00-21swapper/009:45:560
1653428825,18sleep00-21swapper/009:45:560
535928611,14sleep00-21swapper/010:51:220
3139228612,20sleep10-21swapper/107:06:141
2608028615,51sleep125587-21/usr/sbin/munin08:31:231
63912856,45sleep00-21swapper/007:27:330
2481328516,45sleep124965-21head08:26:271
2481328516,45sleep124965-21head08:26:271
2355528522,40sleep323647-21fschecks_time11:51:143
1676228524,49sleep016806-21processes08:01:220
159562847,19sleep30-21swapper/311:26:143
323042837,57sleep20-21swapper/212:16:302
2922728224,31sleep20-21swapper/210:25:562
2222928223,43sleep122253-21munin-run08:20:531
2222928223,43sleep122253-21munin-run08:20:531
526228127,35sleep35285-21apt-get12:35:533
41912800,6sleep20-21swapper/207:21:162
2599728024,32sleep20-21swapper/211:56:512
2988327924,23sleep30-21swapper/307:01:163
2343527922,27sleep20-21swapper/211:51:052
209227913,52sleep22133-21tr10:41:192
205627919,42sleep22145-21ls07:15:562
703827819,41sleep17140-21cut07:31:141
2637127812,52sleep126438-21df10:16:101
1237227815,32sleep20-21swapper/211:15:562
2237527614,44sleep322390-21/usr/sbin/munin11:46:223
2237527614,44sleep322390-21/usr/sbin/munin11:46:223
351227515,27sleep20-21swapper/210:46:152
309227522,32sleep33205-21timerwakeupswit12:26:243
3015827527,25sleep20-21swapper/207:01:232
421227413,48sleep3364-21ntpd07:21:183
2657827420,26sleep20-21swapper/210:16:152
1441727416,36sleep114423-21modprobe07:55:531
2800227315,33sleep128018-21fschecks_count10:21:141
2429727324,23sleep30-21swapper/310:10:553
594527210,38sleep36009-21wc07:26:233
1610627214,20sleep30-21swapper/308:00:563
130362727,11sleep00-21swapper/009:31:430
74302715,12sleep00-21swapper/009:16:040
284627112,13sleep30-21swapper/309:01:133
2382727117,30sleep20-21swapper/208:25:552
2382727117,30sleep20-21swapper/208:25:552
2165727124,31sleep321802-21sed08:16:273
2165727124,31sleep321802-21sed08:16:273
2015827112,16sleep20-21swapper/211:40:552
1807827119,27sleep20-21swapper/209:50:552
1807827119,27sleep20-21swapper/209:50:552
136127120,32sleep3683-21/usr/sbin/munin07:11:253
813727016,33sleep18278-21sed09:16:261
2962527023,31sleep129674-21cpuspeed07:01:101
2874827011,11sleep30-21swapper/312:06:173
275827012,41sleep32773-21/usr/sbin/munin07:16:223
256527018,26sleep20-21swapper/207:16:172
644626921,26sleep10-21swapper/110:56:121
3120426918,32sleep231232-21munin-run12:15:532
2687126916,34sleep326894-21munin-run08:35:533
119726911,26sleep10-21swapper/107:11:231
812726815,35sleep38185-21/usr/sbin/munin11:01:153
792426817,24sleep20-21swapper/209:16:232
2205326820,21sleep10-21swapper/110:01:241
2205326820,21sleep10-21swapper/110:01:241
1986726813,39sleep219875-21/usr/sbin/munin09:56:042
1986726813,39sleep219875-21/usr/sbin/munin09:56:042
147226826,19sleep30-21swapper/312:21:233
1109826820,30sleep311159-21/usr/sbin/munin11:11:123
76312677,14sleep30-21swapper/311:00:543
1501026720,31sleep215051-21ls07:56:192
1501026720,31sleep215051-21ls07:56:192
949926618,20sleep20-21swapper/209:21:232
690826612,38sleep36909-21sendmail-mta09:12:563
360126610,12sleep10-21swapper/107:20:561
2345926618,36sleep223496-21ps10:06:212
1313126621,28sleep213149-21apt07:51:062
1308926616,23sleep10-21swapper/111:16:231
1077626616,37sleep210840-21latency_hist11:10:552
395126516,23sleep30-21swapper/310:46:253
3077126513,38sleep130878-21latency_hist10:30:561
300226423,19sleep30-21swapper/310:45:563
2953526419,24sleep20-21swapper/208:41:262
2953526419,24sleep20-21swapper/208:41:262
2613026420,24sleep30-21swapper/310:15:553
206226419,19sleep10-21swapper/110:41:181
1740826415,23sleep10-21swapper/111:31:131
1740826415,23sleep10-21swapper/111:31:131
1481326415,34sleep114819-21df_inode07:56:141
1481326415,34sleep114819-21df_inode07:56:141
985626322,19sleep30-21swapper/307:40:573
411126321,23sleep10-21swapper/107:21:161
3251626317,29sleep332530-21munin-node07:10:563
2598426322,25sleep226029-21awk08:31:202
1802426318,22sleep10-21swapper/108:06:151
1750726320,25sleep30-21swapper/311:31:153
1750726320,25sleep30-21swapper/311:31:153
1191126319,21sleep30-21swapper/309:30:573
1095326312,15sleep20-21swapper/209:26:202
1878726218,25sleep318839-21processes09:51:243
1878726218,25sleep318839-21processes09:51:243
1685726215,29sleep116893-21/usr/sbin/munin09:46:141
1685726215,29sleep116893-21/usr/sbin/munin09:46:141
3204026117,21sleep30-21swapper/312:16:233
300372618,11sleep30-21swapper/308:43:213
300372618,11sleep30-21swapper/308:43:213
298982617,16sleep10-21swapper/108:41:331
298982617,16sleep10-21swapper/108:41:331
2065326117,24sleep220677-21munin-run08:15:532
63526015,23sleep20-21swapper/210:36:262
63526015,23sleep20-21swapper/210:36:262
628226019,22sleep20-21swapper/207:26:262
497226016,23sleep30-21swapper/310:51:133
2839526023,20sleep30-21swapper/310:21:253
2740826021,17sleep30-21swapper/312:01:243
272022606,13sleep10-21swapper/112:01:181
1388226019,25sleep313935-21/usr/sbin/munin09:36:143
1365026015,24sleep10-21swapper/109:36:041
1281926018,19sleep30-21swapper/309:31:263
1169726020,21sleep30-21swapper/307:46:143
750225912,26sleep20-21swapper/210:59:162
575525924,20sleep10-21swapper/107:26:171
1330725918,22sleep30-21swapper/307:51:153
101542598,12sleep10-21swapper/109:24:291
1006325918,20sleep20-21swapper/207:41:082
1006325918,20sleep20-21swapper/207:41:082
3262025815,24sleep132674-21users12:16:331
2883925816,27sleep228846-21ntpq12:06:202
1918225819,21sleep30-21swapper/311:36:163
1918225819,21sleep30-21swapper/311:36:163
1840525814,20sleep20-21swapper/209:51:152
1840525814,20sleep20-21swapper/209:51:152
126772589,33sleep27431-21kworker/2:211:16:122
1101625820,22sleep10-21swapper/109:26:231
1024525822,20sleep30-21swapper/307:41:133
1024525822,20sleep30-21swapper/307:41:133
647925715,17sleep10-21swapper/109:11:251
2781525713,27sleep227843-21nfs4_client08:36:292
2781525713,27sleep227843-21nfs4_client08:36:292
2767825718,18sleep30-21swapper/310:20:563
1927525716,19sleep20-21swapper/211:36:192
1876825717,24sleep118779-21ntpq09:51:221
1876825717,24sleep118779-21ntpq09:51:221
1343825721,16sleep10-21swapper/109:35:561
633325615,25sleep26341-21awk09:11:222
259952566,16sleep20-21swapper/210:15:102
2486325612,23sleep10-21swapper/110:11:131
2191325617,22sleep20-21swapper/211:46:082
2191325617,22sleep20-21swapper/211:46:082
1717425613,25sleep317219-21/usr/sbin/munin09:46:203
1717425613,25sleep317219-21/usr/sbin/munin09:46:203
838925517,24sleep28454-21awk11:01:222
3207725514,24sleep332074-21apt-get08:51:083
3184125520,18sleep30-21swapper/308:50:563
264025515,20sleep10-21swapper/112:26:141
2614625512,16sleep30-21swapper/311:56:523
1605925517,20sleep10-21swapper/111:26:161
128352557,39sleep10-21swapper/107:50:281
926925418,17sleep30-21swapper/309:21:163
3052425413,24sleep130518-21apt-get08:46:091
3000225421,16sleep10-21swapper/112:11:121
276125412,25sleep22757-21apt-get09:01:082
2430825413,24sleep324337-21irqstats08:26:163
2430825413,24sleep324337-21irqstats08:26:163
226242545,10sleep30-21swapper/310:04:493
226242545,10sleep30-21swapper/310:04:493
21392546,36sleep20-21swapper/208:56:402
1996625412,25sleep120062-21sed08:11:241
1881325412,25sleep118807-21apt-get11:36:061
1881325412,25sleep118807-21apt-get11:36:061
1051825414,25sleep110525-21ntp_states07:41:201
1051825414,25sleep110525-21ntp_states07:41:201
854425313,24sleep18590-21df07:36:111
854425313,24sleep18590-21df07:36:111
2322325314,17sleep10-21swapper/111:50:561
2322325314,17sleep10-21swapper/111:50:561
2272525314,24sleep322775-21if_eth008:21:163
2272525314,24sleep322775-21if_eth008:21:163
2033825311,29sleep320357-21apt-get11:41:053
3254425212,20sleep20-21swapper/208:51:232
301225211,25sleep23091-21sed12:26:232
2033725216,21sleep120348-21ntp_states09:56:191
2033725216,21sleep120348-21ntp_states09:56:191
1423325215,22sleep30-21swapper/311:21:143
120825213,23sleep31204-21apt-get08:56:093
1123225213,23sleep111281-21irqstats11:11:141
103882525,10sleep30-21swapper/311:06:323
797325115,20sleep38058-21mailstats09:16:243
2970725115,21sleep129847-21latency_hist12:10:551
2946425112,23sleep329531-21df10:26:093
1826525111,17sleep20-21swapper/208:06:202
2943525012,23sleep229455-21apt-get10:26:042
2591725014,18sleep10-21swapper/111:56:501
2072025012,19sleep30-21swapper/308:15:543
1441625014,20sleep20-21swapper/211:21:172
2226224911,23sleep222279-21munin-run08:20:542
2226224911,23sleep222279-21munin-run08:20:542
1731424915,18sleep20-21swapper/211:31:102
1731424915,18sleep20-21swapper/211:31:102
1636124913,20sleep116367-21df_inode08:01:101
3128924813,14sleep10-21swapper/110:31:161
3128924813,14sleep10-21swapper/110:31:161
198952488,15sleep30-21swapper/309:56:093
198952488,15sleep30-21swapper/309:56:093
104024814,16sleep10-21swapper/112:21:131
405024711,21sleep24077-21df12:31:112
166824714,17sleep10-21swapper/108:56:231
3128624611,20sleep231292-21diskstats07:06:112
69392456,15sleep20-21swapper/207:31:072
2167324510,14sleep10-21swapper/111:45:551
2249324410,13sleep10-21swapper/108:21:051
2249324410,13sleep10-21swapper/108:21:051
313412428,16sleep20-21swapper/208:46:262
156792417,13sleep10-21swapper/109:41:211
156792417,13sleep10-21swapper/109:41:211
878824012,15sleep30-21swapper/307:36:173
274192409,17sleep30-21swapper/308:36:063
274192409,17sleep30-21swapper/308:36:063
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional