You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-12 - 04:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Sun Apr 12, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
131672737615,7348sleep026801-21kworker/0:020:42:230
9055263811,596sleep124961-1kworker/1:0H23:56:311
18567246811,414sleep118673-21fschecks_time21:01:501
30938237113,342sleep231023-21mailstats19:56:582
23163234842,289sleep223156-21apt-get21:16:452
159902300109,34sleep10-21swapper/119:11:341
21453228777,187sleep121485-21sendmail00:36:281
199482276145,81sleep10-21swapper/122:46:451
16952224456,173sleep116976-21fschecks_count22:37:121
10834224415,193sleep110844-21acpi00:01:401
32154223463,114sleep10-21swapper/123:26:301
8820222250,109sleep10-21swapper/120:31:291
22195221753,122sleep122327-21timerwakeupswit21:12:041
11909220947,16sleep10-21swapper/122:21:501
16012220815,180sleep216136-21sed20:52:022
14957219911,174sleep00-21swapper/020:50:220
746921989,168sleep17612-21grep22:07:041
746921989,168sleep17612-21grep22:07:041
18622219759,25sleep10-21swapper/122:41:521
5786219020,134sleep15868-21switchtime22:02:431
21895218715,116sleep121942-21ntp_offset22:51:541
10065216921,119sleep00-21swapper/022:16:400
2708821679,16sleep10-21swapper/119:46:311
2942121609,114sleep129492-21swap19:52:041
6310215511,102sleep16376-21idleruntime23:46:491
6310215511,102sleep16376-21idleruntime23:46:491
31569215410,55sleep10-21swapper/121:41:561
31569215410,55sleep10-21swapper/121:41:561
27093214934,94sleep10-21swapper/121:27:021
27538214862,28sleep10-21swapper/119:46:511
12732214712,119sleep212811-21awk20:42:002
12331214661,70sleep112437-21fschecks_time20:41:511
15234214123,99sleep015256-21latency_hist00:16:290
1781214028,17sleep10-21swapper/121:51:411
30819213915,23sleep10-21swapper/119:56:531
19094213613,108sleep319104-21ntpq00:26:573
12783213449,61sleep10-21swapper/100:06:531
22174213219,91sleep322331-21unixbench_multi22:52:003
9112212746,20sleep10-21swapper/120:31:451
17465212678,17sleep10-21swapper/120:57:011
1020521260,32sleep210209-21fschecks_count22:16:572
1577212512,98sleep11608-21fschecks_count23:31:471
4463212238,20sleep10-21swapper/120:16:431
1581921219,55sleep10-21swapper/100:16:501
32716212063,13sleep10-21swapper/121:46:441
32716212063,13sleep10-21swapper/121:46:441
28012212021,81sleep327529-21apt-get21:31:403
11356212032,20sleep10-21swapper/122:21:291
3019421188,17sleep10-21swapper/121:36:591
3019421188,17sleep10-21swapper/121:36:591
2583521185,11sleep10-21swapper/121:24:481
125621159,86sleep10-21swapper/123:31:311
2011221147,93sleep120188-21cat19:22:001
352321137,15sleep10-21swapper/123:36:561
16769211313,84sleep316266-21apt-get22:36:373
1187221136,12sleep10-21swapper/120:40:341
15911211213,75sleep20-21swapper/200:16:512
15336211116,61sleep114241-21cyclictest20:51:441
1801521105,12sleep10-21swapper/100:22:071
1801521105,12sleep10-21swapper/100:22:071
8360210925,61sleep38397-21fschecks_count22:11:503
5637210912,84sleep213441-21snmpd23:43:272
4616210916,59sleep114241-21cyclictest23:41:451
1493321095,47sleep10-21swapper/100:12:461
15111210721,69sleep215120-21fschecks_count22:31:522
1319421078,63sleep10-21swapper/122:26:461
7502210614,76sleep07450-21latency_hist23:51:330
7502210614,76sleep07450-21latency_hist23:51:330
28456210619,58sleep10-21swapper/123:12:011
2410621065,46sleep10-21swapper/122:58:321
1958621017,59sleep113441-21snmpd00:27:241
30881210013,21sleep30-21swapper/319:56:553
20399210010,31sleep120443-21ntp_kernel_err21:06:551
15674210016,67sleep315819-21sed22:32:053
99872997,77sleep110117-21grep23:57:031
62802997,57sleep16298-21grep20:21:521
39972999,27sleep14090-21systemd-run20:16:311
39972999,27sleep14090-21systemd-run20:16:311
3209229812,68sleep132179-21fschecks_time20:01:491
228352989,61sleep10-21swapper/119:31:491
228352989,61sleep10-21swapper/119:31:491
2512929613,40sleep10-21swapper/123:01:591
197792967,12sleep00-21swapper/022:46:320
136729612,30sleep31370-21irqstats20:06:523
45172958,58sleep00-21swapper/020:17:230
345329516,21sleep10-21swapper/121:56:461
3240829514,66sleep332453-21ntp_offset20:01:553
23902956,51sleep10-21swapper/120:11:301
46129415,59sleep3511-21/usr/sbin/munin21:46:503
46129415,59sleep3511-21/usr/sbin/munin21:46:503
2897329414,56sleep328852-21/usr/sbin/munin19:51:543
218622946,12sleep00-21swapper/022:51:530
1144929425,40sleep011536-21threads00:01:590
241452937,53sleep1357-21ntpd21:17:571
2018929316,64sleep220318-21sed19:22:012
2204129214,60sleep222063-21/usr/sbin/munin21:12:002
2470629127,35sleep30-21swapper/321:21:433
280052906,45sleep1191-21jbd2/mmcblk2p2-21:31:341
1238228921,26sleep00-21swapper/000:06:390
1377228821,50sleep213787-21awk19:02:022
1296128830,30sleep10-21swapper/119:01:491
1289828811,64sleep25304-21kworker/u8:122:26:272
2093428718,56sleep221083-21unixbench_singl00:32:012
1718728717,46sleep216888-21/usr/sbin/munin20:56:542
93872866,45sleep10-21swapper/122:14:311
2588428621,44sleep225888-21modprobe23:06:292
3001128512,53sleep030049-21/usr/sbin/munin21:36:540
3001128512,53sleep030049-21/usr/sbin/munin21:36:540
1972828525,40sleep019741-21modprobe21:06:290
191202857,30sleep113441-21snmpd19:19:331
827328215,46sleep37541-21/usr/sbin/munin20:27:033
652828228,33sleep00-21swapper/020:21:590
244382820,8sleep30-21swapper/321:21:293
2000528220,31sleep00-21swapper/021:06:410
1495428249,16sleep10-21swapper/119:07:011
817028116,44sleep28270-21timerandwakeup20:27:022
2480328120,42sleep224896-21sort21:21:502
1504528114,51sleep2222-21systemd-journal20:51:292
1952228017,31sleep00-21swapper/022:45:230
1881128018,31sleep20-21swapper/222:41:582
149862798,14sleep10-21swapper/122:31:431
2113327813,51sleep20-21swapper/221:09:302
1777327824,33sleep317895-21timerwakeupswit00:22:013
1777327824,33sleep317895-21timerwakeupswit00:22:013
1419027820,39sleep314209-21munin-run19:06:313
2119227740,19sleep20-21swapper/219:26:492
1710327724,34sleep317244-21unixbench_multi19:12:003
1577427714,49sleep315881-21sed19:11:323
2431927620,42sleep024412-21grep19:36:490
2108027612,47sleep121099-21apt-get19:26:401
1057227612,45sleep31-21systemd00:01:293
250732756,13sleep10-21swapper/119:37:021
1313727530,28sleep013151-21grep19:01:520
939327419,17sleep00-21swapper/023:56:500
896827418,37sleep39033-21modprobe23:56:293
1727727217,38sleep317285-21needreboot20:56:553
1669327213,45sleep216782-21cat19:11:512
587127119,30sleep20-21swapper/222:02:442
400727125,26sleep30-21swapper/321:57:003
374827117,24sleep00-21swapper/021:56:530
2063427123,28sleep30-21swapper/321:07:013
159327116,32sleep21076-21/usr/sbin/munin20:06:592
1545727111,50sleep00-21swapper/019:08:050
409327020,23sleep20-21swapper/220:16:322
409327020,23sleep20-21swapper/220:16:322
1671227021,25sleep00-21swapper/020:56:320
1049727014,28sleep30-21swapper/320:36:323
2967426920,11sleep10-21swapper/123:16:531
2967426920,11sleep10-21swapper/123:16:531
2364526919,33sleep223722-21sort22:57:002
2307226913,34sleep00-21swapper/022:56:450
2124826924,27sleep20-21swapper/222:51:302
539826818,32sleep05456-21sort22:02:340
314326815,32sleep20-21swapper/221:56:302
2739726829,10sleep10-21swapper/123:11:041
2739726829,10sleep10-21swapper/123:11:041
262272686,49sleep00-21swapper/021:26:360
1932126818,24sleep20-21swapper/219:21:322
1730726817,25sleep20-21swapper/222:37:252
272726713,22sleep30-21swapper/320:11:493
272726713,22sleep30-21swapper/320:11:493
1852026716,37sleep218546-21/usr/sbin/munin21:01:472
1487526728,28sleep10-21swapper/120:49:181
903726616,33sleep09028-40mandb23:56:300
392826627,21sleep20-21swapper/221:56:592
320852665,11sleep00-21swapper/021:42:070
320852665,11sleep00-21swapper/021:42:070
1660626616,34sleep00-21swapper/019:11:490
1414926622,28sleep30-21swapper/320:46:543
888226515,32sleep28984-21/usr/sbin/munin22:12:012
647026515,29sleep20-21swapper/220:21:572
2814226513,38sleep228240-21users19:47:122
275226518,22sleep30-21swapper/323:36:303
224292659,28sleep00-21swapper/019:31:320
1356126516,28sleep20-21swapper/222:26:552
793426420,29sleep00-21swapper/020:26:560
643626418,25sleep30-21swapper/322:06:293
572526413,24sleep30-21swapper/320:21:303
35672646,11sleep30-21swapper/323:36:583
2833426415,27sleep00-21swapper/023:12:000
1858126421,21sleep00-21swapper/019:16:580
1203326421,21sleep30-21swapper/320:41:333
1136126419,18sleep30-21swapper/300:01:583
840326318,22sleep00-21swapper/022:11:510
2978726318,29sleep229849-21/usr/sbin/munin23:16:582
2978726318,29sleep229849-21/usr/sbin/munin23:16:582
2262726313,33sleep013441-21snmpd19:31:400
2262726313,33sleep013441-21snmpd19:31:400
970226214,37sleep29736-21ntp_kernel_pll_23:56:562
2842526219,23sleep20-21swapper/221:32:032
2678726221,22sleep00-21swapper/019:43:190
210862620,8sleep30-21swapper/300:32:013
501126116,20sleep30-21swapper/323:41:553
3056526115,22sleep00-21swapper/023:21:290
3056526115,22sleep00-21swapper/023:21:290
282926117,22sleep00-21swapper/023:36:310
2733026112,23sleep00-21swapper/019:46:460
2157926113,25sleep00-21swapper/021:11:460
1422926114,30sleep314324-21/usr/sbin/munin00:11:513
1122726118,23sleep20-21swapper/200:01:532
76972606,42sleep30-21swapper/322:07:503
76972606,42sleep30-21swapper/322:07:503
677026016,24sleep30-21swapper/323:47:003
677026016,24sleep30-21swapper/323:47:003
36592607,40sleep00-21swapper/020:13:230
36592607,40sleep00-21swapper/020:13:230
300412604,45sleep00-21swapper/019:55:180
2957426019,24sleep30-21swapper/323:16:513
2957426019,24sleep30-21swapper/323:16:513
2930326018,23sleep029333-21munin-run21:36:300
2472426018,18sleep00-21swapper/023:01:500
2340026017,16sleep00-21swapper/021:16:510
2001926022,16sleep30-21swapper/319:21:583
1781326012,34sleep017825-21vmstat20:57:040
3211025921,22sleep30-21swapper/323:26:283
254702596,44sleep30-21swapper/319:41:163
252552596,10sleep20-21swapper/219:38:282
1340025919,24sleep30-21swapper/322:26:523
321562586,10sleep30-21swapper/321:43:033
321562586,10sleep30-21swapper/321:43:033
2635325826,15sleep20-21swapper/219:41:592
2283325819,17sleep20-21swapper/222:56:312
224762586,12sleep30-21swapper/321:12:063
1844825811,31sleep218601-21latency_hist00:26:322
1504125818,22sleep20-21swapper/219:07:022
1487525811,21sleep20-21swapper/200:12:022
1240625813,27sleep312401-21apt-get00:06:423
672925714,27sleep06794-21ls22:06:460
672925714,27sleep06794-21ls22:06:460
62625714,21sleep20-21swapper/221:46:532
62625714,21sleep20-21swapper/221:46:532
44625719,16sleep00-21swapper/023:26:580
438025714,18sleep00-21swapper/023:41:310
3267625712,30sleep2300-21memory23:26:522
3110525713,29sleep031221-21timerwakeupswit19:57:000
3029325715,24sleep20-21swapper/221:37:002
3029325715,24sleep20-21swapper/221:37:002
2617525712,29sleep226242-21df23:06:452
2617525712,29sleep226242-21df23:06:452
258712575,11sleep00-21swapper/021:25:150
176925715,24sleep01893-21sed20:07:010
1728825714,25sleep017294-21ntp_states22:37:230
1194425715,25sleep011978-21munin-run20:41:300
1140625721,19sleep30-21swapper/322:21:313
1044525719,21sleep20-21swapper/220:36:312
911125613,25sleep09105-21apt-get20:31:440
647825617,19sleep30-21swapper/320:21:583
3111425614,16sleep20-21swapper/223:21:512
1805525617,22sleep318081-21df19:16:453
120525619,21sleep30-21swapper/323:31:293
1125925612,24sleep20-21swapper/220:36:592
3202725513,24sleep232073-21users21:42:052
3202725513,24sleep232073-21users21:42:052
3200625512,25sleep032000-21apt-get20:01:440
307125516,19sleep20-21swapper/223:36:452
2658325515,23sleep326624-21ntp_states23:06:543
2658325515,23sleep326624-21ntp_states23:06:543
244825514,22sleep00-21swapper/021:52:000
1640325513,25sleep316450-21users00:17:003
159322555,37sleep00-21swapper/022:32:260
1334225516,24sleep20-21swapper/200:07:222
125582557,38sleep00-21swapper/022:22:050
680425414,16sleep20-21swapper/222:06:482
680425414,16sleep20-21swapper/222:06:482
2741325419,19sleep30-21swapper/319:46:493
2703625412,23sleep30-21swapper/319:46:293
2678425416,18sleep00-21swapper/023:06:590
2678425416,18sleep00-21swapper/023:06:590
2462725415,20sleep30-21swapper/323:01:453
222625413,17sleep30-21swapper/321:51:553
1970825416,19sleep30-21swapper/322:46:313
1904825417,22sleep219092-21ntpq00:26:562
165242545,39sleep00-21swapper/000:17:500
147425412,25sleep21486-21apt-get23:31:402
778125313,24sleep27787-21df_inode23:51:472
1995725314,24sleep020097-21latency_hist00:31:310
1412725310,26sleep00-21swapper/022:27:220
1266725314,24sleep312686-21ntp_states20:41:573
83425214,19sleep00-21swapper/021:47:000
83425214,19sleep00-21swapper/021:47:000
445925211,25sleep24456-21apt-get20:16:402
3085625211,26sleep030923-21df23:21:450
289592524,17sleep30-21swapper/323:16:073
2772425212,24sleep227793-21df23:11:462
2110825213,24sleep321175-21df19:26:463
1720225210,16sleep00-21swapper/000:21:500
1720225210,16sleep00-21swapper/000:21:500
2292325112,16sleep30-21swapper/319:31:513
2292325112,16sleep30-21swapper/319:31:513
2086825120,17sleep00-21swapper/019:26:310
2034825117,17sleep30-21swapper/322:46:553
2686025014,23sleep326937-21grep21:27:003
1573125018,18sleep30-21swapper/320:51:543
1768024914,15sleep20-21swapper/200:22:002
274272488,28sleep2367-21sshd21:30:222
2272724811,23sleep222734-21diskstats19:31:462
2272724811,23sleep222734-21diskstats19:31:462
2949324614,19sleep20-21swapper/219:52:052
2334624612,20sleep323435-21memory22:56:533
1200824612,15sleep30-21swapper/322:21:523
2959624511,17sleep30-21swapper/321:36:413
2959624511,17sleep30-21swapper/321:36:413
2019324510,18sleep00-21swapper/000:31:430
3200324410,17sleep20-21swapper/220:01:422
50572429,19sleep05068-21ntp_states23:41:560
669824110,13sleep20-21swapper/223:46:592
669824110,13sleep20-21swapper/223:46:592
1854724111,15sleep30-21swapper/321:01:483
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional