You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-18 - 22:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot7s.osadl.org (updated Thu Dec 18, 2025 12:43:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
204822345115,3419sleep09020-21kworker/u8:108:05:270
204822345115,3419sleep09020-21kworker/u8:108:05:270
2478928867,865sleep124830-21ntpq10:00:061
1043828838,862sleep110441-21systemd-tmpfile10:58:501
1043828838,862sleep110441-21systemd-tmpfile10:58:501
15906288220,844sleep215935-21/usr/sbin/munin07:50:062
11788241520,377sleep311790-21sshd11:00:503
3866241058,333sleep13966-21dpkg07:14:431
28405235142,268sleep128483-21fschecks_time11:55:001
20042229435,185sleep120198-21apt-key08:04:431
30722228511,258sleep030813-21fschecks_time10:20:000
30722228511,258sleep030813-21fschecks_time10:20:000
579122807,262sleep3369-21snmpd07:17:483
579122807,262sleep3369-21snmpd07:17:483
9606226515,228sleep39625-21kernelversion07:30:043
9606226515,228sleep39625-21kernelversion07:30:043
30307225911,235sleep130336-21modprobe10:19:401
5371223813,208sleep35514-21awk10:40:143
32460223010,206sleep332487-21awk08:40:563
3072222945,167sleep13115-21ntpq08:50:071
3072222945,167sleep13115-21ntpq08:50:071
8489222475,61sleep10-21swapper/110:50:141
8489222475,61sleep10-21swapper/110:50:141
7102222217,182sleep37064-21latency_hist09:04:433
26471221921,173sleep026220-21dpkg11:49:410
26471221921,173sleep026220-21dpkg11:49:410
7702221812,158sleep17746-21ntp_kernel_err09:05:051
3141221365,68sleep13159-21fschecks_count10:35:031
8653220615,174sleep28692-21ntpq12:35:072
8653220615,174sleep28692-21ntpq12:35:072
18888220538,77sleep10-21swapper/109:40:151
8253220334,80sleep10-21swapper/112:34:501
8253220334,80sleep10-21swapper/112:34:501
5247220317,166sleep25340-21mailstats07:16:022
5247220317,166sleep25340-21mailstats07:16:022
31879219411,126sleep131909-21munin-run10:24:411
31879219411,126sleep131909-21munin-run10:24:411
5463219260,93sleep15477-21modprobe08:59:401
5463219260,93sleep15477-21modprobe08:59:401
11771218610,111sleep111910-21latency_hist09:19:431
26846218511,159sleep00-21swapper/008:24:450
26846218511,159sleep00-21swapper/008:24:450
606218446,81sleep10-21swapper/112:10:021
2616821779,154sleep026221-21interrupts10:05:010
2616821779,154sleep026221-21interrupts10:05:010
27908217519,38sleep10-21swapper/110:10:071
13637217567,92sleep113670-21fschecks_count09:25:001
13637217567,92sleep113670-21fschecks_count09:25:001
4904217435,118sleep10-21swapper/112:24:401
4904217435,118sleep10-21swapper/112:24:401
16191217245,73sleep116335-21unixbench_singl11:15:141
16191217245,73sleep116335-21unixbench_singl11:15:141
5794216640,81sleep10-21swapper/107:17:501
5794216640,81sleep10-21swapper/107:17:501
2690521627,18sleep10-21swapper/108:25:111
2690521627,18sleep10-21swapper/108:25:111
31193215515,125sleep331264-21latency_hist12:04:433
456215354,82sleep1511-21find10:25:261
456215354,82sleep1511-21find10:25:261
2217214853,64sleep12251-21cyclictest07:09:421
32668214555,44sleep10-21swapper/108:41:021
25472214422,47sleep325511-21ntpq08:20:043
11251214340,44sleep10-21swapper/107:35:071
11251214340,44sleep10-21swapper/107:35:071
23699214050,66sleep10-21swapper/111:40:091
20027213877,49sleep120063-21cat09:45:071
28487213511,112sleep00-21swapper/010:11:070
11221213512,110sleep111261-21ntp_kernel_pll_11:00:071
1775721348,68sleep1369-21snmpd11:20:271
2091121329,64sleep120411-21/usr/sbin/munin08:06:041
2091121329,64sleep120411-21/usr/sbin/munin08:06:041
12756213228,90sleep112907-21sed09:20:161
12756213228,90sleep112907-21sed09:20:161
31464213116,52sleep131482-21apt-get08:39:411
31464213116,52sleep131482-21apt-get08:39:411
15814213013,100sleep115950-21grep09:30:131
587021275,86sleep15900-21grep09:00:001
30414212518,22sleep20-21swapper/212:00:142
1796621259,80sleep118043-21latency_hist09:39:421
12210212513,98sleep012270-21cat07:39:420
12210212513,98sleep012270-21cat07:39:420
2735021239,77sleep127417-21awk11:50:441
15644212217,83sleep215436-21/usr/sbin/munin11:15:022
15644212217,83sleep215436-21/usr/sbin/munin11:15:022
15186212233,28sleep115218-21systemd-run11:14:411
9802211840,19sleep10-21swapper/107:30:101
9802211840,19sleep10-21swapper/107:30:101
30582211819,84sleep030689-21unixbench_multi12:00:160
29048211855,18sleep10-21swapper/108:30:161
15539211815,58sleep10-21swapper/107:49:571
3055211611,25sleep00-21swapper/010:34:580
26484211525,26sleep10-21swapper/111:49:421
26484211525,26sleep10-21swapper/111:49:421
14073211311,40sleep214080-21diskstats07:45:012
5186211230,65sleep15262-21switchtime10:40:121
2249221125,16sleep00-21swapper/009:53:550
21484211114,48sleep10-21swapper/109:50:041
25934211022,60sleep225953-21apt-get10:04:512
25934211022,60sleep225953-21apt-get10:04:512
2428210833,15sleep10-21swapper/108:49:421
13162210823,70sleep013272-21sed11:05:150
3967210715,74sleep24153-21fschecks_count07:14:442
19302210713,78sleep319361-21sendmail_mailqu11:25:133
17209210525,65sleep117287-21perl07:55:021
31894210443,19sleep10-21swapper/112:05:091
18474210412,76sleep018493-21kernelversion09:40:060
7883210219,66sleep10-21swapper/107:25:011
9315210010,20sleep00-21swapper/007:29:530
9315210010,20sleep00-21swapper/007:29:530
26857210018,63sleep326386-21apt-get08:24:523
26857210018,63sleep326386-21apt-get08:24:523
262862995,41sleep10-21swapper/108:23:451
2352729918,18sleep10-21swapper/108:14:571
134662985,39sleep10-21swapper/107:41:331
1418029723,54sleep10-21swapper/107:45:041
1400529715,65sleep214065-21/usr/sbin/munin11:10:022
566929625,14sleep10-21swapper/112:25:131
566929625,14sleep10-21swapper/112:25:131
2668729615,63sleep326707-21/usr/sbin/munin11:49:533
308722945,13sleep10-21swapper/112:01:141
2402029415,60sleep323501-21/usr/sbin/munin08:15:113
1858229414,64sleep018671-21latency_hist11:24:420
979629311,66sleep09863-21awk10:55:090
979629311,66sleep09863-21awk10:55:090
287829321,15sleep10-21swapper/112:15:151
287829321,15sleep10-21swapper/112:15:151
803629225,18sleep10-21swapper/112:34:421
531829218,54sleep25341-21tune2fs12:25:012
531829218,54sleep25341-21tune2fs12:25:012
2604329217,15sleep10-21swapper/110:04:581
2604329217,15sleep10-21swapper/110:04:581
2409429018,13sleep10-21swapper/109:59:411
2409429018,13sleep10-21swapper/109:59:411
1245828913,59sleep212463-21diskstats11:05:012
415628818,44sleep33751-21apt-get07:14:453
622828710,20sleep10-21swapper/107:19:521
3017728719,48sleep029694-21apt-get08:34:510
3017728719,48sleep029694-21apt-get08:34:510
234228713,56sleep32359-21kernelversion12:15:053
234228713,56sleep32359-21kernelversion12:15:053
209492879,41sleep121041-21cat11:30:161
104312878,57sleep30-21swapper/309:14:573
2266928518,31sleep20-21swapper/211:35:262
127762858,64sleep112815-21ntp_kernel_pll_11:05:081
74422848,15sleep10-21swapper/110:49:411
2567028420,41sleep325695-21idleruntime-cro10:04:403
21042840,6sleep00-21swapper/012:14:570
21042840,6sleep00-21swapper/012:14:570
1716728418,47sleep017208-21fschecks_count07:55:010
2896428317,47sleep229081-21wc11:55:132
232352808,52sleep10-21swapper/108:14:411
232352808,52sleep10-21swapper/108:14:411
2229128014,32sleep122316-21awk11:35:171
1871728019,27sleep30-21swapper/309:40:133
101902808,18sleep110254-21latency_hist09:14:421
787627915,49sleep27940-21/usr/sbin/munin09:05:102
126227918,39sleep21387-21grep10:29:432
126227918,39sleep21387-21grep10:29:432
644827821,29sleep00-21swapper/012:29:400
644827821,29sleep00-21swapper/012:29:400
614027816,25sleep00-21swapper/010:44:510
1728027821,36sleep317269-21apt-get11:19:563
1664127821,34sleep316708-21df09:35:003
1516827726,34sleep00-21swapper/011:14:400
3217727622,29sleep20-21swapper/208:40:512
137827613,49sleep21390-21/usr/sbin/munin08:45:032
1216227613,39sleep00-21swapper/011:04:430
2641827513,47sleep226433-21idleruntime-cro11:49:392
2641827513,47sleep226433-21idleruntime-cro11:49:392
189142758,12sleep10-21swapper/111:25:031
147627525,28sleep30-21swapper/307:05:093
2483227411,46sleep224865-21ntp_states10:00:072
2248127413,25sleep00-21swapper/008:10:110
2248127413,25sleep00-21swapper/008:10:110
210832740,7sleep20-21swapper/208:06:082
210832740,7sleep20-21swapper/208:06:082
1536627413,44sleep015460-21memory09:30:040
1050427423,33sleep210534-21modprobe10:59:402
1050427423,33sleep210534-21modprobe10:59:402
377927315,29sleep20-21swapper/212:20:052
377927315,29sleep20-21swapper/212:20:052
311727318,22sleep20-21swapper/208:50:082
311727318,22sleep20-21swapper/208:50:082
2478627317,24sleep00-21swapper/008:19:400
1987727315,41sleep219904-21/usr/sbin/munin09:45:042
1853127324,32sleep218558-21modprobe11:24:402
3193427220,36sleep232026-21apt-config10:24:422
3193427220,36sleep232026-21apt-config10:24:422
312627212,17sleep30-21swapper/308:50:093
312627212,17sleep30-21swapper/308:50:093
2963727211,41sleep029760-21wc11:59:430
2719827215,39sleep027235-21sh11:50:380
24482725,12sleep10-21swapper/110:30:351
117427228,22sleep00-21swapper/007:05:020
226027122,26sleep20-21swapper/207:09:432
1734127119,34sleep30-21swapper/307:55:043
1604627120,30sleep00-21swapper/007:50:120
836527015,32sleep08486-21cat10:50:130
836527015,32sleep08486-21cat10:50:130
760227015,38sleep37661-21/usr/sbin/munin09:05:033
3038727022,23sleep30-21swapper/310:19:423
2174927010,46sleep021813-21latency_hist11:34:430
2007427016,41sleep21-21systemd11:29:402
1367727019,34sleep313648-21latency_hist11:09:423
1302727025,23sleep20-21swapper/207:40:132
63442696,43sleep00-21swapper/007:20:030
434426916,32sleep04494-21unixbench_multi12:20:190
434426916,32sleep04494-21unixbench_multi12:20:190
2302326910,11sleep30-21swapper/308:11:523
2302326910,11sleep30-21swapper/308:11:523
2115126923,30sleep30-21swapper/308:06:103
2115126923,30sleep30-21swapper/308:06:103
157926911,42sleep01619-21ps08:45:090
103372696,12sleep20-21swapper/207:31:202
103372696,12sleep20-21swapper/207:31:202
973126818,33sleep39793-21/usr/sbin/munin10:55:083
973126818,33sleep39793-21/usr/sbin/munin10:55:083
825826820,31sleep28293-21ps07:25:102
800426820,30sleep38054-21/usr/sbin/munin07:25:033
733326810,17sleep00-21swapper/009:04:560
447526819,28sleep04564-21memory07:14:500
447526819,28sleep04564-21memory07:14:500
2981126824,21sleep00-21swapper/008:34:430
285226815,31sleep00-21swapper/007:10:020
248742689,44sleep00-21swapper/010:00:090
1435226812,38sleep214451-21users09:25:132
1435226812,38sleep214451-21users09:25:132
1256026813,24sleep30-21swapper/309:20:143
1256026813,24sleep30-21swapper/309:20:143
1213126820,32sleep212136-21modprobe07:39:402
1213126820,32sleep212136-21modprobe07:39:402
165022679,10sleep30-21swapper/311:16:493
165022679,10sleep30-21swapper/311:16:493
3157126616,30sleep30-21swapper/308:39:433
3157126616,30sleep30-21swapper/308:39:433
2348426613,15sleep00-21swapper/009:55:120
2348426613,15sleep00-21swapper/009:55:120
213362667,47sleep30-21swapper/311:30:593
1780626611,41sleep0369-21snmpd09:38:360
459826521,28sleep24617-21apt-get10:39:532
1457326514,37sleep314665-21cat07:45:143
1435926527,24sleep013971-21/usr/sbin/munin07:45:070
1013926519,29sleep30-21swapper/309:14:403
656226423,27sleep20-21swapper/210:45:062
480826416,27sleep00-21swapper/010:40:030
256442640,7sleep00-21swapper/008:20:080
244862649,12sleep20-21swapper/208:15:512
2376326413,35sleep323842-21cat11:40:113
2225326410,17sleep30-21swapper/309:50:513
179126422,17sleep30-21swapper/308:45:123
750826315,29sleep37644-21/usr/sbin/munin12:30:203
674426318,24sleep20-21swapper/207:20:142
2192426313,30sleep021989-21cpuspeed11:35:070
2109126316,23sleep00-21swapper/009:49:430
1115226317,28sleep30-21swapper/307:35:053
1115226317,28sleep30-21swapper/307:35:053
101226317,25sleep31093-21mailstats12:10:123
837326214,19sleep00-21swapper/007:25:120
673926217,22sleep30-21swapper/310:45:113
37482628,41sleep00-21swapper/008:52:240
37482628,41sleep00-21swapper/008:52:240
30726216,28sleep232697-21latency_hist12:09:432
2915626212,30sleep229297-21unixbench_singl08:30:172
1672126221,27sleep016795-21apt-config11:19:410
1672126221,27sleep016795-21apt-config11:19:410
3254326118,24sleep30-21swapper/312:07:503
3228326115,17sleep00-21swapper/008:40:530
2357326118,18sleep00-21swapper/011:40:040
2022526116,30sleep320266-21processes09:45:133
158926114,27sleep01611-21fschecks_count10:30:010
3207526015,29sleep032197-21sed12:05:120
2949926015,27sleep00-21swapper/010:15:070
2741826019,19sleep20-21swapper/211:50:452
2733626014,24sleep20-21swapper/210:09:442
2332726016,23sleep20-21swapper/209:55:102
2332726016,23sleep20-21swapper/209:55:102
1818226020,19sleep20-21swapper/209:39:512
1747926018,26sleep20-21swapper/207:55:072
1326026018,27sleep213294-21munin-run09:24:412
1326026018,27sleep213294-21munin-run09:24:412
678925814,22sleep00-21swapper/012:29:560
2944425817,25sleep20-21swapper/210:15:052
1438625817,24sleep30-21swapper/311:10:113
1381325821,21sleep00-21swapper/009:25:030
1381325821,21sleep00-21swapper/009:25:030
916025715,20sleep00-21swapper/009:10:050
672825717,23sleep06732-21modprobe09:00:400
2347025717,22sleep223478-21apt-get11:39:542
1865225714,27sleep218645-21apt-get07:59:552
1311125717,25sleep313226-21timerwakeupswit07:40:143
489825617,17sleep20-21swapper/208:55:172
489825617,17sleep20-21swapper/208:55:172
421125614,25sleep04205-21apt-get08:54:540
421125614,25sleep04205-21apt-get08:54:540
2755225613,25sleep227671-21/usr/sbin/munin08:25:442
2755225613,25sleep227671-21/usr/sbin/munin08:25:442
2675925610,31sleep326860-21users10:05:123
2675925610,31sleep326860-21users10:05:123
1886025610,32sleep018912-21if_eth008:00:020
1337625619,20sleep00-21swapper/007:40:230
3237825510,22sleep30-21swapper/310:24:513
3237825510,22sleep30-21swapper/310:24:513
302625517,19sleep20-21swapper/210:34:522
2876625515,16sleep30-21swapper/311:55:103
2514925515,19sleep30-21swapper/311:45:043
2514925515,19sleep30-21swapper/311:45:043
224052557,37sleep20-21swapper/209:52:482
2200725517,18sleep30-21swapper/311:35:113
201525515,25sleep22106-21mailstats10:30:152
1262925510,17sleep00-21swapper/009:20:150
1262925510,17sleep00-21swapper/009:20:150
972925417,16sleep30-21swapper/312:39:443
972925417,16sleep30-21swapper/312:39:443
89082545,16sleep20-21swapper/210:53:582
89082545,16sleep20-21swapper/210:53:582
650625415,20sleep30-21swapper/307:20:063
3062525412,25sleep330616-21apt-get10:19:543
3062525412,25sleep330616-21apt-get10:19:543
228425411,27sleep20-21swapper/212:15:042
228425411,27sleep20-21swapper/212:15:042
196225419,15sleep30-21swapper/310:30:123
1876825414,23sleep318858-21perl08:00:013
1097525410,15sleep20-21swapper/209:15:122
828425313,19sleep30-21swapper/310:50:123
828425313,19sleep30-21swapper/310:50:123
708325317,22sleep20-21swapper/209:04:422
672925318,18sleep30-21swapper/312:29:553
672925318,18sleep30-21swapper/312:29:553
1043325313,18sleep00-21swapper/009:14:580
98625214,25sleep21011-21proc_pri12:10:112
2519325213,18sleep20-21swapper/208:19:592
2283925212,24sleep322834-21apt-get09:54:553
2283925212,24sleep322834-21apt-get09:54:553
2198625212,24sleep221979-21apt-get08:09:572
2198625212,24sleep221979-21apt-get08:09:572
1509125214,18sleep20-21swapper/209:29:542
1395025215,21sleep30-21swapper/309:25:063
1395025215,21sleep30-21swapper/309:25:063
1161925212,19sleep20-21swapper/211:00:162
470125112,24sleep30-21swapper/308:55:143
470125112,24sleep30-21swapper/308:55:143
3254925113,23sleep032611-21perl10:25:170
3254925113,23sleep032611-21perl10:25:170
3152725115,15sleep20-21swapper/208:39:422
3152725115,15sleep20-21swapper/208:39:422
1561225113,20sleep315621-21diskstats07:49:593
889625012,21sleep28962-21df09:09:592
297282506,18sleep30-21swapper/308:34:423
1936325011,22sleep019434-21sensors11:25:140
83612498,15sleep00-21swapper/012:34:570
83612498,15sleep00-21swapper/012:34:570
739124912,18sleep20-21swapper/212:30:182
52124815,17sleep00-21swapper/012:09:570
411024813,16sleep30-21swapper/312:20:143
411024813,16sleep30-21swapper/312:20:143
166392487,18sleep20-21swapper/209:34:592
1552224710,21sleep315546-21ntp_states09:30:073
437224615,17sleep30-21swapper/310:39:413
2759224613,17sleep30-21swapper/310:10:013
2036924611,20sleep220363-21apt-get11:29:572
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional