You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-29 - 07:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Mon Dec 29, 2025 00:43:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6728243118,209sleep26779-21sort21:14:252
6728243118,209sleep26779-21sort21:14:252
359724317,404sleep217249-21kworker/u8:321:04:162
15348229214,121sleep215401-21cpuspeed23:24:262
296592246127,100sleep229673-21fschecks_count20:44:252
296592246127,100sleep229673-21fschecks_count20:44:252
30590223472,19sleep20-21swapper/222:29:242
12959222697,107sleep20-21swapper/223:14:372
12959222697,107sleep20-21swapper/223:14:372
941622160,6sleep20-21swapper/223:04:272
941622160,6sleep20-21swapper/223:04:272
19215221315,89sleep20-21swapper/221:54:312
13301221340,89sleep20-21swapper/219:54:172
32435220610,183sleep232496-21fschecks_time22:35:302
2681921949,170sleep226941-21timerandwakeup23:59:382
15846219034,112sleep215985-21unixbench_multi19:59:592
10759218477,26sleep20-21swapper/223:09:252
32514218276,23sleep20-21swapper/200:19:252
32514218276,23sleep20-21swapper/200:19:252
356218017,104sleep20-21swapper/220:54:262
235121759,16sleep00-21swapper/019:19:350
17291217417,143sleep017327-21idleruntime-cro21:49:040
2474216411,114sleep22615-21sed20:59:392
2370216411,75sleep22450-21fschecks_time19:19:372
18469216319,105sleep20-21swapper/220:09:202
24826215863,83sleep224868-21if_eth023:54:262
29231215546,66sleep20-21swapper/222:25:002
21304214974,12sleep20-21swapper/221:59:362
7562214812,84sleep20-21swapper/222:59:202
1060321485,11sleep30-21swapper/319:44:293
28533214710,83sleep228741-21cat22:24:072
28533214710,83sleep228741-21cat22:24:072
31883214342,18sleep20-21swapper/219:09:512
31883214342,18sleep20-21swapper/219:09:512
28116214017,110sleep128135-21fschecks_count20:39:231
28116214017,110sleep128135-21fschecks_count20:39:231
21276214046,82sleep221356-21apt-config20:19:052
25464213925,86sleep20-21swapper/220:29:352
14982213427,95sleep215064-21mailstats21:39:332
14982213427,95sleep215064-21mailstats21:39:332
11506213341,57sleep211592-21perl21:29:272
711121316,71sleep20-21swapper/222:57:052
711121316,71sleep20-21swapper/222:57:052
4626213112,76sleep24683-21idleruntime22:49:232
1688621318,67sleep20-21swapper/223:29:232
19089213020,88sleep019204-21cat23:34:380
1916212931,86sleep21951-21ntpq00:24:302
329521288,107sleep23343-21ntp_kernel_pll_22:44:302
14257212814,98sleep014393-21latency_hist21:39:060
8507212716,19sleep20-21swapper/219:39:072
18905212536,75sleep219049-21grep21:54:072
2429321238,102sleep224380-21basename22:09:372
13411212214,90sleep113492-21grep21:34:351
6048212117,81sleep20-21swapper/219:30:192
16308212014,22sleep20-21swapper/221:44:302
8965211918,83sleep08985-21cut19:39:290
3652211922,74sleep13792-21grep22:44:371
4269211813,91sleep34343-21find21:05:053
22769211815,87sleep022844-21sensors_temp22:04:340
27451211712,91sleep127464-21kernelversion22:20:021
27451211712,91sleep127464-21kernelversion22:20:021
2655821169,93sleep326574-21needreboot23:59:313
21462211514,85sleep022860-1kworker/0:1H20:19:070
4178211415,70sleep04180-21cron22:49:040
26673211433,63sleep226682-21munin-run22:19:062
5003211119,13sleep20-21swapper/219:28:252
21803211119,72sleep021296-21apt-get20:19:150
21803211119,72sleep021296-21apt-get20:19:150
31603210843,52sleep20-21swapper/220:49:422
23493210810,28sleep20-21swapper/222:09:052
13797210818,45sleep10-21swapper/123:19:211
8915210712,78sleep09035-21sed21:19:360
8915210712,78sleep09035-21sed21:19:360
4826210615,29sleep20-21swapper/200:34:192
4826210615,29sleep20-21swapper/200:34:192
1514221057,13sleep20-21swapper/223:24:142
381210312,16sleep20-21swapper/219:14:112
381210312,16sleep20-21swapper/219:14:112
29004210223,59sleep3228-21systemd-journal00:09:043
29004210223,59sleep3228-21systemd-journal00:09:043
28921210218,67sleep117249-21kworker/u8:322:24:181
9930210113,67sleep210020-21fschecks_time21:24:272
12244210112,42sleep20-21swapper/219:49:332
25994210025,50sleep326047-21grep23:59:073
1544210018,24sleep10-21swapper/120:59:061
1024129921,62sleep20-21swapper/219:44:202
3128529723,54sleep331300-21sort19:09:363
3128529723,54sleep331300-21sort19:09:363
2645729736,15sleep20-21swapper/220:34:192
239922968,12sleep10-21swapper/122:09:281
1268729615,64sleep112795-21latency_hist21:34:071
555729524,46sleep05067-21apt-get19:29:160
219322956,77sleep20-21swapper/220:19:332
219322956,77sleep20-21swapper/220:19:332
2141229514,57sleep321559-21unixbench_singl21:59:373
245062937,74sleep20-21swapper/220:27:292
2316729315,57sleep30-21swapper/323:49:263
2316729315,57sleep30-21swapper/323:49:263
727729117,49sleep07348-21cat21:14:390
727729117,49sleep07348-21cat21:14:390
51792917,15sleep10-21swapper/121:09:191
288932917,15sleep00-21swapper/022:24:130
2843029114,62sleep328471-21ntpq20:39:293
2843029114,62sleep328471-21ntpq20:39:293
3084029034,15sleep20-21swapper/200:14:202
200802909,23sleep20-21swapper/223:39:242
178392906,74sleep00-21swapper/023:30:140
531128927,46sleep05067-21apt-get19:29:080
236572896,69sleep20-21swapper/223:49:372
236572896,69sleep20-21swapper/223:49:372
140792886,71sleep20-21swapper/221:37:472
819128716,55sleep08269-21find22:59:420
2813628733,14sleep20-21swapper/220:39:242
2813628733,14sleep20-21swapper/220:39:242
88222867,35sleep20-21swapper/219:39:272
3133228613,56sleep131410-21/usr/sbin/munin19:09:381
3133228613,56sleep131410-21/usr/sbin/munin19:09:381
2799928634,13sleep20-21swapper/200:04:272
2799928634,13sleep20-21swapper/200:04:272
2467028615,54sleep124702-21cut20:29:051
1459228613,54sleep114699-21grep21:39:241
1459228613,54sleep114699-21grep21:39:241
218222840,9sleep00-21swapper/023:44:300
1813328420,40sleep318167-21munin-plugin-st23:34:043
1774528412,58sleep117749-21sshd20:06:321
1774528412,58sleep117749-21sshd20:06:321
876328313,51sleep18778-21/usr/sbin/munin21:19:331
876328313,51sleep18778-21/usr/sbin/munin21:19:331
71822837,48sleep10-21swapper/121:14:361
71822837,48sleep10-21swapper/121:14:361
1802728313,56sleep018038-21ntp_states21:49:320
1521128314,53sleep315220-21diskstats19:59:233
92372825,66sleep20-21swapper/221:19:582
92372825,66sleep20-21swapper/221:19:582
78522817,62sleep34887-21kworker/u8:121:17:213
78522817,62sleep34887-21kworker/u8:121:17:213
3083928115,49sleep330874-21/usr/sbin/munin22:29:293
2988928131,31sleep30-21swapper/319:04:403
2988928131,31sleep30-21swapper/319:04:403
2500128116,34sleep30-21swapper/320:29:243
187528118,46sleep11898-21cut20:59:261
750028016,45sleep07531-21/usr/sbin/munin19:34:290
355128020,48sleep20-21swapper/200:29:342
355128020,48sleep20-21swapper/200:29:342
3140828014,48sleep031445-21latency20:49:360
302528015,46sleep03047-21apt-config21:03:290
1886328019,43sleep118892-21/usr/sbin/munin20:09:471
129962809,58sleep314510-21kworker/3:123:14:383
129962809,58sleep314510-21kworker/3:123:14:383
2438127925,29sleep30-21swapper/322:09:383
176532796,63sleep30-21swapper/320:05:213
176532796,63sleep30-21swapper/320:05:213
166862797,33sleep20-21swapper/220:04:192
166862797,33sleep20-21swapper/220:04:192
1122927924,27sleep10-21swapper/123:09:361
559127810,49sleep20-21swapper/221:09:332
2905727821,38sleep129143-21latency_hist00:09:061
2905727821,38sleep129143-21latency_hist00:09:061
1163927816,46sleep011658-21sshd21:29:290
106527826,31sleep10-21swapper/122:39:041
3054827710,50sleep20-21swapper/200:14:042
3054827710,50sleep20-21swapper/200:14:042
2994027723,25sleep30-21swapper/300:09:373
2994027723,25sleep30-21swapper/300:09:373
456927617,36sleep34335-21apt-get00:34:083
456927617,36sleep34335-21apt-get00:34:083
2711127626,32sleep326624-21apt-get22:19:153
2711127626,32sleep326624-21apt-get22:19:153
2357027615,44sleep123656-21mailstats23:49:361
2357027615,44sleep123656-21mailstats23:49:361
227892767,57sleep222837-21systemd-run23:49:052
1474027615,43sleep314952-21latency_hist19:59:073
845927522,25sleep10-21swapper/119:39:061
428327514,37sleep14345-21wc19:24:361
3227227522,33sleep032252-21apt-get22:34:310
2717227510,12sleep00-21swapper/000:00:170
3130327418,38sleep031330-21fschecks_count19:09:370
3130327418,38sleep031330-21fschecks_count19:09:370
3026327420,37sleep030393-21wc19:04:490
3026327420,37sleep030393-21wc19:04:490
1300927426,26sleep30-21swapper/321:34:253
46027317,28sleep00-21swapper/000:19:350
46027317,28sleep00-21swapper/000:19:350
2538027318,42sleep025401-21awk20:29:330
1521027311,17sleep00-21swapper/023:24:200
61527210,25sleep30-21swapper/319:14:163
61527210,25sleep30-21swapper/319:14:163
55242724,11sleep10-21swapper/122:51:311
827227119,34sleep18344-21ls22:59:431
2154127116,30sleep10-21swapper/123:44:231
3151727010,46sleep1369-21snmpd22:31:171
2676227017,31sleep326885-21latency_hist22:19:073
648426920,23sleep00-21swapper/021:14:070
282702698,49sleep00-21swapper/022:22:230
282702698,49sleep00-21swapper/022:22:230
2587326916,29sleep00-21swapper/023:59:040
222412697,16sleep30-21swapper/322:04:213
2045326914,39sleep120561-21sed21:59:061
381126815,23sleep10-21swapper/100:29:371
381126815,23sleep10-21swapper/100:29:371
333926812,41sleep03423-21latency00:29:280
333926812,41sleep03423-21latency00:29:280
26862689,38sleep20-21swapper/222:44:062
2646026818,31sleep026453-21apt-get20:34:210
218412689,45sleep321908-21df20:19:263
218412689,45sleep321908-21df20:19:263
189232686,12sleep20-21swapper/223:34:362
1458026818,36sleep114655-21apt-key19:59:051
953126716,23sleep00-21swapper/023:04:300
953126716,23sleep00-21swapper/023:04:300
599926719,26sleep00-21swapper/000:37:440
599926719,26sleep00-21swapper/000:37:440
2829126722,24sleep00-21swapper/020:39:260
2829126722,24sleep00-21swapper/020:39:260
27202674,51sleep00-21swapper/000:28:190
849026614,37sleep38502-21vmstat22:59:453
2048626614,38sleep320524-21/usr/sbin/munin20:14:383
2037326619,30sleep320401-21munin-run21:59:043
1844326620,23sleep30-21swapper/320:09:173
1347826620,28sleep013514-21modprobe23:19:030
1347826620,28sleep013514-21modprobe23:19:030
1231826612,23sleep00-21swapper/019:49:350
2779426515,34sleep127901-21latency_hist20:39:071
206326514,36sleep02191-21/usr/sbin/munin22:39:360
2957626414,26sleep30-21swapper/320:44:223
2957626414,26sleep30-21swapper/320:44:223
2811926414,21sleep00-21swapper/000:04:290
2811926414,21sleep00-21swapper/000:04:290
259312648,13sleep00-21swapper/022:14:360
1668926415,35sleep016742-21ls20:04:210
1668926415,35sleep016742-21ls20:04:210
133226419,22sleep00-21swapper/019:14:420
133226419,22sleep00-21swapper/019:14:420
780526310,29sleep30-21swapper/319:34:363
499126321,18sleep10-21swapper/121:09:071
494026321,21sleep00-21swapper/021:09:060
123026325,22sleep10-21swapper/100:24:041
123026325,22sleep10-21swapper/100:24:041
1200926315,27sleep00-21swapper/023:14:070
572026218,26sleep35749-21munin-run22:54:043
3252426212,35sleep332614-21fschecks_time00:19:263
3252426212,35sleep332614-21fschecks_time00:19:263
2026326236,13sleep00-21swapper/021:57:360
1067526215,26sleep310741-21cpuspeed23:09:223
742726118,27sleep30-21swapper/322:59:083
742726118,27sleep30-21swapper/322:59:083
628726119,20sleep00-21swapper/022:54:330
628726119,20sleep00-21swapper/022:54:330
3164726110,25sleep30-21swapper/320:49:433
2957326119,25sleep029569-21apt-get20:44:190
2957326119,25sleep029569-21apt-get20:44:190
18472618,11sleep10-21swapper/122:39:331
1765526115,22sleep30-21swapper/321:49:233
1221726118,17sleep10-21swapper/123:14:201
1221726118,17sleep10-21swapper/123:14:201
31826019,21sleep10-21swapper/119:14:101
31826019,21sleep10-21swapper/119:14:101
1008026015,17sleep00-21swapper/021:24:290
254725912,23sleep10-21swapper/119:19:401
2337225915,26sleep323367-21apt-get20:24:193
2180425913,28sleep321820-21/usr/sbin/munin23:44:303
2057025914,23sleep10-21swapper/123:39:351
1847525916,24sleep00-21swapper/020:09:220
1604025916,26sleep316035-21apt-get21:44:223
1178325915,27sleep111777-21apt-get19:49:201
3081325819,20sleep10-21swapper/100:14:151
2958325815,22sleep30-21swapper/322:25:093
2594625817,25sleep125958-21munin-run23:59:051
2203525813,23sleep10-21swapper/122:04:071
1544425816,27sleep115500-21tune2fs23:24:381
1510225814,26sleep114605-21apt-get19:59:131
1495525818,24sleep015088-21apt-key19:59:080
505225713,21sleep10-21swapper/100:34:461
505225713,21sleep10-21swapper/100:34:461
327182577,11sleep30-21swapper/322:35:353
3219125715,19sleep00-21swapper/000:19:070
3019325714,25sleep130202-21/usr/sbin/munin00:09:391
3019325714,25sleep130202-21/usr/sbin/munin00:09:391
3006825718,18sleep00-21swapper/000:09:380
3006825718,18sleep00-21swapper/000:09:380
2334825712,27sleep00-21swapper/020:24:160
2318025712,18sleep00-21swapper/023:49:270
2318025712,18sleep00-21swapper/023:49:270
2083725717,24sleep00-21swapper/021:59:260
165125718,17sleep112-21rcu_sched00:24:251
1154625713,28sleep011668-21wc19:49:060
921225615,24sleep19215-21diskstats23:04:231
921225615,24sleep19215-21diskstats23:04:231
870325614,24sleep18697-21apt-get19:39:221
2532425615,25sleep125317-21apt-get22:14:211
1803925617,24sleep118047-21ntpq21:49:331
518225514,24sleep35177-21apt-get21:09:213
323282556,18sleep10-21swapper/120:53:241
310302557,11sleep30-21swapper/300:14:253
309825514,24sleep33126-21munin-run21:04:043
3054025525,18sleep10-21swapper/119:04:581
3054025525,18sleep10-21swapper/119:04:581
2771125517,23sleep31-21systemd20:39:053
2750325516,19sleep10-21swapper/100:04:051
209425514,20sleep30-21swapper/300:24:353
1494925517,22sleep014955-21ntp_states21:39:310
1494925517,22sleep014955-21ntp_states21:39:310
1159325515,18sleep30-21swapper/321:29:283
599425413,23sleep16012-21apt-get22:54:221
599425413,23sleep16012-21apt-get22:54:221
489925415,22sleep34926-21df00:34:243
489925415,22sleep34926-21df00:34:243
2983525413,22sleep10-21swapper/120:44:281
2983525413,22sleep10-21swapper/120:44:281
2284525418,19sleep10-21swapper/122:04:361
2036225414,23sleep30-21swapper/323:39:293
1624925415,17sleep10-21swapper/121:44:291
146025414,24sleep31465-21diskstats22:39:223
1021625413,25sleep30-21swapper/321:24:323
54732536,35sleep00-21swapper/022:50:500
2256825314,21sleep10-21swapper/120:19:571
2256825314,21sleep10-21swapper/120:19:571
1386625313,23sleep313872-21df_inode23:19:233
1353725311,24sleep013596-21irqstats19:54:280
995225213,21sleep39986-21munin-run19:44:053
317025218,17sleep30-21swapper/319:19:503
315825214,18sleep30-21swapper/322:44:273
1842425214,21sleep118418-21apt-get23:34:181
1749025213,23sleep116861-21/usr/sbin/munin23:29:421
558525113,22sleep35577-21apt-get19:29:193
3010425122,16sleep230168-21mailstats19:04:472
3010425122,16sleep230168-21mailstats19:04:472
2622025117,14sleep10-21swapper/120:34:061
1900225119,19sleep30-21swapper/323:34:373
3266825013,23sleep332665-21apt-get20:54:193
2508725013,15sleep00-21swapper/022:14:070
1999425011,23sleep019990-21apt-get23:39:180
1663225010,24sleep016642-21munin-run23:29:050
1196525014,21sleep311758-21/usr/sbin/munin19:49:263
3275124912,22sleep032755-21diskstats20:54:240
3047024911,21sleep030464-21apt-get22:29:180
2122824913,21sleep121229-21sshd20:18:271
106592496,16sleep10-21swapper/119:44:311
829924813,20sleep38295-21apt-get21:19:193
829924813,20sleep38295-21apt-get21:19:193
645024711,19sleep16548-21users19:30:261
1506524612,15sleep30-21swapper/321:39:343
1506524612,15sleep30-21swapper/321:39:343
101702464,30sleep30-21swapper/323:06:193
101702464,30sleep30-21swapper/323:06:193
519724510,16sleep30-21swapper/319:29:073
98442448,18sleep10-21swapper/121:24:221
1609824411,16sleep30-21swapper/323:24:523
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional