You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-06 - 01:47

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Wed Aug 05, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1663283717,804sleep11675-21/usr/sbin/munin12:32:471
15831232220,274sleep115867-21idleruntime-cro11:37:241
15831232220,274sleep115867-21idleruntime-cro11:37:241
19193226954,149sleep3225-21systemd-journal11:47:243
29950226046,170sleep330088-21unixbench_singl12:17:563
9612221935,119sleep39615-21cron11:17:233
30917221325,175sleep330981-21idleruntime12:22:443
15194220215,125sleep30-21swapper/309:52:453
12687220267,106sleep30-21swapper/309:42:273
32048219610,127sleep332174-21latency_hist08:07:273
3333217290,61sleep33374-21ntpq10:57:493
18623217030,125sleep318664-21ntp_kernel_pll_10:02:503
10025216865,62sleep30-21swapper/311:17:443
1487216613,73sleep21510-21fschecks_count12:32:442
29242216522,122sleep129334-21sort10:37:451
16428215911,89sleep316429-21cpu11:38:073
11564215911,67sleep311583-21fschecks_count11:22:443
349215715,119sleep3420-21switchtime10:47:563
349215715,119sleep3420-21switchtime10:47:563
26943215628,59sleep30-21swapper/307:27:263
307214910,84sleep3358-21processes12:27:533
22737214710,112sleep30-21swapper/310:17:283
22420214233,70sleep322478-21munin-run11:57:253
467921418,14sleep00-21swapper/008:47:250
21032214111,112sleep00-21swapper/011:52:370
24737213681,15sleep324863-21cpu07:12:283
1497021347,88sleep315005-21ntpq11:32:503
1497021347,88sleep315005-21ntpq11:32:503
2610821285,11sleep30-21swapper/307:20:153
2733021247,79sleep327389-21date10:32:263
24280212321,27sleep20-21swapper/212:02:452
1074121209,57sleep30-21swapper/309:27:373
1074121209,57sleep30-21swapper/309:27:373
774521175,102sleep00-21swapper/011:08:280
774521175,102sleep00-21swapper/011:08:280
32674211418,75sleep232751-21latency_hist08:12:272
6766211310,54sleep36771-21find11:07:343
6766211310,54sleep36771-21find11:07:343
2074721135,28sleep30-21swapper/310:08:363
3238621105,28sleep30-21swapper/308:09:173
499921095,29sleep30-21swapper/308:48:533
13553210816,20sleep00-21swapper/009:47:330
25788210715,72sleep025896-21latency_hist10:27:260
730821065,32sleep30-21swapper/309:05:493
14574210610,77sleep214623-21/usr/sbin/munin11:32:412
14574210610,77sleep214623-21/usr/sbin/munin11:32:412
406821058,14sleep30-21swapper/308:42:273
21335210526,63sleep30-21swapper/310:12:393
5454210414,75sleep15542-21apt-key08:52:271
8609210316,71sleep18621-21kernelversion11:12:491
16771210323,57sleep216832-21fschecks_time09:57:442
3099121027,85sleep3364-21snmpd07:57:343
10427210113,67sleep110636-21dpkg09:27:271
10427210113,67sleep110636-21dpkg09:27:271
27939210039,41sleep30-21swapper/310:32:503
829429930,17sleep30-21swapper/309:12:273
3231229913,69sleep232321-21df_inode10:47:432
3231229913,69sleep232321-21df_inode10:47:432
101152998,75sleep00-21swapper/011:17:450
297092988,80sleep3364-21snmpd07:47:373
2287629817,60sleep022367-21apt-get11:57:340
1317629818,69sleep30-21swapper/309:46:113
3082329710,73sleep030901-21/usr/sbin/munin10:42:440
273029743,42sleep32756-21latency_hist12:37:263
2074429722,14sleep30-21swapper/311:52:243
951729619,66sleep30-21swapper/309:21:113
65582967,73sleep36706-21latency_hist11:07:263
2447429622,15sleep30-21swapper/310:22:373
2897229515,64sleep229082-21latency_hist12:17:262
132092959,72sleep313269-21if_err_eth011:27:453
1800029421,16sleep30-21swapper/310:02:263
277629313,62sleep02928-21latency_hist08:32:270
277629313,62sleep02928-21latency_hist08:32:270
2741529329,13sleep30-21swapper/312:12:273
1985729320,42sleep10-21swapper/110:07:411
1205729310,69sleep312188-21latency_hist09:37:273
463629223,14sleep30-21swapper/308:47:243
996729119,61sleep30-21swapper/309:23:503
996729119,61sleep30-21swapper/309:23:503
90722909,27sleep39220-21sed09:17:283
304229014,54sleep12932-21/usr/sbin/munin10:57:431
2647729021,13sleep30-21swapper/312:07:503
2320529011,27sleep30-21swapper/311:58:063
278502897,34sleep30-21swapper/307:33:193
213128921,13sleep30-21swapper/308:27:263
1031228830,31sleep00-21swapper/009:27:260
1031228830,31sleep00-21swapper/009:27:260
350628713,52sleep03575-21cat10:57:540
322128733,12sleep30-21swapper/308:35:413
276328733,37sleep30-21swapper/308:32:263
276328733,37sleep30-21swapper/308:32:263
2753028718,53sleep127555-21idleruntime-cro07:32:241
2753028718,53sleep127555-21idleruntime-cro07:32:241
263828728,33sleep22661-21munin-run10:57:242
2403028612,19sleep30-21swapper/307:07:273
2385528615,30sleep10-21swapper/111:58:241
325202850,7sleep00-21swapper/010:47:480
325202850,7sleep00-21swapper/010:47:480
1481728518,12sleep30-21swapper/309:52:243
12932857,10sleep30-21swapper/308:20:113
646428433,39sleep30-21swapper/308:58:063
646428433,39sleep30-21swapper/308:58:063
2015928413,19sleep10-21swapper/111:47:541
1638328428,33sleep216379-21apt-get11:37:372
903328314,55sleep28989-21latency_hist09:17:272
3074628329,41sleep30-21swapper/307:57:263
247612826,14sleep30-21swapper/312:02:583
570628110,14sleep30-21swapper/308:52:293
685227912,51sleep06994-21latency_hist09:02:270
685227912,51sleep06994-21latency_hist09:02:270
295762797,14sleep30-21swapper/310:37:513
26942798,16sleep30-21swapper/310:57:253
2067227811,54sleep0364-21snmpd11:51:280
1924627826,27sleep10-21swapper/111:47:261
272542775,29sleep30-21swapper/307:28:473
272542775,29sleep30-21swapper/307:28:473
1416727720,43sleep214246-21ls09:47:512
10142778,12sleep00-21swapper/012:31:090
1695427622,28sleep10-21swapper/109:57:481
2639527517,42sleep026429-21/usr/sbin/munin10:27:500
1794427522,25sleep00-21swapper/011:42:440
3208927319,35sleep132032-21latency_hist12:27:261
1433527315,38sleep014463-21latency_hist11:32:260
1184727313,44sleep011856-21grep11:22:490
3040227222,31sleep030434-21munin-run10:42:240
2671727219,26sleep00-21swapper/012:07:560
2949727019,34sleep129619-21latency_hist07:47:261
2758327024,20sleep00-21swapper/007:32:270
2758327024,20sleep00-21swapper/007:32:270
2499527022,29sleep10-21swapper/112:03:041
199452700,8sleep20-21swapper/211:47:502
1125727024,24sleep00-21swapper/009:32:270
1125727024,24sleep00-21swapper/009:32:270
306922699,18sleep30-21swapper/310:42:393
3047026920,27sleep00-21swapper/012:22:250
2641626915,31sleep10-21swapper/112:07:481
2350126915,37sleep123593-21/usr/sbin/munin10:17:541
2153626913,34sleep10-21swapper/110:12:451
1486926919,26sleep10-21swapper/109:52:261
3070226819,24sleep20-21swapper/207:57:252
2494226817,24sleep20-21swapper/207:12:312
210012686,13sleep10-21swapper/111:52:301
57022677,42sleep10-21swapper/111:02:521
259092675,12sleep20-21swapper/207:17:362
1636726723,21sleep20-21swapper/209:57:242
111126717,24sleep20-21swapper/212:32:242
680026624,25sleep16819-21modprobe09:02:241
680026624,25sleep16819-21modprobe09:02:241
495926514,37sleep25006-21/usr/sbin/munin11:02:242
2982326512,11sleep20-21swapper/212:17:552
2823926527,21sleep20-21swapper/207:37:262
271926518,22sleep00-21swapper/010:57:260
194652655,47sleep20-21swapper/210:07:012
1588326515,21sleep10-21swapper/109:52:591
63942646,44sleep20-21swapper/211:06:002
319962645,18sleep10-21swapper/108:07:251
290862646,10sleep30-21swapper/307:42:403
260002645,13sleep10-21swapper/107:18:481
2476226411,38sleep124844-21/usr/sbin/munin10:22:471
225412646,11sleep00-21swapper/010:16:360
1514926418,25sleep00-21swapper/011:32:550
1514926418,25sleep00-21swapper/011:32:550
2768426310,24sleep00-21swapper/012:12:440
1721826316,24sleep00-21swapper/009:57:550
156726312,26sleep10-21swapper/110:52:461
1263526322,23sleep00-21swapper/009:42:240
698926218,23sleep10-21swapper/111:07:441
698926218,23sleep10-21swapper/111:07:441
287032625,10sleep30-21swapper/307:40:543
2569726218,30sleep125809-21latency_hist07:17:271
2142726216,31sleep20-21swapper/211:52:482
1273526216,21sleep20-21swapper/211:27:242
5092618,41sleep30-21swapper/308:13:413
3105626124,21sleep10-21swapper/107:58:251
3084526121,20sleep20-21swapper/212:22:432
2933626116,27sleep229394-21/usr/sbin/munin10:37:462
228852615,12sleep00-21swapper/010:17:290
1642626117,26sleep116379-21apt-get11:38:051
151826111,34sleep11669-21sed08:22:271
554326014,25sleep25281-21apt-get08:52:282
3012726017,17sleep10-21swapper/107:52:271
2764426019,21sleep00-21swapper/010:32:430
2036126012,34sleep220457-21sed10:07:542
1107326011,29sleep20-21swapper/211:20:582
966125916,20sleep10-21swapper/109:22:271
749425918,26sleep27570-21sed09:07:262
2629525920,18sleep20-21swapper/210:27:482
2461925922,22sleep20-21swapper/207:12:272
241972595,11sleep30-21swapper/307:07:313
1136625917,22sleep111122-21apt-get09:32:281
1136625917,22sleep111122-21apt-get09:32:281
3270325812,29sleep132218-21/usr/sbin/munin10:47:541
3270325812,29sleep132218-21/usr/sbin/munin10:47:541
3007325816,22sleep20-21swapper/207:52:242
2632425817,20sleep00-21swapper/007:22:260
254912586,41sleep00-21swapper/007:15:440
2396925811,32sleep224029-21latency_hist07:07:262
2189125813,29sleep221975-21sed10:12:552
214525814,18sleep00-21swapper/008:27:270
1680525816,23sleep00-21swapper/011:38:240
1556725814,22sleep00-21swapper/009:52:540
1311125815,28sleep113116-21/usr/sbin/munin11:27:431
883525717,17sleep00-21swapper/011:12:560
2920725714,25sleep129202-21apt-get12:17:371
271682577,18sleep20-21swapper/212:10:402
2602525715,24sleep126020-21apt-get10:27:391
102082576,40sleep20-21swapper/209:27:002
102082576,40sleep20-21swapper/209:27:002
811825615,23sleep18204-21apt-config09:12:251
335025615,17sleep00-21swapper/008:37:240
3206225618,18sleep00-21swapper/012:27:250
2889925616,17sleep20-21swapper/207:42:272
2814225616,19sleep10-21swapper/110:32:561
1978925613,24sleep019856-21df10:07:400
880225515,16sleep20-21swapper/211:12:542
80242555,10sleep20-21swapper/211:12:012
80242555,10sleep20-21swapper/211:12:012
742925516,22sleep00-21swapper/009:07:240
33312555,10sleep10-21swapper/108:37:091
3262425518,20sleep00-21swapper/008:12:250
3200225514,16sleep20-21swapper/210:47:272
3090225518,19sleep10-21swapper/110:42:451
2820725512,19sleep20-21swapper/210:32:572
2550425517,17sleep00-21swapper/012:04:280
209225513,21sleep20-21swapper/208:27:242
1851825519,16sleep20-21swapper/211:42:582
1831825512,18sleep00-21swapper/010:02:430
180125516,21sleep00-21swapper/012:32:500
125112554,10sleep10-21swapper/111:24:321
1160225517,22sleep111122-21apt-get09:32:321
1123825517,16sleep10-21swapper/111:22:261
820525418,19sleep20-21swapper/209:12:262
2696025414,16sleep00-21swapper/007:27:270
2690225414,23sleep226935-21munin-run07:27:252
1832625415,19sleep10-21swapper/110:02:441
150525415,18sleep00-21swapper/008:22:260
1302825417,18sleep20-21swapper/211:27:402
318242535,37sleep00-21swapper/008:05:100
310042536,35sleep00-21swapper/007:57:430
2844725313,23sleep128500-21users12:12:571
2573525318,20sleep225761-21munin-run10:27:242
242932536,36sleep10-21swapper/107:08:471
1267625314,24sleep212686-21munin-run09:42:262
67442525,11sleep20-21swapper/209:01:402
67442525,11sleep20-21swapper/209:01:402
532925214,23sleep01-21systemd08:52:260
3033525226,13sleep00-21swapper/007:52:340
245692529,17sleep00-21swapper/007:12:250
807125112,23sleep08117-21apt-config09:12:240
567225113,22sleep05701-21ntp_states11:02:510
576525011,19sleep10-21swapper/108:52:351
469025010,25sleep14794-21latency_hist08:47:261
306052506,34sleep10-21swapper/107:56:061
265932505,12sleep10-21swapper/107:23:201
1327825017,16sleep00-21swapper/009:47:250
576124912,21sleep25281-21apt-get08:52:322
405824918,15sleep20-21swapper/208:42:262
61082485,35sleep00-21swapper/008:57:020
746224712,18sleep10-21swapper/109:07:251
286962475,32sleep10-21swapper/107:40:481
88824617,15sleep20-21swapper/208:17:272
80824610,23sleep0833-21/usr/sbin/munin08:17:250
3238724612,19sleep132392-21diskstats12:27:431
2815924612,17sleep00-21swapper/007:37:240
129902465,10sleep20-21swapper/209:43:472
275012456,30sleep20-21swapper/207:32:012
275012456,30sleep20-21swapper/207:32:012
2427224514,15sleep20-21swapper/210:22:272
1160624514,18sleep00-21swapper/009:32:350
2948624411,20sleep029496-21munin-run07:47:250
115302446,14sleep20-21swapper/209:32:292
106382445,13sleep20-21swapper/209:27:282
106382445,13sleep20-21swapper/209:27:282
1332324312,18sleep10-21swapper/109:47:261
282012429,15sleep10-21swapper/107:37:251
8772419,18sleep10-21swapper/108:17:261
856924112,15sleep00-21swapper/009:12:320
2845324113,14sleep00-21swapper/007:37:360
3263924010,15sleep10-21swapper/108:12:261
244842398,16sleep20-21swapper/212:02:492
96512388,15sleep00-21swapper/009:22:260
85702386,15sleep10-21swapper/109:12:331
305523612,13sleep20-21swapper/208:33:302
126482368,15sleep10-21swapper/109:42:251
314072358,14sleep20-21swapper/208:02:252
2971123510,14sleep00-21swapper/007:47:380
290732356,11sleep20-21swapper/207:42:312
22482354,11sleep10-21swapper/108:27:281
22482354,11sleep10-21swapper/108:27:281
94732349,15sleep20-21swapper/209:20:382
40082347,13sleep00-21swapper/008:42:240
27232348,13sleep20-21swapper/208:32:252
27232348,13sleep20-21swapper/208:32:252
37062336,15sleep10-21swapper/108:38:261
320372336,13sleep20-21swapper/208:07:262
239022339,12sleep00-21swapper/007:07:240
239032327,13sleep10-21swapper/107:07:251
18132329,13sleep10-21swapper/108:23:441
13122327,14sleep20-21swapper/208:20:262
43622318,12sleep20-21swapper/208:43:482
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional