You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-23 - 05:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Sun Aug 23, 2026 00:43:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
262742337812,3352sleep225766-21apt-get00:24:312
14059277715,743sleep014135-21/usr/sbin/munin23:44:480
23949264613,611sleep016506-1kworker/0:0H22:34:310
23949264613,611sleep016506-1kworker/0:0H22:34:310
759225358,514sleep37633-21ntpq19:59:453
17914240225,353sleep217944-40ogrotate23:59:192
29254240033,270sleep328870-21/usr/sbin/munin22:49:443
13378236558,203sleep313391-21modprobe20:19:193
19691235721,279sleep319845-21latency_hist20:39:223
1576229913,272sleep11581-21ntpq23:04:481
11819229111,208sleep311849-21munin-plugin-st20:14:183
13657228787,176sleep30-21swapper/323:44:383
2903922800,18sleep323482-21cyclictest21:09:213
4493226016,227sleep04538-21/usr/sbin/munin19:49:440
24942224454,106sleep30-21swapper/320:54:443
25918222825,51sleep30-21swapper/319:14:433
29713222510,200sleep029720-21sh21:09:470
2635221613,147sleep32710-21fschecks_time19:44:433
22787221615,184sleep322949-21awk19:04:503
28945221447,154sleep028967-21df22:49:370
17060221370,27sleep30-21swapper/320:29:413
23581221016,52sleep323734-21gpgv22:34:213
23581221016,52sleep323734-21gpgv22:34:213
31631220945,41sleep30-21swapper/319:34:213
25019220917,127sleep30-21swapper/319:14:223
20290220431,131sleep320371-21mailstats00:04:493
32472220315,174sleep332499-21fschecks_count21:19:393
4216219243,91sleep30-21swapper/323:14:343
4216219243,91sleep30-21swapper/323:14:343
28805219018,151sleep128947-21/usr/sbin/munin00:30:071
1604221849,63sleep316122-21fschecks_time22:09:383
3062321737,117sleep330667-21basename22:54:403
8640216812,141sleep28648-21awk21:44:462
21171216814,14sleep30-21swapper/320:44:183
2920216615,136sleep03017-21cat21:29:210
18428216522,122sleep118492-21fschecks_time20:34:391
30982216420,123sleep131101-21timerwakeupswit19:29:511
10248215813,130sleep310395-21cut23:34:213
280221558,61sleep32896-21systemctl21:29:203
28778215212,48sleep323482-21cyclictest19:24:363
17330215122,23sleep30-21swapper/323:54:483
25801214810,98sleep30-21swapper/322:39:383
1556921438,123sleep315606-21ps23:49:473
22073214115,57sleep222212-21sed00:09:532
13271214012,88sleep313302-21idleruntime-cro23:44:183
27672213818,46sleep327558-21latency_hist00:29:203
22644213617,106sleep122651-21open_files22:29:471
22644213617,106sleep122651-21open_files22:29:471
4576213534,47sleep34611-21sed21:34:213
3687213542,82sleep3367-21snmpd23:11:153
13121213217,66sleep313187-21latency21:59:413
24951213164,48sleep30-21swapper/300:19:473
1627213169,40sleep30-21swapper/319:39:503
1589213159,15sleep30-21swapper/323:04:503
6605213014,57sleep36623-21apt-get21:39:303
13579213020,88sleep013721-21unixbench_singl21:59:500
23585212910,81sleep323722-21grep00:14:513
21947212910,44sleep30-21swapper/322:29:213
12610212528,37sleep30-21swapper/321:59:213
32266212316,59sleep332303-21ntp_kernel_pll_19:34:453
29888212228,43sleep329956-21ls00:34:423
7414212034,67sleep323482-21cyclictest23:24:363
4077212014,17sleep00-21swapper/021:29:540
25750212019,81sleep225764-21idleruntime-cro00:24:192
19295212023,46sleep30-21swapper/322:19:403
588821188,43sleep35774-21/usr/sbin/munin23:19:393
27758211812,17sleep00-21swapper/022:44:440
17957211713,89sleep217966-21ntpq22:14:462
27511211654,48sleep327572-21idleruntime22:44:393
12102211615,82sleep012162-21/usr/sbin/munin23:39:390
2307211511,90sleep22378-21cat19:44:212
32069211442,59sleep332135-21modprobe21:19:193
8548211311,48sleep30-21swapper/321:44:433
26144211348,45sleep30-21swapper/320:59:373
21436211214,80sleep021439-21/usr/sbin/munin00:09:390
21834211032,62sleep30-21swapper/300:09:503
15513211021,73sleep115535-21kernelversion20:24:421
983121096,11sleep30-21swapper/323:30:133
32273210910,43sleep30-21swapper/322:59:413
24056210938,19sleep30-21swapper/322:34:383
24056210938,19sleep30-21swapper/322:34:383
7430210817,17sleep10-21swapper/123:24:381
28040210818,71sleep027567-21apt-get00:29:300
24669210713,81sleep024724-21basename00:19:400
1922021055,11sleep30-21swapper/300:00:333
8534210419,61sleep00-21swapper/020:04:220
1396821046,19sleep10-21swapper/120:19:411
23507210310,77sleep222872-21/usr/sbin/munin00:14:502
16019210343,16sleep30-21swapper/320:24:523
29139210219,12sleep30-21swapper/300:31:553
22159210216,28sleep00-21swapper/022:29:300
22159210216,28sleep00-21swapper/022:29:300
406821019,39sleep323482-21cyclictest19:49:313
31386210116,65sleep230778-21/usr/sbin/munin21:14:502
2948529913,69sleep229629-21unixbench_singl19:24:512
317382960,8sleep00-21swapper/019:34:210
2897829513,67sleep228980-21entropy22:49:392
109929514,65sleep21117-21apt-get23:04:322
932629416,63sleep09400-21switchtime20:04:510
1648929411,14sleep10-21swapper/122:09:491
1164329414,67sleep111658-21netstat21:54:421
1030629411,67sleep010398-21cat21:49:490
2952429317,53sleep30-21swapper/321:09:423
267042936,11sleep10-21swapper/122:39:571
1721829345,30sleep0289-21dbus-daemon22:14:200
490529118,50sleep14996-21fschecks_time21:34:391
153292905,10sleep30-21swapper/322:04:583
3055628912,17sleep10-21swapper/121:14:201
2668128921,54sleep10-21swapper/100:25:061
1877328926,45sleep30-21swapper/322:19:193
731328810,61sleep07390-21grep21:40:480
68672887,12sleep10-21swapper/119:58:581
2596828814,56sleep126162-21latency_hist00:24:221
406628714,57sleep225016-21munin-node19:49:302
1537728615,55sleep115381-21kernelversion23:49:411
222542847,15sleep20-21swapper/222:29:362
222542847,15sleep20-21swapper/222:29:362
1519428313,54sleep215241-21cut23:49:382
2937928216,46sleep129454-21mailstats22:49:501
2415728217,43sleep10-21swapper/119:09:481
89792819,52sleep30-21swapper/320:04:413
1370128119,46sleep30-21swapper/320:19:353
2642428021,41sleep226117-21/usr/sbin/munin20:59:442
2337828019,28sleep20-21swapper/220:49:422
848127915,45sleep18483-21cron20:04:191
2848727926,40sleep328513-21munin-run19:24:203
2532927912,14sleep20-21swapper/219:14:252
238127924,34sleep00-21swapper/023:09:190
2146327917,46sleep30-21swapper/320:44:353
1107627917,47sleep211201-21sed23:34:482
1107627917,47sleep211201-21sed23:34:482
75527813,46sleep2811-21latency_hist19:39:212
43827824,26sleep10-21swapper/121:19:501
909927714,24sleep20-21swapper/223:29:392
909927714,24sleep20-21swapper/223:29:392
271632776,14sleep00-21swapper/021:01:110
1345927715,44sleep113535-21sed20:19:211
667027619,12sleep30-21swapper/319:56:263
3169827620,38sleep131725-21munin-run22:59:191
1090727618,11sleep30-21swapper/321:53:153
67302757,15sleep20-21swapper/221:39:572
325072750,8sleep20-21swapper/222:59:472
302127521,33sleep12765-21apt-get21:29:221
3008727516,45sleep030105-21latency_hist19:29:210
2407327518,10sleep30-21swapper/320:51:413
2336827521,34sleep123397-21munin-run19:09:201
2265027525,30sleep10-21swapper/100:14:201
2000527522,28sleep10-21swapper/120:39:401
1062127520,23sleep00-21swapper/023:34:380
1062127520,23sleep00-21swapper/023:34:380
1989027424,25sleep10-21swapper/100:04:391
48172738,49sleep30-21swapper/321:34:333
296427314,43sleep13001-21ntp_states19:44:491
2640527316,37sleep226488-21timerandwakeup22:39:542
19182736,11sleep00-21swapper/019:39:590
69762729,16sleep00-21swapper/019:59:210
2832127218,36sleep127674-21/usr/sbin/munin21:04:521
138727219,34sleep11424-21awk19:39:441
2860227121,30sleep228631-21munin-run22:49:182
138182716,36sleep10-21swapper/123:44:401
898927012,33sleep19098-21perl23:29:381
898927012,33sleep19098-21perl23:29:381
548227021,32sleep25483-21cron23:19:182
548227021,32sleep25483-21cron23:19:182
2518027016,39sleep125304-21sed20:54:501
1070427022,23sleep00-21swapper/020:10:120
556026913,35sleep05625-21cat23:19:200
556026913,35sleep05625-21cat23:19:200
1771226921,25sleep00-21swapper/022:14:400
1248226917,32sleep10-21swapper/123:39:481
1139526918,41sleep30-21swapper/323:35:263
1139526918,41sleep30-21swapper/323:35:263
923326814,27sleep20-21swapper/220:04:482
2824726819,26sleep00-21swapper/021:04:510
949226715,28sleep00-21swapper/021:49:200
2530526721,23sleep20-21swapper/220:54:512
2336326713,25sleep00-21swapper/000:14:480
2071926719,26sleep10-21swapper/122:24:381
3130826611,18sleep10-21swapper/121:14:491
2541026621,26sleep20-21swapper/222:39:212
2541026621,26sleep20-21swapper/222:39:212
1468026616,23sleep00-21swapper/022:04:430
1195926617,26sleep20-21swapper/221:54:522
865526515,25sleep00-21swapper/023:29:210
67926513,36sleep1754-21systemd-run19:39:201
644126516,25sleep20-21swapper/223:19:502
2938226515,30sleep129484-21/usr/sbin/munin19:24:501
1909026512,21sleep219157-21df22:19:352
1871026522,23sleep218755-21awk20:34:452
421426416,29sleep14208-21apt-get23:14:321
421426416,29sleep14208-21apt-get23:14:321
3096826421,23sleep20-21swapper/200:39:212
2806326414,33sleep228069-21/usr/sbin/munin21:04:442
1786826415,34sleep117869-21sshd20:31:571
197626313,32sleep22055-21/usr/sbin/munin21:24:472
1882026317,19sleep20-21swapper/223:59:522
2007126213,20sleep20-21swapper/220:39:412
1592226212,32sleep216018-21timerwakeupswit20:24:512
1106826217,24sleep10-21swapper/120:10:211
1098926219,25sleep211000-21modprobe21:54:192
771226113,22sleep20-21swapper/223:24:432
620226118,23sleep20-21swapper/221:39:212
540326114,30sleep05495-21/usr/sbin/munin21:34:510
3247426115,30sleep10-21swapper/122:59:471
303226113,28sleep03094-21/usr/sbin/munin19:44:520
2978926121,18sleep10-21swapper/100:34:401
2425326112,23sleep10-21swapper/100:19:211
2017226114,32sleep220201-21/usr/sbin/munin00:04:442
1885326112,37sleep118880-21latency_hist22:19:211
1531926113,18sleep00-21swapper/020:24:370
2720426014,24sleep10-21swapper/119:19:291
264226014,20sleep20-21swapper/223:09:292
2436226019,18sleep10-21swapper/120:54:221
2277126028,15sleep00-21swapper/020:49:200
1416426018,25sleep214139-21latency_hist22:04:202
1136226014,20sleep00-21swapper/021:54:370
469525915,30sleep14790-21sed19:49:491
3056525911,35sleep230618-21cat21:14:212
2698625914,25sleep10-21swapper/119:19:211
2590425914,24sleep00-21swapper/020:59:210
2454425918,18sleep10-21swapper/122:34:521
2454425918,18sleep10-21swapper/122:34:521
2100125911,19sleep20-21swapper/222:24:442
2063425915,27sleep020629-21apt-get22:24:330
1449725915,22sleep10-21swapper/122:04:401
579825814,27sleep15793-21apt-get23:19:331
50925811,30sleep00-21swapper/023:00:190
37522585,10sleep20-21swapper/219:48:412
324225819,25sleep12765-21apt-get21:29:311
262862586,37sleep00-21swapper/000:24:370
2235025816,22sleep222379-21irqstats19:04:412
200012584,12sleep00-21swapper/022:19:530
981525712,32sleep19864-21grep21:49:381
3238725713,27sleep232383-21apt-get21:19:342
3069225715,19sleep20-21swapper/219:29:442
2211125710,32sleep00-21swapper/019:04:350
161452575,41sleep00-21swapper/023:51:300
1405825717,20sleep20-21swapper/220:19:422
418725610,21sleep20-21swapper/221:29:562
3230525611,28sleep032324-21ntp_states19:34:460
3041625615,16sleep20-21swapper/222:54:312
284482565,12sleep10-21swapper/122:47:201
2369425615,24sleep223691-21apt-get19:09:332
1837325613,18sleep00-21swapper/023:59:410
132142564,10sleep00-21swapper/020:17:100
794325518,20sleep10-21swapper/121:44:211
2962125516,25sleep029614-21apt-get00:34:340
2759225512,22sleep20-21swapper/219:19:422
2740725514,18sleep10-21swapper/121:04:191
2037325514,17sleep00-21swapper/000:04:500
117252554,10sleep20-21swapper/220:13:042
877325411,25sleep18767-21apt-get20:04:351
844925415,22sleep10-21swapper/121:44:411
2626825419,20sleep00-21swapper/022:39:520
2024025413,25sleep120241-21sshd22:22:571
1665025412,25sleep116647-21apt-get23:54:301
1595625413,24sleep215950-21apt-get22:09:332
91652539,17sleep00-21swapper/023:29:420
91652539,17sleep00-21swapper/023:29:420
3161225312,23sleep031703-21users21:14:520
2852825313,27sleep028537-21systemd-run19:24:210
2827225317,22sleep20-21swapper/200:29:562
273825310,29sleep02746-21df23:09:350
2562225313,19sleep00-21swapper/019:14:320
224122536,38sleep20-21swapper/220:45:222
2157425312,23sleep121670-21perl20:44:411
129492538,33sleep20-21swapper/223:40:092
1051125312,16sleep10-21swapper/123:34:351
1051125312,16sleep10-21swapper/123:34:351
1745925213,21sleep017551-21users23:54:490
1274225213,24sleep212869-21timerwakeupswit20:14:502
721725114,17sleep20-21swapper/219:59:362
1571825116,19sleep20-21swapper/222:09:212
2432725010,19sleep00-21swapper/020:54:210
1687925018,17sleep00-21swapper/020:29:380
1383325012,21sleep213926-21memory23:44:412
124725019,18sleep00-21swapper/021:24:210
2018824910,16sleep00-21swapper/020:39:430
1806224912,22sleep218100-21unin-run20:34:202
1322124914,22sleep113261-21ntp_kernel_err21:59:431
597824813,18sleep20-21swapper/219:54:432
3044024811,21sleep030435-21apt-get22:54:330
1411824811,22sleep014150-21ntp_states20:19:450
710724710,16sleep00-21swapper/023:24:210
311424710,22sleep13120-21ntp_states23:09:441
1846024712,15sleep10-21swapper/123:59:421
2397224610,19sleep024062-21memory19:09:430
1834224310,15sleep00-21swapper/020:34:340
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional