You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-31 - 03:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Tue Mar 31, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
265392137812,1353sleep124660-21kworker/1:122:31:381
3226727008,679sleep232304-21ntpq19:22:382
23030246315,432sleep123033-21/usr/sbin/munin00:02:331
29223237313,344sleep229287-21mailstats19:12:402
20997236620,322sleep320883-40mandb23:57:143
27719230090,174sleep20-21swapper/200:17:312
6874226497,27sleep226573-21cyclictest21:27:412
29655226224,46sleep226573-21cyclictest00:22:402
24795225711,232sleep224803-21apt-config22:24:332
19309225341,195sleep20-21swapper/223:52:122
733222397,220sleep27340-21ntpq23:12:412
3485223857,65sleep23524-21modprobe23:02:112
28824223316,201sleep128918-21cut19:12:311
13425223215,196sleep013447-21fschecks_count23:32:330
32585223036,180sleep232599-21cpu21:07:312
32585223036,180sleep232599-21cpu21:07:312
17256222513,195sleep317230-21latency_hist22:02:143
26808221411,156sleep226848-21uname20:47:402
26808221411,156sleep226848-21uname20:47:402
3235322046,13sleep00-21swapper/000:32:300
2891421868,165sleep028922-21ntpq22:37:390
23325218620,84sleep123341-21fschecks_count20:37:341
23407218225,136sleep20-21swapper/200:02:422
14598218016,145sleep214637-21run-parts23:37:112
21660217316,30sleep20-21swapper/220:32:262
22272217140,114sleep20-21swapper/222:17:302
1985217110,58sleep20-21swapper/219:28:152
26880216932,24sleep20-21swapper/222:32:282
3332216314,28sleep20-21swapper/219:32:432
30988216321,35sleep231007-21kernelversion00:27:322
17962216010,132sleep118000-21processes22:02:401
30782215892,50sleep20-21swapper/221:02:142
10547215115,119sleep20-21swapper/219:57:142
10511214916,112sleep20-21swapper/223:22:422
31549214640,42sleep20-21swapper/222:47:262
4104214522,100sleep14255-21latency_hist19:37:141
6201214310,120sleep26203-21/usr/sbin/munin19:42:322
6201214310,120sleep26203-21/usr/sbin/munin19:42:322
32013214118,90sleep00-21swapper/022:47:400
28042214016,107sleep228081-21modprobe19:12:122
20918214022,103sleep21-21systemd23:57:132
26401213921,47sleep20-21swapper/200:12:352
1612121369,59sleep20-21swapper/221:57:322
3234213316,98sleep20-21swapper/221:17:242
11351213214,19sleep20-21swapper/221:42:312
19021213149,15sleep20-21swapper/222:07:252
3751212812,49sleep226573-21cyclictest23:02:242
32343212414,95sleep232469-21grep21:07:142
32343212414,95sleep232469-21grep21:07:142
27437212318,86sleep127336-21latency_hist00:17:141
30810212210,49sleep230879-21mailstats19:17:412
30810212210,49sleep230879-21mailstats19:17:412
20720212218,89sleep220799-21timerandwakeup20:27:422
6088212010,90sleep20-21swapper/221:27:132
22115212020,71sleep122260-21unixbench_singl23:57:531
22115212020,71sleep122260-21unixbench_singl23:57:531
15220211810,15sleep00-21swapper/023:37:380
26786211714,16sleep00-21swapper/020:47:380
26786211714,16sleep00-21swapper/020:47:380
1014921167,62sleep20-21swapper/221:37:382
18934211411,39sleep218544-21/usr/sbin/munin20:22:372
1244211415,75sleep31454-21apt-config00:37:143
12227211210,38sleep20-21swapper/223:27:432
1860521117,50sleep00-21swapper/022:05:170
16831211111,86sleep216440-21/usr/sbin/munin23:42:362
2922621108,59sleep229316-21latency_hist20:57:132
2922621108,59sleep229316-21latency_hist20:57:132
6084210819,68sleep15600-21apt-get19:42:231
6084210819,68sleep15600-21apt-get19:42:231
19661210819,71sleep119794-21grep22:07:451
1465221089,32sleep20-21swapper/220:07:432
20830210719,70sleep120883-40mandb23:57:121
2066621077,33sleep20-21swapper/222:12:302
26508210236,46sleep026531-21latency_hist19:07:130
25116210263,20sleep20-21swapper/200:07:422
11053210222,65sleep20-21swapper/219:57:342
459929912,72sleep14672-21sed19:37:341
54522987,81sleep00-21swapper/019:40:280
2224529820,64sleep322251-21df_inode22:17:273
91129725,60sleep20-21swapper/222:52:352
1512829724,57sleep20-21swapper/223:37:362
386729610,67sleep13969-21/usr/sbin/munin21:17:431
1541029524,55sleep20-21swapper/220:12:242
1463529511,63sleep20-21swapper/221:52:362
2866429411,63sleep20-21swapper/222:37:332
117829315,56sleep11248-21sensors_temp22:52:421
635329221,10sleep20-21swapper/223:09:142
560029213,64sleep05637-21grep23:07:380
3227629211,59sleep20-21swapper/200:32:282
3009229214,57sleep329977-21/usr/sbin/munin22:42:313
2936729210,65sleep029497-21cat19:12:420
1688529219,57sleep016889-21modprobe21:58:110
2928929115,62sleep329364-21sort19:12:413
271629120,11sleep20-21swapper/221:14:212
271629120,11sleep20-21swapper/221:14:212
2633629119,10sleep20-21swapper/222:29:052
123402918,17sleep20-21swapper/220:02:262
931929020,10sleep20-21swapper/221:36:092
3099329018,10sleep20-21swapper/222:43:472
230529021,49sleep12384-21fschecks_time22:57:301
1801129022,56sleep20-21swapper/222:02:412
86352899,63sleep20-21swapper/223:17:292
28912897,73sleep00-21swapper/021:16:330
28912897,73sleep00-21swapper/021:16:330
2249628919,10sleep20-21swapper/200:00:122
2249628919,10sleep20-21swapper/200:00:122
880928819,11sleep20-21swapper/219:50:442
880928819,11sleep20-21swapper/219:50:442
656428813,23sleep00-21swapper/021:27:230
1722528819,54sleep20-21swapper/220:17:322
480728721,40sleep14801-21apt-get21:22:271
3119328713,61sleep031239-21processes00:27:380
270582877,68sleep00-21swapper/000:14:020
2584628720,56sleep20-21swapper/220:44:422
251428724,26sleep10-21swapper/119:32:131
535328616,10sleep20-21swapper/219:39:132
1809528616,50sleep317988-21/usr/sbin/munin23:47:293
67228514,55sleep0701-21munin-run19:27:120
2576028514,49sleep325837-21switchtime19:02:413
2454228513,55sleep024466-21/usr/sbin/munin00:07:280
1281528548,21sleep20-21swapper/221:47:272
3057728418,50sleep130661-21switchtime22:42:411
2824428318,14sleep30-21swapper/322:37:143
2716128312,56sleep127245-21/usr/sbin/munin22:32:351
1130228319,39sleep00-21swapper/023:27:120
2894128217,42sleep029064-21latency_hist00:22:130
2809328224,38sleep028218-21latency_hist19:12:140
1810728122,39sleep018214-21fschecks_time23:47:300
1184428111,17sleep10-21swapper/121:42:421
2436828015,11sleep20-21swapper/220:40:342
2381428013,48sleep023874-21fschecks_time22:22:330
1473328013,52sleep014779-21/usr/sbin/munin21:52:380
128352808,45sleep30-21swapper/321:47:283
457427916,27sleep00-21swapper/021:22:140
1777227915,46sleep317862-21ls23:47:133
2109627813,17sleep10-21swapper/122:12:411
1347027820,30sleep30-21swapper/323:32:403
976127713,16sleep10-21swapper/123:22:131
396727711,52sleep34025-21if_eth023:02:333
3236027712,46sleep132388-21fschecks_count00:32:311
322112777,17sleep00-21swapper/021:06:200
322112777,17sleep00-21swapper/021:06:200
22172777,51sleep30-21swapper/322:57:243
71727616,46sleep1741-21munin-run19:27:131
1562227622,38sleep115631-21modprobe21:57:111
312327523,27sleep00-21swapper/019:32:370
2609027516,46sleep326133-21date20:47:133
2519727515,43sleep225264-21df19:02:272
2135627517,38sleep320843-21apt-get23:57:223
2135627517,38sleep320843-21apt-get23:57:223
151027518,40sleep01572-21cat19:28:010
1180827517,28sleep10-21swapper/123:27:331
628827415,26sleep00-21swapper/021:27:150
1287527415,37sleep012989-21dpkg23:32:130
3170527318,26sleep10-21swapper/121:02:451
3170527318,26sleep10-21swapper/121:02:451
1914327328,28sleep00-21swapper/020:22:430
823727217,41sleep08335-21fschecks_time21:32:320
3276627220,25sleep30-21swapper/321:07:363
3276627220,25sleep30-21swapper/321:07:363
2517027217,40sleep125186-21awk19:02:221
1415827212,42sleep114247-21/usr/sbin/munin20:07:341
2914627121,32sleep029170-21munin-run20:57:110
2914627121,32sleep029170-21munin-run20:57:110
2809427121,31sleep30-21swapper/300:17:403
275427112,33sleep32746-21apt-get19:32:283
1124627120,34sleep311261-21/usr/sbin/munin19:57:393
921227020,23sleep10-21swapper/119:52:281
250412700,7sleep10-21swapper/100:07:401
2373727015,35sleep323213-21/usr/sbin/munin20:37:443
1176727015,24sleep00-21swapper/021:42:410
85842698,45sleep00-21swapper/019:47:510
85842698,45sleep00-21swapper/019:47:510
3049626916,23sleep00-21swapper/022:42:400
2559126914,24sleep30-21swapper/322:27:373
713926822,25sleep10-21swapper/121:27:481
3266726813,42sleep132670-21entropy21:07:341
3266726813,42sleep132670-21entropy21:07:341
1209526819,20sleep10-21swapper/120:02:131
1164526820,30sleep311746-21fschecks_time23:27:313
2185826719,26sleep321883-21munin-run22:17:113
1921626714,21sleep10-21swapper/120:22:441
1308326719,29sleep113160-21latency21:47:341
2679926617,25sleep10-21swapper/120:47:391
2679926617,25sleep10-21swapper/120:47:391
2292026616,32sleep322940-21latency_hist20:37:123
1827726615,35sleep118279-21cron20:22:121
1356026615,22sleep10-21swapper/123:32:421
309352659,24sleep10-21swapper/100:27:311
2714426513,21sleep00-21swapper/022:32:340
72552647,47sleep3758-21sendmail-mta19:45:403
72552647,47sleep3758-21sendmail-mta19:45:403
562426419,27sleep35649-21munin-run19:42:133
3055626413,24sleep30-21swapper/320:59:573
1673626415,22sleep10-21swapper/123:42:351
1426326419,27sleep314308-21ntpq20:07:363
1046826421,27sleep010498-21munin-run19:57:110
97026312,33sleep31009-21memory22:52:363
3194526316,34sleep131982-21sed22:47:371
1770226315,21sleep00-21swapper/020:17:430
1396126315,31sleep013967-21df_inode20:07:290
2539126212,22sleep00-21swapper/020:42:420
246412625,12sleep30-21swapper/322:22:483
2384526215,28sleep323857-21vmstat00:02:473
2382026218,22sleep00-21swapper/020:37:450
165162625,11sleep10-21swapper/120:14:491
1564026217,25sleep30-21swapper/320:12:333
1518526219,19sleep30-21swapper/323:37:373
68426113,21sleep00-21swapper/021:07:450
68426113,21sleep00-21swapper/021:07:450
3109426115,18sleep30-21swapper/321:02:313
3109426115,18sleep30-21swapper/321:02:313
2990826118,24sleep10-21swapper/100:22:431
2019826115,21sleep30-21swapper/320:27:313
1366726113,31sleep013720-21users21:47:450
1060326117,24sleep30-21swapper/323:22:433
832626011,23sleep30-21swapper/319:47:423
832626011,23sleep30-21swapper/319:47:423
742826022,22sleep00-21swapper/019:47:140
742826022,22sleep00-21swapper/019:47:140
727326012,36sleep0357-21ntpd23:12:390
713426018,30sleep17153-21uname23:12:361
2944926017,29sleep029514-21latency00:22:340
2486726016,24sleep124924-21grep20:42:311
2348826019,20sleep10-21swapper/122:22:141
2081826020,23sleep020829-21modprobe23:57:110
206872608,23sleep00-21swapper/022:12:320
1678326011,33sleep016898-21sed20:17:140
1602526015,20sleep10-21swapper/121:57:311
1436526012,20sleep30-21swapper/321:52:293
128826012,27sleep30-21swapper/319:27:313
1018926021,17sleep10-21swapper/121:37:411
999125916,22sleep30-21swapper/321:37:353
892125911,25sleep10-21swapper/119:52:121
892125911,25sleep10-21swapper/119:52:121
3032725912,30sleep130396-21df19:17:281
3032725912,30sleep130396-21df19:17:281
2764825922,19sleep30-21swapper/320:52:143
2764825922,19sleep30-21swapper/320:52:143
2446525916,26sleep324484-21apt-get00:07:233
2136025913,22sleep00-21swapper/023:57:230
2136025913,22sleep00-21swapper/023:57:230
663425818,22sleep30-21swapper/323:12:143
614225814,24sleep36278-21cut21:27:143
402825815,24sleep04046-21irqstats23:02:340
2541125817,21sleep00-21swapper/022:27:340
2349625818,22sleep00-21swapper/000:02:440
1287925813,32sleep112953-21swap20:02:421
1280325821,20sleep30-21swapper/320:02:403
101125821,19sleep00-21swapper/022:52:370
972225715,18sleep00-21swapper/021:37:280
3203525719,20sleep30-21swapper/300:32:133
1800925716,25sleep118006-21apt-get23:47:231
541725611,27sleep15477-21/usr/sbin/munin23:07:351
3230625615,25sleep332325-21ntp_states19:22:393
3059325618,17sleep30-21swapper/319:17:343
3059325618,17sleep30-21swapper/319:17:343
3032425616,24sleep030326-21cpu19:17:270
3032425616,24sleep030326-21cpu19:17:270
297002565,11sleep30-21swapper/322:42:023
2944025613,26sleep029459-21apt-get20:57:230
2649425613,26sleep126500-21ntp_states00:12:381
2187025617,21sleep10-21swapper/120:32:331
2985425513,26sleep129892-21/usr/sbin/munin20:57:361
814125414,24sleep38136-21apt-get21:32:263
3182125412,24sleep331915-21memory22:47:353
2225325413,18sleep00-21swapper/022:17:280
1281925413,25sleep012877-21sendmail_mailqu20:02:410
928625312,23sleep39295-21diskstats19:52:303
2419625314,25sleep123700-21/usr/sbin/munin22:22:421
2032225311,20sleep10-21swapper/120:27:331
1823225313,21sleep30-21swapper/320:21:373
164825316,20sleep31667-21apt-get21:12:243
164825316,20sleep31667-21apt-get21:12:243
1492225316,19sleep10-21swapper/123:37:301
660425214,24sleep36615-21grep21:27:343
456725214,23sleep30-21swapper/321:22:133
221925212,23sleep02213-21apt-get22:57:250
2808225115,20sleep328091-21munin-run19:12:133
2796225113,21sleep327968-21diskstats20:52:303
2796225113,21sleep327968-21diskstats20:52:303
217025112,26sleep12238-21tr21:12:421
217025112,26sleep12238-21tr21:12:421
2011525112,23sleep020182-21df20:27:280
1905525118,17sleep30-21swapper/320:22:423
1904225112,23sleep319040-21apt-get22:07:263
1078025113,22sleep010777-21apt-get19:57:240
2939825012,20sleep329447-21irqstats00:22:333
2845725016,16sleep10-21swapper/122:37:281
2787725012,22sleep127880-21apt-get20:52:241
2787725012,22sleep127880-21apt-get20:52:241
1809725014,21sleep30-21swapper/322:02:423
175602505,37sleep00-21swapper/023:45:080
851824913,18sleep10-21swapper/121:32:371
845324911,22sleep08447-21apt-get23:17:230
2604124911,23sleep326110-21df00:12:283
1000724916,15sleep10-21swapper/123:22:291
869424811,21sleep38722-21irqstats23:17:303
503624813,21sleep15064-21munin-run23:07:111
2746424814,17sleep10-21swapper/119:07:431
216632488,14sleep00-21swapper/020:32:280
1124324812,19sleep311261-21apt-get21:42:233
822024715,19sleep38314-21sed23:17:143
1946324712,20sleep019500-21ntp_states22:07:400
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional