You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-06 - 19:21

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Sun Sep 06, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
137102171010,1689sleep119677-1kworker/1:1H10:58:381
15288284117,804sleep115301-21/usr/sbin/munin09:18:551
20119244668,360sleep220307-21latency_hist07:53:292
10135234215,312sleep028031-21kworker/u8:209:03:440
10135234215,312sleep028031-21kworker/u8:209:03:440
22193228684,177sleep222232-21fschecks_count07:58:462
28581226430,214sleep228643-21latency_hist10:03:282
30605225830,211sleep230753-21latency_hist11:53:292
26929225317,216sleep227079-21unixbench_singl11:39:002
308512247132,19sleep20-21swapper/210:08:572
308512247132,19sleep20-21swapper/210:08:572
14505222652,153sleep25441-21cyclictest09:18:272
11708221129,145sleep211744-21ntp_kernel_err12:33:542
11708221129,145sleep211744-21ntp_kernel_err12:33:542
989022096,193sleep2367-21snmpd10:45:552
2054322079,177sleep20-21swapper/207:53:502
20816220615,167sleep220966-21latency_hist09:38:292
32016220314,171sleep232110-21switchtime08:28:582
19029220215,158sleep20-21swapper/211:13:572
3643219943,19sleep20-21swapper/212:08:512
32624218411,159sleep232632-21basename11:58:372
3362218015,145sleep21-21systemd08:43:272
3362218015,145sleep21-21systemd08:43:272
1532521809,154sleep215389-21sendmail-msp07:38:282
541421789,19sleep212-21rcu_sched08:48:482
16013217718,24sleep20-21swapper/207:38:522
1530221759,153sleep215363-21find09:18:562
760721739,144sleep20-21swapper/207:13:452
10795217310,147sleep210917-21sed09:04:142
10795217310,147sleep210917-21sed09:04:142
1516217021,133sleep21557-21proc_pri10:18:542
17669216810,145sleep2289-21dbus-daemon09:28:282
15004216728,16sleep20-21swapper/211:03:272
32180216323,123sleep30-21swapper/310:13:513
12387216330,99sleep212449-21processes10:53:542
550621629,83sleep20-21swapper/212:13:522
17383216239,22sleep20-21swapper/211:08:562
28888216150,23sleep20-21swapper/208:18:572
4689216018,128sleep34756-21swap10:28:573
898021568,29sleep20-21swapper/208:58:572
2113421515,10sleep20-21swapper/211:22:122
2267621508,37sleep20-21swapper/209:43:462
2267621508,37sleep20-21swapper/209:43:462
25665214835,48sleep10-21swapper/109:53:381
1518214813,121sleep01559-21cut12:03:280
5895214510,88sleep20-21swapper/210:33:482
2482821457,123sleep224988-21awk09:48:592
2482821457,123sleep224988-21awk09:48:592
3213214416,106sleep03316-21timerandwakeup10:23:570
31237214011,95sleep231280-21ntp_kernel_pll_11:53:532
31237214011,95sleep231280-21ntp_kernel_pll_11:53:532
1137214021,95sleep11150-21fschecks_count10:18:451
1168521399,92sleep211823-21latency_hist10:53:302
30835213717,104sleep130850-21/usr/sbin/munin10:08:561
30835213717,104sleep130850-21/usr/sbin/munin10:08:561
2574921355,36sleep20-21swapper/211:36:452
9598213219,21sleep30-21swapper/310:43:583
16131213225,89sleep30-21swapper/309:23:293
709321309,77sleep20-21swapper/212:18:522
1766821308,85sleep217727-21sendmail_mailqu07:43:572
881321299,81sleep20-21swapper/212:23:572
881321299,81sleep20-21swapper/212:23:572
21654212328,26sleep20-21swapper/209:38:582
3082211815,84sleep23143-21wc10:23:552
2926921188,18sleep20-21swapper/210:03:532
2360211815,82sleep00-21swapper/010:23:280
32143211645,47sleep01-21systemd11:58:270
32143211645,47sleep01-21systemd11:58:270
4866211421,78sleep15009-21sed10:28:591
3015211465,35sleep23046-21basename12:08:272
28482211316,82sleep328628-21unixbench_singl11:43:573
6017211212,56sleep20-21swapper/207:08:522
3021211216,23sleep30-21swapper/310:23:523
19866211213,72sleep319443-21/usr/sbin/munin09:33:543
2243321108,17sleep212-21rcu_sched11:24:022
12987211018,75sleep312991-21true09:13:273
1150021096,91sleep00-21swapper/010:51:420
24972210813,79sleep125065-21latency_hist08:08:301
778321079,59sleep20-21swapper/210:38:572
809321058,86sleep28097-21munin-run08:58:292
2587621058,54sleep20-21swapper/209:53:472
830210413,75sleep1984-21latency_hist10:18:291
1361321045,64sleep20-21swapper/207:31:212
5096210316,43sleep30-21swapper/312:13:403
2387321038,37sleep20-21swapper/211:28:582
989921029,34sleep20-21swapper/207:19:182
989921029,34sleep20-21swapper/207:19:182
32427210211,56sleep20-21swapper/210:13:582
3047210112,69sleep33118-21apt-key12:08:283
2883621005,59sleep20-21swapper/211:46:182
1170121007,54sleep20-21swapper/209:08:462
11449210042,28sleep011514-21grep07:23:560
11449210042,28sleep011514-21grep07:23:560
626529916,61sleep16397-21grep07:08:591
2803629912,71sleep21-21systemd08:18:282
41452988,52sleep20-21swapper/210:28:392
2652529813,70sleep126671-21latency_hist08:13:311
211529814,67sleep02165-21cpuspeed08:38:450
211529814,67sleep02165-21cpuspeed08:38:450
939629721,57sleep38887-21apt-get07:18:383
939629721,57sleep38887-21apt-get07:18:383
2797829720,52sleep127717-21/usr/sbin/munin11:43:471
2268129715,68sleep322757-21fschecks_time09:43:473
2268129715,68sleep322757-21fschecks_time09:43:473
100592967,32sleep20-21swapper/212:28:492
620129516,22sleep00-21swapper/007:08:580
325902959,73sleep0367-21snmpd08:32:070
2533229415,63sleep125477-21unixbench_singl11:33:581
1840729361,15sleep20-21swapper/209:28:572
827929220,57sleep18402-21timerwakeupswit07:14:001
365929220,30sleep20-21swapper/208:43:432
80529115,62sleep0752-21latency_hist10:18:280
1105729142,32sleep20-21swapper/207:23:462
1105729142,32sleep20-21swapper/207:23:462
2926629015,62sleep329274-21acpi11:48:373
266092880,7sleep00-21swapper/011:38:520
1138928818,49sleep311429-21munin-run09:08:273
1138928818,49sleep311429-21munin-run09:08:273
53528722,30sleep20-21swapper/208:33:422
299662876,29sleep20-21swapper/208:23:452
254212878,14sleep20-21swapper/208:08:472
3058928613,57sleep10-21swapper/111:53:281
162592865,27sleep20-21swapper/211:04:402
155928619,49sleep31631-21mailstats10:18:553
303692859,17sleep30-21swapper/310:08:423
303692859,17sleep30-21swapper/310:08:423
2276128514,51sleep022814-21idleruntime09:43:480
2276128514,51sleep022814-21idleruntime09:43:480
729628419,38sleep07374-21fschecks_count07:13:340
3209628420,43sleep332115-21apt-get11:58:263
3209628420,43sleep332115-21apt-get11:58:263
2231728425,28sleep00-21swapper/007:58:480
2230128429,30sleep30-21swapper/309:43:273
737028312,54sleep17444-21find08:54:001
1319428326,25sleep30-21swapper/310:58:273
800128215,54sleep38039-21ntpq07:13:543
116902827,15sleep10-21swapper/109:08:411
2304428118,42sleep17051-21munin-node11:28:371
1142628117,31sleep10-21swapper/112:33:491
1142628117,31sleep10-21swapper/112:33:491
90928029,25sleep30-21swapper/311:59:153
1791728019,46sleep118055-21unixbench_singl07:44:001
1587128014,52sleep00-21swapper/009:20:430
1141028021,42sleep311419-21/usr/sbin/munin12:33:473
1141028021,42sleep311419-21/usr/sbin/munin12:33:473
695427930,24sleep30-21swapper/307:13:293
2179827913,49sleep021867-21unin-run07:58:280
716027818,22sleep10-21swapper/112:18:551
557227812,53sleep15636-21ps12:13:551
2776127810,54sleep0367-21snmpd08:15:110
266742786,21sleep30-21swapper/311:38:573
2387627817,44sleep323926-21if_eth008:03:523
804227715,18sleep00-21swapper/012:23:300
1887127715,48sleep018950-21basename11:13:550
1673827713,43sleep016345-21/usr/sbin/munin09:23:500
51327611,50sleep1532-21apt-get08:33:411
2366627627,38sleep223658-21apt-get08:03:432
1573127625,32sleep115747-21sort07:38:471
695927514,47sleep36981-21kernelversion12:18:493
189532750,8sleep10-21swapper/107:48:471
2159927311,46sleep021494-21/usr/sbin/munin11:23:440
278027224,17sleep30-21swapper/308:39:003
278027224,17sleep30-21swapper/308:39:003
2577727215,37sleep025784-21/usr/sbin/munin09:53:450
2390527215,43sleep124025-21date09:48:291
2390527215,43sleep124025-21date09:48:291
3037927123,26sleep10-21swapper/108:23:561
964427021,24sleep10-21swapper/109:03:281
81127020,32sleep0873-21cat08:33:520
460527029,20sleep10-21swapper/107:03:551
406327027,21sleep00-21swapper/007:03:420
35682706,14sleep10-21swapper/110:24:431
265082707,49sleep30-21swapper/308:13:293
2141827013,43sleep00-21swapper/009:38:520
2052627018,36sleep320521-21apt-get07:53:423
414526918,27sleep30-21swapper/307:03:453
3152426926,21sleep30-21swapper/308:28:473
2942026913,29sleep00-21swapper/011:48:460
1160926914,30sleep00-21swapper/012:33:520
1160926914,30sleep00-21swapper/012:33:520
774426814,40sleep17780-21ps10:38:561
579826821,26sleep15892-21fschecks_time10:33:471
3265926811,16sleep10-21swapper/111:58:451
308332686,14sleep00-21swapper/008:24:240
1844926825,23sleep00-21swapper/007:48:270
1838826819,23sleep30-21swapper/311:13:413
1800726816,20sleep00-21swapper/009:28:460
1746126815,24sleep30-21swapper/307:43:493
261652676,15sleep00-21swapper/008:09:340
406026619,29sleep24056-21apt-get07:03:402
3084326617,31sleep130910-21/usr/sbin/munin11:53:441
3084326617,31sleep130910-21/usr/sbin/munin11:53:441
2802426619,32sleep128034-21modprobe08:18:271
2594926620,19sleep30-21swapper/311:38:283
2139726616,31sleep121451-21users07:54:131
205992666,12sleep10-21swapper/109:36:471
1543426616,20sleep10-21swapper/107:38:311
485826518,26sleep00-21swapper/012:13:290
2024326513,26sleep30-21swapper/311:18:503
202826414,24sleep00-21swapper/012:03:480
1151626415,20sleep10-21swapper/107:23:571
1151626415,20sleep10-21swapper/107:23:571
646726314,32sleep16495-21munin-run08:53:271
2237626316,22sleep10-21swapper/107:58:491
173526317,22sleep10-21swapper/112:03:371
1288126318,29sleep112890-21/usr/sbin/munin07:28:501
1163226314,27sleep311660-21munin-run10:53:273
476226213,31sleep04864-21timerandwakeup10:28:580
2726826215,34sleep127269-21cpu09:58:431
129502625,13sleep00-21swapper/010:55:260
1021726213,34sleep110221-21diskstats09:03:491
1021726213,34sleep110221-21diskstats09:03:491
518326119,25sleep00-21swapper/012:13:460
2910426119,22sleep30-21swapper/310:03:503
2286226119,25sleep123010-21sed11:28:291
1888826117,30sleep018952-21/usr/sbin/munin07:48:470
1819026120,24sleep318342-21latency_hist11:13:293
1508126117,18sleep00-21swapper/011:03:290
2488926024,21sleep00-21swapper/011:33:490
2158326017,29sleep121653-21sensors_temp09:38:571
2006126014,27sleep00-21swapper/011:18:470
1526926016,27sleep115288-21apt-get11:03:381
882425913,24sleep00-21swapper/008:58:550
2702725914,29sleep327162-21cat09:58:293
1862925911,29sleep10-21swapper/111:13:471
1261425920,22sleep112762-21sshd10:53:571
951325818,19sleep00-21swapper/007:18:470
951325818,19sleep00-21swapper/007:18:470
821225816,23sleep30-21swapper/312:23:373
821225816,23sleep30-21swapper/312:23:373
728325813,28sleep07277-21apt-get10:38:420
66522584,42sleep00-21swapper/010:34:270
35182589,18sleep10-21swapper/112:08:381
2720925816,18sleep30-21swapper/308:13:533
2496225813,30sleep024971-21munin-run08:08:290
2202325819,15sleep30-21swapper/311:23:553
1695425818,21sleep00-21swapper/007:43:300
939925715,22sleep10-21swapper/110:43:561
879925717,25sleep38822-21/usr/sbin/munin08:58:543
2902825714,25sleep129082-21/usr/sbin/munin10:03:481
2726725714,26sleep027261-21apt-get09:58:420
1838425719,21sleep10-21swapper/109:28:561
1768125715,17sleep30-21swapper/309:28:293
627925613,27sleep36368-21cat10:33:573
433825611,27sleep14466-21/usr/sbin/munin08:43:591
3064225610,27sleep030710-21wc10:08:510
3064225610,27sleep030710-21wc10:08:510
2437925616,19sleep00-21swapper/011:33:280
1947225615,25sleep019461-21apt-get09:33:430
1585325619,19sleep30-21swapper/307:38:493
365625513,24sleep03650-21apt-get08:43:410
294502556,38sleep00-21swapper/008:21:330
2843025515,17sleep30-21swapper/308:18:453
2544725511,20sleep30-21swapper/309:53:283
2544725511,20sleep30-21swapper/309:53:283
2209025514,18sleep30-21swapper/307:58:393
172902558,12sleep10-21swapper/109:24:031
1684325511,17sleep10-21swapper/111:08:431
1556525513,24sleep015659-21memory11:03:490
1455325520,18sleep30-21swapper/307:33:563
1372625515,23sleep313813-21mailstats09:13:563
1324125513,25sleep113235-21apt-get09:13:421
548625413,14sleep00-21swapper/008:48:500
1847825414,20sleep30-21swapper/309:28:583
1374225413,27sleep313757-21grep10:58:443
1297425415,24sleep012980-21ntp_states07:28:530
1106525415,16sleep30-21swapper/307:23:473
1106525415,16sleep30-21swapper/307:23:473
368825314,25sleep33766-21fw_packets12:08:523
2422125316,18sleep00-21swapper/009:48:450
2422125316,18sleep00-21swapper/009:48:450
2414525314,23sleep124139-21apt-get09:48:421
2414525314,23sleep124139-21apt-get09:48:421
2327625317,16sleep30-21swapper/311:28:473
1080925316,21sleep110828-21ntp_states10:48:531
104525319,15sleep30-21swapper/308:33:593
976125211,20sleep30-21swapper/312:28:383
496325211,23sleep30-21swapper/308:48:293
2989225212,23sleep329887-21apt-get08:23:423
2834125213,23sleep128336-21apt-get08:18:391
1430725214,20sleep00-21swapper/007:33:490
1251925211,23sleep312515-21apt-get07:28:393
1028525216,20sleep00-21swapper/012:28:550
978625111,24sleep19780-21apt-get12:28:401
2882325112,24sleep028890-21df10:03:420
1348925113,24sleep013512-21irqstats09:13:490
2774525012,17sleep00-21swapper/011:43:420
2745325014,21sleep127482-21munin-run11:43:271
953724911,21sleep19547-21munin-run12:28:281
953724911,21sleep19547-21munin-run12:28:281
1073224913,16sleep10-21swapper/107:23:291
1073224913,16sleep10-21swapper/107:23:291
2748824811,18sleep30-21swapper/309:58:493
1717024811,20sleep017208-21memory11:08:500
1660124815,16sleep30-21swapper/311:08:293
833624710,16sleep00-21swapper/012:23:470
833624710,16sleep00-21swapper/012:23:470
80982479,17sleep30-21swapper/308:58:303
199712479,15sleep10-21swapper/111:18:401
1225424715,21sleep012335-21switchtime09:09:010
88732468,15sleep00-21swapper/010:43:430
728024610,15sleep00-21swapper/008:53:590
21682467,16sleep10-21swapper/108:38:461
21682467,16sleep10-21swapper/108:38:461
1906924610,20sleep319108-21memory07:48:503
772124512,18sleep37734-21ntp_states10:38:543
315092459,13sleep10-21swapper/108:28:451
56562439,19sleep35667-21ntp_states08:48:533
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional