You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-16 - 16:35

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Tue Jun 16, 2026 12:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
224752147911,1451sleep122476-21ntp_states09:32:561
224752147911,1451sleep122476-21ntp_states09:32:561
15616253113,506sleep111012-21kworker/1:012:37:001
739524368,412sleep07415-21apt-get10:27:380
739524368,412sleep07415-21apt-get10:27:380
4047239123,349sleep04188-21unixbench_singl08:33:010
21725230911,284sleep221773-21unin-run09:32:292
9347229291,44sleep09383-21ntp_kernel_err10:32:510
9347229291,44sleep09383-21ntp_kernel_err10:32:510
11107228116,250sleep311166-21/usr/sbin/munin10:37:593
18066225517,225sleep218098-21unin-run11:02:292
9623223017,73sleep09679-21fschecks_count08:52:470
19561222310,200sleep1194-21jbd2/mmcblk2p2-11:07:001
4697222271,26sleep04737-21grep10:17:530
1208222052,29sleep00-21swapper/011:52:300
1208222052,29sleep00-21swapper/011:52:300
15474221712,162sleep015522-21latency10:52:500
16671221413,175sleep00-21swapper/009:13:010
27440220415,163sleep00-21swapper/011:32:310
1209822048,184sleep3363-21snmpd08:58:513
13931220127,149sleep013934-21cron09:07:280
2423821937,173sleep023567-21/usr/sbin/munin09:38:000
2423821937,173sleep023567-21/usr/sbin/munin09:38:000
17855219216,162sleep117921-21awk09:17:591
10336219113,156sleep00-21swapper/007:12:340
657218112,153sleep0685-21grep10:07:270
1913217716,79sleep00-21swapper/010:08:120
2922121749,25sleep00-21swapper/011:37:400
126282172129,19sleep00-21swapper/012:27:300
537921708,146sleep00-21swapper/012:02:510
5369217012,17sleep00-21swapper/008:37:570
5369217012,17sleep00-21swapper/008:37:570
24543217024,127sleep00-21swapper/011:22:380
15650216811,135sleep00-21swapper/007:27:530
6912216636,109sleep36965-21sshd08:42:573
6912216636,109sleep36965-21sshd08:42:573
1948621666,12sleep00-21swapper/011:06:020
24934216422,22sleep10-21swapper/111:22:511
27756215867,22sleep00-21swapper/011:32:470
26643215820,98sleep01-21systemd09:47:280
26643215820,98sleep01-21systemd09:47:280
23145214810,99sleep023238-21grep07:52:470
23145214810,99sleep023238-21grep07:52:470
2091921477,123sleep00-21swapper/009:27:540
19643214719,113sleep219658-21cat11:07:292
4885214419,108sleep24956-21ls10:17:592
28239214312,83sleep028253-21systemd-run09:52:290
3101321406,13sleep00-21swapper/011:42:480
3039721396,99sleep00-21swapper/008:15:320
3462213411,106sleep12959-21apt-get11:57:351
1815921349,111sleep018182-21cat07:37:290
1158213433,82sleep3677-21apt-get10:07:383
176821337,45sleep00-21swapper/008:27:520
31270213215,96sleep331332-21wc11:42:563
9963212912,99sleep010045-21latency_hist07:12:300
9963212912,99sleep010045-21latency_hist07:12:300
1236621290,8sleep20-21swapper/207:17:482
1736121278,36sleep00-21swapper/010:57:590
6508212638,69sleep00-21swapper/008:42:450
6508212638,69sleep00-21swapper/008:42:450
1468521255,84sleep00-21swapper/010:49:310
864121247,81sleep08698-21ps12:12:570
14632212315,86sleep20-21swapper/207:23:022
1324621235,12sleep00-21swapper/007:19:570
16518212018,80sleep316669-21unixbench_singl09:13:003
13743212016,89sleep213753-21forks10:47:472
2744721188,74sleep01-21systemd08:07:290
13870211840,11sleep00-21swapper/012:29:240
3892211738,65sleep00-21swapper/011:58:560
14801211719,26sleep30-21swapper/312:32:543
14161211725,68sleep314248-21latency_hist12:32:293
29106211555,42sleep08449-21cyclictest09:53:020
11201211565,29sleep011258-21fschecks_count08:57:470
8067211422,62sleep20-21swapper/208:47:482
8067211422,62sleep20-21swapper/208:47:482
1521211410,85sleep21165-21apt-get11:52:322
144621149,29sleep01651-21sed08:27:310
25344211212,22sleep00-21swapper/007:58:000
2040821127,14sleep00-21swapper/007:42:540
11575211210,81sleep30-21swapper/307:13:553
440210818,66sleep1512-21ls11:47:581
440210818,66sleep1512-21ls11:47:581
3537210720,65sleep33465-21/usr/sbin/munin11:58:143
20156210727,63sleep120215-21latency11:07:491
20156210727,63sleep120215-21latency11:07:491
13038210633,40sleep00-21swapper/010:43:130
711121056,88sleep00-21swapper/010:27:100
711121056,88sleep00-21swapper/010:27:100
26393210518,68sleep326094-21/usr/sbin/munin11:27:503
25370210527,59sleep324865-21apt-get09:42:393
25370210527,59sleep324865-21apt-get09:42:393
15544210514,75sleep215615-21latency_hist09:12:312
1418821057,82sleep00-21swapper/007:22:560
8379210435,49sleep08412-21latency_hist07:07:300
26380210311,73sleep226392-21/usr/sbin/munin11:27:492
2241921037,19sleep00-21swapper/007:48:030
752921025,25sleep00-21swapper/012:08:080
20926210220,21sleep10-21swapper/109:27:551
386821016,83sleep00-21swapper/010:15:430
1952021019,20sleep019607-21ls09:22:580
14717210117,15sleep00-21swapper/009:07:560
14399210169,20sleep012492-21ntpd12:32:430
8191210057,21sleep00-21swapper/008:47:500
8191210057,21sleep00-21swapper/008:47:500
6237210019,59sleep26385-21latency_hist08:42:312
28276210018,66sleep328375-21cat11:32:583
2398921009,78sleep224025-21ntpq09:37:532
2398921009,78sleep224025-21ntpq09:37:532
2321421005,83sleep00-21swapper/009:36:420
2321421005,83sleep00-21swapper/009:36:420
22515210024,42sleep00-21swapper/011:15:030
19436210033,56sleep00-21swapper/007:39:310
10433210021,51sleep110589-21grep12:17:581
177029914,71sleep11837-21df08:27:531
1713029922,61sleep117216-21grep09:17:301
1191829916,65sleep212046-21timerwakeupswit12:23:002
2869129821,42sleep20-21swapper/209:52:392
282402987,18sleep028315-21mailstats08:07:570
16422979,15sleep00-21swapper/011:52:340
884929618,61sleep39001-21awk12:13:003
1737129664,20sleep017345-21/usr/sbin/munin09:17:450
101992967,14sleep00-21swapper/012:17:530
1017629520,58sleep210280-21munin-node07:12:322
3029329319,36sleep20-21swapper/209:57:402
3132329211,65sleep00-21swapper/008:17:550
2820329221,55sleep228275-21sensors_temp11:32:572
1586529215,64sleep215899-21fschecks_count09:12:472
953029123,42sleep19677-21latency_hist12:17:301
302962906,14sleep00-21swapper/009:57:420
2281529017,60sleep322926-21date07:52:303
1373328918,17sleep10-21swapper/107:22:451
117812887,11sleep00-21swapper/010:42:000
2370128617,45sleep123781-21fschecks_time09:37:481
2370128617,45sleep123781-21fschecks_time09:37:481
213928619,53sleep22176-21ntp_kernel_err08:28:022
1782328617,43sleep317829-21ntp_states09:17:563
1187228621,49sleep212020-21latency_hist10:42:302
552028517,53sleep25643-21cat08:37:592
198112855,11sleep00-21swapper/011:07:310
198112855,11sleep00-21swapper/011:07:310
1186428420,44sleep111889-21munin-run07:17:281
1100728423,32sleep20-21swapper/212:22:272
877128316,32sleep20-21swapper/212:12:592
400628320,41sleep34035-21munin-plugin-st10:17:283
2348728330,42sleep023521-21ps11:17:570
2112428314,50sleep10-21swapper/107:46:221
1517828314,15sleep10-21swapper/110:52:391
1267028323,35sleep30-21swapper/309:02:433
126428315,53sleep21327-21cmp08:27:292
2999128219,39sleep130135-21sed11:38:011
1804928220,44sleep10-21swapper/111:02:281
2196828118,47sleep20-21swapper/207:47:572
1709228023,41sleep317108-21uname07:32:523
2335427919,26sleep30-21swapper/309:37:303
2335427919,26sleep30-21swapper/309:37:303
871727711,34sleep30-21swapper/307:07:453
871727711,34sleep30-21swapper/307:07:453
188127722,32sleep11650-21/usr/sbin/munin11:52:591
3248627617,35sleep132606-21timerwakeupswit10:03:001
2669527619,24sleep20-21swapper/208:02:592
1891027622,30sleep20-21swapper/209:22:402
1842327624,23sleep30-21swapper/311:02:463
1402527623,40sleep314052-21netstat10:47:523
132827623,28sleep31431-21gpgv08:27:303
86827517,39sleep31075-21date10:07:303
3241327521,30sleep132410-21apt-get08:22:401
1901827527,23sleep20-21swapper/207:38:002
1723127518,36sleep217275-21/usr/sbin/munin07:32:552
762827418,34sleep27678-21/usr/sbin/munin10:27:482
762827418,34sleep27678-21/usr/sbin/munin10:27:482
1654127426,28sleep30-21swapper/307:32:283
73027314,43sleep2866-21latency_hist10:07:292
2126027317,37sleep321301-21latency_hist07:47:303
1945927313,26sleep30-21swapper/309:22:573
1399527314,42sleep214064-21latency_hist09:07:302
2944427225,23sleep20-21swapper/211:37:502
2701727220,24sleep20-21swapper/209:47:472
905827116,39sleep29063-21diskstats10:32:452
905827116,39sleep29063-21diskstats10:32:452
875927125,22sleep30-21swapper/310:32:303
875927125,22sleep30-21swapper/310:32:303
649227116,27sleep30-21swapper/310:23:013
649227116,27sleep30-21swapper/310:23:013
3103327119,30sleep131182-21awk09:57:591
3054927129,25sleep212-21rcu_sched08:17:282
2542127112,30sleep10-21swapper/109:43:091
2542127112,30sleep10-21swapper/109:43:091
2872027020,32sleep128710-21apt-get09:52:421
2508627025,29sleep325092-21ntp_states07:57:553
2044327017,27sleep20-21swapper/207:42:562
1581427011,25sleep10-21swapper/107:27:581
991226916,35sleep29947-21idleruntime-cro07:12:282
991226916,35sleep29947-21idleruntime-cro07:12:282
81592696,48sleep28160-21192.168.115.60-07:05:222
236926935,14sleep30-21swapper/311:53:093
2329226928,20sleep20-21swapper/211:17:512
2023726913,22sleep20-21swapper/211:07:502
2023726913,22sleep20-21swapper/211:07:502
924626821,27sleep19247-21cron08:52:271
924626821,27sleep19247-21cron08:52:271
44212687,40sleep30-21swapper/310:17:483
2973726815,28sleep10-21swapper/108:12:551
2737926818,31sleep127404-21munin-run11:32:281
265572687,13sleep10-21swapper/108:02:541
1314026824,25sleep10-21swapper/109:02:571
119126821,24sleep10-21swapper/110:07:441
607326714,40sleep3363-21snmpd08:41:353
607326714,40sleep3363-21snmpd08:41:353
3217726718,23sleep30-21swapper/308:22:303
980526617,32sleep39841-21cat08:52:503
408526623,24sleep10-21swapper/110:17:301
2949026619,25sleep30-21swapper/308:12:493
2814326614,29sleep127644-21/usr/sbin/munin11:32:561
2439326625,22sleep30-21swapper/307:57:313
2439326625,22sleep30-21swapper/307:57:313
2324226621,24sleep10-21swapper/107:52:481
2324226621,24sleep10-21swapper/107:52:481
2114626621,28sleep30-21swapper/311:12:273
2114626621,28sleep30-21swapper/311:12:273
1285126618,20sleep10-21swapper/112:27:431
109432667,11sleep30-21swapper/312:21:403
464626521,24sleep10-21swapper/108:37:301
3160626514,38sleep3284-21dbus-daemon10:02:293
310426518,28sleep20-21swapper/210:12:502
629026411,39sleep36293-21sshd12:07:103
1818326415,33sleep118326-21cat07:37:301
1342126421,30sleep113531-21latency_hist10:47:301
717426312,35sleep27321-21awk08:43:002
717426312,35sleep27321-21awk08:43:002
2865026313,21sleep10-21swapper/108:08:031
2241826322,22sleep30-21swapper/311:13:573
1653426319,25sleep316544-21munin-run10:57:293
669126222,25sleep30-21swapper/312:07:473
612426217,20sleep20-21swapper/210:22:522
612426217,20sleep20-21swapper/210:22:522
542726216,28sleep15464-21sed12:02:521
258942625,11sleep10-21swapper/108:02:251
2072526216,16sleep30-21swapper/309:27:493
945626120,20sleep10-21swapper/110:32:561
945626120,20sleep10-21swapper/110:32:561
897826115,26sleep38973-21apt-get10:32:423
897826115,26sleep38973-21apt-get10:32:423
385026117,27sleep23919-21awk08:32:592
306626121,17sleep30-21swapper/308:32:313
2764626119,20sleep30-21swapper/309:48:003
1900626118,20sleep10-21swapper/109:22:471
353626017,25sleep23483-21apt-get11:58:132
281772607,11sleep10-21swapper/109:52:141
1287926019,18sleep30-21swapper/310:43:003
3106425918,17sleep10-21swapper/111:42:491
2819925918,27sleep328236-21ps08:07:563
1500825917,18sleep10-21swapper/110:52:301
1453025912,30sleep214609-21fw_packets12:32:492
1164925920,18sleep20-21swapper/208:57:572
841425817,18sleep10-21swapper/107:07:311
791725818,18sleep10-21swapper/112:12:291
688525813,29sleep16891-21ntp_states08:42:551
688525813,29sleep16891-21ntp_states08:42:551
559325816,24sleep25664-21sensors_temp12:02:572
330425815,26sleep33299-21apt-get08:32:433
2594825817,18sleep20-21swapper/208:02:302
2563725818,23sleep225686-21irqstats09:43:142
2563725818,23sleep225686-21irqstats09:43:142
2494825818,15sleep20-21swapper/211:22:522
2362725819,18sleep20-21swapper/207:52:582
2362725819,18sleep20-21swapper/207:52:582
1349325818,19sleep30-21swapper/307:22:303
1264725817,24sleep112682-21grep07:17:561
1166325819,25sleep30-21swapper/312:22:533
1111725818,24sleep111184-21df08:57:441
1056125816,23sleep110628-21df10:37:451
873725717,21sleep10-21swapper/107:07:471
873725717,21sleep10-21swapper/107:07:471
3125525722,19sleep20-21swapper/211:42:552
3123525718,19sleep10-21swapper/108:17:521
1887125713,31sleep318885-21awk07:37:573
151632577,11sleep10-21swapper/109:08:141
1432025721,21sleep30-21swapper/309:07:473
929725618,20sleep30-21swapper/308:52:293
929725618,20sleep30-21swapper/308:52:293
832625611,22sleep30-21swapper/307:07:293
567425613,16sleep10-21swapper/110:22:301
3234525614,22sleep20-21swapper/211:47:432
3234525614,22sleep20-21swapper/211:47:432
2944425619,22sleep329577-21awk09:53:093
2499825616,23sleep325022-21ntp_states11:22:533
2208025619,22sleep222190-21sort11:12:592
1415225616,26sleep314184-21ntp_states07:22:553
2352425513,26sleep123600-21/usr/sbin/munin11:17:581
2186625516,23sleep121889-21ntp_states11:12:541
1760025517,24sleep217638-21irqstats09:17:512
960725411,18sleep20-21swapper/208:52:452
579825416,17sleep10-21swapper/108:38:021
579825416,17sleep10-21swapper/108:38:021
2310925410,29sleep323151-21fschecks_count11:17:483
2196025411,27sleep121967-21ntp_states07:47:561
1894625412,21sleep10-21swapper/107:37:591
1338125418,16sleep20-21swapper/212:27:562
1275225310,23sleep20-21swapper/210:42:592
952925213,22sleep19531-21apt-get08:52:401
3176525210,26sleep231774-21vmstat08:18:012
2932425216,21sleep329330-21diskstats11:37:473
2384125211,25sleep323863-21irqstats09:37:503
2384125211,25sleep323863-21irqstats09:37:503
2248225215,18sleep20-21swapper/209:32:572
2248225215,18sleep20-21swapper/209:32:572
869025113,22sleep10-21swapper/110:32:281
869025113,22sleep10-21swapper/110:32:281
716325112,16sleep20-21swapper/212:07:582
1520725110,15sleep20-21swapper/210:52:442
131725115,19sleep20-21swapper/210:07:542
2819025011,23sleep228196-21ntp_states08:07:552
178872505,34sleep30-21swapper/311:00:243
115024912,22sleep31206-21systemd-run11:52:283
115024912,22sleep31206-21systemd-run11:52:283
1536524811,22sleep215366-21diskstats07:27:462
2645624713,18sleep30-21swapper/308:02:523
1254224714,20sleep112543-21sshd12:27:151
624624412,15sleep10-21swapper/110:22:551
624624412,15sleep10-21swapper/110:22:551
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional