You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-30 - 22:06

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Thu Jul 30, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6184252098,5154sleep14755-21kworker/1:109:31:331
14395274212,677sleep114341-21latency_hist09:57:491
9501246518,433sleep29546-21/usr/sbin/munin11:23:222
9501246518,433sleep29546-21/usr/sbin/munin11:23:222
9217240015,327sleep19312-21perl11:23:171
9217240015,327sleep19312-21perl11:23:171
26782234349,242sleep00-21swapper/007:12:580
6644231112,284sleep36651-21df_inode09:33:043
200192311101,190sleep10-21swapper/108:33:041
200192311101,190sleep10-21swapper/108:33:041
178232281131,28sleep10-21swapper/110:08:061
28350225764,176sleep128417-21cat12:23:131
14529225770,170sleep114565-21sendmail08:17:461
12579224516,213sleep312602-21/usr/sbin/munin11:33:083
12579224516,213sleep312602-21/usr/sbin/munin11:33:083
29693223619,74sleep129732-21latency_hist07:22:481
7329223521,196sleep17379-21cpuspeed11:18:001
7329223521,196sleep17379-21cpuspeed11:18:001
15954223414,197sleep116079-21timerwakeupswit11:43:161
27565222249,123sleep127674-21latency_hist12:22:491
1605222271,22sleep10-21swapper/109:17:481
16972218617,156sleep317008-21fschecks_count11:48:063
12820218634,79sleep10-21swapper/109:52:481
8916218513,153sleep18146-21/usr/sbin/munin09:38:191
8916218513,153sleep18146-21/usr/sbin/munin09:38:191
26841218519,151sleep226856-21unin-run10:37:472
20743217720,137sleep020878-21grep11:58:210
21929217045,112sleep121968-21ntp_kernel_err12:03:101
30138216744,56sleep130205-21mailstats09:03:151
19550216657,25sleep10-21swapper/110:13:071
25132216510,138sleep1369-21sshd10:31:321
31571216119,122sleep10-21swapper/107:28:051
16555216012,93sleep116579-21netstat10:03:101
8258215615,128sleep08269-21fschecks_count09:38:050
8258215615,128sleep08269-21fschecks_count09:38:050
19739215611,128sleep219747-21ntpq10:13:122
27353215315,87sleep127367-21grep10:38:061
10938215213,78sleep10-21swapper/111:28:091
10938215213,78sleep10-21swapper/111:28:091
1627215193,41sleep11689-21processes10:58:541
19801214715,111sleep019882-21mailstats10:13:140
27288214611,69sleep127352-21sendmail_mailst07:13:151
2878621450,11sleep30-21swapper/310:43:043
3231214310,77sleep10-21swapper/109:22:491
400214112,115sleep0467-21idleruntime09:13:070
19400214114,112sleep1369-21sshd08:32:351
19400214114,112sleep1369-21sshd08:32:351
11479213817,69sleep111597-21latency_hist08:07:491
11479213817,69sleep111597-21latency_hist08:07:491
2504213791,21sleep10-21swapper/107:38:231
22175213614,69sleep122262-21grep10:22:481
29545213516,94sleep30-21swapper/312:28:083
3153921307,74sleep131564-21awk12:33:351
21272213013,67sleep121294-21munin-run12:02:461
1575021307,113sleep1364-21snmpd10:00:521
546221288,69sleep11-21systemd11:12:471
15961212714,88sleep016114-21wc10:02:480
21564212611,100sleep021577-21systemd08:37:470
21564212611,100sleep021577-21systemd08:37:470
1887121259,105sleep2369-21sshd08:28:332
1887121259,105sleep2369-21sshd08:28:332
2393211821,79sleep21877-21apt-get07:38:032
2958821178,63sleep1364-21snmpd10:43:441
28405211725,70sleep328450-21latency_hist10:42:493
27354211612,87sleep227419-21switchtime07:13:162
29314211510,87sleep10-21swapper/109:02:471
28728211213,83sleep328730-21sshd12:23:313
31854211115,72sleep332000-21unixbench_multi09:08:223
2394210916,73sleep32384-21apt-get07:38:043
707210713,28sleep1810-21apt-config10:57:491
707210713,28sleep1810-21apt-config10:57:491
3808210717,73sleep10-21swapper/109:23:091
25386210717,68sleep224827-21/usr/sbin/munin07:08:162
27723210615,79sleep227850-21timerwakeupswit10:38:162
25518210633,61sleep125565-21grep10:33:021
1794721068,14sleep30-21swapper/310:08:083
17704210514,17sleep20-21swapper/208:27:482
12874210415,69sleep212990-21timerwakeupswit11:33:152
12874210415,69sleep212990-21timerwakeupswit11:33:152
6320210317,70sleep26396-21wc11:13:162
330210317,28sleep00-21swapper/007:32:480
9218210213,20sleep10-21swapper/107:58:171
8042210212,69sleep10-21swapper/107:54:331
6326210214,71sleep36365-21latency_hist09:32:483
6218210112,73sleep36369-21awk07:48:183
6148210111,40sleep15720-21/usr/sbin/munin11:13:111
22224210111,30sleep10-21swapper/108:38:131
293232998,34sleep10-21swapper/112:27:571
2785029815,66sleep027874-21/usr/sbin/munin12:23:000
2305329811,73sleep223055-21sshd08:42:332
1512329831,47sleep10-21swapper/108:18:101
590529711,32sleep10-21swapper/107:48:111
2492229713,16sleep10-21swapper/108:48:011
1462929723,55sleep314630-21cpu09:57:593
368229612,67sleep33421-21/usr/sbin/munin09:23:073
32072966,55sleep10-21swapper/111:03:181
307712968,35sleep10-21swapper/110:48:201
85329420,53sleep2890-21wc07:33:112
741229410,65sleep00-21swapper/011:18:030
741229410,65sleep00-21swapper/011:18:030
1651529412,12sleep30-21swapper/308:23:053
1314529419,55sleep313205-21fschecks_time09:53:063
121712949,34sleep10-21swapper/108:08:131
121712949,34sleep10-21swapper/108:08:131
1383729318,58sleep313914-21switchtime08:13:163
2609029117,60sleep326162-21wc10:33:163
224972918,17sleep10-21swapper/110:23:041
2111929116,61sleep221143-21modprobe08:33:252
2111929116,61sleep221143-21modprobe08:33:252
2943228911,63sleep029443-21awk12:28:050
2395828915,37sleep20-21swapper/210:27:582
824828816,55sleep38256-21/usr/sbin/munin09:38:043
824828816,55sleep38256-21/usr/sbin/munin09:38:043
238302887,70sleep123864-21ps08:43:171
238302887,70sleep123864-21ps08:43:171
1899228711,61sleep019006-21latency_hist10:12:460
1415628711,64sleep114172-21netstat11:38:111
1415628711,64sleep114172-21netstat11:38:111
1197728724,34sleep30-21swapper/311:32:483
1197728724,34sleep30-21swapper/311:32:483
314302865,19sleep10-21swapper/109:08:111
2623128620,44sleep00-21swapper/008:52:490
1347428619,53sleep013512-21awk11:37:450
1347428619,53sleep013512-21awk11:37:450
39422858,14sleep10-21swapper/107:42:591
246662859,58sleep10-21swapper/112:12:591
2347828523,49sleep323551-21cat07:03:453
1666128514,50sleep10-21swapper/108:23:071
3191228421,40sleep232055-21grep12:33:442
241328416,48sleep32425-21latency_hist11:02:483
239972840,5sleep10-21swapper/107:03:541
1276728420,48sleep312784-21sh09:52:463
1226228411,54sleep012329-21/usr/sbin/munin08:08:170
1226228411,54sleep012329-21/usr/sbin/munin08:08:170
776628328,12sleep10-21swapper/109:36:371
152528321,43sleep3225-21systemd-journal09:17:463
1351528324,15sleep10-21swapper/111:37:461
1351528324,15sleep10-21swapper/111:37:461
3042628213,13sleep10-21swapper/107:23:171
1840828120,11sleep10-21swapper/111:52:571
306932807,11sleep00-21swapper/009:05:360
2801928016,50sleep228041-21dpkg08:57:552
235552807,13sleep10-21swapper/112:08:151
120682800,7sleep20-21swapper/209:48:202
23602798,19sleep00-21swapper/007:37:530
2266527910,50sleep00-21swapper/008:38:200
1843527922,34sleep20-21swapper/211:53:022
143752790,8sleep00-21swapper/011:38:170
143752790,8sleep00-21swapper/011:38:170
271472787,12sleep10-21swapper/108:53:181
2514927817,44sleep325210-21/usr/sbin/munin12:13:143
2878927718,40sleep028826-21ntpq07:18:110
1742127723,40sleep10-21swapper/111:48:161
1004427710,14sleep10-21swapper/109:43:071
3115927613,48sleep331174-21grep12:33:213
3087027616,40sleep230996-21/usr/sbin/munin10:48:212
211327614,29sleep20-21swapper/209:18:092
145942768,14sleep20-21swapper/211:38:202
145942768,14sleep20-21swapper/211:38:202
97527517,12sleep10-21swapper/107:33:141
603427520,29sleep30-21swapper/311:13:093
3274527527,24sleep20-21swapper/207:32:462
2576727515,12sleep10-21swapper/107:08:231
2216327522,37sleep322247-21timerandwakeup12:03:163
2125627517,24sleep00-21swapper/010:18:110
1849227524,29sleep30-21swapper/308:28:173
1849227524,29sleep30-21swapper/308:28:173
237652740,7sleep20-21swapper/210:27:492
1681427418,33sleep016939-21wc10:03:170
1302027415,46sleep313063-21cut08:12:483
1302027415,46sleep313063-21cut08:12:483
1112727414,39sleep210642-21/usr/sbin/munin11:28:142
1112727414,39sleep210642-21/usr/sbin/munin11:28:142
1039627414,42sleep010432-21ntp_states09:43:220
445727313,42sleep24470-21/usr/sbin/munin11:08:082
2885027311,43sleep328948-21users08:58:173
2825227312,44sleep228258-21ntp_states12:23:102
2475227315,41sleep224840-21tune2fs12:13:052
1034527319,30sleep20-21swapper/209:43:202
2660727216,27sleep30-21swapper/307:12:533
249472728,50sleep30-21swapper/310:29:083
753427119,23sleep20-21swapper/211:18:052
753427119,23sleep20-21swapper/211:18:052
2617627117,32sleep326297-21latency_hist07:12:493
1042227113,28sleep30-21swapper/311:27:473
1042227113,28sleep30-21swapper/311:27:473
2391827018,32sleep023996-21/usr/sbin/munin07:03:540
1674027024,27sleep30-21swapper/310:03:163
684226918,26sleep00-21swapper/009:33:090
3181526914,39sleep331825-21forks10:53:023
3181526914,39sleep331825-21forks10:53:023
2268126914,17sleep00-21swapper/010:23:070
2197026926,23sleep20-21swapper/212:03:112
1375026917,37sleep213813-21awk08:13:152
1367526918,34sleep213799-21sed09:53:182
269042680,10sleep00-21swapper/008:53:120
1955326821,30sleep019586-21munin-run08:32:470
1955326821,30sleep019586-21munin-run08:32:470
285926712,41sleep02902-21if_eth011:03:090
2132226718,34sleep321289-21latency_hist12:02:483
199926717,23sleep00-21swapper/007:37:490
1814326721,30sleep20-21swapper/211:52:452
153252677,13sleep30-21swapper/308:18:173
1123926721,24sleep00-21swapper/011:28:160
1123926721,24sleep00-21swapper/011:28:160
1046226710,15sleep00-21swapper/011:27:480
51432666,12sleep00-21swapper/011:08:420
3200926614,34sleep032075-21users09:08:230
3065226616,32sleep230741-21apt-config12:32:472
2864026615,36sleep228646-21grep10:43:002
2436826616,35sleep324376-21modprobe12:12:453
296602658,45sleep00-21swapper/010:44:400
2521926514,21sleep00-21swapper/007:08:100
529726418,32sleep05315-21grep09:28:130
470526413,34sleep24734-21munin-run09:27:462
2307226422,24sleep30-21swapper/308:42:473
701226322,21sleep30-21swapper/307:52:583
1542226313,34sleep215495-21fschecks_time11:43:042
1169326310,35sleep211714-21apt-get08:07:582
1169326310,35sleep211714-21apt-get08:07:582
106426315,29sleep01082-21apt10:57:560
99672625,11sleep30-21swapper/309:43:053
518326213,31sleep25237-21/usr/sbin/munin09:28:112
408826212,24sleep00-21swapper/009:23:160
3250526217,21sleep20-21swapper/209:12:492
3182826219,19sleep00-21swapper/010:53:030
3182826219,19sleep00-21swapper/010:53:030
3065626216,29sleep330688-21/usr/sbin/munin10:48:183
2647226223,21sleep20-21swapper/208:53:032
2457926220,21sleep20-21swapper/207:07:472
1976026220,19sleep30-21swapper/311:57:473
18472629,14sleep00-21swapper/009:18:030
1533826221,18sleep30-21swapper/311:43:013
431126114,29sleep04337-21netstat07:43:100
3188426115,21sleep30-21swapper/307:28:113
2666226115,20sleep30-21swapper/308:53:083
153862616,11sleep00-21swapper/008:18:180
53492604,10sleep30-21swapper/311:11:243
3271226015,27sleep232706-21apt-get09:13:012
2987426020,23sleep20-21swapper/209:03:082
2528026020,23sleep00-21swapper/010:32:480
1978926014,20sleep00-21swapper/011:57:480
179942606,10sleep00-21swapper/011:50:500
1667126012,25sleep20-21swapper/210:03:152
948625913,19sleep20-21swapper/209:42:482
948625913,19sleep20-21swapper/209:42:482
730625917,26sleep27337-21/usr/sbin/munin07:53:092
2968925913,32sleep329738-21cut09:03:053
2086525916,26sleep220859-21apt-get10:18:012
2043625912,29sleep30-21swapper/308:33:143
2043625912,29sleep30-21swapper/308:33:143
1943425913,28sleep319488-21/usr/sbin/munin10:13:053
1687225922,19sleep00-21swapper/008:23:140
108725915,27sleep21084-21apt-get10:57:582
974625818,18sleep00-21swapper/011:23:280
898025813,28sleep39014-21/usr/sbin/munin07:58:103
2845525811,22sleep00-21swapper/010:42:500
228212586,12sleep00-21swapper/012:07:450
1335025813,17sleep00-21swapper/008:13:050
451825717,26sleep20-21swapper/207:43:162
326625715,25sleep23363-21/usr/sbin/munin11:03:192
2490225718,22sleep024934-21irqstats12:13:070
1308425715,18sleep00-21swapper/009:53:030
396125616,24sleep34072-21date11:07:483
2847625620,19sleep20-21swapper/207:18:052
2617725617,22sleep20-21swapper/208:52:472
2386725613,24sleep20-21swapper/208:43:182
2386725613,24sleep20-21swapper/208:43:182
2095425618,20sleep30-21swapper/310:18:063
2000825613,25sleep220001-21apt-get11:58:022
1489925613,21sleep20-21swapper/209:58:062
656725517,22sleep26564-21apt-get09:32:592
610125514,26sleep26217-21sed07:48:172
3002625511,19sleep30-21swapper/307:23:063
2378925513,24sleep223915-21unixbench_multi12:08:202
1024625514,19sleep00-21swapper/008:03:040
1024625514,19sleep00-21swapper/008:03:040
932125413,17sleep20-21swapper/207:58:182
59662548,20sleep00-21swapper/007:48:150
3239425414,25sleep032405-21modprobe12:37:460
2994025412,25sleep229933-21apt-get07:23:012
2361525415,18sleep00-21swapper/012:08:180
460725312,24sleep34731-21sed07:43:173
323172539,29sleep232405-21threads10:53:142
323172539,29sleep232405-21threads10:53:142
233422539,29sleep023361-21apt-get08:43:000
233422539,29sleep023361-21apt-get08:43:000
1178025311,24sleep00-21swapper/009:48:110
103825314,23sleep31044-21ntp_states07:33:163
3069125213,23sleep030770-21sendmail_mailtr10:48:190
3012325214,23sleep030159-21idleruntime07:23:070
2837525212,23sleep328386-21apt-get07:17:583
2625125212,23sleep026245-21apt-get12:17:590
701425110,16sleep00-21swapper/011:17:470
272602517,10sleep20-21swapper/212:19:312
1493825111,27sleep215033-21fschecks_time08:18:072
2825925012,23sleep028315-21awk08:58:050
230425014,21sleep32307-21ntp_states09:18:153
1588225018,18sleep00-21swapper/011:43:150
1855324912,22sleep218639-21users10:08:202
1429624914,18sleep30-21swapper/311:38:173
1429624914,18sleep30-21swapper/311:38:173
2566524813,15sleep30-21swapper/308:48:203
2560224810,19sleep00-21swapper/010:33:050
2344624812,21sleep323452-21diskstats08:43:073
2344624812,21sleep323452-21diskstats08:43:073
2248724814,17sleep30-21swapper/310:23:023
101462487,15sleep20-21swapper/208:02:582
101462487,15sleep20-21swapper/208:02:582
1023724712,21sleep310244-21df_inode08:03:033
1023724712,21sleep310244-21df_inode08:03:033
2632524610,14sleep30-21swapper/312:18:023
962424412,18sleep09625-21sshd07:59:330
77552436,14sleep30-21swapper/311:18:103
77552436,14sleep30-21swapper/311:18:103
70402439,18sleep07107-21df07:53:030
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional