You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-17 - 11:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Fri Jul 17, 2026 00:43:51)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
265912655215,6522sleep221149-21kworker/2:221:59:032
265912655215,6522sleep221149-21kworker/2:221:59:032
295242267714,2645sleep029491-21apt-get22:09:030
126042132059,1245sleep012605-21ntp_states21:14:030
1876928898,869sleep318807-21ntpq21:34:013
25844259024,543sleep225862-21cron21:58:352
4140256816,537sleep24177-21ntpq00:14:012
269492486285,100sleep30-21swapper/323:43:543
18336234343,120sleep318494-21unixbench_multi19:49:063
2474233140,202sleep32570-21unixbench_multi22:24:083
1272223288,196sleep30-21swapper/321:14:063
13245229442,130sleep30-21swapper/319:33:563
5336228338,110sleep35387-21cpuspeed00:18:503
3068322727,253sleep330779-21fschecks_time20:28:553
27768226728,203sleep327854-21basename20:19:103
27768226728,203sleep327854-21basename20:19:103
11958224742,176sleep30-21swapper/322:54:083
26855223041,32sleep327040-21gpgv20:18:383
14495222574,25sleep30-21swapper/323:03:533
7186222446,85sleep30-21swapper/322:39:063
4837221211,132sleep34945-21latency_hist22:33:373
25584220516,170sleep325293-21/usr/sbin/munin23:38:573
10473220520,168sleep110606-21head22:49:091
10473220520,168sleep110606-21head22:49:091
8689219825,52sleep00-21swapper/000:29:010
15768219842,25sleep30-21swapper/321:24:033
15768219842,25sleep30-21swapper/321:24:033
9087218732,107sleep30-21swapper/321:03:523
16501218653,86sleep30-21swapper/323:09:053
195721808,158sleep32031-21latency_hist00:08:373
1521217850,48sleep30-21swapper/320:38:573
17418217415,145sleep217395-21latency_hist19:48:372
7678217138,50sleep30-21swapper/320:58:563
7678217138,50sleep30-21swapper/320:58:563
4964216616,134sleep35061-21timerandwakeup20:49:063
10593216528,119sleep310659-21sort21:08:493
10593216528,119sleep310659-21sort21:08:493
19467215556,87sleep319511-21ntpq23:18:593
786521529,130sleep1369-21sshd22:43:031
3505215139,14sleep30-21swapper/300:13:373
16571215171,43sleep316606-21ps19:44:063
29025214910,41sleep30-21swapper/323:49:063
29025214910,41sleep30-21swapper/323:49:063
2429821389,116sleep3364-21snmpd21:53:353
24248213843,52sleep324321-21awk23:34:053
32734213611,80sleep30-21swapper/322:18:553
32734213611,80sleep30-21swapper/322:18:553
5152213338,48sleep30-21swapper/322:33:533
14333213210,86sleep314417-21expr21:19:073
22196212824,75sleep122190-21apt-get23:28:501
9956212517,62sleep30-21swapper/319:23:513
39321247,49sleep3364-21snmpd00:03:363
937212335,70sleep1947-21modprobe20:38:341
4011212328,59sleep34021-21kernelversion00:13:583
24670212214,92sleep124675-21/usr/sbin/munin21:53:531
13266212216,22sleep00-21swapper/022:59:010
21503212032,46sleep30-21swapper/323:24:113
16906211914,91sleep00-21swapper/021:29:050
12740211818,79sleep012755-21latency_hist19:33:370
12740211818,79sleep012755-21latency_hist19:33:370
8146211734,39sleep30-21swapper/300:28:353
2425721176,96sleep00-21swapper/021:53:040
28113211611,89sleep328148-21ntp_kernel_pll_22:04:013
11230211621,75sleep111171-21latency_hist19:28:371
30680211515,10sleep30-21swapper/322:12:293
2057721159,69sleep320697-21timerwakeupswit21:39:063
17516211324,17sleep30-21swapper/323:13:503
17516211324,17sleep30-21swapper/323:13:503
17447211323,71sleep117521-21find21:29:221
2196021119,19sleep30-21swapper/321:44:063
20446211127,75sleep3364-21snmpd19:58:233
31070211018,11sleep30-21swapper/323:56:373
24962211010,62sleep324999-21ntpq21:53:593
30298210927,70sleep31-21systemd20:28:353
7036210718,11sleep30-21swapper/320:56:583
7036210718,11sleep30-21swapper/320:56:583
31221210738,49sleep231246-21munin-run23:58:342
8668210627,43sleep38663-21apt-get00:28:483
26308210612,77sleep026375-21sort21:58:570
26308210612,77sleep026375-21sort21:58:570
24484210510,78sleep024559-21find20:09:060
11553210316,68sleep211568-21cat00:38:362
25192210239,48sleep30-21swapper/320:13:373
2931210120,55sleep13073-21head00:09:071
2536221019,20sleep10-21swapper/123:38:511
8822210026,62sleep38901-21grep19:19:033
1927210010,77sleep02044-21cat20:39:060
10631210014,70sleep210770-21awk19:24:062
398529917,67sleep13995-21ntp_states22:29:151
297152998,14sleep00-21swapper/023:53:360
297152998,14sleep00-21swapper/023:53:360
85932979,29sleep38629-21ntpq22:44:023
3105529711,16sleep30-21swapper/322:13:503
638929422,51sleep06452-21/usr/sbin/munin20:54:040
638929422,51sleep06452-21/usr/sbin/munin20:54:040
49102949,15sleep10-21swapper/119:08:551
49102949,15sleep10-21swapper/119:08:551
226902947,12sleep00-21swapper/023:29:050
1332929411,49sleep30-21swapper/322:59:043
1631829315,55sleep016084-21/usr/sbin/munin19:43:590
1100029324,45sleep211028-21munin-run22:53:342
1100029324,45sleep211028-21munin-run22:53:342
743529217,56sleep17561-21/usr/sbin/munin19:14:081
41529211,67sleep0552-21latency_hist00:03:370
381129214,57sleep13717-21/usr/sbin/munin00:13:521
411128932,42sleep34254-21date20:48:383
1605728917,51sleep116113-21fschecks_count23:08:551
1192128918,53sleep211981-21sendmail_mailst19:29:032
1192128918,53sleep211981-21sendmail_mailst19:29:032
10142899,41sleep3957-21latency_hist20:38:373
41628818,45sleep2531-21unixbench_singl20:34:072
3110728816,54sleep131131-21/usr/sbin/munin22:13:521
261572888,37sleep30-21swapper/321:58:483
261572888,37sleep30-21swapper/321:58:483
158852889,50sleep30-21swapper/319:43:373
1093628817,50sleep110198-21/usr/sbin/munin00:34:071
72642879,64sleep37350-21mailstats19:14:063
336728717,56sleep03433-21ls19:03:540
336728717,56sleep03433-21ls19:03:540
179402875,70sleep30-21swapper/321:31:473
213228617,28sleep00-21swapper/022:24:010
890328519,47sleep08977-21sed19:19:040
645328512,48sleep15880-21/usr/sbin/munin20:54:051
645328512,48sleep15880-21/usr/sbin/munin20:54:051
2686428520,37sleep127005-21unixbench_multi21:59:101
2686428520,37sleep127005-21unixbench_multi21:59:101
1516828519,30sleep00-21swapper/019:39:060
1450828530,33sleep10-21swapper/123:03:551
107662857,41sleep310831-21sshd00:34:053
2270328428,15sleep30-21swapper/320:04:003
2270328428,15sleep30-21swapper/320:04:003
2089928413,55sleep30-21swapper/319:58:553
2089928413,55sleep30-21swapper/319:58:553
1351928431,29sleep20-21swapper/221:18:372
3114228317,44sleep131209-21grep20:29:061
1143928318,48sleep111440-21cpu19:28:501
1143928318,48sleep111440-21cpu19:28:501
279932820,6sleep10-21swapper/122:03:581
223772817,35sleep30-21swapper/323:28:563
828628024,29sleep10-21swapper/100:28:371
329128029,27sleep10-21swapper/119:03:511
329128029,27sleep10-21swapper/119:03:511
2889328017,50sleep128956-21sshd23:49:041
2889328017,50sleep128956-21sshd23:49:041
1918328016,46sleep119199-21fschecks_count23:18:521
1273828028,40sleep312724-21/usr/sbin/munin19:33:363
1273828028,40sleep312724-21/usr/sbin/munin19:33:363
79082799,50sleep00-21swapper/020:59:010
79082799,50sleep00-21swapper/020:59:010
2364327917,47sleep023618-21latency_hist20:08:370
2364327917,47sleep023618-21latency_hist20:08:370
1424227918,46sleep114279-21cat19:38:351
2359127822,39sleep123593-21cron20:08:341
2359127822,39sleep123593-21cron20:08:341
1077627817,49sleep010837-21if_err_eth021:08:540
1077627817,49sleep010837-21if_err_eth021:08:540
574327718,40sleep05855-21timerwakeupswit22:34:060
1899127718,27sleep20-21swapper/219:53:382
549127617,36sleep25628-21grep19:09:082
549127617,36sleep25628-21grep19:09:082
478127615,42sleep04790-21/usr/sbin/munin19:08:460
478127615,42sleep04790-21/usr/sbin/munin19:08:460
453327625,31sleep34550-21munin-run19:08:373
453327625,31sleep34550-21munin-run19:08:373
3015927620,39sleep130262-21grep22:09:361
2114727618,29sleep00-21swapper/019:59:010
2114727618,29sleep00-21swapper/019:59:010
1116627618,18sleep20-21swapper/221:09:042
1116627618,18sleep20-21swapper/221:09:042
1380427512,45sleep213811-21/usr/sbin/munin21:18:532
204072746,56sleep00-21swapper/021:39:030
1791427414,17sleep20-21swapper/223:14:012
1791427414,17sleep20-21swapper/223:14:012
1601227419,31sleep216145-21sed21:24:062
1601227419,31sleep216145-21sed21:24:062
137132748,16sleep00-21swapper/021:18:470
881427320,28sleep10-21swapper/122:44:081
443527319,29sleep00-21swapper/020:48:550
405927323,30sleep04084-21munin-run20:48:350
1658527320,30sleep116713-21ls21:28:381
1658527320,30sleep116713-21ls21:28:381
480627224,32sleep30-21swapper/319:08:493
480627224,32sleep30-21swapper/319:08:493
3081727226,28sleep20-21swapper/222:13:372
2595927226,25sleep00-21swapper/023:39:060
1348227218,39sleep113518-21latency_hist21:18:361
1080827214,44sleep0364-21snmpd19:24:120
1018527221,14sleep30-21swapper/322:49:033
1018527221,14sleep30-21swapper/322:49:033
2848027116,39sleep228613-21/usr/sbin/munin22:04:082
2204527121,31sleep122112-21/usr/sbin/munin21:44:081
951127018,36sleep29662-21date22:48:372
3078527021,22sleep00-21swapper/020:28:560
3003427017,39sleep030040-21df_inode23:53:520
2726227015,29sleep10-21swapper/120:18:561
2726227015,29sleep10-21swapper/120:18:561
2104727018,36sleep221111-21cat19:58:592
2104727018,36sleep221111-21cat19:58:592
1892927019,25sleep00-21swapper/021:34:060
1503427015,27sleep20-21swapper/219:39:042
3182026916,24sleep20-21swapper/223:58:592
811226819,31sleep10-21swapper/120:59:061
811226819,31sleep10-21swapper/120:59:061
3271526810,32sleep10-21swapper/122:18:531
3271526810,32sleep10-21swapper/122:18:531
2904426813,23sleep00-21swapper/020:23:560
2897226816,34sleep028979-21modprobe22:08:340
2073726817,29sleep10-21swapper/123:23:571
149226815,38sleep01467-21latency_hist22:23:360
149226815,38sleep01467-21latency_hist22:23:360
1478326812,41sleep214397-21/usr/sbin/munin23:04:002
1313326813,26sleep10-21swapper/122:58:581
810326720,32sleep28222-21date19:18:372
2280226720,31sleep222849-21cat21:48:372
2119926724,21sleep00-21swapper/023:24:080
842326610,31sleep10-21swapper/119:18:521
3067226612,35sleep230682-21/usr/sbin/munin20:28:542
2851326616,29sleep028541-21munin-run20:23:350
2851326616,29sleep028541-21munin-run20:23:350
1920426612,35sleep219235-21cpu19:53:502
5812656,46sleep00-21swapper/020:34:090
348926528,18sleep20-21swapper/219:03:562
348926528,18sleep20-21swapper/219:03:562
1030726521,27sleep010341-21fschecks_count00:33:530
974526416,20sleep00-21swapper/022:48:520
974526416,20sleep00-21swapper/022:48:520
894426416,20sleep20-21swapper/200:29:432
802426415,32sleep08048-21munin-run19:18:350
3114226419,24sleep00-21swapper/022:13:550
187072646,18sleep00-21swapper/019:50:570
1491126412,36sleep014979-21mailstats23:04:060
1129126417,29sleep111285-21apt-get22:53:491
298626318,31sleep12987-21cron22:28:341
225862638,45sleep00-21swapper/021:46:260
209922636,11sleep10-21swapper/121:40:531
1896026323,18sleep10-21swapper/119:53:371
1723826319,20sleep20-21swapper/221:29:152
908226221,24sleep20-21swapper/219:19:062
3215926217,21sleep10-21swapper/123:59:101
320426214,34sleep23252-21cat20:44:012
301912629,37sleep130232-21irqstats23:53:571
299026217,20sleep20-21swapper/222:28:352
166726216,29sleep21704-21/usr/sbin/munin20:39:002
1503826214,20sleep10-21swapper/121:23:371
1147526218,24sleep00-21swapper/022:53:560
77862616,43sleep00-21swapper/022:42:010
64826116,26sleep00-21swapper/000:03:530
560626119,22sleep20-21swapper/200:18:572
3036926117,26sleep130515-21latency_hist20:28:371
2564126120,18sleep20-21swapper/220:14:002
204526120,20sleep10-21swapper/120:39:071
2021226123,18sleep20-21swapper/221:38:562
1643926113,29sleep216500-21/usr/sbin/munin23:09:042
156862617,11sleep10-21swapper/119:41:541
1305526119,26sleep213061-21df_inode19:33:512
903726011,25sleep20-21swapper/221:03:512
665026010,24sleep10-21swapper/100:23:371
633426015,22sleep10-21swapper/122:38:351
2907926014,27sleep229032-21apt-get20:24:182
2604926016,22sleep00-21swapper/020:14:110
2500126016,26sleep1278-21dbus-daemon23:38:351
2387726017,26sleep223874-21apt-get20:08:482
185642606,10sleep10-21swapper/123:15:461
185642606,10sleep10-21swapper/123:15:461
1364426013,28sleep013756-21/usr/sbin/munin19:34:060
566825918,18sleep20-21swapper/220:53:382
547425915,21sleep20-21swapper/222:34:002
3145125914,22sleep20-21swapper/222:14:012
2817425914,26sleep20-21swapper/223:48:372
2421025920,24sleep20-21swapper/223:34:042
1284225918,24sleep20-21swapper/222:58:472
958125820,19sleep00-21swapper/021:04:060
348325816,26sleep23003-21apt-get22:28:432
2724225823,20sleep10-21swapper/123:44:001
2669525813,27sleep126723-21munin-run20:18:361
1909425817,19sleep20-21swapper/221:34:082
2895725716,19sleep20-21swapper/223:49:052
2895725716,19sleep20-21swapper/223:49:052
1971325714,17sleep00-21swapper/019:54:040
44925613,24sleep2543-21memory22:18:582
44925613,24sleep2543-21memory22:18:582
2864225618,23sleep20-21swapper/220:23:372
2864225618,23sleep20-21swapper/220:23:372
2592225617,23sleep025995-21date21:58:370
2388125619,18sleep10-21swapper/120:08:511
1920325611,27sleep119197-21apt-get19:53:491
91682557,10sleep00-21swapper/022:44:560
885325512,19sleep10-21swapper/121:03:381
86125513,28sleep20-21swapper/200:03:592
435125513,25sleep14421-21df20:48:521
1126625512,26sleep20-21swapper/222:53:462
874525412,26sleep2278-21dbus-daemon21:03:352
874525412,26sleep2278-21dbus-daemon21:03:352
819825412,18sleep00-21swapper/000:28:360
79852545,10sleep20-21swapper/200:26:322
58792547,25sleep20-21swapper/220:53:462
58792547,25sleep20-21swapper/220:53:462
2686725411,17sleep00-21swapper/023:43:510
1899825416,17sleep10-21swapper/121:34:071
1668525418,20sleep10-21swapper/119:44:081
116262546,11sleep10-21swapper/121:10:411
116262546,11sleep10-21swapper/121:10:411
280925315,18sleep00-21swapper/000:09:060
2233625313,24sleep20-21swapper/220:03:512
2233625313,24sleep20-21swapper/220:03:512
1797225315,22sleep017978-21ntp_states23:14:030
1797225315,22sleep017978-21ntp_states23:14:030
664825214,21sleep26644-21apt-get22:38:482
553025214,23sleep05584-21if_eth000:18:550
2539025215,22sleep225399-21diskstats23:38:522
2311925214,22sleep123123-21diskstats21:48:531
1951425212,22sleep019551-21ntp_states23:19:000
2312425115,16sleep20-21swapper/221:48:542
2212425011,14sleep10-21swapper/120:03:381
2212425011,14sleep10-21swapper/120:03:381
1910025012,23sleep219167-21df23:18:492
1624025011,23sleep016286-21sed23:08:580
867924912,22sleep18663-21apt-get00:28:541
64424910,23sleep1640-21apt-get00:03:501
372324912,19sleep03737-21apt-get00:13:470
348624913,15sleep00-21swapper/022:28:450
2285324713,18sleep20-21swapper/223:29:072
1535224711,20sleep015359-21diskstats21:23:520
1535224711,20sleep015359-21diskstats21:23:520
28042447,14sleep10-21swapper/120:43:511
219072447,14sleep20-21swapper/223:28:342
22652438,17sleep20-21swapper/200:08:522
238272429,15sleep00-21swapper/023:33:530
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional