You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-04 - 05:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Fri Sep 04, 2026 00:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
17968234393,117sleep118023-21fschecks_count19:38:571
1662225611,204sleep110270-21kworker/u8:022:13:361
30400225171,25sleep10-21swapper/120:18:401
30021224613,219sleep130024-21cron20:18:361
393822398,172sleep13980-21ntpq22:19:011
530223612,210sleep0616-21fw_packets22:08:590
23123222813,199sleep323201-21threads19:54:083
23123222813,199sleep323201-21threads19:54:083
3459222736,171sleep13606-21unixbench_singl23:59:281
5984220082,29sleep16017-21fschecks_count20:43:551
15005218879,21sleep10-21swapper/121:13:381
28826218731,120sleep128869-21ntpq23:39:011
23846218513,157sleep123920-21sort21:39:091
17313218525,142sleep117399-21mailstats21:19:071
22880218417,95sleep123012-21timerwakeupswit23:19:071
25698218213,156sleep125734-21ntpq23:29:041
10040217713,151sleep07915-21latencyplot19:13:450
4711217576,17sleep10-21swapper/120:38:591
32300217510,153sleep132362-21cat20:23:571
32300217510,153sleep132362-21cat20:23:571
10771217514,76sleep10-21swapper/100:23:581
529217073,21sleep10-21swapper/123:53:381
2137216916,138sleep12198-21cat22:13:561
9266216728,104sleep10-21swapper/119:12:041
4113216160,20sleep10-21swapper/120:38:381
18855216046,33sleep118923-21latency_hist23:08:381
7956215214,83sleep18104-21latency_hist22:33:381
32215214515,114sleep032298-21fschecks_time20:23:560
32215214515,114sleep032298-21fschecks_time20:23:560
30576214310,67sleep10-21swapper/120:18:521
30576214310,67sleep10-21swapper/120:18:521
19185213813,90sleep119264-21fschecks_time23:08:581
5656213713,109sleep05633-21latency_hist20:43:380
20329213722,90sleep220341-21vmstat19:44:102
8804213619,94sleep28913-21timerandwakeup22:34:082
2460213612,111sleep12590-21apt-key23:58:391
1172921347,58sleep10-21swapper/119:18:541
11890212912,68sleep111897-21munin-run21:03:371
772921219,91sleep10-21swapper/100:14:011
6394211811,91sleep26486-21/usr/sbin/munin00:09:052
31817211711,90sleep031876-21/usr/sbin/munin23:49:010
31306211712,67sleep131450-21latency_hist23:48:381
15745211719,22sleep10-21swapper/122:58:371
12619211212,85sleep212598-21latency_hist22:48:372
12619211212,85sleep212598-21latency_hist22:48:372
11425211212,83sleep211529-21cat19:18:392
1407211119,75sleep31470-21if_eth020:28:563
19891211032,61sleep120096-21sed21:28:401
22971210920,67sleep023034-21sendmail-msp21:38:350
16128210912,41sleep116269-21modprobe19:33:401
15772210916,73sleep215918-21wc22:58:382
30609210726,50sleep330610-21apt-get20:19:153
30609210726,50sleep330610-21apt-get20:19:153
22529210729,57sleep10-21swapper/119:53:541
22529210729,57sleep10-21swapper/119:53:541
21323210721,65sleep20-21swapper/223:14:082
12617210712,76sleep012679-21/usr/sbin/munin21:04:040
12617210712,76sleep012679-21/usr/sbin/munin21:04:040
11320210713,78sleep10-21swapper/122:43:511
11320210713,78sleep10-21swapper/122:43:511
10738210734,58sleep110857-21sed19:14:061
4359210634,57sleep14460-21grep00:03:561
20171210514,19sleep10-21swapper/121:28:561
15863210517,72sleep115982-21/usr/sbin/munin21:14:061
900210425,51sleep2874-21apt-get23:54:042
8212210412,76sleep28102-21/usr/sbin/munin19:08:542
20183210432,58sleep120328-21uptime19:44:091
1845210413,17sleep10-21swapper/120:29:061
19188210215,74sleep219236-21latency_hist19:43:382
27060210111,24sleep00-21swapper/021:49:050
27060210111,24sleep00-21swapper/021:49:050
25036210130,57sleep125059-21munin-run23:28:361
19237210112,22sleep30-21swapper/319:43:393
631629932,52sleep16388-21mailstats00:09:041
2559829815,70sleep325657-21latency23:29:023
2619229711,65sleep125633-21/usr/sbin/munin20:04:071
2619229711,65sleep125633-21/usr/sbin/munin20:04:071
2874929626,38sleep20-21swapper/220:13:492
2874929626,38sleep20-21swapper/220:13:492
2220129611,58sleep10-21swapper/121:34:041
720529514,65sleep07242-21date20:48:380
508929515,66sleep05142-21ls22:23:520
2541629524,51sleep025480-21sort23:28:590
1898029513,66sleep219058-21timerandwakeup21:24:092
1137529510,71sleep311403-21users00:24:083
780129431,46sleep07842-21awk19:08:360
269429420,49sleep02845-21unixbench_singl22:14:080
2166629416,65sleep00-21swapper/023:15:350
136012945,14sleep30-21swapper/300:33:533
936829313,14sleep10-21swapper/120:54:001
936829313,14sleep10-21swapper/120:54:001
248362938,72sleep124830-21apt-get21:43:491
2238429310,65sleep30-21swapper/321:34:063
3020829227,53sleep10-21swapper/121:59:031
2730229121,15sleep10-21swapper/123:34:051
2563229113,58sleep325651-21apt-get20:03:493
2563229113,58sleep325651-21apt-get20:03:493
241032917,17sleep10-21swapper/119:58:501
1365529014,21sleep10-21swapper/119:24:031
1026629011,67sleep110356-21sort00:23:371
315152890,8sleep20-21swapper/220:19:582
315152890,8sleep20-21swapper/220:19:582
1502828913,16sleep10-21swapper/119:29:001
920828816,53sleep28977-21/usr/sbin/munin20:53:572
920828816,53sleep28977-21/usr/sbin/munin20:53:572
3179228815,19sleep10-21swapper/122:04:081
2872028816,54sleep328812-21load23:38:593
269012887,15sleep10-21swapper/121:49:021
269012887,15sleep10-21swapper/121:49:021
1598828821,13sleep10-21swapper/122:58:491
1368428814,59sleep213695-21awk00:33:562
134502887,16sleep10-21swapper/121:08:371
134502887,16sleep10-21swapper/121:08:371
93528722,40sleep3952-21idleruntime-cro20:28:353
93528722,40sleep3952-21idleruntime-cro20:28:353
3139228722,43sleep331484-21tune2fs22:03:573
2536728720,40sleep225400-21munin-plugin-st20:03:362
1976628717,46sleep019888-21cat21:28:390
1494928711,60sleep014889-21latency_hist00:38:380
1329528717,46sleep213185-21/usr/sbin/munin19:23:562
2736328614,57sleep027440-21tr23:34:080
222872868,65sleep122288-21munin-run19:53:381
222872868,65sleep122288-21munin-run19:53:381
1998228614,52sleep320053-21ls19:44:073
177028622,43sleep01844-21switchtime20:29:050
1107928518,48sleep011208-21ls22:43:380
629528426,48sleep10-21swapper/122:27:571
489628415,54sleep34967-21switchtime20:39:053
1791128410,62sleep217941-21awk23:03:592
1285828418,49sleep10-21swapper/100:29:101
114872845,17sleep00-21swapper/000:25:060
2232428312,36sleep220066-21kworker/2:123:18:562
100122836,16sleep10-21swapper/122:38:591
826628219,17sleep10-21swapper/122:33:541
744028219,12sleep10-21swapper/120:48:481
3073628211,17sleep20-21swapper/223:44:072
2642828222,39sleep326112-21apt-get21:48:403
2817128110,15sleep10-21swapper/121:53:501
2817128110,15sleep10-21swapper/121:53:501
1384628117,14sleep10-21swapper/100:33:591
743427913,47sleep27440-21/usr/sbin/munin00:13:542
2766627922,44sleep10-21swapper/120:09:041
226262788,15sleep30-21swapper/323:19:013
178562789,55sleep017859-21acpi19:38:470
116632778,49sleep30-21swapper/319:18:523
707227612,46sleep07081-21modprobe00:13:350
335127614,46sleep03457-21ls23:59:270
2053627613,44sleep220547-21/usr/sbin/munin21:29:052
2159827518,10sleep10-21swapper/123:14:431
1384427519,10sleep10-21swapper/122:50:011
1384427519,10sleep10-21swapper/122:50:011
1297427518,30sleep20-21swapper/219:23:402
1106727520,26sleep30-21swapper/320:59:073
27442745,10sleep00-21swapper/020:33:450
1703427417,23sleep00-21swapper/021:18:580
827927311,20sleep30-21swapper/322:33:563
710827321,29sleep10-21swapper/119:04:061
1692427320,36sleep317030-21tune2fs21:18:573
330127220,23sleep20-21swapper/222:18:382
2821127220,29sleep228339-21latency_hist23:38:382
2152327219,30sleep00-21swapper/021:33:390
203962726,16sleep00-21swapper/019:44:530
100422728,41sleep20-21swapper/222:39:002
770427116,28sleep30-21swapper/320:48:583
662627122,37sleep36661-21fschecks_count19:03:553
661827129,21sleep20-21swapper/219:03:542
508727114,26sleep30-21swapper/322:23:513
34127115,29sleep30-21swapper/323:53:363
3017727121,22sleep30-21swapper/323:43:553
2417127115,40sleep324178-21/usr/sbin/munin19:58:533
2147327118,35sleep321519-21cat21:33:383
1331027122,33sleep213337-21grep00:33:352
397727010,43sleep24024-21systemd-run00:03:362
1096227018,33sleep210977-21/usr/sbin/munin00:24:032
2709026919,34sleep227124-21ps21:49:072
2709026919,34sleep227124-21ps21:49:072
1189826918,23sleep20-21swapper/221:03:382
950126816,32sleep39701-21irqstats19:13:403
2508926821,24sleep00-21swapper/023:28:390
2900326712,40sleep229086-21switchtime23:39:062
1638326711,44sleep216278-21/usr/sbin/munin19:33:532
1561126715,27sleep30-21swapper/321:14:003
1264826724,20sleep30-21swapper/322:48:383
1264826724,20sleep30-21swapper/322:48:383
1184726717,35sleep01-21systemd21:03:360
3012426620,29sleep230168-21ntpq21:59:002
2463926623,24sleep20-21swapper/221:43:382
231026622,21sleep00-21swapper/023:58:380
1567426610,41sleep015744-21systemd-run22:58:360
1524326623,16sleep20-21swapper/221:13:512
121402669,43sleep012161-21fschecks_count00:28:570
835926517,27sleep00-21swapper/022:33:580
590026523,27sleep25968-21/usr/sbin/munin20:43:522
892726411,36sleep28946-21apt-get00:18:462
2863226413,38sleep328647-21open_inodes21:54:053
2863226413,38sleep328647-21open_inodes21:54:053
1034226414,19sleep30-21swapper/320:58:393
1034226414,19sleep30-21swapper/320:58:393
3130926316,24sleep00-21swapper/022:03:540
2280426320,23sleep00-21swapper/023:19:060
1607826319,24sleep00-21swapper/022:58:560
648726216,23sleep30-21swapper/300:09:063
3091226220,24sleep00-21swapper/020:19:480
3091226220,24sleep00-21swapper/020:19:480
2343926217,24sleep30-21swapper/321:38:593
710126117,29sleep27133-21/usr/sbin/munin22:29:072
1213526123,18sleep30-21swapper/321:03:513
1213526123,18sleep30-21swapper/321:03:513
934826015,23sleep00-21swapper/000:19:000
294012607,39sleep00-21swapper/021:58:130
294012607,39sleep00-21swapper/021:58:130
286126015,22sleep30-21swapper/320:33:563
2702426017,30sleep327058-21/usr/sbin/munin21:49:043
2702426017,30sleep327058-21/usr/sbin/munin21:49:043
2484026015,19sleep00-21swapper/021:43:520
214602609,16sleep30-21swapper/323:14:093
1926526015,29sleep20-21swapper/223:08:592
1221726016,30sleep212110-21/usr/sbin/munin21:03:542
1221726016,30sleep212110-21/usr/sbin/munin21:03:542
472225915,28sleep24766-21/usr/sbin/munin20:39:002
2862525912,28sleep228631-21ntp_states21:54:042
2862525912,28sleep228631-21ntp_states21:54:042
2851425918,20sleep30-21swapper/320:13:383
1417525917,16sleep20-21swapper/222:53:372
1417525917,16sleep20-21swapper/222:53:372
1412425913,29sleep014143-21munin-run22:53:350
1412425913,29sleep014143-21munin-run22:53:350
119925910,19sleep20-21swapper/220:28:472
992225815,19sleep30-21swapper/319:13:443
784525817,23sleep27888-21/usr/sbin/munin20:49:012
380125815,19sleep20-21swapper/222:18:582
2809925819,23sleep20-21swapper/220:09:092
2611125815,24sleep20-21swapper/220:04:042
2611125815,24sleep20-21swapper/220:04:042
1775025817,17sleep30-21swapper/323:03:553
1658125814,26sleep00-21swapper/021:18:370
1341725815,20sleep00-21swapper/019:23:580
78222575,12sleep00-21swapper/000:14:040
721525715,20sleep00-21swapper/022:29:090
551525710,30sleep25539-21ntp_states22:24:032
2377425714,26sleep2367-21snmpd23:23:482
2303125716,18sleep20-21swapper/219:54:072
2303125716,18sleep20-21swapper/219:54:072
1905925714,21sleep30-21swapper/321:24:103
1411325715,18sleep20-21swapper/221:09:032
900225613,26sleep08996-21apt-get20:53:510
900225613,26sleep08996-21apt-get20:53:510
865925615,16sleep00-21swapper/019:09:050
2171025615,24sleep221706-21apt-get21:33:492
209822569,27sleep30-21swapper/319:48:563
209822569,27sleep30-21swapper/319:48:563
1647725612,24sleep216559-21mailstats22:59:062
1175825611,25sleep00-21swapper/019:18:570
381625512,17sleep30-21swapper/322:18:593
351625513,23sleep03535-21apt-get22:18:480
2998225511,25sleep330016-21irqstats21:58:573
2941425516,19sleep00-21swapper/020:14:070
2941425516,19sleep00-21swapper/020:14:070
2724425514,18sleep30-21swapper/323:34:033
2419125510,20sleep00-21swapper/021:39:120
192525519,21sleep30-21swapper/322:13:463
1415825513,26sleep014178-21open_inodes21:09:050
483625419,19sleep30-21swapper/300:04:063
2418025411,18sleep00-21swapper/019:58:540
2244225410,18sleep00-21swapper/019:53:410
2244225410,18sleep00-21swapper/019:53:410
1764425415,16sleep30-21swapper/319:38:383
1475825414,23sleep014753-21apt-get19:28:500
713425311,18sleep30-21swapper/322:29:083
2720525313,23sleep027199-21apt-get20:08:500
2696925315,23sleep00-21swapper/020:08:380
2696925315,23sleep00-21swapper/020:08:380
2434925311,20sleep30-21swapper/323:24:063
1936825311,23sleep00-21swapper/023:09:010
1884425317,21sleep01-21systemd23:08:370
1599025313,23sleep315985-21apt-get22:58:513
1205425314,24sleep312050-21apt-get00:28:523
3260525215,17sleep20-21swapper/222:08:372
1629925213,22sleep016296-21apt-get19:33:470
1521425214,23sleep315220-21ntp_states19:29:063
87125112,25sleep3359-21apt-get23:53:483
6012516,34sleep20-21swapper/220:24:192
6012516,34sleep20-21swapper/220:24:192
40325112,24sleep3398-21apt-get22:08:543
3006925113,22sleep030075-21df_inode23:43:520
2250225111,23sleep022521-21apt-get19:53:490
2250225111,23sleep022521-21apt-get19:53:490
146832518,30sleep3367-21snmpd21:10:093
1320225114,21sleep313204-21apt-get19:23:493
97442507,15sleep30-21swapper/322:38:493
47182508,18sleep00-21swapper/000:04:030
364025015,22sleep32680-21/usr/sbin/munin23:59:303
3201625013,18sleep20-21swapper/223:49:072
719824917,17sleep30-21swapper/320:48:373
338624911,16sleep20-21swapper/220:34:072
3012624916,18sleep30-21swapper/320:18:383
3012624916,18sleep30-21swapper/320:18:383
2098124912,22sleep220973-21apt-get19:48:542
2098124912,22sleep220973-21apt-get19:48:542
1917024912,20sleep319178-21df_inode23:08:563
2853624810,21sleep028631-21perl23:38:560
2554824712,19sleep20-21swapper/223:29:012
1808324712,19sleep20-21swapper/221:23:362
1441224711,23sleep314427-21grep22:53:503
1199224715,17sleep00-21swapper/022:44:070
1199224715,17sleep00-21swapper/022:44:070
1021424710,20sleep310246-21munin-run00:23:353
81332469,16sleep00-21swapper/020:49:070
40224611,16sleep20-21swapper/222:08:532
2974724611,18sleep00-21swapper/021:58:500
1308724611,21sleep313138-21irqstats22:49:023
1308724611,21sleep313138-21irqstats22:49:023
248112429,15sleep20-21swapper/221:43:462
147602429,17sleep20-21swapper/219:28:522
762924113,16sleep30-21swapper/300:13:593
431524111,13sleep20-21swapper/200:03:532
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional