You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-28 - 23:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Thu May 28, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
127029775,964sleep029084-1kworker/0:2H09:59:000
17368274029,673sleep217376-21ntpq07:24:032
17368274029,673sleep217376-21ntpq07:24:032
3102250414,472sleep13144-21unin-run08:23:371
1105924958,474sleep311067-21ntpq10:29:033
1381244985,318sleep21510-21touch11:43:382
1200923678,345sleep212069-21cat07:08:572
1415823550,8sleep30-21swapper/308:58:383
9623234110,319sleep29637-21fschecks_count12:08:532
1156323387,319sleep211616-21ntpq08:49:012
2369323206,40sleep20-21swapper/209:26:482
1380229613,266sleep11412-21fschecks_count09:59:211
2320228817,254sleep32377-21/usr/sbin/munin11:45:083
2320228817,254sleep32377-21/usr/sbin/munin11:45:083
2664922479,224sleep026650-21sshd09:34:580
7753224311,215sleep37774-21latency_hist12:03:373
31244224119,205sleep031333-21grep08:08:540
13414221518,180sleep213559-21grep12:19:072
3403221029,146sleep23512-21perl11:48:552
21401221018,122sleep23108-1kworker/2:1H09:19:012
21272220337,152sleep221326-21idleruntime-cro07:38:362
1744220326,106sleep20-21swapper/211:43:462
1744220326,106sleep20-21swapper/211:43:462
31415219716,87sleep20-21swapper/211:34:052
645219511,102sleep20-21swapper/208:14:012
9609218410,139sleep29622-21cpu08:43:532
17237217628,108sleep217284-21ntpq10:49:002
3505217419,96sleep23660-21grep10:04:072
8545217327,81sleep28609-21awk08:39:042
18668217310,150sleep218754-21cat12:38:372
18860217131,99sleep218926-21sendmail_mailqu10:54:052
5497217012,74sleep20-21swapper/210:11:582
5497217012,74sleep20-21swapper/210:11:582
493621698,87sleep24985-21awk08:28:512
493621698,87sleep24985-21awk08:28:512
2546921689,14sleep00-21swapper/011:14:090
1317421617,125sleep213016-21kworker/2:210:35:272
11060216023,33sleep20-21swapper/212:13:462
183321547,90sleep2363-21snmpd08:18:502
4699215271,15sleep20-21swapper/208:28:392
4699215271,15sleep20-21swapper/208:28:392
31890214821,110sleep20-21swapper/209:53:492
16048214816,118sleep216066-21tr12:28:562
30814214614,116sleep330833-21apt-get11:33:463
2362221398,92sleep223690-21systemctl07:44:052
23405213519,97sleep323424-21uname07:43:583
26240213439,77sleep226315-21latency_hist07:53:382
25676213326,40sleep20-21swapper/209:33:512
28797213224,82sleep20-21swapper/209:43:522
14312213062,52sleep20-21swapper/212:23:552
8802212918,17sleep10-21swapper/108:39:071
1497521299,93sleep215121-21unixbench_multi08:59:072
22053212861,51sleep20-21swapper/207:39:052
1473212828,87sleep21512-21idleruntime09:59:262
4983212711,102sleep05094-21sed10:09:050
4983212711,102sleep05094-21sed10:09:050
18461212725,42sleep218436-21/usr/sbin/munin07:28:512
15735212664,17sleep20-21swapper/207:19:022
6208212419,26sleep20-21swapper/210:14:002
4125212217,19sleep10-21swapper/110:08:381
24692211811,44sleep20-21swapper/211:13:552
26290211676,25sleep226380-21fschecks_time11:18:552
26290211676,25sleep226380-21fschecks_time11:18:552
21828211314,79sleep021863-21/usr/sbin/munin07:38:590
7737211237,12sleep20-21swapper/208:38:362
7737211237,12sleep20-21swapper/208:38:362
3470211114,76sleep33528-21fschecks_count08:23:553
3470211114,76sleep33528-21fschecks_count08:23:553
19977211010,82sleep320048-21sensors_temp09:14:063
16080211020,71sleep315568-21apt-get09:03:483
1628621099,86sleep216356-21fschecks_time09:04:522
9726210812,15sleep20-21swapper/210:24:052
8187210820,70sleep08311-21timerwakeupswit10:19:110
28143210829,67sleep228187-21ntpq11:24:002
28143210829,67sleep228187-21ntpq11:24:002
30065210733,13sleep20-21swapper/208:04:042
681521069,82sleep26864-21awk11:59:002
28052210622,17sleep20-21swapper/207:58:512
28052210622,17sleep20-21swapper/207:58:512
1086821058,84sleep210886-21kernelversion10:28:572
616210415,71sleep1642-21netstat08:14:001
24099210320,34sleep30-21swapper/309:28:473
23331210321,15sleep20-21swapper/211:08:582
20709210219,61sleep20-21swapper/209:18:372
8439210114,73sleep38449-21ntp_states12:04:023
5303210126,15sleep20-21swapper/211:54:002
30601210164,18sleep20-21swapper/211:33:372
30601210164,18sleep20-21swapper/211:33:372
2925429914,68sleep129260-21ntp_states09:44:031
2756029917,69sleep027698-21sed11:23:380
2003629928,16sleep220037-21diskstats10:58:532
2003629928,16sleep220037-21diskstats10:58:532
186062998,75sleep218744-21grep09:09:322
1790929914,68sleep117879-21apt-get09:09:031
1050529923,55sleep210211-21/usr/sbin/munin07:03:572
1186729812,70sleep011899-21munin-run10:33:360
2619229730,37sleep30-21swapper/311:18:483
2619229730,37sleep30-21swapper/311:18:483
2174129614,18sleep10-21swapper/109:19:081
133212968,76sleep213410-21timerandwakeup08:54:062
1132029610,72sleep0363-21snmpd07:06:210
278842950,7sleep10-21swapper/109:39:061
2412629522,56sleep20-21swapper/209:28:502
1751629511,68sleep117690-21latency_hist09:08:381
1410629511,25sleep214145-21ntpq10:38:592
86132948,72sleep38673-21awk08:39:053
2539329314,64sleep325400-21open_files07:49:073
245029311,59sleep12552-21sed11:45:101
245029311,59sleep12552-21sed11:45:101
316932928,31sleep20-21swapper/208:09:052
215082929,19sleep20-21swapper/211:03:492
215082929,19sleep20-21swapper/211:03:492
159929215,57sleep01737-21latency_hist08:18:390
126129223,48sleep31243-21apt-get09:58:553
2818929115,63sleep328199-21ntp_states11:24:013
2818929115,63sleep328199-21ntp_states11:24:013
2618629121,45sleep026206-21munin-run07:53:360
1428629132,34sleep114208-21/usr/sbin/munin12:23:541
1513029013,63sleep015275-21sed07:18:400
1394629019,54sleep3307-21in:imuxsock12:23:363
79912899,63sleep07992-21http12:03:510
79102897,71sleep27954-21ntpq10:19:042
282852895,72sleep20-21swapper/209:40:452
2204628910,13sleep10-21swapper/107:39:041
1450228921,28sleep00-21swapper/008:58:560
135022896,17sleep20-21swapper/207:13:412
2212528813,60sleep322235-21timerandwakeup11:04:073
2212528813,60sleep322235-21timerandwakeup11:04:073
1554428724,35sleep00-21swapper/012:28:380
50642867,11sleep10-21swapper/111:53:551
1575828626,42sleep20-21swapper/210:44:032
1489728511,52sleep10-21swapper/108:59:061
2674028424,31sleep00-21swapper/011:19:060
2671928313,53sleep20-21swapper/207:53:592
2671928313,53sleep20-21swapper/207:53:592
1659528318,49sleep015753-21/usr/sbin/munin12:29:060
1512628312,29sleep30-21swapper/308:59:083
1340628313,46sleep113500-21df07:13:401
1032228322,29sleep30-21swapper/307:03:543
1845728224,37sleep318454-21apt-get07:28:483
618528120,40sleep16206-21kernelversion10:13:591
2647728115,47sleep326496-21cpu07:53:513
2647728115,47sleep326496-21cpu07:53:513
1040228123,34sleep30-21swapper/310:28:383
4272800,6sleep30-21swapper/311:39:023
2576028013,54sleep125809-21fschecks_count09:33:541
1081128024,27sleep10-21swapper/107:04:041
685227817,24sleep30-21swapper/308:34:013
685227817,24sleep30-21swapper/308:34:013
33822788,56sleep10-21swapper/108:23:491
33822788,56sleep10-21swapper/108:23:491
314732789,33sleep20-21swapper/209:51:552
2774327810,15sleep10-21swapper/111:23:511
2774327810,15sleep10-21swapper/111:23:511
1788427810,34sleep30-21swapper/309:08:493
778127728,20sleep00-21swapper/012:03:390
2304727716,44sleep123111-21ps09:24:051
2171727719,28sleep10-21swapper/111:03:571
2171727719,28sleep10-21swapper/111:03:571
3202427620,29sleep00-21swapper/009:53:550
553727514,15sleep30-21swapper/311:54:053
2247027538,19sleep30-21swapper/307:39:103
1703927525,24sleep10-21swapper/112:33:351
1662127527,29sleep30-21swapper/307:23:353
1499327518,42sleep314994-21sshd07:17:583
2937127416,40sleep129471-21latency_hist08:03:381
2937127416,40sleep129471-21latency_hist08:03:381
849527317,41sleep20-21swapper/212:04:052
974127211,47sleep09791-21if_eth012:08:550
3081727219,37sleep130856-21/usr/sbin/munin09:49:041
2587827210,46sleep325953-21systemd-run11:18:363
1588827212,43sleep015964-21grep10:44:050
225527111,41sleep02297-21ntp_states11:45:050
225527111,41sleep02297-21ntp_states11:45:050
1051627114,43sleep11-21systemd12:09:571
968327018,22sleep00-21swapper/008:43:550
964127016,28sleep30-21swapper/312:08:543
3196427014,25sleep10-21swapper/109:53:521
2810327022,27sleep00-21swapper/007:58:530
2810327022,27sleep00-21swapper/007:58:530
192227013,42sleep01977-21fschecks_count08:18:560
1414727019,23sleep30-21swapper/310:39:003
697926917,31sleep00-21swapper/008:34:060
697926917,31sleep00-21swapper/008:34:060
3001026914,24sleep00-21swapper/008:04:020
2595826919,25sleep10-21swapper/111:18:381
2056626921,26sleep30-21swapper/310:59:063
2056626921,26sleep30-21swapper/310:59:063
1823926917,35sleep118380-21cat07:28:381
1823926917,35sleep118380-21cat07:28:381
1714326913,36sleep00-21swapper/010:48:580
730426813,40sleep07452-21latency_hist10:18:380
2842626813,41sleep028562-21awk11:24:060
2842626813,41sleep028562-21awk11:24:060
2574426817,25sleep30-21swapper/309:33:523
2439626816,24sleep00-21swapper/007:48:360
1730226818,21sleep10-21swapper/110:49:031
2666126715,22sleep10-21swapper/107:53:581
2666126715,22sleep10-21swapper/107:53:581
1646026725,24sleep30-21swapper/312:29:053
1163126717,19sleep00-21swapper/012:14:040
2721026613,31sleep027345-21grep07:54:090
2721026613,31sleep027345-21grep07:54:090
2441326620,27sleep124512-21sed07:48:371
2037726612,34sleep00-21swapper/007:33:580
1743926622,25sleep30-21swapper/310:49:053
988026520,20sleep10-21swapper/108:44:001
3012826520,26sleep030274-21grep09:48:380
45032647,11sleep00-21swapper/008:26:470
45032647,11sleep00-21swapper/008:26:470
2968626416,30sleep129693-21/usr/sbin/munin08:03:551
1450826414,32sleep314600-21/usr/sbin/munin12:24:013
936726313,25sleep00-21swapper/008:43:380
603926319,27sleep36125-21/usr/sbin/munin10:13:573
36126312,27sleep00-21swapper/011:39:000
280926316,29sleep02803-21apt-get10:03:480
2044326317,29sleep10-21swapper/110:59:041
2044326317,29sleep10-21swapper/110:59:041
1971926316,32sleep019795-21sed10:58:390
178542636,11sleep00-21swapper/007:24:330
178542636,11sleep00-21swapper/007:24:330
122282635,19sleep00-21swapper/008:51:050
483226218,31sleep14847-21open_inodes10:09:021
483226218,31sleep14847-21open_inodes10:09:021
3063726220,18sleep00-21swapper/009:48:590
497826116,20sleep00-21swapper/011:53:530
3249926119,21sleep10-21swapper/108:13:401
2005126115,30sleep020137-21fschecks_time10:58:550
2005126115,30sleep020137-21fschecks_time10:58:550
1920226112,32sleep019335-21latency_hist09:13:380
163942616,10sleep30-21swapper/310:46:433
130772616,12sleep30-21swapper/310:34:113
1130326119,18sleep10-21swapper/108:48:561
108112619,12sleep10-21swapper/110:28:561
462126013,25sleep00-21swapper/011:53:370
328026013,22sleep10-21swapper/111:48:461
3165426018,21sleep30-21swapper/309:53:383
2380426018,18sleep30-21swapper/311:09:073
2324726019,19sleep00-21swapper/011:08:560
2278426017,22sleep00-21swapper/009:23:560
1381926014,29sleep113879-21fschecks_time10:38:541
127552605,10sleep10-21swapper/107:09:201
1223926016,28sleep112240-21/usr/sbin/munin10:33:531
931425914,25sleep10-21swapper/110:23:561
85825917,18sleep30-21swapper/309:58:383
1794425924,19sleep30-21swapper/312:34:063
1238025916,27sleep30-21swapper/307:09:063
1182625916,26sleep011820-21apt-get07:08:510
535425814,17sleep00-21swapper/008:29:010
535425814,17sleep00-21swapper/008:29:010
2805325816,17sleep30-21swapper/307:58:523
2805325816,17sleep30-21swapper/307:58:523
2680825815,22sleep10-21swapper/111:19:071
2680825815,22sleep10-21swapper/111:19:071
2546725815,26sleep125536-21/usr/sbin/munin07:49:091
210425816,17sleep30-21swapper/308:18:593
204552587,40sleep00-21swapper/009:15:280
1941625816,25sleep119435-21apt-get09:13:471
168792585,42sleep30-21swapper/307:23:433
168792585,42sleep30-21swapper/307:23:433
1582725820,18sleep10-21swapper/107:19:051
1405125817,23sleep10-21swapper/112:23:381
1269425815,26sleep012763-21sensors_temp10:34:040
1236425815,20sleep30-21swapper/312:18:353
2990625716,20sleep00-21swapper/011:29:070
2990625716,20sleep00-21swapper/011:29:070
1609325712,22sleep00-21swapper/009:03:530
87422565,10sleep30-21swapper/310:23:213
554425614,29sleep15617-21grep08:29:061
554425614,29sleep15617-21grep08:29:061
53142564,12sleep30-21swapper/308:29:003
53142564,12sleep30-21swapper/308:29:003
2936325613,24sleep129369-21df_inode11:28:511
2936325613,24sleep129369-21df_inode11:28:511
2926625610,17sleep30-21swapper/311:28:453
2926625610,17sleep30-21swapper/311:28:453
2305625614,25sleep123051-21apt-get11:08:491
2224125616,17sleep00-21swapper/011:04:080
2224125616,17sleep00-21swapper/011:04:080
1536425614,27sleep015358-21apt-get07:18:520
1244325612,23sleep10-21swapper/112:18:371
2780625511,21sleep00-21swapper/009:39:060
253442557,39sleep00-21swapper/009:33:040
1779225512,17sleep00-21swapper/012:34:030
1400225512,28sleep014062-21latency10:38:570
1008625518,19sleep30-21swapper/308:44:063
648225413,24sleep16477-21apt-get08:33:501
648225413,24sleep16477-21apt-get08:33:501
40925415,23sleep1416-21ntp_states11:39:011
3176425418,19sleep30-21swapper/308:09:063
2412525411,25sleep124120-21apt-get09:28:491
2135325419,19sleep10-21swapper/107:38:391
2102925414,22sleep021023-21apt-get09:18:510
2000825413,24sleep320002-21apt-get07:33:493
1858725417,23sleep1284-21dbus-daemon12:38:361
1161825414,24sleep311620-21ntp_states08:49:023
799325311,15sleep10-21swapper/112:03:521
770225313,24sleep17730-21munin-run12:03:351
764225311,26sleep17717-21perl10:18:591
42682538,34sleep30-21swapper/311:49:453
2233725313,24sleep322475-21cat09:23:383
479225212,22sleep34821-21ntp_states10:09:003
479225212,22sleep34821-21ntp_states10:09:003
2257625213,23sleep322644-21df09:23:513
82625116,21sleep3914-21sed08:14:063
131192519,16sleep10-21swapper/108:54:011
301925015,19sleep30-21swapper/310:03:563
195852507,35sleep10-21swapper/107:31:531
1525325014,21sleep115272-21apt-get10:43:461
1293925015,17sleep00-21swapper/012:18:570
300282494,11sleep30-21swapper/309:48:173
2960924912,21sleep329677-21df08:03:533
1874824911,17sleep10-21swapper/110:53:591
1837624914,19sleep30-21swapper/310:53:493
945124812,16sleep00-21swapper/010:23:590
1845924813,18sleep00-21swapper/007:28:490
135652487,14sleep00-21swapper/008:54:080
3217124711,21sleep132181-21munin-run11:38:361
3084324710,16sleep00-21swapper/011:33:510
282512464,9sleep30-21swapper/309:40:183
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional