You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-02 - 16:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Fri Jan 02, 2026 12:43:50)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21398269319,6911sleep314990-21kworker/3:007:13:283
24389276714,736sleep024427-21/usr/sbin/munin07:19:180
580027458,702sleep35841-21ntpq08:04:163
25940238519,341sleep016523-1kworker/0:1H09:09:240
23171236418,188sleep323326-21unixbench_multi08:59:213
20445236215,334sleep320447-21cpu12:19:043
20445236215,334sleep320447-21cpu12:19:043
26076236110,190sleep325812-21/usr/sbin/munin09:09:553
15564230725,269sleep315597-21ntpq10:19:143
15564230725,269sleep315597-21ntpq10:19:143
2484023047,232sleep324880-21ntpq10:49:143
25442227724,238sleep325481-21ntpq12:34:153
29699226314,209sleep329844-21tail09:19:203
2514225951,186sleep32591-21latency_hist09:38:513
11134225816,222sleep311172-21latency11:49:123
11134225816,222sleep311172-21latency11:49:123
38652251109,29sleep30-21swapper/307:59:033
18033224864,35sleep318181-21latency_hist10:28:513
31973223951,171sleep332057-21latency_hist11:13:503
819822298,209sleep0369-21snmpd08:13:340
22627222941,173sleep322671-21systemd-run10:43:493
29753222523,186sleep329872-21timerwakeupswit11:04:213
20016222445,120sleep30-21swapper/310:34:123
9844222229,180sleep39900-21sed08:18:503
236752209119,70sleep323809-21unixbench_singl10:44:223
18625220719,150sleep318666-21ntp_kernel_err10:29:153
10653220740,16sleep30-21swapper/311:48:513
6383220548,100sleep30-21swapper/309:49:213
29908219866,94sleep330044-21latency_hist07:38:523
29908219866,94sleep330044-21latency_hist07:38:523
31596219510,173sleep0369-21snmpd09:26:080
21540219413,163sleep021584-21fw_packets10:39:100
1479221918,172sleep0369-21snmpd10:18:060
1479221918,172sleep0369-21snmpd10:18:060
2379218913,154sleep30-21swapper/307:54:073
73221879,24sleep30-21swapper/307:49:063
73221879,24sleep30-21swapper/307:49:063
29454217918,144sleep229465-21/usr/sbin/munin07:34:272
29454217918,144sleep229465-21/usr/sbin/munin07:34:272
8248217530,107sleep38288-21uname11:39:183
14164217410,13sleep30-21swapper/310:14:213
14164217410,13sleep30-21swapper/310:14:213
21743216217,19sleep00-21swapper/012:23:500
21743216217,19sleep00-21swapper/012:23:500
3204216011,40sleep30-21swapper/311:24:083
3204216011,40sleep30-21swapper/311:24:083
17073215835,85sleep317035-21latency_hist12:08:503
13243215642,49sleep30-21swapper/308:29:083
26270215265,48sleep326311-21latency10:54:113
13659214711,101sleep313747-21systemd-run11:58:493
31127214640,42sleep331207-21switchtime09:24:213
27695214413,119sleep327733-21wc09:14:113
167621448,12sleep10-21swapper/111:19:091
17577214143,62sleep317590-21latency_hist08:43:503
24596213821,44sleep30-21swapper/307:19:213
25805213721,97sleep125315-21apt-get09:09:001
24461213733,71sleep124455-21apt-get10:49:041
1445213519,95sleep11587-21sed07:49:281
1445213519,95sleep11587-21sed07:49:281
10213213515,46sleep110237-21basename10:03:491
24119213110,68sleep30-21swapper/309:04:063
24119213110,68sleep30-21swapper/309:04:063
15176213014,97sleep00-21swapper/011:59:590
1189421279,74sleep30-21swapper/308:24:113
20717212530,80sleep320867-21latency_hist08:53:523
1162621256,99sleep00-21swapper/008:24:040
447212416,92sleep0470-21ntp_kernel_pll_11:15:000
24016212216,63sleep30-21swapper/312:29:183
13749212221,82sleep013843-21latency_hist11:58:500
4833212113,90sleep30-21swapper/309:44:203
6814212023,71sleep30-21swapper/311:34:213
3210421197,69sleep30-21swapper/309:29:063
31040211816,81sleep031066-21netstat11:09:140
30094211410,41sleep30-21swapper/307:39:003
22956211420,41sleep30-21swapper/307:15:143
22956211420,41sleep30-21swapper/307:15:143
1932721147,70sleep319389-21ps12:14:203
1932721147,70sleep319389-21ps12:14:203
27184211317,70sleep20-21swapper/209:13:502
1470421138,42sleep30-21swapper/308:34:013
22383211215,78sleep022417-21/usr/sbin/munin12:24:140
736121107,69sleep37401-21ntpq08:09:133
18097210912,81sleep118154-21latency08:44:131
18097210912,81sleep118154-21latency08:44:131
14794210926,30sleep10-21swapper/108:34:071
454221080,5sleep20-21swapper/211:28:502
454221080,5sleep20-21swapper/211:28:502
29495210812,82sleep029534-21ntpq11:04:140
2556621077,62sleep325669-21perl07:24:103
2415321070,9sleep10-21swapper/112:29:191
10501210710,56sleep30-21swapper/310:04:033
32452210610,18sleep30-21swapper/311:14:063
2151321060,11sleep00-21swapper/008:54:200
2151321060,11sleep00-21swapper/008:54:200
11320210611,82sleep011321-21ntp_states11:49:170
11320210611,82sleep011321-21ntp_states11:49:170
27490210529,54sleep30-21swapper/307:29:233
9018210416,70sleep19108-21mailstats08:14:241
8637210419,64sleep28692-21awk09:58:492
18618210316,65sleep018693-21latency_hist12:13:520
4427210015,69sleep14540-21cat11:28:501
4427210015,69sleep14540-21cat11:28:501
229229912,20sleep10-21swapper/107:54:011
1215629814,70sleep112161-21systemd-run11:53:501
1215629814,70sleep112161-21systemd-run11:53:501
939829623,32sleep30-21swapper/309:59:193
3242029622,52sleep131941-21apt-get11:14:001
2017729616,66sleep120201-21cpu07:09:061
365129512,68sleep23662-21vmstat09:39:232
365129512,68sleep23662-21vmstat09:39:232
289429535,44sleep33037-21sed11:23:513
602829413,62sleep16104-21/usr/sbin/munin08:04:231
1800829414,43sleep318071-21cat08:44:113
1800829414,43sleep318071-21cat08:44:113
3245129315,22sleep20-21swapper/211:14:052
3185629315,36sleep10-21swapper/109:28:511
1525229222,54sleep115291-21cut10:19:081
1525229222,54sleep115291-21cut10:19:081
42032915,74sleep00-21swapper/011:26:350
42032915,74sleep00-21swapper/011:26:350
2599229110,24sleep00-21swapper/010:54:000
1117729113,62sleep011291-21sed10:04:210
2244829015,39sleep30-21swapper/312:24:173
3049128912,62sleep030520-21/usr/sbin/munin07:39:130
870528810,65sleep0369-21snmpd11:39:310
1422128711,35sleep30-21swapper/311:59:053
759428625,43sleep30-21swapper/309:54:103
610528621,43sleep26252-21grep08:04:242
1547828612,59sleep215517-21modprobe12:03:492
2484928511,50sleep024991-21latency_hist12:33:510
2356028516,53sleep20-21swapper/210:44:212
1162228522,39sleep111619-21apt-get08:24:011
297728420,44sleep03108-21unixbench_multi07:54:200
172792837,13sleep30-21swapper/312:08:543
101752838,42sleep310176-21diskstats08:19:153
275732828,14sleep30-21swapper/310:59:043
1690128215,43sleep116256-21/usr/sbin/munin08:39:241
962028114,20sleep00-21swapper/009:59:240
464128114,33sleep30-21swapper/311:29:003
318852817,60sleep30-21swapper/307:44:133
3175328012,49sleep131826-21fschecks_time07:44:111
2554828019,41sleep225565-21fschecks_count07:24:092
243628022,41sleep12495-21sendmail09:38:491
2159428014,50sleep121707-21timerwakeupswit08:54:221
2159428014,50sleep121707-21timerwakeupswit08:54:221
1588828014,43sleep215141-21/usr/sbin/munin10:19:212
1588828014,43sleep215141-21/usr/sbin/munin10:19:212
1985327914,46sleep019908-21/usr/sbin/munin08:49:210
1712527913,16sleep20-21swapper/210:24:132
518127821,24sleep20-21swapper/208:03:512
2271727814,44sleep122859-21sed10:43:511
2193627818,27sleep30-21swapper/308:55:463
2193627818,27sleep30-21swapper/308:55:463
2149627819,46sleep221605-21apt-key07:13:512
704827722,34sleep10-21swapper/108:09:071
584427720,24sleep10-21swapper/109:49:071
389627718,26sleep30-21swapper/309:42:263
389627718,26sleep30-21swapper/309:42:263
2179727719,37sleep10-21swapper/112:23:511
534927613,25sleep10-21swapper/111:29:231
2422327614,20sleep00-21swapper/010:48:510
2961227530,28sleep10-21swapper/111:04:191
2919027517,28sleep00-21swapper/009:19:090
2567127413,41sleep025711-21if_eth007:24:110
2954127310,48sleep129611-21/usr/sbin/munin09:19:181
2864327321,24sleep00-21swapper/007:34:130
2864327321,24sleep00-21swapper/007:34:130
2567927318,20sleep10-21swapper/112:34:211
621327218,26sleep00-21swapper/011:34:060
3218127211,14sleep20-21swapper/209:29:092
3142627223,28sleep10-21swapper/107:43:511
2762127215,40sleep127719-21cat07:29:251
1484927218,38sleep214879-21awk10:18:482
1484927218,38sleep214879-21awk10:18:482
504127116,31sleep25183-21unixbench_singl09:44:232
2826827122,25sleep10-21swapper/110:59:221
2804927116,24sleep20-21swapper/210:59:192
2007827113,27sleep10-21swapper/110:34:141
1984827126,27sleep119878-21munin-run07:08:491
1638327117,38sleep216471-21grep08:39:132
1647127020,24sleep00-21swapper/010:23:510
1647127020,24sleep00-21swapper/010:23:510
931526915,36sleep09413-21users08:14:270
747026915,32sleep17528-21fschecks_count09:54:081
746526913,40sleep07469-21diskstats09:54:070
2222526922,26sleep122259-21users07:13:581
2222526922,26sleep122259-21users07:13:581
1955726918,28sleep20-21swapper/210:33:512
3115826828,18sleep10-21swapper/111:09:201
2939626823,21sleep20-21swapper/211:04:122
129772686,16sleep00-21swapper/010:09:380
1250526830,19sleep30-21swapper/310:09:163
37042675,10sleep00-21swapper/009:39:530
37042675,10sleep00-21swapper/009:39:530
2207126716,30sleep221441-21apt-get07:13:552
2207126716,30sleep221441-21apt-get07:13:552
2071126718,29sleep20-21swapper/212:19:112
2071126718,29sleep20-21swapper/212:19:112
1936826718,27sleep10-21swapper/108:49:051
1852726719,28sleep10-21swapper/110:29:131
1384426716,36sleep113968-21sed08:29:221
639426612,14sleep10-21swapper/111:34:111
1776426618,27sleep10-21swapper/112:09:171
1707226610,26sleep00-21swapper/010:24:110
146802667,13sleep00-21swapper/008:33:580
432826514,22sleep00-21swapper/007:59:170
3124126516,24sleep20-21swapper/211:09:212
1713026518,22sleep00-21swapper/012:08:510
838026416,25sleep10-21swapper/111:39:201
2722026422,17sleep20-21swapper/207:29:142
2229226419,26sleep20-21swapper/210:39:282
140426410,23sleep00-21swapper/011:18:520
3034926313,34sleep230502-21date09:23:522
1637726316,31sleep116478-21sed12:04:201
312026216,29sleep13149-21/usr/sbin/munin09:39:131
312026216,29sleep13149-21/usr/sbin/munin09:39:131
240412626,11sleep10-21swapper/109:04:041
240412626,11sleep10-21swapper/109:04:041
2372326213,25sleep10-21swapper/109:03:271
213702629,40sleep00-21swapper/007:13:040
2051926216,22sleep10-21swapper/112:19:061
1086326218,27sleep210859-21apt-get11:49:022
1086326218,27sleep210859-21apt-get11:49:022
929926114,31sleep29316-21/usr/sbin/munin11:44:022
2541726116,22sleep00-21swapper/009:08:510
2541726116,22sleep00-21swapper/009:08:510
232052618,12sleep20-21swapper/212:28:242
1904026118,26sleep219098-21/usr/sbin/munin12:14:112
1904026118,26sleep219098-21/usr/sbin/munin12:14:112
1384526115,21sleep10-21swapper/111:58:511
1017826122,21sleep00-21swapper/008:19:160
2353326020,24sleep20-21swapper/212:29:052
1029726020,18sleep20-21swapper/208:19:182
78822595,12sleep20-21swapper/208:09:282
547925914,26sleep25570-21users11:29:242
1835525916,23sleep00-21swapper/008:44:200
1835525916,23sleep00-21swapper/008:44:200
1731725915,20sleep20-21swapper/212:09:062
1473425915,27sleep10-21swapper/111:59:521
1442125919,24sleep01-21systemd08:33:500
45825817,24sleep2467-21modprobe07:48:492
3211925815,22sleep00-21swapper/007:44:180
309522586,12sleep20-21swapper/209:24:162
3069225819,21sleep20-21swapper/207:39:202
2742725814,27sleep027420-21apt-get09:14:040
2333325814,29sleep023392-21users08:59:220
2225225815,22sleep222379-21grep08:58:522
2225225815,22sleep222379-21grep08:58:522
1904925820,25sleep019058-21ntp_states07:04:170
1759325812,23sleep00-21swapper/008:43:510
1248325816,27sleep212492-21/usr/sbin/munin11:54:072
1100725819,19sleep20-21swapper/210:04:182
848625713,28sleep28490-21find08:14:052
407525718,19sleep10-21swapper/107:59:101
2816625714,18sleep10-21swapper/109:14:221
2291725715,27sleep122911-21apt-get10:44:031
1689225713,20sleep10-21swapper/110:24:081
1335225717,18sleep20-21swapper/208:29:112
914925613,20sleep10-21swapper/111:43:521
895425621,17sleep20-21swapper/209:59:052
2602325612,32sleep0369-21snmpd12:34:520
2257525616,24sleep222587-21fschecks_count08:59:082
1544325616,25sleep215576-21sed08:34:202
95525516,20sleep10-21swapper/109:33:521
632525515,19sleep20-21swapper/209:49:202
560225519,17sleep10-21swapper/109:48:511
466425513,25sleep04660-21apt-get11:29:010
2680925513,24sleep126901-21users10:54:211
20242558,31sleep22056-21systemd-run07:53:492
20242558,31sleep22056-21systemd-run07:53:492
2019225515,20sleep10-21swapper/112:18:501
2019225515,20sleep10-21swapper/112:18:501
1903625514,23sleep319047-21ntp_states07:04:163
175825519,22sleep01835-21sort09:34:200
1019925512,21sleep10-21swapper/108:19:171
953325413,21sleep00-21swapper/011:44:120
667525413,25sleep01-21systemd08:08:500
584325414,25sleep05834-21apt-get09:49:060
2763725412,16sleep20-21swapper/209:14:102
2231225412,22sleep00-21swapper/007:14:010
2231225412,22sleep00-21swapper/007:14:010
1364025415,22sleep113662-21df10:14:071
1364025415,22sleep113662-21df10:14:071
1333825414,24sleep1228-21systemd-journal10:13:501
814925313,18sleep20-21swapper/211:39:132
2621225314,23sleep226269-21irqstats10:54:102
2529425317,22sleep225340-21http09:08:492
2529425317,22sleep225340-21http09:08:492
2445225314,23sleep224455-21apt-get10:49:012
2389525315,21sleep20-21swapper/207:19:002
236925313,15sleep20-21swapper/207:54:062
1990025314,22sleep219919-21munin-run07:08:502
1849325314,19sleep00-21swapper/010:29:120
1587925314,23sleep20-21swapper/212:04:092
2766025216,19sleep00-21swapper/010:59:090
1988125213,23sleep019885-21diskstats10:34:090
1096725214,23sleep111004-21ps10:04:171
279332515,37sleep00-21swapper/007:29:320
1982425112,24sleep219830-21ntp_states08:49:192
3255825014,23sleep032569-21open_files09:29:190
287225010,16sleep20-21swapper/211:23:502
2543624914,15sleep10-21swapper/107:24:001
1939224915,18sleep00-21swapper/012:14:210
1939224915,18sleep00-21swapper/012:14:210
1293024915,17sleep20-21swapper/208:28:512
1249824914,18sleep20-21swapper/210:09:152
2576224810,24sleep225881-21unixbench_multi12:34:222
2432524818,14sleep10-21swapper/107:19:151
53172475,31sleep00-21swapper/009:45:470
225224714,17sleep20-21swapper/209:36:282
103424714,17sleep00-21swapper/007:49:190
103424714,17sleep00-21swapper/007:49:190
661924511,21sleep26661-21ntpq11:34:162
1782624510,13sleep20-21swapper/208:44:062
1782624510,13sleep20-21swapper/208:44:062
224182447,14sleep10-21swapper/112:24:151
285592437,14sleep10-21swapper/107:34:101
285592437,14sleep10-21swapper/107:34:101
247102438,17sleep00-21swapper/010:49:110
197992407,13sleep20-21swapper/210:34:072
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional