You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-21 - 01:46

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Sun Sep 20, 2026 12:43:59)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
162632503415,5006sleep114669-21kworker/1:110:39:551
12682123015,1202sleep016486-21kworker/0:208:12:380
2850826524,615sleep328619-1kworker/3:2H09:37:583
8474259813,572sleep38548-21cat11:58:073
8474259813,572sleep38548-21cat11:58:073
3210224739,446sleep332232-21apt-config11:32:383
7996237618,337sleep18093-21fschecks_time11:57:561
7996237618,337sleep18093-21fschecks_time11:57:561
18074233816,306sleep218191-21cat10:47:392
19945229385,192sleep319979-21cut10:52:573
5048228410,196sleep35087-21grep08:23:033
4486228064,199sleep34502-21idleruntime-cro11:47:373
3269222719,247sleep332765-21sed08:08:013
20269226346,29sleep30-21swapper/307:28:163
185832257118,42sleep318647-21cat10:48:003
185832257118,42sleep318647-21cat10:48:003
6036225317,215sleep3289-21dbus-daemon11:52:373
21800222711,204sleep021888-21cat07:33:010
2586322269,20sleep326016-21grep11:12:393
2586322269,20sleep326016-21grep11:12:393
22399221719,180sleep122483-21awk09:18:091
22399221719,180sleep122483-21awk09:18:091
9930221110,162sleep310036-21timerandwakeup08:38:103
22746218811,163sleep322772-21latency_hist11:02:383
32233218414,150sleep30-21swapper/308:07:413
2288221749,97sleep0367-21snmpd09:20:230
2288221749,97sleep0367-21snmpd09:20:230
765121738,69sleep37673-21sort11:57:383
1821021699,17sleep30-21swapper/307:23:073
1821021699,17sleep30-21swapper/307:23:073
21805216512,136sleep321774-21/usr/sbin/munin09:17:553
21805216512,136sleep321774-21/usr/sbin/munin09:17:553
1485216510,91sleep30-21swapper/311:37:393
1288321659,121sleep312913-21kernelversion07:08:003
103832165133,18sleep30-21swapper/310:22:573
4410216410,140sleep34489-21latency_hist08:22:393
1515216117,126sleep01511-21apt-get08:12:510
24357216017,96sleep324382-21modprobe07:42:373
1808721607,143sleep3781-21sendmail-mta09:04:063
1808721607,143sleep3781-21sendmail-mta09:04:063
5736215811,66sleep35763-21sshd10:07:583
1946215815,124sleep30-21swapper/309:57:373
734521558,135sleep37405-21cat10:12:593
2176421548,134sleep321502-21/usr/sbin/munin07:33:003
1503221537,79sleep30-21swapper/308:55:063
26151215110,95sleep326177-21munin-run09:32:373
1616221517,130sleep316225-21ps12:23:063
1616221517,130sleep316225-21ps12:23:063
3135621508,59sleep327943-21kworker/u8:309:47:393
3135621508,59sleep327943-21kworker/u8:309:47:393
774821478,96sleep30-21swapper/308:32:533
14438214323,98sleep00-21swapper/007:12:460
3026421425,11sleep30-21swapper/311:24:513
15489214214,112sleep015576-21sed10:38:000
1223821426,11sleep30-21swapper/312:12:163
2884321416,12sleep30-21swapper/307:55:373
19294214111,89sleep30-21swapper/309:08:103
527821395,41sleep30-21swapper/310:07:023
272421395,43sleep30-21swapper/311:39:493
1989921386,10sleep30-21swapper/312:35:423
3565213412,40sleep33611-21find10:02:373
2402021345,27sleep30-21swapper/311:04:403
1579821316,42sleep30-21swapper/307:17:053
2383521308,86sleep323919-21grep09:23:083
15754212822,23sleep30-21swapper/310:38:073
13828212471,41sleep313901-21systemd-run12:17:373
19473212316,24sleep00-21swapper/012:33:090
18195212211,29sleep10-21swapper/107:22:561
18195212211,29sleep10-21swapper/107:22:561
26057212131,12sleep30-21swapper/311:12:443
26057212131,12sleep30-21swapper/311:12:443
7135211914,92sleep17137-21acpi10:12:491
1642521167,61sleep30-21swapper/307:17:583
1642521167,61sleep30-21swapper/307:17:583
13966211328,68sleep214100-21wc07:12:392
27970211112,16sleep30-21swapper/309:37:383
1883211113,75sleep30-21swapper/308:13:023
20211210910,16sleep30-21swapper/309:12:503
13062210911,17sleep30-21swapper/310:28:543
27079210813,77sleep027113-21/usr/sbin/munin09:33:230
8559210644,43sleep30-21swapper/310:17:403
29773210515,74sleep029843-21switchtime11:23:070
27857210518,59sleep30-21swapper/311:17:573
5696210412,76sleep05700-21diskstats10:07:550
31384210414,75sleep131504-21timerandwakeup11:28:081
1599421048,14sleep30-21swapper/308:58:043
15328210417,69sleep215469-21latency_hist08:57:392
1105721049,52sleep30-21swapper/308:42:583
1105721049,52sleep30-21swapper/308:42:583
620521038,16sleep30-21swapper/308:27:563
14655210314,71sleep214659-21diskstats07:12:562
83021020,8sleep10-21swapper/108:08:101
17715210215,16sleep00-21swapper/007:22:390
17715210215,16sleep00-21swapper/007:22:390
30626210111,74sleep030693-21switchtime09:43:080
30626210111,74sleep030693-21switchtime09:43:080
24911210117,70sleep124920-21/usr/sbin/munin07:42:591
17927210114,69sleep218049-21unixbench_multi12:28:102
2642221007,43sleep30-21swapper/307:47:583
24408210020,21sleep10-21swapper/107:42:391
1792121007,82sleep317721-21kworker/3:210:46:193
2616729912,70sleep126186-21apt-get07:47:481
2011929812,71sleep120154-21latency_hist12:37:381
1896429715,65sleep219109-21unixbench_singl07:23:342
1896429715,65sleep219109-21unixbench_singl07:23:342
144429715,70sleep21483-21latency_hist11:37:382
143812976,12sleep30-21swapper/312:17:573
143812976,12sleep30-21swapper/312:17:573
136222967,14sleep30-21swapper/310:32:503
124902966,13sleep30-21swapper/308:47:553
250142957,11sleep30-21swapper/307:43:023
302362938,53sleep30-21swapper/307:58:163
1130229318,61sleep211364-21wc07:03:032
1097429323,44sleep211054-21fschecks_time08:42:572
1097429323,44sleep211054-21fschecks_time08:42:572
104512936,12sleep30-21swapper/312:04:123
104512936,12sleep30-21swapper/312:04:123
1081129222,54sleep310986-21latency_hist07:02:393
2468829115,60sleep224840-21latency_hist09:27:392
133929119,20sleep20-21swapper/208:12:402
3952909,62sleep0444-21unin-run09:52:370
3952909,62sleep0444-21unin-run09:52:370
175122900,7sleep10-21swapper/109:03:021
175122900,7sleep10-21swapper/109:03:021
1068029016,58sleep110823-21cut08:42:401
1002829018,49sleep29428-21/usr/sbin/munin12:03:062
1002829018,49sleep29428-21/usr/sbin/munin12:03:062
608728913,26sleep00-21swapper/011:52:380
325252895,11sleep30-21swapper/309:48:243
325252895,11sleep30-21swapper/309:48:243
1076528919,21sleep10-21swapper/112:07:381
1076528919,21sleep10-21swapper/112:07:381
172142875,11sleep30-21swapper/312:27:473
403028620,46sleep13546-21apt-get10:02:481
2226028632,34sleep122403-21grep07:33:101
2216228616,51sleep222301-21grep10:58:092
1780428615,57sleep117931-21cat07:22:401
1780428615,57sleep117931-21cat07:22:401
1700028414,55sleep217053-21latency_hist12:27:372
1700028414,55sleep217053-21latency_hist12:27:372
1004128421,41sleep010182-21unixbench_singl08:38:110
1835728319,44sleep218392-21awk09:07:362
1835728319,44sleep218392-21awk09:07:362
1412028324,29sleep10-21swapper/112:17:471
1412028324,29sleep10-21swapper/112:17:471
353228223,40sleep23560-21munin-run10:02:362
1991928128,33sleep1312-21in:imuxsock09:12:361
1505428010,52sleep114549-21/usr/sbin/munin07:13:071
1432328015,49sleep214445-21unixbench_multi10:33:092
473727916,48sleep14762-21grep08:22:571
3223827920,31sleep00-21swapper/011:32:400
1764427925,35sleep00-21swapper/012:28:040
1228627926,31sleep012299-21modprobe07:07:370
1152227916,47sleep111644-21timerwakeupswit08:43:081
1152227916,47sleep111644-21timerwakeupswit08:43:081
629027819,43sleep26331-21uniq08:27:592
2463027818,43sleep124646-21sort11:07:571
2463027818,43sleep124646-21sort11:07:571
1793227820,32sleep20-21swapper/207:22:412
1793227820,32sleep20-21swapper/207:22:412
140027814,39sleep0663-21/usr/sbin/munin09:53:100
1840127721,28sleep00-21swapper/010:47:570
1840127721,28sleep00-21swapper/010:47:570
1605527715,50sleep216146-21cut08:58:082
641127620,36sleep16299-21/usr/sbin/munin11:52:561
2356127514,17sleep20-21swapper/207:38:042
107672750,6sleep00-21swapper/010:23:050
558827410,47sleep15607-21apt-get10:07:481
3002127424,22sleep00-21swapper/007:58:090
2516827419,30sleep20-21swapper/209:28:002
2074227417,34sleep120886-21unixbench_singl07:28:261
435927316,39sleep14387-21idleruntime-cro08:22:371
224862737,12sleep00-21swapper/010:59:480
21342737,13sleep00-21swapper/011:38:050
1555027317,38sleep115524-21/usr/sbin/munin08:57:541
1538927326,29sleep20-21swapper/210:37:582
973427215,39sleep29776-21ntp_states08:38:052
811827211,25sleep00-21swapper/008:33:020
3135727216,36sleep131417-21/usr/sbin/munin08:03:061
248127221,25sleep00-21swapper/009:57:580
2310527219,34sleep223114-21latency_hist09:22:392
2310527219,34sleep223114-21latency_hist09:22:392
3100727115,22sleep00-21swapper/011:27:590
2846527119,36sleep127969-21apt-get09:37:481
828227015,39sleep18353-21/usr/sbin/munin08:33:071
632027011,25sleep20-21swapper/211:52:492
374727018,26sleep00-21swapper/008:18:080
2922026918,24sleep20-21swapper/211:22:522
2879626921,24sleep00-21swapper/009:38:200
284672697,21sleep20-21swapper/209:37:492
2697926916,38sleep227025-21sed09:33:212
1422626921,29sleep014230-21diskstats12:17:540
1422626921,29sleep014230-21diskstats12:17:540
1329226919,23sleep00-21swapper/007:08:090
1298526921,23sleep10-21swapper/107:08:021
453826816,21sleep10-21swapper/111:47:391
236726812,37sleep12489-21/usr/sbin/munin11:38:101
1854126811,39sleep10-21swapper/112:32:371
1208126817,28sleep00-21swapper/010:27:500
1184426814,45sleep00-21swapper/008:44:000
1184426814,45sleep00-21swapper/008:44:000
1005826816,22sleep20-21swapper/210:22:392
850726715,27sleep20-21swapper/210:17:392
359026711,14sleep20-21swapper/208:18:062
2730226717,31sleep20-21swapper/211:16:552
2730226717,31sleep20-21swapper/211:16:552
2015626716,30sleep220074-21latency_hist12:37:392
1414926712,41sleep014239-21grep10:33:070
1397226718,35sleep114029-21cpuspeed08:52:541
1151526720,31sleep00-21swapper/012:08:090
2628226621,25sleep126326-21munin-run09:32:391
2286226619,31sleep222987-21grep07:37:392
921426518,32sleep09250-21wc10:18:050
88826516,31sleep11018-21/usr/sbin/munin11:33:171
57072657,48sleep00-21swapper/008:25:020
499126516,20sleep20-21swapper/211:48:002
44526523,19sleep10-21swapper/109:52:381
44526523,19sleep10-21swapper/109:52:381
2350726517,33sleep023554-21/usr/sbin/munin07:38:020
204426518,29sleep10-21swapper/109:57:391
1676426519,27sleep10-21swapper/110:42:531
140862656,15sleep10-21swapper/110:33:041
1263826513,37sleep112647-21diskstats12:12:551
1023226518,21sleep00-21swapper/012:03:080
1023226518,21sleep00-21swapper/012:03:080
925126413,23sleep10-21swapper/110:18:061
809526418,17sleep20-21swapper/211:57:572
809526418,17sleep20-21swapper/211:57:572
2030626411,38sleep220316-21ntpq10:53:042
2000226418,20sleep00-21swapper/007:28:080
1962026416,24sleep20-21swapper/210:52:402
1962026416,24sleep20-21swapper/210:52:402
1695126414,34sleep116999-21systemd-run12:27:361
1695126414,34sleep116999-21systemd-run12:27:361
3161626318,28sleep031758-21grep08:03:100
2344826316,19sleep10-21swapper/111:03:041
213842635,10sleep00-21swapper/009:16:230
1306226324,17sleep00-21swapper/012:13:060
588826217,28sleep25942-21sed10:08:022
36352627,13sleep00-21swapper/011:43:050
2984426212,30sleep129963-21/usr/sbin/munin11:23:081
1540426221,23sleep20-21swapper/212:22:372
1540426221,23sleep20-21swapper/212:22:372
1230126220,25sleep112357-21unin-run07:07:381
835426116,26sleep28430-21/usr/sbin/munin08:33:082
332226115,20sleep20-21swapper/211:42:592
167542617,41sleep00-21swapper/012:25:010
167542617,41sleep00-21swapper/012:25:010
1522626114,30sleep00-21swapper/008:57:370
562226016,19sleep10-21swapper/111:48:111
3205226012,18sleep20-21swapper/211:32:372
2754026015,23sleep00-21swapper/007:52:400
2584626016,19sleep20-21swapper/211:12:382
2584626016,19sleep20-21swapper/211:12:382
2515526017,26sleep025245-21grep07:43:070
2429126014,29sleep224302-21munin-run11:07:382
2099626014,28sleep221115-21users09:13:102
1834126014,27sleep018354-21fschecks_count07:23:200
1834126014,27sleep018354-21fschecks_count07:23:200
132402606,20sleep10-21swapper/110:31:101
1310126018,18sleep10-21swapper/108:48:091
2791725913,27sleep027945-21irqstats11:17:580
1831025917,25sleep118307-21apt-get10:47:501
1831025917,25sleep118307-21apt-get10:47:501
1098225916,23sleep211001-21apt-get12:07:512
482725814,26sleep04974-21head10:03:310
3218325818,19sleep20-21swapper/208:07:402
2952825811,18sleep20-21swapper/207:57:592
2820325822,22sleep20-21swapper/207:53:062
2767225819,23sleep127665-21apt-get11:17:501
2480625820,18sleep00-21swapper/011:08:000
2480625820,18sleep00-21swapper/011:08:000
162302588,14sleep00-21swapper/008:58:100
632325715,25sleep06317-21apt-get11:52:510
2831725716,25sleep00-21swapper/007:53:080
2619425715,27sleep026281-21sh09:32:380
2088825715,26sleep120994-21timerwakeupswit09:13:091
2032725713,26sleep00-21swapper/010:53:060
1721925716,23sleep017225-21ntp_states10:43:050
1367825713,26sleep013702-21munin-run08:52:360
284162565,13sleep20-21swapper/211:18:082
2299225615,25sleep022985-21apt-get11:02:510
225042565,41sleep10-21swapper/111:00:021
1165725612,27sleep111798-21cat10:27:391
916525518,23sleep09200-21awk12:02:360
916525518,23sleep09200-21awk12:02:360
723725515,24sleep07246-21diskstats10:12:560
465225513,19sleep20-21swapper/208:22:542
2667525516,25sleep226739-21/usr/sbin/munin07:48:052
211052555,10sleep20-21swapper/207:30:572
198872558,36sleep00-21swapper/009:12:110
1372825515,18sleep20-21swapper/208:52:382
3222925414,20sleep20-21swapper/209:48:072
3222925414,20sleep20-21swapper/209:48:072
2426225413,21sleep00-21swapper/009:23:130
1993125415,23sleep119937-21df_inode10:52:551
7612538,19sleep20-21swapper/209:52:552
628525316,21sleep16289-21diskstats08:27:581
374625314,18sleep10-21swapper/110:02:391
1056625314,25sleep210616-21sed10:23:002
546625215,20sleep25354-21latency_hist10:07:412
3072025216,18sleep10-21swapper/109:43:091
3072025216,18sleep10-21swapper/109:43:091
3004725215,23sleep230099-21cpuspeed09:42:542
3004725215,23sleep230099-21cpuspeed09:42:542
2771425221,16sleep10-21swapper/107:52:491
299125118,16sleep20-21swapper/211:42:392
2629825116,17sleep00-21swapper/011:12:570
2629825116,17sleep00-21swapper/011:12:570
2464625114,21sleep224639-21apt-get07:42:522
1866525112,23sleep218655-21apt-get09:07:532
159525114,21sleep11602-21diskstats08:12:561
1256125114,22sleep212555-21apt-get12:12:522
1179925112,24sleep211982-21apt-config10:27:402
2664824911,16sleep10-21swapper/109:32:471
2332024914,17sleep10-21swapper/109:22:501
1716824810,21sleep017179-21df_inode09:02:540
1716824810,21sleep017179-21df_inode09:02:540
1100724812,18sleep10-21swapper/112:07:541
2616424711,21sleep126171-21diskstats11:12:541
2616424711,21sleep126171-21diskstats11:12:541
673824514,16sleep00-21swapper/008:28:090
3242024510,17sleep20-21swapper/208:07:522
293182448,20sleep129385-21df07:57:541
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional