You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-26 - 16:51

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System (updated Thu Mar 26, 2026 12:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2296027785,739sleep218030-21kworker/2:011:31:332
2296027785,739sleep218030-21kworker/2:011:31:332
25315266723,625sleep325379-21/usr/sbin/munin08:12:583
19963245850,64sleep20-21swapper/209:37:592
532023927,373sleep25365-21cat12:17:462
19323238520,344sleep019460-21/usr/sbin/munin07:53:200
7956225931,191sleep28028-21mailstats07:18:002
6772223622,192sleep16811-21cut12:22:461
849221614,115sleep2420-21/usr/sbin/munin12:02:512
849221614,115sleep2420-21/usr/sbin/munin12:02:512
28669220712,177sleep328759-21grep08:23:163
32012219537,59sleep20-21swapper/210:17:292
21754219510,123sleep221903-21tail09:43:042
16504219017,157sleep316541-21latency_hist07:47:293
2447221888,90sleep20-21swapper/209:52:462
13667218011,151sleep013742-21find09:17:580
11844217730,110sleep211954-21latency_hist07:32:302
20115217110,147sleep220151-21cpu07:57:422
10326216436,27sleep20-21swapper/209:07:582
1196821627,140sleep212034-21mailstats09:12:592
2610921609,136sleep226186-21sed08:17:292
11225216025,124sleep211302-21grep10:52:562
11225216025,124sleep211302-21grep10:52:562
1038421609,106sleep210479-21timerandwakeup12:32:572
27747215834,68sleep227923-21apt-config08:22:302
24516215729,58sleep20-21swapper/208:12:292
6448215648,21sleep20-21swapper/210:37:572
1897214729,24sleep20-21swapper/208:42:422
1897214729,24sleep20-21swapper/208:42:422
12082214719,18sleep20-21swapper/207:32:422
10158214719,109sleep310192-21proc_pri09:07:553
28963214621,103sleep329092-21grep11:47:583
15355214668,39sleep20-21swapper/207:42:482
723221445,128sleep00-21swapper/010:42:190
30994214124,94sleep031123-21latency_hist08:32:300
30994214124,94sleep031123-21latency_hist08:32:300
27058214129,100sleep227102-21ntpq11:42:522
25163214117,104sleep325298-21grep09:53:153
22926214131,72sleep222961-21munin-run08:07:282
18762214011,113sleep00-21swapper/011:17:460
31977213911,89sleep232112-21head08:33:002
31977213911,89sleep232112-21head08:33:002
5002213428,46sleep20-21swapper/208:52:432
28763213418,94sleep028842-21xargs08:23:170
31103213213,98sleep00-21swapper/011:57:280
2968121297,59sleep20-21swapper/208:27:392
2968121297,59sleep20-21swapper/208:27:392
28202212929,88sleep228212-21find11:47:372
9778212814,88sleep30-21swapper/312:32:413
15882212821,91sleep116034-21awk07:42:591
3669212726,52sleep20-21swapper/207:07:302
29630212710,40sleep20-21swapper/210:07:562
29630212710,40sleep20-21swapper/210:07:562
11303212719,84sleep311441-21grep10:52:573
11303212719,84sleep311441-21grep10:52:573
714212518,90sleep3730-21kernelversion12:02:483
714212518,90sleep3730-21kernelversion12:02:483
24576212534,51sleep224651-21systemd-run11:37:272
20490212517,95sleep320532-21sed07:57:513
1853521249,79sleep218525-21apt-get07:52:412
23000212331,56sleep223042-21ntp_kernel_pll_09:47:532
1706212311,93sleep11716-21kworker/u8:012:07:271
1706212311,93sleep11716-21kworker/u8:012:07:271
8320212214,94sleep18362-21fschecks_count12:27:471
2011721180,9sleep30-21swapper/309:37:593
17175211815,65sleep10-21swapper/111:12:461
12166211716,19sleep30-21swapper/310:57:333
9496211613,83sleep20-21swapper/207:22:572
17465211620,74sleep317443-21latency_hist09:32:313
18143211218,78sleep318302-21apt-config07:52:303
13602211128,17sleep20-21swapper/211:02:292
7054211022,40sleep20-21swapper/212:22:512
4958211013,25sleep20-21swapper/210:32:562
29759211027,46sleep229771-21apt-get11:52:382
15804211017,71sleep00-21swapper/007:42:580
11881210812,82sleep111955-21systemctl10:57:271
11881210812,82sleep111955-21systemctl10:57:271
631821077,32sleep20-21swapper/207:12:552
3423210739,31sleep20-21swapper/208:47:392
3313210720,21sleep20-21swapper/210:27:552
24614210712,82sleep024672-21if_eth009:53:040
3228721067,14sleep10-21swapper/110:17:451
31811210521,39sleep20-21swapper/211:57:552
13745210519,65sleep113883-21grep09:17:591
10334210534,55sleep110353-21modprobe10:52:261
10334210534,55sleep110353-21modprobe10:52:261
7322210413,73sleep07469-21awk08:57:590
14691210322,35sleep20-21swapper/209:22:442
31435210222,63sleep131459-21fschecks_count11:57:461
25091210226,40sleep224593-21apt-get11:37:312
1711121025,62sleep20-21swapper/209:28:342
5815210110,74sleep05943-21sed10:37:310
4899210114,42sleep20-21swapper/212:17:302
29864210119,61sleep329901-21fschecks_count11:52:473
2258210122,64sleep32278-21cut12:07:493
1766621015,10sleep20-21swapper/207:48:052
9152210011,62sleep00-21swapper/010:47:450
9152210011,62sleep00-21swapper/010:47:450
21770210013,71sleep021766-21apt-get11:27:390
21770210013,71sleep021766-21apt-get11:27:390
19157210010,24sleep10-21swapper/111:17:551
2465529919,61sleep324741-21dpkg11:37:283
1774829922,37sleep20-21swapper/209:32:422
91652979,51sleep29227-21grep10:47:472
91652979,51sleep29227-21grep10:47:472
359529716,62sleep33660-21fw_packets08:47:483
286672975,10sleep20-21swapper/210:05:542
86662965,35sleep20-21swapper/210:45:532
86662965,35sleep20-21swapper/210:45:532
2969029616,61sleep329661-21/usr/sbin/munin08:27:443
2969029616,61sleep329661-21/usr/sbin/munin08:27:443
1676329611,64sleep016132-21/usr/sbin/munin09:28:000
706429514,59sleep16552-21/usr/sbin/munin08:57:551
411929527,48sleep14139-21munin-run10:32:271
207012957,71sleep220767-21wc11:22:572
207012957,71sleep220767-21wc11:22:572
199629521,13sleep20-21swapper/212:07:402
1843829537,37sleep00-21swapper/011:17:300
139642958,48sleep213983-21apt-get11:02:372
858529420,35sleep20-21swapper/209:02:532
2729629420,22sleep00-21swapper/011:42:580
1383929411,39sleep20-21swapper/207:37:472
1836329313,63sleep018491-21sed09:32:560
2650029211,62sleep10-21swapper/108:17:491
16722917,48sleep21716-21ntpq10:22:512
944428914,59sleep19473-21latency_hist09:07:291
530928814,33sleep20-21swapper/207:12:312
299762885,11sleep10-21swapper/110:07:591
299762885,11sleep10-21swapper/110:07:591
166128831,15sleep20-21swapper/208:42:302
130622888,37sleep20-21swapper/209:17:442
4102878,38sleep20-21swapper/210:17:582
2862228721,51sleep128628-21ntpq08:23:141
281362867,40sleep228125-21apt-get08:22:422
138728513,50sleep01390-21/usr/sbin/munin10:22:440
2798928415,16sleep30-21swapper/311:47:303
258072848,40sleep225942-21modprobe09:57:302
258072848,40sleep225942-21modprobe09:57:302
2485228415,53sleep024907-21fschecks_count08:12:470
2319228420,48sleep023211-21apt-get08:07:380
1646728422,22sleep10-21swapper/109:27:521
251328313,52sleep12629-21timerandwakeup08:42:571
251328313,52sleep12629-21timerandwakeup08:42:571
1855628218,32sleep30-21swapper/307:52:593
2072628118,39sleep120840-21timerandwakeup07:57:581
198428116,50sleep12125-21unixbench_multi10:22:581
1210728124,34sleep012232-21wc09:13:010
2813928021,39sleep028247-21timerandwakeup10:02:570
3134827920,36sleep331345-21apt-get11:57:393
2083327915,48sleep020935-21/usr/sbin/munin11:22:590
2083327915,48sleep020935-21/usr/sbin/munin11:22:590
523327824,38sleep15278-21munin-node07:12:301
523327824,38sleep15278-21munin-node07:12:301
2455527819,38sleep324704-21latency_hist08:12:303
2134327724,25sleep10-21swapper/109:42:551
52927615,45sleep3652-21/usr/sbin/munin10:17:593
2474327514,44sleep00-21swapper/011:37:290
2066627523,29sleep00-21swapper/009:42:300
587127421,31sleep10-21swapper/112:17:571
52212749,52sleep00-21swapper/012:17:440
3248827421,37sleep032528-21sh08:37:270
3248827421,37sleep032528-21sh08:37:270
200027416,43sleep12070-21df12:07:431
1062127317,31sleep00-21swapper/007:27:470
71827217,41sleep3729-21netstat08:37:503
550527215,37sleep05584-21awk08:52:580
2174727212,21sleep30-21swapper/311:27:383
2174727212,21sleep30-21swapper/311:27:383
1683627213,46sleep016906-21fschecks_time07:47:470
925927123,20sleep30-21swapper/307:22:493
626027122,31sleep16314-21modprobe08:57:271
31022717,53sleep00-21swapper/012:10:290
2807327115,41sleep328137-21/usr/sbin/munin10:02:563
2232427121,25sleep10-21swapper/111:27:571
2232427121,25sleep10-21swapper/111:27:571
183592717,52sleep218385-21munin-run11:17:282
936027014,41sleep19390-21/usr/sbin/munin07:22:511
92932707,52sleep20-21swapper/212:29:542
2638927016,29sleep10-21swapper/111:42:271
1964227019,39sleep20-21swapper/211:18:412
1113827025,25sleep30-21swapper/307:27:583
445726917,33sleep14383-21/usr/sbin/munin10:32:431
1794826913,43sleep317969-21irqstats09:32:473
659326823,24sleep30-21swapper/307:13:003
416926819,24sleep30-21swapper/310:32:293
183626815,23sleep30-21swapper/310:22:563
813026716,37sleep38164-21cpu09:02:423
389626711,38sleep20-21swapper/207:07:382
315826720,30sleep33175-21idleruntime-cro08:47:273
315826720,30sleep33175-21idleruntime-cro08:47:273
2648126722,22sleep30-21swapper/309:57:563
2648126722,22sleep30-21swapper/309:57:563
167042676,29sleep20-21swapper/211:11:142
798826621,19sleep30-21swapper/312:27:283
75182665,27sleep20-21swapper/208:58:212
482026619,25sleep30-21swapper/312:17:263
359926616,28sleep03628-21munin-run07:07:280
295392666,13sleep00-21swapper/010:07:540
295392666,13sleep00-21swapper/010:07:540
2632426625,21sleep30-21swapper/308:17:393
790926512,25sleep00-21swapper/007:17:580
666226511,37sleep36698-21/usr/sbin/munin08:57:453
633926516,19sleep30-21swapper/308:57:293
1946126516,20sleep10-21swapper/107:53:211
944726420,28sleep39480-21/usr/sbin/munin10:47:523
944726420,28sleep39480-21/usr/sbin/munin10:47:523
44392645,46sleep00-21swapper/012:12:590
3266726416,23sleep00-21swapper/012:02:290
2970726419,26sleep329820-21timerwakeupswit10:07:573
2970726419,26sleep329820-21timerwakeupswit10:07:573
44526315,31sleep0439-21apt-get12:02:410
44526315,31sleep0439-21apt-get12:02:410
44502634,47sleep00-21swapper/008:49:020
2953626312,36sleep129594-21cat11:52:281
2606226310,25sleep10-21swapper/109:57:451
2606226310,25sleep10-21swapper/109:57:451
1567826312,38sleep113441-21snmpd09:25:021
1525126314,35sleep315253-21cron11:07:273
1496526316,25sleep00-21swapper/007:42:290
1365826316,30sleep10-21swapper/107:37:421
896426213,21sleep00-21swapper/007:22:380
874426219,31sleep38834-21mailstats12:27:573
728426223,20sleep10-21swapper/110:42:291
2515126217,26sleep30-21swapper/311:37:563
238212626,44sleep00-21swapper/009:51:470
1351426214,29sleep113542-21munin-run11:02:281
1203526219,27sleep312102-21cat09:13:003
1109026217,19sleep00-21swapper/010:52:540
1109026217,19sleep00-21swapper/010:52:540
821026117,22sleep30-21swapper/310:42:593
821026117,22sleep30-21swapper/310:42:593
3270326113,20sleep00-21swapper/010:17:560
2420626017,18sleep10-21swapper/109:52:321
2392726013,34sleep124018-21timerandwakeup11:32:581
2311926014,21sleep30-21swapper/309:47:583
2123426013,30sleep321292-21sort09:42:533
2103226021,20sleep00-21swapper/009:42:380
1800526016,27sleep118048-21gpgv07:52:281
1346626021,23sleep013512-21apt-config11:02:270
1108026018,23sleep10-21swapper/109:12:301
599025918,24sleep36057-21df10:37:463
51132597,11sleep00-21swapper/007:11:360
51132597,11sleep00-21swapper/007:11:360
440625916,25sleep34401-21apt-get10:32:413
2211925919,24sleep322121-21ntp_states08:02:523
1500825916,21sleep00-21swapper/009:22:510
1217125915,22sleep30-21swapper/307:32:473
1213325914,20sleep00-21swapper/007:32:440
1100825911,11sleep10-21swapper/107:27:561
93712584,42sleep00-21swapper/009:07:150
518225822,21sleep35210-21basename07:12:273
518225822,21sleep35210-21basename07:12:273
3111125827,16sleep10-21swapper/110:12:561
2681325814,18sleep00-21swapper/008:17:560
226662587,17sleep10-21swapper/108:04:071
1779425815,25sleep117882-21fschecks_time09:32:451
1437025820,19sleep30-21swapper/309:22:293
1194525816,22sleep10-21swapper/109:12:581
884025718,20sleep18948-21latency_hist10:47:291
884025718,20sleep18948-21latency_hist10:47:291
878225716,26sleep18924-21cat07:22:311
802525717,22sleep00-21swapper/012:27:290
3026425721,18sleep10-21swapper/111:52:571
282025716,23sleep32908-21mailstats07:02:553
2080025716,23sleep120975-21gpgv09:42:311
1875725717,20sleep30-21swapper/311:17:453
823425618,19sleep00-21swapper/012:27:410
789925620,19sleep00-21swapper/010:42:520
789925620,19sleep00-21swapper/010:42:520
559325616,23sleep05628-21ntp_states07:12:330
2729425614,27sleep027322-21latency_hist10:02:290
2729425614,27sleep027322-21latency_hist10:02:290
2234925617,19sleep00-21swapper/008:02:570
1346125616,18sleep10-21swapper/107:37:301
594125515,22sleep15948-21diskstats07:12:461
3123825514,25sleep331305-21df08:32:453
3123825514,25sleep331305-21df08:32:453
1554425515,24sleep315535-21apt-get11:07:433
1456625511,27sleep014648-21mailstats11:04:010
1396325516,23sleep113481-21apt-get11:02:361
397325412,25sleep14005-21ntp_states12:12:521
285362545,34sleep10-21swapper/110:04:121
268012548,17sleep10-21swapper/111:42:471
2550725413,20sleep10-21swapper/108:13:001
2515325415,23sleep025122-21apt-get11:37:570
2142925419,18sleep00-21swapper/008:02:290
1373025414,24sleep313739-21diskstats07:37:443
3189525312,23sleep031967-21systemd-run10:17:270
1634625314,21sleep016444-21users11:08:000
1103725316,18sleep00-21swapper/012:37:270
83825214,16sleep00-21swapper/008:37:550
285125214,23sleep02846-21apt-get10:27:410
1099325212,24sleep011079-21apt-config09:12:280
977625114,22sleep19771-21apt-get12:32:391
568225113,19sleep30-21swapper/312:17:553
3126725115,15sleep30-21swapper/310:12:583
260425111,26sleep12309-21/usr/sbin/munin07:02:491
2125825118,14sleep10-21swapper/111:24:331
2125825118,14sleep10-21swapper/111:24:331
197025111,15sleep00-21swapper/008:42:440
197025111,15sleep00-21swapper/008:42:440
2822625011,22sleep028222-21apt-get11:47:390
2359725016,15sleep10-21swapper/108:07:511
1591725016,18sleep30-21swapper/309:27:303
789724913,16sleep30-21swapper/307:17:573
344824911,22sleep13442-21apt-get08:47:421
3185024915,15sleep10-21swapper/108:32:591
3185024915,15sleep10-21swapper/108:32:591
2067724912,21sleep320683-21ntp_states11:22:543
2067724912,21sleep320683-21ntp_states11:22:543
1604524914,17sleep10-21swapper/111:07:571
90624812,16sleep10-21swapper/108:37:561
246732478,14sleep10-21swapper/109:53:051
306072417,24sleep10-21swapper/108:28:071
306072417,24sleep10-21swapper/108:28:071
179392407,16sleep30-21swapper/311:13:003
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional