You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-02 - 21:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackbslot7s.osadl.org (updated Thu Jul 02, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
262322653419,6501sleep018477-21kworker/0:208:50:030
244782112013,1085sleep04757-21kworker/0:210:30:110
1408325458,521sleep114086-21apt-get11:39:381
6355241519,379sleep36412-21/usr/sbin/munin07:49:493
3092623566,330sleep1364-21snmpd12:30:471
15748232238,70sleep10-21swapper/108:19:431
23725232148,146sleep123752-21munin-plugin-st10:29:271
237822290120,50sleep10-21swapper/108:44:591
29045228987,153sleep129138-21latency_hist07:19:301
29045228987,153sleep129138-21latency_hist07:19:301
8546227666,102sleep10-21swapper/109:39:471
24462273114,107sleep12212-21/usr/sbin/munin09:19:481
22628227212,245sleep222672-21ntpq08:40:232
21004224924,204sleep221021-21fschecks_count10:19:472
10493224573,55sleep110532-21latency_hist11:29:281
8951223532,183sleep19008-21idleruntime-cro07:59:281
27103223437,149sleep127178-21apt-config10:39:291
16323221936,139sleep116385-21fschecks_time10:04:471
192222109,157sleep11947-21munin-plugin-st09:19:281
643122069,161sleep16477-21ntp_kernel_pll_11:14:511
643122069,161sleep16477-21ntp_kernel_pll_11:14:511
29281220516,111sleep10-21swapper/107:19:421
29281220516,111sleep10-21swapper/107:19:421
8921219914,170sleep28961-21ps09:39:562
2875021990,23sleep125525-21cyclictest12:24:521
32754219818,128sleep10-21swapper/107:29:521
31324219451,63sleep131296-21latency_hist09:09:301
27783219119,159sleep227844-21fschecks_time07:14:472
27783219119,159sleep227844-21fschecks_time07:14:472
30743218815,158sleep230895-21sed09:04:592
3106621865,20sleep10-21swapper/107:24:511
3106621865,20sleep10-21swapper/107:24:511
5948218454,109sleep10-21swapper/109:29:581
18244217716,147sleep318258-21awk10:09:543
1591821759,151sleep016007-21cat11:44:500
1738221747,31sleep10-21swapper/108:24:471
27600217312,145sleep227094-21apt-get10:39:382
1897921738,45sleep10-21swapper/111:54:401
1897921738,45sleep10-21swapper/111:54:401
2719217217,41sleep22811-21latency_hist11:04:302
16182216910,144sleep316253-21sensors11:44:573
27294215826,69sleep127376-21mailstats12:19:561
23724215217,120sleep023744-21latency_hist08:44:530
15735215230,68sleep10-21swapper/111:44:471
21912215133,47sleep10-21swapper/112:04:291
1973421479,122sleep019880-21sed11:55:040
1973421479,122sleep019880-21sed11:55:040
7772214546,42sleep10-21swapper/111:19:471
7772214546,42sleep10-21swapper/111:19:471
598214424,99sleep0740-21grep07:29:590
295521418,17sleep10-21swapper/111:04:411
9984214073,54sleep19985-21/usr/sbin/munin09:44:401
395221407,87sleep14011-21basename09:24:491
14361214016,104sleep214365-21cron09:59:272
5819213518,58sleep10-21swapper/111:14:301
1070221336,35sleep10-21swapper/111:29:331
16432213222,84sleep316582-21unixbench_multi08:20:013
5853213122,86sleep25999-21latency_hist11:14:312
4359213112,34sleep10-21swapper/107:44:301
4359213112,34sleep10-21swapper/107:44:301
20858213024,71sleep120892-21ps08:34:571
9500212911,65sleep10-21swapper/107:59:401
2321421285,30sleep10-21swapper/108:41:111
3249121265,31sleep10-21swapper/109:10:111
1823021255,28sleep10-21swapper/111:51:331
7146212349,51sleep10-21swapper/109:34:501
12180212355,49sleep10-21swapper/109:49:571
32105212141,25sleep10-21swapper/112:34:591
1507521219,75sleep115111-21ntpq09:59:511
18956212019,81sleep318476-21apt-get11:54:383
18956212019,81sleep318476-21apt-get11:54:383
486921198,51sleep14939-21latency07:44:501
25864211834,39sleep10-21swapper/112:14:561
26889211645,31sleep10-21swapper/108:54:441
24819211534,29sleep10-21swapper/110:30:201
1976211413,17sleep00-21swapper/007:34:560
27841211311,80sleep027927-21/usr/sbin/munin10:40:530
14748211343,49sleep10-21swapper/108:14:581
26010211249,42sleep10-21swapper/107:09:531
8100211112,84sleep08108-21awk11:19:540
8100211112,84sleep08108-21awk11:19:540
5519211115,83sleep35562-21basename09:29:483
11336210831,65sleep111412-21latency08:04:491
30815210716,77sleep230851-21expr10:49:482
3385210614,73sleep33416-21memory07:39:513
3385210614,73sleep33416-21memory07:39:513
2975621067,16sleep10-21swapper/110:44:581
23432210514,75sleep023572-21latency_hist12:09:290
17701210515,78sleep217788-21basename11:49:572
31277210410,47sleep10-21swapper/110:49:591
28910210426,15sleep10-21swapper/108:59:571
483210328,36sleep10-21swapper/110:55:001
24923210226,41sleep124953-21munin-run12:14:271
24923210226,41sleep124953-21munin-run12:14:271
2726921019,13sleep00-21swapper/012:19:550
16605210017,65sleep216211-21/usr/sbin/munin10:04:522
533429910,77sleep25376-21grep09:29:432
661429816,59sleep26679-21awk07:49:562
1944629822,48sleep319524-21find08:29:583
13642985,28sleep10-21swapper/110:59:321
2501629724,49sleep225033-21ls08:49:282
73172968,28sleep10-21swapper/107:53:221
199922965,12sleep10-21swapper/108:33:031
740229517,54sleep37345-21latency_hist11:19:293
740229517,54sleep37345-21latency_hist11:19:293
2440829523,50sleep324473-21fschecks_time10:30:103
3234029416,62sleep332401-21users12:35:013
302712949,70sleep00-21swapper/007:21:030
302712949,70sleep00-21swapper/007:21:030
2984529420,60sleep229961-21sort10:44:592
2800329318,52sleep327602-21/usr/sbin/munin10:40:563
147029211,24sleep20-21swapper/207:34:422
2907729112,64sleep029221-21awk12:25:000
180302918,18sleep10-21swapper/110:09:481
1282929126,46sleep10-21swapper/111:35:021
102592916,11sleep30-21swapper/311:27:043
3196129015,16sleep20-21swapper/212:34:562
778128933,30sleep07873-21fschecks_time07:54:460
221182897,17sleep10-21swapper/112:04:421
64212879,65sleep00-21swapper/011:14:500
64212879,65sleep00-21swapper/011:14:500
2146428732,40sleep10-21swapper/110:19:581
133552877,43sleep113502-21awk08:09:581
1155428712,55sleep211529-21/usr/sbin/munin09:49:422
1425128628,32sleep20-21swapper/208:14:472
276262858,18sleep127620-21apt-get10:39:411
1960128426,37sleep10-21swapper/110:14:491
274128327,19sleep11-21systemd07:39:291
2497628219,18sleep00-21swapper/012:14:300
2497628219,18sleep00-21swapper/012:14:300
2558528113,16sleep20-21swapper/212:14:482
1863428117,37sleep30-21swapper/311:54:303
2582428023,30sleep30-21swapper/308:49:573
2386528023,43sleep023899-21cut08:45:020
3058727925,26sleep30-21swapper/307:24:293
3058727925,26sleep30-21swapper/307:24:293
112942798,15sleep00-21swapper/008:04:480
3110827813,19sleep00-21swapper/010:49:540
2462327817,41sleep324683-21/usr/sbin/munin07:04:553
41227710,52sleep0482-21sed10:54:590
3156327725,27sleep20-21swapper/209:09:432
3048427714,45sleep330547-21/usr/sbin/munin09:04:563
2271927716,44sleep322757-21processes08:40:283
123327718,35sleep31360-21latency_hist07:34:303
453227617,29sleep00-21swapper/009:25:000
2015227512,40sleep20-21swapper/208:34:292
1387127513,20sleep30-21swapper/308:14:283
1160527518,43sleep311629-21df_inode09:49:433
2745027418,31sleep30-21swapper/312:19:583
1400727428,11sleep10-21swapper/109:55:021
421527322,37sleep34217-21cron11:09:273
33727314,15sleep20-21swapper/207:29:532
2639627328,34sleep10-21swapper/108:52:101
260627212,46sleep02635-21nfs4_client09:19:510
2053527213,44sleep220554-21apt-get11:59:382
1530927213,35sleep3364-21snmpd09:59:573
2815827113,42sleep228308-21cat12:24:302
2726227118,23sleep30-21swapper/310:39:313
2708227117,28sleep20-21swapper/207:14:312
2707327128,23sleep00-21swapper/010:39:280
1356827118,28sleep00-21swapper/009:54:570
354927022,23sleep20-21swapper/209:24:302
3274127015,41sleep2313-21ps10:54:572
2047127012,41sleep320478-21/usr/sbin/munin08:34:473
1737227015,24sleep00-21swapper/008:24:460
1015227020,25sleep20-21swapper/209:44:492
922326916,25sleep00-21swapper/007:59:310
1081026916,24sleep00-21swapper/011:29:440
2541126821,29sleep10-21swapper/107:09:291
2293326817,32sleep022953-21/usr/sbin/munin10:24:580
1091626819,26sleep30-21swapper/311:29:473
1027226819,23sleep00-21swapper/009:44:510
954426718,25sleep30-21swapper/308:00:073
79682675,51sleep00-21swapper/009:37:120
769126715,36sleep37692-21sshd07:54:413
2977226720,26sleep20-21swapper/209:04:302
266826716,26sleep00-21swapper/011:04:280
2129326712,40sleep221306-21vmstat08:35:022
1860526722,21sleep30-21swapper/308:29:303
641526617,18sleep00-21swapper/007:49:500
440226612,38sleep34525-21cat09:24:593
262122664,52sleep00-21swapper/012:16:000
2548026614,25sleep20-21swapper/207:09:302
2473826624,25sleep224744-21ntp_states10:30:172
2419726616,31sleep30-21swapper/308:45:093
1933326618,30sleep319404-21/usr/sbin/munin10:14:433
1115326629,20sleep20-21swapper/208:04:462
521526518,28sleep35360-21/usr/sbin/munin11:10:003
3147026524,22sleep330705-21/usr/sbin/munin10:50:013
2621426512,35sleep326245-21/usr/sbin/munin07:10:003
2257926515,32sleep222603-21/usr/sbin/munin12:04:552
2164026526,22sleep30-21swapper/310:20:003
1319126521,26sleep30-21swapper/309:54:483
2912426412,25sleep00-21swapper/009:00:000
2392526414,33sleep323937-21/usr/sbin/munin12:09:493
2392526414,33sleep323937-21/usr/sbin/munin12:09:493
2077826412,37sleep020830-21irqstats11:59:480
1005426414,21sleep20-21swapper/208:00:342
2405526318,27sleep224093-21/usr/sbin/munin12:09:522
2405526318,27sleep224093-21/usr/sbin/munin12:09:522
1952826314,29sleep019661-21sed08:29:590
1523226315,34sleep215308-21switchtime09:59:562
3092026218,19sleep30-21swapper/307:24:493
3092026218,19sleep30-21swapper/307:24:493
1008426212,36sleep0364-21snmpd11:24:590
43332619,28sleep20-21swapper/209:24:582
3041326112,33sleep230427-21awk12:29:552
2602226117,21sleep20-21swapper/210:34:492
2380326122,22sleep30-21swapper/310:29:293
1008126120,18sleep30-21swapper/309:44:463
964026021,24sleep29646-21ntp_states11:24:532
585426022,21sleep20-21swapper/207:49:272
47822606,13sleep30-21swapper/307:44:483
2996226013,29sleep329155-21/usr/sbin/munin10:45:003
2897026019,19sleep30-21swapper/312:24:593
2686426016,21sleep00-21swapper/008:54:430
2251126010,19sleep30-21swapper/312:04:523
2001326020,20sleep20-21swapper/210:14:582
1717126015,27sleep017190-21apt-get11:49:380
1540626023,19sleep30-21swapper/311:44:293
824425918,28sleep38336-21cat11:19:573
824425918,28sleep38336-21cat11:19:573
33932595,13sleep30-21swapper/309:23:053
3042825915,28sleep330486-21/usr/sbin/munin12:29:563
2871325913,18sleep30-21swapper/308:59:513
1904425916,28sleep019045-21cron10:14:270
16602597,11sleep20-21swapper/209:16:102
1658425914,26sleep016653-21/usr/sbin/munin08:20:020
1225825914,28sleep212262-21find11:34:382
74025816,23sleep00-21swapper/009:14:460
305125811,19sleep00-21swapper/007:39:440
305125811,19sleep00-21swapper/007:39:440
2614825816,24sleep00-21swapper/007:09:560
2612725818,25sleep226219-21users08:50:002
1986725819,23sleep019933-21grep10:14:560
172382586,11sleep00-21swapper/010:06:080
122642587,16sleep30-21swapper/311:34:393
734625715,19sleep30-21swapper/309:34:573
331825714,28sleep23384-21wc07:39:502
331825714,28sleep23384-21wc07:39:502
3203125716,19sleep30-21swapper/309:09:573
295625722,20sleep20-21swapper/211:04:422
2744825716,23sleep327566-21munin-node07:14:363
2744825716,23sleep327566-21munin-node07:14:363
214832576,10sleep30-21swapper/312:00:073
1805425717,19sleep20-21swapper/210:09:492
1545125724,16sleep00-21swapper/011:44:300
150472576,12sleep00-21swapper/009:59:500
754225617,19sleep20-21swapper/209:35:002
588025619,18sleep00-21swapper/009:29:570
3131625620,18sleep00-21swapper/007:24:580
3131625620,18sleep00-21swapper/007:24:580
2421625615,24sleep024227-21df_inode07:04:440
24182565,15sleep30-21swapper/311:01:163
1898525615,25sleep218977-21apt-get11:54:412
1898525615,25sleep218977-21apt-get11:54:412
116325618,24sleep31309-21latency_hist10:59:303
828225510,30sleep00-21swapper/009:39:300
468925518,16sleep00-21swapper/011:09:490
305052555,10sleep20-21swapper/207:24:042
305052555,10sleep20-21swapper/207:24:042
2957625514,20sleep00-21swapper/010:44:530
2835525516,24sleep228351-21apt-get12:24:422
2375625514,26sleep223788-21systemd-run10:29:282
894925416,16sleep20-21swapper/211:24:292
894925416,16sleep20-21swapper/211:24:292
845425415,22sleep38450-21apt-get09:39:413
605225414,25sleep26048-21apt-get11:14:392
605225414,25sleep26048-21apt-get11:14:392
1623425413,24sleep316230-21apt-get10:04:413
1416425414,22sleep314232-21df08:14:443
1266725415,25sleep012709-21ntp_kernel_pll_11:34:570
1210625420,19sleep00-21swapper/009:49:560
467725313,16sleep00-21swapper/007:44:450
3147325313,17sleep20-21swapper/207:25:012
3147325313,17sleep20-21swapper/207:25:012
2820625312,18sleep30-21swapper/308:59:313
2693725315,22sleep226943-21df_inode08:54:452
1535925319,17sleep20-21swapper/211:44:282
1467125316,24sleep014747-21systemctl08:14:570
3007325214,22sleep030082-21diskstats09:04:450
1261125213,23sleep012605-21apt-get08:09:400
2756825111,24sleep027618-21munin-node07:14:370
2756825111,24sleep027618-21munin-node07:14:370
24022515,35sleep30-21swapper/307:35:153
2039425112,24sleep020388-21apt-get08:34:440
1574925113,19sleep20-21swapper/208:19:442
1350325112,23sleep213515-21users08:09:592
1082325117,18sleep20-21swapper/211:29:462
317372509,22sleep00-21swapper/012:34:500
2995125012,21sleep029945-21apt-get12:29:410
2571625013,21sleep325789-21mailstats12:14:543
118682508,14sleep30-21swapper/308:04:593
86972497,31sleep20-21swapper/207:56:102
3156124913,20sleep031556-21apt-get09:09:410
73224811,17sleep30-21swapper/309:14:453
1729824615,18sleep217300-21cpu08:24:442
1581424514,16sleep20-21swapper/211:44:482
2091724411,18sleep00-21swapper/010:19:400
335424217,13sleep00-21swapper/011:04:520
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional