You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-28 - 17:40

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackbslot7s.osadl.org (updated Tue Apr 28, 2026 00:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
20014287455,8727sleep218447-21kworker/2:120:58:132
2027280418,768sleep02067-21/usr/sbin/munin20:01:020
15878266814,640sleep315885-21/usr/sbin/munin20:45:503
2606258718,551sleep32694-21/usr/sbin/munin23:30:543
4366252912,501sleep24383-21awk20:10:332
12143239215,365sleep212188-21grep00:01:142
6004223719,202sleep36080-21latency_hist20:15:343
6004223719,202sleep36080-21latency_hist20:15:343
21948222016,185sleep322080-21grep19:21:043
919221570,26sleep20-21swapper/219:56:072
8500221010,187sleep27998-21apt-get22:05:472
6726220711,178sleep26808-21cut20:16:022
28501220320,27sleep20-21swapper/223:10:352
9987220114,123sleep210049-21modprobe23:55:322
16881219319,147sleep20-21swapper/200:15:352
16881219319,147sleep20-21swapper/200:15:352
6476219029,146sleep26484-21latency_hist22:00:342
23090219010,160sleep223093-21acpi19:25:432
368521887,170sleep213441-21snmpd23:34:292
8857217740,126sleep213441-21snmpd20:23:082
8857217740,126sleep213441-21snmpd20:23:082
18038217361,39sleep218182-21unixbench_singl00:16:202
18038217361,39sleep218182-21unixbench_singl00:16:202
2787921687,148sleep227935-21tune2fs19:40:532
2231721635,12sleep20-21swapper/221:05:562
29313216169,25sleep20-21swapper/223:11:032
14667216111,136sleep113441-21snmpd19:00:481
32225216030,18sleep20-21swapper/219:55:352
22634215657,29sleep222706-21fschecks_time22:50:532
22634215657,29sleep222706-21fschecks_time22:50:532
29272215016,69sleep20-21swapper/221:27:112
2266121509,123sleep222768-21cut19:25:352
2136214916,109sleep32213-21fschecks_time21:45:543
20499214811,60sleep20-21swapper/200:25:522
486214414,113sleep1562-21mailstats19:56:021
22134214016,108sleep322194-21fschecks_time21:05:533
19432213915,103sleep319496-21mailstats20:56:003
30006213522,94sleep330026-21latency_hist23:15:323
4060213328,89sleep30-21swapper/321:51:023
15205212541,71sleep215260-21latency_hist00:10:342
19500212411,91sleep00-21swapper/020:56:010
31163212317,77sleep20-21swapper/221:35:352
19418212124,48sleep10-21swapper/122:40:501
10763212016,25sleep20-21swapper/220:30:352
28366211917,16sleep10-21swapper/121:25:531
3586211872,35sleep23597-21grep21:50:492
8059211711,57sleep20-21swapper/222:05:352
25638211755,46sleep20-21swapper/223:00:462
25638211755,46sleep20-21swapper/223:00:462
2179521150,6sleep00-21swapper/000:30:340
23096211423,71sleep322613-21apt-get19:25:443
21233211442,59sleep221308-21cat22:45:562
15516211418,80sleep315522-21df_inode00:10:523
15516211418,80sleep315522-21df_inode00:10:523
15142211415,81sleep015202-21/usr/sbin/munin19:01:030
10286211414,78sleep20-21swapper/223:55:442
17917211314,83sleep118036-21timerandwakeup00:16:191
17917211314,83sleep118036-21timerandwakeup00:16:191
17513211311,29sleep217533-21apt19:10:332
1575211021,18sleep20-21swapper/223:26:012
14901211038,55sleep20-21swapper/220:41:052
22385210912,20sleep20-21swapper/200:31:012
21051210912,30sleep121137-21sed00:26:041
1092210936,59sleep21169-21grep21:41:042
8510210824,52sleep18493-21apt-get22:05:541
27867210810,16sleep00-21swapper/019:40:510
26332210840,56sleep226365-21grep19:35:532
9736210713,77sleep30-21swapper/320:26:003
8490210617,72sleep18537-21grep23:50:351
19470210535,52sleep20-21swapper/219:15:342
26851210310,77sleep126925-21find19:36:041
2070321037,12sleep20-21swapper/219:16:102
12786210316,66sleep312829-21fschecks_count20:35:533
22526210211,45sleep00-21swapper/000:31:030
24770210117,67sleep324873-21perl19:30:533
18341210019,63sleep018400-21/usr/sbin/munin22:36:010
13142210031,52sleep20-21swapper/220:36:002
265129959,19sleep20-21swapper/221:46:052
2367929917,67sleep323689-21forks21:10:533
474029861,16sleep20-21swapper/220:10:532
474029861,16sleep20-21swapper/220:10:532
1696629816,64sleep117099-21sed22:31:011
109342980,8sleep00-21swapper/022:11:050
2454729713,68sleep10-21swapper/122:56:001
2260429713,68sleep122757-21awk00:31:041
1109329712,22sleep20-21swapper/220:30:432
80262969,11sleep30-21swapper/320:20:553
80262969,11sleep30-21swapper/320:20:553
1909829634,46sleep20-21swapper/200:20:562
1909829634,46sleep20-21swapper/200:20:562
822329520,19sleep30-21swapper/322:05:363
3059229517,61sleep130624-21kernelversion23:15:551
1762729529,49sleep20-21swapper/222:35:342
2359329461,15sleep20-21swapper/221:10:482
1127129437,12sleep20-21swapper/222:14:552
1672329318,56sleep016792-21modprobe00:15:330
1672329318,56sleep016792-21modprobe00:15:330
1007429310,61sleep010021-40mandb23:55:340
801329220,21sleep10-21swapper/122:05:341
592429223,43sleep15935-21modprobe20:15:321
592429223,43sleep15935-21modprobe20:15:321
53942929,67sleep25473-21cat23:40:352
3008729214,64sleep130171-21latency_hist23:15:341
2464129229,12sleep20-21swapper/219:30:442
255229030,11sleep20-21swapper/220:02:092
1433829029,12sleep20-21swapper/222:24:122
3264528929,12sleep20-21swapper/221:40:242
2165428927,12sleep20-21swapper/221:04:222
2227228814,57sleep022317-21latency_hist22:50:340
3037628729,11sleep20-21swapper/219:47:442
2690128726,39sleep026929-21munin-run23:05:320
2690128726,39sleep026929-21munin-run23:05:320
1213328714,54sleep112097-21apt-get00:01:091
190912866,67sleep20-21swapper/222:40:072
1289528615,53sleep212922-21cut22:20:322
269628517,31sleep00-21swapper/023:30:560
235922857,66sleep00-21swapper/022:53:170
235922857,66sleep00-21swapper/022:53:170
201728515,57sleep32025-21ntpq20:01:013
1576728528,10sleep20-21swapper/220:45:412
1493428520,44sleep214955-21kernelversion19:00:572
3099228450,17sleep20-21swapper/223:16:042
240792848,16sleep20-21swapper/222:55:452
2327528417,55sleep020195-1kworker/0:0H19:26:540
1776828459,12sleep20-21swapper/220:50:582
625628317,50sleep06387-21cat23:41:050
56312828,16sleep20-21swapper/223:40:472
1455828223,38sleep314653-21/usr/sbin/munin00:06:403
727428110,59sleep27297-21tune2fs23:45:512
561828115,51sleep15626-21ntpq21:56:011
1976128024,29sleep00-21swapper/019:15:440
195452809,52sleep20-21swapper/222:40:552
1473728016,46sleep20-21swapper/222:25:492
833427918,45sleep325884-21systemd-journal23:49:113
366927925,30sleep10-21swapper/121:50:521
679927817,36sleep36887-21tune2fs22:00:513
624327816,47sleep00-21swapper/020:15:470
2839827718,34sleep10-21swapper/119:41:041
246042776,58sleep00-21swapper/021:12:240
226427726,26sleep30-21swapper/323:30:343
1577027716,25sleep00-21swapper/020:45:430
97952766,47sleep10-21swapper/120:26:021
617927613,44sleep36254-21wc23:41:043
3243927614,46sleep032458-21apt-get19:55:450
318732767,15sleep20-21swapper/223:20:482
2563927510,49sleep125155-21/usr/sbin/munin21:16:001
2168427420,30sleep30-21swapper/300:30:343
1840127418,29sleep10-21swapper/122:36:021
1747927417,41sleep017491-21fschecks_time00:16:050
1747927417,41sleep017491-21fschecks_time00:16:050
1400227423,24sleep00-21swapper/020:40:340
347027319,32sleep33086-21/usr/sbin/munin20:05:573
3125527319,33sleep130862-21/usr/sbin/munin19:50:571
287227321,26sleep10-21swapper/120:05:351
2170127318,24sleep00-21swapper/019:20:570
1077827320,32sleep310927-21grep22:11:053
2758927215,38sleep127626-21sed23:05:571
2645627218,32sleep00-21swapper/019:35:550
1880427213,43sleep318823-21apt-get00:20:453
1880427213,43sleep318823-21apt-get00:20:453
184032726,55sleep016897-21kworker/0:120:52:540
3032927114,24sleep10-21swapper/121:31:021
255682718,52sleep225596-21ntpq21:15:572
2180827122,31sleep121857-21grep21:05:341
16592715,17sleep00-21swapper/021:44:250
1473427115,25sleep00-21swapper/020:41:030
128732719,51sleep00-21swapper/022:20:140
1669827016,39sleep116713-21awk20:46:041
88612696,12sleep00-21swapper/020:23:100
88612696,12sleep00-21swapper/020:23:100
779226918,38sleep17823-21cpu20:20:491
779226918,38sleep17823-21cpu20:20:491
44522699,16sleep00-21swapper/023:35:560
2109526919,23sleep30-21swapper/319:20:353
1866426922,21sleep30-21swapper/319:11:033
2105026823,28sleep321138-21tune2fs22:45:533
180126820,22sleep10-21swapper/121:45:351
415726713,23sleep10-21swapper/123:35:491
3270726720,24sleep10-21swapper/121:40:351
230026717,28sleep00-21swapper/023:30:350
2231826715,26sleep10-21swapper/122:50:351
2231826715,26sleep10-21swapper/122:50:351
206092676,13sleep00-21swapper/021:00:530
1754726717,33sleep017600-21sendmail22:35:320
1654026714,33sleep116622-21/usr/sbin/munin19:05:561
1540926719,32sleep315565-21awk19:01:063
1482126711,40sleep114832-21fschecks_count22:25:521
767626616,26sleep00-21swapper/023:46:020
560726617,35sleep05617-21ntpq21:56:000
334726621,20sleep00-21swapper/021:50:350
3115426619,23sleep30-21swapper/319:50:553
3097226621,23sleep00-21swapper/019:50:520
3029126614,38sleep030326-21ps21:31:010
1375826620,21sleep30-21swapper/322:21:023
72626515,36sleep0796-21sed23:25:350
2982526515,26sleep30-21swapper/321:30:483
2804026519,20sleep30-21swapper/321:25:353
266552655,19sleep10-21swapper/121:20:361
2071726520,26sleep00-21swapper/022:45:340
2017626518,20sleep10-21swapper/119:15:571
955226411,39sleep013441-21snmpd22:07:510
443026420,22sleep10-21swapper/120:10:361
365926416,19sleep00-21swapper/021:50:510
271682647,41sleep20-21swapper/221:20:582
2672926412,25sleep00-21swapper/021:20:470
2409926419,17sleep10-21swapper/121:11:031
2408426421,26sleep324081-21apt-get22:55:463
1762226412,33sleep317681-21/usr/sbin/munin20:50:553
136912649,37sleep013754-21apt-key00:05:360
1281526423,25sleep00-21swapper/000:01:320
1168126413,26sleep00-21swapper/020:31:060
797426321,21sleep00-21swapper/022:05:330
563526318,28sleep15702-21df23:40:501
475326324,23sleep30-21swapper/323:36:033
3064826319,21sleep30-21swapper/319:50:353
3002126321,16sleep00-21swapper/019:46:040
199092637,12sleep10-21swapper/120:56:501
1725026323,24sleep30-21swapper/300:15:443
1725026323,24sleep30-21swapper/300:15:443
3081926213,26sleep00-21swapper/023:16:020
1665526213,31sleep316701-21/usr/sbin/munin19:05:583
1613226219,23sleep10-21swapper/100:11:061
1613226219,23sleep10-21swapper/100:11:061
1531226215,28sleep015409-21/usr/sbin/munin22:26:040
932926121,25sleep39420-21sed23:51:013
348126114,26sleep00-21swapper/020:05:580
284012614,12sleep30-21swapper/323:09:553
2291726124,17sleep30-21swapper/300:32:113
1649326118,26sleep316546-21/usr/sbin/munin22:30:513
1059626117,22sleep10-21swapper/122:11:031
2885326014,25sleep00-21swapper/021:26:040
2335626018,21sleep10-21swapper/121:10:351
2270726022,20sleep30-21swapper/322:50:543
2270726022,20sleep30-21swapper/322:50:543
1909726024,21sleep31-21systemd22:40:123
1835126017,19sleep00-21swapper/019:10:560
1398426014,32sleep113870-21/usr/sbin/munin00:05:531
1183026016,19sleep10-21swapper/100:00:361
2980525925,14sleep10-21swapper/119:46:011
2453925912,21sleep00-21swapper/022:55:590
1480825917,26sleep314814-21df_inode22:25:503
3247425824,20sleep10-21swapper/123:21:031
3208025815,22sleep30-21swapper/321:36:053
3186725814,29sleep00-21swapper/021:36:020
2876525815,28sleep328828-21awk21:26:033
2716825811,19sleep00-21swapper/023:05:440
258525822,18sleep10-21swapper/121:46:041
2466525816,25sleep024661-21apt-get19:30:460
2077325817,25sleep320855-21/usr/sbin/munin21:00:563
1320525816,17sleep10-21swapper/120:36:031
624425711,22sleep10-21swapper/120:15:491
230262575,10sleep00-21swapper/021:06:590
1783425716,26sleep117896-21/usr/sbin/munin20:51:011
755325616,17sleep30-21swapper/320:20:353
648625618,16sleep30-21swapper/322:00:353
2671525620,17sleep30-21swapper/319:36:023
2096225615,26sleep120957-21apt-get22:45:471
2910425515,19sleep00-21swapper/023:10:570
2312425516,23sleep123110-21apt-get19:25:501
1887625514,28sleep018904-21sh00:20:500
1887625514,28sleep018904-21sh00:20:500
1625425514,25sleep016249-21apt-get19:05:470
1422325512,18sleep30-21swapper/320:40:453
977625414,23sleep09794-21ntp_states20:26:010
48525414,22sleep3480-21apt-get21:40:493
2910225419,18sleep30-21swapper/319:45:343
2650125415,18sleep00-21swapper/021:20:350
1350025416,18sleep30-21swapper/300:05:353
1161125417,19sleep30-21swapper/320:31:053
484025313,23sleep04892-21cat20:10:550
484025313,23sleep04892-21cat20:10:550
3233925319,16sleep30-21swapper/323:21:013
257422535,11sleep30-21swapper/321:16:023
1797025314,19sleep10-21swapper/119:10:401
1192125314,21sleep312006-21memory22:15:573
1061225316,22sleep310679-21memory23:55:583
1031025313,24sleep010306-21apt-get23:55:470
2610225218,17sleep10-21swapper/123:01:011
2610225218,17sleep10-21swapper/123:01:011
2600825213,23sleep126130-21latency_hist19:35:351
258625214,22sleep12595-21diskstats23:30:521
1385525212,25sleep013159-21/usr/sbin/munin22:21:030
137725216,20sleep11414-21ntp_states23:25:561
872625114,21sleep18793-21df23:50:471
45525114,21sleep3461-21ntp_states19:56:003
2259225116,19sleep10-21swapper/119:25:341
1941525012,22sleep319412-21apt-get22:40:483
189052507,17sleep10-21swapper/100:20:511
189052507,17sleep10-21swapper/100:20:511
1170725012,16sleep10-21swapper/122:15:511
1139925021,15sleep10-21swapper/120:30:591
3030424913,21sleep330319-21apt-get23:15:453
2571724812,20sleep325724-21diskstats23:00:513
2571724812,20sleep325724-21diskstats23:00:513
1398724713,20sleep114085-21users22:21:041
152812466,31sleep10-21swapper/120:42:411
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional