You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-09 - 22:04

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Note that this system runs a non-RT kernel.
Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackbslot7s.osadl.org (updated Mon Mar 09, 2026 12:43:49)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
224162234511,2321sleep221983-21kworker/2:207:13:332
17296213365,1276sleep04927-1kworker/0:1H08:38:510
61322113011,1103sleep215823-21kworker/u8:308:03:402
23266287418,840sleep223297-21/usr/sbin/munin12:23:512
15835277518,741sleep115866-21/usr/sbin/munin08:33:511
6983259014,561sleep37017-21/usr/sbin/munin09:48:513
19519246820,431sleep119590-21cat12:13:281
25293241216,376sleep125268-21latency_hist10:48:281
15551240645,314sleep015592-21fschecks_count08:33:460
23578237224,325sleep123716-21grep12:23:581
16047231021,274sleep316049-21sshd11:59:213
256192251105,55sleep0222-21systemd-journal07:23:280
475224934,200sleep0497-21fschecks_count09:28:450
1487122368,215sleep014903-21cat11:58:290
7395223436,154sleep07398-21cron08:08:260
28414223312,204sleep028547-21latency_hist10:58:290
25628223026,192sleep025719-21cut10:48:480
23245222851,120sleep00-21swapper/007:13:550
13434222021,181sleep313517-21/usr/sbin/munin10:08:573
13522221118,178sleep012818-21/usr/sbin/munin10:08:580
457621947,172sleep14656-21find11:23:581
29068217711,18sleep00-21swapper/009:13:590
84921675,124sleep00-21swapper/011:13:430
84921675,124sleep00-21swapper/011:13:430
20960216214,133sleep121030-21switchtime08:48:571
27880216040,101sleep00-21swapper/009:13:270
4103215897,43sleep00-21swapper/007:58:290
24352215749,70sleep04927-1kworker/0:1H10:43:510
11377215315,23sleep00-21swapper/010:03:460
956621526,134sleep00-21swapper/011:39:030
956621526,134sleep00-21swapper/011:39:030
27287215222,114sleep10-21swapper/110:53:491
2411221528,131sleep024186-21find08:58:580
5459215034,101sleep05043-21/usr/sbin/munin09:43:510
16158214813,76sleep00-21swapper/010:18:470
25746214730,102sleep025883-21grep09:03:580
25746214730,102sleep025883-21grep09:03:580
23510214515,119sleep00-21swapper/010:41:000
23510214515,119sleep00-21swapper/010:41:000
1109121448,19sleep00-21swapper/008:18:480
2964421429,22sleep00-21swapper/007:33:580
3103621418,121sleep031037-21ntp_states07:38:540
929221395,17sleep00-21swapper/009:56:470
4777213718,21sleep012-21rcu_sched09:43:270
30292213717,98sleep330334-21fschecks_count11:03:453
3019921368,112sleep00-21swapper/011:03:380
7022213539,61sleep00-21swapper/009:48:520
16435213311,106sleep016581-21latency_hist12:03:300
5775213213,22sleep00-21swapper/008:03:300
2073521326,89sleep00-21swapper/012:14:400
12408213220,17sleep00-21swapper/011:49:120
932521318,100sleep09415-21fschecks_time08:13:470
2570221315,115sleep00-21swapper/012:33:260
11510213129,16sleep00-21swapper/011:48:280
8332212911,31sleep08390-21sendmail-msp11:38:260
8332212911,31sleep08390-21sendmail-msp11:38:260
32604212925,19sleep00-21swapper/007:43:550
2959921295,17sleep00-21swapper/010:59:260
23341212910,97sleep00-21swapper/012:23:530
20828212927,19sleep020875-21grep07:08:420
27250212713,94sleep00-21swapper/012:38:270
27250212713,94sleep00-21swapper/012:38:270
2554212627,76sleep00-21swapper/009:33:580
12035212519,88sleep112178-21head10:03:591
31055212410,75sleep00-21swapper/009:23:280
21906212417,92sleep121967-21switchtime12:18:571
27309212324,76sleep127475-21unixbench_multi09:09:001
19843212320,84sleep219931-21tune2fs12:13:452
437821198,73sleep04422-21ntpq11:23:530
19279211920,75sleep119281-21/usr/sbin/munin10:28:371
19880211824,77sleep320027-21sed07:03:573
10479211716,77sleep110629-21grep09:58:581
18613211621,81sleep00-21swapper/012:08:510
12844211518,30sleep10-21swapper/110:08:431
1000821158,16sleep00-21swapper/009:58:490
18915211419,76sleep119030-21latency_hist10:28:291
4323211131,39sleep30-21swapper/307:58:403
243421118,23sleep02424-21apt-get11:18:440
243421118,23sleep02424-21apt-get11:18:440
22699211122,72sleep222716-21cut10:38:482
22699211122,72sleep222716-21cut10:38:482
1792221107,50sleep00-21swapper/012:08:260
8944210927,65sleep08977-21modprobe08:13:270
24062210819,62sleep20-21swapper/210:43:452
25995210721,66sleep125887-21/usr/sbin/munin07:23:451
28933210614,72sleep329066-21/usr/sbin/munin09:13:583
15463210622,29sleep00-21swapper/010:15:570
2036221055,88sleep00-21swapper/010:30:470
28713210422,64sleep228734-21awk07:33:272
1982121035,87sleep00-21swapper/008:45:010
27163210215,69sleep327188-21fschecks_count10:53:473
2225221028,60sleep022265-21awk12:19:000
22847210174,15sleep00-21swapper/008:54:010
22847210174,15sleep00-21swapper/008:54:010
14420210126,19sleep014510-21grep08:28:540
32737210023,52sleep20-21swapper/207:43:572
2519210018,65sleep12539-21fschecks_count11:18:491
2519210018,65sleep12539-21fschecks_count11:18:491
23508210014,68sleep123575-21df08:58:421
3037129915,69sleep130487-21sed09:18:571
283229824,55sleep32873-21ntpq11:18:553
283229824,55sleep32873-21ntpq11:18:553
1283929622,44sleep212836-21apt-get10:08:402
326122958,51sleep00-21swapper/009:28:290
911829410,71sleep19119-21sshd09:54:341
2047029414,63sleep320561-21fschecks_time08:48:473
2104629316,57sleep120840-21/usr/sbin/munin10:33:471
1428229313,60sleep314313-21/usr/sbin/munin08:28:483
1324229323,50sleep213269-21munin-plugin-st11:53:262
2948929210,69sleep229558-21mailstats07:33:562
269862925,11sleep00-21swapper/007:26:060
561129016,46sleep00-21swapper/011:28:450
4122905,11sleep00-21swapper/011:11:370
215022905,10sleep00-21swapper/008:51:390
187242906,31sleep00-21swapper/010:27:440
2286428813,64sleep322923-21fschecks_count07:13:463
213252887,44sleep00-21swapper/010:33:540
23652876,28sleep00-21swapper/007:52:030
1488228716,47sleep114363-21/usr/sbin/munin10:13:561
122928619,48sleep11273-21ntpq11:13:521
122928619,48sleep11273-21ntpq11:13:521
1090828623,48sleep110981-21grep08:18:451
2735628524,37sleep227509-21grep12:38:292
2735628524,37sleep227509-21grep12:38:292
1857828521,41sleep30-21swapper/312:08:503
134342855,27sleep00-21swapper/008:25:590
130428517,25sleep10-21swapper/107:48:451
2991728418,52sleep129929-21sh11:03:261
2704128422,41sleep327078-21ntp_states09:08:543
284132835,46sleep00-21swapper/007:29:340
1129828317,42sleep211361-21find08:18:542
309182829,12sleep30-21swapper/309:22:143
744728119,37sleep37592-21latency_hist08:08:293
3048828120,40sleep230625-21head09:18:582
2318028111,53sleep123225-21/usr/sbin/munin07:13:521
2221028118,47sleep322336-21sort07:13:303
1145428118,48sleep311508-21apt-config11:48:273
2036728022,33sleep10-21swapper/112:13:571
2872627918,16sleep10-21swapper/109:13:561
103242797,11sleep10-21swapper/108:16:111
2113527813,15sleep30-21swapper/310:33:493
2473927714,28sleep30-21swapper/312:28:493
1692427717,42sleep317131-21latency_hist08:38:303
158827721,27sleep30-21swapper/307:48:513
1573627720,34sleep215842-21latency_hist10:18:292
3175627628,20sleep10-21swapper/109:23:541
1909427618,23sleep10-21swapper/108:43:491
1889027622,28sleep20-21swapper/212:08:572
816527512,48sleep38226-21/usr/sbin/munin08:08:583
3221227514,45sleep132218-21ntp_states11:08:561
22472757,10sleep20-21swapper/207:50:312
2086227510,44sleep20-21swapper/210:33:392
1606127516,39sleep316156-21perl10:18:463
1447327523,23sleep30-21swapper/310:13:463
1400727418,27sleep20-21swapper/211:53:552
87927338,20sleep20-21swapper/209:28:552
2310627311,35sleep30-21swapper/312:23:483
2252327321,23sleep30-21swapper/310:38:453
2252327321,23sleep30-21swapper/310:38:453
990627227,21sleep30-21swapper/311:43:263
990627227,21sleep30-21swapper/311:43:263
328027221,29sleep30-21swapper/309:38:293
2715227227,35sleep20-21swapper/207:28:142
2012527216,38sleep320169-21ntpq12:13:503
1682627221,31sleep216920-21apt-config08:38:292
2610727017,26sleep30-21swapper/307:23:473
2572227015,39sleep125780-21if_eth010:48:491
254682708,51sleep10-21swapper/112:30:241
2406927014,40sleep324080-21awk10:43:463
1798127020,20sleep20-21swapper/210:23:542
1169127023,28sleep211722-21ntp_kernel_pll_10:03:522
401826912,36sleep14100-21/usr/sbin/munin09:39:001
334626917,32sleep33439-21/usr/sbin/munin07:53:563
2752426917,38sleep227533-21uname10:53:542
2618226912,42sleep326240-21if_eth012:33:493
2618226912,42sleep326240-21if_eth012:33:493
236522695,52sleep10-21swapper/110:42:511
236522695,52sleep10-21swapper/110:42:511
213002695,10sleep20-21swapper/208:49:022
1557126911,25sleep20-21swapper/211:58:552
113182697,10sleep20-21swapper/211:46:402
323492689,11sleep30-21swapper/309:25:413
2865626822,29sleep30-21swapper/310:58:443
2709526815,40sleep227155-21processes09:08:572
2553226816,27sleep10-21swapper/109:03:551
2553226816,27sleep10-21swapper/109:03:551
242002689,11sleep30-21swapper/307:18:303
1517426820,30sleep115176-21cron08:33:271
936126722,24sleep10-21swapper/111:39:001
936126722,24sleep10-21swapper/111:39:001
552926711,29sleep30-21swapper/309:43:543
3075726720,18sleep20-21swapper/207:38:482
212726716,36sleep21921-21/usr/sbin/munin09:33:482
1932526729,23sleep00-21swapper/007:03:460
1519426714,38sleep115199-21entropy11:58:461
1237826717,21sleep30-21swapper/311:49:113
561526613,36sleep15709-21fschecks_time11:28:461
2880226618,21sleep20-21swapper/209:13:572
2462126622,23sleep10-21swapper/107:18:491
1960826629,20sleep10-21swapper/107:03:511
974026517,19sleep20-21swapper/209:58:432
969026517,33sleep39697-21ntp_states08:13:543
96412659,42sleep20-21swapper/208:13:532
410026514,20sleep30-21swapper/311:23:483
2471226518,24sleep20-21swapper/212:28:482
1260126513,35sleep312693-21latency_hist10:08:293
2332726418,24sleep10-21swapper/108:58:301
192026419,28sleep11939-21apt-get09:33:391
1797626416,25sleep30-21swapper/312:08:293
1730526420,20sleep20-21swapper/208:38:572
15402646,50sleep30-21swapper/309:31:593
879126315,24sleep20-21swapper/211:38:492
879126315,24sleep20-21swapper/211:38:492
841226322,23sleep30-21swapper/311:38:293
841226322,23sleep30-21swapper/311:38:293
395326323,25sleep20-21swapper/211:23:382
3245626317,25sleep20-21swapper/209:27:062
2678726315,34sleep326834-21systemd-run10:53:273
1919826321,22sleep30-21swapper/308:43:513
1027426318,28sleep110281-21/usr/sbin/munin11:43:441
219326222,20sleep20-21swapper/211:18:302
219326222,20sleep20-21swapper/211:18:302
2181426218,19sleep30-21swapper/312:18:553
2121726217,29sleep121255-21ntp_kernel_pll_07:08:511
1399726219,29sleep214027-21fschecks_count08:28:432
899426119,25sleep29084-21ls08:13:292
3033726122,23sleep330377-21sed07:38:293
1891426122,21sleep20-21swapper/208:43:452
93126017,19sleep30-21swapper/311:13:463
93126017,19sleep30-21swapper/311:13:463
65442608,17sleep20-21swapper/211:30:212
2239626012,20sleep20-21swapper/208:53:552
2239626012,20sleep20-21swapper/208:53:552
2213126016,29sleep222196-21munin-node07:13:292
1929926022,23sleep218795-21apt-get10:28:382
175862608,15sleep10-21swapper/112:04:071
280125912,23sleep20-21swapper/211:18:542
280125912,23sleep20-21swapper/211:18:542
2392225915,29sleep323961-21ntp_states08:58:523
2282925917,26sleep322843-21/usr/sbin/munin08:54:003
2282925917,26sleep322843-21/usr/sbin/munin08:54:003
1923825916,26sleep219235-21apt-get07:03:402
506725813,27sleep15062-21apt-get09:43:401
297225818,22sleep20-21swapper/207:53:462
1089125817,25sleep310899-21df_inode08:18:433
816325716,17sleep20-21swapper/209:53:422
552225714,25sleep25528-21ntp_states09:43:532
49925710,29sleep1601-21perl09:28:461
313225722,20sleep10-21swapper/107:53:491
3067025718,27sleep130756-21cut07:38:471
2936725725,18sleep10-21swapper/107:33:511
2599425713,26sleep125987-21apt-get12:33:431
2599425713,26sleep125987-21apt-get12:33:431
1716425714,26sleep217243-21awk12:03:562
1266925718,22sleep112736-21memory08:23:501
1258725716,18sleep10-21swapper/111:49:181
109312578,40sleep30-21swapper/310:02:313
683625620,18sleep30-21swapper/311:33:293
395225618,23sleep23976-21ntp_states09:38:572
2637325620,21sleep226382-21munin-run09:08:282
2637325620,21sleep226382-21munin-run09:08:282
247002567,37sleep20-21swapper/209:02:312
2180725615,24sleep221813-21ntp_states12:18:542
1605725617,20sleep20-21swapper/210:18:452
2873825516,22sleep328762-21munin-run07:33:283
2610725513,22sleep20-21swapper/210:48:582
2256925519,21sleep10-21swapper/108:53:581
1724825521,22sleep316745-21apt-get08:38:383
792225419,15sleep10-21swapper/109:53:291
766225414,19sleep10-21swapper/108:08:401
560925415,25sleep25673-21dpkg08:03:272
434725412,24sleep14342-21apt-get07:58:421
397725413,25sleep33985-21ntp_states09:38:583
3221425421,15sleep30-21swapper/307:43:463
3058225414,23sleep330578-21apt-get07:38:413
1363725415,20sleep113737-21perl11:53:461
1243525416,18sleep30-21swapper/308:23:443
691425313,20sleep10-21swapper/108:04:231
670125321,18sleep20-21swapper/209:48:462
1932925313,23sleep319300-21apt-get10:28:563
1751725313,24sleep317514-21apt-get10:23:393
1738825316,19sleep10-21swapper/108:39:001
7812524,32sleep10-21swapper/107:46:351
3173425213,23sleep231753-21apt-get11:08:402
2865525213,22sleep228647-21apt-get10:58:432
2746125217,22sleep127509-21awk07:28:421
753625117,16sleep20-21swapper/211:33:562
753625117,16sleep20-21swapper/211:33:562
2638325114,22sleep326449-21cat09:08:293
2638325114,22sleep326449-21cat09:08:293
1283625116,18sleep20-21swapper/208:23:552
815825012,22sleep38155-21apt-get09:53:393
1728025012,23sleep117331-21latency_hist10:23:291
3020524911,22sleep130202-21apt-get11:03:391
2497724917,18sleep225103-21cat07:18:582
1406824916,15sleep30-21swapper/311:53:563
752924812,21sleep17535-21ntp_states11:33:551
752924812,21sleep17535-21ntp_states11:33:551
3028724711,21sleep230291-21diskstats11:03:442
318092418,15sleep30-21swapper/311:08:453
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional