You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-06 - 11:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sun Sep 06, 2026 00:43:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
332621130,5sleep1460699cyclictest00:08:141
1470521050,2sleep30-21swapper/323:04:143
243942920,5sleep2460799cyclictest23:55:142
259692900,6sleep2460799cyclictest21:59:562
4442870,4sleep2446-21sh21:30:162
280702870,2sleep10-21swapper/123:59:241
4605998013,59cyclictest0-21swapper/022:55:300
86662760,3sleep38669-21aten_rbpower_en20:50:323
4092710,1sleep30-21swapper/322:45:583
460599692,52cyclictest0-21swapper/020:30:450
222712670,2sleep10-21swapper/121:16:431
460599663,49cyclictest0-21swapper/000:00:460
460599642,48cyclictest0-21swapper/021:25:380
460599626,33cyclictest16738-21ssh23:45:300
4605996247,3cyclictest0-21swapper/023:50:310
4605996243,6cyclictest6461-21unixbench_singl21:35:450
4605996235,16cyclictest0-21swapper/022:30:340
460599621,45cyclictest26916-21fschecks_time22:00:360
460599621,42cyclictest14311-21grep00:20:470
121922620,5sleep3460899cyclictest00:19:193
4605996136,6cyclictest7850irq/56-dwc_otg_20:20:160
4605996127,11cyclictest7907-21grep20:45:430
460599611,40cyclictest0-21swapper/020:35:450
460699608,4cyclictest0-21swapper/119:10:291
4605996016,41cyclictest28352-21timerandwakeup22:40:440
197662600,2sleep20-21swapper/200:28:032
4605995939,10cyclictest0-21swapper/021:23:450
4605995935,5cyclictest7850irq/56-dwc_otg_23:25:170
460599592,44cyclictest0-21swapper/022:52:480
4605995924,11cyclictest4876-21cut00:10:310
460599591,36cyclictest20989-21unixbench_multi23:10:460
4605995912,33cyclictest31740-21switchtime22:05:420
4605995912,33cyclictest31740-21switchtime22:05:420
4605995839,11cyclictest7750irq/56-dwc_otg21:15:420
4605995832,9cyclictest17299-21wget19:45:310
4605995830,15cyclictest14816-21grep21:05:410
460599582,42cyclictest15694-21fschecks_count19:40:330
460599582,40cyclictest0-21swapper/020:25:460
4605995739,8cyclictest16038-21grep22:25:430
4605995734,4cyclictest7750irq/56-dwc_otg23:05:350
460599573,33cyclictest0-21swapper/020:05:510
4605995726,11cyclictest0-21swapper/023:30:320
460599571,53cyclictest27830-21cron20:15:160
460599566,36cyclictest6171-21cron19:15:160
4605995632,4cyclictest60-21kswapd021:50:230
4605995628,12cyclictest0-21swapper/020:55:420
4605995625,12cyclictest23335-21grep22:35:330
4605995614,26cyclictest133742chrt23:40:430
460599561,38cyclictest9345-21tail19:20:460
460599561,33cyclictest0-21swapper/022:20:360
4605995530,12cyclictest10683-21grep21:40:460
4605995530,12cyclictest10683-21grep21:40:460
4605995524,10cyclictest10163-21grep19:25:310
460599552,40cyclictest0-21swapper/021:10:390
460599552,37cyclictest0-21swapper/000:15:380
460599551,44cyclictest0-21swapper/020:51:450
460599551,40cyclictest25507-21ntp_kernel_pll_23:55:390
460599545,37cyclictest8471-21fschecks_count23:35:330
4605995435,8cyclictest7850irq/56-dwc_otg_21:30:440
460599541,37cyclictest0-21swapper/019:30:380
37552540,2sleep23717-21ssh22:10:472
4605995333,9cyclictest0-21swapper/000:28:030
4605995330,8cyclictest14152-21latency19:35:370
460599532,43cyclictest0-21swapper/022:45:320
460599532,35cyclictest0-21swapper/020:40:430
460599531,16cyclictest813-21ssh23:25:440
4605995230,6cyclictest7750irq/56-dwc_otg00:35:470
4605995230,11cyclictest7850irq/56-dwc_otg_21:45:360
460599521,42cyclictest0-21swapper/000:05:220
4605995130,7cyclictest7950irq/56-dwc_otg_23:00:230
460599512,36cyclictest0-21swapper/022:15:420
460599511,37cyclictest0-21swapper/023:15:320
460599511,36cyclictest1180-21runrttasks21:55:300
4605995110,5cyclictest3730-21grep22:10:470
460699498,25cyclictest32171-21ssh23:25:231
460699491,9cyclictest7750irq/56-dwc_otg20:40:431
4605994932,7cyclictest0-21swapper/000:33:230
4605994926,9cyclictest0-21swapper/021:00:410
4605994921,11cyclictest30404-21latency20:20:360
460599481,37cyclictest0-21swapper/019:55:340
285342480,7sleep1460699cyclictest22:40:451
460699471,30cyclictest0-21swapper/123:45:261
4605994724,11cyclictest7090-21ntp_kernel_err19:15:390
4608994624,12cyclictest15567-21ssh23:05:243
41132462,36sleep00-21swapper/019:07:280
460699455,24cyclictest7750irq/56-dwc_otg22:50:161
4605994523,10cyclictest19059-21acpi19:50:280
301872450,5sleep130176-21ssh21:25:451
460699448,24cyclictest0-21swapper/121:35:231
460699447,24cyclictest0-21swapper/122:00:311
460699447,23cyclictest0-21swapper/120:45:221
460699447,22cyclictest0-21swapper/120:15:301
460699447,22cyclictest0-21swapper/119:20:311
460599441,29cyclictest22662-21/usr/sbin/munin20:00:300
263102440,4sleep115698-21diskmemload22:39:041
4607994316,15cyclictest30889-21ssh22:44:552
460699436,23cyclictest0-21swapper/119:55:301
460699431,4cyclictest18928-21ls19:50:161
4608994220,10cyclictest1180-21runrttasks22:37:393
460699426,21cyclictest0-21swapper/120:35:211
460699425,28cyclictest7750irq/56-dwc_otg21:55:011
460699424,25cyclictest7750irq/56-dwc_otg23:35:331
460699422,20cyclictest0-21swapper/123:23:461
326952420,6sleep3460899cyclictest20:25:453
460699418,17cyclictest0-21swapper/121:20:221
460699412,24cyclictest0-21swapper/122:50:351
216032410,2sleep20-21swapper/222:33:012
4608994018,10cyclictest0-21swapper/322:05:443
4608994018,10cyclictest0-21swapper/322:05:443
460699407,20cyclictest0-21swapper/100:20:311
460699405,23cyclictest7750irq/56-dwc_otg21:30:481
4608993919,4cyclictest687050wget20:45:293
4608993912,8cyclictest7950irq/56-dwc_otg_21:50:313
4607993917,11cyclictest1972350ssh23:10:262
460699397,15cyclictest0-21swapper/120:05:291
460699396,26cyclictest3951-21ssh23:30:211
460699391,27cyclictest7750irq/56-dwc_otg00:37:561
460699391,23cyclictest0-21swapper/120:25:311
460899385,5cyclictest7850irq/56-dwc_otg_21:30:483
4608993814,13cyclictest2280750ssh00:31:033
460799389,17cyclictest11779-1kworker/u9:121:14:422
460799384,7cyclictest7850irq/56-dwc_otg_23:00:312
460699387,21cyclictest0-21swapper/121:10:301
4606993814,8cyclictest7750irq/56-dwc_otg19:25:431
4606993814,4cyclictest12628-21fschecks_time23:40:331
460699381,15cyclictest21-21rcuc/122:55:301
460699381,12cyclictest7750irq/56-dwc_otg22:32:371
4608993717,9cyclictest381ktimersoftd/322:40:313
4608993717,5cyclictest8852-21sh22:56:153
4608993714,12cyclictest1504450ssh00:21:583
4608993713,7cyclictest0-21swapper/320:20:343
460799373,5cyclictest7950irq/56-dwc_otg_23:55:432
4607993715,5cyclictest1180-21runrttasks21:08:462
460799371,19cyclictest15520-21wget19:40:302
460699379,6cyclictest32020-21grep22:05:441
460699379,6cyclictest32020-21grep22:05:441
460699376,21cyclictest7750irq/56-dwc_otg21:40:431
460699376,21cyclictest7750irq/56-dwc_otg21:40:431
4606993715,7cyclictest221ktimersoftd/121:58:511
4608993617,4cyclictest26986-21if_eth022:00:373
4608993616,4cyclictest1107-21dhclient22:11:593
460899361,5cyclictest26989-21ssh23:57:253
460799363,6cyclictest7950irq/56-dwc_otg_00:35:322
4607993616,5cyclictest1180-21runrttasks00:33:312
460799361,3cyclictest7950irq/56-dwc_otg_19:20:382
460699368,19cyclictest7750irq/56-dwc_otg00:10:321
460699364,11cyclictest7750irq/56-dwc_otg23:10:461
460699363,10cyclictest14972-21head21:45:451
460699361,4cyclictest7750irq/56-dwc_otg22:25:281
460699361,25cyclictest7750irq/56-dwc_otg21:00:161
4606993611,4cyclictest0-21swapper/122:25:161
460899354,4cyclictest7750irq/56-dwc_otg23:35:163
460899352,5cyclictest7850irq/56-dwc_otg_22:20:343
4608993518,6cyclictest28691-21ls23:20:393
4608993516,7cyclictest21290-21sh23:11:103
460899351,19cyclictest20479-21aten_rbpower_en21:15:303
460799351,5cyclictest10461-21ssh22:20:172
460799351,4cyclictest12671-21ls23:40:342
4607993514,6cyclictest10866-21aten_rbpower_en22:20:312
460799351,3cyclictest0-21swapper/222:45:172
460699356,6cyclictest7750irq/56-dwc_otg20:30:451
460699353,5cyclictest30110-21unixbench_multi00:00:461
4606993516,4cyclictest7750irq/56-dwc_otg22:10:361
460699351,23cyclictest7750irq/56-dwc_otg22:20:171
175042350,2sleep20-21swapper/221:50:092
4608993426,5cyclictest29081-21wget21:25:313
4608993425,6cyclictest9772-21sh21:40:353
4608993425,6cyclictest9772-21sh21:40:353
4608993413,7cyclictest32721-21apt-get00:05:163
4607993423,5cyclictest6849-21ssh22:55:092
4607993421,7cyclictest7850irq/56-dwc_otg_22:03:282
460799341,8cyclictest7989-21cron19:20:162
460799341,3cyclictest0-21swapper/223:27:182
460699344,5cyclictest18933-21ssh00:26:311
460699344,20cyclictest7750irq/56-dwc_otg19:50:341
460699342,24cyclictest0-21swapper/120:50:391
4606993418,9cyclictest7750irq/56-dwc_otg23:15:231
460699341,14cyclictest7750irq/56-dwc_otg20:00:431
175052340,1sleep20-21swapper/223:06:472
460899335,4cyclictest7950irq/56-dwc_otg_19:30:443
460899333,11cyclictest7850irq/56-dwc_otg_20:00:293
460899332,15cyclictest0-21swapper/323:50:453
4608993316,4cyclictest4792-21/usr/sbin/munin00:10:313
4608993313,6cyclictest14226-21cat19:35:383
4608993311,5cyclictest381ktimersoftd/320:15:363
4608993311,4cyclictest25353-21sed20:05:423
460799338,9cyclictest0-21swapper/200:22:382
4607993318,5cyclictest25536-21ssh22:37:402
460799331,4cyclictest639-21/usr/sbin/munin00:05:432
4607993314,7cyclictest1180-21runrttasks21:17:432
460799331,14cyclictest28576-21ssh21:25:092
4607993310,6cyclictest1288-21diskstats20:30:332
460699338,12cyclictest14791-21awk21:05:401
460699331,3cyclictest7750irq/56-dwc_otg00:15:371
460699331,17cyclictest7750irq/56-dwc_otg23:50:511
460899327,4cyclictest7950irq/56-dwc_otg_20:35:163
460899326,4cyclictest7850irq/56-dwc_otg_00:05:333
460899324,4cyclictest7850irq/56-dwc_otg_22:30:453
460899324,4cyclictest7750irq/56-dwc_otg21:20:403
460899323,4cyclictest0-21swapper/300:37:583
4608993225,5cyclictest19209-21ssh23:48:173
460899322,10cyclictest0-21swapper/321:49:053
4608993218,5cyclictest19487-21sh00:27:333
4608993217,5cyclictest17341-21aten_rbpower_po19:45:313
460899321,4cyclictest7850irq/56-dwc_otg_21:10:443
460899321,25cyclictest0-21swapper/322:27:293
460799328,9cyclictest7850irq/56-dwc_otg_00:20:162
460799327,5cyclictest7950irq/56-dwc_otg_20:25:172
460799326,5cyclictest4985-21latency_hist21:35:182
460799326,3cyclictest7750irq/56-dwc_otg00:00:312
4607993218,12cyclictest7950irq/56-dwc_otg_23:20:442
4607993215,5cyclictest118950lldpd22:57:332
4607993210,5cyclictest1180-21runrttasks20:48:162
4607993210,5cyclictest0-21swapper/222:25:292
4607993210,11cyclictest7750irq/56-dwc_otg23:35:342
460699326,4cyclictest7750irq/56-dwc_otg19:35:341
4606993214,5cyclictest15698-21diskmemload00:31:131
460899318,4cyclictest7750irq/56-dwc_otg23:15:363
460899317,9cyclictest7950irq/56-dwc_otg_21:00:443
460899317,4cyclictest0-21swapper/323:37:463
460899317,12cyclictest0-21swapper/321:37:063
460899315,5cyclictest19072-21taskset19:50:313
460899314,4cyclictest0-21swapper/321:55:463
460899313,12cyclictest12285-21ssh23:40:283
460899312,5cyclictest7950irq/56-dwc_otg_19:15:303
460899311,5cyclictest3001-21ssh23:29:123
460799319,9cyclictest0-21swapper/223:45:312
460799316,5cyclictest7850irq/56-dwc_otg_19:25:452
460799312,3cyclictest0-21swapper/220:25:352
4607993117,4cyclictest13135-21/usr/sbin/munin21:00:412
460799311,5cyclictest31006-21diskstats22:05:322
460799311,5cyclictest31006-21diskstats22:05:322
460799311,5cyclictest11475-21grep20:55:432
4607993115,7cyclictest22076-21latency_hist21:55:172
4607993113,11cyclictest9800-21fschecks_time21:40:352
4607993113,11cyclictest9800-21fschecks_time21:40:352
460699316,13cyclictest13385-21grep21:00:441
4606993123,4cyclictest16751-21ls23:05:451
460699312,12cyclictest7750irq/56-dwc_otg19:44:561
460699311,7cyclictest11429-21taskset23:00:231
460699311,6cyclictest0-21swapper/120:10:451
50452300,1sleep30-21swapper/322:51:503
460899306,4cyclictest0-21swapper/322:15:433
4608993017,4cyclictest6177-21dhclient-script20:41:523
460899301,4cyclictest0-21swapper/319:20:323
4608993013,6cyclictest11151-21unixbench_multi19:25:453
460799309,6cyclictest9301-21ntpq20:50:412
460799307,4cyclictest7850irq/56-dwc_otg_00:10:362
460799305,5cyclictest7950irq/56-dwc_otg_19:50:312
460799303,3cyclictest7750irq/56-dwc_otg23:15:332
4607993013,4cyclictest17303-21aten_rbpower_en19:45:312
4607993012,4cyclictest13302-21taskset19:33:572
460699301,5cyclictest12524-21grep19:30:381
41062303,18sleep320188-21snmpd19:07:233
40002304,20sleep20-21swapper/219:06:012
4608992922,5cyclictest15450-21/usr/sbin/munin19:40:353
460899291,4cyclictest14125-21/usr/sbin/munin21:05:343
4608992913,5cyclictest28048-21latency_hist20:15:173
460799296,5cyclictest7950irq/56-dwc_otg_20:40:482
460799294,4cyclictest7850irq/56-dwc_otg_19:10:452
460799292,6cyclictest27946-21latency_hist20:15:172
460799291,5cyclictest7950irq/56-dwc_otg_20:05:292
460799291,3cyclictest0-21swapper/223:30:262
460799291,3cyclictest0-21swapper/221:40:172
4606992921,5cyclictest221ktimersoftd/119:15:281
460899281,3cyclictest7950irq/56-dwc_otg_19:55:303
4608992813,5cyclictest11671-21resolvconf20:56:303
460799289,10cyclictest14865-21taskset19:36:002
460799281,5cyclictest7850irq/56-dwc_otg_20:40:162
460699284,13cyclictest7950irq/56-dwc_otg_20:20:391
4608992720,5cyclictest5361-21open_inodes19:10:413
460799276,5cyclictest29686-21taskset20:20:162
460799276,4cyclictest7850irq/56-dwc_otg_19:55:272
33582275,9sleep10-21swapper/119:05:391
460899261,4cyclictest4637-21dpkg20:40:173
460799244,4cyclictest7950irq/56-dwc_otg_20:00:262
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional