You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-20 - 20:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Mon Jul 20, 2026 12:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
27588211080,5sleep30-21swapper/307:05:363
2853422860,2sleep128539-21unixbench_singl07:05:501
292039910419,67cyclictest457-21sh10:20:420
29203991005,69cyclictest0-21swapper/010:05:340
199562980,2sleep00-21swapper/011:24:490
63802970,2sleep10-21swapper/111:47:551
29203999459,21cyclictest0-21swapper/009:45:240
2920399931,55cyclictest7520-21kworker/u8:409:55:460
202252920,4sleep320221-21ssh11:25:203
257482870,2sleep30-21swapper/312:11:383
2920399864,55cyclictest8236-21ssh10:30:330
2920399864,55cyclictest8236-21ssh10:30:330
2920399804,46cyclictest0-21swapper/010:17:220
271542790,2sleep30-21swapper/311:33:223
2920799763,45cyclictest0-21swapper/309:55:473
2920599768,49cyclictest0-21swapper/110:17:201
2920399767,65cyclictest11596-21wget12:35:350
29206997514,47cyclictest7950irq/56-dwc_otg_10:20:522
2920599743,44cyclictest221ktimersoftd/110:20:401
2920399746,36cyclictest0-21swapper/009:30:310
2920399744,51cyclictest0-21swapper/007:50:340
227812740,3sleep10-21swapper/110:08:271
12112720,2sleep30-21swapper/310:20:533
2920599703,11cyclictest8225-21apt-get10:30:351
2920599703,11cyclictest8225-21apt-get10:30:351
29203997045,9cyclictest0-21swapper/009:11:480
2920399704,54cyclictest0-21swapper/012:20:340
2920399704,54cyclictest0-21swapper/012:20:340
2920399700,50cyclictest32641-21grep09:40:450
2920399700,50cyclictest32641-21grep09:40:450
29206996819,25cyclictest301ktimersoftd/209:55:462
29203996830,22cyclictest0-21swapper/012:14:070
29205996724,26cyclictest12575-21ssh09:55:491
2920399676,40cyclictest16339-21sort12:00:390
29203996636,19cyclictest0-21swapper/010:55:460
40432650,2sleep30-21swapper/307:30:343
2920799652,36cyclictest8050irq/56-dwc_otg_09:40:333
2920799652,36cyclictest8050irq/56-dwc_otg_09:40:333
2920699658,37cyclictest8237-21kthreadd10:30:332
2920699658,37cyclictest8237-21kthreadd10:30:332
2920399658,54cyclictest101ktimersoftd/011:30:450
2920399657,39cyclictest0-21swapper/008:05:350
29206996410,20cyclictest30434-21sh10:17:222
2920399641,42cyclictest0-21swapper/011:17:550
2920799633,40cyclictest7950irq/56-dwc_otg_09:25:323
2920799633,39cyclictest0-21swapper/310:17:203
2920699637,35cyclictest7950irq/56-dwc_otg_10:05:342
2920399638,50cyclictest0-21swapper/011:25:340
2920799623,12cyclictest20035-21apt-get10:05:323
2920399619,24cyclictest0-21swapper/008:15:340
29203996134,13cyclictest8050irq/56-dwc_otg_11:50:210
2920799609,30cyclictest7950irq/56-dwc_otg_10:30:333
2920799609,30cyclictest7950irq/56-dwc_otg_10:30:333
29203996021,24cyclictest3918-21idleruntime-cro09:00:200
29203995941,9cyclictest0-21swapper/012:30:340
29203995938,10cyclictest0-21swapper/008:40:460
29203995934,6cyclictest23597-21cron12:10:210
2920399592,42cyclictest0-21swapper/008:35:380
2920399592,36cyclictest15955-21aten_rbpower_cu09:20:350
2920399592,34cyclictest0-21swapper/010:10:340
2920399591,44cyclictest28903-21sort09:35:500
29203995910,28cyclictest0-21swapper/011:55:340
2920799581,4cyclictest15933-21ssh10:00:353
29203995847,4cyclictest0-21swapper/011:52:360
2920399584,51cyclictest101ktimersoftd/007:35:200
2920399584,51cyclictest101ktimersoftd/007:25:210
2920399584,33cyclictest8579-21ls07:40:480
29203995828,17cyclictest8050irq/56-dwc_otg_09:25:510
2920399582,37cyclictest17114-21unixbench_multi10:00:500
29203995813,41cyclictest101ktimersoftd/007:25:410
29203995810,44cyclictest13221-21aten_rbpower_po07:55:350
29203995737,12cyclictest0-21swapper/009:19:390
29203995733,14cyclictest24396-21latency_hist10:50:210
29203995712,40cyclictest101ktimersoftd/011:10:330
2920399562,28cyclictest32253-21unixbench-2d08:45:510
2920399561,34cyclictest0-21swapper/009:05:320
2920799558,32cyclictest0-21swapper/311:15:373
2920599557,33cyclictest0-21swapper/111:50:351
2920399554,29cyclictest28990-21/usr/sbin/munin11:35:410
2920399553,23cyclictest0-21swapper/012:15:330
29203995517,24cyclictest6948-21unixbench_singl07:35:480
2920399549,42cyclictest7735-21sh11:09:170
2920399542,9cyclictest0-21swapper/011:00:320
2920399542,9cyclictest0-21swapper/011:00:320
29203995425,12cyclictest0-21swapper/011:44:480
29203995421,15cyclictest0-21swapper/008:00:480
2920399541,40cyclictest25210-21ls10:50:420
2920599538,31cyclictest0-21swapper/111:15:371
29203995331,9cyclictest0-21swapper/008:30:490
29203995331,9cyclictest0-21swapper/008:30:490
29203995326,14cyclictest101ktimersoftd/007:45:500
29203995325,5cyclictest7950irq/56-dwc_otg_08:20:500
29203995311,6cyclictest101ktimersoftd/012:27:260
29207995220,18cyclictest381ktimersoftd/311:05:353
29203995238,5cyclictest7950irq/56-dwc_otg_09:55:210
29203995233,4cyclictest24591-21missed_timers08:25:430
29203995231,9cyclictest18918-21basename08:10:400
29203995223,5cyclictest17448-21timerwakeupswit10:40:500
2920399521,28cyclictest31314-21fschecks_time07:15:390
2920599517,32cyclictest0-21swapper/107:25:321
2920399512,6cyclictest12575-21fschecks_time10:35:400
2920799508,27cyclictest0-21swapper/309:20:353
29203995029,5cyclictest0-21swapper/009:04:240
2920599491,27cyclictest19752-21latency_hist12:05:211
2920399491,35cyclictest0-21swapper/010:26:020
213322490,6sleep32920799cyclictest11:25:453
2920599477,28cyclictest0-21swapper/107:15:351
2920599474,24cyclictest12508-21ls12:35:491
2920799461,31cyclictest28127-21ls09:35:373
29203994619,6cyclictest1618-21grep08:50:480
2920799457,23cyclictest11934-21ssh09:15:363
29205994517,6cyclictest23629-21idleruntime-cro12:10:211
29205994516,12cyclictest2070250wget11:25:351
2920399451,32cyclictest0-21swapper/007:12:540
288302453,13sleep00-21swapper/007:09:020
29205994414,22cyclictest4590-21sed12:25:501
2920599441,30cyclictest24956-21ssh11:30:411
122692441,36sleep312332-21ssh10:35:363
2920799437,23cyclictest0-21swapper/307:25:323
29207994315,6cyclictest7179-21latency_hist12:30:213
2920799428,19cyclictest0-21swapper/307:50:273
2920799426,22cyclictest7999-21ssh11:50:353
2920799425,25cyclictest8050irq/56-dwc_otg_09:45:393
2920799423,17cyclictest8050irq/56-dwc_otg_12:35:353
2920799422,5cyclictest8050irq/56-dwc_otg_09:00:333
2920699427,21cyclictest0-21swapper/212:00:332
2920599426,24cyclictest15857-21ssh10:00:291
29205994219,5cyclictest12298-21fschecks_time09:15:411
29205994215,12cyclictest14382-21rm11:57:511
29205994212,12cyclictest0-21swapper/109:20:391
24242420,2sleep20-21swapper/207:25:342
149402420,1sleep10-21swapper/110:38:271
315682411,5sleep20-21swapper/212:20:122
29205994118,11cyclictest221ktimersoftd/112:20:091
2920599411,26cyclictest595-21latency11:40:411
2920799401,5cyclictest8050irq/56-dwc_otg_12:00:333
29206994017,13cyclictest0-21swapper/209:32:392
2920599405,5cyclictest27689-21ssh12:15:201
2920599402,29cyclictest0-21swapper/110:51:141
29205994020,11cyclictest8050irq/56-dwc_otg_09:45:481
175252400,2sleep20-21swapper/210:01:232
315972390,9sleep28050irq/56-dwc_otg_09:40:212
2920799398,18cyclictest8050irq/56-dwc_otg_10:10:493
2920799395,21cyclictest8050irq/56-dwc_otg_12:06:243
2920799391,3cyclictest0-21swapper/308:10:333
2920799391,16cyclictest27988-21ssh12:15:323
29206993919,5cyclictest3915-21if_wlan009:45:412
2920599395,7cyclictest7950irq/56-dwc_otg_08:30:491
2920599395,7cyclictest7950irq/56-dwc_otg_08:30:491
29205993915,9cyclictest8050irq/56-dwc_otg_11:36:381
2920799388,8cyclictest0-21swapper/311:45:363
2920799387,21cyclictest0-21swapper/307:15:353
2920799384,24cyclictest2229-21ssh12:23:303
2920799384,24cyclictest2229-21ssh12:23:303
2920799382,8cyclictest9409-21/usr/sbin/munin07:45:473
29207993811,5cyclictest7850irq/56-dwc_otg10:45:503
2920699384,4cyclictest7950irq/56-dwc_otg_12:35:352
2920699381,29cyclictest21036-21ls09:25:502
2920599387,6cyclictest7950irq/56-dwc_otg_09:25:431
29205993820,5cyclictest28708-21/usr/sbin/munin10:55:501
2920599381,9cyclictest28667-21ntp_kernel_pll_09:35:461
2920799374,18cyclictest8050irq/56-dwc_otg_12:35:223
29206993725,8cyclictest3181-21cron12:25:212
29206993725,8cyclictest3181-21cron12:25:212
29206993720,5cyclictest11566-21idleruntime-cro09:15:212
2920699371,7cyclictest2401850/usr/sbin/munin12:10:472
29206993713,15cyclictest0-21swapper/211:35:262
29206993711,13cyclictest1493-21grep07:20:492
2920599378,18cyclictest0-21swapper/107:50:271
2920599376,5cyclictest7850irq/56-dwc_otg10:10:391
2920599373,7cyclictest7955-21aten_rbpower_en09:10:351
29205993716,11cyclictest24852-21grep09:30:481
2920599371,29cyclictest29149-21munin-run08:40:201
2920599371,26cyclictest14128-21ls07:55:491
29205993711,15cyclictest0-21swapper/111:14:421
2920799366,20cyclictest0-21swapper/311:55:373
2920799361,6cyclictest11640-21latency_hist09:15:213
2920699369,4cyclictest0-21swapper/211:25:412
29206993616,9cyclictest8050irq/56-dwc_otg_12:30:382
29206993613,14cyclictest0-21swapper/209:42:152
29206993613,14cyclictest0-21swapper/209:42:152
2920599367,4cyclictest7950irq/56-dwc_otg_07:45:491
2920599366,23cyclictest8050irq/56-dwc_otg_08:25:431
2920599364,5cyclictest0-21swapper/111:10:211
2920599363,6cyclictest7950irq/56-dwc_otg_10:25:511
2920599363,6cyclictest0-21swapper/109:00:321
2920599362,8cyclictest0-21swapper/112:22:291
2920599362,8cyclictest0-21swapper/112:22:291
29205993627,6cyclictest221ktimersoftd/108:20:331
2920599361,6cyclictest11347-21ssh12:35:221
2920599361,6cyclictest11263-21ssh09:55:201
29205993616,12cyclictest2228-21ssh09:43:421
29205993616,12cyclictest2228-21ssh09:43:421
2920599361,5cyclictest16607-21ssh11:20:351
2920799358,15cyclictest29637-21ls11:35:513
2920799357,16cyclictest8050irq/56-dwc_otg_09:30:503
2920799353,19cyclictest8050irq/56-dwc_otg_11:10:403
2920799352,16cyclictest11279-21apt09:55:203
29207993515,10cyclictest850-21ssh11:40:473
29207993512,16cyclictest0-21swapper/310:55:333
2920699359,13cyclictest0-21swapper/212:07:392
2920699354,4cyclictest0-21swapper/212:27:282
29206993516,10cyclictest31-21ksoftirqd/210:25:342
29206993512,8cyclictest505-21cron08:00:202
29206993510,13cyclictest0-21swapper/208:50:432
2920599355,5cyclictest7850irq/56-dwc_otg10:40:571
2920599351,4cyclictest16057-21grep08:00:501
19252350,1sleep10-21swapper/111:01:091
19252350,1sleep10-21swapper/111:01:091
130052351,28sleep213021-21ntp_states10:35:472
2920799341,8cyclictest7643-21latency_hist09:10:213
2920699348,4cyclictest0-21swapper/210:51:392
2920699347,12cyclictest7950irq/56-dwc_otg_11:10:432
2920699346,7cyclictest7850irq/56-dwc_otg11:55:402
2920699345,6cyclictest7850irq/56-dwc_otg11:10:212
29206993428,4cyclictest301ktimersoftd/209:05:352
29206993420,5cyclictest301ktimersoftd/211:03:422
29206993420,5cyclictest301ktimersoftd/211:03:422
29206993416,5cyclictest8121-21sh09:50:412
29206993410,12cyclictest0-21swapper/211:22:552
2920599341,8cyclictest20463-21wget08:15:361
29205993410,4cyclictest3507-21unixbench_singl08:55:501
2920799338,13cyclictest0-21swapper/307:55:463
2920799334,15cyclictest8050irq/56-dwc_otg_08:50:393
2920799333,26cyclictest8050irq/56-dwc_otg_07:20:313
2920799332,5cyclictest7950irq/56-dwc_otg_08:05:443
2920699339,9cyclictest7950irq/56-dwc_otg_10:10:322
2920699338,15cyclictest727-21runrttasks11:17:552
2920699336,19cyclictest0-21swapper/208:15:342
2920699335,6cyclictest7950irq/56-dwc_otg_09:15:382
2920699334,5cyclictest7850irq/56-dwc_otg09:20:322
2920699333,15cyclictest8050irq/56-dwc_otg_11:50:362
2920699332,17cyclictest0-21swapper/208:05:362
2920699331,6cyclictest24729-21ps08:25:472
29206993315,4cyclictest31-21ksoftirqd/210:55:392
2920599332,6cyclictest1080-21ntpq07:20:431
2920599332,5cyclictest29703-21fschecks_time08:40:381
2920599331,6cyclictest6483-21tr07:35:421
2920599331,5cyclictest31535-21tune2fs08:45:401
2920599331,5cyclictest0-21swapper/108:05:411
29205993314,10cyclictest22922-21sh10:47:551
2920799323,14cyclictest8050irq/56-dwc_otg_08:30:493
2920799323,14cyclictest8050irq/56-dwc_otg_08:30:493
2920799322,3cyclictest7950irq/56-dwc_otg_10:51:203
2920699326,18cyclictest61-21kswapd007:50:272
2920699325,5cyclictest8050irq/56-dwc_otg_07:20:212
2920699324,6cyclictest31-21ksoftirqd/211:30:422
2920699322,8cyclictest20151-21sh10:45:212
29206993212,13cyclictest7575-21ssh11:50:142
2920599329,13cyclictest0-21swapper/109:10:211
2920799319,13cyclictest381ktimersoftd/308:15:363
2920799317,16cyclictest381ktimersoftd/310:25:343
2920799313,6cyclictest7850irq/56-dwc_otg08:50:203
29207993116,5cyclictest15950-21timerwakeupswit08:00:493
2920799311,4cyclictest373-21ssh11:00:313
2920799311,4cyclictest373-21ssh11:00:313
2920799311,11cyclictest13896-21kworker/3:110:40:483
2920699318,9cyclictest31-21ksoftirqd/208:00:452
2920699311,6cyclictest26865-21unixbench_multi08:30:492
2920699311,6cyclictest26865-21unixbench_multi08:30:492
29206993115,5cyclictest31-21ksoftirqd/207:45:422
2920699311,4cyclictest0-21swapper/211:40:392
2920799303,5cyclictest8050irq/56-dwc_otg_07:35:483
2920699309,12cyclictest8050irq/56-dwc_otg_09:00:202
2920699307,4cyclictest0-21swapper/210:47:212
2920699302,15cyclictest29-21rcuc/207:40:482
2920699301,16cyclictest6882-21sed07:35:482
29206993010,5cyclictest2770050wget08:35:362
2920599302,6cyclictest1692-21unixbench_singl08:50:491
2920599301,5cyclictest8296-21ntp_kernel_pll_07:40:431
2920799293,6cyclictest24537-21memory08:25:433
29207992912,10cyclictest381ktimersoftd/307:10:443
2920699297,5cyclictest8050irq/56-dwc_otg_07:30:342
2920699294,5cyclictest7850irq/56-dwc_otg08:20:432
2920699292,7cyclictest30374-21head07:10:492
2920699292,7cyclictest18773-21diskstats08:10:382
29206992920,6cyclictest31456-21fschecks_count08:45:392
2920699291,14cyclictest7950irq/56-dwc_otg_09:00:352
2920599291,5cyclictest0-21swapper/107:10:421
29205992914,8cyclictest727-21runrttasks08:13:541
2920599291,19cyclictest0-21swapper/107:30:411
273732296,9sleep20-21swapper/207:05:322
2920799281,16cyclictest0-21swapper/307:40:343
2920699285,5cyclictest7950irq/56-dwc_otg_08:40:362
2920799279,9cyclictest3542-21who08:55:503
2920799275,4cyclictest7950irq/56-dwc_otg_08:40:203
2920799271,5cyclictest0-21swapper/308:40:343
2920799271,4cyclictest0-21swapper/308:20:393
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional