You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-10 - 18:10

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Thu Sep 10, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1992221470,6sleep179799cyclictest09:58:161
797991323,98cyclictest10442sleep109:35:341
797991323,98cyclictest10442sleep109:35:341
2250121240,15sleep279899cyclictest11:20:312
7969912119,72cyclictest0-21swapper/009:35:340
7969912119,72cyclictest0-21swapper/009:35:340
797991176,85cyclictest14672-21ssh10:30:451
7969911625,62cyclictest21405-21basename10:00:320
7969911340,47cyclictest9504-21/usr/sbin/munin10:25:400
7969911340,47cyclictest9504-21/usr/sbin/munin10:25:400
7989911110,18cyclictest7750irq/56-dwc_otg10:00:332
798991073,26cyclictest7750irq/56-dwc_otg09:35:332
798991073,26cyclictest7750irq/56-dwc_otg09:35:332
7969910731,24cyclictest14760-21cat10:30:460
7969910630,27cyclictest5261-21fschecks_time09:40:370
3067221060,6sleep079699cyclictest10:50:370
796991021,53cyclictest0-21swapper/011:45:290
796991016,67cyclictest0-21swapper/007:10:310
799991008,17cyclictest1083-21fschecks_count09:35:343
799991008,17cyclictest1083-21fschecks_count09:35:343
79999985,12cyclictest21401-21aten_rbpower_cu10:00:323
799999710,9cyclictest7850irq/56-dwc_otg_09:55:463
79799962,73cyclictest21434-21aten_rbpower_en10:00:321
79699966,74cyclictest0-21swapper/008:10:320
796999623,43cyclictest0-21swapper/009:55:460
79899945,11cyclictest10314-21ssh10:25:432
79899945,11cyclictest10314-21ssh10:25:432
79799942,67cyclictest10308-21ntp_states10:25:411
79799942,67cyclictest10308-21ntp_states10:25:411
210032890,2sleep00-21swapper/012:36:010
797998816,46cyclictest5178-21ssh09:40:351
79699885,27cyclictest18381-21ssh11:15:320
796998613,55cyclictest12499-21unixbench_singl12:25:480
796998111,47cyclictest21906-21sed08:05:460
796998111,47cyclictest21906-21sed08:05:460
79899803,8cyclictest0-21swapper/209:55:462
796997712,52cyclictest0-21swapper/011:35:190
286702770,5sleep079699cyclictest10:09:150
191332770,2sleep30-21swapper/311:55:343
79899763,9cyclictest5179-21diskstats09:40:362
796997621,31cyclictest17868-21apt-get10:35:310
796997615,41cyclictest9205-21munin-plugin-st10:25:180
79699758,52cyclictest31834-21latency12:10:380
79699758,52cyclictest31834-21latency12:10:380
796997514,43cyclictest8000-21ssh11:41:020
796997513,48cyclictest28577-21ssh11:26:590
796997511,48cyclictest23815-21ssh12:00:420
79699742,51cyclictest0-21swapper/009:20:190
796997423,26cyclictest18629-21latency_hist11:55:190
796997333,10cyclictest1180-21runrttasks09:05:310
79999724,47cyclictest0-21swapper/309:40:403
79699726,37cyclictest0-21swapper/007:30:310
79699725,47cyclictest12794-21cron09:50:180
796997218,33cyclictest465-21awk09:35:190
79699718,46cyclictest12140-21gpg-connect-age09:10:200
79699712,55cyclictest0-21swapper/008:00:310
79699712,55cyclictest0-21swapper/008:00:310
796997112,42cyclictest1861-21/usr/sbin/munin10:55:280
796997011,45cyclictest28426-21ssh10:47:410
796997011,45cyclictest28426-21ssh10:47:410
796997010,45cyclictest30191-21ssh12:08:590
134912700,2sleep00-21swapper/007:45:190
79699698,42cyclictest0-21swapper/009:25:250
79699697,45cyclictest0-21swapper/012:31:380
79699697,45cyclictest0-21swapper/012:31:380
79699695,54cyclictest0-21swapper/011:58:120
796996914,40cyclictest6692-21runrttasks12:20:110
796996910,46cyclictest5017-21ssh11:38:080
79699689,41cyclictest0-21swapper/011:01:380
796996826,3cyclictest0-21swapper/010:40:290
79699681,48cyclictest23737-21grep11:20:460
797996710,34cyclictest7750irq/56-dwc_otg10:55:191
79699679,40cyclictest1349-21tr08:40:340
79699672,43cyclictest27963-21latency_hist08:25:190
796996710,41cyclictest4725-21ssh10:19:410
79699652,46cyclictest24759-21sed09:25:200
796996513,38cyclictest16772-21ssh09:55:020
796996416,37cyclictest467-21ssh10:14:300
796996410,50cyclictest10055-21latency_hist09:05:190
79799638,37cyclictest0-21swapper/112:35:251
79699635,38cyclictest0-21swapper/007:20:470
79799627,41cyclictest23458-21if_err_wlan012:00:371
79699629,42cyclictest0-21swapper/007:55:140
79699628,39cyclictest31835-21taskset08:35:240
79699626,38cyclictest0-21swapper/007:39:260
79699623,39cyclictest0-21swapper/008:15:440
796996215,30cyclictest10441-21fschecks_time11:05:350
79799616,38cyclictest7750irq/56-dwc_otg09:15:421
79799611,5cyclictest2551-21aten_rbpower_vo11:35:321
79699611,41cyclictest0-21swapper/008:30:450
79699601,30cyclictest14619-21fschecks_time11:10:380
796996011,34cyclictest7438-21fschecks_time12:20:350
79799595,40cyclictest7750irq/56-dwc_otg12:23:511
79799586,41cyclictest7950irq/56-dwc_otg_10:45:251
79799586,41cyclictest7950irq/56-dwc_otg_10:45:251
79799586,40cyclictest0-21swapper/109:05:321
79799585,5cyclictest7750irq/56-dwc_otg12:20:191
79799581,4cyclictest7850irq/56-dwc_otg_08:00:401
79799581,4cyclictest7850irq/56-dwc_otg_08:00:401
79799581,45cyclictest7850irq/56-dwc_otg_09:30:191
796995828,13cyclictest7850irq/56-dwc_otg_08:50:470
796995816,36cyclictest17208-21apt-get07:55:290
796995810,35cyclictest0-21swapper/008:26:550
81512570,2sleep28154-21aten_rbpower_po07:30:322
79799574,39cyclictest0-21swapper/110:22:501
79799571,8cyclictest30435-21grep10:10:421
79899563,33cyclictest0-21swapper/210:30:462
79799565,5cyclictest17874-21ssh10:35:311
79799562,8cyclictest7950irq/56-dwc_otg_11:55:131
79799561,5cyclictest7750irq/56-dwc_otg11:10:141
79799561,10cyclictest7850irq/56-dwc_otg_11:30:151
79699565,43cyclictest4111-21taskset07:19:030
79799559,3cyclictest7950irq/56-dwc_otg_12:31:521
79799559,3cyclictest7950irq/56-dwc_otg_12:31:521
79799556,7cyclictest7950irq/56-dwc_otg_12:25:531
79799556,36cyclictest0-21swapper/110:15:421
79799553,3cyclictest0-21swapper/109:21:161
79799551,5cyclictest0-21swapper/107:55:191
79699552,37cyclictest0-21swapper/007:45:420
326502553,42sleep20-21swapper/207:05:592
799995419,9cyclictest7850irq/56-dwc_otg_10:25:413
799995419,9cyclictest7850irq/56-dwc_otg_10:25:413
79799546,37cyclictest7850irq/56-dwc_otg_09:10:341
79799545,40cyclictest7950irq/56-dwc_otg_07:55:401
79799544,4cyclictest0-21swapper/109:45:331
79799542,6cyclictest9333-21chrt07:31:401
79799542,5cyclictest31941-21aten_rbpower_po08:35:331
79799542,3cyclictest0-21swapper/111:32:161
79699548,36cyclictest0-21swapper/008:46:180
150222540,6sleep179799cyclictest11:10:431
79799532,8cyclictest7750irq/56-dwc_otg09:53:401
79699536,34cyclictest7504-21runrttasks07:26:220
79799524,7cyclictest7950irq/56-dwc_otg_12:09:111
79799523,7cyclictest7950irq/56-dwc_otg_09:33:011
79799521,6cyclictest13855-21fschecks_count07:45:361
79799521,3cyclictest0-21swapper/110:40:371
799995113,12cyclictest14651-21gzip10:30:443
79799515,3cyclictest7850irq/56-dwc_otg_11:58:011
79799512,41cyclictest0-21swapper/107:16:361
79799511,9cyclictest0-21swapper/110:08:411
797995117,20cyclictest6214-21ls07:25:191
79799511,6cyclictest3371-21fschecks_time08:45:351
797995116,32cyclictest1180-21runrttasks11:18:521
797995112,36cyclictest6535-21/usr/sbin/munin11:40:341
797995111,36cyclictest10471-21sed07:35:381
79699517,41cyclictest0-21swapper/008:59:490
79799501,8cyclictest20830-21aten_rbpower_cu08:05:321
79799501,8cyclictest20830-21aten_rbpower_cu08:05:321
79799501,7cyclictest23230-21ntpq11:20:401
79799501,4cyclictest5080-21ssh10:59:151
79799501,11cyclictest0-21swapper/111:04:561
79799501,10cyclictest0-21swapper/108:59:571
797995010,37cyclictest11100-21cat11:45:361
79799491,9cyclictest0-21swapper/108:30:281
79799491,8cyclictest31585-21grep12:10:351
79799491,8cyclictest31585-21grep12:10:351
79799491,8cyclictest24707-21fschecks_time08:15:351
79799491,7cyclictest6257-21/usr/sbin/munin07:25:311
79799482,8cyclictest26915-21wc08:20:381
79799482,7cyclictest5334-21wc08:50:391
797994817,28cyclictest22620-21/usr/sbin/munin08:10:291
79799481,3cyclictest0-21swapper/108:40:391
79799481,3cyclictest0-21swapper/107:45:191
79799473,6cyclictest1101-21tune2fs07:10:351
79799471,8cyclictest28562-21cut08:25:361
79799471,7cyclictest8703-21fschecks_count09:00:361
79999467,27cyclictest0-21swapper/308:50:333
327612462,36sleep00-21swapper/007:07:270
79899454,3cyclictest7950irq/56-dwc_otg_09:50:312
798994419,9cyclictest0-21swapper/211:30:512
44602420,3sleep24465-21acpi07:20:302
79999416,25cyclictest0-21swapper/310:20:313
798994110,19cyclictest0-21swapper/211:25:322
300972400,1sleep30-21swapper/310:50:203
239812400,1sleep20-21swapper/209:24:152
79999393,26cyclictest7950irq/56-dwc_otg_09:20:293
799993911,8cyclictest21896-21grep08:05:463
799993911,8cyclictest21896-21grep08:05:463
799993816,11cyclictest22120-21sh12:38:043
798993817,7cyclictest3108-21grep10:55:462
799993723,8cyclictest381ktimersoftd/309:33:083
799993722,5cyclictest20448-21idleruntime-cro09:20:193
79999371,27cyclictest22767-21cut11:20:353
79899378,18cyclictest301ktimersoftd/207:40:392
798993713,6cyclictest9379-21sh12:22:342
798993710,5cyclictest9561-21unixbench_multi09:00:472
160852370,3sleep3795-21cyclictest09:15:103
79999365,15cyclictest7850irq/56-dwc_otg_08:25:363
79999362,6cyclictest7950irq/56-dwc_otg_07:45:303
799993617,4cyclictest27723-21ssh12:05:393
799993617,10cyclictest20969-21sh11:18:093
799993616,4cyclictest14905-21aten_rbpower_vo11:50:333
799993610,5cyclictest7850irq/56-dwc_otg_12:20:303
799993610,5cyclictest0-21swapper/312:30:453
799993610,5cyclictest0-21swapper/312:30:453
798993616,8cyclictest20432-21sh11:56:112
79999359,3cyclictest0-21swapper/311:30:593
79999356,17cyclictest0-21swapper/310:44:433
79999356,13cyclictest9531-21ssh09:45:383
799993514,5cyclictest11598-21smartpower11:45:433
799993512,5cyclictest19676-21grep08:00:423
799993512,5cyclictest19676-21grep08:00:423
79999351,16cyclictest26934-21ssh12:05:193
79899352,6cyclictest9064-21aten_rbpower_cu09:45:322
79899351,25cyclictest6557-21fschecks_time07:25:342
79899351,20cyclictest20571-21latency_hist08:05:202
79899351,20cyclictest20571-21latency_hist08:05:202
79899351,10cyclictest7950irq/56-dwc_otg_12:25:332
48512351,4sleep24861-21ssh10:19:572
79999348,5cyclictest7850irq/56-dwc_otg_12:15:273
79999346,17cyclictest7750irq/56-dwc_otg10:45:353
79999346,17cyclictest7750irq/56-dwc_otg10:45:353
79999345,16cyclictest14448-21cpuspeed11:10:363
79999341,7cyclictest31597-21cron08:35:183
799993412,6cyclictest28096-21ssh10:08:113
798993418,9cyclictest31178-21sh12:10:212
798993418,9cyclictest31178-21sh12:10:212
79899341,25cyclictest21568-21ssh12:37:042
798993411,13cyclictest0-21swapper/211:45:332
79999339,12cyclictest0-21swapper/311:05:443
79999338,15cyclictest0-21swapper/309:51:303
79999332,15cyclictest25023-21ssh09:25:313
79999331,7cyclictest4335-21ssh11:36:523
799993311,6cyclictest15866-21cut07:50:383
799993310,4cyclictest0-21swapper/311:01:463
799993310,19cyclictest39-21ksoftirqd/310:35:323
79899334,5cyclictest0-21swapper/211:09:142
79899332,24cyclictest7950irq/56-dwc_otg_09:25:352
798993316,5cyclictest24512-21aten_rbpower_po08:15:322
79899331,5cyclictest12987-21ls09:10:422
79899331,24cyclictest7674-21timerwakeupswit11:40:452
798993311,5cyclictest2634-21/usr/sbin/munin07:15:362
798993310,15cyclictest0-21swapper/210:05:472
79999329,13cyclictest39-21ksoftirqd/311:27:423
799993214,5cyclictest7482-21swap11:40:433
799993212,5cyclictest2493-21latency_hist07:15:193
799993210,4cyclictest0-21swapper/310:13:063
79899329,13cyclictest0-21swapper/210:50:442
79899328,5cyclictest0-21swapper/210:35:382
79899327,13cyclictest7950irq/56-dwc_otg_08:45:372
798993212,4cyclictest26811-21latency_hist12:05:192
79999317,5cyclictest7950irq/56-dwc_otg_08:20:193
79999316,14cyclictest3064-21unixbench-2d10:55:463
79999315,4cyclictest7850irq/56-dwc_otg_08:40:463
799993116,6cyclictest3845-21ssh10:18:043
79999311,26cyclictest8400-21uniq07:30:353
79999311,19cyclictest24185-21latency_hist08:15:193
79899317,5cyclictest7750irq/56-dwc_otg12:35:192
79899317,5cyclictest7750irq/56-dwc_otg12:35:192
79899317,12cyclictest7950irq/56-dwc_otg_09:15:352
79899316,9cyclictest7950irq/56-dwc_otg_10:40:292
79899315,5cyclictest4218-21sed12:15:452
79899313,16cyclictest31744-21latency_hist08:35:192
79899312,4cyclictest4842-21apt-get08:50:312
798993121,7cyclictest1180-21runrttasks09:32:512
79899311,5cyclictest0-21swapper/208:05:322
79899311,5cyclictest0-21swapper/208:05:322
79899311,24cyclictest0-21swapper/210:21:572
79899311,23cyclictest0-21swapper/211:40:102
178642310,2sleep20-21swapper/207:55:402
79999308,4cyclictest7950irq/56-dwc_otg_08:58:163
79999304,5cyclictest7850irq/56-dwc_otg_12:15:193
79999304,5cyclictest7850irq/56-dwc_otg_12:15:193
79899308,5cyclictest301ktimersoftd/211:50:452
79899307,10cyclictest7950irq/56-dwc_otg_09:05:432
79899305,19cyclictest7950irq/56-dwc_otg_10:47:232
79899305,19cyclictest7950irq/56-dwc_otg_10:47:232
79899304,13cyclictest7850irq/56-dwc_otg_11:10:382
79899303,5cyclictest7850irq/56-dwc_otg_08:25:352
79899301,7cyclictest23711-21grep08:10:462
79899301,20cyclictest0-21swapper/207:10:342
154742300,1sleep20-21swapper/207:50:322
79999297,10cyclictest7950irq/56-dwc_otg_12:30:193
79999291,22cyclictest9452-21timerwakeupswit09:00:463
799992910,4cyclictest3172-21sort07:15:383
79899291,14cyclictest0-21swapper/208:37:162
79899291,12cyclictest0-21swapper/211:00:462
321992295,15sleep332275-21smartpower07:05:443
79999286,4cyclictest7850irq/56-dwc_otg_07:20:393
79999282,5cyclictest6712-21/usr/sbin/munin07:25:363
799992818,7cyclictest27972-21apt-config08:25:193
79999281,24cyclictest13384-21ls07:45:193
79899283,3cyclictest0-21swapper/210:10:282
79899281,6cyclictest0-21swapper/208:40:442
798992812,6cyclictest0-21swapper/211:15:302
79899281,15cyclictest7750irq/56-dwc_otg07:35:452
3572283,15sleep10-21swapper/107:08:171
799992712,5cyclictest4250-21taskset08:46:073
79999271,15cyclictest10534-21cut09:05:363
79899271,19cyclictest0-21swapper/212:05:322
799992614,9cyclictest32174-21cut08:35:363
79899264,10cyclictest7850irq/56-dwc_otg_09:00:192
798992617,7cyclictest7950irq/56-dwc_otg_08:20:382
79899261,13cyclictest0-21swapper/207:45:462
79999254,9cyclictest7950irq/56-dwc_otg_07:58:053
79999251,13cyclictest0-21swapper/307:37:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional