You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-06 - 15:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sat Dec 06, 2025 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3148521730,5sleep03274699cyclictest12:30:410
32746991056,75cyclictest0-21swapper/008:25:280
32746999440,36cyclictest0-21swapper/011:20:280
319902900,2sleep30-21swapper/308:35:343
3274699877,62cyclictest0-21swapper/007:25:270
227092860,2sleep10-21swapper/109:25:321
154812800,3sleep315464-21ssh11:30:293
3274699796,25cyclictest0-21swapper/012:29:580
3274699796,25cyclictest0-21swapper/012:29:580
32746997814,14cyclictest32743-21cyclictest12:15:260
64032770,4sleep09-21ksoftirqd/011:17:140
3274699776,47cyclictest0-21swapper/011:30:280
288142730,1sleep30-21swapper/310:18:483
132942730,4sleep13274799cyclictest11:26:251
3274699727,54cyclictest0-21swapper/010:05:280
280912720,2sleep20-21swapper/210:17:142
32747997110,41cyclictest4143-21latency_hist11:15:151
250542710,2sleep20-21swapper/209:29:312
3274699695,26cyclictest29861-21ssh10:20:260
32746996937,18cyclictest0-21swapper/009:55:260
32746996918,28cyclictest2531-21fschecks_time07:15:310
3274799667,36cyclictest0-21swapper/110:55:291
3274799663,46cyclictest0-21swapper/109:44:481
3274799661,42cyclictest26602-21aten_rbpower_po12:25:311
3274799661,42cyclictest26602-21aten_rbpower_po12:25:311
3274799653,47cyclictest7950irq/56-dwc_otg_11:50:001
3274799645,50cyclictest1783-21sh10:25:391
3274799644,42cyclictest7950irq/56-dwc_otg_11:45:141
3274799644,42cyclictest7950irq/56-dwc_otg_11:45:141
3274799643,45cyclictest7750irq/56-dwc_otg09:31:021
3274799643,45cyclictest7750irq/56-dwc_otg09:31:021
3274799641,48cyclictest6557-21memory12:00:371
3274799641,48cyclictest6557-21memory12:00:371
3274799641,47cyclictest18069-21ssh10:49:241
3274799633,4cyclictest11420-21diskmemload12:35:491
3274799633,48cyclictest7950irq/56-dwc_otg_11:34:221
3274799631,44cyclictest7850irq/56-dwc_otg_12:11:571
3274799625,37cyclictest0-21swapper/109:15:411
3274799623,4cyclictest7950irq/56-dwc_otg_08:45:271
3274799623,45cyclictest0-21swapper/111:21:271
3274799623,37cyclictest7750irq/56-dwc_otg07:10:461
3274799622,41cyclictest7950irq/56-dwc_otg_08:40:141
3274799621,3cyclictest7750irq/56-dwc_otg12:20:301
110462620,2sleep20-21swapper/212:05:432
3274799612,47cyclictest0-21swapper/110:38:131
3274799612,40cyclictest26847-21ntp_states10:15:381
32746996125,5cyclictest0-21swapper/008:05:250
3274799601,6cyclictest4713-21/usr/sbin/munin10:30:271
3274799601,5cyclictest29877-21grep11:05:321
32747996012,30cyclictest4436-21/usr/sbin/munin08:50:341
3274699606,42cyclictest0-21swapper/008:20:190
3274699602,34cyclictest8763-21grep09:50:360
32746996014,26cyclictest2019-21cpu11:55:290
3274799598,42cyclictest938-1kworker/u9:211:56:041
3274799595,45cyclictest451-21rs:main1
3274799594,41cyclictest7750irq/56-dwc_otg09:50:361
3274799592,36cyclictest0-21swapper/112:08:401
3274799591,45cyclictest10687-21grep07:35:431
3274699596,7cyclictest0-21swapper/007:45:250
32746995921,5cyclictest0-21swapper/009:30:300
32746995921,5cyclictest0-21swapper/009:30:300
32746995920,23cyclictest1120-21runrttasks11:45:150
32746995920,23cyclictest1120-21runrttasks11:45:150
3274799582,38cyclictest0-21swapper/110:09:411
3274799582,38cyclictest0-21swapper/107:18:271
3274799581,38cyclictest6934-21grep07:25:411
32746995824,5cyclictest0-21swapper/012:35:430
3274699582,38cyclictest1133-21fschecks_time08:40:320
32746995821,8cyclictest0-21swapper/010:15:140
32746995821,8cyclictest0-21swapper/010:15:140
32746995815,22cyclictest16381-21/usr/sbin/munin10:00:400
3274699581,31cyclictest22432-21aten_rbpower_vo08:10:320
3274799573,5cyclictest7750irq/56-dwc_otg10:53:391
3274799572,42cyclictest31002-21grep10:20:431
3274799572,36cyclictest0-21swapper/111:16:191
3274799571,44cyclictest31209-21apt08:35:131
3274799571,44cyclictest0-21swapper/109:47:261
3274799571,3cyclictest0-21swapper/111:03:291
32746995736,11cyclictest8429-21grep07:30:390
32746995732,7cyclictest7850irq/56-dwc_otg_12:00:390
32746995732,7cyclictest7850irq/56-dwc_otg_12:00:390
3274699571,30cyclictest0-21swapper/007:50:270
3274699571,15cyclictest30685-21kworker/u8:107:10:290
3274799566,39cyclictest7950irq/56-dwc_otg_07:20:431
3274799562,37cyclictest0-21swapper/112:34:281
3274799561,5cyclictest19534-21ssh09:20:381
32746995636,11cyclictest101ktimersoftd/012:05:440
32746995629,16cyclictest9-21ksoftirqd/008:55:430
3274699561,41cyclictest0-21swapper/010:35:260
3274699561,17cyclictest4241-21ls07:20:290
175132560,2sleep20-21swapper/212:14:332
146332560,2sleep30-21swapper/310:44:363
3274799554,37cyclictest7950irq/56-dwc_otg_12:20:051
3274799552,5cyclictest1028-21diskstats08:40:301
3274799552,41cyclictest32446-21ssh11:52:061
3274799551,3cyclictest0-21swapper/111:35:301
3274699554,35cyclictest9-21ksoftirqd/007:35:400
3274699552,43cyclictest0-21swapper/011:05:440
3274699552,29cyclictest0-21swapper/008:15:400
3274699551,4cyclictest0-21swapper/012:20:240
3274699551,36cyclictest9-21ksoftirqd/010:40:330
3274699551,30cyclictest23481-21unixbench_multi10:55:470
169922550,2sleep20-21swapper/210:47:052
85832540,5sleep23274899cyclictest11:20:342
3274799547,42cyclictest11422-21aten_rbpower_cu07:40:271
3274799546,19cyclictest0-21swapper/109:05:291
3274799542,39cyclictest0-21swapper/109:39:021
3274799542,35cyclictest0-21swapper/107:56:151
32747995418,33cyclictest9245-21latency_hist07:35:141
3274799541,5cyclictest19441-21tail08:00:401
3274699544,11cyclictest0-21swapper/007:55:290
3274699542,35cyclictest13-21rcuc/011:00:370
3274699541,33cyclictest23225-21ps09:25:390
3274799532,43cyclictest0-21swapper/109:14:391
3274799531,3cyclictest0-21swapper/109:59:541
3274699538,22cyclictest5546-21grep09:45:420
3274699536,29cyclictest0-21swapper/009:00:300
3274699536,29cyclictest0-21swapper/009:00:300
3274699534,27cyclictest0-21swapper/008:45:440
3274699532,30cyclictest29479-21aten_rbpower_po09:35:300
208032530,2sleep20-21swapper/209:22:272
3274999526,7cyclictest0-21swapper/307:40:253
3274799525,37cyclictest7750irq/56-dwc_otg10:01:461
3274799522,5cyclictest26015-21apt-get08:20:201
3274699526,27cyclictest0-21swapper/009:40:280
32746995220,14cyclictest7950irq/56-dwc_otg_09:10:130
322352523,10sleep00-21swapper/007:07:070
32749995116,7cyclictest7950irq/56-dwc_otg_11:45:303
3274799514,30cyclictest7750irq/56-dwc_otg10:10:461
3274799514,30cyclictest7750irq/56-dwc_otg10:10:461
3274799511,4cyclictest21970-21apt-get08:10:201
3274799511,3cyclictest7950irq/56-dwc_otg_08:25:381
32746995134,3cyclictest0-21swapper/009:25:150
32746995125,14cyclictest12955-21unixbench_multi09:10:440
3274699511,29cyclictest26916-21proc_pri10:15:390
323002510,5sleep33274999cyclictest09:39:273
3274799502,45cyclictest7821-21idleruntime-cro09:00:141
3274799501,46cyclictest24150-21grep08:15:291
3274799501,45cyclictest22222-21sed08:10:151
32747995014,33cyclictest147132chrt07:49:521
3274699503,29cyclictest30394-21ps08:30:400
3274699502,32cyclictest0-21swapper/008:00:320
32746995021,5cyclictest15207-21kworker/u8:211:45:200
32746995018,5cyclictest0-21swapper/008:50:290
3274699501,36cyclictest0-21swapper/010:26:030
3274799492,43cyclictest8576-21latency09:00:361
3274799492,43cyclictest8576-21latency09:00:361
3274699491,24cyclictest5894-21unixbench-2d10:30:450
3274699482,24cyclictest0-21swapper/011:30:140
32746994821,9cyclictest101ktimersoftd/011:54:480
3274699481,26cyclictest0-21swapper/009:15:290
3274999477,19cyclictest0-21swapper/307:35:203
3274799471,7cyclictest15090-21aten_rbpower_vo07:50:291
3274699472,29cyclictest0-21swapper/011:35:360
3274699461,23cyclictest0-21swapper/012:10:310
32746994611,6cyclictest7850irq/56-dwc_otg_10:50:280
32749994511,4cyclictest7950irq/56-dwc_otg_12:10:243
3274699452,8cyclictest16287-21grep10:45:420
3274699451,32cyclictest0-21swapper/011:11:560
140862450,2sleep30-21swapper/311:28:083
3274999447,22cyclictest0-21swapper/309:00:293
3274999447,22cyclictest0-21swapper/309:00:293
32746994425,4cyclictest7950irq/56-dwc_otg_07:43:190
3274999439,22cyclictest0-21swapper/309:40:443
3274899420,28cyclictest0-21swapper/210:10:142
3274699427,18cyclictest32026-21seq08:35:350
3274999409,7cyclictest0-21swapper/311:10:343
3274999402,4cyclictest0-21swapper/310:55:313
3274999398,11cyclictest0-21swapper/311:35:393
32749993918,11cyclictest13353-21run-parts12:09:273
3274999390,23cyclictest7950irq/56-dwc_otg_07:45:303
3274899391,24cyclictest0-21swapper/207:15:272
3274999387,13cyclictest7950irq/56-dwc_otg_10:39:353
32749993822,14cyclictest0-21swapper/308:30:273
32749993817,4cyclictest1861-21/usr/sbin/munin11:55:363
324732387,19sleep332480-21modprobe07:10:143
32749993721,12cyclictest0-21swapper/310:30:293
3274999371,22cyclictest28767-21ls08:25:413
3274999371,20cyclictest22506-21ssh10:10:153
3274999368,22cyclictest0-21swapper/308:10:193
3274999367,24cyclictest0-21swapper/310:10:283
3274999367,24cyclictest0-21swapper/310:10:283
3274999362,5cyclictest7950irq/56-dwc_otg_07:25:423
32749993612,11cyclictest11420-21diskmemload09:22:463
3274899364,4cyclictest7750irq/56-dwc_otg08:20:262
3274899362,6cyclictest0-21swapper/208:35:272
3274899362,20cyclictest0-21swapper/209:30:472
3274899362,20cyclictest0-21swapper/209:30:472
32748993611,5cyclictest29-21rcuc/209:35:442
3274999356,5cyclictest11802-21ssh09:55:253
3274999356,22cyclictest0-21swapper/307:50:283
3274999351,8cyclictest1614-21head07:10:533
32749993515,4cyclictest381ktimersoftd/309:10:283
32749993514,5cyclictest381ktimersoftd/312:27:513
32749993514,5cyclictest381ktimersoftd/312:27:513
3274999351,20cyclictest0-21swapper/307:20:273
3274999351,17cyclictest7633-21ssh11:19:563
3274999351,16cyclictest0-21swapper/309:28:563
32748993514,5cyclictest11660-21fschecks_count07:40:312
32748993510,6cyclictest5701-21unixbench_singl11:15:432
3274999346,13cyclictest7950irq/56-dwc_otg_12:30:453
3274999342,21cyclictest7950irq/56-dwc_otg_08:15:293
3274999342,18cyclictest7950irq/56-dwc_otg_08:50:373
32749993415,5cyclictest2463-21/usr/sbin/munin12:35:453
32749993412,11cyclictest18349-21sh09:20:073
32749993411,9cyclictest11420-21diskmemload12:21:543
3274899342,20cyclictest0-21swapper/208:40:572
32748993418,9cyclictest9041-21switchtime09:50:402
3274999336,24cyclictest4430-21rm09:45:283
3274999336,15cyclictest0-21swapper/308:55:293
3274999334,15cyclictest0-21swapper/311:50:383
3274999332,21cyclictest0-21swapper/311:08:213
3274999331,6cyclictest7850irq/56-dwc_otg_09:55:143
3274999331,18cyclictest7850irq/56-dwc_otg_11:02:423
3274999331,18cyclictest0-21swapper/311:24:573
3274899337,17cyclictest0-21swapper/209:00:292
3274899337,17cyclictest0-21swapper/209:00:292
3274899336,5cyclictest7950irq/56-dwc_otg_09:10:302
3274899332,3cyclictest7850irq/56-dwc_otg_08:15:292
3274899332,19cyclictest6881-21timerwakeupswit07:25:412
32748993319,5cyclictest301ktimersoftd/210:40:402
3274899331,8cyclictest1280-21ssh11:54:432
32748993312,5cyclictest11420-21diskmemload12:15:392
3274899331,21cyclictest28443-21ls08:25:362
3274899331,21cyclictest0-21swapper/209:40:212
3274899331,19cyclictest0-21swapper/209:05:552
3274899331,17cyclictest0-21swapper/212:28:042
3274899331,17cyclictest0-21swapper/212:28:042
3274899331,17cyclictest0-21swapper/211:38:002
32748993311,6cyclictest29079-21taskset11:04:512
3274899331,14cyclictest3710-21grep12:35:592
3274999329,4cyclictest0-21swapper/310:50:373
3274999327,11cyclictest0-21swapper/310:27:583
3274999326,14cyclictest0-21swapper/307:30:293
32749993215,9cyclictest20664-21sh12:17:433
32749993215,4cyclictest27601-21latency_hist08:25:143
3274999321,12cyclictest0-21swapper/308:45:303
3274899329,4cyclictest0-21swapper/211:25:562
3274899328,4cyclictest2176-21latency_hist12:35:142
3274899326,5cyclictest1028-21ssh10:25:272
3274899325,12cyclictest7950irq/56-dwc_otg_12:00:442
3274899325,12cyclictest7950irq/56-dwc_otg_12:00:442
3274899323,11cyclictest0-21swapper/211:40:362
3274899323,11cyclictest0-21swapper/211:40:362
3274899321,3cyclictest7850irq/56-dwc_otg_10:00:272
3274899321,22cyclictest13453-21cut07:45:302
3274899321,21cyclictest5130-21idleruntime10:30:352
3274899321,21cyclictest4692-21ssh09:45:322
3274899321,18cyclictest7750irq/56-dwc_otg11:11:422
3274899321,16cyclictest29697-21aten_rbpower_po11:05:292
3274899321,14cyclictest0-21swapper/211:33:252
316362327,12sleep20-21swapper/207:05:392
300502320,2sleep20-21swapper/208:30:342
3274999317,8cyclictest22914-21fschecks_count11:40:333
3274999317,8cyclictest22914-21fschecks_count11:40:333
3274999317,4cyclictest7950irq/56-dwc_otg_10:20:353
3274999312,4cyclictest21965-21modprobe08:10:143
3274999311,19cyclictest0-21swapper/309:33:053
3274999311,19cyclictest0-21swapper/309:33:053
3274899315,17cyclictest7950irq/56-dwc_otg_10:35:332
32748993114,10cyclictest10769-21vmstat07:35:442
3274899311,23cyclictest7950irq/56-dwc_otg_07:55:432
3274899311,21cyclictest0-21swapper/210:50:272
3274899311,21cyclictest0-21swapper/208:52:182
3274899311,20cyclictest23318-21timerandwakeup10:55:452
3274899311,20cyclictest0-21swapper/211:49:562
3274899311,16cyclictest0-21swapper/210:10:252
3274899311,16cyclictest0-21swapper/210:10:252
3274899311,12cyclictest7850irq/56-dwc_otg_09:55:252
3274999308,11cyclictest37-21rcuc/308:41:223
32749993019,6cyclictest0-21swapper/310:00:263
3274999301,5cyclictest9842-21/usr/sbin/munin09:05:383
3274899308,4cyclictest0-21swapper/207:10:502
3274899304,4cyclictest22920-21ssh12:20:332
32748993013,8cyclictest9370-21cat07:35:142
3274899301,19cyclictest0-21swapper/211:56:112
3274899301,16cyclictest4131-21wget07:20:282
3274899301,15cyclictest2737-21aten_rbpower_vo08:45:312
3274899301,12cyclictest0-21swapper/207:50:402
3274999298,4cyclictest0-21swapper/312:00:323
3274999298,4cyclictest0-21swapper/312:00:323
32749992923,4cyclictest16910-21aten_rbpower_vo07:55:303
3274999292,24cyclictest7950irq/56-dwc_otg_10:45:273
3274899292,16cyclictest0-21swapper/208:10:492
3274899291,21cyclictest0-21swapper/209:18:172
3274899291,18cyclictest0-21swapper/208:05:402
32748992910,12cyclictest322622chrt10:23:092
323872296,14sleep10-21swapper/107:09:061
3274999286,5cyclictest7950irq/56-dwc_otg_07:15:403
32749992811,5cyclictest1120-21runrttasks08:00:333
3274899281,20cyclictest0-21swapper/208:04:572
3274899267,17cyclictest0-21swapper/208:55:292
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional