You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-02 - 23:48

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot3s.osadl.org (updated Fri Jan 02, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1294999927,25cyclictest0-21swapper/007:25:250
12949998713,55cyclictest0-21swapper/007:10:140
12949997820,9cyclictest0-21swapper/009:35:230
17762770,1sleep10-21swapper/112:35:291
1295399763,46cyclictest0-21swapper/307:10:153
12949997610,37cyclictest0-21swapper/012:20:250
1294999742,40cyclictest0-21swapper/009:10:230
311562730,3sleep21295199cyclictest08:00:182
1294999737,22cyclictest0-21swapper/009:40:250
12949997111,29cyclictest0-21swapper/010:35:260
1295099703,33cyclictest0-21swapper/107:10:141
324342690,5sleep21295199cyclictest09:30:392
183842680,3sleep218391-21switchtime11:50:382
1295399684,45cyclictest7950irq/56-dwc_otg_08:05:383
282722670,5sleep01294999cyclictest09:20:310
1295399671,55cyclictest15125-21grep07:15:303
1295099662,10cyclictest7750irq/56-dwc_otg11:55:251
1294999662,37cyclictest393-21grep11:00:400
1294999662,28cyclictest0-21swapper/012:10:240
1295399655,36cyclictest7950irq/56-dwc_otg_07:25:263
1295399653,49cyclictest7950irq/56-dwc_otg_08:20:413
1295399652,44cyclictest24154-21fschecks_time07:40:293
1295399647,38cyclictest25744-21cut07:45:293
1295399642,50cyclictest28707-21sed10:50:343
1295099644,46cyclictest7750irq/56-dwc_otg08:30:241
12949996415,10cyclictest0-21swapper/009:15:240
1294999641,33cyclictest0-21swapper/008:10:240
1294999641,24cyclictest0-21swapper/010:00:210
1295399631,42cyclictest7950irq/56-dwc_otg_07:20:263
1295399631,33cyclictest7750irq/56-dwc_otg12:00:413
1294999631,31cyclictest0-21swapper/008:05:250
96902620,2sleep20-21swapper/208:30:252
1295399622,42cyclictest0-21swapper/308:15:353
1295399621,37cyclictest7750irq/56-dwc_otg10:05:273
1295399618,32cyclictest7950irq/56-dwc_otg_08:00:393
1295099611,38cyclictest23-21ksoftirqd/110:05:231
1294999616,31cyclictest0-21swapper/011:30:410
1294999615,29cyclictest15761-21aten_rbpower_en11:45:280
12949996134,17cyclictest21376-21fschecks_count12:00:290
1295399602,46cyclictest7950irq/56-dwc_otg_07:50:383
1295399601,4cyclictest32074-21latency09:30:333
1294999607,50cyclictest0-21swapper/007:20:230
1294999605,27cyclictest0-21swapper/012:35:120
1294999602,32cyclictest0-21swapper/009:00:120
1295399591,5cyclictest21320-21unixbench_multi07:30:423
1295399581,4cyclictest29386-21apt-get09:25:183
1295399581,43cyclictest25501-21runrttasks09:12:093
12950995817,19cyclictest887-21runrttasks09:35:231
1294999586,31cyclictest22984-21grep12:05:250
1294999583,29cyclictest0-21swapper/010:30:250
12949995832,3cyclictest0-21swapper/010:05:230
12949995822,5cyclictest0-21swapper/009:05:220
1295399572,40cyclictest0-21swapper/312:06:073
1295399571,45cyclictest19375-21aten_rbpower_po11:55:273
12949995738,5cyclictest101ktimersoftd/010:55:390
12949995737,9cyclictest6658-21ntpq08:20:350
12949995724,17cyclictest0-21swapper/008:30:390
1294999571,34cyclictest18307-21grep10:20:400
1295399563,36cyclictest0-21swapper/310:30:303
1295399561,42cyclictest0-21swapper/308:58:513
1295399561,38cyclictest4976-21fschecks_count11:15:313
12949995616,27cyclictest0-21swapper/007:55:220
1295399552,40cyclictest0-21swapper/309:06:323
1295399551,37cyclictest0-21swapper/310:58:513
12949995532,10cyclictest27788-21sort07:50:300
1294999551,45cyclictest0-21swapper/007:40:220
1295399544,38cyclictest26039-21runrttasks12:11:583
1295399542,36cyclictest30479-21cut12:25:293
1295399541,40cyclictest16860-21runrttasks10:19:293
1295399541,36cyclictest0-21swapper/309:03:543
1295399541,36cyclictest0-21swapper/309:03:543
1295099542,9cyclictest0-21swapper/107:15:261
1295099542,12cyclictest7750irq/56-dwc_otg10:20:241
12949995427,6cyclictest7950irq/56-dwc_otg_11:55:300
12949995425,3cyclictest0-21swapper/011:40:250
1294999542,32cyclictest6400-21aten_rbpower_po09:50:260
1294999542,32cyclictest2253-21unixbench_singl11:05:400
12949995423,16cyclictest4829-21fschecks_time09:45:310
1294999542,23cyclictest0-21swapper/008:25:230
1294999542,23cyclictest0-21swapper/008:25:230
1294999541,50cyclictest9-21ksoftirqd/010:50:240
1295199535,11cyclictest7750irq/56-dwc_otg09:25:252
1294999536,33cyclictest0-21swapper/009:25:180
12949995332,4cyclictest14846-21aten_rbpower_en07:15:270
1294999531,29cyclictest31444-21grep08:00:280
12949995310,6cyclictest0-21swapper/012:15:250
12949995310,6cyclictest0-21swapper/012:15:250
1295399527,8cyclictest7950irq/56-dwc_otg_10:10:233
1295399525,43cyclictest1356-21/usr/sbin/munin09:35:313
1295399522,4cyclictest7950irq/56-dwc_otg_07:35:313
1295399522,4cyclictest7950irq/56-dwc_otg_07:35:313
1295399521,37cyclictest7850irq/56-dwc_otg_11:35:293
1295399521,35cyclictest0-21swapper/310:47:393
1295399521,35cyclictest0-21swapper/310:47:393
1295099524,8cyclictest0-21swapper/111:45:251
1294999523,3cyclictest0-21swapper/008:15:290
12949995233,4cyclictest25737-21cut07:45:290
1294999522,29cyclictest24626-21tr10:40:260
1294999522,28cyclictest17745-21cut11:50:290
1295399514,44cyclictest2519-21chrt11:08:213
1295399512,46cyclictest28685-21fschecks_time12:20:303
1295399511,7cyclictest6807-21grep09:50:323
1295399511,5cyclictest6641-21/usr/sbin/munin11:20:313
1295399511,4cyclictest3970-21ls11:10:413
1295399511,47cyclictest2793-21aten_rbpower_po09:40:263
1295399511,46cyclictest27098-21grep09:15:403
1295399511,45cyclictest20737-21sort10:30:133
1295399511,37cyclictest29259-21/usr/sbin/munin07:55:363
12953995111,38cyclictest17348-21cat11:50:123
1295099514,11cyclictest7750irq/56-dwc_otg07:50:241
12950995113,6cyclictest7950irq/56-dwc_otg_10:40:241
1294999518,29cyclictest7850irq/56-dwc_otg_11:30:130
1294999518,29cyclictest7850irq/56-dwc_otg_11:30:130
12949995128,13cyclictest23084-21ls07:35:390
12949995128,13cyclictest23084-21ls07:35:390
1294999512,31cyclictest6765-21wget11:20:260
12949995122,6cyclictest0-21swapper/008:45:250
1294999511,36cyclictest0-21swapper/009:55:250
1294999511,36cyclictest0-21swapper/009:55:250
1294999511,35cyclictest17203-21uniq08:50:300
1295399508,39cyclictest10708-21unixbench_multi08:30:393
1295399501,46cyclictest23746-21grep10:35:413
1295399501,40cyclictest0-21swapper/308:28:343
1295399501,40cyclictest0-21swapper/308:28:343
1295099504,5cyclictest7750irq/56-dwc_otg08:50:261
12949995035,4cyclictest0-21swapper/009:03:530
12949995035,4cyclictest0-21swapper/009:03:530
12949995035,3cyclictest0-21swapper/012:25:260
1294999501,28cyclictest16035-21awk10:15:360
1295399496,28cyclictest7950irq/56-dwc_otg_08:10:343
1295399492,44cyclictest11384-21unixbench_multi11:30:413
1295399492,35cyclictest0-21swapper/308:48:353
1295399491,4cyclictest24627-21/usr/sbin/munin10:40:263
1295399491,45cyclictest29570-21latency_hist09:25:133
1295399491,3cyclictest0-21swapper/308:40:233
1295399491,2cyclictest0-21swapper/309:55:223
1295399491,2cyclictest0-21swapper/309:55:223
1295099496,7cyclictest7950irq/56-dwc_otg_11:20:241
12949994921,16cyclictest0-21swapper/010:45:330
12949994921,16cyclictest0-21swapper/010:45:330
1295399482,42cyclictest8974-21latency11:25:333
1295399482,42cyclictest8974-21latency11:25:333
1295399481,6cyclictest327-21/usr/sbin/munin12:30:373
1295399481,4cyclictest12977-21kworker/u8:108:50:183
1295399481,44cyclictest11395-21/usr/sbin/munin08:35:283
1295399481,43cyclictest2427-21grep12:35:393
1295399481,3cyclictest0-21swapper/311:40:233
1295399481,36cyclictest0-21swapper/310:02:423
1295399481,2cyclictest0-21swapper/310:20:223
12951994831,12cyclictest12948-21cyclictest07:10:142
1294999484,21cyclictest0-21swapper/009:30:350
1295399471,3cyclictest31739-21apt-get11:00:223
1295399471,36cyclictest0-21swapper/309:46:113
1295099477,26cyclictest0-21swapper/108:35:231
12949994723,12cyclictest0-21swapper/011:40:120
1294999471,16cyclictest20551-21cut07:30:320
1295399461,4cyclictest18320-21/usr/sbin/munin11:50:373
1295399461,41cyclictest26666-21aten_rbpower_vo12:15:283
1295399461,41cyclictest26666-21aten_rbpower_vo12:15:283
1295099467,25cyclictest0-21swapper/111:35:261
1295099467,24cyclictest0-21swapper/109:10:251
1295099462,29cyclictest0-21swapper/108:40:251
1295099461,15cyclictest7950irq/56-dwc_otg_12:10:401
12950994611,23cyclictest0-21swapper/107:25:251
12949994624,12cyclictest14973-21taskset10:13:340
12949994620,14cyclictest101ktimersoftd/011:15:420
120772462,7sleep00-21swapper/007:05:410
1295199457,25cyclictest0-21swapper/209:00:252
1295199457,25cyclictest0-21swapper/209:00:252
1295199453,14cyclictest0-21swapper/208:15:302
12951994526,5cyclictest6321-21idleruntime-cro11:20:122
1295099458,24cyclictest0-21swapper/108:00:181
1295099457,26cyclictest0-21swapper/111:25:261
1295099457,26cyclictest0-21swapper/111:25:261
1295099457,24cyclictest0-21swapper/110:35:261
12949994516,20cyclictest0-21swapper/012:38:200
1295199442,4cyclictest0-21swapper/207:45:282
1295099447,27cyclictest0-21swapper/110:30:261
12950994424,5cyclictest15471-21aten_rbpower_vo10:15:281
1295199437,24cyclictest0-21swapper/209:50:262
1295199432,4cyclictest0-21swapper/207:40:242
1295099436,27cyclictest0-21swapper/111:15:271
12949994318,15cyclictest0-21swapper/010:27:280
12949994318,14cyclictest12612-21chrt08:35:470
1294999431,22cyclictest14913-21latency_hist08:45:120
1295099427,27cyclictest0-21swapper/110:00:231
1295199417,21cyclictest0-21swapper/211:05:252
1295199415,7cyclictest7750irq/56-dwc_otg10:25:212
1295099416,24cyclictest0-21swapper/112:15:251
1295099416,24cyclictest0-21swapper/112:15:251
1295099416,23cyclictest0-21swapper/111:30:251
1295099415,29cyclictest7950irq/56-dwc_otg_10:25:211
1295099413,21cyclictest7850irq/56-dwc_otg_07:45:261
12950994119,5cyclictest22850-21sensors07:35:371
12950994119,5cyclictest22850-21sensors07:35:371
12950994115,12cyclictest7950irq/56-dwc_otg_11:05:361
1295099407,22cyclictest0-21swapper/108:20:261
1295099402,23cyclictest0-21swapper/111:40:251
1295099402,22cyclictest0-21swapper/109:15:241
1294999401,22cyclictest3259-21fschecks_time11:10:300
1295199397,23cyclictest0-21swapper/211:45:272
1295199391,26cyclictest20426-21ls07:30:292
1295199378,6cyclictest7850irq/56-dwc_otg_10:40:132
1295199372,11cyclictest0-21swapper/212:20:252
1295199371,6cyclictest7750irq/56-dwc_otg10:20:252
1295099377,17cyclictest7850irq/56-dwc_otg_07:40:231
1295099373,20cyclictest0-21swapper/108:05:251
1295099367,15cyclictest7950irq/56-dwc_otg_11:00:361
1295099366,6cyclictest7950irq/56-dwc_otg_09:00:401
1295099366,6cyclictest7950irq/56-dwc_otg_09:00:401
1295199357,25cyclictest0-21swapper/209:40:252
1295199355,5cyclictest24498-21/usr/sbin/munin10:40:222
12951993517,5cyclictest16761-21latency_hist08:50:122
1295199351,11cyclictest7750irq/56-dwc_otg12:25:252
1295099354,7cyclictest21937-21latency_hist07:35:131
1295099352,24cyclictest0-21swapper/109:25:261
1295099351,23cyclictest0-21swapper/109:05:231
1295199347,15cyclictest0-21swapper/207:55:242
1295199342,16cyclictest7850irq/56-dwc_otg_12:05:122
1295099347,14cyclictest7950irq/56-dwc_otg_07:20:401
1295099346,11cyclictest221ktimersoftd/109:51:411
1295099345,14cyclictest1247-21latency_hist12:35:121
1295099341,8cyclictest8079-21tune2fs08:25:291
1295099341,8cyclictest8079-21tune2fs08:25:291
1295199336,6cyclictest7850irq/56-dwc_otg_11:00:122
1295199336,15cyclictest0-21swapper/208:05:252
1295199335,6cyclictest7850irq/56-dwc_otg_11:00:252
1295199334,10cyclictest28580-21tail07:50:412
1295199332,4cyclictest0-21swapper/209:35:242
1295199332,18cyclictest0-21swapper/209:50:122
1295099335,15cyclictest11-21rcu_preempt12:25:231
1295099334,21cyclictest7850irq/56-dwc_otg_10:10:391
1295099333,3cyclictest7750irq/56-dwc_otg08:10:221
1295099332,8cyclictest0-21swapper/111:10:261
12950993317,7cyclictest7750irq/56-dwc_otg11:55:121
118472330,2sleep10-21swapper/107:05:361
1295199327,16cyclictest0-21swapper/207:35:242
1295199327,16cyclictest0-21swapper/207:35:242
1295199325,6cyclictest7850irq/56-dwc_otg_09:05:372
1295199324,14cyclictest7950irq/56-dwc_otg_12:15:272
1295199324,14cyclictest7950irq/56-dwc_otg_12:15:272
1295199322,6cyclictest7950irq/56-dwc_otg_11:40:292
12951993222,7cyclictest16367-21sh07:20:122
1295199321,6cyclictest7750irq/56-dwc_otg08:55:422
12951993215,5cyclictest20362-21unixbench-2d11:55:412
1295199321,3cyclictest7950irq/56-dwc_otg_11:25:412
1295199321,3cyclictest7950irq/56-dwc_otg_11:25:412
1295199321,22cyclictest24344-21cpuspeed09:10:262
1295099328,5cyclictest7850irq/56-dwc_otg_12:05:221
1295099321,6cyclictest7750irq/56-dwc_otg10:50:301
1295099321,4cyclictest7750irq/56-dwc_otg09:40:291
12950993214,5cyclictest1125-21runrttasks07:57:221
1295099321,20cyclictest26913-21missed_timers10:45:331
1295099321,20cyclictest26913-21missed_timers10:45:331
119732328,11sleep30-21swapper/307:05:393
1295199316,17cyclictest301ktimersoftd/211:20:132
1295199316,13cyclictest0-21swapper/212:10:262
1295199314,18cyclictest7950irq/56-dwc_otg_09:15:262
1295199312,17cyclictest0-21swapper/210:49:332
1295199312,17cyclictest0-21swapper/210:49:332
1295199311,12cyclictest0-21swapper/211:30:232
1295099316,14cyclictest7950irq/56-dwc_otg_09:45:241
1295099314,3cyclictest7750irq/56-dwc_otg12:00:231
1295099311,26cyclictest28883-21latency12:20:321
1295199305,7cyclictest7850irq/56-dwc_otg_08:45:122
1295199305,5cyclictest7850irq/56-dwc_otg_08:35:302
1295199305,5cyclictest31-21ksoftirqd/210:00:282
1295199302,18cyclictest7950irq/56-dwc_otg_11:35:342
1295199302,14cyclictest7850irq/56-dwc_otg_08:23:172
1295199301,25cyclictest24504-21latency_hist12:10:132
1295199301,19cyclictest18474-21wget07:25:262
1295199301,11cyclictest7750irq/56-dwc_otg10:20:132
1295099309,4cyclictest7750irq/56-dwc_otg08:55:311
12950993024,4cyclictest9683-21idleruntime-cro10:00:121
12950993024,4cyclictest9683-21idleruntime-cro10:00:121
1295099302,13cyclictest0-21swapper/108:45:361
1295099301,4cyclictest30487-21/usr/sbin/munin10:55:321
1295199293,5cyclictest7850irq/56-dwc_otg_08:50:182
1295199292,17cyclictest0-21swapper/212:35:312
1295199291,5cyclictest2938-21aten_rbpower_cu11:10:262
1295199291,16cyclictest0-21swapper/208:13:282
12951992910,10cyclictest7950irq/56-dwc_otg_10:05:342
1295099294,7cyclictest7850irq/56-dwc_otg_09:20:341
1295099292,7cyclictest7750irq/56-dwc_otg08:15:281
1295099291,7cyclictest32733-21taskset09:30:421
1295199289,5cyclictest0-21swapper/212:30:302
1295199282,16cyclictest7950irq/56-dwc_otg_10:50:412
1295199281,5cyclictest27809-21acpi09:20:222
1295199275,11cyclictest7950irq/56-dwc_otg_09:55:332
1295199275,11cyclictest7950irq/56-dwc_otg_09:55:332
1295199272,4cyclictest7850irq/56-dwc_otg_10:10:282
1295199272,3cyclictest7750irq/56-dwc_otg07:20:302
1295199271,6cyclictest21426-21/usr/sbin/munin10:30:342
111432274,14sleep211209-21/usr/sbin/munin07:05:262
1295199264,5cyclictest7850irq/56-dwc_otg_08:25:382
1295199264,5cyclictest7850irq/56-dwc_otg_08:25:382
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional