You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-30 - 20:37

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Mon Mar 30, 2026 12:45:06)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
152699659,17cyclictest131rcu_preempt08:05:132
152699627,9cyclictest27-21ksoftirqd/210:45:142
152699601,7cyclictest131rcu_preempt07:45:132
152699572,20cyclictest131rcu_preempt08:30:122
152699561,8cyclictest131rcu_preempt08:45:132
152699542,30cyclictest16867-21ssh12:25:142
152699511,41cyclictest13601-21ssh10:00:122
150999472,42cyclictest18508-21ssh11:50:160
153599459,27cyclictest28366irq/58-eth008:30:143
153599451,26cyclictest28366irq/58-eth008:20:123
152699443,21cyclictest14888-21ssh09:25:152
1526994411,8cyclictest131rcu_preempt08:25:112
153599436,23cyclictest28366irq/58-eth008:10:133
151799431,33cyclictest4510-21ssh09:55:151
152699427,28cyclictest27-21ksoftirqd/211:45:162
153599413,26cyclictest28366irq/58-eth010:30:123
152699407,24cyclictest27-21ksoftirqd/210:50:132
152699392,23cyclictest3208-21ssh10:30:132
151799392,24cyclictest22709-21ssh10:05:131
153599383,26cyclictest28366irq/58-eth010:10:133
152699381,30cyclictest21344-21ssh10:40:112
151799380,27cyclictest0-21swapper/107:10:101
152699366,17cyclictest27-21ksoftirqd/207:55:152
151799362,30cyclictest27555-21ssh11:55:161
151799362,30cyclictest27555-21ssh11:55:151
150999365,3cyclictest131rcu_preempt09:35:110
153599344,20cyclictest28366irq/58-eth011:05:133
152699342,20cyclictest22346irq/40-dwc2_hso11:25:132
1526993313,14cyclictest27-21ksoftirqd/211:30:112
152699331,13cyclictest19646-21diskmemload09:10:132
152699330,30cyclictest924-21snmpd10:15:132
150999334,5cyclictest12-21ksoftirqd/010:55:290
152699321,6cyclictest131rcu_preempt12:00:132
1509993219,9cyclictest11279-21apt-get07:20:140
153599312,19cyclictest28366irq/58-eth009:15:123
153599312,17cyclictest28366irq/58-eth011:00:143
153599311,20cyclictest28366irq/58-eth010:50:143
1526993116,9cyclictest27-21ksoftirqd/210:35:152
1526993116,9cyclictest27-21ksoftirqd/210:35:142
1526993110,6cyclictest0-21swapper/208:00:162
151799314,24cyclictest23927-21ssh09:30:121
151799313,21cyclictest28920-21ssh11:20:111
1526993021,6cyclictest27-21ksoftirqd/208:55:132
151799301,22cyclictest10815-21sh11:10:131
153599293,10cyclictest28366irq/58-eth011:40:133
1526992913,10cyclictest27-21ksoftirqd/209:45:152
1517992923,2cyclictest17348-21apt-get08:00:131
1517992912,4cyclictest21-21ksoftirqd/110:15:121
150999299,5cyclictest12-21ksoftirqd/009:20:190
150999294,4cyclictest12-21ksoftirqd/010:08:140
1535992819,7cyclictest0-21swapper/311:35:133
152699282,9cyclictest27-21ksoftirqd/211:50:172
152699282,18cyclictest22346irq/40-dwc2_hso11:20:122
152699282,17cyclictest22346irq/40-dwc2_hso09:05:132
152699282,10cyclictest27-21ksoftirqd/208:15:142
151799281,20cyclictest26614-21ssh10:25:151
150999288,6cyclictest12-21ksoftirqd/010:45:230
150999288,3cyclictest131rcu_preempt10:40:130
150999286,5cyclictest12-21ksoftirqd/012:00:250
1509992811,8cyclictest131rcu_preempt08:00:140
153599274,10cyclictest28366irq/58-eth009:20:133
153599273,13cyclictest28366irq/58-eth011:30:123
152699279,4cyclictest131rcu_preempt07:20:182
152699273,15cyclictest22346irq/40-dwc2_hso10:55:132
151799277,3cyclictest131rcu_preempt10:50:281
151799276,8cyclictest131rcu_preempt09:27:211
151799274,5cyclictest21-21ksoftirqd/111:25:221
151799274,3cyclictest131rcu_preempt12:28:261
151799271,20cyclictest30396-21ssh10:45:131
1517992711,5cyclictest21-21ksoftirqd/110:00:081
150999279,3cyclictest131rcu_preempt12:25:250
150999278,7cyclictest131rcu_preempt12:20:180
150999278,3cyclictest131rcu_preempt11:20:250
150999273,7cyclictest12-21ksoftirqd/011:05:290
150999273,6cyclictest12-21ksoftirqd/009:45:310
150999272,4cyclictest12-21ksoftirqd/010:00:230
150999271,8cyclictest12185-21kthreadcore11:10:210
150999271,6cyclictest12-21ksoftirqd/011:05:020
153599264,15cyclictest0-21swapper/312:10:123
1535992618,5cyclictest28366irq/58-eth012:30:153
152699263,16cyclictest22346irq/40-dwc2_hso08:20:132
152699262,19cyclictest0-21swapper/207:35:142
152699261,17cyclictest14657-21ls09:25:012
151799268,9cyclictest21-21ksoftirqd/109:51:311
151799266,4cyclictest21-21ksoftirqd/111:44:571
151799265,5cyclictest21-21ksoftirqd/112:10:141
151799265,4cyclictest21-21ksoftirqd/110:56:131
151799264,8cyclictest131rcu_preempt09:41:081
151799264,4cyclictest21-21ksoftirqd/109:15:041
151799264,11cyclictest21-21ksoftirqd/108:50:201
151799263,4cyclictest21-21ksoftirqd/111:45:151
1517992612,11cyclictest4269-21apt-get08:20:131
151799261,18cyclictest31240-21ssh12:15:141
1517992610,10cyclictest0-21swapper/111:30:361
1517992610,10cyclictest0-21swapper/109:49:031
150999268,3cyclictest131rcu_preempt11:25:230
150999266,9cyclictest12-21ksoftirqd/010:30:100
150999266,5cyclictest12-21ksoftirqd/009:30:220
150999264,9cyclictest12-21ksoftirqd/009:12:160
150999263,9cyclictest12-21ksoftirqd/012:05:350
150999263,6cyclictest12-21ksoftirqd/008:55:130
150999263,5cyclictest12-21ksoftirqd/011:45:220
150999263,4cyclictest12-21ksoftirqd/010:00:010
150999263,15cyclictest25884-21ssh12:30:130
150999262,6cyclictest12-21ksoftirqd/011:55:140
150999262,6cyclictest12-21ksoftirqd/011:55:130
150999262,5cyclictest12-21ksoftirqd/012:35:250
150999262,4cyclictest12-21ksoftirqd/009:50:140
150999261,7cyclictest12-21ksoftirqd/010:25:260
1509992614,6cyclictest0-21swapper/007:10:110
1509992613,8cyclictest12-21ksoftirqd/009:25:130
1509992611,7cyclictest0-21swapper/010:54:270
1509992611,3cyclictest131rcu_preempt11:30:140
1509992611,3cyclictest0-21swapper/009:41:420
1509992610,5cyclictest12-21ksoftirqd/011:35:220
152699259,10cyclictest29204-21ssh09:15:142
152699257,15cyclictest28052-21cron08:45:012
152699254,14cyclictest22346irq/40-dwc2_hso12:10:152
152699253,16cyclictest1918-21ssh11:05:152
152699253,15cyclictest22346irq/40-dwc2_hso10:20:122
152699253,14cyclictest22346irq/40-dwc2_hso11:40:132
151799259,7cyclictest131rcu_preempt09:20:081
151799257,9cyclictest21-21ksoftirqd/111:05:041
151799253,5cyclictest21-21ksoftirqd/107:55:181
151799253,5cyclictest21-21ksoftirqd/107:40:191
151799253,4cyclictest21-21ksoftirqd/110:30:021
1517992512,7cyclictest0-21swapper/109:35:421
151799251,18cyclictest13204-21ssh12:05:171
150999258,3cyclictest131rcu_preempt08:25:240
150999257,3cyclictest131rcu_preempt10:20:130
150999256,7cyclictest12-21ksoftirqd/007:39:530
150999255,6cyclictest12-21ksoftirqd/012:15:300
150999255,5cyclictest12-21ksoftirqd/009:05:220
150999255,5cyclictest12-21ksoftirqd/009:00:150
150999254,8cyclictest12-21ksoftirqd/007:15:220
150999254,4cyclictest12-21ksoftirqd/010:10:200
150999253,5cyclictest12-21ksoftirqd/012:10:250
150999253,4cyclictest12-21ksoftirqd/011:17:480
150999252,5cyclictest12-21ksoftirqd/008:40:240
150999252,4cyclictest12-21ksoftirqd/008:05:270
150999251,6cyclictest12-21ksoftirqd/009:17:130
1509992511,3cyclictest131rcu_preempt08:15:150
152699243,8cyclictest22718-21ssh10:05:132
152699242,16cyclictest22346irq/40-dwc2_hso09:00:112
1526992412,9cyclictest0-21swapper/212:37:252
152699241,17cyclictest27828-21ssh09:50:122
152699241,13cyclictest131rcu_preempt10:10:172
151799247,3cyclictest131rcu_preempt10:35:151
151799247,3cyclictest131rcu_preempt10:35:141
151799246,6cyclictest21-21ksoftirqd/109:10:201
151799246,5cyclictest21-21ksoftirqd/111:35:551
151799246,3cyclictest131rcu_preempt11:50:431
151799246,3cyclictest131rcu_preempt11:00:141
151799246,3cyclictest131rcu_preempt08:35:151
151799245,8cyclictest21-21ksoftirqd/112:38:081
151799245,8cyclictest21-21ksoftirqd/107:15:191
151799245,3cyclictest131rcu_preempt09:05:301
151799245,3cyclictest131rcu_preempt08:15:151
151799244,4cyclictest21-21ksoftirqd/112:00:301
151799243,7cyclictest21-21ksoftirqd/112:30:161
151799243,5cyclictest21-21ksoftirqd/110:20:281
151799243,5cyclictest21-21ksoftirqd/108:25:241
1517992411,7cyclictest21-21ksoftirqd/110:41:461
1517992410,7cyclictest21-21ksoftirqd/111:16:261
150999247,2cyclictest131rcu_preempt07:50:250
150999244,5cyclictest151rcuc/008:50:180
150999243,4cyclictest12-21ksoftirqd/011:40:160
150999241,5cyclictest12-21ksoftirqd/010:35:150
150999241,5cyclictest12-21ksoftirqd/010:35:140
150999240,7cyclictest8605-21ssh10:15:160
153599235,12cyclictest28366irq/58-eth009:10:153
153599234,13cyclictest0-21swapper/310:45:133
152699236,8cyclictest2289-21latency12:15:222
152699235,8cyclictest131rcu_preempt09:58:072
152699232,6cyclictest131rcu_preempt07:30:252
1526992313,5cyclictest27-21ksoftirqd/208:51:222
151799235,2cyclictest131rcu_preempt08:10:161
151799234,4cyclictest21-21ksoftirqd/109:00:201
151799233,7cyclictest21-21ksoftirqd/112:20:131
151799233,5cyclictest21-21ksoftirqd/107:25:121
151799232,4cyclictest21-21ksoftirqd/110:10:221
1517992315,4cyclictest2905-21apt-get07:45:121
151799230,7cyclictest21-21ksoftirqd/107:35:281
150999235,3cyclictest131rcu_preempt08:10:220
150999234,4cyclictest12-21ksoftirqd/008:45:190
150999234,4cyclictest12-21ksoftirqd/007:40:210
150999233,4cyclictest12-21ksoftirqd/008:35:200
150999233,4cyclictest12-21ksoftirqd/007:55:180
150999233,4cyclictest12-21ksoftirqd/007:25:160
1535992211,8cyclictest28366irq/58-eth010:15:143
1535992210,9cyclictest0-21swapper/312:35:213
1535992210,9cyclictest0-21swapper/310:59:173
152699227,12cyclictest30180-21kthreadcore08:10:222
152699223,16cyclictest16283-21sed07:25:152
152699221,18cyclictest10061-21ps09:40:162
152699221,15cyclictest30642-21grep07:40:152
1526992210,9cyclictest0-21swapper/211:00:292
152699220,14cyclictest27709-21processes09:30:262
151799224,3cyclictest131rcu_preempt08:55:181
151799223,4cyclictest21-21ksoftirqd/108:05:271
151799222,4cyclictest21-21ksoftirqd/108:30:281
151799222,4cyclictest21-21ksoftirqd/107:30:141
150999223,5cyclictest12-21ksoftirqd/008:30:170
150999222,4cyclictest12-21ksoftirqd/007:30:130
153599219,9cyclictest0-21swapper/312:23:463
153599213,12cyclictest0-21swapper/311:20:133
153599211,14cyclictest0-21swapper/310:00:133
1535992111,4cyclictest0-21swapper/308:35:123
152699217,11cyclictest26706-21df_inode10:25:152
152699214,12cyclictest4275-21ssh09:35:252
152699213,14cyclictest14421-21kthreadcore12:05:222
152699212,10cyclictest3190-21kthreadcore07:10:202
1526992115,3cyclictest20500-21ssh11:15:212
152699211,17cyclictest29245-21kthreadcore12:30:232
152699211,12cyclictest31627-21sed11:55:322
152699211,12cyclictest31627-21sed11:55:312
152699211,11cyclictest23917-21fschecks_time11:35:172
152699211,10cyclictest7845-21fschecks_count07:50:162
152699210,4cyclictest131rcu_preempt11:10:312
151799216,3cyclictest131rcu_preempt08:40:271
151799213,4cyclictest21-21ksoftirqd/108:45:201
151799212,16cyclictest11558-21grep07:50:301
150999213,4cyclictest12-21ksoftirqd/008:20:280
150999212,5cyclictest12-21ksoftirqd/007:45:230
153599209,9cyclictest0-21swapper/309:50:203
153599209,4cyclictest0-21swapper/312:25:223
1535992013,5cyclictest0-21swapper/312:04:463
1535992012,5cyclictest0-21swapper/311:10:143
1535992011,6cyclictest28366irq/58-eth009:30:123
1535992010,8cyclictest0-21swapper/311:47:163
1535992010,8cyclictest0-21swapper/310:25:283
152699201,9cyclictest9650-21kthreadcore12:20:212
152699201,7cyclictest131rcu_preempt07:15:162
152699201,10cyclictest18746-21tune2fs08:35:162
151799202,4cyclictest21-21ksoftirqd/107:20:261
153599198,9cyclictest0-21swapper/309:45:333
153599197,9cyclictest0-21swapper/312:05:143
153599192,9cyclictest28366irq/58-eth010:20:103
1535991912,4cyclictest0-21swapper/312:15:203
1535991911,6cyclictest28366irq/58-eth009:25:143
1535991910,7cyclictest0-21swapper/311:15:493
1535991910,7cyclictest0-21swapper/310:39:493
1535991910,7cyclictest0-21swapper/310:39:493
1535991910,6cyclictest0-21swapper/311:51:033
152699191,15cyclictest1595-21latency_hist07:10:012
153599189,6cyclictest0-21swapper/310:40:313
153599189,6cyclictest0-21swapper/309:35:103
153599182,10cyclictest0-21swapper/308:50:143
153599182,10cyclictest0-21swapper/307:35:143
153599181,12cyclictest0-21swapper/310:05:163
1535991810,6cyclictest0-21swapper/309:40:053
153599179,6cyclictest0-21swapper/311:25:243
153599172,9cyclictest0-21swapper/308:40:133
153599172,9cyclictest0-21swapper/307:55:143
153599172,10cyclictest0-21swapper/309:05:133
153599171,14cyclictest0-21swapper/309:55:173
153599171,10cyclictest0-21swapper/311:55:313
153599171,10cyclictest0-21swapper/311:55:303
150999173,4cyclictest12-21ksoftirqd/007:10:010
153599169,4cyclictest28366irq/58-eth008:25:113
153599154,7cyclictest28366irq/58-eth007:30:123
151799151,10cyclictest1574-21latency_hist07:10:021
153599149,3cyclictest28466irq/59-eth007:45:133
153599149,3cyclictest0-21swapper/307:15:313
153599138,3cyclictest0-21swapper/307:10:193
153599137,4cyclictest33-21ksoftirqd/308:45:253
153599136,5cyclictest33-21ksoftirqd/307:20:293
153599133,8cyclictest0-21swapper/309:00:143
153599133,8cyclictest0-21swapper/308:00:153
153599133,7cyclictest0-21swapper/308:15:143
153599133,7cyclictest0-21swapper/308:05:123
153599132,9cyclictest0-21swapper/307:40:133
153599127,3cyclictest0-21swapper/307:27:433
153599124,5cyclictest33-21ksoftirqd/308:55:133
153599121,9cyclictest0-21swapper/307:50:183
153599101,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional