You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-11 - 17:43

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Apr 11, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
132999695,48cyclictest2295-21ssh11:05:141
132999646,7cyclictest21-21ksoftirqd/108:55:121
133999582,34cyclictest22917-21ssh12:10:132
133999508,7cyclictest271ksoftirqd/211:15:122
1339994811,28cyclictest27-21ksoftirqd/208:45:122
132999460,36cyclictest21910-21ssh10:40:131
133999391,27cyclictest17847-21ssh12:25:142
1350993826,5cyclictest28366irq/58-eth009:20:133
1339993828,8cyclictest0-21swapper/210:05:112
133999380,27cyclictest13974-21ssh10:00:132
132999371,28cyclictest24204-21ssh11:35:111
132999361,26cyclictest20404-21ssh11:15:141
1339993511,17cyclictest0-21swapper/211:30:202
132999351,26cyclictest32073-21ssh10:10:131
132999347,5cyclictest131rcu_preempt07:35:121
132999341,26cyclictest8820-21ssh12:20:161
1315993418,11cyclictest28168-21apt-get11:55:130
1339993311,15cyclictest0-21swapper/210:44:402
132999330,19cyclictest12755-21ssh10:35:131
133999327,16cyclictest0-21swapper/211:10:342
1339993216,14cyclictest5866-21ssh10:30:222
1339993213,13cyclictest0-21swapper/209:55:142
1339993210,12cyclictest27-21ksoftirqd/211:50:302
132999326,4cyclictest131rcu_preempt08:50:141
1329993212,3cyclictest0-21swapper/108:20:141
133999313,21cyclictest0-21swapper/207:20:142
1339993113,5cyclictest27-21ksoftirqd/210:10:182
1339993110,15cyclictest0-21swapper/212:09:062
1339993110,15cyclictest0-21swapper/212:03:402
1339993110,15cyclictest0-21swapper/209:46:242
131599317,6cyclictest12-21ksoftirqd/010:00:120
1315993118,7cyclictest3026-21apt-get07:45:140
133999309,15cyclictest0-21swapper/211:58:582
133999309,15cyclictest0-21swapper/209:33:092
1339993013,11cyclictest0-21swapper/210:46:582
133999301,22cyclictest8791-21ssh10:15:142
132999308,13cyclictest21-21ksoftirqd/110:15:131
131599304,6cyclictest12-21ksoftirqd/012:35:190
131599303,22cyclictest4955-21sh09:55:140
133999298,15cyclictest0-21swapper/212:33:182
133999293,17cyclictest22346irq/40-dwc2_hso09:05:122
133999292,18cyclictest22346irq/40-dwc2_hso07:55:132
1339992912,11cyclictest0-21swapper/209:20:282
133999291,15cyclictest27-21ksoftirqd/211:05:212
1339992910,14cyclictest0-21swapper/210:58:582
1329992912,8cyclictest21-21ksoftirqd/111:30:021
1329992911,9cyclictest21-21ksoftirqd/111:50:011
131599291,8cyclictest13923-21awk11:45:290
133999289,15cyclictest27-21ksoftirqd/209:40:522
133999288,14cyclictest27-21ksoftirqd/209:25:262
133999286,16cyclictest27-21ksoftirqd/212:36:142
133999286,16cyclictest27-21ksoftirqd/211:01:532
133999282,19cyclictest0-21swapper/208:30:142
1339992819,5cyclictest261rcuc/207:50:142
133999280,19cyclictest22346irq/40-dwc2_hso08:20:142
132999288,5cyclictest21-21ksoftirqd/112:30:481
132999282,19cyclictest4921-21ssh12:16:031
132999282,19cyclictest4921-21ssh12:16:031
1329992813,7cyclictest21-21ksoftirqd/111:20:171
132999281,22cyclictest15219-21ssh11:30:151
1329992811,4cyclictest21-21ksoftirqd/110:35:001
1329992810,8cyclictest21-21ksoftirqd/108:35:271
131599287,5cyclictest12-21ksoftirqd/011:00:320
131599285,5cyclictest12-21ksoftirqd/011:25:300
131599284,9cyclictest12294-21basename10:35:010
131599284,4cyclictest12-21ksoftirqd/010:15:230
131599282,4cyclictest12-21ksoftirqd/009:25:260
1315992820,6cyclictest31996-21ssh12:15:130
1315992820,6cyclictest31996-21ssh12:15:120
1315992811,5cyclictest0-21swapper/009:22:190
133999275,13cyclictest27-21ksoftirqd/210:35:252
1339992712,8cyclictest0-21swapper/209:18:012
133999271,15cyclictest27-21ksoftirqd/210:55:022
132999279,8cyclictest21-21ksoftirqd/107:40:281
132999278,9cyclictest13964-21apt-get10:00:131
132999278,8cyclictest21-21ksoftirqd/108:30:301
1329992711,7cyclictest0-21swapper/112:13:231
1329992710,9cyclictest21-21ksoftirqd/108:15:011
1329992710,8cyclictest0-21swapper/109:00:151
131599278,3cyclictest131rcu_preempt12:03:580
131599276,8cyclictest12-21ksoftirqd/010:25:240
131599276,8cyclictest12-21ksoftirqd/010:25:230
131599275,5cyclictest12-21ksoftirqd/012:10:260
131599275,5cyclictest12-21ksoftirqd/009:40:240
131599274,5cyclictest12-21ksoftirqd/011:35:290
131599274,5cyclictest12-21ksoftirqd/010:40:250
131599274,5cyclictest12-21ksoftirqd/009:50:240
131599274,5cyclictest11456-21ssh12:20:230
131599274,4cyclictest12-21ksoftirqd/008:15:010
131599274,4cyclictest12-21ksoftirqd/007:55:010
131599273,6cyclictest12-21ksoftirqd/011:50:290
1315992721,3cyclictest0-21swapper/010:45:140
131599271,9cyclictest12-21ksoftirqd/012:06:110
131599271,7cyclictest9246-21cat08:25:010
131599271,7cyclictest12-21ksoftirqd/009:45:160
1315992710,11cyclictest22956-21apt-get10:05:140
131599270,9cyclictest5621-21ssh10:11:470
133999268,14cyclictest27-21ksoftirqd/211:40:122
133999267,4cyclictest27-21ksoftirqd/210:20:282
133999267,4cyclictest27-21ksoftirqd/208:05:192
133999267,14cyclictest0-21swapper/209:54:052
133999267,11cyclictest131rcu_preempt11:20:302
133999265,6cyclictest131rcu_preempt12:20:282
1339992617,4cyclictest27-21ksoftirqd/209:12:532
1339992617,4cyclictest27-21ksoftirqd/209:12:522
133999261,16cyclictest24180-21find11:35:112
133999260,15cyclictest1056-21find08:50:122
132999269,8cyclictest21-21ksoftirqd/109:25:321
132999267,3cyclictest131rcu_preempt11:43:561
132999266,8cyclictest21-21ksoftirqd/111:50:561
132999266,4cyclictest21-21ksoftirqd/112:05:281
132999262,19cyclictest11081-21ssh12:38:511
1329992611,9cyclictest0-21swapper/109:30:461
1329992611,5cyclictest21-21ksoftirqd/108:25:181
131599268,6cyclictest12-21ksoftirqd/010:20:130
131599266,6cyclictest12-21ksoftirqd/011:05:250
131599264,5cyclictest12-21ksoftirqd/011:10:290
131599264,5cyclictest12-21ksoftirqd/008:15:250
131599264,5cyclictest12-21ksoftirqd/008:15:250
131599264,4cyclictest12-21ksoftirqd/011:40:270
131599264,4cyclictest12-21ksoftirqd/008:55:010
131599264,4cyclictest12-21ksoftirqd/008:30:290
131599263,6cyclictest12-21ksoftirqd/012:31:240
131599263,4cyclictest12-21ksoftirqd/008:25:230
131599262,8cyclictest12-21ksoftirqd/009:30:240
131599262,20cyclictest20189-21diskmemload10:55:120
131599261,7cyclictest23829-21latency_hist08:40:010
131599261,7cyclictest19752-21kthreadcore08:00:220
131599261,6cyclictest12-21ksoftirqd/011:20:020
131599260,6cyclictest20699-21kthreadcore12:25:220
133999258,11cyclictest27-21ksoftirqd/209:05:012
133999258,10cyclictest131rcu_preempt09:00:022
133999257,12cyclictest0-21swapper/211:28:022
133999255,4cyclictest27-21ksoftirqd/209:35:192
133999255,12cyclictest27-21ksoftirqd/212:17:322
133999255,12cyclictest27-21ksoftirqd/212:17:322
133999251,8cyclictest27-21ksoftirqd/208:00:222
1339992510,6cyclictest27-21ksoftirqd/211:47:082
132999259,8cyclictest21-21ksoftirqd/112:04:511
132999259,8cyclictest21-21ksoftirqd/109:45:191
132999257,9cyclictest21-21ksoftirqd/109:16:241
132999257,9cyclictest21-21ksoftirqd/107:10:171
132999257,4cyclictest21-21ksoftirqd/110:22:241
132999254,4cyclictest131rcu_preempt09:10:191
132999254,4cyclictest131rcu_preempt09:10:181
132999254,3cyclictest131rcu_preempt09:20:131
132999252,5cyclictest21-21ksoftirqd/110:45:271
1329992512,4cyclictest21-21ksoftirqd/109:50:221
1329992511,4cyclictest21-21ksoftirqd/109:55:151
1329992510,9cyclictest21-21ksoftirqd/108:40:271
1329992510,9cyclictest21-21ksoftirqd/108:40:271
1329992510,6cyclictest131rcu_preempt12:28:211
1329992510,6cyclictest0-21swapper/110:07:281
131599255,3cyclictest131rcu_preempt09:35:180
131599254,6cyclictest12-21ksoftirqd/008:55:280
131599254,5cyclictest12-21ksoftirqd/010:52:380
131599253,7cyclictest12-21ksoftirqd/007:35:130
131599253,6cyclictest12-21ksoftirqd/011:30:140
131599253,10cyclictest12-21ksoftirqd/009:00:290
131599251,7cyclictest30531-21cut09:15:190
131599251,5cyclictest12-21ksoftirqd/011:15:130
131599250,7cyclictest12272-21apt-get10:35:030
133999249,13cyclictest27-21ksoftirqd/210:27:182
133999249,13cyclictest27-21ksoftirqd/210:27:172
133999249,12cyclictest27-21ksoftirqd/207:45:262
133999248,7cyclictest27-21ksoftirqd/207:45:012
133999244,8cyclictest131rcu_preempt08:15:192
133999244,8cyclictest131rcu_preempt08:15:182
132999249,8cyclictest21-21ksoftirqd/111:55:271
132999249,8cyclictest21-21ksoftirqd/110:55:221
132999249,3cyclictest2347-21kthreadcore09:35:191
132999248,8cyclictest21-21ksoftirqd/111:10:111
132999248,8cyclictest21-21ksoftirqd/107:55:021
132999241,14cyclictest14841-21cat09:40:321
131599246,6cyclictest12-21ksoftirqd/007:55:180
131599245,10cyclictest131rcu_preempt08:40:190
131599245,10cyclictest131rcu_preempt08:40:180
131599244,5cyclictest12-21ksoftirqd/008:05:150
131599243,7cyclictest12-21ksoftirqd/009:05:190
131599243,6cyclictest12-21ksoftirqd/008:45:130
131599243,5cyclictest12-21ksoftirqd/007:20:220
131599242,7cyclictest12-21ksoftirqd/007:15:240
131599242,5cyclictest12-21ksoftirqd/009:10:270
131599242,5cyclictest12-21ksoftirqd/009:10:260
131599241,7cyclictest12-21ksoftirqd/007:25:300
131599241,7cyclictest12-21ksoftirqd/007:10:170
1350992318,3cyclictest321rcuc/308:30:133
1350992318,3cyclictest321rcuc/308:10:123
1350992311,9cyclictest0-21swapper/310:20:283
133999236,3cyclictest131rcu_preempt08:10:212
133999234,3cyclictest2418-21kthreadcore07:10:192
133999233,4cyclictest9610-21irqcore08:25:172
133999233,11cyclictest27-21ksoftirqd/207:15:292
133999231,15cyclictest25622-21/usr/sbin/munin07:35:122
132999239,3cyclictest131rcu_preempt07:45:201
132999238,4cyclictest21-21ksoftirqd/110:25:231
132999238,4cyclictest21-21ksoftirqd/110:25:221
132999237,8cyclictest131rcu_preempt07:30:271
132999237,6cyclictest131rcu_preempt09:10:011
132999234,5cyclictest21-21ksoftirqd/111:02:351
132999232,13cyclictest20189-21diskmemload10:50:121
131599233,6cyclictest12-21ksoftirqd/007:40:120
131599233,5cyclictest12-21ksoftirqd/007:30:210
1350992217,3cyclictest321rcuc/311:30:383
1350992217,3cyclictest321rcuc/311:26:373
1350992216,4cyclictest321rcuc/309:54:343
1350992213,5cyclictest0-21swapper/310:15:133
1350992212,7cyclictest0-21swapper/310:37:553
1350992212,6cyclictest28366irq/58-eth012:00:133
133999228,7cyclictest27-21ksoftirqd/207:25:152
133999227,12cyclictest0-21swapper/208:35:302
132999223,7cyclictest131rcu_preempt07:25:051
132999221,7cyclictest15441-21kthreadcore07:55:221
1329992210,4cyclictest21-21ksoftirqd/108:00:181
132999220,7cyclictest25456-21kthreadcore08:05:221
1350992116,3cyclictest321rcuc/310:29:433
1350992116,3cyclictest321rcuc/310:29:433
1350992116,3cyclictest321rcuc/310:12:273
1350992116,3cyclictest321rcuc/307:30:183
1350992115,4cyclictest28366irq/58-eth009:35:143
1350992115,4cyclictest0-21swapper/309:15:153
1350992114,5cyclictest0-21swapper/311:37:433
1350992114,5cyclictest0-21swapper/310:41:093
1350992114,5cyclictest0-21swapper/309:44:443
1350992112,7cyclictest0-21swapper/311:13:313
1350992112,7cyclictest0-21swapper/311:08:133
1350992111,7cyclictest0-21swapper/312:05:323
1350992111,7cyclictest0-21swapper/311:00:473
1350992111,5cyclictest28366irq/58-eth011:45:153
135099211,15cyclictest0-21swapper/312:10:153
133999211,8cyclictest27-21ksoftirqd/208:45:022
133999211,8cyclictest27-21ksoftirqd/208:45:012
132999219,3cyclictest131rcu_preempt08:45:251
132999217,5cyclictest21-21ksoftirqd/107:15:141
1329992111,3cyclictest21-21ksoftirqd/108:15:151
1329992111,3cyclictest21-21ksoftirqd/108:15:141
135099209,8cyclictest28366irq/58-eth009:25:143
135099208,4cyclictest0-21swapper/310:00:143
1350992014,4cyclictest321rcuc/310:32:513
1350992014,4cyclictest321rcuc/308:45:293
1350992014,4cyclictest0-21swapper/312:36:363
1350992014,4cyclictest0-21swapper/311:50:193
1350992014,4cyclictest0-21swapper/307:25:253
1350992013,5cyclictest0-21swapper/312:30:323
1350992013,5cyclictest0-21swapper/311:20:143
1350992013,5cyclictest0-21swapper/311:15:133
1350992013,4cyclictest0-21swapper/310:06:263
1350992011,6cyclictest28366irq/58-eth010:55:133
133999200,5cyclictest24491-21sed07:30:282
132999208,3cyclictest131rcu_preempt07:20:181
135099196,10cyclictest28366irq/58-eth008:35:133
135099193,6cyclictest28366irq/58-eth012:25:123
1350991914,3cyclictest321rcuc/309:05:013
1350991914,3cyclictest321rcuc/308:57:503
1350991914,3cyclictest321rcuc/307:25:003
1350991913,4cyclictest321rcuc/307:15:143
1350991913,4cyclictest0-21swapper/312:20:193
1350991913,4cyclictest0-21swapper/312:15:033
1350991913,4cyclictest0-21swapper/312:15:023
1350991913,4cyclictest0-21swapper/311:55:123
1350991913,4cyclictest0-21swapper/311:40:243
1350991913,4cyclictest0-21swapper/310:51:043
1350991913,4cyclictest0-21swapper/310:45:413
1350991913,4cyclictest0-21swapper/309:55:353
1350991913,4cyclictest0-21swapper/309:47:563
1350991913,4cyclictest0-21swapper/309:30:053
1350991913,4cyclictest0-21swapper/309:11:043
1350991913,4cyclictest0-21swapper/309:11:043
133999199,8cyclictest0-21swapper/207:10:012
132999191,3cyclictest0-21swapper/107:10:021
1350991813,3cyclictest0-21swapper/307:50:263
1350991813,3cyclictest0-21swapper/307:10:183
1350991812,4cyclictest0-21swapper/307:45:243
1350991712,3cyclictest0-21swapper/309:05:173
1350991711,4cyclictest0-21swapper/308:20:213
1350991711,4cyclictest0-21swapper/307:40:153
135099171,12cyclictest0-21swapper/308:00:173
1350991611,3cyclictest0-21swapper/307:55:243
135099161,12cyclictest0-21swapper/308:50:163
135099161,12cyclictest0-21swapper/307:35:173
135099151,11cyclictest0-21swapper/308:15:303
135099151,11cyclictest0-21swapper/308:15:293
1350991510,3cyclictest0-21swapper/308:40:023
1350991510,3cyclictest0-21swapper/308:40:013
1350991510,3cyclictest0-21swapper/308:05:163
135099148,4cyclictest0-21swapper/308:27:513
131599141,10cyclictest1322-21latency_hist07:10:010
13509991,5cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional