You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-08 - 09:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot3s.osadl.org (updated Fri May 08, 2026 00:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1115026030,5sleep00-21swapper/004:26:050
1115026030,5sleep00-21swapper/004:26:050
1122122560,2sleep30-21swapper/304:26:083
1122122560,2sleep30-21swapper/304:26:083
1296299943,70cyclictest0-21swapper/004:30:580
214602920,2sleep30-21swapper/309:38:063
1296299872,73cyclictest0-21swapper/006:10:560
1296299851,23cyclictest16679-21aten_rbpower_vo08:16:100
1296299832,34cyclictest8385-21fschecks_time08:06:130
1296299821,27cyclictest0-21swapper/009:20:560
47082810,1sleep30-21swapper/309:17:453
12962998025,5cyclictest21882-21cut07:06:100
1296299792,37cyclictest0-21swapper/005:51:220
1296299778,15cyclictest0-21swapper/009:39:240
155302760,5sleep11296399cyclictest06:06:161
45532750,1sleep20-21swapper/208:01:192
12962997521,3cyclictest0-21swapper/005:45:550
69072740,2sleep111-21rcu_preempt08:41:241
12962997430,3cyclictest0-21swapper/009:06:210
1296299742,58cyclictest0-21swapper/006:41:070
1296299732,29cyclictest0-21swapper/008:47:010
12962997320,4cyclictest27450-21latency_hist07:50:550
1296299731,27cyclictest28763-21runrttasks06:36:120
12962997222,4cyclictest10612-21/usr/sbin/munin07:31:150
12962997221,12cyclictest7850irq/56-dwc_otg_04:45:550
1296299721,24cyclictest0-21swapper/008:21:080
254082710,2sleep20-21swapper/207:47:342
1296299711,38cyclictest0-21swapper/006:47:270
1296299701,33cyclictest23906-21sed09:01:240
12962996925,28cyclictest0-21swapper/006:26:070
1296299692,32cyclictest0-21swapper/005:02:300
1296299692,24cyclictest0-21swapper/006:51:220
1296299692,24cyclictest0-21swapper/006:51:220
12962996920,12cyclictest0-21swapper/006:22:010
1296299681,39cyclictest0-21swapper/008:35:490
12963996623,11cyclictest7950irq/56-dwc_otg_08:56:071
1296299662,37cyclictest0-21swapper/007:02:190
12962996619,5cyclictest0-21swapper/006:16:090
1296299655,27cyclictest12961-21cyclictest09:52:140
74062640,2sleep10-21swapper/108:05:281
1296299641,30cyclictest0-21swapper/008:55:260
256382630,2sleep10-21swapper/107:10:531
1296299632,40cyclictest0-21swapper/009:25:560
1296299622,50cyclictest13524-21ssh06:56:120
12962996026,4cyclictest7750irq/56-dwc_otg09:41:110
12962996026,4cyclictest7750irq/56-dwc_otg09:41:110
1296299601,35cyclictest0-21swapper/009:50:450
12962995945,6cyclictest7750irq/56-dwc_otg07:55:550
1296299594,32cyclictest0-21swapper/005:16:070
1296299591,47cyclictest0-21swapper/006:33:490
1296299587,42cyclictest0-21swapper/008:56:090
1296299573,37cyclictest0-21swapper/005:56:050
1296299572,46cyclictest5240-21sed05:36:240
1296299572,46cyclictest5240-21sed05:36:240
12962995724,5cyclictest7850irq/56-dwc_otg_08:11:080
1296299571,48cyclictest0-21swapper/008:01:170
1296299571,44cyclictest0-21swapper/009:30:550
12964995637,16cyclictest7950irq/56-dwc_otg_09:16:082
1296299566,28cyclictest30553-21fschecks_count07:16:120
1296299552,42cyclictest0-21swapper/008:26:210
1296299552,39cyclictest0-21swapper/008:38:360
1296599543,36cyclictest0-21swapper/304:30:583
12962995430,4cyclictest0-21swapper/009:31:090
1296299541,49cyclictest22261-21cut04:56:120
1296299541,44cyclictest4532-21ssh09:56:260
1296599531,10cyclictest7750irq/56-dwc_otg04:41:083
1296299532,36cyclictest0-21swapper/005:46:150
1296299532,34cyclictest0-21swapper/007:25:550
12962995322,12cyclictest7850irq/56-dwc_otg_07:41:230
12962995322,12cyclictest7850irq/56-dwc_otg_07:41:230
1296299531,37cyclictest29924-21ssh07:15:550
12962995238,5cyclictest7750irq/56-dwc_otg05:10:560
1296299522,37cyclictest0-21swapper/007:38:330
1296299522,35cyclictest0-21swapper/008:41:060
12962995128,5cyclictest20902-21ntp_kernel_pll_04:51:180
1296299512,39cyclictest0-21swapper/006:12:020
1296299512,39cyclictest0-21swapper/006:12:020
1296299512,38cyclictest0-21swapper/004:39:200
1296299511,39cyclictest0-21swapper/009:15:260
1296299511,39cyclictest0-21swapper/009:15:260
1296299511,26cyclictest757-21cut05:26:110
1296599507,17cyclictest0-21swapper/306:56:093
1296399505,27cyclictest7950irq/56-dwc_otg_06:11:151
1296399505,27cyclictest7950irq/56-dwc_otg_06:11:151
1296299502,37cyclictest0-21swapper/005:15:020
241882490,6sleep2301ktimersoftd/207:08:202
1296499491,22cyclictest7950irq/56-dwc_otg_05:21:072
1296299491,36cyclictest0-21swapper/005:31:060
1296599487,26cyclictest0-21swapper/305:56:083
1296399483,10cyclictest0-21swapper/107:56:071
1296299481,4cyclictest7237-21grep07:26:200
1296299481,36cyclictest32691-21ntpq07:56:180
1296599472,9cyclictest7750irq/56-dwc_otg06:01:063
12963994711,18cyclictest7750irq/56-dwc_otg07:26:091
12962994729,9cyclictest7950irq/56-dwc_otg_04:46:250
1296599467,24cyclictest0-21swapper/308:21:083
1296599467,24cyclictest0-21swapper/305:16:013
1296399462,11cyclictest0-21swapper/107:51:091
12963994620,6cyclictest14691-21latency_hist04:35:561
1296599457,24cyclictest0-21swapper/305:06:103
1296599451,25cyclictest0-21swapper/308:41:093
1296399451,5cyclictest19975-21ntp_states09:36:191
1296299451,34cyclictest0-21swapper/005:21:080
1296599447,24cyclictest0-21swapper/306:21:083
1296599447,23cyclictest0-21swapper/305:31:083
1296499449,24cyclictest30715-21grep08:31:252
12964994428,9cyclictest32347-21ssh07:17:422
1296399448,7cyclictest7750irq/56-dwc_otg04:56:061
1296399442,4cyclictest7950irq/56-dwc_otg_06:41:071
1296399441,19cyclictest5200-21latency_hist08:40:561
12962994427,6cyclictest7950irq/56-dwc_otg_06:01:150
1296599437,25cyclictest0-21swapper/307:11:093
1296599437,23cyclictest0-21swapper/305:41:083
1296599436,22cyclictest0-21swapper/304:56:083
1296599434,22cyclictest26814-21ssh09:06:073
12965994322,7cyclictest10963-21fschecks_time07:31:143
1296499433,25cyclictest0-21swapper/204:51:092
1296499431,18cyclictest26826-21ssh09:06:092
12963994319,4cyclictest9313-21idleruntime-cro05:50:551
266182421,2sleep20-21swapper/205:06:242
1296599429,5cyclictest7750irq/56-dwc_otg07:56:243
12965994222,5cyclictest349150wget09:56:123
1296499426,14cyclictest4818-21ssh09:56:432
12964994213,15cyclictest0-21swapper/208:36:202
12963994217,4cyclictest26340-21fschecks_count07:11:131
12963994211,5cyclictest15405-21sort08:51:241
12963994210,4cyclictest0-21swapper/108:26:241
180532410,1sleep20-21swapper/209:34:272
1296499415,23cyclictest4628-21diskmemload06:46:022
1296499414,12cyclictest31-21ksoftirqd/206:31:182
1296499413,12cyclictest0-21swapper/204:30:582
1296399412,8cyclictest0-21swapper/106:46:071
12963994126,7cyclictest7950irq/56-dwc_otg_09:16:141
12963994117,5cyclictest32532-21sh09:11:541
12963994117,5cyclictest32532-21sh09:11:541
91552400,1sleep10-21swapper/109:23:191
12965994020,5cyclictest3040550wget07:16:093
12965994019,5cyclictest39-21ksoftirqd/305:21:203
1296499403,5cyclictest7850irq/56-dwc_otg_08:56:072
1296399405,4cyclictest7750irq/56-dwc_otg07:31:081
1296599398,25cyclictest0-21swapper/305:26:083
1296499394,24cyclictest31-21ksoftirqd/205:50:552
1296499394,20cyclictest0-21swapper/206:06:162
12963993911,5cyclictest0-21swapper/106:36:171
1296599389,7cyclictest7750irq/56-dwc_otg09:46:243
1296599389,17cyclictest0-21swapper/308:52:193
1296599385,4cyclictest0-21swapper/309:43:023
1296599385,4cyclictest0-21swapper/309:43:023
1296599385,21cyclictest25113-21ssh08:26:103
12965993811,16cyclictest37-21rcuc/307:05:573
12965993810,13cyclictest7750irq/56-dwc_otg07:28:383
1296499389,18cyclictest0-21swapper/205:36:082
1296499389,18cyclictest0-21swapper/205:36:082
12964993819,5cyclictest22585-21sh07:45:052
12964993819,5cyclictest22585-21sh07:45:052
12964993817,5cyclictest4154-21sh07:22:402
12964993813,13cyclictest0-21swapper/206:36:242
12963993817,5cyclictest13250-21sh06:56:081
1296599376,19cyclictest0-21swapper/307:01:553
1296599373,13cyclictest39-21ksoftirqd/309:26:233
1296599372,3cyclictest0-21swapper/307:41:083
1296599372,3cyclictest0-21swapper/307:41:083
1296599371,22cyclictest16274-21ssh08:15:563
12965993710,18cyclictest28466-21ssh06:36:063
1296499375,21cyclictest61-21kswapd009:26:062
12964993717,4cyclictest22330-21sh08:21:452
12963993715,5cyclictest16464-21sh09:31:321
12963993710,4cyclictest2078-21/usr/sbin/munin07:21:241
1296599368,7cyclictest39-21ksoftirqd/306:46:223
1296599367,19cyclictest39-21ksoftirqd/308:02:173
1296599365,5cyclictest39-21ksoftirqd/305:51:093
12965993621,4cyclictest23851-21sh09:01:243
12965993611,5cyclictest24333-21/usr/sbin/munin07:46:193
12964993620,9cyclictest16010-21sh08:52:062
12964993620,10cyclictest9083-21ssh07:28:512
1296399368,15cyclictest0-21swapper/105:35:551
1296399366,5cyclictest7950irq/56-dwc_otg_06:51:101
1296399366,5cyclictest7950irq/56-dwc_otg_06:51:101
1296399365,6cyclictest936-21seq08:35:551
1296399365,3cyclictest0-21swapper/105:26:061
1296399363,17cyclictest7950irq/56-dwc_otg_09:51:121
1296399362,5cyclictest0-21swapper/107:01:261
1296399361,13cyclictest0-21swapper/107:17:561
1296599357,15cyclictest0-21swapper/306:31:073
1296599356,15cyclictest0-21swapper/306:06:093
1296599355,16cyclictest7850irq/56-dwc_otg_04:36:253
12965993518,5cyclictest8026-21sh08:06:083
12965993517,4cyclictest7750irq/56-dwc_otg04:51:093
12965993516,5cyclictest19104-21/usr/sbin/munin06:16:133
1296599351,25cyclictest7850irq/56-dwc_otg_09:21:173
1296499358,10cyclictest11172-21grep07:31:172
1296499357,14cyclictest908-21runrttasks08:06:102
1296499356,5cyclictest22668-21/usr/sbin/munin09:01:062
1296499355,21cyclictest19218-21ssh09:36:092
1296499355,17cyclictest0-21swapper/208:11:132
1296499353,25cyclictest0-21swapper/207:40:532
1296499352,20cyclictest0-21swapper/204:56:152
12964993517,9cyclictest2676750rm09:45:142
12964993517,9cyclictest2676750rm09:45:142
12964993516,12cyclictest26276-21ssh07:11:132
1296499351,24cyclictest16372-21latency_hist04:40:562
1296499351,20cyclictest29199-21gpgv05:15:572
1296499351,13cyclictest17592-21aten_rbpower_en07:01:112
1296399357,13cyclictest7950irq/56-dwc_otg_09:56:191
1296399355,6cyclictest15708-21ssh07:36:221
1296399352,10cyclictest0-21swapper/105:36:081
1296399352,10cyclictest0-21swapper/105:36:081
12963993516,4cyclictest22700-21sh09:01:091
12963993515,5cyclictest22056-21idleruntime-cro06:25:551
1296399351,22cyclictest26202-21missed_timers05:06:171
1296599341,9cyclictest7750irq/56-dwc_otg08:36:083
12965993415,4cyclictest37-21rcuc/308:47:063
1296499348,16cyclictest0-21swapper/205:31:122
1296499345,17cyclictest0-21swapper/205:01:082
1296499342,24cyclictest0-21swapper/208:46:342
1296499342,21cyclictest0-21swapper/205:56:112
12964993421,5cyclictest1141-21ssh07:56:582
12964993418,10cyclictest5188-21kworker/u8:209:51:082
12964993413,5cyclictest29355-21sh07:52:022
1296399347,14cyclictest7850irq/56-dwc_otg_08:11:141
1296399345,7cyclictest7850irq/56-dwc_otg_05:21:221
1296399342,5cyclictest0-21swapper/104:51:241
12963993419,4cyclictest12010-21latency_hist08:10:561
12963993418,5cyclictest23856-21diskmemload07:41:241
12963993418,5cyclictest23856-21diskmemload07:41:241
12963993416,7cyclictest7850irq/56-dwc_otg_09:06:091
1296399341,4cyclictest28667-21ls09:46:231
1296399341,23cyclictest29037-21gpgv05:15:561
1296399341,23cyclictest0-21swapper/105:41:351
12963993410,10cyclictest0-21swapper/108:23:331
1296599337,6cyclictest7850irq/56-dwc_otg_07:51:263
1296599337,14cyclictest0-21swapper/306:11:013
1296599337,14cyclictest0-21swapper/306:11:013
1296599333,5cyclictest39-21ksoftirqd/306:41:133
1296599331,5cyclictest23436-21grep06:26:243
1296499339,13cyclictest22320-21aten_rbpower_cu06:26:102
1296499337,15cyclictest0-21swapper/206:41:092
12964993313,4cyclictest0-21swapper/208:16:092
1296499331,26cyclictest0-21swapper/209:21:152
1296499331,14cyclictest0-21swapper/206:56:092
1296399333,7cyclictest7950irq/56-dwc_otg_05:01:121
1296399331,6cyclictest23726-21fschecks_time09:41:161
1296399331,6cyclictest23726-21fschecks_time09:41:161
1296399331,5cyclictest24722-21users07:46:231
167162320,2sleep10-21swapper/104:41:101
1296599326,9cyclictest39-21ksoftirqd/307:22:583
1296599325,20cyclictest31642-21ssh09:51:083
1296599324,7cyclictest39-21ksoftirqd/308:15:593
1296599324,14cyclictest7850irq/56-dwc_otg_05:11:133
1296599323,18cyclictest15853-21ntpq09:31:193
1296599321,18cyclictest2459-21ls09:15:563
1296599321,18cyclictest2459-21ls09:15:563
1296599321,14cyclictest9933-21switchtime06:51:233
1296599321,14cyclictest9933-21switchtime06:51:233
12965993211,4cyclictest7950irq/56-dwc_otg_05:46:093
1296499328,5cyclictest7750irq/56-dwc_otg08:26:172
1296499326,16cyclictest20804-21fschecks_time06:21:122
1296499324,9cyclictest0-21swapper/206:51:082
1296499324,9cyclictest0-21swapper/206:51:082
1296499323,17cyclictest0-21swapper/205:16:012
1296499322,5cyclictest0-21swapper/209:46:172
1296499321,5cyclictest30913-21ssh09:11:042
1296499321,5cyclictest30913-21ssh09:11:042
1296499321,16cyclictest18160-21cron04:45:552
12964993210,5cyclictest8568-21sh08:44:132
1296399329,5cyclictest7950irq/56-dwc_otg_06:31:091
1296399325,5cyclictest7950irq/56-dwc_otg_05:56:121
1296399324,5cyclictest7950irq/56-dwc_otg_06:01:221
1296399322,7cyclictest22319-21aten_rbpower_cu06:26:101
1296399321,19cyclictest16877-21uniq08:16:131
294372310,1sleep30-21swapper/308:31:073
1296599318,4cyclictest0-21swapper/308:58:243
1296599311,22cyclictest4451-21uniq05:36:143
1296599311,22cyclictest4451-21uniq05:36:143
1296599311,16cyclictest39-21ksoftirqd/307:39:513
1296499316,16cyclictest18946-21sort06:16:122
1296499315,18cyclictest908-21runrttasks05:54:452
1296499315,14cyclictest0-21swapper/205:45:262
12964993118,7cyclictest301ktimersoftd/206:01:172
1296399317,12cyclictest0-21swapper/109:28:121
1296399313,5cyclictest7950irq/56-dwc_otg_04:46:181
1296599302,4cyclictest7850irq/56-dwc_otg_04:46:113
1296599301,17cyclictest0-21swapper/305:01:203
1296499294,12cyclictest301ktimersoftd/206:11:022
1296499294,12cyclictest301ktimersoftd/206:11:022
1296499291,12cyclictest7950irq/56-dwc_otg_04:48:082
1296399294,4cyclictest7750irq/56-dwc_otg05:18:371
1296399293,16cyclictest7950irq/56-dwc_otg_08:49:221
12963992911,8cyclictest23-21ksoftirqd/106:16:181
127432295,11sleep10-21swapper/104:30:571
127432295,11sleep10-21swapper/104:30:571
12964992810,4cyclictest804-21cut05:26:112
1296399277,7cyclictest7850irq/56-dwc_otg_04:36:121
1296399274,5cyclictest7950irq/56-dwc_otg_05:51:161
113872253,7sleep20-21swapper/204:26:122
113872253,7sleep20-21swapper/204:26:122
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional