You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-28 - 04:25

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Thu May 28, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3000991921,189cyclictest13-21rcuc/021:10:370
3000991921,189cyclictest13-21rcuc/021:10:370
16532960,5sleep3300399cyclictest22:11:263
222952940,3sleep22999-21cyclictest21:19:352
164592810,2sleep00-21swapper/022:30:150
300099801,52cyclictest0-21swapper/023:00:160
186292780,1sleep10-21swapper/121:15:151
300099778,55cyclictest0-21swapper/000:00:180
140962760,3sleep114095-21sort23:05:311
322242750,5sleep17850irq/56-dwc_otg00:06:381
205892750,3sleep013-21rcuc/022:35:140
3000997332,6cyclictest0-21swapper/023:35:160
300399722,6cyclictest0-21swapper/300:00:163
300099718,54cyclictest0-21swapper/000:15:180
3000997133,5cyclictest0-21swapper/020:20:150
285722700,2sleep10-21swapper/121:25:381
298092690,1sleep20-21swapper/222:06:212
212002680,3sleep20-21swapper/221:55:472
300099678,31cyclictest0-21swapper/019:40:320
24682653,12sleep10-21swapper/119:06:371
15692650,2sleep10-21swapper/120:35:061
300099615,21cyclictest0-21swapper/022:45:150
3002996014,34cyclictest0-21swapper/222:45:152
300099602,38cyclictest29964-21fschecks_time23:25:210
300199593,25cyclictest7850irq/56-dwc_otg20:05:161
300099594,32cyclictest0-21swapper/022:20:190
3000995930,5cyclictest7850irq/56-dwc_otg21:55:050
3000995930,5cyclictest7850irq/56-dwc_otg21:55:050
300099582,8cyclictest3468-21fschecks_count21:35:230
300399576,22cyclictest0-21swapper/320:45:173
300199576,15cyclictest7850irq/56-dwc_otg21:20:151
3000995722,6cyclictest0-21swapper/000:30:150
3000995722,6cyclictest0-21swapper/000:30:150
3000995719,6cyclictest27742-21latency_hist22:05:050
300099565,28cyclictest31678-21fschecks_time21:30:210
300099565,28cyclictest31678-21fschecks_time21:30:210
300099561,31cyclictest23269-21runrttasks20:05:180
3001995544,7cyclictest21-21rcuc/123:10:161
300099552,23cyclictest0-21swapper/000:20:330
300399545,12cyclictest8914-21ssh23:00:153
300099544,20cyclictest22856-21ssh00:35:180
300099543,5cyclictest0-21swapper/021:45:150
3000995427,14cyclictest8050irq/56-dwc_otg_22:30:050
3000995419,4cyclictest1133-21runrttasks23:53:010
290012540,1sleep00-21swapper/021:26:250
59052530,3sleep20-21swapper/223:35:232
3003995310,24cyclictest27613-21ssh22:04:553
3000995327,5cyclictest7950irq/56-dwc_otg_22:10:050
3000995310,25cyclictest22980-21aten_rbpower_en21:20:170
5712520,1sleep00-21swapper/000:08:060
300399529,28cyclictest28324-21ssh22:44:383
300399528,13cyclictest0-21swapper/319:10:153
300399522,31cyclictest0-21swapper/322:55:163
300199523,32cyclictest8050irq/56-dwc_otg_22:45:151
3000995225,4cyclictest0-21swapper/019:35:050
3000995222,16cyclictest8050irq/56-dwc_otg_23:10:180
300099521,32cyclictest0-21swapper/022:10:110
203902520,2sleep30-21swapper/323:13:583
300199513,4cyclictest0-21swapper/122:40:161
300099512,26cyclictest21725-21fschecks_time20:00:220
300099511,28cyclictest28696-21cat22:45:050
300099511,26cyclictest5214-21grep19:15:220
300099511,23cyclictest22422-21cut23:55:230
300099511,23cyclictest22422-21cut23:55:230
300099511,20cyclictest809550ssh21:40:270
300399502,7cyclictest0-21swapper/322:20:173
300399500,27cyclictest22726-21latency_hist21:20:053
3000995029,4cyclictest25092-21wget20:10:190
300099502,29cyclictest0-21swapper/020:30:260
300099502,29cyclictest0-21swapper/020:30:260
3000995022,4cyclictest0-21swapper/021:15:270
3001994912,22cyclictest0-21swapper/120:15:171
3000994939,5cyclictest7850irq/56-dwc_otg23:45:040
3000994934,6cyclictest8050irq/56-dwc_otg_21:10:050
300099492,10cyclictest0-21swapper/019:10:260
300099491,27cyclictest6032-21sed22:55:330
300099491,27cyclictest13071-21ssh23:05:170
33462480,3sleep23349-21fschecks_time19:10:222
300399489,22cyclictest9693-21ssh21:42:283
300399488,21cyclictest21864-21ssh00:33:563
300399488,21cyclictest21864-21ssh00:33:563
300399487,26cyclictest0-21swapper/320:40:183
300399487,25cyclictest611-21ssh22:50:173
300399487,23cyclictest7460-21ssh23:36:433
300399485,18cyclictest17158-21ls22:30:253
3003994810,20cyclictest31773-21ssh00:05:483
3003994810,12cyclictest7950irq/56-dwc_otg_21:00:333
300199488,27cyclictest0-21swapper/120:55:171
300099484,23cyclictest2482-21timerwakeupswit23:30:320
300099484,23cyclictest2482-21timerwakeupswit23:30:320
300099482,24cyclictest0-21swapper/019:55:240
3000994822,14cyclictest0-21swapper/019:20:180
3000994816,7cyclictest2060-21wget00:10:180
300099481,28cyclictest0-21swapper/020:15:150
300099481,25cyclictest1890-21sed22:50:330
3000994810,22cyclictest0-21swapper/023:15:150
300399477,25cyclictest0-21swapper/300:10:183
300399475,25cyclictest23583-21cat21:20:253
3003994710,14cyclictest12771-21ssh00:22:273
300099476,25cyclictest0-21swapper/020:50:180
3000994713,17cyclictest16882-21sh00:27:230
300399468,23cyclictest0-21swapper/319:55:173
300399462,9cyclictest8050irq/56-dwc_otg_23:40:163
300199461,19cyclictest8050irq/56-dwc_otg_22:05:051
3000994628,4cyclictest7950irq/56-dwc_otg_23:20:170
3000994620,16cyclictest0-21swapper/019:29:020
300399457,24cyclictest0-21swapper/321:05:173
300399454,7cyclictest0-21swapper/320:55:163
3003994530,8cyclictest8050irq/56-dwc_otg_22:47:243
3003994517,5cyclictest381ktimersoftd/300:20:043
3003994516,14cyclictest8050irq/56-dwc_otg_21:56:523
300199457,24cyclictest0-21swapper/119:55:171
300199457,23cyclictest0-21swapper/119:20:171
3001994525,4cyclictest23-21ksoftirqd/121:00:191
300099451,41cyclictest101ktimersoftd/020:26:040
300099451,11cyclictest0-21swapper/020:40:050
26312453,35sleep00-21swapper/019:08:470
300399446,20cyclictest17559-21rm00:28:383
300399443,10cyclictest14057-21fschecks_count23:45:223
3003994423,9cyclictest8050irq/56-dwc_otg_21:11:423
3003994423,9cyclictest8050irq/56-dwc_otg_21:11:423
300399442,19cyclictest8050irq/56-dwc_otg_00:00:063
300399442,19cyclictest8050irq/56-dwc_otg_00:00:063
3003994415,7cyclictest7850irq/56-dwc_otg22:05:163
3003994414,10cyclictest8050irq/56-dwc_otg_19:40:183
300299447,22cyclictest0-21swapper/219:25:172
300199446,29cyclictest26201-21ssh00:00:181
3000994419,7cyclictest18446-21proc_pri19:50:280
300099441,24cyclictest0-21swapper/021:56:210
300099441,23cyclictest16635-21ps19:45:290
300099441,22cyclictest12251-21latency_hist21:05:050
300399436,21cyclictest19544-21ssh23:51:083
300399435,24cyclictest3113-21ssh21:35:113
3003994321,12cyclictest14034-21diskmemload00:35:213
3003994310,17cyclictest30257-21ls23:25:243
300199436,24cyclictest7850irq/56-dwc_otg00:15:171
3000994318,5cyclictest0-21swapper/022:15:150
3000994316,18cyclictest0-21swapper/020:42:570
316882420,5sleep1300199cyclictest23:26:531
300399424,5cyclictest0-21swapper/320:25:153
300399424,20cyclictest7950irq/56-dwc_otg_22:19:203
3003994218,3cyclictest8050irq/56-dwc_otg_23:20:153
3003994213,11cyclictest8050irq/56-dwc_otg_21:31:153
3003994213,11cyclictest8050irq/56-dwc_otg_21:31:153
3003994211,12cyclictest12489-21ls21:45:313
3003994210,9cyclictest32245-21apt-get20:30:163
3003994210,9cyclictest32245-21apt-get20:30:163
300299427,25cyclictest0-21swapper/223:15:182
300199425,5cyclictest7850irq/56-dwc_otg20:00:181
300099423,18cyclictest9986-21unixbench_singl20:55:320
3000994224,4cyclictest7850irq/56-dwc_otg20:46:050
300099422,21cyclictest13752-21/usr/sbin/munin23:45:180
3003994124,4cyclictest8050irq/56-dwc_otg_21:53:423
3003994124,4cyclictest8050irq/56-dwc_otg_21:53:423
3003994113,10cyclictest8050irq/56-dwc_otg_20:35:283
300299416,21cyclictest0-21swapper/223:55:182
300299416,21cyclictest0-21swapper/223:55:182
300299413,21cyclictest0-21swapper/200:25:052
300299413,18cyclictest0-21swapper/220:50:162
300299411,17cyclictest0-21swapper/221:20:322
3001994122,4cyclictest221ktimersoftd/121:46:121
84902400,2sleep10-21swapper/121:40:321
3003994014,13cyclictest8050irq/56-dwc_otg_19:15:423
300299403,8cyclictest0-21swapper/223:00:162
3002994031,6cyclictest301ktimersoftd/200:30:152
3002994031,6cyclictest301ktimersoftd/200:30:152
300199407,23cyclictest0-21swapper/119:40:111
300199403,24cyclictest20395-21ssh21:55:261
3001994020,13cyclictest23-21ksoftirqd/123:59:511
3001994020,13cyclictest23-21ksoftirqd/123:59:511
178712400,2sleep10-21swapper/119:50:181
300399397,25cyclictest0-21swapper/319:30:193
300399391,12cyclictest23872-21ntpq20:05:263
300299397,22cyclictest0-21swapper/223:05:172
3002993916,5cyclictest21323-21ssh22:35:262
300199398,5cyclictest7850irq/56-dwc_otg23:00:321
300199393,19cyclictest0-21swapper/121:11:481
300199393,19cyclictest0-21swapper/121:11:481
3001993917,5cyclictest1064-21dhclient22:19:381
300099391,22cyclictest0-21swapper/019:35:220
300099391,22cyclictest0-21swapper/019:35:220
35182380,1sleep20-21swapper/223:32:052
35182380,1sleep20-21swapper/223:32:052
300399388,22cyclictest29689-21ssh21:27:403
300399388,17cyclictest8050irq/56-dwc_otg_23:07:423
300399388,13cyclictest20856-21ssh22:35:203
300399384,13cyclictest26186-21/usr/sbin/munin20:10:343
300299385,22cyclictest20197-21ls19:55:252
300199388,5cyclictest10899-21sed23:40:331
300199385,6cyclictest7850irq/56-dwc_otg00:15:051
300199383,9cyclictest10404-21ssh00:20:151
300199383,5cyclictest0-21swapper/120:30:171
3001993812,6cyclictest0-21swapper/100:25:261
3001993811,4cyclictest0-21swapper/123:33:151
3001993811,4cyclictest0-21swapper/123:33:151
85892370,6sleep2300299cyclictest00:17:222
300399376,19cyclictest0-21swapper/319:23:533
3003993711,11cyclictest3067-21ssh23:31:153
3003993711,11cyclictest3067-21ssh23:31:153
3003993710,11cyclictest24933-21rm23:19:403
300299377,21cyclictest0-21swapper/222:30:172
300299375,23cyclictest8050irq/56-dwc_otg_23:50:312
3002993716,11cyclictest14781-21ssh00:25:202
300299371,26cyclictest3766-21ssh21:35:262
300299371,22cyclictest17382-21cpuspeed23:10:192
300299371,20cyclictest12809-21ls21:05:222
300199379,20cyclictest0-21swapper/122:23:491
300199376,7cyclictest7950irq/56-dwc_otg_00:30:221
300199376,7cyclictest7950irq/56-dwc_otg_00:30:221
3001993720,5cyclictest14556-21sh23:45:281
300399367,24cyclictest0-21swapper/319:50:163
300399365,19cyclictest12338-21ssh22:25:183
300399361,12cyclictest23101-21latency_hist20:05:053
300299368,13cyclictest32193-21latency_hist20:30:062
300299366,21cyclictest0-21swapper/223:45:182
300299365,21cyclictest0-21swapper/221:00:182
300299362,15cyclictest0-21swapper/223:20:142
3002993618,5cyclictest29325-21ntp_states20:20:272
300299361,4cyclictest7950irq/56-dwc_otg_20:10:162
300199366,5cyclictest0-21swapper/120:50:161
300199365,6cyclictest9462-21latency_hist23:40:051
3001993616,5cyclictest20223-21sh23:52:231
300299358,5cyclictest7950irq/56-dwc_otg_00:10:162
300299356,6cyclictest7950irq/56-dwc_otg_22:50:332
3002993516,12cyclictest7926-21/usr/sbin/munin19:20:322
300199357,7cyclictest8050irq/56-dwc_otg_23:20:041
300199351,24cyclictest31933-21latency21:30:241
300199351,24cyclictest31933-21latency21:30:241
3001993512,4cyclictest0-21swapper/122:30:171
300399345,6cyclictest7950irq/56-dwc_otg_20:15:323
300299345,6cyclictest7850irq/56-dwc_otg22:20:172
300299345,22cyclictest7950irq/56-dwc_otg_23:25:212
300299345,13cyclictest7950irq/56-dwc_otg_00:05:052
300299343,6cyclictest1371-21munin-plugin-st20:35:052
300299343,6cyclictest1371-21munin-plugin-st20:35:052
3002993411,5cyclictest12525-21idleruntime19:35:232
3002993411,5cyclictest12525-21idleruntime19:35:232
3002993410,5cyclictest7950irq/56-dwc_otg_00:35:182
3001993413,5cyclictest23-21ksoftirqd/122:05:241
300199341,23cyclictest0-21swapper/120:45:051
300199341,20cyclictest1634-21ssh22:50:311
300199341,18cyclictest26700-21ssh00:40:051
300199341,14cyclictest7850irq/56-dwc_otg20:40:051
258802340,3sleep121-21rcuc/120:10:311
300399337,15cyclictest0-21swapper/319:45:173
300399334,12cyclictest12153-21chrt19:35:103
300399334,12cyclictest12153-21chrt19:35:103
300299335,22cyclictest0-21swapper/222:15:312
300299334,20cyclictest7950irq/56-dwc_otg_23:40:402
3002993322,8cyclictest27900-21latency_hist22:05:052
300299331,8cyclictest7384-21ssh22:57:542
3002993314,17cyclictest1753550ssh21:13:462
3002993314,17cyclictest1753550ssh21:13:462
300299331,22cyclictest25341-21latency22:40:252
300199338,6cyclictest7850irq/56-dwc_otg21:50:291
300199338,6cyclictest7850irq/56-dwc_otg21:50:291
300199332,5cyclictest0-21swapper/120:45:151
300199331,23cyclictest0-21swapper/122:10:351
3001993311,7cyclictest8050irq/56-dwc_otg_20:20:211
300199331,16cyclictest25996-21irqstats23:20:221
300199331,16cyclictest13635-21users22:25:341
300399327,14cyclictest8050irq/56-dwc_otg_20:20:173
300399325,11cyclictest8050irq/56-dwc_otg_19:25:173
300399321,4cyclictest8050irq/56-dwc_otg_20:50:173
300299325,5cyclictest7950irq/56-dwc_otg_19:40:322
300299322,5cyclictest7950irq/56-dwc_otg_21:30:212
300299322,5cyclictest7950irq/56-dwc_otg_21:30:212
300299321,5cyclictest5059-21latency_hist20:45:052
300199323,4cyclictest7950irq/56-dwc_otg_19:15:191
300199321,4cyclictest7850irq/56-dwc_otg22:55:201
300299317,21cyclictest0-21swapper/219:45:172
300299315,5cyclictest7850irq/56-dwc_otg21:50:322
300299315,5cyclictest7850irq/56-dwc_otg21:50:322
300299315,15cyclictest2496-21ls20:35:312
300299315,13cyclictest27679-21/usr/sbin/munin21:25:222
3002993123,5cyclictest8050irq/56-dwc_otg_22:25:192
300299311,8cyclictest0-21swapper/220:05:152
300299311,20cyclictest27165-21idleruntime20:15:222
300299311,14cyclictest7950irq/56-dwc_otg_21:40:242
300199317,14cyclictest0-21swapper/121:35:101
300199315,5cyclictest7950irq/56-dwc_otg_22:35:201
300199312,6cyclictest7850irq/56-dwc_otg20:25:201
300199311,5cyclictest8050irq/56-dwc_otg_19:25:211
300299306,17cyclictest7950irq/56-dwc_otg_22:10:092
300299303,5cyclictest7950irq/56-dwc_otg_19:15:222
300299303,4cyclictest0-21swapper/220:55:212
300299302,20cyclictest7950irq/56-dwc_otg_20:05:042
3002993018,4cyclictest19332-21latency_hist19:55:052
3002993011,7cyclictest10703-21fschecks_time19:30:232
105822300,3sleep1300199cyclictest19:30:211
300299291,20cyclictest0-21swapper/221:45:172
300199299,5cyclictest23-21ksoftirqd/119:10:221
300199291,6cyclictest7850irq/56-dwc_otg21:05:231
3001992914,8cyclictest12284-21wget19:35:191
3001992914,8cyclictest12284-21wget19:35:191
300299282,7cyclictest14034-21diskmemload00:05:252
300199282,4cyclictest7950irq/56-dwc_otg_19:45:251
300299275,4cyclictest8050irq/56-dwc_otg_20:45:282
26282263,16sleep20-21swapper/219:08:452
25922264,14sleep30-21swapper/319:08:173
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional