You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-15 - 17:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Wed Apr 15, 2026 12:45:18)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
18763997132,26cyclictest35966irq/58-eth012:15:143
18728996021,9cyclictest12-21ksoftirqd/011:45:120
1875399564,38cyclictest25656-21ssh11:45:112
1874499513,38cyclictest20183-21ssh10:30:121
1874499513,38cyclictest20183-21ssh10:30:121
1874499504,34cyclictest21737-21ssh09:55:141
1874499504,34cyclictest21737-21ssh09:55:131
1872899467,36cyclictest11275-21ssh11:55:130
1872899467,36cyclictest11275-21ssh11:55:130
18763994231,8cyclictest35966irq/58-eth012:30:113
1875399423,19cyclictest31146-21ssh11:30:112
18753994221,17cyclictest27346irq/40-dwc2_hso09:20:102
18728994128,6cyclictest4754-21diskmemload11:35:130
1872899412,32cyclictest5956-21ssh10:40:150
18753993923,6cyclictest27-21ksoftirqd/210:15:112
1875399392,29cyclictest4736-21cat12:25:302
1876399381,21cyclictest35966irq/58-eth012:10:113
18763993811,17cyclictest35966irq/58-eth010:15:123
1876399376,22cyclictest35966irq/58-eth010:20:103
1876399374,21cyclictest35966irq/58-eth010:35:133
18753993721,10cyclictest29340-1kworker/u9:110:03:292
18753993710,14cyclictest2047-21ssh10:20:112
18763993619,10cyclictest35966irq/58-eth009:10:123
1875399362,28cyclictest18611-21ssh10:28:382
1876399351,4cyclictest0-21swapper/308:35:103
1876399351,30cyclictest35966irq/58-eth009:35:123
1875399353,23cyclictest32303-21ssh09:25:132
18753993516,12cyclictest0-21swapper/211:20:112
1872899358,21cyclictest12-21ksoftirqd/008:25:140
1872899351,23cyclictest27048-21ssh09:40:110
1876399343,22cyclictest35966irq/58-eth010:00:123
1876399343,10cyclictest35966irq/58-eth009:45:123
1874499345,17cyclictest27484-21ssh11:10:141
1874499345,17cyclictest27484-21ssh11:10:131
1872899342,25cyclictest4032-21ssh11:15:110
1876399337,22cyclictest0-21swapper/310:30:133
1876399337,22cyclictest0-21swapper/310:30:123
1876399333,17cyclictest35966irq/58-eth010:45:133
1876399332,21cyclictest35966irq/58-eth012:25:133
1875399333,23cyclictest14174-21ssh09:15:142
1875399326,13cyclictest27346irq/40-dwc2_hso07:50:132
1875399326,13cyclictest27346irq/40-dwc2_hso07:50:122
18763993115,9cyclictest0-21swapper/310:25:133
1875399312,18cyclictest27346irq/40-dwc2_hso07:15:112
1874499311,28cyclictest0-21swapper/107:30:111
1874499311,28cyclictest0-21swapper/107:30:101
1872899314,20cyclictest12-21ksoftirqd/009:35:130
1872899313,18cyclictest8946-21ssh09:30:110
1872899312,23cyclictest3401-21ssh10:37:290
1872899311,3cyclictest131rcu_preempt07:10:130
1876399303,3cyclictest0-21swapper/311:00:183
1875399305,16cyclictest27346irq/40-dwc2_hso11:15:122
1875399303,16cyclictest27346irq/40-dwc2_hso09:00:122
1876399293,12cyclictest35966irq/58-eth009:15:123
1876399291,3cyclictest0-21swapper/311:30:133
1875399293,17cyclictest27-21ksoftirqd/211:00:142
1875399292,21cyclictest22138-21unixbench_singl09:35:272
1875399292,11cyclictest15078-21ssh12:31:302
18753992912,9cyclictest27-21ksoftirqd/207:55:132
1874499293,19cyclictest27058-21ssh09:40:131
1874499292,20cyclictest9392-21ssh12:30:121
1872899292,21cyclictest2578-21usage_Anybus10:00:290
1872899291,21cyclictest2057-21ssh10:20:110
1876399281,4cyclictest0-21swapper/309:58:293
1876399281,4cyclictest0-21swapper/309:58:283
1876399281,15cyclictest35966irq/58-eth007:30:113
1876399281,15cyclictest35966irq/58-eth007:30:103
18753992812,9cyclictest0-21swapper/208:00:132
1874499282,23cyclictest32304-21ssh09:25:121
1874499282,18cyclictest13453-21cat09:30:291
1874499281,22cyclictest0-21swapper/108:55:131
1874499281,12cyclictest3653-21ssh09:45:131
18728992812,11cyclictest151rcuc/011:00:150
1876399271,9cyclictest0-21swapper/311:35:213
1875399273,18cyclictest0-21swapper/210:37:302
1875399272,17cyclictest0-21swapper/212:35:132
1875399272,15cyclictest14924-21sh10:45:162
1875399272,10cyclictest0-21swapper/211:25:142
1875399272,10cyclictest0-21swapper/208:50:132
1875399272,10cyclictest0-21swapper/208:50:122
1874499272,21cyclictest12667-21ssh09:50:141
1874499271,22cyclictest18518-21ssh11:05:131
1872899272,21cyclictest23968-21ssh10:50:140
1876399263,16cyclictest0-21swapper/312:38:373
18763992618,5cyclictest35966irq/58-eth010:40:143
1876399261,4cyclictest0-21swapper/309:50:133
1876399261,4cyclictest0-21swapper/307:15:113
1876399261,16cyclictest35966irq/58-eth011:15:103
1876399261,10cyclictest0-21swapper/311:05:213
1875399266,13cyclictest27-21ksoftirqd/207:25:112
1875399264,16cyclictest27346irq/40-dwc2_hso08:55:122
1875399263,17cyclictest27346irq/40-dwc2_hso07:45:152
1875399263,15cyclictest27346irq/40-dwc2_hso08:05:122
18753992615,2cyclictest0-21swapper/211:05:112
1875399261,14cyclictest16797-21ssh11:40:152
1874499264,18cyclictest454-21ssh10:55:131
1874499263,19cyclictest18351-21ssh12:35:141
1874499262,19cyclictest16722-21ssh11:40:161
1874499262,18cyclictest25379-21sh10:15:131
1874499261,20cyclictest14155-21apt-get09:15:121
1872899263,17cyclictest14925-21ssh10:45:150
1872899262,17cyclictest7335-21ssh10:05:120
18728992613,5cyclictest12-21ksoftirqd/010:15:110
1876399253,6cyclictest35966irq/58-eth008:50:113
1876399253,6cyclictest35966irq/58-eth008:50:103
1876399252,13cyclictest35966irq/58-eth011:20:123
1876399251,5cyclictest0-21swapper/311:43:213
1876399251,4cyclictest0-21swapper/312:00:153
1876399251,4cyclictest0-21swapper/310:50:203
1876399251,3cyclictest0-21swapper/309:40:273
1876399251,3cyclictest0-21swapper/307:20:143
18763992513,9cyclictest35966irq/58-eth010:55:123
1875399252,17cyclictest27346irq/40-dwc2_hso08:30:122
1875399252,14cyclictest27346irq/40-dwc2_hso09:40:122
18753992516,5cyclictest23967-21sh10:50:132
1875399251,11cyclictest4015-21kthreadcore11:50:202
1875399251,11cyclictest4015-21kthreadcore11:50:202
1874499252,21cyclictest5792-21ssh12:10:111
1874499251,21cyclictest29262-21ssh10:35:131
1872899252,18cyclictest7223-21ssh09:28:260
18728992515,6cyclictest0-21swapper/012:05:140
1872899251,19cyclictest22225-21ssh11:25:140
1876399246,4cyclictest0-21swapper/312:09:133
1876399241,7cyclictest0-21swapper/311:25:113
1876399241,6cyclictest0-21swapper/311:50:143
1876399241,6cyclictest0-21swapper/311:50:133
1876399241,5cyclictest0-21swapper/309:20:533
1876399241,5cyclictest0-21swapper/307:40:223
1876399241,4cyclictest0-21swapper/311:14:543
1876399241,4cyclictest0-21swapper/311:14:543
1876399241,4cyclictest0-21swapper/309:30:193
1876399241,4cyclictest0-21swapper/307:35:273
1876399241,3cyclictest0-21swapper/312:20:153
1876399241,3cyclictest0-21swapper/310:05:203
1876399241,3cyclictest0-21swapper/308:15:283
18763992412,8cyclictest35966irq/58-eth009:25:133
1875399242,20cyclictest0-21swapper/207:20:132
1875399242,15cyclictest27346irq/40-dwc2_hso07:35:132
18753992412,5cyclictest26898-21ssh10:32:412
18753992412,5cyclictest26898-21ssh10:32:402
18753992411,9cyclictest0-21swapper/210:55:112
1875399240,11cyclictest14528-21latency_hist10:45:012
1874499243,15cyclictest17988-21apt-get09:35:111
1872899242,16cyclictest14798-21ssh12:15:140
1872899241,14cyclictest9368-21ssh12:30:110
1876399234,15cyclictest35966irq/58-eth009:00:113
1876399231,5cyclictest0-21swapper/311:56:103
1876399231,5cyclictest0-21swapper/311:56:093
1876399231,5cyclictest0-21swapper/310:10:313
1876399231,5cyclictest0-21swapper/309:00:013
1876399231,5cyclictest0-21swapper/308:40:293
1876399231,4cyclictest0-21swapper/309:08:303
18763992314,6cyclictest35966irq/58-eth011:45:123
1876399231,3cyclictest0-21swapper/308:45:193
1875399233,16cyclictest27346irq/40-dwc2_hso12:10:112
18753992311,8cyclictest8920-21/usr/sbin/munin09:30:252
1874499232,14cyclictest16367-21ssh10:10:131
18744992317,3cyclictest0-21swapper/110:40:141
1874499231,19cyclictest20574-21ls08:15:281
1874499231,17cyclictest30795-21sh10:00:131
1872899231,18cyclictest31374-21cpuspeed08:30:140
1872899231,18cyclictest23059-21dpkg09:20:020
1872899231,18cyclictest21888-21cut08:20:160
1872899231,16cyclictest16356-21ssh10:10:120
18728992310,8cyclictest20177-21sh10:30:130
18728992310,8cyclictest20177-21sh10:30:120
1876399222,4cyclictest0-21swapper/307:10:153
1876399221,5cyclictest0-21swapper/308:25:163
1876399221,5cyclictest0-21swapper/308:20:163
1876399221,4cyclictest0-21swapper/308:30:223
1875399222,7cyclictest0-21swapper/208:45:122
1875399222,15cyclictest0-21swapper/209:56:272
1875399222,15cyclictest0-21swapper/209:56:272
1875399222,15cyclictest0-21swapper/209:50:282
1875399222,14cyclictest27346irq/40-dwc2_hso07:10:132
1875399221,5cyclictest16930-21df08:15:152
18753992211,9cyclictest22486-21ssh12:18:572
1875399221,17cyclictest19161-21ssh10:10:222
1874499229,9cyclictest21-21ksoftirqd/109:10:141
1874499221,7cyclictest15199-21grep10:25:291
18744992217,3cyclictest0-21swapper/112:25:131
1874499221,18cyclictest13419-21sshd11:00:291
1874499221,16cyclictest20307-21ssh12:00:131
1874499221,16cyclictest13155-21ssh11:20:131
18744992210,3cyclictest131rcu_preempt12:20:221
18728992215,5cyclictest0-21swapper/011:05:130
18728992214,5cyclictest32371-21sh12:25:010
1872899221,16cyclictest21747-21ssh09:55:140
1872899221,16cyclictest21747-21ssh09:55:130
1876399211,5cyclictest0-21swapper/308:10:173
1876399211,5cyclictest0-21swapper/308:00:213
1876399211,5cyclictest0-21swapper/307:55:233
1876399211,5cyclictest0-21swapper/307:50:193
1876399211,5cyclictest0-21swapper/307:50:183
1876399211,5cyclictest0-21swapper/307:45:173
1876399211,4cyclictest0-21swapper/307:25:153
1875399213,15cyclictest28986-21kthreadcore08:25:222
1875399212,16cyclictest9665-21grep07:30:272
1875399212,16cyclictest9665-21grep07:30:262
1875399212,14cyclictest0-21swapper/212:05:132
1875399212,13cyclictest7724-21ssh11:35:142
1875399211,7cyclictest7566-21grep10:05:142
1875399211,6cyclictest23762-21ssh12:20:132
18753992111,8cyclictest0-21swapper/212:00:242
1875399211,12cyclictest10399-21sort08:40:202
1875399211,11cyclictest7513-21ssh09:45:272
1875399211,10cyclictest4820-21cat09:10:022
1874499211,18cyclictest7320-21latency_hist11:35:021
1874499211,18cyclictest4052-21ps08:35:161
1874499211,15cyclictest22386-21cut11:25:151
1872899218,10cyclictest5251-21df09:10:140
1872899213,13cyclictest11107-21ssh10:25:120
1872899212,15cyclictest20305-21sh12:00:130
1872899212,14cyclictest23212-21users12:35:300
1876399201,5cyclictest0-21swapper/308:05:183
1876399201,4cyclictest0-21swapper/307:10:013
1875399209,9cyclictest25051-21/usr/sbin/munin08:20:222
1875399207,4cyclictest15524-21cpu07:40:132
1875399205,7cyclictest4133-21cut08:35:172
1875399202,16cyclictest12259-21grep08:10:152
18753992013,4cyclictest0-21swapper/209:10:122
1875399201,15cyclictest29313-21kthreadcore11:10:222
1875399201,15cyclictest29313-21kthreadcore11:10:212
1874499201,14cyclictest18394-21ssh12:15:241
1874499201,10cyclictest27001-21cat07:15:211
18728992013,5cyclictest0-21swapper/007:50:160
18728992013,5cyclictest0-21swapper/007:50:160
1875399191,9cyclictest12931-21kthreadcore11:55:212
1875399191,9cyclictest12931-21kthreadcore11:55:202
1875399191,15cyclictest18777-21grep07:10:022
1874499192,13cyclictest18399-21ssh10:45:251
18744991913,4cyclictest0-21swapper/108:00:291
18744991913,3cyclictest0-21swapper/110:05:141
18744991912,4cyclictest0-21swapper/108:25:291
1874499191,14cyclictest28630-21apt-get07:20:111
1874499191,13cyclictest2312-21ssh11:50:141
1874499191,13cyclictest2312-21ssh11:50:141
1874499191,13cyclictest0-21swapper/108:40:151
18744991910,7cyclictest0-21swapper/109:21:311
1872899198,4cyclictest14801-1kworker/0:0H08:10:210
1872899193,11cyclictest0-21swapper/011:30:120
1872899191,6cyclictest5877-21df12:10:120
1872899191,15cyclictest73-21kswapd009:45:140
1872899191,14cyclictest16846-21apt-get08:15:130
1872899191,13cyclictest16721-21sh11:40:150
1872899190,17cyclictest27370-21ls07:15:250
1874499189,7cyclictest8901-21cat11:15:291
1874499185,10cyclictest28013-21unixbench_multi10:50:281
1874499183,12cyclictest26365-21irqstats11:45:171
18744991811,5cyclictest0-21swapper/107:40:261
1874499181,14cyclictest10808-21kthreadcore11:35:221
1874499181,11cyclictest32649-21cpuspeed09:05:141
1872899186,9cyclictest21867-21date11:25:010
1872899184,11cyclictest3276-21kthreadcore12:25:220
1872899182,11cyclictest26922-21ssh09:20:230
1872899181,13cyclictest5881-21ssh11:50:240
1872899181,13cyclictest5881-21ssh11:50:230
1874499179,6cyclictest14151-21ntpq07:35:231
1874499179,5cyclictest30846-21kthreadcore09:00:211
1874499178,7cyclictest12039-21apt-get08:10:131
1874499177,7cyclictest14612-21kthreadcore11:55:231
1874499177,7cyclictest14612-21kthreadcore11:55:221
1874499172,12cyclictest16680-21wc08:45:211
1874499172,11cyclictest29349-21diskstats12:05:151
1874499171,5cyclictest25433-21grep07:50:161
1874499171,5cyclictest25433-21grep07:50:151
18744991711,4cyclictest0-21swapper/108:50:281
18744991711,4cyclictest0-21swapper/108:50:281
1874499171,13cyclictest32469-21tr08:30:201
1874499171,13cyclictest2030-21apt-get10:20:091
1872899178,7cyclictest6989-21users08:00:310
1872899178,7cyclictest30136-21diskstats07:55:160
1872899178,7cyclictest22331-21cut07:45:220
1872899178,6cyclictest1822-21cut10:55:180
1872899177,7cyclictest12-21ksoftirqd/011:10:150
1872899177,7cyclictest12-21ksoftirqd/011:10:140
1872899173,11cyclictest1512-21sed07:25:160
1872899172,11cyclictest0-21swapper/008:40:140
1872899171,6cyclictest2048-21kthreadcore09:05:210
18728991711,4cyclictest0-21swapper/007:40:280
1872899171,13cyclictest12640-21ssh09:50:120
1872899171,12cyclictest29704-21tr09:00:190
1874499169,5cyclictest0-21swapper/107:55:191
1874499167,6cyclictest8042-21kthreadcore08:05:191
1874499166,5cyclictest0-21swapper/108:20:131
1874499161,7cyclictest18889-21/usr/sbin/munin07:10:241
1874499161,12cyclictest21043-21uname07:45:191
1874499161,11cyclictest2174-21kthreadcore07:25:191
1872899163,10cyclictest5934-21fschecks_count07:30:150
1872899163,10cyclictest5934-21fschecks_count07:30:140
1872899163,10cyclictest24563-21kthreadcore08:55:210
1872899162,10cyclictest10712-21find07:35:110
1872899162,10cyclictest0-21swapper/007:20:040
1872899161,12cyclictest10642-21ntp_192.168.11408:05:250
1872899161,11cyclictest4189-21irqcore08:35:160
18728991610,4cyclictest18268-21entropy08:50:140
18728991610,4cyclictest18268-21entropy08:50:130
1872899151,10cyclictest18776-21latency_hist07:10:010
18728991510,3cyclictest13462-21fschecks_count08:45:140
1874499141,10cyclictest18804-21latency_hist07:10:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional