You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-28 - 19:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sun Jun 28, 2026 12:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2085399711,17cyclictest12-21ksoftirqd/011:30:140
2085399603,47cyclictest9552-21ssh10:05:130
2086999553,40cyclictest28123-21ssh11:45:142
2086999542,37cyclictest20876-21ssh11:05:142
2086199501,39cyclictest4335-21ssh10:20:131
2086199492,34cyclictest23905-21ssh09:55:141
2086199485,39cyclictest1000-21ssh11:30:141
2086999441,22cyclictest11836-21ssh11:00:122
2087799425,26cyclictest35966irq/58-eth012:15:133
2087799424,25cyclictest35966irq/58-eth009:15:123
2086199427,3cyclictest131rcu_preempt09:45:131
2087799413,33cyclictest35966irq/58-eth009:20:113
2086999415,21cyclictest31902-21rm12:05:122
2087799404,22cyclictest35966irq/58-eth010:05:133
2086199403,6cyclictest131rcu_preempt07:45:121
2086199401,32cyclictest3072-21ssh12:25:131
2087799397,20cyclictest0-21swapper/310:00:143
20853993815,14cyclictest630-21Async09:36:310
2085399381,32cyclictest26159-21ssh10:50:130
20877993714,17cyclictest35966irq/58-eth012:00:113
2086199371,25cyclictest0-21swapper/107:55:131
20877993627,5cyclictest35966irq/58-eth009:45:143
20877993622,9cyclictest0-21swapper/310:50:123
2085399363,9cyclictest31480-21ssh10:35:130
2085399362,30cyclictest551-21ssh10:00:130
2087799354,22cyclictest35966irq/58-eth010:35:143
2086199353,21cyclictest13790-21ssh11:55:121
20853993526,4cyclictest131rcu_preempt12:30:120
2085399351,31cyclictest27599-21ssh10:15:120
2085399351,26cyclictest12-21ksoftirqd/008:00:120
2086999343,15cyclictest877-21lldpd12:20:102
2086199342,24cyclictest20040-21ssh09:35:131
2085399343,28cyclictest22952-21ssh12:00:140
2085399342,29cyclictest17145-21ssh10:45:120
2087799332,21cyclictest35966irq/58-eth011:20:123
20877993311,14cyclictest35966irq/58-eth012:05:123
2086999335,21cyclictest27346irq/40-dwc2_hso07:50:122
2086999332,24cyclictest20036-21ssh09:35:132
2086999332,21cyclictest17450-21ssh12:15:132
2086199336,14cyclictest14738-21ssh09:50:121
2086199334,18cyclictest26128-21ssh10:50:111
2085399336,18cyclictest11002-21ssh09:30:100
2085399335,23cyclictest18550-21ssh10:10:120
20853993321,6cyclictest13370-21sh10:25:140
20877993215,11cyclictest35966irq/58-eth010:15:123
2086999321,27cyclictest15335-21ssh11:20:122
2087799314,15cyclictest0-21swapper/310:45:113
20877993117,7cyclictest321rcuc/312:30:123
2086999314,15cyclictest27346irq/40-dwc2_hso07:15:142
2086199311,4cyclictest131rcu_preempt08:30:131
2086199311,23cyclictest13372-21ssh10:25:131
2085399312,23cyclictest25300-21ssh09:20:110
2085399311,24cyclictest29863-21ssh11:10:140
2087799302,15cyclictest35966irq/58-eth010:30:133
2086999306,15cyclictest27346irq/40-dwc2_hso09:00:142
2086999305,18cyclictest27346irq/40-dwc2_hso07:35:122
2086999303,19cyclictest27346irq/40-dwc2_hso07:55:142
20869993018,10cyclictest0-21swapper/209:10:142
2086199301,5cyclictest0-21swapper/110:05:111
2085399303,18cyclictest26449-21ssh12:20:100
2087799293,19cyclictest0-21swapper/312:21:023
2086999294,18cyclictest27346irq/40-dwc2_hso10:30:142
20869992919,6cyclictest27-21ksoftirqd/210:35:132
2086199299,12cyclictest131rcu_preempt12:30:111
2085399291,24cyclictest8093-21ssh10:40:130
2087799282,15cyclictest35966irq/58-eth011:35:123
2086999286,14cyclictest27346irq/40-dwc2_hso07:25:132
2086999283,15cyclictest27346irq/40-dwc2_hso11:35:122
2086999283,14cyclictest27346irq/40-dwc2_hso08:20:132
2086999282,19cyclictest0-21swapper/208:45:132
2086999282,11cyclictest17892-21ssh12:31:392
2085399282,21cyclictest1953-21ssh09:25:140
20853992813,9cyclictest12-21ksoftirqd/008:35:180
2087799272,15cyclictest35966irq/58-eth010:40:133
2086999273,14cyclictest27346irq/40-dwc2_hso08:25:122
20869992717,7cyclictest4329-21ssh10:20:122
2086999271,19cyclictest29098-21ssh09:40:142
2086199271,19cyclictest28139-21ssh11:45:151
2085399273,19cyclictest22421-21ssh10:30:130
20877992618,5cyclictest33-21ksoftirqd/309:35:123
2086999268,8cyclictest27-21ksoftirqd/210:45:142
2086999266,12cyclictest27346irq/40-dwc2_hso12:10:132
2086999266,12cyclictest27346irq/40-dwc2_hso12:10:132
2086999265,12cyclictest27346irq/40-dwc2_hso11:25:132
2086999263,16cyclictest27346irq/40-dwc2_hso08:15:122
2086999263,15cyclictest27346irq/40-dwc2_hso07:40:132
2086999262,14cyclictest27346irq/40-dwc2_hso08:10:122
2086999261,15cyclictest27598-21ssh10:15:122
2086999261,10cyclictest1169-21cut10:00:192
2086199267,11cyclictest0-21swapper/108:00:121
2086199262,2cyclictest131rcu_preempt09:10:121
2085399262,18cyclictest26413-21ssh12:36:050
20853992618,5cyclictest0-21swapper/009:15:120
2087799253,15cyclictest35966irq/58-eth008:25:133
2087799253,11cyclictest0-21swapper/312:25:143
20877992515,5cyclictest33-21ksoftirqd/309:05:143
2087799251,19cyclictest0-21swapper/312:10:143
2087799251,19cyclictest0-21swapper/312:10:143
2086999255,12cyclictest27346irq/40-dwc2_hso07:10:142
2086999254,13cyclictest27346irq/40-dwc2_hso07:45:132
2086999253,18cyclictest27346irq/40-dwc2_hso09:55:152
2086999252,8cyclictest28573-21ssh12:38:402
2086999252,16cyclictest6354-21ssh11:15:132
2086199256,15cyclictest21-21ksoftirqd/109:15:131
2085399251,19cyclictest6344-21ssh11:15:120
2085399251,18cyclictest15349-21sh11:20:130
2087799247,13cyclictest35966irq/58-eth011:30:143
2087799243,14cyclictest0-21swapper/311:00:133
2086999249,12cyclictest23147-21sh11:40:292
2086999242,19cyclictest0-21swapper/208:30:152
2086199242,17cyclictest29845-21apt-get11:10:121
2085399242,17cyclictest2751-21ssh10:55:100
2087799236,12cyclictest0-21swapper/311:45:153
2086999236,14cyclictest22664-21wc12:00:012
2086999234,15cyclictest4932-21grep11:30:312
2086999232,18cyclictest8322-21uniq10:40:152
2086999232,16cyclictest0-21swapper/212:29:402
20869992312,9cyclictest11205-21ssh09:46:142
2086199232,15cyclictest27608-21ssh10:15:131
2086199231,17cyclictest2069-21ssh09:25:161
2085399237,11cyclictest12-21ksoftirqd/007:15:140
20853992314,4cyclictest131rcu_preempt09:05:140
2085399231,18cyclictest21108-21cut07:10:160
20853992310,9cyclictest0-21swapper/011:00:130
2087799221,15cyclictest0-21swapper/311:25:123
20877992210,9cyclictest0-21swapper/311:15:163
20877992210,8cyclictest35966irq/58-eth009:55:143
2086999224,9cyclictest24613-21cut12:00:212
2086999223,15cyclictest13541-21date11:55:022
2086999222,14cyclictest27346irq/40-dwc2_hso11:10:132
20869992211,9cyclictest16171-21ssh09:15:052
2086999221,12cyclictest896766ssh10:57:032
2086999221,11cyclictest24064-21unixbench_singl08:50:262
2086199223,16cyclictest21093-21ssh12:35:141
2086199222,4cyclictest211ksoftirqd/112:10:461
2086199222,4cyclictest211ksoftirqd/112:10:461
2086199221,17cyclictest4695-21ssh11:50:111
2086199221,16cyclictest25942-21cut07:15:161
2086199221,12cyclictest21-21ksoftirqd/107:50:131
2085399229,8cyclictest12-21ksoftirqd/008:30:150
2085399221,16cyclictest4514-21ls11:50:010
2085399221,16cyclictest24575-21ssh09:55:200
2085399221,10cyclictest22454-21munin-plugin-st12:00:010
2087799219,9cyclictest0-21swapper/309:50:533
20877992113,5cyclictest0-21swapper/309:40:113
2086999218,10cyclictest25827-21kthreadcore08:55:202
2086999214,9cyclictest9747-21fschecks_time10:05:162
2086999213,14cyclictest14180-21kthreadcore10:25:212
2086999212,15cyclictest4918-21irqcore08:00:162
2086999211,7cyclictest0-21swapper/209:20:122
20869992116,3cyclictest19073-21ssh09:50:292
2086199211,4cyclictest0-21swapper/112:05:021
20861992111,7cyclictest17904-21ps10:45:191
2086199211,16cyclictest14042-21apt-get08:10:121
20861992110,8cyclictest211ksoftirqd/109:40:141
20861992110,8cyclictest15324-21apt-get11:20:111
2085399217,3cyclictest131rcu_preempt07:55:120
20853992112,7cyclictest24087-21ssh12:17:390
20853992112,5cyclictest0-21swapper/008:50:120
2085399211,16cyclictest32634-21missed_timers09:40:240
2087799208,9cyclictest35966irq/58-eth012:35:133
2087799202,12cyclictest35966irq/58-eth011:55:123
20877992011,6cyclictest35966irq/58-eth009:25:143
20877992011,6cyclictest0-21swapper/311:00:013
2086999201,9cyclictest7934-21cpu07:30:142
2086999201,9cyclictest7379-21kthreadcore08:35:252
2086999201,9cyclictest12548-21cat08:05:222
2086999201,9cyclictest12548-21cat08:05:222
2086999201,11cyclictest1915-21/usr/sbin/munin09:25:312
2086999201,10cyclictest3171-21kthreadcore09:05:182
2086999201,10cyclictest26365-21fschecks_time10:50:152
2086999201,10cyclictest20045-21kthreadcore10:10:202
2086999200,12cyclictest12082-21kthreadcore09:30:182
2086199209,8cyclictest29909-21df_inode09:00:161
2086199208,3cyclictest131rcu_preempt09:20:161
2086199201,14cyclictest31451-21ssh10:35:121
2086199200,3cyclictest131rcu_preempt08:55:141
2085399202,6cyclictest53646usb-storage07:25:050
2085399202,2cyclictest131rcu_preempt09:55:010
20853992013,4cyclictest0-21swapper/011:50:160
2085399201,2cyclictest131rcu_preempt11:36:000
2085399201,13cyclictest25145-21cut08:55:180
2085399201,13cyclictest23085-21timerwakeupswit11:40:280
2087799192,13cyclictest321rcuc/311:50:113
2087799191,13cyclictest35966irq/58-eth007:35:123
2086999191,8cyclictest30917-21hwlatdetect07:20:162
2086999191,15cyclictest11516-21kthreadcore08:40:202
2086999191,15cyclictest11516-21kthreadcore08:40:202
2086199199,7cyclictest22173-21cron07:45:011
2086199195,9cyclictest0-21swapper/107:30:131
2086199192,3cyclictest131rcu_preempt08:45:131
2086199192,13cyclictest0-21swapper/107:35:131
2086199191,2cyclictest131rcu_preempt10:10:121
2086199191,15cyclictest4231-21kthreadcore07:25:201
2086199191,14cyclictest24350-21ssh11:25:141
2085399192,11cyclictest0-21swapper/012:05:120
20853991911,6cyclictest26934-21ssh09:37:570
20853991911,6cyclictest18594-21cron08:15:020
20853991911,5cyclictest8667-21sh09:10:200
2085399191,14cyclictest9353-21ssh09:45:250
2085399191,10cyclictest131rcu_preempt10:22:370
2087799183,9cyclictest36066irq/59-eth011:10:143
2087799182,12cyclictest35966irq/58-eth009:10:143
2087799182,10cyclictest0-21swapper/307:25:053
20877991812,4cyclictest0-21swapper/309:34:013
20877991811,4cyclictest35966irq/58-eth011:40:223
2087799181,12cyclictest0-21swapper/307:30:273
20877991810,6cyclictest0-21swapper/310:28:443
20877991810,5cyclictest35966irq/58-eth011:05:143
2087799180,14cyclictest35966irq/58-eth010:20:133
2086199187,3cyclictest131rcu_preempt08:55:011
2086199186,3cyclictest131rcu_preempt11:35:211
2086199183,4cyclictest21-21ksoftirqd/111:17:221
2086199183,10cyclictest0-21swapper/110:55:111
2086199181,8cyclictest2003-21cat07:20:281
2086199181,5cyclictest21038-21ntp_192.168.11412:15:241
2086199181,3cyclictest131rcu_preempt12:06:241
2086199181,14cyclictest12025-21ls10:40:281
2086199181,11cyclictest0-21swapper/108:40:141
2086199181,11cyclictest0-21swapper/108:40:131
2085399183,3cyclictest131rcu_preempt08:45:140
20853991812,4cyclictest19465-21kthreadcore07:40:200
20853991812,3cyclictest12-21ksoftirqd/011:25:140
2085399181,14cyclictest22415-21apt-get07:45:120
2085399181,14cyclictest12851-21fschecks_count07:35:150
2087799171,13cyclictest0-21swapper/308:00:133
2087799171,10cyclictest0-21swapper/308:30:283
2087799171,10cyclictest0-21swapper/308:15:053
2086999170,14cyclictest20928-21ls07:10:022
2086199179,5cyclictest20865-21ssh11:05:131
2086199177,7cyclictest21179-21perl07:10:171
2086199173,11cyclictest0-21swapper/110:30:211
2086199172,4cyclictest131rcu_preempt11:00:141
2086199171,8cyclictest27314-21sendmail_mailqu08:20:261
20861991712,3cyclictest0-21swapper/109:30:401
20861991710,4cyclictest2551-21kthreadcore10:00:211
20861991710,4cyclictest0-21swapper/108:25:121
2086199170,6cyclictest28497-21kthreadcore12:20:181
2085399174,10cyclictest2914-21cmp07:25:020
2085399172,2cyclictest131rcu_preempt08:20:190
2085399172,12cyclictest22655-21kthreadcore11:05:200
2085399171,8cyclictest28718-21fschecks_count08:25:160
2085399171,3cyclictest131rcu_preempt07:30:270
2085399171,2cyclictest131rcu_preempt09:00:240
2085399171,2cyclictest131rcu_preempt07:50:250
2085399170,5cyclictest9299-21/usr/sbin/munin08:05:150
2085399170,5cyclictest9299-21/usr/sbin/munin08:05:150
2085399170,2cyclictest131rcu_preempt12:11:170
2085399170,2cyclictest131rcu_preempt12:11:170
2087799162,10cyclictest0-21swapper/307:20:133
2087799161,12cyclictest35966irq/58-eth010:10:123
2086199162,10cyclictest5714-21apt-get08:35:171
2086199162,10cyclictest22704-21ssh11:40:241
2086199161,12cyclictest9406-21perl08:05:151
2086199161,12cyclictest9406-21perl08:05:151
2086199161,12cyclictest18600-21apt-get08:15:051
2085399164,9cyclictest19408-21irqcore08:15:170
2085399161,12cyclictest10524-21apt-get08:40:150
2085399161,12cyclictest10524-21apt-get08:40:140
2087799152,10cyclictest0-21swapper/307:55:143
2087799151,8cyclictest0-21swapper/308:15:013
2087799151,7cyclictest0-21swapper/308:50:013
2086199152,10cyclictest2397-21uniq09:05:161
2087799146,6cyclictest0-21swapper/307:43:393
2087799143,5cyclictest0-21swapper/308:40:143
2087799143,5cyclictest0-21swapper/308:40:143
2087799142,9cyclictest0-21swapper/307:10:153
2087799142,8cyclictest0-21swapper/308:35:133
2087799142,10cyclictest0-21swapper/307:45:133
2087799141,9cyclictest0-21swapper/309:00:153
2085399141,9cyclictest20866-21latency_hist07:10:020
2087799138,3cyclictest0-21swapper/308:21:483
2087799135,6cyclictest33-21ksoftirqd/308:55:183
2087799131,7cyclictest0-21swapper/308:05:163
2087799131,7cyclictest0-21swapper/308:05:153
2087799131,4cyclictest0-21swapper/307:15:293
2087799131,10cyclictest0-21swapper/308:50:193
2086199131,9cyclictest20886-21grep07:10:021
2087799122,6cyclictest0-21swapper/307:50:133
2087799111,7cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional