You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-15 - 18:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Fri May 15, 2026 12:45:15)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
910699623,48cyclictest23409-21ssh12:25:120
911599596,42cyclictest30520-21ssh11:35:111
9106995920,26cyclictest12-21ksoftirqd/007:45:130
910699524,38cyclictest4479-21ssh09:15:130
910699521,37cyclictest10861-21ssh12:00:130
912599502,37cyclictest32422-21ssh12:30:122
911599504,33cyclictest16222-21ssh11:45:131
911599481,39cyclictest27857-21ssh09:10:101
911599472,37cyclictest12065-21ssh09:55:131
910699472,5cyclictest131rcu_preempt08:55:110
910699472,34cyclictest3594-21ssh11:20:120
911599451,29cyclictest22629-21ssh09:25:131
911599451,29cyclictest22629-21ssh09:25:121
910699442,30cyclictest2990-21ssh09:50:130
9133994235,4cyclictest35966irq/58-eth011:45:133
911599429,6cyclictest131rcu_preempt10:45:141
911599429,6cyclictest131rcu_preempt10:45:131
910699425,29cyclictest32412-21ssh11:00:130
910699424,27cyclictest28766-21rm10:40:150
9115994113,24cyclictest21-21ksoftirqd/112:30:121
910699414,6cyclictest131rcu_preempt09:05:150
913399392,20cyclictest35966irq/58-eth011:15:143
913399392,20cyclictest35966irq/58-eth011:15:133
912599392,29cyclictest30209-21ssh10:05:122
911599395,26cyclictest15868-21rm10:15:111
910699392,25cyclictest18014-21ssh11:10:120
910699392,25cyclictest18014-21ssh11:10:110
911599382,26cyclictest13611-21ssh09:20:151
910699383,27cyclictest14347-21ssh10:50:110
910699381,30cyclictest17317-21ssh09:40:110
913399374,22cyclictest35966irq/58-eth012:00:143
912599372,19cyclictest26291-21ssh11:14:412
912599372,19cyclictest26291-21ssh11:14:402
912599371,19cyclictest0-21swapper/207:45:112
912599362,18cyclictest29393-21rm10:04:312
912599360,33cyclictest0-21swapper/208:15:142
912599352,28cyclictest2111-21ssh11:18:442
912599352,28cyclictest2111-21ssh11:18:432
912599352,17cyclictest11202-21ssh10:46:442
912599352,17cyclictest11202-21ssh10:46:432
911599351,3cyclictest131rcu_preempt09:00:141
913399341,23cyclictest35966irq/58-eth009:45:123
912599345,22cyclictest0-21swapper/209:10:112
912599340,22cyclictest15875-21ssh10:15:112
913399333,24cyclictest35966irq/58-eth010:10:133
913399333,21cyclictest35966irq/58-eth009:30:123
913399333,21cyclictest35966irq/58-eth009:30:113
913399331,21cyclictest35966irq/58-eth010:15:113
912599332,27cyclictest8944-21ssh12:35:122
911599336,19cyclictest21-21ksoftirqd/108:15:121
910699331,22cyclictest0-21swapper/012:05:120
913399322,21cyclictest35966irq/58-eth007:15:143
9133993217,9cyclictest35966irq/58-eth012:20:143
9133993214,14cyclictest35966irq/58-eth009:55:133
912599323,18cyclictest27346irq/40-dwc2_hso08:45:132
911599322,24cyclictest1686-21ssh10:25:121
910699325,18cyclictest5337-21ssh12:15:120
910699322,24cyclictest28800-21ssh12:10:140
913399313,17cyclictest35966irq/58-eth008:15:133
913399312,24cyclictest35966irq/58-eth010:05:123
911599312,24cyclictest31659-21ssh09:30:121
911599312,24cyclictest31659-21ssh09:30:111
910699312,22cyclictest12695-21ssh10:11:430
913399306,15cyclictest35966irq/58-eth011:20:113
912599304,14cyclictest27346irq/40-dwc2_hso08:30:112
912599302,19cyclictest26369-21ssh09:45:132
9125993013,12cyclictest10685-21ssh10:30:112
911599303,20cyclictest28785-21ssh12:10:121
913399293,21cyclictest35966irq/58-eth010:25:123
913399293,18cyclictest35966irq/58-eth011:10:123
913399293,18cyclictest35966irq/58-eth011:10:113
9133992914,11cyclictest35966irq/58-eth010:50:123
912599293,6cyclictest271ksoftirqd/210:10:262
912599292,11cyclictest17335-21ssh11:09:402
9125992916,4cyclictest0-21swapper/211:50:112
911599297,16cyclictest21-21ksoftirqd/110:35:121
911599292,21cyclictest8386-21ssh11:04:441
911599292,19cyclictest6821-21ssh10:10:131
910699292,21cyclictest4499-21ssh10:07:440
9106992914,9cyclictest9098-21cyclictest11:15:130
9106992914,9cyclictest9098-21cyclictest11:15:130
913399287,8cyclictest35966irq/58-eth012:10:143
913399283,19cyclictest35966irq/58-eth007:25:123
913399283,18cyclictest0-21swapper/311:05:443
913399281,17cyclictest35966irq/58-eth010:40:143
912599284,14cyclictest27346irq/40-dwc2_hso08:20:122
912599282,18cyclictest27346irq/40-dwc2_hso10:35:132
9125992818,3cyclictest261rcuc/212:23:072
9125992814,8cyclictest536-21usb-storage08:55:102
912599280,18cyclictest27346irq/40-dwc2_hso12:15:132
910699282,20cyclictest23452-21ssh10:18:450
910699282,18cyclictest10704-21ssh10:30:130
9106992821,4cyclictest0-21swapper/010:20:120
9133992714,8cyclictest35966irq/58-eth012:05:253
912599273,4cyclictest27-21ksoftirqd/212:00:202
912599273,13cyclictest0-21swapper/209:37:202
912599271,20cyclictest19778-21ssh12:05:132
9125992710,13cyclictest21557-21ssh11:30:122
911599273,16cyclictest21-21ksoftirqd/108:35:131
911599272,7cyclictest21-21ksoftirqd/108:40:121
911599272,19cyclictest17323-21ssh11:09:401
911599271,18cyclictest25301-21ssh11:50:141
910699272,19cyclictest27229-21ssh10:38:450
913399262,21cyclictest35966irq/58-eth010:45:133
913399262,21cyclictest35966irq/58-eth010:45:123
9133992611,7cyclictest35966irq/58-eth012:15:283
912599266,8cyclictest27-21ksoftirqd/209:15:032
912599265,13cyclictest27346irq/40-dwc2_hso09:20:152
912599263,14cyclictest27346irq/40-dwc2_hso07:35:122
912599263,13cyclictest0-21swapper/210:52:192
912599262,9cyclictest4574-21ssh10:44:342
912599262,21cyclictest0-21swapper/209:50:132
912599262,16cyclictest27346irq/40-dwc2_hso11:25:142
912599262,16cyclictest27346irq/40-dwc2_hso09:30:122
912599262,16cyclictest27346irq/40-dwc2_hso09:30:112
912599261,21cyclictest1829-21ssh11:55:132
912599261,16cyclictest23415-21ssh12:25:112
9115992618,6cyclictest21-21ksoftirqd/108:00:131
910699269,5cyclictest31630-21/usr/sbin/munin09:30:270
910699269,5cyclictest31630-21/usr/sbin/munin09:30:270
913399252,19cyclictest35966irq/58-eth009:35:143
913399252,17cyclictest0-21swapper/310:23:443
912599253,4cyclictest271ksoftirqd/207:50:162
912599252,18cyclictest30044-21ssh10:57:432
912599252,11cyclictest3603-21ssh11:20:132
9125992515,7cyclictest261rcuc/211:37:492
912599251,17cyclictest0-21swapper/208:25:052
912599250,17cyclictest27346irq/40-dwc2_hso09:00:132
9115992512,4cyclictest21-21ksoftirqd/108:50:121
913399243,14cyclictest0-21swapper/310:30:123
912599244,14cyclictest0-21swapper/208:10:092
912599242,18cyclictest0-21swapper/210:25:132
912599242,17cyclictest12066-21ssh09:55:122
912599242,12cyclictest24951-21ssh10:20:132
9125992416,7cyclictest261rcuc/211:42:502
911599242,17cyclictest21562-21ssh11:30:131
911599241,21cyclictest7168-21ssh11:40:131
911599241,19cyclictest5437-21ls12:15:151
911599241,17cyclictest6179-21ssh10:09:431
911599241,17cyclictest23337-21ssh10:55:121
910699249,11cyclictest28333-21ssh09:10:170
910699242,17cyclictest844-21munin-node10:45:140
910699242,17cyclictest844-21munin-node10:45:130
910699240,4cyclictest131rcu_preempt11:50:120
913399233,16cyclictest35966irq/58-eth011:40:133
913399233,13cyclictest0-21swapper/311:04:433
913399232,13cyclictest0-21swapper/311:55:123
913399232,11cyclictest35966irq/58-eth008:20:123
912599239,6cyclictest1944-21cron08:10:022
912599235,8cyclictest27-21ksoftirqd/212:11:052
912599235,7cyclictest27-21ksoftirqd/211:45:422
912599233,4cyclictest27-21ksoftirqd/209:40:262
9125992314,7cyclictest261rcuc/209:26:502
9125992314,7cyclictest261rcuc/209:26:492
912599231,19cyclictest0-21swapper/208:40:132
912599231,15cyclictest20351-21cut07:55:162
912599231,11cyclictest1262-21cut11:00:192
911599234,4cyclictest131rcu_preempt11:10:131
911599234,4cyclictest131rcu_preempt11:10:131
911599233,15cyclictest21-21ksoftirqd/108:30:131
9115992310,3cyclictest21-21ksoftirqd/107:40:101
910699236,13cyclictest0-21swapper/007:20:140
910699232,17cyclictest1840-21ssh11:55:130
910699231,18cyclictest28462-21find07:30:130
913399221,13cyclictest0-21swapper/308:50:113
9125992215,4cyclictest22907-21apt-get09:05:132
911599221,8cyclictest13817-21kthreadcore12:00:211
9115992212,3cyclictest0-21swapper/110:22:191
9115992210,3cyclictest21-21ksoftirqd/111:28:371
910699223,11cyclictest8341-21ssh09:35:120
910699221,18cyclictest23766-21ls07:25:150
913399218,8cyclictest0-21swapper/312:25:123
913399215,9cyclictest0-21swapper/311:25:153
9133992112,6cyclictest0-21swapper/312:30:143
912599214,14cyclictest30984-21latency_hist08:40:022
912599211,11cyclictest21434-21kthreadcore07:20:222
912599211,10cyclictest25051-21ls08:00:142
912599210,4cyclictest27-21ksoftirqd/207:40:162
912599210,10cyclictest17737-21switchtime07:15:282
911599219,3cyclictest21-21ksoftirqd/112:26:551
911599218,3cyclictest21-21ksoftirqd/110:42:441
911599213,7cyclictest21-21ksoftirqd/109:35:161
911599212,14cyclictest9319-21kworker/1:207:15:151
911599211,3cyclictest0-21swapper/110:32:391
9115992113,5cyclictest0-21swapper/110:50:151
9115992111,7cyclictest8135-21ntpq09:15:261
911599211,16cyclictest595-21latency_hist07:35:011
910699219,3cyclictest131rcu_preempt09:20:220
910699218,4cyclictest131rcu_preempt11:40:200
910699211,15cyclictest29939-21fschecks_count08:05:160
910699211,15cyclictest24866-21ssh10:00:250
910699211,12cyclictest14455-21ssh12:20:150
910699210,15cyclictest2378-21ssh10:25:180
913399209,6cyclictest0-21swapper/309:52:113
913399203,12cyclictest0-21swapper/310:04:303
9133992012,4cyclictest0-21swapper/310:55:133
912599206,5cyclictest26099-21kthreadcore07:25:202
912599203,4cyclictest27-21ksoftirqd/208:50:182
912599201,4cyclictest27-21ksoftirqd/207:10:182
912599201,4cyclictest27-21ksoftirqd/207:10:182
9125992011,6cyclictest28477-21apt-get07:30:152
911599209,3cyclictest21-21ksoftirqd/109:40:421
911599209,3cyclictest131rcu_preempt12:35:181
911599209,2cyclictest21-21ksoftirqd/111:16:461
911599209,2cyclictest21-21ksoftirqd/111:16:451
911599207,3cyclictest21-21ksoftirqd/109:50:561
9115992014,4cyclictest0-21swapper/110:00:161
9115992014,4cyclictest0-21swapper/107:50:141
9115992012,6cyclictest18062-21ssh12:20:261
911599201,13cyclictest30258-21ls09:45:291
910699204,3cyclictest131rcu_preempt07:40:020
910699204,10cyclictest0-21swapper/008:35:130
910699202,15cyclictest9546-21idleruntime07:10:160
910699202,15cyclictest9546-21idleruntime07:10:150
910699202,14cyclictest8945-21sh12:35:120
913399199,8cyclictest0-21swapper/310:37:273
913399193,11cyclictest0-21swapper/311:33:443
913399192,11cyclictest0-21swapper/308:45:053
913399192,10cyclictest0-21swapper/307:50:123
913399191,13cyclictest0-21swapper/309:25:143
913399191,13cyclictest0-21swapper/309:25:133
913399191,12cyclictest0-21swapper/308:25:153
911599191,9cyclictest23715-21wc07:55:261
911599191,3cyclictest0-21swapper/111:58:351
911599191,2cyclictest0-21swapper/111:24:331
9115991912,4cyclictest21-21ksoftirqd/112:05:141
9115991911,5cyclictest1-21systemd07:13:571
9115991911,5cyclictest1-21systemd07:13:561
911599190,15cyclictest910-21cpuspeed07:35:141
910699192,13cyclictest12594-21ssh11:25:150
910699192,11cyclictest14055-21ssh11:05:440
910699191,8cyclictest23736-21irqcore10:55:170
910699191,15cyclictest22792-21cut09:25:160
910699191,15cyclictest22792-21cut09:25:150
910699191,14cyclictest27476-21diskmemload09:45:140
9106991910,7cyclictest16210-21apt-get11:45:120
910699190,14cyclictest5833-21fschecks_count07:40:140
913399189,6cyclictest0-21swapper/309:15:273
913399183,11cyclictest0-21swapper/309:20:143
913399182,12cyclictest0-21swapper/312:35:113
913399182,10cyclictest0-21swapper/309:00:143
913399182,10cyclictest0-21swapper/308:55:053
913399182,10cyclictest0-21swapper/308:35:133
913399182,10cyclictest0-21swapper/308:10:123
913399182,10cyclictest0-21swapper/308:05:143
913399182,10cyclictest0-21swapper/308:00:133
913399182,10cyclictest0-21swapper/307:55:053
9133991811,5cyclictest0-21swapper/311:50:303
913399181,13cyclictest0-21swapper/307:20:223
911599188,2cyclictest21-21ksoftirqd/109:05:231
911599183,9cyclictest0-21swapper/107:48:401
911599181,14cyclictest29814-21find08:05:111
910699185,10cyclictest3511-21gpgv08:45:020
910699182,4cyclictest53646usb-storage08:15:110
910699182,13cyclictest14122-21apt-get07:15:120
910699181,9cyclictest2239-21kthreadcore12:30:190
910699181,9cyclictest22271-21tr08:30:180
910699181,8cyclictest812-21kthreadcore11:35:210
913399179,6cyclictest0-21swapper/309:41:293
913399179,5cyclictest0-21swapper/309:10:263
913399172,10cyclictest0-21swapper/308:40:143
913399172,10cyclictest0-21swapper/307:35:143
913399172,10cyclictest0-21swapper/307:30:143
912599171,11cyclictest9223-21latency_hist07:10:012
911599171,3cyclictest0-21swapper/108:20:141
9115991712,4cyclictest0-21swapper/108:25:301
9115991712,3cyclictest536-21usb-storage08:55:051
910699173,11cyclictest26082-21kthreadcore08:00:180
910699173,11cyclictest20617-21grep08:25:280
910699171,8cyclictest19643-21kthreadcore09:00:210
910699171,3cyclictest131rcu_preempt09:55:150
9106991711,4cyclictest25235-21ntp_offset11:30:250
910699171,12cyclictest16457-21apt-get08:25:020
910699171,11cyclictest20075-21cat07:55:010
913399168,5cyclictest35966irq/58-eth011:35:123
913399161,8cyclictest0-21swapper/309:05:133
911599168,6cyclictest23455-21cron07:25:031
911599167,7cyclictest21-21ksoftirqd/108:10:101
911599161,4cyclictest53646usb-storage08:45:051
911599161,10cyclictest0-21swapper/107:20:131
910699169,4cyclictest4150-21cut08:45:180
910699161,6cyclictest8761-21irqcore08:50:170
910699161,12cyclictest2798-21irqrtprio08:10:150
910699161,12cyclictest19854-21apt-get07:55:030
913399152,10cyclictest0-21swapper/307:40:123
913399142,9cyclictest0-21swapper/308:30:133
911599142,9cyclictest9049-21latency_hist07:10:011
910699142,10cyclictest9098-21cyclictest07:10:020
913399131,7cyclictest0-21swapper/307:15:013
913399131,7cyclictest0-21swapper/307:15:003
913399121,9cyclictest0-21swapper/307:45:183
913399101,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional