You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-18 - 14:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Apr 18, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
25857997056,10cyclictest35966irq/58-eth009:30:133
2585799555,35cyclictest35966irq/58-eth010:10:133
25857994829,8cyclictest35966irq/58-eth010:05:133
2583399481,31cyclictest23356-21ssh10:10:120
25857994612,24cyclictest35966irq/58-eth009:35:133
2585799441,27cyclictest35966irq/58-eth008:30:153
2584199442,33cyclictest21301-21ssh09:15:161
25833994338,3cyclictest11937-21diskmemload09:35:130
2583399423,34cyclictest17968-21ssh11:55:120
2585799392,22cyclictest35966irq/58-eth011:05:143
2584999392,28cyclictest25106-21ssh12:35:142
2584199391,31cyclictest6910-21ssh09:25:121
2584999385,15cyclictest131rcu_preempt09:10:162
2584999361,22cyclictest14241-21sh10:05:122
2584199363,5cyclictest131rcu_preempt07:10:121
2583399364,7cyclictest131rcu_preempt07:40:150
2585799353,21cyclictest35966irq/58-eth009:10:153
2585799352,19cyclictest35966irq/58-eth009:05:133
2585799351,21cyclictest35966irq/58-eth010:00:123
2584999356,12cyclictest27-21ksoftirqd/211:46:552
2585799333,15cyclictest35966irq/58-eth011:10:123
2585799332,17cyclictest35966irq/58-eth011:20:123
2585799325,15cyclictest35966irq/58-eth010:35:153
2585799322,19cyclictest35966irq/58-eth011:40:133
2584999312,24cyclictest19652-21ssh11:36:052
2584999303,22cyclictest0-21swapper/211:13:572
2584999302,17cyclictest27346irq/40-dwc2_hso12:20:122
2584999301,18cyclictest27346irq/40-dwc2_hso08:40:122
2583399302,21cyclictest16061-21ssh11:00:140
2585799292,19cyclictest35966irq/58-eth010:45:123
2585799292,13cyclictest35966irq/58-eth012:35:133
2584999293,21cyclictest0-21swapper/212:12:562
2584999292,20cyclictest7103-21ssh10:55:132
2584999292,20cyclictest0-21swapper/207:20:142
25849992915,7cyclictest27-21ksoftirqd/209:40:122
25849992915,7cyclictest27-21ksoftirqd/209:40:112
25857992813,9cyclictest35966irq/58-eth012:30:143
2584999284,16cyclictest27346irq/40-dwc2_hso08:20:112
2584999284,15cyclictest27346irq/40-dwc2_hso11:30:112
2583399281,4cyclictest1296-21munin-run11:10:010
2585799272,17cyclictest35966irq/58-eth010:40:123
2585799272,14cyclictest35966irq/58-eth008:20:133
2584999277,10cyclictest19603-21sort11:00:242
2584999275,15cyclictest0-21swapper/207:35:122
2584999273,20cyclictest0-21swapper/212:09:572
2584999273,19cyclictest0-21swapper/211:18:562
2584999273,14cyclictest27346irq/40-dwc2_hso08:25:132
2584999272,8cyclictest30979-21ssh11:43:562
2584999272,18cyclictest0-21swapper/211:50:132
2584999272,18cyclictest0-21swapper/208:15:152
25849992719,4cyclictest27-21ksoftirqd/210:10:132
2583399273,20cyclictest12-21ksoftirqd/007:10:120
2585799261,13cyclictest35966irq/58-eth011:15:133
2584999262,9cyclictest15636-21ssh12:29:572
2584999262,10cyclictest0-21swapper/208:45:142
2584199262,19cyclictest28454-21sh12:17:561
2583399262,19cyclictest28455-21rm12:17:560
2583399261,3cyclictest131rcu_preempt08:20:100
2583399260,23cyclictest18077-21sh10:25:140
2583399260,20cyclictest10708-21ssh09:45:220
2583399260,20cyclictest10708-21ssh09:45:210
2584999258,11cyclictest0-21swapper/208:05:162
2584999252,8cyclictest2244-21ssh12:03:562
2584999252,17cyclictest27346irq/40-dwc2_hso09:30:132
2584999252,14cyclictest27346irq/40-dwc2_hso07:30:152
2584999251,9cyclictest28675-21ssh09:55:132
2584999251,16cyclictest30328-21grep08:55:172
2584999251,16cyclictest30328-21grep08:55:162
2584999251,14cyclictest3776-21ssh10:35:162
2584999251,14cyclictest19645-21ssh09:50:172
2584999251,10cyclictest32515-21cpuspeed10:15:142
2584999250,16cyclictest27346irq/40-dwc2_hso08:10:132
2583399252,17cyclictest30312-21ssh09:20:140
25833992512,4cyclictest11937-21diskmemload10:05:200
2583399251,19cyclictest21988-21ssh11:38:560
2583399251,15cyclictest1594-21/usr/sbin/munin09:40:140
2583399251,15cyclictest1594-21/usr/sbin/munin09:40:130
2585799242,14cyclictest35966irq/58-eth010:20:113
2584999248,12cyclictest10538-21kthreadcore08:00:202
2584999248,12cyclictest10538-21kthreadcore08:00:192
25849992412,9cyclictest29139-21ssh10:48:502
2583399242,19cyclictest9074-21ssh10:20:130
2583399242,18cyclictest8959-21apt-get11:50:130
2583399241,20cyclictest32438-21sh10:15:150
2583399241,18cyclictest5252-21ssh11:30:120
2583399241,17cyclictest28769-21ssh11:25:130
2585799237,11cyclictest0-21swapper/311:36:053
2585799237,10cyclictest0-21swapper/312:17:563
25857992313,7cyclictest35966irq/58-eth012:20:143
2584999235,15cyclictest18473-21irqcore10:25:182
2584999234,13cyclictest27346irq/40-dwc2_hso11:25:122
2584999233,11cyclictest9802-21kthreadcore10:20:192
2584999232,6cyclictest0-21swapper/211:06:552
2584999232,17cyclictest24679-21sh12:35:012
2584999232,15cyclictest0-21swapper/211:20:052
25849992312,9cyclictest28648-21ssh12:18:112
2584999231,15cyclictest0-21swapper/208:30:042
2584199231,18cyclictest7341-21ssh10:35:271
25841992311,7cyclictest21-21ksoftirqd/108:10:131
2583399235,4cyclictest131rcu_preempt12:05:190
2583399231,7cyclictest23465-21ntp_kernel_pll_11:20:240
25833992312,8cyclictest10643-21awk09:25:250
25833992312,5cyclictest12-21ksoftirqd/007:35:120
2583399231,17cyclictest15914-21ssh09:30:130
25833992310,3cyclictest131rcu_preempt10:55:130
2585799227,10cyclictest35966irq/58-eth012:05:143
2584999223,16cyclictest27851-21kthreadcore10:30:182
2584999223,16cyclictest22176-21tr09:15:202
2584999223,11cyclictest8674-21kthreadcore09:05:202
2584999222,9cyclictest26171-21kthreadcore09:35:202
2584999222,8cyclictest0-21swapper/207:45:132
2584999222,15cyclictest0-21swapper/210:50:132
25849992215,4cyclictest27-21ksoftirqd/210:00:142
25849992211,9cyclictest12877-21ssh09:26:482
2584999221,15cyclictest32620-21kthreadcore09:20:232
2584999221,13cyclictest27346irq/40-dwc2_hso10:40:122
2584999221,10cyclictest30561-21sh07:15:022
2584199221,19cyclictest16099-21ssh12:30:141
2583399221,17cyclictest27558-21cut07:45:160
2583399220,5cyclictest7842-21kthreadcore10:00:220
2583399220,17cyclictest3072-21ssh12:04:550
2585799214,8cyclictest35966irq/58-eth011:55:143
2584999215,13cyclictest14722-21unixbench_singl08:35:292
2584999212,9cyclictest5632-21kthreadcore07:55:182
2584999212,4cyclictest0-21swapper/207:25:142
2584999212,13cyclictest27346irq/40-dwc2_hso08:50:142
2584999212,11cyclictest25906-21latency_hist07:10:012
2584999211,10cyclictest1678-21kthreadcore07:15:232
2584999211,10cyclictest1678-21kthreadcore07:15:222
2584199214,11cyclictest201rcuc/107:40:121
2584199212,15cyclictest13294-21irqstats10:40:181
25841992114,4cyclictest131rcu_preempt11:35:141
25841992112,6cyclictest24940-21cat10:10:191
2584199211,12cyclictest11662-21cut11:50:211
25841992111,2cyclictest0-21swapper/111:29:171
2584199211,11cyclictest0-21swapper/109:47:541
2584199211,11cyclictest0-21swapper/109:47:531
2583399216,4cyclictest29394-21kernelversion09:55:180
2583399212,14cyclictest14027-21apt-get08:05:030
2583399211,3cyclictest131rcu_preempt12:35:120
2583399211,17cyclictest27680-21ls08:15:290
25833992111,6cyclictest0-21swapper/008:10:150
2583399211,13cyclictest20676-21cut08:45:160
25833992110,9cyclictest29886-21cron09:20:010
2583399210,3cyclictest131rcu_preempt09:50:140
2583399210,14cyclictest3295-21sshd07:20:140
2585799202,11cyclictest0-21swapper/307:35:163
2585799201,12cyclictest0-21swapper/312:10:133
2584999206,6cyclictest11748-21kthreadcore09:45:292
2584999206,6cyclictest11748-21kthreadcore09:45:282
2584999204,7cyclictest22517-21apt-get07:40:152
2584999202,9cyclictest2729-21fschecks_count09:00:162
2584999201,9cyclictest2163-21kthreadcore07:50:222
2584999201,16cyclictest19823-21kthreadcore11:55:202
2584199209,3cyclictest131rcu_preempt10:50:261
2584199204,9cyclictest0-21swapper/111:55:131
2584199204,2cyclictest131rcu_preempt10:45:531
2584199202,14cyclictest1201-21apt-get08:25:121
25841992015,3cyclictest0-21swapper/112:35:141
25841992012,6cyclictest30073-21sshd09:35:491
2584199201,14cyclictest27376-21ls10:30:171
2584199201,13cyclictest16055-21sh11:00:131
2583399201,6cyclictest3390-21kthreadcore11:10:190
2583399201,6cyclictest13159-21ps10:40:170
2583399201,4cyclictest0-21swapper/007:50:170
2583399201,2cyclictest131rcu_preempt08:30:150
2583399201,17cyclictest8308-21timerandwakeup07:55:270
2583399201,16cyclictest30641-21kworker/u8:011:40:140
2583399201,14cyclictest3665-21ssh10:35:150
25833992010,3cyclictest11937-21diskmemload10:50:130
2585799199,7cyclictest0-21swapper/310:55:223
2585799199,6cyclictest0-21swapper/312:26:373
2585799198,8cyclictest0-21swapper/309:16:553
2585799193,9cyclictest35966irq/58-eth009:45:243
2585799193,9cyclictest35966irq/58-eth009:45:233
2585799192,12cyclictest0-21swapper/311:46:553
25857991911,6cyclictest0-21swapper/309:55:013
25857991910,7cyclictest35966irq/58-eth011:00:123
25857991910,6cyclictest0-21swapper/311:30:183
2584199199,6cyclictest15712-21cut12:10:221
2584199195,12cyclictest3574-21ssh12:05:141
2584199194,3cyclictest131rcu_preempt11:15:241
2584199192,12cyclictest25819-21cyclictest09:10:141
2584199191,8cyclictest9329-21grep10:55:231
2584199191,6cyclictest16564-21xargs07:30:291
25841991914,3cyclictest0-21swapper/109:00:171
2584199191,16cyclictest1578-21ssh09:40:131
2584199191,16cyclictest1578-21ssh09:40:121
25841991911,5cyclictest5204-21/usr/sbin/munin11:30:281
2584199191,14cyclictest18767-21irqrtprio10:25:181
2584199191,14cyclictest12827-21ps10:20:271
2583399193,3cyclictest131rcu_preempt09:05:310
2583399193,3cyclictest131rcu_preempt07:15:130
2583399193,3cyclictest131rcu_preempt07:15:130
2583399191,6cyclictest11937-21diskmemload10:30:130
2583399191,4cyclictest5753-21apt08:30:000
2583399191,4cyclictest0-21swapper/008:40:290
2583399191,3cyclictest131rcu_preempt09:00:140
2583399191,2cyclictest131rcu_preempt11:47:550
2583399191,15cyclictest7190-21ssh12:25:130
25833991910,7cyclictest20445-21cat12:30:300
2583399190,8cyclictest26002-21sed08:50:180
2585799189,6cyclictest0-21swapper/309:40:263
2585799189,6cyclictest0-21swapper/309:40:263
2585799183,9cyclictest35966irq/58-eth007:20:133
2585799182,9cyclictest0-21swapper/307:45:133
2585799182,8cyclictest0-21swapper/308:40:103
2585799182,12cyclictest0-21swapper/312:04:553
2585799182,10cyclictest35966irq/58-eth010:50:123
2585799182,10cyclictest0-21swapper/311:50:043
2585799182,10cyclictest0-21swapper/309:55:043
2585799182,10cyclictest0-21swapper/308:10:043
2585799182,10cyclictest0-21swapper/308:05:153
2585799182,10cyclictest0-21swapper/307:55:143
2585799182,10cyclictest0-21swapper/307:25:143
2585799182,10cyclictest0-21swapper/307:15:053
2585799182,10cyclictest0-21swapper/307:15:043
25857991811,5cyclictest0-21swapper/309:25:183
2585799181,11cyclictest0-21swapper/311:25:133
2585799181,11cyclictest0-21swapper/308:35:173
25857991810,6cyclictest0-21swapper/310:26:583
25857991810,6cyclictest0-21swapper/310:19:333
25857991810,6cyclictest0-21swapper/309:20:323
2584199186,9cyclictest11686-21kthreadcore08:35:201
2584199182,8cyclictest0-21swapper/107:35:291
2584199182,3cyclictest4515-21ssh12:22:171
2584199182,13cyclictest7830-21kthreadcore10:00:211
2584199181,8cyclictest16349-21kthreadcore10:05:191
2584199181,6cyclictest19046-21missed_timers08:40:201
2584199181,3cyclictest131rcu_preempt11:45:181
2584199181,3cyclictest131rcu_preempt09:05:281
25841991812,4cyclictest0-21swapper/111:05:251
25841991811,5cyclictest0-21swapper/111:20:101
2584199181,14cyclictest25328-21grep08:50:021
2584199181,13cyclictest32437-21ssh10:15:141
2584199181,13cyclictest28665-21ssh09:55:131
2584199181,13cyclictest26997-21ssh12:00:131
2584199181,10cyclictest1595-21ntpq09:20:261
2583399186,9cyclictest12045-21kthreadcore08:35:200
2583399182,3cyclictest131rcu_preempt12:22:360
2583399181,7cyclictest21561-21cpu10:45:140
2583399181,4cyclictest0-21swapper/009:11:320
2583399181,4cyclictest0-21swapper/007:30:200
2583399181,11cyclictest11937-21diskmemload11:18:560
2583399180,15cyclictest12593-21ssh12:10:120
2585799179,6cyclictest0-21swapper/310:30:113
2585799179,5cyclictest35966irq/58-eth008:00:153
2585799179,5cyclictest35966irq/58-eth008:00:153
2585799172,11cyclictest35966irq/58-eth008:25:123
2585799172,10cyclictest0-21swapper/307:50:143
2585799171,12cyclictest0-21swapper/308:15:143
2584199179,6cyclictest2336-21cron09:00:011
2584199179,6cyclictest2336-21cron09:00:001
2584199174,10cyclictest31438-21irqrtprio07:15:191
2584199174,10cyclictest31438-21irqrtprio07:15:181
2584199173,11cyclictest19644-21diskmemload09:50:161
2584199173,10cyclictest20028-21sed09:30:281
2584199171,13cyclictest23841-21irqrtprio11:40:171
25841991710,5cyclictest27482-21ls07:45:161
25841991710,5cyclictest12640-21sshd07:29:571
25841991710,4cyclictest7693-21irqcore12:25:181
2584199170,4cyclictest53646usb-storage08:50:101
2583399171,5cyclictest9730-21kthreadcore07:25:210
2583399171,2cyclictest131rcu_preempt08:00:160
2583399171,2cyclictest131rcu_preempt08:00:150
2585799162,9cyclictest0-21swapper/308:55:153
2585799162,9cyclictest0-21swapper/308:55:143
2585799162,9cyclictest0-21swapper/308:50:033
2585799161,12cyclictest0-21swapper/308:45:283
2584199164,9cyclictest7054-21tr07:55:211
2584199162,12cyclictest32421-21fschecks_time07:50:161
2584199162,11cyclictest14572-21idleruntime08:05:181
2584199161,8cyclictest11036-21kthreadcore08:00:211
2584199161,8cyclictest11036-21kthreadcore08:00:201
2584199161,2cyclictest131rcu_preempt08:30:201
2584199161,12cyclictest24376-21irqcore08:15:181
2584199161,11cyclictest1649-21acpi11:10:111
2583399164,9cyclictest32358-21kthreadcore08:55:210
2583399164,9cyclictest32358-21kthreadcore08:55:210
25857991510,3cyclictest0-21swapper/307:42:563
2584199151,10cyclictest135-21systemd-journal07:20:121
2584199150,11cyclictest31297-21tr08:20:201
2584199141,9cyclictest25867-21grep07:10:021
2583399131,10cyclictest25855-21latency_hist07:10:010
2585799127,3cyclictest0-21swapper/309:00:503
2585799123,7cyclictest0-21swapper/307:10:213
2585799122,7cyclictest0-21swapper/307:30:143
258579991,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional