You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-10 - 18:33

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Thu Sep 10, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
314998518,10cyclictest131rcu_preempt09:45:120
32499552,39cyclictest31726-21ssh09:35:101
314995513,8cyclictest131rcu_preempt08:45:110
31499532,41cyclictest5810-21ssh12:20:130
32499473,4cyclictest131rcu_preempt09:25:111
32499472,36cyclictest22713-21ssh09:30:121
32499453,7cyclictest131rcu_preempt07:50:121
31499433,31cyclictest5507-21ssh10:50:140
32499411,20cyclictest14852-21ssh12:25:151
31499392,28cyclictest2109-21ssh12:00:120
35099381,11cyclictest35966irq/58-eth012:10:133
31499385,17cyclictest0-21swapper/007:40:120
314993824,10cyclictest151rcuc/012:35:140
33599376,21cyclictest27-21ksoftirqd/212:25:142
33599372,19cyclictest11143-21ntp_states11:45:252
33599362,17cyclictest24772-21rm09:48:252
32499361,28cyclictest21400-21ssh10:05:141
31499363,22cyclictest12-21ksoftirqd/008:15:120
33599355,19cyclictest0-21swapper/209:30:122
35099341,6cyclictest35966irq/58-eth009:40:123
32499347,15cyclictest21-21ksoftirqd/110:00:121
32499342,20cyclictest4599-21ssh09:20:121
32499340,25cyclictest3275-21ssh09:55:131
33599337,17cyclictest27-21ksoftirqd/207:50:122
35099322,16cyclictest35966irq/58-eth012:15:103
35099321,3cyclictest0-21swapper/307:45:143
35099321,3cyclictest0-21swapper/307:45:133
33599321,18cyclictest13632-21ssh09:25:112
32499323,2cyclictest131rcu_preempt08:25:121
35099315,15cyclictest35966irq/58-eth009:15:123
35099311,14cyclictest35966irq/58-eth009:50:133
33599316,15cyclictest27346irq/40-dwc2_hso09:35:112
33599316,13cyclictest27346irq/40-dwc2_hso09:05:122
335993110,10cyclictest11162-21diskmemload12:05:152
31499312,25cyclictest26513-21ssh09:50:130
35099304,18cyclictest35966irq/58-eth007:20:133
35099303,16cyclictest35966irq/58-eth011:10:163
35099300,17cyclictest35966irq/58-eth010:55:133
33599306,11cyclictest27346irq/40-dwc2_hso09:15:122
33599304,18cyclictest27346irq/40-dwc2_hso10:15:122
33599302,22cyclictest5445-21ssh10:30:242
32499303,21cyclictest17429-21ssh09:45:131
31499305,21cyclictest8423-21ssh09:40:130
31499302,22cyclictest11930-21ssh10:15:300
35099292,14cyclictest35966irq/58-eth011:25:123
35099292,14cyclictest35966irq/58-eth011:25:123
350992917,8cyclictest35966irq/58-eth012:25:143
32499290,19cyclictest0-21swapper/109:05:131
31499293,19cyclictest18194-21ssh11:15:140
32499282,20cyclictest11163-21rm12:05:141
324992812,12cyclictest21-21ksoftirqd/107:25:101
31499287,12cyclictest12-21ksoftirqd/010:20:120
31499281,21cyclictest1443-21ssh10:30:010
350992710,13cyclictest35966irq/58-eth011:45:133
33599276,12cyclictest27346irq/40-dwc2_hso11:10:142
33599273,10cyclictest0-21swapper/210:10:302
33599272,16cyclictest27346irq/40-dwc2_hso07:25:122
33599272,14cyclictest27346irq/40-dwc2_hso11:40:122
32499272,19cyclictest30399-21/usr/sbin/munin10:10:301
31499274,17cyclictest14493-21ssh10:55:140
31499272,19cyclictest3365-21ssh09:35:250
35099267,16cyclictest35966irq/58-eth009:25:263
35099263,15cyclictest0-21swapper/309:35:253
35099261,16cyclictest0-21swapper/311:05:113
33599264,12cyclictest27346irq/40-dwc2_hso07:20:122
32499263,17cyclictest27246-21ssh11:20:131
324992614,4cyclictest7864-21irqcore11:45:171
32499261,21cyclictest18463-21diskmemload11:30:171
31499263,18cyclictest151rcuc/009:55:140
31499261,19cyclictest12-21ksoftirqd/007:35:100
35099259,4cyclictest0-21swapper/309:23:373
35099251,3cyclictest0-21swapper/310:30:193
33599255,13cyclictest27346irq/40-dwc2_hso11:30:162
33599252,17cyclictest0-21swapper/209:40:252
33599252,15cyclictest27346irq/40-dwc2_hso07:15:122
33599252,15cyclictest27346irq/40-dwc2_hso07:15:122
335992513,9cyclictest0-21swapper/208:35:162
33599250,15cyclictest3048-21find08:20:112
32499251,19cyclictest2271-21ssh09:54:251
32499251,12cyclictest18468-21apt-get09:10:031
31499253,5cyclictest131rcu_preempt09:15:150
314992514,8cyclictest0-21swapper/007:20:280
35099248,4cyclictest0-21swapper/311:16:373
35099247,14cyclictest0-21swapper/311:43:323
35099243,14cyclictest0-21swapper/310:00:123
35099242,8cyclictest0-21swapper/311:20:133
35099241,6cyclictest0-21swapper/310:20:203
35099241,6cyclictest0-21swapper/308:00:293
35099241,3cyclictest0-21swapper/310:10:143
350992413,4cyclictest0-21swapper/312:05:133
33599249,12cyclictest20891-21kthreadcore12:10:182
33599242,11cyclictest25178-21ssh10:25:132
33599241,12cyclictest21407-21sh10:05:132
32499248,10cyclictest630-21Async09:36:311
32499242,17cyclictest18463-21diskmemload11:10:161
32499241,16cyclictest21-21ksoftirqd/110:30:121
31499244,13cyclictest12-21ksoftirqd/008:40:130
314992415,6cyclictest0-21swapper/011:40:150
35099239,10cyclictest0-21swapper/312:35:293
35099236,3cyclictest35966irq/58-eth012:20:373
35099235,5cyclictest35966irq/58-eth010:40:173
35099231,7cyclictest0-21swapper/310:50:303
35099231,7cyclictest0-21swapper/310:05:313
35099231,7cyclictest0-21swapper/309:55:213
35099231,6cyclictest0-21swapper/312:00:253
35099231,6cyclictest0-21swapper/310:35:193
35099231,6cyclictest0-21swapper/309:45:173
35099231,6cyclictest0-21swapper/309:30:273
35099231,6cyclictest0-21swapper/308:45:153
350992310,8cyclictest35966irq/58-eth011:30:193
33599234,14cyclictest27346irq/40-dwc2_hso10:40:132
33599231,8cyclictest11518-21fschecks_time07:55:152
335992311,9cyclictest0-21swapper/210:20:112
335992311,5cyclictest9422-21ssh12:03:262
32499231,18cyclictest4252-21find08:55:121
32499231,17cyclictest16313-21cut08:00:161
31499232,17cyclictest1857-21ssh10:30:130
31499232,14cyclictest32526-21ssh11:05:110
31499231,18cyclictest18463-21diskmemload10:10:130
31499231,16cyclictest16531-21ssh11:50:120
31499231,16cyclictest16531-21ssh11:50:120
35099225,8cyclictest0-21swapper/307:15:133
35099225,8cyclictest0-21swapper/307:15:123
35099221,6cyclictest0-21swapper/311:55:013
35099221,6cyclictest0-21swapper/311:55:013
35099221,6cyclictest0-21swapper/311:00:273
35099221,6cyclictest0-21swapper/310:45:223
35099221,6cyclictest0-21swapper/310:25:263
35099221,6cyclictest0-21swapper/310:15:153
35099221,6cyclictest0-21swapper/309:10:253
35099221,6cyclictest0-21swapper/309:05:163
35099221,6cyclictest0-21swapper/308:55:153
35099221,6cyclictest0-21swapper/308:30:173
35099221,6cyclictest0-21swapper/308:25:283
35099221,6cyclictest0-21swapper/308:20:253
35099221,6cyclictest0-21swapper/308:15:193
35099221,6cyclictest0-21swapper/308:15:013
35099221,6cyclictest0-21swapper/307:40:153
35099221,6cyclictest0-21swapper/307:35:013
35099221,6cyclictest0-21swapper/307:25:193
35099221,14cyclictest0-21swapper/311:35:123
35099221,13cyclictest35966irq/58-eth008:40:133
33599226,13cyclictest465-21sort07:10:112
33599225,11cyclictest27346irq/40-dwc2_hso10:35:142
33599223,9cyclictest32381-21latency_hist12:35:022
33599223,9cyclictest32381-21latency_hist12:35:022
33599221,17cyclictest27032-21ssh11:00:232
335992211,4cyclictest0-21swapper/211:20:222
33599221,12cyclictest27695-21ssh09:15:022
33599221,11cyclictest29377-21cut10:45:182
33599221,10cyclictest7389-21ntp_192.168.11411:25:242
33599221,10cyclictest7389-21ntp_192.168.11411:25:242
335992210,9cyclictest20217-21ssh10:04:042
32499223,14cyclictest8399-21ssh09:40:121
32499221,18cyclictest28882-21rm10:45:131
32499221,10cyclictest0-21swapper/112:35:221
324992210,3cyclictest131rcu_preempt11:35:191
314992216,4cyclictest0-21swapper/008:55:260
314992215,4cyclictest6606-21grep08:20:250
31499221,17cyclictest21847-21ssh11:35:130
31499221,16cyclictest12-21ksoftirqd/011:45:130
31499221,15cyclictest18815-21ssh09:10:040
31499220,16cyclictest13886-21apt-get09:05:120
35099211,6cyclictest0-21swapper/312:30:193
35099211,6cyclictest0-21swapper/312:30:193
35099211,6cyclictest0-21swapper/311:55:193
35099211,6cyclictest0-21swapper/309:00:163
35099211,6cyclictest0-21swapper/307:55:273
35099211,6cyclictest0-21swapper/307:50:193
35099211,6cyclictest0-21swapper/307:35:203
35099211,6cyclictest0-21swapper/307:10:203
33599212,10cyclictest25983-21kthreadcore07:35:192
33599212,10cyclictest13830-21latency_hist09:05:032
33599211,9cyclictest4877-21cut09:20:162
33599211,8cyclictest32580-21ssh11:05:132
33599211,16cyclictest20205-21ps07:30:172
33599211,11cyclictest27222-21uname09:50:182
33599211,11cyclictest16559-21ssh11:50:142
33599211,11cyclictest16559-21ssh11:50:132
33599211,10cyclictest6393-21kthreadcore09:55:242
33599211,10cyclictest32551-21grep12:15:212
33599211,10cyclictest15663-21kthreadcore10:55:192
335992110,4cyclictest9090-21netstat10:50:242
33599210,16cyclictest2800-21latency_hist08:20:012
32499218,10cyclictest14844-21sshd07:55:251
32499211,9cyclictest0-21swapper/112:02:261
32499211,9cyclictest0-21swapper/109:18:351
324992112,5cyclictest0-21swapper/110:25:141
32499211,17cyclictest32578-21ssh12:35:031
32499211,17cyclictest32578-21ssh12:35:021
32499211,17cyclictest14466-21apt-get10:55:111
32499211,16cyclictest5171-21ls07:15:011
32499211,12cyclictest29137-21ssh12:15:111
32499211,11cyclictest0-21swapper/111:02:241
32499211,10cyclictest0-21swapper/110:51:371
32499210,11cyclictest710-21cron08:10:021
31499212,14cyclictest630-21Async09:36:310
31499212,14cyclictest16071-21ssh10:00:270
31499211,3cyclictest131rcu_preempt07:45:240
31499211,3cyclictest131rcu_preempt07:45:240
314992113,6cyclictest0-21swapper/007:10:230
31499211,2cyclictest131rcu_preempt11:59:090
314992112,7cyclictest300-21cyclictest11:00:130
314992111,7cyclictest17319-21cat11:30:340
314992111,4cyclictest0-21swapper/012:10:110
31499210,15cyclictest23401-21sed10:05:210
35099201,6cyclictest0-21swapper/308:50:203
35099201,6cyclictest0-21swapper/308:35:213
33599208,9cyclictest991-21cut12:35:192
33599208,9cyclictest0-21swapper/208:55:152
33599207,5cyclictest2727-21kthreadcore07:45:202
33599207,5cyclictest2727-21kthreadcore07:45:202
33599206,5cyclictest27-21ksoftirqd/208:05:282
33599206,4cyclictest0-21swapper/211:35:292
33599202,8cyclictest22684-21irqcore08:40:182
33599202,14cyclictest27229-21fschecks_count08:45:152
33599202,10cyclictest29683-21unixbench_singl08:10:282
33599201,9cyclictest31853-21/usr/sbin/munin08:50:152
33599201,8cyclictest9202-21kthreadcore12:20:222
33599201,5cyclictest0-21swapper/211:15:152
33599201,10cyclictest370-21latency_hist07:10:022
33599200,11cyclictest16235-21sort08:00:142
32499202,3cyclictest131rcu_preempt11:05:181
32499201,15cyclictest30746-21cut08:15:161
324992010,7cyclictest6352-21sed12:20:171
31499203,3cyclictest131rcu_preempt08:25:240
31499202,16cyclictest12645-21kthreadcore10:35:220
31499202,14cyclictest4614-21ssh09:20:120
31499201,9cyclictest131rcu_preempt11:11:330
31499201,3cyclictest131rcu_preempt11:21:520
31499201,10cyclictest20176-21ps10:40:170
33599191,9cyclictest13194-21irqrtprio08:30:172
335991914,3cyclictest27-21ksoftirqd/207:40:122
33599191,11cyclictest8371-21irqrtprio08:25:182
33599191,10cyclictest25533-21ssh11:55:132
32499199,7cyclictest18731-21irqrtprio11:15:181
32499196,10cyclictest21122-21sh12:10:181
32499194,9cyclictest21-21ksoftirqd/108:30:131
32499192,3cyclictest0-21swapper/110:40:281
32499191,9cyclictest14080-21kthreadcore10:35:241
32499191,9cyclictest0-21swapper/110:16:541
32499191,7cyclictest32202-21kthreadcore07:40:211
32499191,14cyclictest7454-21ssh11:25:261
32499191,14cyclictest7454-21ssh11:25:251
32499191,11cyclictest27141-21cpuspeed08:45:141
31499192,2cyclictest131rcu_preempt07:15:270
31499192,2cyclictest131rcu_preempt07:15:270
314991911,5cyclictest4001-21fschecks_time11:25:160
314991911,5cyclictest4001-21fschecks_time11:25:150
31499191,15cyclictest24188-21tail12:30:170
31499191,15cyclictest24188-21tail12:30:170
314991911,5cyclictest11111-21/usr/sbin/munin12:05:120
31499191,13cyclictest29237-21ssh12:15:120
32499187,7cyclictest21-21ksoftirqd/107:30:131
32499182,12cyclictest25517-21apt-get11:55:131
32499182,10cyclictest0-21swapper/108:35:131
32499181,4cyclictest18596-21kthreadcore11:50:211
32499181,4cyclictest18596-21kthreadcore11:50:201
32499181,14cyclictest29306-21ls07:40:021
32499181,14cyclictest26254-21irqcore08:10:181
32499181,14cyclictest18670-21tr10:20:201
32499181,11cyclictest5289-21find07:15:111
32499181,11cyclictest5289-21find07:15:111
31499186,9cyclictest28014-21kthreadcore08:10:210
31499182,8cyclictest131rcu_preempt09:32:580
31499182,11cyclictest0-21swapper/008:30:140
31499181,2cyclictest131rcu_preempt09:29:470
31499181,2cyclictest131rcu_preempt07:25:300
314991811,5cyclictest24808-21grep08:05:280
31499181,12cyclictest28988-21diskstats10:45:150
31499181,11cyclictest4100-21ls08:55:020
314991810,5cyclictest15166-21if_err_eth112:25:170
32499170,14cyclictest1489-21kthreadcore08:50:211
31499171,7cyclictest9073-21/usr/sbin/munin09:00:250
32499163,8cyclictest0-21swapper/107:20:141
32499161,11cyclictest9461-21cut09:00:181
32499161,11cyclictest3256-21fschecks_time08:20:151
32499161,11cyclictest22366-21df08:40:151
31499163,8cyclictest12-21ksoftirqd/007:30:110
31499163,10cyclictest11359-21cpu07:55:130
31499162,11cyclictest22151-21latency_hist08:40:010
31499161,12cyclictest7557-21kthreadcore07:50:200
35099151,10cyclictest0-21swapper/308:05:143
32499151,11cyclictest371-21latency_hist07:10:021
32499151,10cyclictest1895-21fschecks_count07:45:161
32499151,10cyclictest1895-21fschecks_count07:45:161
31499141,9cyclictest336-21latency_hist07:10:010
35099101,6cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional