You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-18 - 05:57

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackaslot8s.osadl.org (updated Thu Jun 18, 2026 00:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
222492201187,8sleep30-21swapper/319:05:423
224452179101,27sleep00-21swapper/019:08:160
223502175104,63sleep20-21swapper/219:07:012
222772170117,46sleep10-21swapper/119:06:041
10472780,1sleep10-21swapper/119:35:001
315752650,4sleep2271rcuc/219:30:122
7072630,1sleep10-21swapper/123:26:591
25152610,2sleep20-21swapper/221:28:052
297552600,2sleep20-21swapper/223:00:182
22854996020,23cyclictest9963-21diskmemload23:10:411
199662600,3sleep3331rcuc/321:40:223
4472590,1sleep00-21swapper/023:26:450
291302580,2sleep00-21swapper/000:35:480
231002550,2sleep20-21swapper/219:10:142
95062540,1sleep00-21swapper/021:56:480
202322520,3sleep228-21ksoftirqd/221:16:492
193352520,1sleep30-21swapper/322:28:463
262432510,2sleep30-21swapper/322:57:553
141042500,2sleep00-21swapper/000:00:410
194842490,1sleep019489-21head00:05:180
150662480,1sleep20-21swapper/200:01:382
76142470,2sleep20-21swapper/221:55:212
52042450,1sleep00-21swapper/000:18:550
313832450,2sleep30-21swapper/323:25:243
161072450,2sleep10-21swapper/122:25:311
130632450,2sleep20-21swapper/223:35:552
130632450,2sleep20-21swapper/223:35:552
198912440,3sleep00-21swapper/022:29:190
216902410,2sleep10-21swapper/121:18:161
22854993813,7cyclictest29064-21seq21:47:211
2285499352,14cyclictest4882-21taskset21:30:131
22854993517,15cyclictest23048-21sh22:06:541
22854993511,6cyclictest7023-21seq23:31:341
22854993511,6cyclictest6671-21expr21:55:131
2285499331,9cyclictest24433-21ssh23:45:101
2285499328,7cyclictest8651-21ntpq22:20:201
2285499323,8cyclictest32093-21apt-get21:50:091
22854993213,3cyclictest636-21runrttasks21:12:281
22854993211,4cyclictest1845-21sh23:04:151
22854993210,5cyclictest29675-21date20:40:011
22854993210,4cyclictest16590-21expr23:15:121
2285699310,28cyclictest14396-21rm21:36:183
22855993123,5cyclictest9963-21diskmemload21:31:142
2285499319,4cyclictest0-21swapper/123:50:241
2285499318,6cyclictest7770-21seq21:04:351
2285499318,5cyclictest24781-21ls19:15:011
2285499318,20cyclictest0-21swapper/100:11:111
22854993112,3cyclictest5370-21/usr/sbin/munin23:55:221
22854993111,4cyclictest26510-21sh21:21:211
2285499311,10cyclictest31118-21gltestperf21:25:151
2285499310,29cyclictest0-21swapper/123:22:141
2285699300,28cyclictest10254-21ssh21:57:333
2285599301,26cyclictest13683-21rm21:35:362
2285499309,3cyclictest21004-21cpuspeed_turbos20:20:131
2285499308,5cyclictest5865-21cat21:00:011
2285499308,5cyclictest12333-21grep21:35:151
2285499308,3cyclictest25171-21sh21:45:001
2285499307,7cyclictest1649-21sed22:15:171
2285499306,9cyclictest0-21swapper/123:05:221
22853993024,4cyclictest23879-21/usr/sbin/munin23:20:170
2285399301,3cyclictest4966-21ssh21:30:140
2285399301,26cyclictest23809-21apt-get23:45:030
2285399300,27cyclictest24043-21ssh22:07:540
322062290,1chrt0-21swapper/223:26:032
2285699291,25cyclictest32622-21ssh21:25:443
2285699291,25cyclictest30105-21ssh21:24:583
2285699290,5cyclictest6362-21cat00:19:593
2285599291,25cyclictest26561-21ssh23:45:492
2285499299,3cyclictest389-21fschecks_count00:15:151
2285499298,5cyclictest25271-21expr19:15:131
2285499291,9cyclictest8651-21cpu22:45:111
22854992911,3cyclictest21196-21cstates22:30:131
2285499291,11cyclictest6217-21grep19:45:151
2285499291,10cyclictest20343-21timerandwakeup00:05:271
22856992820,6cyclictest1695-21ssh21:50:413
2285699281,3cyclictest2808-21ssh23:52:493
2285699281,2cyclictest0-21swapper/321:47:433
2285699281,25cyclictest5070-21ssh22:17:553
2285699281,25cyclictest10370-21ssh21:34:143
2285699281,23cyclictest0-21swapper/322:32:033
2285699281,23cyclictest0-21swapper/322:01:273
2285599283,24cyclictest28-21ksoftirqd/222:17:552
22855992822,3cyclictest14190-21ssh22:25:062
22855992820,5cyclictest27577-21netstat22:10:202
2285599281,25cyclictest30800-21ssh21:49:072
2285599280,25cyclictest26816-21ssh23:22:232
2285499289,6cyclictest52850irq/140-enp3s0-00:20:121
2285499287,5cyclictest14368-21tr20:05:121
2285499285,9cyclictest0-21swapper/122:35:221
22854992821,5cyclictest18556-21ssh22:04:101
2285499281,8cyclictest12154-21ssh00:00:131
22854992812,3cyclictest15308-21/usr/sbin/munin22:50:211
22854992811,3cyclictest8041-21/usr/sbin/munin21:05:241
22854992810,3cyclictest27634-21sh22:10:221
22853992824,2cyclictest0-21swapper/021:41:380
22853992820,6cyclictest12284-21expr23:35:230
22853992820,6cyclictest12284-21expr23:35:220
2285399281,25cyclictest0-21swapper/022:19:300
207982280,2sleep00-21swapper/000:30:140
22856992722,3cyclictest7981-21ssh23:32:343
2285699271,24cyclictest0-21swapper/323:45:473
2285699271,24cyclictest0-21swapper/322:07:423
2285599273,1cyclictest0-21swapper/223:52:482
2285599271,2cyclictest0-21swapper/222:08:062
2285599271,24cyclictest0-21swapper/223:55:182
2285599271,24cyclictest0-21swapper/221:24:212
2285499272,10cyclictest28050-21ntp_states00:35:191
22854992720,6cyclictest0-21swapper/123:35:231
22854992720,6cyclictest0-21swapper/123:35:221
22853992723,2cyclictest6301-21apt-get00:20:010
22853992723,2cyclictest15686-21ssh22:01:150
22853992719,5cyclictest6196-21diskmemload23:06:540
2285399271,24cyclictest0-21swapper/023:10:270
2285399271,24cyclictest0-21swapper/022:55:320
2285399271,24cyclictest0-21swapper/022:39:310
2285399271,24cyclictest0-21swapper/022:31:250
2285399271,24cyclictest0-21swapper/021:51:120
2285399270,25cyclictest1927-21ssh21:27:280
22856992619,4cyclictest27576-21netstat22:10:193
22856992616,7cyclictest14957-21python300:25:233
2285699261,23cyclictest0-21swapper/322:23:433
2285699261,1cyclictest0-21swapper/323:20:193
22855992623,2cyclictest0-21swapper/222:39:322
22855992622,2cyclictest0-21swapper/222:31:592
2285599261,2cyclictest0-21swapper/223:31:482
2285599261,2cyclictest0-21swapper/222:24:082
2285599261,24cyclictest0-21swapper/222:04:482
2285599261,23cyclictest0-21swapper/221:40:062
2285599261,23cyclictest0-21swapper/200:25:192
22854992617,6cyclictest22570-21/usr/sbin/munin22:55:151
22854992612,3cyclictest16396-21/usr/sbin/munin20:10:241
22853992623,2cyclictest0-21swapper/021:24:570
22853992622,2cyclictest0-21swapper/023:31:550
2285399261,2cyclictest0-21swapper/023:50:190
2285399261,24cyclictest0-21swapper/021:35:460
2285399261,1cyclictest0-21swapper/022:23:520
2285399261,1cyclictest0-21swapper/021:47:550
22856992512,9cyclictest28704-21grep00:35:233
2285699251,1cyclictest0-21swapper/323:10:193
2285699250,23cyclictest6263-21ssh23:55:183
22855992523,1cyclictest0-21swapper/221:50:392
2285599251,2cyclictest0-21swapper/223:10:192
2285599251,1cyclictest0-21swapper/222:55:332
2285499251,5cyclictest7000-21ssh22:43:471
22853992523,1cyclictest0-21swapper/022:10:220
129772250,2sleep30-21swapper/322:48:043
2285699244,17cyclictest34-21ksoftirqd/323:00:193
22856992422,1cyclictest0-21swapper/322:35:183
22856992421,2cyclictest0-21swapper/300:20:013
22855992422,1cyclictest0-21swapper/200:20:012
2285599241,7cyclictest2976-21grep22:40:202
22853992421,2cyclictest0-21swapper/023:55:170
161902240,1chrt0-21swapper/022:50:180
291612230,4sleep029177-21ssh00:12:210
22856992313,7cyclictest25748-21df_inode20:30:123
22856992313,7cyclictest25748-21df_inode20:30:113
2285599236,3cyclictest141rcu_preempt21:10:002
22855992313,7cyclictest52750irq/139-enp3s0-22:45:122
53302220,1sleep30-21swapper/322:42:023
22856992211,8cyclictest12472-21df_inode20:00:123
22854992211,8cyclictest3995-21tune2fs19:40:171
22854992211,8cyclictest3995-21tune2fs19:40:161
22854992211,3cyclictest29925-21idleruntime19:25:151
22853992210,3cyclictest53322ssh22:42:010
2285699213,9cyclictest34-21ksoftirqd/321:10:113
22856992113,6cyclictest27021-21cat19:20:013
22856992113,6cyclictest19370-21perl00:05:173
2285599219,10cyclictest636-21runrttasks23:41:042
22855992116,3cyclictest6391-21latency_hist00:20:002
22854992112,6cyclictest26123-21cat20:30:161
22854992112,6cyclictest26123-21cat20:30:161
2285399218,10cyclictest29670-21missed_timers23:00:180
22853992114,4cyclictest11183-21latency21:10:180
22853992111,7cyclictest6847-21tr19:45:220
2285699204,3cyclictest9963-21diskmemload23:44:153
22856992013,4cyclictest636-21runrttasks00:32:073
22856992012,6cyclictest8864-21which19:50:213
22856992012,6cyclictest5795-21date21:00:013
22856992012,6cyclictest31452-21find19:30:093
22856992012,6cyclictest11140-21sensors_temp19:55:233
22856992012,5cyclictest6173-21expr21:00:133
22856992012,5cyclictest21796-21cat22:55:013
2285599203,15cyclictest28655-21grep00:35:222
22855992012,6cyclictest560-21dbus-daemon23:20:002
22855992012,6cyclictest14509-21cat20:05:122
22855992012,6cyclictest1352-21expr19:35:132
22855992012,5cyclictest31233-21grep20:40:262
22854992012,6cyclictest20342-21seq20:18:481
22854992012,6cyclictest1084-21seq20:46:221
2285399202,12cyclictest3771-21apt-get20:55:110
22853992014,4cyclictest1992-21uniq20:50:160
22853992013,4cyclictest10865-21sh22:45:550
22853992012,6cyclictest8799-21sed21:05:170
22853992012,6cyclictest7583-21seq21:02:100
22853992012,6cyclictest29328-21seq20:38:350
22853992011,6cyclictest30115-21cat19:25:180
22853992011,6cyclictest28805-21seq19:21:300
22853992010,6cyclictest13616-21apt-get00:25:120
2285399200,17cyclictest10982-21awk19:55:220
2285699199,2cyclictest34-21ksoftirqd/300:00:143
2285699191,6cyclictest26154-21hddtemp_smartct00:10:163
22856991914,3cyclictest4450-21netstat19:40:213
22856991914,3cyclictest4450-21netstat19:40:213
22856991914,3cyclictest23297-21fschecks_time19:10:153
22856991913,4cyclictest17473-21sh21:15:133
22856991913,4cyclictest14937-21kernelversion20:05:173
22856991912,5cyclictest31648-21cron20:44:593
22856991912,5cyclictest21146-21cat20:20:143
22856991912,5cyclictest17410-21ntpq23:15:193
22856991911,6cyclictest30097-21memory19:25:183
22856991911,6cyclictest17719-21sed20:10:233
2285699191,16cyclictest10848-21ssh23:35:113
2285699191,16cyclictest10848-21ssh23:35:113
22856991911,5cyclictest8348-21cat21:05:133
22856991911,5cyclictest29726-21cut20:40:013
22856991911,5cyclictest27638-21perl19:20:153
2285599192,3cyclictest9963-21diskmemload00:31:082
2285599191,4cyclictest0-21swapper/223:05:022
22855991914,3cyclictest7822-21dump-pmu-power21:05:002
22855991913,3cyclictest28445-21xargs19:20:252
22855991912,4cyclictest9963-21diskmemload00:10:072
22855991911,6cyclictest9150-21sed19:50:252
22855991911,6cyclictest6222-21sed19:45:142
2285499192,12cyclictest23077-21apt-get20:25:121
22854991912,4cyclictest636-21runrttasks23:44:141
22854991911,6cyclictest12424-21expr20:00:131
22854991911,5cyclictest1560-21uniq19:35:141
22854991910,7cyclictest20667-21df_inode00:30:121
2285399192,12cyclictest22894-21apt-get19:10:120
22853991913,4cyclictest26917-21cron19:20:010
22853991911,6cyclictest5348-21seq19:43:460
22853991911,6cyclictest5348-21seq19:43:450
22853991911,6cyclictest19474-21ntp_states20:15:190
22853991911,5cyclictest18085-21needreboot21:15:200
2285699189,6cyclictest3846-21latency23:05:183
22856991812,4cyclictest2516-21timerwakeupswit19:35:253
22856991811,5cyclictest2752-21grep20:50:223
22856991811,5cyclictest23510-21tune2fs20:25:153
22856991811,5cyclictest18979-21tune2fs20:15:143
22856991810,6cyclictest6467-21memory19:45:173
2285599186,9cyclictest0-21swapper/220:50:162
2285599181,5cyclictest0-21swapper/221:14:532
22855991812,4cyclictest28384-21sed20:35:182
22855991812,4cyclictest18532-21/usr/sbin/munin20:15:132
22855991811,5cyclictest30537-21sed19:25:232
22855991811,5cyclictest19690-21cat00:05:202
22855991810,6cyclictest31889-21cat20:45:092
22855991810,6cyclictest15815-21uniq22:50:142
22854991811,4cyclictest14581-21needreboot00:25:191
22854991810,6cyclictest11777-21seq19:58:591
22854991810,5cyclictest9931-21date19:55:001
2285399189,6cyclictest26581-21seq20:30:210
2285399189,6cyclictest26581-21seq20:30:210
2285399189,6cyclictest13378-21date20:00:220
2285399182,11cyclictest7846-21apt-get19:50:130
22853991811,5cyclictest24690-21seq20:26:400
22853991811,5cyclictest18854-21mailstats23:40:210
22853991811,5cyclictest15095-21cat20:05:180
22853991810,6cyclictest31728-21expr19:30:110
22853991810,6cyclictest30673-21ntp_kernel_pll_20:40:210
22853991810,5cyclictest20916-21find20:20:100
22853991810,5cyclictest17781-21sed23:15:220
22856991710,5cyclictest32071-21cat20:45:123
2285599178,8cyclictest26962-21sendmail19:20:002
2285599172,13cyclictest13640-21unixbench_singl20:00:252
22855991711,4cyclictest28-21ksoftirqd/220:30:012
22855991711,4cyclictest28-21ksoftirqd/220:30:002
22855991710,5cyclictest23713-21cat20:25:172
22855991710,5cyclictest10984-21grep19:55:212
22854991710,5cyclictest3288-21seq20:52:121
22854991710,5cyclictest31865-21cat20:45:011
22854991710,5cyclictest27735-21expr19:20:171
2285399179,5cyclictest372-21uname20:45:200
2285399172,11cyclictest1197-21apt-get19:35:130
2285599169,5cyclictest560-21dbus-daemon21:00:012
2285599169,5cyclictest4065-21grep19:40:172
2285599169,5cyclictest4065-21grep19:40:162
2285599169,5cyclictest21619-21wc20:20:172
22853991611,3cyclictest16396-21/usr/sbin/munin20:10:150
2285599159,4cyclictest17622-21switchtime20:10:232
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional