You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-04 - 20:26

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Mon May 04, 2026 12:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
36362257233,16sleep20-21swapper/207:06:512
35952152129,16sleep30-21swapper/307:06:203
37012140102,26sleep00-21swapper/007:07:420
35332140104,25sleep10-21swapper/107:05:311
2494021010,2sleep30-21swapper/310:21:543
204032900,4sleep320402-21ssh09:30:253
128272620,3sleep312831-21fschecks_time11:25:153
77862610,1sleep30-21swapper/309:45:233
5492600,2sleep10-21swapper/111:40:141
50282570,1sleep10-21swapper/110:07:201
221582550,1sleep122157-21ssh11:08:031
264322530,2sleep00-21swapper/009:35:200
93432520,1sleep00-21swapper/012:34:430
194102500,2sleep00-21swapper/009:54:340
95562490,3chrt141rcu_preempt09:23:051
95562490,3chrt141rcu_preempt09:23:041
292752490,1sleep20-21swapper/210:25:182
50162480,1sleep20-21swapper/210:30:522
265342480,2sleep00-21swapper/011:59:430
145672470,8sleep334-21ksoftirqd/310:38:393
45902450,2sleep20-21swapper/211:43:032
10752420,2sleep10-21swapper/112:28:131
252492400,1sleep10-21swapper/109:58:401
415499352,10cyclictest14969-21ssh10:15:122
175392330,2chrt12-21ksoftirqd/012:15:490
415399293,13cyclictest10083-21apt-get12:35:111
4155992818,7cyclictest31041-21expr09:15:143
4153992823,3cyclictest18406-21ssh09:53:341
88262270,12sleep228-21ksoftirqd/209:22:222
88262270,12sleep228-21ksoftirqd/209:22:212
4155992714,9cyclictest560-21dbus-daemon09:50:013
4153992715,10cyclictest17092-21fschecks_time10:40:161
415399271,19cyclictest0-21swapper/107:12:351
415399261,11cyclictest266722gltestperf08:00:151
415299261,1cyclictest0-21swapper/007:13:230
415599251,23cyclictest0-21swapper/307:12:543
4154992523,1cyclictest0-21swapper/207:12:382
4154992514,8cyclictest1-21systemd07:40:002
415499251,23cyclictest0-21swapper/209:53:272
415299259,6cyclictest18172-21df_inode11:05:160
4152992511,11cyclictest7238-21grep09:45:170
142802250,4sleep20-21swapper/212:37:542
415599242,14cyclictest19497-21chrt09:00:123
4155992415,7cyclictest560-21dbus-daemon08:40:003
4154992410,5cyclictest23252-21grep10:20:262
415399242,15cyclictest75-21kswapd012:20:111
266022240,1sleep20-21swapper/210:46:572
146472240,2sleep10-21swapper/111:26:001
54932230,2sleep00-21swapper/009:44:180
415599233,9cyclictest7201-21grep11:20:253
415599231,7cyclictest16293-21python310:15:233
415499231,6cyclictest32337-21grep11:40:012
415499231,6cyclictest32337-21grep11:40:002
4154992313,7cyclictest9031-21df07:20:142
415399236,10cyclictest23999-21diskmemload09:10:131
415399232,14cyclictest640-21snmpd12:10:131
415399232,14cyclictest640-21snmpd07:55:111
4153992313,7cyclictest20413-21ntpq09:00:201
4153992313,6cyclictest52850irq/140-enp3s0-11:00:121
4153992311,10cyclictest13293-21ssh11:49:581
4152992315,6cyclictest28798-21users10:00:260
121732230,2sleep20-21swapper/209:49:182
97082220,1sleep30-21swapper/311:46:253
415599221,8cyclictest15139-21cat11:50:243
415599221,7cyclictest30032-21timerwakeupswit10:25:243
415599220,8cyclictest8689-21grep10:10:153
415499229,11cyclictest24253-21latency_hist11:10:002
4154992213,7cyclictest17452-21memory10:40:192
4154992213,6cyclictest5929-21df_inode11:20:152
4154992213,6cyclictest53050irq/142-enp3s0-11:25:112
4154992212,8cyclictest560-21dbus-daemon07:34:592
415399222,14cyclictest25487-21ssh09:35:141
4153992215,5cyclictest22-21ksoftirqd/111:25:001
4153992212,6cyclictest7603-21needreboot09:45:201
4153992211,7cyclictest5061-21hwlatdetect10:55:161
415399221,15cyclictest23339-21ssh10:45:121
415399220,10cyclictest20822fschecks_time12:05:141
4152992214,6cyclictest12-21ksoftirqd/011:20:010
263002220,1sleep00-21swapper/010:23:160
133882220,2sleep1211rcuc/110:13:531
415599219,8cyclictest560-21dbus-daemon11:15:003
415599215,3cyclictest141rcu_preempt09:37:113
415599214,4cyclictest23521-21diskmemload12:27:533
415599211,6cyclictest26486-21hddtemp_smartct11:35:173
415599211,6cyclictest26486-21hddtemp_smartct11:35:163
4155992115,4cyclictest6397-21ssh11:44:523
4155992113,6cyclictest9607-21ntpq07:20:193
4155992113,6cyclictest13979-21ntp_states08:45:213
4155992113,6cyclictest12025-21ls11:00:183
4155992112,7cyclictest6390-21awk07:15:133
4155992112,5cyclictest28406-21python310:00:233
4155992112,5cyclictest21557-21mailstats09:55:233
415499219,7cyclictest53050irq/142-enp3s0-10:50:122
415499219,7cyclictest53050irq/142-enp3s0-10:50:112
415499218,11cyclictest2978-21ls10:05:252
415499213,4cyclictest28-21ksoftirqd/209:35:162
4154992113,6cyclictest14644-21seq12:14:392
4154992112,7cyclictest30687-21users12:25:272
4154992112,7cyclictest27032-21cat10:00:122
4154992112,6cyclictest20211-21awk11:30:222
4154992112,5cyclictest0-21swapper/212:30:132
415499211,14cyclictest27278-21ssh12:00:102
415399213,13cyclictest75-21kswapd011:10:121
415399213,13cyclictest16856-31updatedb.mlocat07:37:051
415399213,12cyclictest0-21swapper/109:30:101
415399212,13cyclictest15454-21ssh12:15:121
415399211,6cyclictest31379-21cut11:15:131
4153992115,3cyclictest23521-21diskmemload10:01:201
4153992115,3cyclictest23521-21diskmemload09:41:411
4153992113,6cyclictest10405-21seq08:37:021
4153992113,4cyclictest14375-21gltestperf11:50:171
415299214,7cyclictest141rcu_preempt12:35:110
415299212,13cyclictest11193-21ssh11:00:120
4152992115,4cyclictest17923-21idleruntime08:55:150
4152992115,4cyclictest14914-21/usr/sbin/munin10:15:170
4152992112,7cyclictest26218-21df_inode11:35:150
4152992112,7cyclictest26218-21df_inode11:35:140
4152992112,7cyclictest21505-21seq07:48:380
242292210,2sleep00-21swapper/009:57:400
97992200,9sleep012-21ksoftirqd/009:23:200
97992200,9sleep012-21ksoftirqd/009:23:190
415599204,3cyclictest23521-21diskmemload12:24:583
415599203,14cyclictest2942-21tr10:30:133
415599203,12cyclictest0-21swapper/309:40:123
4155992015,3cyclictest5143-21sendmail11:20:013
4155992012,6cyclictest5070-21sed10:55:173
4155992012,6cyclictest25510-21sed09:10:263
4155992012,6cyclictest16081-21smartctl12:15:173
4155992011,6cyclictest12456-21grep09:25:163
415599201,13cyclictest16411-21ssh10:40:043
415599200,9cyclictest21680-21seq11:31:173
4154992015,3cyclictest15766-21ssh11:50:422
4154992014,4cyclictest5062-21sh12:06:512
4154992012,6cyclictest31230-21find11:15:112
4154992012,6cyclictest31033-21idleruntime08:10:182
4154992012,6cyclictest17770-21perl08:55:142
415399208,3cyclictest52850irq/140-enp3s0-10:20:141
415399203,3cyclictest23521-21diskmemload10:34:201
415399203,15cyclictest10003-21grep10:35:151
4153992015,3cyclictest12745-21sh07:30:001
4153992012,6cyclictest52850irq/140-enp3s0-10:25:241
4153992012,4cyclictest636-21runrttasks12:01:021
4153992012,4cyclictest26242-21entropy11:35:161
4153992012,4cyclictest26242-21entropy11:35:151
4153992012,4cyclictest23521-21diskmemload12:34:061
4153992011,7cyclictest12824-21wc09:25:181
415299208,10cyclictest4088-21mailstats10:30:220
415299208,10cyclictest0-21swapper/010:50:230
415299208,10cyclictest0-21swapper/010:50:220
415299202,13cyclictest8182-21apt-get10:10:130
4152992015,3cyclictest16262-21processes08:50:200
4152992014,4cyclictest28317-21ntp_states12:00:190
4152992013,4cyclictest5495-21sh10:07:460
4152992012,6cyclictest31372-21ntpq08:10:210
4152992012,6cyclictest25082-21fw_packets11:10:150
4152992012,6cyclictest22586-21seq09:05:220
4152992012,6cyclictest14424-21idleruntime11:50:170
4152992011,7cyclictest31002-21seq10:26:120
13512200,3sleep0131rcuc/011:40:200
415599195,12cyclictest3122-21users10:05:273
415599193,3cyclictest23521-21diskmemload12:10:043
415599192,12cyclictest4661-21ssh12:30:243
415599192,11cyclictest636-21runrttasks07:30:123
4155991914,3cyclictest22729-21sendmail_mailst09:05:233
4155991913,4cyclictest9340-21sh09:22:533
4155991913,4cyclictest9340-21sh09:22:523
4155991913,4cyclictest518-21uniq08:15:163
4155991913,3cyclictest17918-21tr11:05:153
4155991912,5cyclictest31088-21ntp_kernel_pll_10:50:203
4155991912,5cyclictest31088-21ntp_kernel_pll_10:50:193
4155991911,6cyclictest5029-21seq12:06:503
4155991911,6cyclictest4566-21sort08:25:123
4155991911,6cyclictest17223-21sed07:40:003
415599191,12cyclictest6620-21apt-get08:30:123
4155991910,6cyclictest10614-21gltestperf12:35:153
415499199,8cyclictest6295-21kworker/u8:211:00:202
415499198,9cyclictest0-21swapper/210:56:032
4154991913,4cyclictest8711-21tr10:10:172
4154991913,4cyclictest21315-21sh09:31:192
4154991913,4cyclictest13934-21sh10:38:012
4154991912,5cyclictest20446-21pluginstate09:00:212
4154991912,5cyclictest1026-21netstat08:15:192
4154991911,6cyclictest9747-21awk08:35:202
4154991911,6cyclictest31852-21open_files09:15:232
4154991911,6cyclictest23548-21wc09:09:592
4154991911,6cyclictest18325-21awk07:40:212
4154991911,6cyclictest18325-21awk07:40:212
4154991911,6cyclictest16140-21if_err_enp3s012:15:182
4154991911,6cyclictest12314-21awk07:25:242
4154991911,5cyclictest32247-21acpi09:40:112
4154991911,5cyclictest15783-21idleruntime08:50:142
4154991910,7cyclictest29475-21python308:05:242
4154991910,7cyclictest24764-21ls07:55:172
415499190,4cyclictest0-21swapper/209:11:062
4153991913,4cyclictest12806-21sh08:45:001
4153991912,5cyclictest52850irq/140-enp3s0-10:15:211
4153991912,5cyclictest18742-21sed07:40:251
4153991912,5cyclictest18742-21sed07:40:241
4153991912,5cyclictest14887-21seq08:49:261
4153991911,6cyclictest4145-21seq08:24:091
4153991911,6cyclictest28734-21cut08:05:151
4153991910,7cyclictest22962-21sed09:05:261
4153991910,6cyclictest52850irq/140-enp3s0-09:15:131
4152991914,3cyclictest22950-21rt-features07:50:210
4152991914,3cyclictest16328-21switchtime07:35:230
4152991913,4cyclictest22534-21sh09:32:300
4152991913,4cyclictest20147-21sh11:30:210
4152991913,4cyclictest13175-21sh10:37:150
4152991913,4cyclictest13025-21sh12:13:020
4152991913,4cyclictest12254-21sh11:48:560
4152991913,3cyclictest23760-21sh09:10:020
4152991912,5cyclictest9740-21seq11:22:500
4152991912,5cyclictest12687-21seq07:29:360
4152991911,6cyclictest8030-21seq07:17:400
4152991911,6cyclictest25386-21cat07:55:240
4152991911,6cyclictest13649-21basename08:45:180
4152991911,6cyclictest10399-21seq07:22:380
4152991911,5cyclictest17862-21mailstats10:40:230
415299191,13cyclictest30784-21ssh09:15:120
415299191,12cyclictest4324-40unattended-upgr12:06:110
4152991910,7cyclictest560-21dbus-daemon09:25:230
4152991910,7cyclictest18498-21ssh11:29:520
415299190,5cyclictest28716-21ssh10:49:040
415599188,7cyclictest28016-21cut12:00:163
415599185,2cyclictest141rcu_preempt11:57:343
415599185,11cyclictest30381-21latency_hist08:10:023
4155991813,3cyclictest25726-21sh10:46:043
4155991811,5cyclictest30672-21expr08:10:133
4155991811,5cyclictest17073-21cron08:55:013
4155991811,5cyclictest12756-21basename07:29:593
4155991811,4cyclictest27509-21cat08:00:243
415599181,14cyclictest18309-21ntp_states07:40:223
415599181,14cyclictest18309-21ntp_states07:40:213
4155991810,6cyclictest22268-21cut07:50:143
4155991810,6cyclictest19280-21sed09:00:003
4155991810,5cyclictest11767-21tail08:40:203
415499188,9cyclictest14574-21ssh09:26:202
415499187,3cyclictest20277-21ssh09:55:112
415499182,11cyclictest13064-21apt-get08:45:122
4154991813,3cyclictest23504-21sh11:56:422
4154991813,3cyclictest11684-21sh11:48:232
4154991811,5cyclictest6511-21ls07:15:132
4154991810,6cyclictest8562-21date08:35:012
4154991810,6cyclictest5749-21cat08:25:232
415399189,6cyclictest513-21grep08:15:161
415399181,9cyclictest0-21swapper/111:55:261
415399181,4cyclictest0-21swapper/110:50:281
415399181,4cyclictest0-21swapper/110:50:271
4153991813,3cyclictest657-21sshd11:30:211
4153991811,5cyclictest22357-21grep07:50:151
4153991811,5cyclictest18152-21sed08:55:181
4153991811,5cyclictest15983-21sed08:50:161
4153991810,6cyclictest8443-21cat07:20:011
4153991810,6cyclictest20882-21tr07:45:231
4153991810,5cyclictest6605-21acpi08:30:101
415299189,6cyclictest7051-21tune2fs08:30:160
415299189,6cyclictest52750irq/139-enp3s0-10:55:130
4152991813,3cyclictest24484-21sh12:20:580
4152991812,4cyclictest28901-21sh12:25:090
4152991811,5cyclictest29990-21seq08:07:280
4152991811,5cyclictest20059-21hostname09:00:170
4152991810,6cyclictest4789-21cat08:25:120
4152991810,6cyclictest27071-21ntpq08:00:190
4152991810,6cyclictest1924-21seq08:18:230
4152991810,6cyclictest17352-21seq07:40:040
4152991810,6cyclictest17352-21seq07:40:030
4152991810,6cyclictest13214-21expr07:30:140
4152991810,6cyclictest10569-21seq08:39:090
4155991710,5cyclictest25212-21sed07:55:233
4155991710,5cyclictest20052-21cat07:45:153
415499179,4cyclictest23521-21diskmemload11:10:012
415499172,11cyclictest19742-21apt-get07:45:132
4154991710,5cyclictest27250-21rt-features08:00:222
4154991710,5cyclictest2514-21ls08:20:152
4154991710,5cyclictest22858-21ls12:20:162
4154991710,5cyclictest12957-21cat08:45:012
4153991711,4cyclictest9388-21cut07:20:171
4152991710,5cyclictest2471-21expr08:20:140
4152991710,5cyclictest11088-21seq08:40:150
4155991611,3cyclictest2898-21hddtemp_smartct08:20:173
415499161,6cyclictest21853-21/usr/sbin/munin07:50:152
415399168,6cyclictest5752-21sed08:25:241
415399163,11cyclictest30587-21cpuspeed_turbos08:10:131
415399162,12cyclictest14462-21grep07:30:251
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional