You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-07 - 01:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Thu Aug 06, 2026 12:45:05)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8065216997,24sleep20-21swapper/207:08:072
81012140105,25sleep30-21swapper/307:08:363
79452138102,25sleep10-21swapper/107:06:341
80262136101,24sleep00-21swapper/007:07:370
157302760,3sleep215726-21cstates11:50:122
221402700,2sleep20-21swapper/209:29:112
248622670,2sleep20-21swapper/211:08:032
314592660,8sleep012-21ksoftirqd/011:37:090
118242650,1sleep20-21swapper/210:10:042
16442600,2sleep30-21swapper/311:15:143
116462580,3sleep111631-21ssh12:10:261
52912550,9sleep228-21ksoftirqd/212:05:412
118732540,2sleep30-21swapper/311:22:323
21192520,2sleep10-21swapper/109:14:201
181992520,2sleep00-21swapper/008:45:140
238292510,4sleep1211rcuc/112:20:131
273792490,1sleep00-21swapper/007:50:210
239272490,1sleep20-21swapper/212:20:142
105152490,1sleep210518-21grep07:15:002
299242470,1sleep00-21swapper/011:11:260
100912450,1sleep10-21swapper/109:44:341
95792440,1chrt0-21swapper/210:07:592
208992410,2sleep30-21swapper/310:15:593
133552410,2chrt0-21swapper/209:46:082
118032400,2sleep20-21swapper/210:34:122
69982390,1sleep10-21swapper/112:31:081
219232330,4chrt0-21swapper/110:17:011
200292290,2chrt0-21swapper/209:51:102
150102290,3chrt28-21ksoftirqd/209:23:412
303202280,11sleep122-21ksoftirqd/112:00:231
218662280,4sleep30-21swapper/311:05:233
236452270,8sleep012-21ksoftirqd/011:06:480
8487992622,2cyclictest0-21swapper/307:15:213
8487992610,6cyclictest17818-21expr09:50:123
848699262,21cyclictest25067-21grep09:30:262
848699261,23cyclictest0-21swapper/207:15:062
8486992612,11cyclictest10814-21awk12:35:002
848599261,24cyclictest0-21swapper/107:14:511
848499260,24cyclictest0-21swapper/007:15:270
187722260,2sleep30-21swapper/307:30:223
118792260,2sleep30-21swapper/312:35:163
848799251,1cyclictest0-21swapper/307:14:053
8484992517,6cyclictest15487-21sh12:38:000
8484992511,12cyclictest3583-21seq12:29:390
848699243,16cyclictest29071-21ssh12:00:122
8485992415,6cyclictest52850irq/140-enp3s0-11:30:121
8484992421,2cyclictest0-21swapper/007:13:570
42802240,1sleep00-21swapper/010:04:360
159812240,1sleep00-21swapper/011:50:140
154342240,3sleep10-21swapper/110:12:111
133592240,10sleep122-21ksoftirqd/111:48:111
848699239,5cyclictest8666-21expr11:20:162
848699232,2cyclictest141rcu_preempt07:50:222
848699232,14cyclictest28788-21sh09:10:042
8486992310,3cyclictest25183-21ssh10:20:012
848599232,14cyclictest15573-21chrt11:50:111
8485992310,3cyclictest141rcu_preempt09:40:001
8022230,2chrt0-21swapper/210:25:212
297072230,2sleep20-21swapper/210:47:042
848799223,14cyclictest4307-21ssh09:40:123
8487992213,6cyclictest24399-21tr12:20:183
8487992212,7cyclictest29327-21seq09:10:143
848699228,8cyclictest27225-21sh11:10:132
8486992215,5cyclictest28-21ksoftirqd/209:10:002
8486992212,7cyclictest12711-21cat08:30:232
8486992210,3cyclictest4813-21unixbench_singl09:15:222
848599222,14cyclictest24237-21seq09:55:131
8485992215,4cyclictest12548-21cron07:20:001
8485992213,7cyclictest31077-21df08:00:141
8485992213,7cyclictest26826-21sensors_temp10:20:211
8485992212,8cyclictest15425-21cron08:40:011
8485992211,7cyclictest4064-21wc09:15:161
76292220,2sleep00-21swapper/012:31:480
8487992114,4cyclictest12997-21/usr/sbin/munin10:35:243
8487992113,6cyclictest30785-21ssh11:12:193
8487992113,6cyclictest1-21systemd12:32:033
8487992112,6cyclictest26361-21load10:20:183
8487992111,8cyclictest32548-21df_inode10:25:153
848799211,15cyclictest5167-21ssh10:05:043
8487992110,8cyclictest10480-21df_inode09:20:153
8486992114,5cyclictest13472-21cron11:00:002
8486992113,6cyclictest13746-21ntp_states07:20:192
8486992113,6cyclictest13746-21ntp_states07:20:192
8486992111,7cyclictest20307-21df08:50:142
848599219,8cyclictest22121-21apt-get08:55:111
848499219,10cyclictest11484-21switchtime09:20:240
8484992111,8cyclictest5561-21run-parts09:40:220
8484992111,7cyclictest31734-21cron10:25:000
8484992111,7cyclictest20034-21cat10:15:230
5722210,2sleep00-21swapper/010:50:110
318322210,1sleep10-21swapper/111:13:231
136592210,1sleep20-21swapper/210:35:162
8487992014,4cyclictest539-21sh09:36:403
8487992012,6cyclictest965-21diskstats10:50:143
8487992012,6cyclictest30092-21ntp_kernel_pll_12:00:203
8487992012,6cyclictest28757-21expr07:55:133
8487992012,6cyclictest19892-21cat08:50:003
8487992012,6cyclictest1797-21seq10:02:033
8487992012,6cyclictest11593-21seq10:58:063
8487992011,6cyclictest25084-21cat09:00:183
8487992011,6cyclictest2030-21cat11:40:003
8487992010,7cyclictest6491-21grep10:30:133
848699209,8cyclictest0-21swapper/209:45:002
848699206,13cyclictest141rcu_preempt12:12:282
848699202,7cyclictest23503-21ssh11:55:242
8486992012,6cyclictest20975-21tr10:40:222
8486992012,6cyclictest15039-21awk08:35:262
8486992011,7cyclictest14787-21memory11:00:192
848599201,7cyclictest5968-21idleruntime-cro10:30:001
8485992014,4cyclictest22-21ksoftirqd/111:40:001
8485992011,7cyclictest4567-21grep08:15:001
8485992011,7cyclictest27205-21sed09:05:191
8485992011,7cyclictest16931-21expr09:25:131
8485992011,7cyclictest11137-21ntpq09:20:201
8485992011,6cyclictest789-21expr08:05:131
8485992011,6cyclictest278-21systemd-journal07:35:001
8485992010,7cyclictest20135-21tune2fs07:35:131
8484992015,3cyclictest3318-21ntp_states11:40:200
8484992011,7cyclictest16270-21smartctl08:40:170
8484992011,6cyclictest52750irq/139-enp3s0-09:10:120
8484992011,6cyclictest17127-21grep12:15:140
8484992010,8cyclictest3113-21/usr/sbin/munin09:15:120
848499200,18cyclictest28618-21seq10:45:570
53492200,2sleep30-21swapper/309:15:513
29842200,3sleep228-21ksoftirqd/210:51:052
199212201,1sleep00-21swapper/011:04:490
164822200,2sleep00-21swapper/010:37:100
848799192,12cyclictest430-21apt-get08:05:093
8487991914,3cyclictest15303-21sh09:48:073
8487991913,4cyclictest3315-21apt-get12:05:123
8487991913,4cyclictest14974-21sh12:13:483
8487991911,6cyclictest24704-21tune2fs09:55:163
8487991911,6cyclictest22622-21readlink11:30:223
8487991911,6cyclictest17410-21load12:15:163
8487991911,5cyclictest9858-21idleruntime08:25:163
8487991911,5cyclictest15618-21cat08:40:013
8487991910,6cyclictest22283-21expr07:40:133
8487991910,5cyclictest0-21swapper/311:00:123
848799190,16cyclictest26986-21head10:45:173
848699199,7cyclictest31621-21kworker/u8:111:15:252
848699195,3cyclictest141rcu_preempt11:35:142
8486991913,4cyclictest3857-21sendmail09:40:002
8486991912,5cyclictest7435-21expr08:20:132
8486991912,5cyclictest5008-21expr08:15:132
8486991911,6cyclictest25082-21memory09:55:202
8486991911,6cyclictest22066-21awk11:30:172
848599199,8cyclictest560-21dbus-daemon09:54:591
848599199,6cyclictest52850irq/140-enp3s0-10:45:121
848599199,6cyclictest52850irq/140-enp3s0-10:35:121
848599193,6cyclictest24471-21ssh09:30:211
848599192,5cyclictest1434-21apt-get11:15:131
848599192,12cyclictest32244-21ssh10:00:231
848599191,8cyclictest15867-21taskset11:00:391
8485991912,4cyclictest5955-21sh11:42:251
8485991912,4cyclictest11231-21sh12:35:111
8485991911,6cyclictest26996-21sed11:10:011
8485991911,6cyclictest22677-21cut11:55:171
8485991910,7cyclictest560-21dbus-daemon11:30:001
8485991910,7cyclictest24899-21cat09:00:161
8485991910,6cyclictest29518-21ntp_states07:55:231
8485991910,6cyclictest11671-21expr08:30:111
8485991910,6cyclictest10843-21seq08:26:091
848499199,7cyclictest798-21df_inode08:05:120
848499192,12cyclictest32200-21sh10:25:120
848499192,11cyclictest15230-21apt-get07:25:120
8484991914,3cyclictest10441-21switchtime08:25:210
8484991913,4cyclictest27294-21sh12:22:340
8484991913,3cyclictest29865-21sh09:35:030
8484991912,5cyclictest3298-21find12:05:110
8484991911,6cyclictest8194-21date10:55:190
8484991911,6cyclictest560-21dbus-daemon07:35:200
8484991911,6cyclictest31911-21python308:00:230
8484991911,6cyclictest24859-21grep09:30:250
8484991911,6cyclictest24460-21awk07:45:140
8484991911,6cyclictest13759-21awk07:20:200
8484991911,6cyclictest13759-21awk07:20:190
8484991910,7cyclictest3918-21date08:10:240
8484991910,6cyclictest52750irq/139-enp3s0-10:30:130
8484991910,6cyclictest18187-21cat07:30:160
160452190,2chrt0-21swapper/208:40:152
112902190,2sleep011288-21df09:45:130
848799189,6cyclictest9371-21fschecks_time11:45:143
848799188,8cyclictest560-21dbus-daemon07:50:003
848799187,9cyclictest0-21swapper/311:50:253
848799182,3cyclictest166862unixbench_singl07:25:253
848799181,7cyclictest3237-21needreboot11:40:193
8487991813,3cyclictest22122-21sh11:55:093
8487991813,3cyclictest15399-21interrupts11:25:163
8487991813,3cyclictest12907-21cpuspeed_turbos07:20:123
8487991813,3cyclictest12907-21cpuspeed_turbos07:20:113
8487991812,4cyclictest13026-21sh10:10:203
8487991811,5cyclictest6149-21grep08:15:253
8487991811,5cyclictest2868-21cat08:10:133
8487991811,5cyclictest27420-21awk09:05:223
8487991811,5cyclictest20732-21ls08:50:183
8487991811,5cyclictest19957-21expr10:40:133
8487991811,5cyclictest17602-21cat09:25:193
8487991811,5cyclictest11746-21entropy08:30:143
848799181,14cyclictest31865-21vmstat12:25:263
8487991811,4cyclictest28310-21diskmemload09:30:103
8487991810,6cyclictest22550-21tune2fs08:55:153
8487991810,5cyclictest31575-21date08:00:193
8487991810,5cyclictest27238-21cat07:50:203
848699188,7cyclictest30754-21apt-get08:00:122
848699188,7cyclictest18195-21cron10:15:002
848699182,11cyclictest75-21kswapd012:15:092
848699182,11cyclictest15193-21taskset07:25:032
8486991813,3cyclictest15769-21sh11:25:202
8486991811,5cyclictest30240-21expr12:25:132
8486991811,5cyclictest19999-21expr07:35:112
848699181,11cyclictest28310-21diskmemload11:43:372
8486991810,6cyclictest278-21systemd-journal08:15:002
8486991810,6cyclictest25026-21uname07:45:202
8486991810,6cyclictest12377-21which12:35:212
848599189,7cyclictest14923-21cat07:25:011
848599189,7cyclictest14923-21cat07:25:011
848599189,6cyclictest11044-21tr09:45:131
848599186,10cyclictest0-21swapper/112:09:471
848599183,9cyclictest7288-21ssh10:55:131
848599182,10cyclictest24114-21chrt07:45:131
8485991812,4cyclictest11552-21sh10:33:561
848599181,12cyclictest19702-21apt-get10:40:121
8485991810,6cyclictest20433-21cut08:50:141
8485991810,5cyclictest8956-21cat11:20:191
848499189,7cyclictest13765-21seq12:12:340
848499189,6cyclictest52750irq/139-enp3s0-11:45:100
848499182,11cyclictest7126-21apt-get08:20:120
848499182,11cyclictest16710-21apt-get09:25:110
848499182,11cyclictest14671-21apt-get11:25:100
8484991813,3cyclictest30481-21idleruntime-cro07:59:590
8484991813,3cyclictest26928-21gltestperf09:05:150
8484991813,3cyclictest26570-21sh09:56:030
8484991813,3cyclictest22879-21idleruntime11:55:170
8484991813,3cyclictest13874-21sh10:10:370
8484991812,4cyclictest9628-21sh10:08:020
8484991812,4cyclictest28310-21diskmemload10:40:310
8484991812,4cyclictest23072-21sh11:30:260
8484991812,4cyclictest20890-21sh09:52:030
8484991811,5cyclictest6112-21cat08:15:230
8484991811,5cyclictest3084-21awk11:15:250
8484991811,5cyclictest21696-21seq08:52:570
8484991811,5cyclictest14112-21smartctl08:35:160
8484991811,5cyclictest12-21ksoftirqd/008:59:590
8484991811,5cyclictest11237-21seq11:21:530
8484991810,6cyclictest29318-21expr12:00:130
8484991810,6cyclictest24625-21cat09:00:150
8484991810,6cyclictest11979-21hostname08:30:150
8487991712,3cyclictest20262-21sh07:35:123
848699179,6cyclictest22087-21expr07:40:122
848699172,4cyclictest141rcu_preempt11:48:592
8486991712,3cyclictest1391-21sh08:05:202
8486991711,4cyclictest28310-21diskmemload10:03:422
8486991710,5cyclictest24493-21expr09:00:132
8486991710,5cyclictest22554-21tune2fs08:55:152
848699170,14cyclictest0-21swapper/208:45:112
848599173,11cyclictest0-21swapper/110:05:131
848599172,11cyclictest16932-21diskstats12:15:121
848599171,7cyclictest3203-21ssh10:51:181
8485991712,3cyclictest15708-21apt08:40:111
848599171,14cyclictest560-21dbus-daemon07:30:001
848599171,11cyclictest30134-21apt-get12:25:121
8484991710,5cyclictest22765-21ls07:40:180
848799169,5cyclictest16243-21sed08:40:163
8487991610,4cyclictest8253-21mailstats08:20:223
848699169,5cyclictest29341-21date07:55:202
848699162,11cyclictest9676-21sort08:25:142
8486991612,2cyclictest17440-21apt-config07:30:012
848599167,6cyclictest22756-21missed_timers07:40:181
8485991611,3cyclictest636-21runrttasks08:15:091
8485991610,4cyclictest26695-21expr07:50:141
848599152,7cyclictest7129-21taskset08:20:111
848599151,12cyclictest17820-21/usr/sbin/munin08:45:221
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional