You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-19 - 23:14

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Tue May 19, 2026 12:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22230221797,96sleep10-21swapper/107:06:571
222042176116,33sleep20-21swapper/207:06:392
221802140104,26sleep30-21swapper/307:06:203
221522136100,25sleep00-21swapper/007:05:560
271662750,2sleep00-21swapper/011:46:370
299202730,2sleep30-21swapper/311:49:253
199002660,1sleep00-21swapper/010:29:280
123002650,2sleep10-21swapper/111:35:221
11482650,10sleep012-21ksoftirqd/010:39:440
11482650,10sleep012-21ksoftirqd/010:39:440
110742650,3sleep228-21ksoftirqd/211:35:122
18052620,1sleep10-21swapper/111:28:001
284272600,2sleep012-21ksoftirqd/010:10:370
200642600,4sleep122-21ksoftirqd/112:05:251
239792560,2sleep30-21swapper/307:10:243
22622550,2sleep30-21swapper/309:51:183
201012550,1sleep00-21swapper/011:17:160
42612510,2sleep00-21swapper/011:54:320
304762510,2sleep10-21swapper/112:37:501
301912510,1sleep30-21swapper/311:25:133
271512500,2sleep10-21swapper/110:10:151
123192500,4sleep10-21swapper/110:47:151
232462490,1sleep20-21swapper/209:19:312
296692460,1sleep00-21swapper/012:37:020
288472390,1sleep20-21swapper/211:24:202
2274099332,11cyclictest5196-21ssh09:30:123
141132330,3sleep30-21swapper/311:12:563
46952320,1sleep30-21swapper/310:41:183
126652320,2sleep20-21swapper/210:47:352
2273999307,12cyclictest28-21ksoftirqd/212:30:262
2273999307,12cyclictest28-21ksoftirqd/210:03:382
22739993012,10cyclictest28-21ksoftirqd/209:41:592
306622290,8sleep228-21ksoftirqd/210:12:532
22739992913,9cyclictest28-21ksoftirqd/209:47:562
22739992910,10cyclictest28-21ksoftirqd/209:26:072
22737992814,4cyclictest5937-21sh09:55:000
2273999278,10cyclictest28-21ksoftirqd/207:38:342
2273999276,12cyclictest28-21ksoftirqd/212:16:332
22739992714,4cyclictest28-21ksoftirqd/209:52:272
22739992713,4cyclictest28-21ksoftirqd/211:46:472
22739992711,9cyclictest28-21ksoftirqd/211:43:332
22739992711,9cyclictest28-21ksoftirqd/209:57:022
22739992710,8cyclictest28-21ksoftirqd/210:57:582
2273899271,4cyclictest22-21ksoftirqd/109:14:591
2273799277,10cyclictest12-21ksoftirqd/010:53:490
22737992725,1cyclictest0-21swapper/007:11:450
2273999269,9cyclictest28-21ksoftirqd/208:52:062
2273999267,2cyclictest141rcu_preempt10:41:372
2273999265,8cyclictest28-21ksoftirqd/212:23:592
2273999264,11cyclictest28-21ksoftirqd/212:02:502
22739992613,4cyclictest28-21ksoftirqd/209:36:362
22739992613,4cyclictest28-21ksoftirqd/209:32:452
22739992610,4cyclictest28-21ksoftirqd/210:09:082
2273999259,4cyclictest28-21ksoftirqd/211:11:332
2273999258,3cyclictest28-21ksoftirqd/212:39:502
2273999257,9cyclictest28-21ksoftirqd/212:06:122
2273999257,10cyclictest28-21ksoftirqd/207:51:072
2273999255,9cyclictest28-21ksoftirqd/210:38:062
2273999255,9cyclictest28-21ksoftirqd/210:38:052
2273999255,11cyclictest28-21ksoftirqd/210:52:312
2273999255,10cyclictest28-21ksoftirqd/211:56:372
2273999255,10cyclictest16567-21cron08:10:002
22739992520,4cyclictest28-21ksoftirqd/212:15:002
22739992520,4cyclictest28-21ksoftirqd/212:14:592
22739992514,3cyclictest28-21ksoftirqd/210:18:562
22739992511,7cyclictest28-21ksoftirqd/209:22:322
22739992511,4cyclictest28-21ksoftirqd/212:30:002
2273899258,3cyclictest22-21ksoftirqd/107:50:161
2273899250,24cyclictest0-21swapper/107:12:141
22740992414,7cyclictest22560-21df10:55:163
2273999249,2cyclictest141rcu_preempt11:09:202
2273999248,9cyclictest28-21ksoftirqd/210:34:432
2273999248,4cyclictest28-21ksoftirqd/211:51:222
22739992419,3cyclictest271rcuc/211:17:012
22739992412,5cyclictest28-21ksoftirqd/211:00:252
22739992411,4cyclictest28-21ksoftirqd/211:30:512
22739992410,4cyclictest28-21ksoftirqd/209:08:592
22739992410,2cyclictest141rcu_preempt10:28:222
2273999240,23cyclictest0-21swapper/207:11:422
2273799249,7cyclictest141rcu_preempt11:13:190
2273799247,8cyclictest0-21swapper/011:09:560
2273799246,5cyclictest12-21ksoftirqd/008:35:230
2273799244,5cyclictest12-21ksoftirqd/012:10:160
2273799244,5cyclictest12-21ksoftirqd/012:10:150
22737992410,12cyclictest14808-21ssh11:37:390
257142230,2sleep10-21swapper/112:10:141
257142230,2sleep10-21swapper/112:10:141
2274099233,10cyclictest5715-21tail11:55:183
2274099232,14cyclictest794-21rm10:15:133
22740992310,10cyclictest19786-21cut12:30:143
2273999239,4cyclictest28-21ksoftirqd/210:22:412
2273999239,4cyclictest28-21ksoftirqd/207:58:252
2273999239,4cyclictest28-21ksoftirqd/207:19:292
2273999238,3cyclictest141rcu_preempt07:45:182
2273999238,2cyclictest141rcu_preempt11:25:212
2273999234,12cyclictest28-21ksoftirqd/208:41:502
2273999231,7cyclictest28-21ksoftirqd/208:10:232
22739992317,4cyclictest28-21ksoftirqd/208:25:012
2273899230,9cyclictest2966-21cat11:05:011
2273799238,9cyclictest25968-21kworker/u8:111:22:060
2273799237,4cyclictest12-21ksoftirqd/010:32:170
2273799237,4cyclictest12-21ksoftirqd/010:16:190
2273799235,3cyclictest141rcu_preempt10:40:250
2273799234,9cyclictest0-21swapper/011:30:520
2273799233,10cyclictest12-21ksoftirqd/007:40:010
22737992311,2cyclictest141rcu_preempt12:02:320
22737992311,10cyclictest131rcuc/009:18:160
186962230,3sleep30-21swapper/309:15:233
306902220,1sleep10-21swapper/109:49:041
22740992212,6cyclictest6488-21cpuspeed_turbos09:55:143
2273999228,4cyclictest28-21ksoftirqd/207:23:532
2273999226,9cyclictest28-21ksoftirqd/209:01:252
22739992216,4cyclictest28-21ksoftirqd/207:35:012
22738992214,6cyclictest560-21dbus-daemon09:40:001
22738992213,6cyclictest3017-21cat10:40:191
22738992213,6cyclictest278-21systemd-journal11:59:591
22738992212,8cyclictest25845-21seq11:21:171
22738992211,9cyclictest1439-21sendmail10:40:001
22738992211,9cyclictest1439-21sendmail10:40:001
2273799229,4cyclictest12-21ksoftirqd/010:59:420
2273799226,4cyclictest12-21ksoftirqd/009:32:280
2273799226,2cyclictest141rcu_preempt12:21:460
2273799223,10cyclictest17580-21ssh11:40:110
2273799222,3cyclictest141rcu_preempt09:35:010
2273799222,15cyclictest10772-21ssh09:10:130
2273799221,5cyclictest12-21ksoftirqd/012:08:580
22737992213,6cyclictest31977-21open_files09:25:210
22737992211,7cyclictest5894-21cut11:55:200
142302220,9sleep334-21ksoftirqd/309:36:023
102292220,1sleep10-21swapper/109:57:351
2274099211,8cyclictest1-21systemd12:36:533
22740992112,6cyclictest22517-21grep08:20:263
22740992111,7cyclictest6889-21cat09:00:183
2274099211,17cyclictest28172-21grep10:10:243
2273999218,4cyclictest28-21ksoftirqd/207:42:522
2273999218,3cyclictest28-21ksoftirqd/207:27:192
2273999218,3cyclictest28-21ksoftirqd/207:27:182
2273999217,4cyclictest28-21ksoftirqd/208:03:342
2273999212,7cyclictest28-21ksoftirqd/208:45:142
2273999212,10cyclictest28-21ksoftirqd/209:11:222
2273999211,7cyclictest28-21ksoftirqd/208:30:172
2273999211,5cyclictest18970-21apt-get08:15:132
22739992111,7cyclictest560-21dbus-daemon08:30:002
22738992114,4cyclictest636-21runrttasks11:32:381
22738992113,6cyclictest26380-21memory08:30:181
22738992113,6cyclictest13497-21forks10:00:151
22738992113,6cyclictest10729-21memory11:10:191
22738992112,7cyclictest3707-21perl11:05:171
22738992112,7cyclictest16431-21seq08:08:181
22738992112,6cyclictest3900-21cat07:40:131
22738992111,7cyclictest11392-21switchtime07:55:221
2273799217,3cyclictest141rcu_preempt10:23:570
2273799217,2cyclictest141rcu_preempt11:29:170
2273799216,8cyclictest141rcu_preempt12:32:450
2273799216,7cyclictest141rcu_preempt09:44:070
2273799216,2cyclictest12-21ksoftirqd/012:15:180
22737992113,6cyclictest13413-21cut12:25:140
22737992112,7cyclictest14142-21ntp_states10:00:200
151942214,2sleep1141rcu_preempt10:25:201
142412210,2sleep30-21swapper/311:37:053
326092200,11sleep334-21ksoftirqd/311:02:263
277162200,1sleep10-21swapper/111:47:111
235002200,1sleep30-21swapper/311:44:403
2274099204,13cyclictest560-21dbus-daemon09:25:013
2274099202,13cyclictest32084-21apt-get12:15:133
2274099201,8cyclictest19613-21pmu-power12:05:213
22740992012,6cyclictest560-21dbus-daemon10:50:003
22740992011,7cyclictest29476-21users08:35:263
2274099200,4cyclictest0-21swapper/310:21:003
2273999200,6cyclictest4132-21apt-get08:55:152
2273899202,7cyclictest21093-21python310:05:221
22738992013,5cyclictest4960-21ls08:55:211
22738992012,6cyclictest5020-21seq10:17:561
22738992012,6cyclictest32480-21entropy12:15:151
22738992012,6cyclictest32136-21cat08:45:001
22738992012,6cyclictest27957-21/usr/sbin/munin08:35:111
22738992012,6cyclictest21821-21seq11:19:001
22738992012,5cyclictest9323-21ntpq09:05:201
22738992011,7cyclictest611-21seq07:32:361
22738992010,7cyclictest2652-21cat07:35:251
22738992010,6cyclictest53050irq/142-enp3s0-12:30:141
2273799209,4cyclictest12-21ksoftirqd/009:45:190
2273799209,2cyclictest141rcu_preempt11:00:300
2273799206,4cyclictest12-21ksoftirqd/010:45:420
2273799204,9cyclictest141rcu_preempt09:23:050
2273799204,3cyclictest141rcu_preempt09:57:020
2273799203,10cyclictest141rcu_preempt10:08:430
22737992016,2cyclictest121ksoftirqd/008:00:220
22737992013,5cyclictest624-21gdbus08:29:590
22737992013,5cyclictest27991-21ntp_offset07:20:210
22737992011,7cyclictest31285-21sensors_temp08:40:230
2274099199,7cyclictest1381-21cut07:35:133
2274099199,6cyclictest26429-21ntpq09:45:223
2274099192,12cyclictest16833-21apt-get08:10:143
2274099191,5cyclictest0-21swapper/310:35:123
2274099191,5cyclictest0-21swapper/310:35:113
22740991912,5cyclictest8910-21gzip07:50:213
22740991912,5cyclictest6103-21wc09:00:003
22740991912,5cyclictest10253-21cat07:55:133
22740991912,4cyclictest4093-21missed_timers11:05:213
22740991911,6cyclictest8792-21cut09:05:153
22740991911,6cyclictest32468-21sed09:25:253
22740991911,6cyclictest32067-21sed07:30:163
22740991911,6cyclictest25132-21cut09:45:003
22740991911,6cyclictest22819-21cron11:20:003
22740991911,5cyclictest52750irq/139-enp3s0-10:07:553
2274099191,12cyclictest12971-21ssh12:25:113
22740991910,7cyclictest7796-21cut07:50:013
22740991910,7cyclictest3882-21df07:40:113
22740991910,6cyclictest30478-21forks08:40:153
22739991912,5cyclictest28-21ksoftirqd/208:40:012
2273899199,4cyclictest4761-21cron09:30:011
2273899197,8cyclictest12345-21apt-get08:00:121
22738991913,4cyclictest23584-21sh09:43:391
22738991913,4cyclictest22-21ksoftirqd/111:00:011
22738991912,5cyclictest20523-21seq11:41:401
22738991911,5cyclictest2001-21cpu08:50:121
22738991910,7cyclictest560-21dbus-daemon07:30:001
22738991910,7cyclictest560-21dbus-daemon07:30:001
2273799198,3cyclictest12-21ksoftirqd/008:30:180
2273799198,2cyclictest12-21ksoftirqd/009:00:150
2273799196,2cyclictest141rcu_preempt07:30:110
2273799191,8cyclictest141rcu_preempt07:25:180
2273799191,8cyclictest141rcu_preempt07:25:180
22737991914,3cyclictest26828-21dump-pmu-power07:20:000
22737991913,5cyclictest12-21ksoftirqd/007:50:010
22737991910,7cyclictest17903-21rt-features08:10:230
327642181,2sleep1141rcu_preempt09:50:181
2274099189,6cyclictest52750irq/139-enp3s0-10:00:113
2274099188,7cyclictest29200-21acpi07:25:123
2274099188,7cyclictest29200-21acpi07:25:113
2274099186,3cyclictest141rcu_preempt10:30:153
2274099182,11cyclictest23274-21apt-get11:20:123
22740991813,3cyclictest15449-21sendmail_mailst10:25:233
2274099181,15cyclictest6446-21ssh11:30:563
22740991811,5cyclictest11937-21ps09:10:233
22740991811,4cyclictest24944-21grep08:25:263
22740991810,6cyclictest7947-21awk12:20:253
22740991810,6cyclictest31417-21cut11:50:143
2273899186,9cyclictest6284-21apt-get12:20:111
2273899181,3cyclictest19177-21users09:15:271
22738991813,3cyclictest312-21gltestperf08:45:161
22738991812,4cyclictest4572-21sh11:54:551
22738991812,4cyclictest22342-21timerwakeupswit10:30:231
22738991811,5cyclictest28535-21unixbench_singl07:20:261
22738991811,5cyclictest26503-21seq07:16:141
22738991811,5cyclictest24436-21ls09:20:111
22738991811,5cyclictest24124-21interrupts08:25:191
22738991811,5cyclictest19867-21ntp_kernel_pll_08:15:211
22738991810,6cyclictest6593-21expr09:00:141
22738991810,6cyclictest5343-21grep09:30:121
22738991810,6cyclictest18417-21seq08:11:201
22738991810,6cyclictest13870-21cat10:25:001
22738991810,6cyclictest13325-21df12:25:121
22738991810,5cyclictest13267-21timerandwakeup12:00:261
2273799189,7cyclictest12-21ksoftirqd/008:10:000
2273799183,3cyclictest141rcu_preempt08:55:210
2273799182,11cyclictest3557-21apt-get07:40:100
2273799182,11cyclictest32271-21apt-get08:45:110
2273799181,8cyclictest141rcu_preempt07:55:140
22737991811,5cyclictest8536-21sed07:50:170
2273799181,10cyclictest19892-21ntp_states08:15:220
2274099178,6cyclictest11819-21apt-get12:00:133
2274099171,6cyclictest27019-21unixbench-2d12:10:263
2274099171,6cyclictest27019-21unixbench-2d12:10:263
22740991712,3cyclictest32244-21/usr/sbin/munin08:45:243
22740991712,3cyclictest25953-21sh08:30:143
22740991712,3cyclictest24879-21munin-setwarn07:15:013
22740991712,3cyclictest1966-21/usr/sbin/munin08:50:223
2273899176,9cyclictest0-21swapper/110:50:111
22738991712,3cyclictest7595-21latency_hist07:50:011
22738991711,4cyclictest21788-21cut08:20:181
2273799172,11cyclictest1992-21apt-get08:50:130
22737991712,3cyclictest22104-21pmu-power08:20:220
22737991711,4cyclictest10454-21sed09:10:000
2274099167,7cyclictest13460-21mailstats08:00:233
2274099167,6cyclictest27242-21cstates07:20:143
2274099161,11cyclictest16173-21idleruntime10:50:173
2274099156,6cyclictest20411-21unixbench_multi08:15:253
22740991510,3cyclictest14766-21cpuspeed_turbos08:05:123
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional