You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-07 - 01:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Mon Jul 06, 2026 12:45:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
23337217196,24sleep20-21swapper/207:07:202
235172165114,29sleep30-21swapper/307:09:413
232892162106,28sleep10-21swapper/107:06:421
235092158102,29sleep00-21swapper/007:09:340
2451821350,1sleep10-21swapper/110:06:381
1096521140,2sleep10-21swapper/111:32:511
137512580,2sleep20-21swapper/210:23:012
4972570,1sleep10-21swapper/111:25:161
294362570,2sleep20-21swapper/207:20:232
112632570,1sleep1211rcuc/110:20:311
289072560,2sleep30-21swapper/312:10:133
176042530,1sleep122-21ksoftirqd/109:15:021
102052520,4sleep10-21swapper/109:33:081
118002510,1sleep00-21swapper/009:10:190
206342460,1sleep20-21swapper/212:04:222
139142460,1sleep30-21swapper/310:23:103
119072420,3chrt0-21swapper/211:10:002
48152410,2chrt0-21swapper/311:28:273
22012410,1sleep20-21swapper/209:26:552
92132390,1chrt0-21swapper/007:50:140
26722370,2chrt0-21swapper/307:35:153
23816993713,4cyclictest16719-21ssh12:00:280
23816993612,5cyclictest21425-21seq10:52:320
59062350,2chrt0-21swapper/112:16:411
23816993411,5cyclictest25281-21wc08:25:250
23816993410,5cyclictest6009-21sed11:05:160
195342340,1sleep30-21swapper/310:50:393
2381699331,18cyclictest0-21swapper/007:21:170
23816993311,6cyclictest14581-21cat12:00:000
23816993311,3cyclictest5466-21sh12:16:150
2381699329,6cyclictest4207-21expr10:40:120
2381699321,5cyclictest27128-21ssh11:45:110
2381699321,13cyclictest560-21dbus-daemon11:20:000
2381699319,5cyclictest17203-21seq11:37:190
2381699311,9cyclictest2670-21ssh11:50:200
23816993112,3cyclictest15074-21sh12:24:070
23816993111,8cyclictest29402-21cut10:35:000
23816993111,3cyclictest27193-21rm10:56:340
23818993018,2cyclictest141rcu_preempt10:34:582
2381699308,5cyclictest13025-21grep08:00:000
2381699302,11cyclictest20940-21python309:40:230
23816993012,4cyclictest31466-21swap07:25:230
23816993011,3cyclictest30032-21sh12:10:230
23816993011,3cyclictest10560-21diskmemload09:57:150
23816993010,3cyclictest30593-21missed_timers10:35:180
23816993010,3cyclictest26482-21sh12:08:270
309062290,1sleep10-21swapper/112:10:591
2381899292,16cyclictest28-21ksoftirqd/211:42:272
23818992910,10cyclictest28-21ksoftirqd/212:38:442
2381699299,4cyclictest12-21ksoftirqd/007:15:000
2381699298,6cyclictest12-21ksoftirqd/009:25:240
2381699297,4cyclictest0-21swapper/011:20:210
2381699295,9cyclictest0-21swapper/011:03:490
23816992911,4cyclictest17770-21swap12:25:240
2381699291,11cyclictest560-21dbus-daemon10:20:000
194522290,2sleep00-21swapper/009:15:250
180372290,2sleep20-21swapper/211:38:092
156202290,2chrt0-21swapper/109:36:451
23818992810,6cyclictest28-21ksoftirqd/211:02:212
2381799281,25cyclictest0-21swapper/107:17:511
2381799281,25cyclictest0-21swapper/107:17:511
2381699289,4cyclictest10560-21diskmemload11:32:420
2381699281,8cyclictest13888-21ssh09:35:220
23816992810,3cyclictest10560-21diskmemload10:21:490
197382280,2sleep20-21swapper/212:27:012
49702270,2sleep30-21swapper/308:55:173
2381899279,4cyclictest28-21ksoftirqd/212:31:372
2381899279,4cyclictest28-21ksoftirqd/209:31:182
2381899279,10cyclictest28-21ksoftirqd/209:46:352
2381899279,10cyclictest28-21ksoftirqd/209:46:352
23818992714,7cyclictest53050irq/142-enp3s0-10:10:112
23818992712,4cyclictest28-21ksoftirqd/210:26:152
23818992710,5cyclictest28-21ksoftirqd/211:30:002
23818992710,4cyclictest28-21ksoftirqd/211:45:302
2381799271,25cyclictest0-21swapper/107:23:401
2381699276,5cyclictest0-21swapper/008:45:140
2381699271,24cyclictest0-21swapper/007:17:400
2381699271,24cyclictest0-21swapper/007:17:400
23816992710,3cyclictest10560-21diskmemload11:10:310
2381699270,13cyclictest2581-21ssh09:50:310
2381999261,23cyclictest0-21swapper/307:20:463
2381899269,9cyclictest28-21ksoftirqd/210:55:252
2381899261,9cyclictest28-21ksoftirqd/209:24:452
23818992614,4cyclictest28-21ksoftirqd/211:30:272
23818992613,4cyclictest28-21ksoftirqd/209:55:562
2381899261,23cyclictest0-21swapper/207:17:112
2381899261,23cyclictest0-21swapper/207:17:112
23818992612,2cyclictest141rcu_preempt10:35:162
23818992611,5cyclictest28-21ksoftirqd/209:17:362
23817992610,5cyclictest22-21ksoftirqd/111:15:231
2381699267,6cyclictest12-21ksoftirqd/007:30:110
2381699261,9cyclictest5712-21/usr/sbin/munin09:30:220
217772260,1sleep30-21swapper/311:16:313
2381999251,23cyclictest0-21swapper/307:18:373
2381999251,23cyclictest0-21swapper/307:18:373
2381899259,4cyclictest28-21ksoftirqd/211:51:352
2381899251,17cyclictest28-21ksoftirqd/209:53:172
23818992511,4cyclictest28-21ksoftirqd/212:17:582
23817992523,1cyclictest0-21swapper/108:22:001
68772240,8sleep122-21ksoftirqd/110:17:541
2381899247,4cyclictest28-21ksoftirqd/212:20:192
2381899246,4cyclictest28-21ksoftirqd/211:16:172
23818992412,5cyclictest28-21ksoftirqd/209:41:362
2381899241,11cyclictest90872cstates07:50:142
23818992410,4cyclictest28-21ksoftirqd/209:12:302
23818992410,2cyclictest141rcu_preempt11:57:412
23817992414,7cyclictest11796-21cut10:45:191
23817992411,3cyclictest5324-21latency_hist09:30:001
23816992417,5cyclictest12-21ksoftirqd/010:05:000
2381699240,4cyclictest24535-21smart_sda12:30:270
90102230,2sleep1211rcuc/109:05:151
2381899239,4cyclictest28-21ksoftirqd/212:05:082
2381899238,4cyclictest28-21ksoftirqd/212:13:322
23818992321,1cyclictest0-21swapper/207:33:152
23818992315,5cyclictest31303-21ntpq08:40:212
23818992313,7cyclictest19312-21users10:50:262
23818992310,2cyclictest141rcu_preempt10:42:322
2381799233,17cyclictest16696-21grep10:25:171
23816992314,7cyclictest5936-21seq08:56:030
20242230,1sleep30-21swapper/311:02:023
23819992213,7cyclictest32664-21which09:25:213
23819992211,8cyclictest9246-21df_inode07:50:153
2381899229,2cyclictest28-21ksoftirqd/211:11:072
2381899225,3cyclictest141rcu_preempt10:46:252
2381899224,9cyclictest28-21ksoftirqd/210:19:382
23818992216,4cyclictest28-21ksoftirqd/207:40:012
2381899221,15cyclictest21556-21ssh10:05:112
23818992211,4cyclictest28-21ksoftirqd/211:22:192
2381899221,10cyclictest0-21swapper/208:55:232
23818992210,4cyclictest28-21ksoftirqd/210:01:102
2381899220,13cyclictest271rcuc/208:35:242
2381799229,9cyclictest754-21ssh09:50:111
23817992213,6cyclictest14570-21apt-get10:00:111
190072220,2sleep30-21swapper/310:26:313
23819992112,7cyclictest11887-21memory09:10:203
23819992111,8cyclictest18180-21seq12:01:553
2381899218,4cyclictest28-21ksoftirqd/207:42:512
2381899216,4cyclictest28-21ksoftirqd/209:35:032
23818992110,5cyclictest28-21ksoftirqd/208:49:322
23818992110,4cyclictest28-21ksoftirqd/207:28:492
23817992113,6cyclictest11754-21cron11:10:001
23817992112,7cyclictest3739-21users08:50:261
23817992112,6cyclictest31291-21switchtime12:35:241
23817992111,8cyclictest30225-21tune2fs10:35:151
23817992111,7cyclictest17818-21df_inode08:10:141
23816992114,4cyclictest636-21runrttasks11:43:270
23819992014,4cyclictest34-21ksoftirqd/310:45:003
23819992014,4cyclictest31640-21sh12:35:273
23819992014,4cyclictest25549-21sh12:31:063
23819992014,4cyclictest12366-21sh11:57:503
23819992013,4cyclictest4481-21sh10:15:303
23819992012,6cyclictest3587-21hostname12:15:173
23819992012,6cyclictest30290-21seq09:24:203
23819992012,6cyclictest24528-21sed07:10:193
23819992012,6cyclictest22878-21tr12:05:243
23819992012,6cyclictest16132-21cat12:25:013
23819992011,7cyclictest560-21dbus-daemon09:20:003
23819992011,7cyclictest24684-21date08:25:183
23819992011,7cyclictest21175-21cat09:40:243
2381999201,15cyclictest6818-21sh11:30:123
23819992010,6cyclictest52950irq/141-enp3s0-11:20:123
2381899209,4cyclictest28-21ksoftirqd/208:00:182
2381899202,12cyclictest636-21runrttasks07:45:122
23818992014,4cyclictest28-21ksoftirqd/208:00:002
23818992012,6cyclictest7083-21smartctl09:00:162
2381799203,4cyclictest0-21swapper/110:58:141
2381799201,5cyclictest0-21swapper/111:42:111
23817992014,4cyclictest22-21ksoftirqd/110:00:011
23817992014,4cyclictest22-21ksoftirqd/107:40:011
23817992012,6cyclictest5968-21seq08:56:261
23817992012,6cyclictest29820-21seq08:36:331
23817992012,6cyclictest28482-21seq09:45:471
23817992012,6cyclictest28482-21seq09:45:471
23817992012,6cyclictest22749-21wc12:30:001
23817992012,6cyclictest12113-21pluginstate09:10:231
23817992011,7cyclictest31949-21seq08:41:101
23817992010,7cyclictest14505-21cron12:00:001
2381699209,9cyclictest32593-21sh10:12:570
2381699209,8cyclictest6846-21expr07:45:140
2381699207,3cyclictest176392switchtime10:25:260
23816992013,4cyclictest24461-21sh09:20:030
23816992012,6cyclictest4414-21cat07:40:100
23816992012,6cyclictest4367-21date07:40:000
23816992012,6cyclictest11789-21needreboot10:45:190
23816992011,7cyclictest15114-21basename08:04:590
144822200,4sleep3341ksoftirqd/310:00:063
2381999199,7cyclictest32762-21basename09:50:003
2381999199,7cyclictest32762-21basename09:49:593
2381999197,10cyclictest12247-21cpuspeed_turbos11:10:133
2381999192,12cyclictest26075-21apt-get08:30:113
23819991914,3cyclictest25091-21sh10:07:123
23819991913,4cyclictest30274-21sh11:46:503
23819991913,4cyclictest28991-21sh10:58:223
23819991913,4cyclictest2719-21sh09:50:383
23819991913,4cyclictest10482-21sh11:08:443
23819991912,5cyclictest24877-21seq11:43:143
23819991911,6cyclictest9580-21seq09:55:303
23819991911,6cyclictest4987-21which07:40:163
23819991911,6cyclictest29299-21sendmail_mailqu08:35:223
23819991911,6cyclictest12403-21sed10:45:243
23819991911,5cyclictest7481-21uptime09:30:263
23819991910,7cyclictest6990-21perl09:00:153
23819991910,7cyclictest31212-21ntpq07:25:203
23819991910,6cyclictest29469-21cat10:35:003
2381899198,3cyclictest141rcu_preempt08:30:162
2381899192,12cyclictest8666-21apt-get09:05:122
23818991914,3cyclictest271rcuc/207:10:172
23818991913,4cyclictest28-21ksoftirqd/208:20:002
23818991911,6cyclictest17434-21date08:10:012
23818991911,6cyclictest17434-21date08:10:012
2381799193,6cyclictest0-21swapper/110:51:071
2381799193,11cyclictest13212-21apt-get08:00:121
23817991914,3cyclictest25362-21uptime07:10:271
23817991913,4cyclictest22-21ksoftirqd/112:25:001
23817991912,5cyclictest5108-21ntp_kernel_pll_10:40:191
23817991912,5cyclictest23447-21expr12:30:131
23817991912,4cyclictest10560-21diskmemload11:21:091
23817991911,6cyclictest29275-21uname10:10:181
23817991911,6cyclictest25364-21interrupts09:20:171
23817991911,6cyclictest14896-21ls11:35:211
23817991910,7cyclictest21806-21awk12:05:141
23817991910,7cyclictest13512-21python311:10:231
23817991910,6cyclictest7825-21timerandwakeup07:45:241
23817991910,6cyclictest28826-21uptime11:45:251
23816991914,3cyclictest1921-21sh11:25:330
23816991910,7cyclictest30940-21ntp_states12:35:210
2381999182,11cyclictest30376-21apt-get08:40:133
2381999182,11cyclictest2234-21apt-get08:50:133
2381999181,8cyclictest3400-21ssh11:50:383
23819991813,3cyclictest14529-21sh09:35:403
23819991812,4cyclictest17761-21sh12:25:243
23819991812,4cyclictest17015-21sh11:37:083
23819991811,5cyclictest11987-21awk07:55:233
23819991810,6cyclictest8538-21wc07:50:003
23819991810,6cyclictest31306-21date10:35:243
23819991810,6cyclictest29617-21run-parts10:10:223
23819991810,5cyclictest8850-21cat09:05:133
2381899186,4cyclictest141rcu_preempt08:50:132
23818991811,5cyclictest23080-21sed08:20:262
2381799189,7cyclictest31541-21tune2fs11:00:151
2381799189,7cyclictest26990-21cat08:30:201
2381799189,6cyclictest9211-21seq07:50:141
2381799181,5cyclictest0-21swapper/111:50:011
23817991813,3cyclictest23167-21sh09:42:141
23817991812,4cyclictest7088-21sh09:00:161
23817991812,4cyclictest32489-21/usr/sbin/munin08:45:261
23817991811,5cyclictest25762-21seq08:29:261
23817991810,6cyclictest4722-21cat07:40:131
23817991810,6cyclictest30809-21expr07:25:151
23817991810,6cyclictest26293-21seq10:32:021
23817991810,6cyclictest15313-21sshd08:05:021
23817991810,6cyclictest15313-21sshd08:05:021
2381699186,10cyclictest0-21swapper/009:45:150
2381699186,10cyclictest0-21swapper/009:45:150
23816991814,2cyclictest22346-21sshd08:20:170
23816991811,5cyclictest21660-21date08:20:010
23816991811,5cyclictest15577-21expr08:05:130
23816991811,5cyclictest15577-21expr08:05:130
23816991810,6cyclictest6847-21cat09:00:120
2381999179,5cyclictest17499-21cpu08:10:133
2381999178,6cyclictest20586-21pmu-power08:15:233
2381999172,11cyclictest32514-21apt-get08:45:123
23819991710,5cyclictest22389-21cat08:20:173
23819991710,5cyclictest13410-21expr08:00:123
2381899178,7cyclictest24443-21gltestperf08:25:152
23818991710,5cyclictest28-21ksoftirqd/208:15:002
23817991712,3cyclictest14970-21/usr/sbin/munin12:00:131
23817991711,4cyclictest356-21fschecks_count07:30:131
23817991710,5cyclictest11478-21tune2fs07:55:171
2381699179,6cyclictest9253-21basename09:05:170
2381699178,6cyclictest21772-21cat10:05:120
23816991711,4cyclictest18714-21sed08:10:230
23816991710,5cyclictest28427-21expr08:35:140
23816991710,5cyclictest2663-21tune2fs08:50:150
23816991710,5cyclictest12-21ksoftirqd/008:40:150
2381999168,5cyclictest32537-21cpuspeed_turbos07:30:123
2381799169,5cyclictest20710-21gzip08:15:231
2381999151,10cyclictest15352-21apt-get08:05:133
2381999151,10cyclictest15352-21apt-get08:05:133
2381699158,5cyclictest26148-21cpuspeed_turbos08:30:120
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional