You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-29 - 17:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Wed Jul 29, 2026 12:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1704216498,34sleep00-21swapper/007:09:130
15512142105,26sleep30-21swapper/307:07:173
14552140103,26sleep10-21swapper/107:06:021
1445213698,26sleep20-21swapper/207:05:542
101912520,2sleep20-21swapper/209:49:572
65872510,1sleep20-21swapper/210:10:242
312812510,1sleep30-21swapper/311:50:133
294382460,1sleep00-21swapper/008:10:250
166342450,1sleep00-21swapper/009:54:330
312692390,1sleep10-21swapper/109:16:141
203799398,17cyclictest15286-21ssh09:53:093
2037993710,10cyclictest21606-21diskmemload09:33:013
197772370,10sleep334-21ksoftirqd/310:20:213
203799368,18cyclictest7544-21ssh10:11:123
203799368,10cyclictest21654-21date09:10:003
203799359,12cyclictest27029-21ssh10:01:333
203799358,9cyclictest21606-21diskmemload10:25:073
203799358,15cyclictest7606-21ssh09:47:213
203799358,10cyclictest21606-21diskmemload10:19:203
203799358,10cyclictest21606-21diskmemload10:19:203
203799356,17cyclictest0-21swapper/310:05:403
203799337,15cyclictest0-21swapper/309:26:303
164572330,2chrt0-21swapper/109:54:231
245882320,2sleep10-21swapper/108:00:211
203799329,9cyclictest29185-21rm09:39:543
203799328,9cyclictest0-21swapper/309:10:533
203799328,6cyclictest31996-21sh09:16:583
203799325,16cyclictest0-21swapper/309:20:033
203799317,8cyclictest1079-21sh09:42:293
203799317,11cyclictest17439-21runrttasks09:55:093
203799307,8cyclictest2078-21apt-get07:10:113
2037992914,12cyclictest10918-21sed11:00:263
2037992910,16cyclictest14480-21grep11:10:183
203799287,6cyclictest11122-21perl07:30:163
2037992810,14cyclictest265991ls08:05:203
2036992815,7cyclictest22050-21apt-get09:10:142
203799275,9cyclictest0-21swapper/312:25:233
203799275,10cyclictest32113-21runrttasks08:19:463
2037992715,9cyclictest24826-21grep11:35:143
2037992713,11cyclictest390-21munin-run10:40:013
2036992721,4cyclictest19603-21/usr/sbin/munin10:20:192
203799264,8cyclictest0-21swapper/312:20:033
203799264,8cyclictest0-21swapper/311:20:273
2037992615,9cyclictest16625-21gltestperf12:30:173
2037992614,9cyclictest23018-21cut11:30:173
2036992616,8cyclictest29841-21grep09:40:132
108972260,3sleep110894-21cron08:45:001
108972260,3sleep110894-21cron08:45:001
2037992515,7cyclictest7663-21cat07:20:253
2037992514,8cyclictest7819-21grep12:10:183
2037992514,8cyclictest5959-21unixbench_singl08:30:263
2037992514,8cyclictest4019-21grep12:00:263
2037992514,8cyclictest29587-21head08:10:243
2037992514,8cyclictest11547-21apt-key12:20:013
2037992513,9cyclictest7443-21grep10:55:133
2037992513,9cyclictest27378-21grep11:40:163
2037992511,11cyclictest21341-21cat07:55:013
2037992511,11cyclictest21341-21cat07:55:003
2037992511,11cyclictest15666-21cat08:55:113
2037992510,13cyclictest3181-21cut10:45:123
2036992518,5cyclictest18950irq/143-ahci[0009:19:592
2035992523,1cyclictest0-21swapper/107:13:121
2034992523,1cyclictest0-21swapper/007:13:160
203799247,12cyclictest0-21swapper/311:15:123
2037992414,8cyclictest12275-21fschecks_time11:05:153
2037992414,7cyclictest3609-21cat08:25:253
2037992414,7cyclictest24168-21grep08:00:173
2037992414,7cyclictest20900-21grep11:25:173
2037992414,7cyclictest18665-21cut12:35:153
2037992414,7cyclictest13034-21idleruntime-cro08:49:593
2037992414,7cyclictest1201-21head08:20:213
2037992413,8cyclictest7221-21ls10:54:593
2037992413,8cyclictest30015-21ps11:45:233
2037992413,8cyclictest15602-21grep07:40:163
2037992413,8cyclictest13667-21sort08:50:163
2037992413,8cyclictest1328-21tune2fs10:40:133
2037992412,9cyclictest29750-21/usr/sbin/munin10:30:223
2037992411,10cyclictest8703-21cat07:25:153
2037992411,10cyclictest6280-21grep12:05:213
2037992411,10cyclictest17849-21expr09:00:143
2037992410,11cyclictest17409-21cstates07:45:143
203699240,23cyclictest0-21swapper/207:13:162
203599242,15cyclictest16394-21sh09:30:121
203499247,3cyclictest141rcu_preempt08:40:000
297952230,3sleep10-21swapper/110:30:211
2037992314,7cyclictest6955-21cpuspeed_turbos08:35:133
2037992313,8cyclictest10414-21unixbench-2d08:40:283
2037992313,8cyclictest10414-21unixbench-2d08:40:273
2037992312,9cyclictest1062-21expr11:55:133
2037992312,8cyclictest21806-21/usr/sbin/munin07:55:153
2037992311,9cyclictest4686-21/usr/sbin/munin07:15:173
2037992311,9cyclictest12872-21grep07:35:143
2035992313,7cyclictest11952-21sed07:30:241
203599231,19cyclictest2773-21perf12:00:001
2034992315,6cyclictest4268-21ls12:00:280
2034992315,6cyclictest18224-21irqstats09:55:170
236372220,3sleep223643-21gltestperf09:35:162
203699224,4cyclictest21606-21diskmemload09:21:292
2035992217,3cyclictest15846-21latency_hist12:30:001
203499224,8cyclictest141rcu_preempt09:22:380
203499222,13cyclictest11957-21ssh10:15:130
203499222,13cyclictest11957-21ssh10:15:120
203499221,9cyclictest312852tr11:50:130
2034992216,3cyclictest21606-21diskmemload09:47:200
2034992213,6cyclictest22850-21grep09:10:200
2034992213,6cyclictest17831-21wc09:30:260
91972210,3sleep10-21swapper/110:12:531
203699211,5cyclictest0-21swapper/209:52:402
2036992115,4cyclictest28-21ksoftirqd/208:00:002
2036992113,6cyclictest5808-21ls08:30:262
2036992113,6cyclictest2371-21users11:55:272
2036992113,6cyclictest21817-21cut09:10:012
2036992113,5cyclictest636-21runrttasks09:56:462
2036992112,7cyclictest19003-21cat11:20:202
2036992111,7cyclictest26168-21ls10:25:182
2035992116,3cyclictest21597-21sh09:10:011
2035992113,6cyclictest13263-21fw_packets07:35:161
2035992113,6cyclictest12171-21diskstats11:05:131
203499213,4cyclictest24196-21apt-get10:00:120
2034992113,6cyclictest9919-21ntpq08:40:220
2034992113,6cyclictest9919-21ntpq08:40:210
2034992113,6cyclictest27209-21awk11:40:150
2034992112,7cyclictest952-21date08:20:180
2034992112,7cyclictest4844-21seq10:48:590
2034992112,7cyclictest12451-21date08:45:230
203499210,6cyclictest20141-21idleruntime-cro12:40:000
203699201,6cyclictest18164-21latency_hist10:20:012
203699201,6cyclictest18164-21latency_hist10:20:002
2036992012,6cyclictest998-21wc08:20:192
2036992012,6cyclictest9632-21cat08:40:192
2036992012,6cyclictest9632-21cat08:40:182
2036992012,6cyclictest21300-21sed07:55:012
2036992012,6cyclictest21300-21sed07:55:012
2036992012,6cyclictest18299-21cat12:35:122
2036992012,6cyclictest17415-21ntpq09:30:222
2036992012,6cyclictest16923-21needreboot12:30:222
2036992012,6cyclictest14901-21ntp_states11:10:232
2036992011,6cyclictest278-21systemd-journal10:35:012
2036992010,7cyclictest11100-21cat09:25:232
203699200,18cyclictest15951-21ntp_kernel_pll_07:40:212
203599209,8cyclictest560-21dbus-daemon08:24:591
203599203,2cyclictest141rcu_preempt09:36:451
203599202,15cyclictest18260-21sort10:20:011
203599202,15cyclictest18260-21sort10:20:001
203599201,4cyclictest21606-21diskmemload09:28:161
2035992012,6cyclictest32520-21seq09:41:281
2035992012,6cyclictest29040-21seq08:10:191
2035992010,7cyclictest22426-21df_inode09:10:151
2035992010,7cyclictest10405-21ntp_states11:00:221
203499209,9cyclictest26351-21cut10:25:200
203499204,3cyclictest21606-21diskmemload09:28:060
203499201,5cyclictest31582-21irqstats10:05:180
203499201,5cyclictest19319-21irqstats10:20:170
2034992014,3cyclictest21606-21diskmemload09:17:470
2034992012,6cyclictest4755-21expr08:30:150
2034992012,6cyclictest29183-21tune2fs10:30:160
2034992012,6cyclictest24599-21cat11:35:000
2034992012,6cyclictest21939-21seq11:25:310
2034992012,6cyclictest21143-21seq07:54:110
2034992012,6cyclictest21143-21seq07:54:100
2034992012,6cyclictest14224-21open_inodes08:50:220
2034992012,6cyclictest10141-21seq07:26:470
2034992012,5cyclictest6046-21ntp_kernel_err12:05:190
2034992011,6cyclictest560-21dbus-daemon11:10:060
203499201,10cyclictest0-21swapper/010:13:460
203699199,7cyclictest27195-21entropy11:40:142
203699198,6cyclictest30863-21ssh10:05:132
203699197,9cyclictest7035-21taskset07:20:192
203699192,14cyclictest10950-21head11:00:272
203699191,5cyclictest0-21swapper/210:01:342
2036991912,5cyclictest5153-21cat12:05:112
2036991912,5cyclictest3651-21sed08:25:242
2036991912,5cyclictest18297-21wc09:00:182
2036991911,6cyclictest9367-21wc12:15:012
2036991911,6cyclictest30444-21awk11:45:272
2036991910,7cyclictest8984-21perl07:25:162
2036991910,7cyclictest14955-21cat07:40:002
203599199,7cyclictest16097-21apt-get12:30:141
203599192,14cyclictest5707-21sed12:05:161
203599192,12cyclictest0-21swapper/107:20:121
2035991913,4cyclictest15038-21acpi07:40:121
2035991912,5cyclictest9396-21cat12:15:001
2035991912,5cyclictest2987-21seq12:00:021
2035991912,4cyclictest32020-21sh10:05:221
2035991911,6cyclictest18937-21sed12:35:181
2035991910,7cyclictest4540-21awk10:45:251
2035991910,7cyclictest26732-21date10:25:241
2035991910,7cyclictest1348-21cut10:40:141
203599190,6cyclictest22-21ksoftirqd/109:56:381
2034991914,3cyclictest1558-21kernelversion10:40:170
2034991912,5cyclictest25752-21find08:05:110
2034991912,5cyclictest19456-21cron09:05:010
2034991912,5cyclictest17329-21date12:30:260
2034991912,5cyclictest12457-21cat11:05:170
2034991911,6cyclictest7492-21expr12:10:150
2034991911,6cyclictest560-21dbus-daemon07:20:000
203499191,16cyclictest53050irq/142-enp3s0-12:15:120
2034991911,6cyclictest1865-21which11:55:220
2034991911,6cyclictest17521-21date09:00:000
2034991911,6cyclictest14134-21cat12:25:150
2034991911,6cyclictest13534-21seq12:23:050
2034991910,7cyclictest16905-21seq07:44:390
203499190,10cyclictest0-21swapper/009:35:050
203699189,7cyclictest968-21cat10:40:122
203699187,8cyclictest26240-21gltestperf08:05:162
203699184,12cyclictest24969-21timerwakeupswit08:00:252
203699182,12cyclictest15421-21apt-get08:55:112
203699182,11cyclictest30259-21apt-get08:15:122
203699182,11cyclictest16116-21apt-get11:15:132
2036991811,5cyclictest5451-21expr10:50:142
2036991811,5cyclictest4206-21cat07:15:122
2036991811,5cyclictest15853-21pluginstate12:29:592
2036991811,5cyclictest10750-21expr07:30:142
203699181,13cyclictest22562-21chrt11:30:132
203699181,12cyclictest3028-21apt-get12:00:142
2036991810,6cyclictest19287-21date07:50:002
2036991810,5cyclictest9396-21cat11:00:002
203599182,2cyclictest141rcu_preempt08:00:011
203599182,11cyclictest9444-21apt-get12:15:141
2035991813,4cyclictest22-21ksoftirqd/111:30:001
2035991813,3cyclictest8354-21processes10:55:221
2035991813,3cyclictest29990-21sh10:04:361
2035991811,5cyclictest30406-21expr08:15:131
2035991811,5cyclictest14200-21expr11:10:161
2035991810,6cyclictest26467-21seq11:38:421
2035991810,6cyclictest18908-21seq07:47:561
2035991810,6cyclictest17104-21seq08:57:351
203599180,7cyclictest40842ls09:45:141
203499189,7cyclictest17200-21apt-get07:45:130
203499189,6cyclictest13229-21tune2fs07:35:160
2034991811,5cyclictest6984-21seq10:54:010
203499181,15cyclictest6910-21gltestperf07:20:170
2034991811,5cyclictest29993-21ls09:40:130
2034991811,5cyclictest11099-21seq11:01:380
2034991811,4cyclictest24199-21hwlatdetect08:00:170
2034991810,6cyclictest8210-21ntp_kernel_err10:55:200
2034991810,6cyclictest32694-21seq10:38:190
2034991810,6cyclictest26452-21seq11:38:310
2034991810,6cyclictest21828-21awk07:55:160
2034991810,5cyclictest2479-21cut08:25:140
203699179,6cyclictest734-21cut11:55:002
203699179,6cyclictest11285-21grep08:45:132
203699172,11cyclictest28120-21apt-get08:10:132
2036991712,3cyclictest6799-21packagekitd08:35:002
2036991712,3cyclictest21380-21pmu-power11:25:232
2036991712,3cyclictest11788-21/usr/sbin/munin12:20:192
2036991711,4cyclictest31161-21expr10:35:132
2036991710,5cyclictest9709-21expr12:15:132
2036991710,5cyclictest28-21ksoftirqd/208:50:132
203599179,6cyclictest16251-21expr11:15:121
203599179,6cyclictest13047-21basename08:50:001
203599178,6cyclictest7250-21date10:55:011
203599176,3cyclictest3623-21cut09:20:161
203599172,11cyclictest6850-21apt-get08:35:131
2035991712,3cyclictest28974-21/usr/sbin/munin11:45:181
2035991712,3cyclictest13924-21interrupts08:50:181
2035991710,5cyclictest5251-21cut07:15:231
2035991710,5cyclictest4482-21cat08:30:111
2035991710,5cyclictest32601-21awk11:50:261
2035991710,5cyclictest28151-21sed11:40:251
2035991710,5cyclictest23628-21rt-features11:30:241
2035991710,5cyclictest20143-21sed07:50:201
2035991710,5cyclictest20143-21sed07:50:191
2035991710,5cyclictest19621-21wc11:20:271
2035991710,5cyclictest19536-21grep10:20:181
2035991710,5cyclictest18203-21cat09:00:161
2035991710,4cyclictest32424-21grep10:35:261
203499179,5cyclictest29244-21expr11:45:150
2034991710,5cyclictest20601-21ntp_kernel_err09:05:200
203699169,5cyclictest3259-21expr10:45:122
203699168,6cyclictest25025-21df11:35:152
203699161,6cyclictest12758-21ntp_states11:05:212
203599169,5cyclictest8770-21expr07:25:151
203599169,5cyclictest12545-21cat12:20:181
2035991611,4cyclictest22-21ksoftirqd/108:30:001
203499169,5cyclictest11146-21grep07:30:160
203499168,6cyclictest16106-21awk11:15:110
203499161,13cyclictest31681-21unixbench_multi08:15:240
2034991611,3cyclictest18256-21/usr/sbin/munin11:20:170
203599151,2cyclictest141rcu_preempt08:05:131
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional