You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-04 - 23:58

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Tue Aug 04, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
993121790,3sleep30-21swapper/307:05:183
108472176100,26sleep20-21swapper/207:05:422
11020217498,25sleep10-21swapper/107:07:571
109072139103,25sleep00-21swapper/007:06:290
1547121270,1sleep30-21swapper/311:47:403
260412640,2sleep00-21swapper/011:55:240
192242640,2sleep00-21swapper/011:25:390
29602610,2sleep00-21swapper/011:38:300
197822560,10sleep012-21ksoftirqd/011:50:260
209092550,1sleep30-21swapper/310:39:183
95652540,2sleep30-21swapper/312:31:233
92462520,1sleep20-21swapper/209:41:022
188222510,1sleep10-21swapper/110:37:111
38982460,2chrt0-21swapper/010:25:340
198052460,3sleep00-21swapper/012:15:140
83132430,1sleep30-21swapper/309:16:443
128042380,1sleep10-21swapper/111:45:211
212662370,2chrt0-21swapper/310:15:143
127662370,1sleep00-21swapper/012:34:370
308232300,3sleep012-21ksoftirqd/009:57:420
308232300,3sleep012-21ksoftirqd/009:57:410
39312290,9sleep334-21ksoftirqd/312:03:223
173622290,2sleep10-21swapper/109:47:311
168042290,4sleep216755-21ssh11:25:012
99702280,2chrt0-21swapper/311:43:523
48982280,2sleep20-21swapper/209:38:252
1145799284,11cyclictest34-21ksoftirqd/311:33:093
1145799284,11cyclictest34-21ksoftirqd/311:33:083
307062270,8sleep1221ksoftirqd/111:59:381
1145499271,25cyclictest0-21swapper/007:12:330
296242260,3chrt0-21swapper/007:50:170
1145499263,15cyclictest30603-21diskmemload11:40:140
35722250,2chrt0-21swapper/212:27:042
1145799253,15cyclictest30603-21diskmemload10:20:133
1145799253,10cyclictest34-21ksoftirqd/310:41:093
1145799252,16cyclictest17320-21ssh11:25:143
11457992515,3cyclictest34-21ksoftirqd/309:55:213
11457992515,3cyclictest34-21ksoftirqd/309:55:203
1145799251,1cyclictest0-21swapper/307:12:283
1145799251,16cyclictest141rcu_preempt12:22:553
1145699251,23cyclictest0-21swapper/207:12:552
11455992523,1cyclictest0-21swapper/107:13:011
1145499254,18cyclictest16336-21kworker/0:010:35:000
11454992517,6cyclictest16674-21ls11:24:590
11454992514,7cyclictest4570-21cut11:15:170
11454992510,8cyclictest7769-21ssh10:05:110
130122240,2sleep20-21swapper/210:33:032
1145799247,5cyclictest34-21ksoftirqd/309:25:143
1145499242,16cyclictest23981-21chrt11:30:120
1145499242,16cyclictest23981-21chrt11:30:110
32512230,3sleep00-21swapper/010:00:500
1145799234,9cyclictest141rcu_preempt12:09:253
1145799232,15cyclictest1181-21ssh10:00:163
11456992313,7cyclictest32251-21fw_packets12:00:152
11455992315,6cyclictest19253-21sh12:15:101
204772220,2sleep30-21swapper/307:30:133
1145799229,11cyclictest25562-21if_enp3s011:55:193
1145799225,8cyclictest34-21ksoftirqd/312:38:333
1145799224,4cyclictest34-21ksoftirqd/311:10:233
11457992212,2cyclictest141rcu_preempt12:15:553
11457992210,3cyclictest34-21ksoftirqd/310:05:223
1145599222,15cyclictest11254-21ssh09:19:251
11455992213,7cyclictest19802-21sed11:50:271
11455992213,6cyclictest19593-21df09:25:151
11455992212,7cyclictest5438-21fw_packets11:40:191
11455992212,6cyclictest10778-21seq10:06:441
1145499222,17cyclictest31070-21apt-get12:00:030
11454992214,5cyclictest52750irq/139-enp3s0-12:35:060
11454992213,5cyclictest14133-21gltestperf09:45:170
1145799219,3cyclictest34-21ksoftirqd/312:10:133
1145799219,2cyclictest141rcu_preempt11:00:343
1145799217,3cyclictest141rcu_preempt11:16:493
1145799216,8cyclictest141rcu_preempt12:25:333
1145799212,13cyclictest24692-21apt-get07:40:143
1145799211,9cyclictest173612ssh09:47:323
11457992113,6cyclictest7362-21tune2fs09:40:163
11457992113,6cyclictest21179-21sed09:50:203
11457992113,6cyclictest17062-21seq10:11:233
11457992112,7cyclictest560-21dbus-daemon07:19:593
11457992112,7cyclictest16239-21perl07:20:163
11457992112,6cyclictest3402-21cut10:50:143
1145799211,18cyclictest6508-21awk09:40:013
11457992110,3cyclictest141rcu_preempt09:10:123
11457992110,3cyclictest141rcu_preempt09:10:113
1145699218,11cyclictest30526-21seq11:10:182
1145699212,13cyclictest13642-21apt-get07:15:132
1145699211,7cyclictest30052-21python310:45:242
1145699211,7cyclictest30052-21python310:45:232
11456992113,6cyclictest8817-21sh11:18:482
11456992112,6cyclictest32568-21ls09:10:262
11456992112,6cyclictest32568-21ls09:10:262
11456992112,6cyclictest27291-21entropy09:55:172
11456992112,6cyclictest27291-21entropy09:55:162
11456992111,7cyclictest16705-21ntpq07:20:202
11455992113,6cyclictest23955-21seq10:40:351
11455992112,7cyclictest21996-21seq08:49:471
11455992112,6cyclictest4171-21diskstats11:15:141
11455992111,7cyclictest8416-21python308:15:231
1145499212,13cyclictest640-21snmpd09:30:120
1145499210,19cyclictest2219-21seq11:13:500
107332211,4sleep210892-21unixbench_singl08:20:252
107332211,4sleep210892-21unixbench_singl08:20:252
278872200,4sleep027893-21cpuspeed_turbos10:20:130
1145799208,2cyclictest34-21ksoftirqd/310:55:143
1145799208,2cyclictest141rcu_preempt10:33:283
1145799203,15cyclictest30391-21unixbench_singl10:45:263
1145799203,15cyclictest30391-21unixbench_singl10:45:253
11457992012,6cyclictest23766-21sed07:35:263
1145699209,7cyclictest17876-21cat11:00:262
11456992015,3cyclictest271rcuc/208:40:002
1145699201,4cyclictest0-21swapper/210:24:402
11456992014,5cyclictest28-21ksoftirqd/209:30:002
11456992014,4cyclictest5830-21sh10:51:352
11456992014,4cyclictest30691-21sh11:59:382
11456992013,4cyclictest24815-21sh10:41:272
11456992013,4cyclictest15790-21sh10:59:562
11456992012,6cyclictest7930-21seq09:16:202
11456992012,6cyclictest560-21dbus-daemon09:04:592
11456992012,6cyclictest32367-21cat07:55:252
11456992012,6cyclictest26436-21cron09:54:592
11456992012,5cyclictest22969-21cat11:05:122
11456992011,7cyclictest25348-21awk08:55:222
11456992011,6cyclictest8604-21memory10:05:182
1145699201,16cyclictest8275-21cron10:30:002
11456992011,6cyclictest19469-21cat12:15:122
1145599209,9cyclictest12478-21/usr/sbin/munin09:20:141
1145599209,8cyclictest22169-21cron07:35:001
1145599209,3cyclictest62212sshd10:03:511
11455992015,3cyclictest17686-21apt-get07:25:011
11455992013,5cyclictest22-21ksoftirqd/112:30:011
11455992013,4cyclictest30603-21diskmemload10:48:411
11455992013,4cyclictest30603-21diskmemload10:48:401
11455992013,4cyclictest25109-21sh09:53:381
11455992012,6cyclictest560-21dbus-daemon09:40:001
11455992012,6cyclictest5459-21diskstats08:10:161
11455992012,6cyclictest13886-21seq08:30:131
11455992012,6cyclictest12625-21seq10:56:421
1145599200,9cyclictest30518-21needreboot11:10:171
1145499202,12cyclictest12351-21apt-get09:20:030
1145499201,4cyclictest0-21swapper/009:54:490
11454992014,4cyclictest18950irq/143-ahci[0010:15:010
11454992012,6cyclictest4591-21seq12:28:060
11454992012,6cyclictest24576-21cat08:55:140
11454992011,7cyclictest26689-21date07:45:000
11454992011,6cyclictest4458-21grep08:05:270
1145499200,8cyclictest15540-21apt07:20:010
169022190,4sleep1211rcuc/112:12:581
133242190,3chrt0-21swapper/011:45:280
1145799198,9cyclictest0-21swapper/311:20:183
1145799198,8cyclictest24370-21switchtime11:05:263
1145799194,13cyclictest181721/usr/sbin/munin11:50:163
1145799191,6cyclictest32671-21ssh11:35:443
11457991911,6cyclictest8449-21sed08:15:253
11457991911,6cyclictest17297-21unixbench_singl08:35:253
11457991910,2cyclictest141rcu_preempt09:34:543
1145699198,9cyclictest0-21swapper/211:50:152
1145699191,6cyclictest13563-21/usr/sbin/munin12:35:232
1145699191,5cyclictest0-21swapper/208:44:572
11456991914,3cyclictest5937-21sh12:05:112
11456991914,3cyclictest29224-21sh12:21:582
11456991914,3cyclictest14432-21sh11:46:372
11456991913,4cyclictest30623-21/usr/sbin/munin11:35:232
11456991913,4cyclictest10013-21sh12:31:502
11456991912,5cyclictest28310-21sh11:33:062
11456991912,5cyclictest28310-21sh11:33:052
11456991911,6cyclictest20127-21find08:45:092
11456991911,5cyclictest16200-21cat10:35:212
11456991911,5cyclictest15707-21cut08:35:002
11456991910,6cyclictest21276-21cat10:15:152
11456991910,6cyclictest14187-21sed09:45:162
1145599197,3cyclictest0-21swapper/110:19:071
1145599192,6cyclictest8686-21apt-get10:30:111
11455991914,3cyclictest32157-21sh12:24:561
11455991913,4cyclictest5905-21sh12:05:081
11455991913,4cyclictest28442-21sh11:09:161
11455991913,4cyclictest10231-21sh12:32:031
11455991911,6cyclictest560-21dbus-daemon08:09:591
11455991911,6cyclictest31245-21grep09:10:151
11455991911,6cyclictest31245-21grep09:10:151
11455991911,6cyclictest26139-21ls09:30:131
11455991911,6cyclictest24849-21tune2fs08:55:161
11455991911,6cyclictest22618-21expr07:35:141
11455991911,6cyclictest19896-21date12:40:011
11455991911,6cyclictest10865-21df_inode11:20:161
11455991910,7cyclictest32179-21tune2fs12:00:161
11455991910,6cyclictest18552-21fschecks_time07:25:171
1145599190,5cyclictest0-21swapper/110:14:311
1145499193,11cyclictest75-21kswapd011:05:120
1145499193,11cyclictest75-21kswapd010:45:110
1145499193,11cyclictest75-21kswapd010:45:110
1145499191,8cyclictest12673-21ssh12:10:120
11454991914,3cyclictest22547-21latency_hist11:05:000
11454991913,4cyclictest11432-21sh09:19:360
11454991912,5cyclictest12-21ksoftirqd/007:40:010
11454991911,6cyclictest30792-21sed09:10:000
11454991911,6cyclictest27473-21ntp_kernel_pll_09:00:210
11454991911,6cyclictest21172-21ntp_kernel_err07:30:200
11454991911,6cyclictest10097-21cut08:20:190
11454991911,6cyclictest10097-21cut08:20:180
11454991910,7cyclictest12893-21cat08:25:260
1145799189,6cyclictest27664-21tr07:45:223
1145799182,11cyclictest5124-21apt-get08:10:143
1145799182,11cyclictest22282-21apt-get08:50:143
1145799182,10cyclictest2009-21apt-get10:25:133
11457991814,2cyclictest9861-21fschecks_time08:20:173
11457991814,2cyclictest9861-21fschecks_time08:20:163
11457991811,5cyclictest31424-21seq07:55:143
11457991811,5cyclictest2245-21grep08:00:253
11457991810,6cyclictest12128-21hddtemp_smartct08:25:183
1145699187,7cyclictest28-21ksoftirqd/212:10:112
1145699182,11cyclictest1155-21apt-get10:00:152
11456991813,3cyclictest24414-21sh07:40:002
11456991813,3cyclictest20185-21sh11:26:362
11456991812,4cyclictest29377-21sh09:32:112
11456991812,4cyclictest15650-21rt-features10:10:222
11456991812,3cyclictest8767-21sh11:42:382
11456991812,3cyclictest15329-21sh09:21:312
11456991811,5cyclictest21621-21cat07:30:252
11456991811,5cyclictest11882-21expr08:25:152
1145699181,12cyclictest5127-21taskset08:10:132
11456991810,6cyclictest7465-21cat08:15:142
11456991810,5cyclictest27398-21cat07:45:182
1145599189,7cyclictest18765-21memory08:40:191
1145599189,6cyclictest29182-21gltestperf09:05:171
1145599188,7cyclictest25611-21sh07:40:221
1145599186,8cyclictest9424-21apt-get08:20:121
1145599186,8cyclictest9424-21apt-get08:20:121
1145599182,9cyclictest30659-21diskmemload11:35:121
1145599182,11cyclictest26885-21apt-get09:55:121
1145599182,11cyclictest26885-21apt-get09:55:121
1145599181,7cyclictest5114-21ssh10:26:491
11455991813,3cyclictest9597-21sh09:41:231
11455991813,3cyclictest6014-21sh10:51:461
11455991813,3cyclictest31049-21sh10:22:011
11455991813,3cyclictest26146-21sh11:30:531
11455991813,3cyclictest26146-21sh11:30:531
11455991813,3cyclictest16275-21sh11:00:101
11455991812,4cyclictest23040-21missed_timers08:50:191
11455991812,3cyclictest20429-21sh11:26:511
11455991811,5cyclictest577-21basename08:00:001
11455991811,5cyclictest29886-21uname07:50:201
11455991811,5cyclictest14929-21cat07:15:251
11455991810,6cyclictest27656-21needreboot07:45:211
1145499189,6cyclictest52750irq/139-enp3s0-10:50:110
1145499189,6cyclictest21480-21timerwakeupswit08:45:230
1145499187,8cyclictest520-21cron08:00:010
1145499183,10cyclictest25997-21apt-get12:20:140
1145499182,11cyclictest839-21apt-get08:00:130
1145499181,8cyclictest636-21runrttasks12:06:040
1145499181,5cyclictest640-21snmpd10:17:100
1145499181,3cyclictest141rcu_preempt10:35:020
11454991813,3cyclictest9618-21/usr/sbin/munin10:55:210
11454991813,3cyclictest9402-21sh09:41:110
11454991813,3cyclictest21050-21sh09:25:340
11454991811,4cyclictest32362-21mailstats09:10:250
11454991811,4cyclictest32362-21mailstats09:10:240
11454991810,6cyclictest7990-21sed08:15:190
11454991810,6cyclictest32616-21apt-get09:35:140
11454991810,6cyclictest17734-21awk08:40:000
11454991810,6cyclictest17304-21users07:20:260
14942171,13sleep21602-21memory08:00:192
1145799178,7cyclictest0-21swapper/309:20:243
1145799178,6cyclictest20227-21tr08:45:113
1145799175,8cyclictest34-21ksoftirqd/308:55:133
1145799174,3cyclictest141rcu_preempt09:00:133
11457991710,5cyclictest3090-21grep08:05:123
11457991710,5cyclictest29662-21sed07:50:173
1145699179,6cyclictest3940-21pmu-power08:05:222
1145699179,6cyclictest29270-21smartctl09:05:182
1145699179,5cyclictest24376-21date08:55:002
11456991712,3cyclictest25025-21sh07:40:162
11456991710,5cyclictest30961-21cut07:55:012
1145599179,6cyclictest15932-21grep08:35:131
1145599178,6cyclictest1958-21mailstats08:00:241
11455991712,3cyclictest12485-21ntp_states08:25:221
11455991710,5cyclictest26908-21seq09:00:151
1145499171,6cyclictest28287-21cron10:45:000
11454991713,2cyclictest14862-21sensors_temp08:30:240
11454991712,3cyclictest23450-21switchtime08:50:250
11454991711,4cyclictest28707-21latency_hist07:50:010
11454991710,5cyclictest19365-21cat07:25:250
1145799169,5cyclictest18217-21cat07:25:153
1145799169,5cyclictest14860-21sensors_temp08:30:243
11457991611,3cyclictest18926-21sh08:40:223
1145699168,6cyclictest18281-21cut07:25:142
1145499169,5cyclictest20055-21cut08:45:020
11457991510,3cyclictest29399-21latency09:05:193
1145499159,4cyclictest5425-21expr08:10:140
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional