You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-08 - 01:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Fri Aug 07, 2026 12:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
228502185110,28sleep20-21swapper/207:05:572
230032179103,69sleep10-21swapper/107:07:571
229232168117,28sleep30-21swapper/307:06:553
231602161101,29sleep00-21swapper/007:09:570
46572690,4sleep228-21ksoftirqd/209:29:182
291912660,1sleep10-21swapper/110:35:181
88572620,2sleep10-21swapper/110:44:491
226362590,3sleep30-21swapper/310:30:193
114472550,1sleep30-21swapper/307:55:163
287842530,1sleep10-21swapper/111:47:141
158012500,2sleep30-21swapper/312:01:313
96012460,1sleep20-21swapper/211:09:212
140412460,2sleep20-21swapper/209:11:402
140412460,2sleep20-21swapper/209:11:392
176792450,1sleep30-21swapper/311:39:373
10172450,2sleep012-21ksoftirqd/011:02:300
229502410,1sleep30-21swapper/309:18:513
2343799346,15cyclictest0-21swapper/109:46:331
2343799336,11cyclictest10643-21diskmemload11:22:121
2343799325,12cyclictest1371-21ssh09:50:231
2343799325,10cyclictest2932-21rm12:15:421
2343799322,9cyclictest0-21swapper/109:20:231
2343799316,8cyclictest636-21runrttasks11:02:571
2343799316,16cyclictest3823-21sh10:16:071
2343799308,9cyclictest10643-21diskmemload11:29:421
2343799306,7cyclictest19366-21ssh12:28:451
2343799304,11cyclictest9339-21ssh09:56:011
2343799303,15cyclictest6470-21cstates11:55:151
2343799296,7cyclictest10643-21diskmemload11:18:221
2343799295,9cyclictest7471-21awk09:30:241
2343799295,9cyclictest0-21swapper/111:07:111
2343799294,8cyclictest9479-21ls10:20:221
2343799294,8cyclictest17487-21ssh11:39:261
2343799294,8cyclictest0-21swapper/111:10:231
2343799294,10cyclictest28962-21ssh10:59:471
2343799292,11cyclictest640-21snmpd12:01:571
2343799291,26cyclictest298862chrt07:25:011
45192280,11sleep122-21ksoftirqd/107:40:101
289952280,8sleep012-21ksoftirqd/011:23:370
2343999282,18cyclictest22900-21sh10:55:113
2343999281,3cyclictest544-21open_files07:30:223
2343999281,3cyclictest544-21open_files07:30:213
2343799286,8cyclictest14954-21ssh09:12:351
2343799286,8cyclictest14954-21ssh09:12:341
2343799285,9cyclictest0-21swapper/112:34:231
2343799285,11cyclictest4981-21ssh09:29:371
2343799283,8cyclictest0-21swapper/110:25:181
2343799283,15cyclictest0-21swapper/109:18:191
2343799283,10cyclictest31778-21ssh12:37:441
23437992824,2cyclictest0-21swapper/107:34:511
23437992824,2cyclictest0-21swapper/107:34:501
2343799282,13cyclictest0-21swapper/110:10:211
2343799282,10cyclictest26638-21ssh10:09:531
23437992815,7cyclictest13767-21ssh10:00:111
2343799281,25cyclictest0-21swapper/107:24:151
23436992819,7cyclictest21899-21ssh11:42:040
2343699281,25cyclictest0-21swapper/007:34:480
2343699281,25cyclictest0-21swapper/007:34:470
65842270,2sleep00-21swapper/011:55:150
319962270,2chrt0-21swapper/309:49:413
2343799279,15cyclictest4944-21grep08:55:151
2343799275,8cyclictest8704-21ntp_states12:20:221
2343799275,8cyclictest10643-21diskmemload11:54:441
2343799273,9cyclictest24113-21ssh12:07:561
2343799271,8cyclictest21034-21ssh09:40:261
2343799271,2cyclictest0-21swapper/108:09:091
2343799271,24cyclictest0-21swapper/107:53:021
2343699271,24cyclictest0-21swapper/007:59:440
2343699271,24cyclictest0-21swapper/007:54:210
2343699271,24cyclictest0-21swapper/007:42:370
2343699271,24cyclictest0-21swapper/007:35:560
2343699271,24cyclictest0-21swapper/007:26:300
2343999261,23cyclictest0-21swapper/308:13:073
2343999261,23cyclictest0-21swapper/307:42:573
2343999261,23cyclictest0-21swapper/307:24:303
2343999260,3cyclictest30839-21ntp_states07:25:213
23438992614,7cyclictest636-21runrttasks10:30:122
2343899261,24cyclictest0-21swapper/208:14:452
2343899261,24cyclictest0-21swapper/207:34:482
2343899261,24cyclictest0-21swapper/207:34:472
2343899261,23cyclictest0-21swapper/207:58:472
2343899261,23cyclictest0-21swapper/207:25:232
2343899260,24cyclictest0-21swapper/207:36:072
2343799269,14cyclictest22142-21tail08:20:141
2343799268,8cyclictest9988-21perl10:45:151
2343799266,6cyclictest10643-21diskmemload09:36:051
23437992624,1cyclictest0-21swapper/108:32:421
23437992624,1cyclictest0-21swapper/107:35:591
23437992619,5cyclictest636-21runrttasks08:13:241
2343799260,24cyclictest0-21swapper/108:02:261
23436992623,2cyclictest0-21swapper/008:17:420
23436992619,4cyclictest991-21latency_hist10:15:010
2343699261,1cyclictest0-21swapper/008:23:320
2343699261,1cyclictest0-21swapper/007:48:110
2343699260,24cyclictest0-21swapper/008:10:350
2343699260,24cyclictest0-21swapper/008:03:000
2343699260,24cyclictest0-21swapper/007:20:220
202552260,3sleep00-21swapper/010:52:470
130892260,2sleep20-21swapper/211:35:222
2343999251,23cyclictest0-21swapper/308:49:083
2343999251,23cyclictest0-21swapper/308:27:453
2343999251,23cyclictest0-21swapper/308:17:483
2343999251,23cyclictest0-21swapper/308:08:503
2343999251,23cyclictest0-21swapper/307:47:423
2343999251,23cyclictest0-21swapper/307:35:413
2343999250,23cyclictest0-21swapper/308:02:213
2343899251,23cyclictest0-21swapper/208:54:412
2343899251,23cyclictest0-21swapper/207:24:152
2343899251,1cyclictest0-21swapper/208:17:102
2343899251,1cyclictest0-21swapper/208:02:242
2343899250,23cyclictest0-21swapper/208:09:182
2343799254,9cyclictest0-21swapper/111:43:081
23437992523,1cyclictest0-21swapper/108:54:501
23437992523,1cyclictest0-21swapper/108:15:421
23437992523,1cyclictest0-21swapper/107:46:561
2343799251,9cyclictest0-21swapper/112:10:271
2343699259,3cyclictest141rcu_preempt09:20:200
25532240,2sleep30-21swapper/310:40:023
243382240,2sleep20-21swapper/211:20:142
2343999241,1cyclictest0-21swapper/307:53:443
23438992422,1cyclictest0-21swapper/208:29:192
23438992422,1cyclictest0-21swapper/207:47:032
23438992416,5cyclictest17163-21ssh11:15:002
2343899241,1cyclictest0-21swapper/207:43:162
23437992413,9cyclictest17534-21ssh10:50:221
23437992412,9cyclictest31596-21grep08:40:221
2343899233,12cyclictest16177-21ssh10:50:112
2343899230,22cyclictest0-21swapper/207:53:212
2343799233,13cyclictest8085-21ssh11:31:421
23437992311,9cyclictest23962-21tail07:10:141
23437992310,10cyclictest25221-21sed08:25:271
23436992312,8cyclictest31993-21sh11:50:120
23439992220,1cyclictest0-21swapper/308:35:373
23439992214,6cyclictest8134-21perl12:20:173
23438992220,1cyclictest0-21swapper/208:23:322
23438992217,3cyclictest28-21ksoftirqd/211:59:592
2343799228,10cyclictest10643-21diskmemload10:30:021
23437992211,8cyclictest10141-21grep09:05:231
2343699222,16cyclictest11079-21apt-get09:10:130
2343699222,16cyclictest11079-21apt-get09:10:130
23436992211,6cyclictest14320-21ssh12:25:120
108212220,1sleep30-21swapper/309:10:013
23439992113,6cyclictest7117-21smartctl09:30:223
2343999211,16cyclictest6278-21ssh11:55:143
2343899211,5cyclictest0-21swapper/212:14:412
23438992113,6cyclictest5961-21users08:55:232
23438992113,6cyclictest28-21ksoftirqd/212:30:202
23438992112,6cyclictest15156-21sh10:25:142
23438992112,6cyclictest13989-21ntp_kernel_pll_12:00:192
23438992112,5cyclictest52950irq/141-enp3s0-09:55:122
2343899211,14cyclictest19733-21ssh12:05:042
23437992111,7cyclictest26577-21ls07:15:221
23437992110,8cyclictest6718-21cpuspeed_turbos09:00:121
23437992110,8cyclictest420-21grep08:45:161
23437992110,8cyclictest11023-21cut07:55:141
2343699218,10cyclictest27773-21cpu12:35:120
23436992119,1cyclictest0-21swapper/008:54:390
23436992115,4cyclictest12-21ksoftirqd/009:55:000
23436992113,6cyclictest19372-21seq11:15:430
23436992113,6cyclictest10697-21sed09:10:000
23436992112,6cyclictest16187-21pluginstate08:05:250
190682210,18sleep319084-21ssh10:04:033
23439992012,6cyclictest31747-21uptime08:40:243
23439992012,6cyclictest27296-21cat12:10:153
23439992011,7cyclictest4921-21perl08:55:143
23439992011,7cyclictest2523-21seq11:27:503
23439992011,6cyclictest22448-21seq11:18:483
23439992011,6cyclictest11583-21needreboot11:10:183
2343899209,7cyclictest52950irq/141-enp3s0-11:40:242
2343899207,11cyclictest23220-21ssh10:06:272
2343899202,12cyclictest26840-21ls11:45:252
23438992014,4cyclictest9353-21sendmail_mailtr10:20:222
23438992014,4cyclictest28-21ksoftirqd/210:40:002
23438992013,5cyclictest9090-21diskstats09:05:122
23438992012,6cyclictest26046-21sed09:20:232
2343899200,17cyclictest6025-21cut11:55:002
23437992011,6cyclictest28893-21missed_timers08:35:201
2343699201,5cyclictest0-21swapper/010:43:350
23436992014,4cyclictest25092-21sh10:08:180
2343699201,3cyclictest0-21swapper/010:29:070
23436992012,6cyclictest9207-21ls12:20:250
23436992012,6cyclictest6950-21df09:00:140
23436992012,6cyclictest6622-21ls11:30:250
23436992012,6cyclictest25248-21seq12:32:550
23436992012,6cyclictest2078-21sed10:15:180
23436992011,7cyclictest4442-21cat08:55:090
23436992010,7cyclictest2296-21cron10:39:590
38872190,3sleep30-21swapper/312:16:413
276512190,1sleep20-21swapper/210:10:152
2343999198,7cyclictest20778-21missed_timers12:05:193
2343999197,7cyclictest53050irq/142-enp3s0-10:50:113
23439991913,4cyclictest9984-21sh10:20:373
23439991913,4cyclictest34-21ksoftirqd/311:05:013
23439991913,4cyclictest3364-21sh10:15:393
23439991913,4cyclictest20046-21sh10:28:583
23439991913,4cyclictest13653-21sh10:47:543
23439991913,4cyclictest10643-21diskmemload09:11:233
23439991913,4cyclictest10643-21diskmemload09:11:223
23439991912,5cyclictest23779-21expr07:10:123
23439991912,4cyclictest23689-21sh12:31:223
23439991911,6cyclictest6093-21netstat11:30:203
23439991911,6cyclictest26181-21smartctl07:15:173
23439991911,6cyclictest1721-21date12:40:003
23439991911,6cyclictest14678-21df_inode12:25:153
2343999191,13cyclictest7050-21runrttasks09:55:113
2343999191,11cyclictest31490-21ssh09:25:103
23439991910,7cyclictest27942-21cut08:35:003
2343899196,3cyclictest141rcu_preempt12:20:152
2343899193,12cyclictest5117-21apt-get11:30:132
2343899192,12cyclictest29647-21apt-get11:00:132
2343899191,8cyclictest0-21swapper/209:41:382
2343899191,5cyclictest0-21swapper/212:35:152
23438991913,4cyclictest4682-21sh10:40:382
23438991911,6cyclictest32292-21wc11:25:242
23438991911,6cyclictest23908-21netstat10:55:182
23438991911,6cyclictest2195-21rt-features12:15:252
23438991911,6cyclictest14488-21interrupts10:00:182
23438991910,6cyclictest524-21idleruntime08:45:162
2343899190,17cyclictest15131-21seq09:36:182
2343699198,7cyclictest8051-21apt-get10:20:120
23436991914,3cyclictest32255-21timerandwakeup11:25:250
23436991914,3cyclictest30224-21sh11:48:410
23436991914,3cyclictest16665-21sh10:01:380
23436991914,3cyclictest13882-21sh09:35:230
23436991914,3cyclictest10453-21sh09:33:210
23436991913,4cyclictest30154-21sh12:12:140
23436991913,4cyclictest12828-21sh11:10:530
23436991912,5cyclictest21608-21uptime12:05:260
23436991911,6cyclictest764-21uptime09:25:240
23436991911,6cyclictest23800-21df07:10:120
23436991911,6cyclictest20545-21tr09:40:220
23436991911,6cyclictest11165-21awk10:45:240
23436991910,7cyclictest25840-21date08:30:000
23436991910,6cyclictest22895-21acpi10:55:100
2343999187,9cyclictest0-21swapper/311:20:063
23439991813,3cyclictest6870-21sh11:06:383
23439991813,3cyclictest31940-21/usr/sbin/munin11:50:253
23439991813,3cyclictest28354-21sh10:35:083
23439991812,4cyclictest20264-21sh11:40:253
23439991812,3cyclictest10643-21diskmemload11:46:083
23439991811,5cyclictest2602-21seq09:50:593
23439991811,5cyclictest19750-21cut09:40:153
23439991810,6cyclictest6618-21apt09:00:103
23439991810,6cyclictest27690-21perl10:10:153
23439991810,6cyclictest22004-21diskstats08:20:143
23439991810,5cyclictest3421-21run-parts08:50:213
23439991810,5cyclictest25865-21mailstats09:20:223
2343899187,9cyclictest560-21dbus-daemon07:19:592
2343899185,4cyclictest15861-21wc12:25:252
2343899182,3cyclictest141rcu_preempt10:45:122
2343899182,11cyclictest5770-21apt-get09:30:102
23438991811,5cyclictest26708-21load08:30:202
23438991811,5cyclictest17936-21expr09:15:112
23438991811,5cyclictest1243-21ls09:50:212
23438991810,6cyclictest3099-21unixbench_singl10:15:282
23438991810,6cyclictest30922-21cut08:40:162
23438991810,6cyclictest28994-21ntp_kernel_pll_08:35:202
2343699188,8cyclictest29992-21awk08:40:000
2343699182,11cyclictest3955-21apt-get11:05:130
23436991813,3cyclictest26628-21ssh09:45:160
23436991813,3cyclictest17784-21cpu09:15:100
23436991813,3cyclictest10643-21diskmemload12:01:540
23436991812,4cyclictest10643-21diskmemload09:55:130
23436991811,5cyclictest22105-21awk10:30:150
23436991811,5cyclictest1015-21sort12:15:130
23439991711,4cyclictest14760-21sh09:35:553
23439991710,5cyclictest20994-21perl10:05:163
2343899179,6cyclictest7773-21sensors09:00:232
2343899178,7cyclictest23568-21grep07:10:132
23438991712,3cyclictest10472-21sh11:10:022
23438991710,5cyclictest27477-21wc09:45:252
2343699179,5cyclictest26534-21hwlatdetect08:30:180
2343699173,9cyclictest11813-21ssh11:35:120
2343699171,11cyclictest32393-21apt-get08:45:110
23436991710,5cyclictest27422-21seq07:19:030
2343699151,10cyclictest31167-21ntp_states08:40:190
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional