You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-28 - 17:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #a, slot #8

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rackaslot8s.osadl.org (updated Sun Jun 28, 2026 12:45:11)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2375421890,8sleep10-21swapper/107:05:131
25240217196,25sleep20-21swapper/207:09:042
57721670,1sleep30-21swapper/310:36:473
250522139102,26sleep30-21swapper/307:06:363
250692138101,26sleep00-21swapper/007:06:500
313722800,3sleep00-21swapper/007:20:240
184032660,2chrt0-21swapper/111:14:411
285672650,2sleep10-21swapper/109:45:201
72842620,1sleep30-21swapper/310:18:053
48052610,1sleep10-21swapper/109:27:421
87752590,2sleep228-21ksoftirqd/207:45:112
324112540,3sleep2271rcuc/211:49:132
146082540,1sleep10-21swapper/111:35:201
221512520,1sleep0131rcuc/009:15:560
304782510,1sleep20-21swapper/208:35:182
316442490,1sleep00-21swapper/009:23:480
250792490,1sleep00-21swapper/008:20:240
246212480,1sleep124622-21sh09:42:271
24492470,2sleep20-21swapper/207:30:132
128912470,1sleep112893-21cat09:10:001
158362460,2sleep10-21swapper/110:24:561
40002440,1sleep20-21swapper/211:04:072
302892440,1sleep10-21swapper/112:11:071
117162440,3sleep10-21swapper/111:33:351
224612430,3sleep30-21swapper/310:29:523
34352410,2chrt0-21swapper/311:03:313
284342410,1sleep30-21swapper/310:57:503
159402370,3sleep00-21swapper/010:48:440
159402370,3sleep00-21swapper/010:48:430
145962370,1sleep10-21swapper/109:10:241
307652350,2sleep20-21swapper/212:11:372
292002330,2sleep10-21swapper/108:30:241
206512330,2sleep10-21swapper/110:28:021
86262300,3chrt0-21swapper/009:00:000
206762300,11sleep012-21ksoftirqd/010:51:450
25588992912,5cyclictest22-21ksoftirqd/112:00:141
25588992912,4cyclictest22-21ksoftirqd/110:57:051
84462280,1sleep38447-21wc12:19:593
25588992821,5cyclictest22-21ksoftirqd/108:05:011
25587992824,2cyclictest0-21swapper/007:10:330
46942270,8sleep012-21ksoftirqd/009:51:360
2558899279,5cyclictest22-21ksoftirqd/111:49:091
2558899275,7cyclictest22-21ksoftirqd/109:36:081
2558899271,25cyclictest0-21swapper/107:10:271
25588992710,5cyclictest22-21ksoftirqd/109:51:451
46092260,2sleep00-21swapper/008:50:120
25590992623,1cyclictest0-21swapper/307:10:333
25589992614,10cyclictest5440-21ssh09:28:202
25589992613,8cyclictest53050irq/142-enp3s0-11:05:052
2558899269,4cyclictest22-21ksoftirqd/111:15:491
2558899268,4cyclictest22-21ksoftirqd/111:28:291
2558899266,6cyclictest22-21ksoftirqd/110:45:151
2558899266,6cyclictest22-21ksoftirqd/110:45:151
2558899263,11cyclictest22-21ksoftirqd/110:50:141
25588992610,5cyclictest22-21ksoftirqd/109:55:401
2558999256,8cyclictest324902cstates07:25:142
25589992522,2cyclictest0-21swapper/207:10:342
2558899259,4cyclictest22-21ksoftirqd/112:27:241
2558899258,9cyclictest22-21ksoftirqd/110:07:561
2558899258,14cyclictest22-21ksoftirqd/109:20:191
25588992518,5cyclictest22-21ksoftirqd/110:40:001
25588992512,4cyclictest22-21ksoftirqd/112:24:111
25588992512,4cyclictest22-21ksoftirqd/111:51:121
2558899251,11cyclictest19888-21apt-get09:15:111
25588992510,5cyclictest22-21ksoftirqd/112:19:071
2558799256,10cyclictest141rcu_preempt10:40:180
25589992415,7cyclictest15241-21ssh09:35:202
2558899249,4cyclictest22-21ksoftirqd/111:23:311
2558899249,4cyclictest22-21ksoftirqd/110:00:171
2558899249,4cyclictest22-21ksoftirqd/110:00:171
2558899248,9cyclictest22-21ksoftirqd/110:13:471
2558899248,4cyclictest22-21ksoftirqd/110:44:591
2558899244,11cyclictest22-21ksoftirqd/112:31:251
2558899241,11cyclictest80772expr11:55:111
25588992410,4cyclictest22-21ksoftirqd/110:34:421
2558799241,8cyclictest25165-21memory10:55:150
25587992416,4cyclictest12789-21diskmemload11:30:280
117402240,2sleep00-21swapper/009:32:590
43982230,1sleep20-21swapper/208:50:112
257432230,2sleep20-21swapper/209:19:362
25590992314,7cyclictest30810-21users12:35:263
25590992314,7cyclictest27116-1kworker/u9:210:09:493
2559099231,18cyclictest14175-21apt-get09:35:123
25589992313,7cyclictest23922-21cat10:30:172
2558899237,4cyclictest22-21ksoftirqd/111:01:361
25588992317,4cyclictest22-21ksoftirqd/108:40:001
25588992317,4cyclictest22-21ksoftirqd/107:39:591
25588992311,4cyclictest22-21ksoftirqd/108:45:241
25588992310,5cyclictest22-21ksoftirqd/112:06:241
25587992319,2cyclictest22582sleep011:50:200
2558799231,7cyclictest16944-21ls12:25:240
2558799230,9cyclictest0-21swapper/012:00:020
84072220,2sleep38405-21ls09:30:173
2559099225,4cyclictest12789-21diskmemload11:38:533
2559099221,12cyclictest25422-21cron11:20:003
2558999225,4cyclictest12789-21diskmemload09:13:522
25589992214,5cyclictest11270-21diskstats07:50:132
25589992213,7cyclictest11468-21fw_packets10:45:152
25589992213,7cyclictest11468-21fw_packets10:45:152
2558999220,5cyclictest0-21swapper/210:25:142
25588992216,4cyclictest22-21ksoftirqd/108:20:001
25588992213,6cyclictest3296-21diskstats10:15:141
25588992210,4cyclictest22-21ksoftirqd/112:35:191
25588992210,4cyclictest22-21ksoftirqd/111:42:401
2558799221,3cyclictest141rcu_preempt10:05:110
25587992213,7cyclictest15254-21awk09:35:210
25587992211,9cyclictest17795-21sh10:25:260
69582210,9sleep334-21ksoftirqd/307:40:143
2559099211,6cyclictest7811-21apt-get11:55:103
25590992113,6cyclictest5955-21awk11:05:153
25590992113,6cyclictest560-21dbus-daemon10:00:003
25590992113,6cyclictest2999-21ntp_kernel_err08:45:203
25590992113,6cyclictest26268-21awk11:20:133
25590992113,6cyclictest24408-21which10:30:223
25590992113,6cyclictest22888-21threads08:15:233
25590992113,6cyclictest1887-21users11:25:263
25590992113,6cyclictest11619-21cat09:05:173
25590992112,7cyclictest21676-21date10:05:003
25590992112,7cyclictest21676-21date10:05:003
25590992112,6cyclictest2841-21idleruntime07:30:173
25590992111,8cyclictest30799-21sh09:46:593
2558999219,4cyclictest9177-21sh09:30:242
2558999215,2cyclictest141rcu_preempt10:05:162
2558999215,14cyclictest141rcu_preempt12:37:252
2558999214,15cyclictest1585-21ssh11:50:142
2558999213,4cyclictest15595-21ntp_states12:00:212
2558999213,15cyclictest6784-21cpuspeed_turbos07:40:132
2558999212,3cyclictest141rcu_preempt11:30:182
2558999211,5cyclictest0-21swapper/211:56:502
25589992114,5cyclictest28-21ksoftirqd/212:10:002
25589992113,6cyclictest19392-21sed08:10:002
25589992112,7cyclictest6444-21idleruntime-cro11:30:002
2558899219,2cyclictest141rcu_preempt11:07:581
2558899217,2cyclictest141rcu_preempt08:05:111
2558899216,8cyclictest22-21ksoftirqd/109:31:301
25588992114,5cyclictest6135-21sh08:55:011
25588992112,7cyclictest6602-21apt-get08:55:121
2558799216,4cyclictest66050lldpd11:25:040
2558799214,3cyclictest12789-21diskmemload10:12:330
2558799212,7cyclictest5557-21ssh11:05:120
25587992116,4cyclictest12-21ksoftirqd/010:19:590
25587992115,4cyclictest52750irq/139-enp3s0-11:10:170
25587992115,4cyclictest32563-21sh10:36:190
25587992114,4cyclictest12789-21diskmemload10:00:110
25587992114,4cyclictest12789-21diskmemload10:00:100
25587992113,6cyclictest14147-21rt-features07:55:220
25587992113,5cyclictest21384-21grep11:40:200
221752210,2sleep30-21swapper/310:53:163
152752210,1sleep30-21swapper/312:00:183
2559099203,15cyclictest6850-21/usr/sbin/munin11:30:263
2559099201,7cyclictest15837-21cstates12:25:153
25590992014,4cyclictest28215-21sendmail_mailst11:45:243
25590992013,5cyclictest9103-21cat12:20:113
25590992013,5cyclictest32693-21cat08:40:153
25590992013,5cyclictest15118-21date08:00:003
25590992012,6cyclictest5113-21sed10:40:173
25590992012,6cyclictest26809-21expr09:20:123
25590992012,6cyclictest13748-21fw_packets09:10:163
25590992012,5cyclictest28947-21ntpq12:10:223
25590992012,5cyclictest18694-21timerwakeupswit08:05:243
25590992010,6cyclictest21179-21ssh11:40:183
2558999209,7cyclictest53050irq/142-enp3s0-10:15:112
2558999204,3cyclictest12789-21diskmemload09:45:042
2558999202,3cyclictest141rcu_preempt11:41:042
2558999202,3cyclictest141rcu_preempt11:17:332
2558999201,6cyclictest0-21swapper/209:20:162
25589992013,5cyclictest30903-21ntpq07:20:212
25589992013,4cyclictest18797-21proc_pri10:50:212
25589992012,6cyclictest9730-21cut09:00:212
25589992012,6cyclictest4581-21wc07:35:132
25589992012,6cyclictest24510-21sed08:20:182
25589992011,7cyclictest10094-21tune2fs10:20:142
25589992011,6cyclictest14309-21sed11:35:172
2558999200,9cyclictest3017650ssh10:10:242
2558999200,5cyclictest0-21swapper/210:59:322
2558899208,3cyclictest141rcu_preempt08:40:251
2558899204,2cyclictest141rcu_preempt07:20:131
25588992014,4cyclictest22-21ksoftirqd/108:24:591
25588992014,4cyclictest22-21ksoftirqd/107:35:011
2558799203,14cyclictest10768-21pmu-power10:20:210
2558799202,15cyclictest31617-21grep11:00:160
2558799202,15cyclictest31264-21ps08:35:270
2558799201,5cyclictest8187-21df_inode11:55:130
2558799201,5cyclictest515-21irqbalance11:36:190
2558799201,5cyclictest26221-21df11:20:130
25587992015,3cyclictest23209-21acpi10:30:110
25587992014,4cyclictest7253-21sh12:18:530
25587992014,4cyclictest30141-21sh09:46:180
25587992013,5cyclictest24448-21seq12:06:570
25587992013,5cyclictest16184-21pluginstate08:00:200
25587992012,6cyclictest8990-21cat12:20:110
25587992012,6cyclictest22761-21expr12:30:140
25587992012,6cyclictest1782-21seq07:29:560
25587992011,8cyclictest27773-21apt-get07:15:130
25587992010,7cyclictest19250-21cron08:09:590
247122200,2sleep30-21swapper/309:42:323
2559099193,13cyclictest22770-21grep12:30:143
2559099192,4cyclictest34-21ksoftirqd/311:13:073
2559099192,15cyclictest4614-21cut07:35:133
2559099192,14cyclictest31562-21wc08:35:293
2559099192,12cyclictest850-21apt-get09:25:133
2559099192,12cyclictest23812-21apt-get08:20:123
2559099191,4cyclictest3877-21rm09:50:453
2559099191,3cyclictest12789-21diskmemload11:53:193
25590991912,5cyclictest6703-21expr08:55:143
25590991912,5cyclictest28994-21ntp_kernel_pll_08:30:203
25590991912,5cyclictest28712-21open_inodes07:15:223
25590991912,5cyclictest26168-21expr08:25:143
25590991912,5cyclictest22491-21basename12:05:233
25590991912,5cyclictest10644-21seq07:49:413
25590991911,6cyclictest5164-21ntpq08:50:183
25590991911,6cyclictest28923-21ls10:10:133
25590991911,6cyclictest11475-21grep10:45:163
25590991911,6cyclictest11475-21grep10:45:153
2558999198,2cyclictest28-21ksoftirqd/210:00:202
2558999198,2cyclictest28-21ksoftirqd/210:00:202
2558999197,3cyclictest53050irq/142-enp3s0-12:15:122
2558999196,4cyclictest53050irq/142-enp3s0-12:25:152
2558999193,2cyclictest141rcu_preempt10:41:082
2558999192,4cyclictest28-21ksoftirqd/211:22:362
2558999192,4cyclictest28-21ksoftirqd/210:35:002
2558999192,2cyclictest141rcu_preempt12:20:072
2558999192,15cyclictest10103-21grep09:55:272
2558999191,3cyclictest141rcu_preempt11:10:212
25589991913,4cyclictest6360-21sh09:53:192
2558999191,2cyclictest0-21swapper/212:32:342
25589991912,5cyclictest21934-21ntp_offset09:40:202
25589991911,6cyclictest1860-21cron08:45:002
25589991910,7cyclictest560-21dbus-daemon07:20:012
25589991910,6cyclictest15994-21missed_timers08:00:182
2558899191,3cyclictest141rcu_preempt07:43:271
25588991911,6cyclictest1289-21ls07:25:251
25588991910,6cyclictest9555-21cut07:45:171
2558799198,2cyclictest12-21ksoftirqd/011:16:530
25587991913,4cyclictest8275-21sh09:55:020
25587991913,4cyclictest714-21sh09:25:050
25587991913,4cyclictest29291-21sed12:10:240
25587991913,4cyclictest12789-21diskmemload09:44:100
25587991912,5cyclictest8279-21seq07:41:440
25587991912,5cyclictest14281-21ntpq09:10:220
25587991911,6cyclictest27638-21basename11:45:180
25587991911,6cyclictest23419-21seq08:18:380
25587991911,6cyclictest10715-21cat07:50:010
25587991911,5cyclictest0-21swapper/012:35:110
25587991910,7cyclictest4180-21wc08:50:010
2558799190,16cyclictest12088-21seq07:50:220
2559099189,6cyclictest32568-21cat07:25:153
2559099184,12cyclictest30144-21cstates07:20:133
25590991811,5cyclictest52850irq/140-enp3s0-09:15:213
25590991811,5cyclictest20619-21tr08:10:233
25590991810,6cyclictest15455-21expr08:00:133
2558999185,2cyclictest141rcu_preempt09:05:212
2558999182,11cyclictest21651-21apt-get08:15:112
25589991810,6cyclictest28775-21cut08:30:192
25588991812,4cyclictest22-21ksoftirqd/108:14:591
25588991811,5cyclictest27113-21sensors08:25:231
25588991810,6cyclictest10933-21apt07:50:111
2558799188,7cyclictest9088-21df_inode09:00:150
2558799183,13cyclictest2077-21apt-key07:30:000
25587991811,5cyclictest6046-21seq07:35:460
25587991811,5cyclictest28455-21uniq08:30:160
25587991811,5cyclictest21442-21cut08:15:000
25587991811,5cyclictest1781-21seq08:44:060
2559099179,6cyclictest8993-21cat09:00:143
2559099179,6cyclictest12898-21sed07:55:013
25590991710,5cyclictest9834-21expr10:20:123
2558999172,11cyclictest19514-21apt-get08:10:132
25589991713,3cyclictest14994-21idleruntime-cro08:00:012
25589991711,4cyclictest26656-21sed08:25:172
25589991710,5cyclictest6758-21cstates08:55:152
25588991711,4cyclictest14076-21proc_pri07:55:221
25588991710,5cyclictest8827-21cat09:00:141
25588991710,5cyclictest29413-21seq07:16:591
25587991710,4cyclictest27430-21grep08:25:260
25587991611,3cyclictest10904-21cpu09:05:130
2558999158,5cyclictest3360-21awk08:45:252
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional