You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-21 - 11:15

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack7slot7s.osadl.org (updated Fri Nov 21, 2025 00:43:52)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21414210188,9sleep10-21swapper/119:06:101
1992529480,9sleep30-21swapper/319:05:173
2152428471,9sleep00-21swapper/019:07:340
150562840,0sleep10-21swapper/123:59:151
2159328366,13sleep20-21swapper/219:08:292
302122780,0sleep10-21swapper/121:20:171
21807993532,2cyclictest32-21ksoftirqd/221:50:002
2181099270,27cyclictest0-21swapper/322:53:353
21807992521,1cyclictest32-21ksoftirqd/221:57:062
2181099210,2cyclictest0-21swapper/323:16:373
21807992119,1cyclictest0-21swapper/221:00:142
21807992116,1cyclictest32-21ksoftirqd/221:25:132
2180699210,2cyclictest0-21swapper/122:27:011
2181099201,2cyclictest141rcu_preempt21:50:223
2181099200,18cyclictest0-21swapper/300:07:123
2181099190,19cyclictest0-21swapper/321:12:333
2181099190,18cyclictest11705-21sh00:21:013
2180799192,1cyclictest141rcu_preempt22:25:332
2180799191,17cyclictest4987-21sed00:15:302
2180799190,19cyclictest0-21swapper/223:44:332
2180699190,18cyclictest0-21swapper/122:46:411
21807991817,1cyclictest0-21swapper/223:25:142
21807991816,1cyclictest23916-21unixbench_multi19:10:272
21807991815,2cyclictest32-21ksoftirqd/221:20:142
2180799180,18cyclictest0-21swapper/219:55:272
2180399186,6cyclictest11979-21munin-run23:10:000
2180399185,7cyclictest29402-21munin-run21:45:010
2180399183,11cyclictest0-21swapper/022:30:140
2180399180,15cyclictest0-21swapper/022:55:140
2180799178,8cyclictest13092-21sh00:22:472
2180799176,2cyclictest4578-21tune2fs21:50:202
21807991716,1cyclictest0-21swapper/223:36:502
21807991715,1cyclictest0-21swapper/222:24:542
2180699173,1cyclictest141rcu_preempt23:52:421
2180399175,6cyclictest224-21systemd-journal22:50:540
2181099160,3cyclictest0-21swapper/300:14:443
2181099160,1cyclictest0-21swapper/323:52:563
2181099160,1cyclictest0-21swapper/322:04:283
2180799161,14cyclictest0-21swapper/223:50:142
2180799160,16cyclictest0-21swapper/221:16:252
21806991614,1cyclictest25-21ksoftirqd/100:25:141
21803991616,0cyclictest0-21swapper/021:11:360
21803991611,4cyclictest24276-21sh23:42:340
2181099151,7cyclictest12136-21idleruntime22:45:193
21810991514,1cyclictest0-21swapper/320:45:173
2180799158,3cyclictest32-21ksoftirqd/222:46:402
2180799152,1cyclictest141rcu_preempt21:44:102
21807991513,2cyclictest32-21ksoftirqd/200:40:002
21807991513,1cyclictest32-21ksoftirqd/200:25:122
21807991513,1cyclictest20300-21ls23:15:192
21807991512,1cyclictest141rcu_preempt22:35:432
21807991511,3cyclictest29534-21dump-pmu-power22:10:002
2180799150,2cyclictest0-21swapper/219:50:132
2180799150,14cyclictest0-21swapper/221:36:292
2180699154,8cyclictest0-21swapper/120:00:151
2180699153,0cyclictest141rcu_preempt22:53:591
2180699151,1cyclictest141rcu_preempt00:01:401
21806991511,4cyclictest10545-21apache223:25:561
2180699150,11cyclictest0-21swapper/122:16:341
22622140,2chrt2260-21grep00:40:000
21810991410,3cyclictest714-21grep23:00:263
21810991410,2cyclictest9501-21grep19:50:163
2181099140,2cyclictest0-21swapper/321:35:223
2180799148,2cyclictest473-21taskset23:00:252
2180799143,1cyclictest141rcu_preempt20:30:282
21807991411,1cyclictest311rcuc/219:20:132
2180799140,13cyclictest24731-21cat20:20:222
2180699148,6cyclictest10223-21munin-node22:42:111
21806991412,2cyclictest0-21swapper/122:14:461
21806991412,1cyclictest241rcuc/100:20:251
21806991410,3cyclictest32119-21fgrep20:35:221
21806991410,3cyclictest16340-21grep20:05:141
2180699140,1cyclictest0-21swapper/123:46:461
2180399147,6cyclictest3348-21perf21:25:010
2180399147,6cyclictest27497-21cut23:45:140
21803991410,2cyclictest11051-21sh19:55:000
2180399140,11cyclictest904-21wget20:40:170
2181099139,3cyclictest29397-21cut00:35:193
21810991313,0cyclictest0-21swapper/322:21:153
21810991312,1cyclictest39-21ksoftirqd/300:30:183
2181099131,1cyclictest0-21swapper/320:35:143
21810991310,2cyclictest5799-21grep23:05:173
21810991310,2cyclictest30171-21cat22:35:003
21810991310,2cyclictest1737-21cat20:40:223
21810991310,2cyclictest1462-21latency_hist19:35:013
21810991310,2cyclictest12077-21cat21:30:293
2181099130,1cyclictest141rcu_preempt23:38:273
2181099130,1cyclictest0-21swapper/300:29:033
2180799139,3cyclictest5661-21sort22:40:212
2180799138,4cyclictest32-21ksoftirqd/200:30:292
2180799135,4cyclictest224-21systemd-journal19:50:002
2180799131,8cyclictest11199-21hwlatdetect21:30:212
21807991312,1cyclictest32-21ksoftirqd/223:06:092
21807991310,2cyclictest21783-21grep20:15:192
2180799130,12cyclictest31138-21grep00:10:292
2180699139,3cyclictest30869-21cron23:00:011
2180699139,3cyclictest25185-21wc20:20:281
2180699139,3cyclictest20812-21wc23:15:261
2180699139,3cyclictest17489-21grep21:35:181
21806991310,2cyclictest9581-21grep20:55:341
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional