You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-06-28 - 21:13

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Sat Jun 28, 2025 12:59:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
52309910791067,6cyclictest7742-21fschecks_time07:30:051
50182563518,30sleep10-21swapper/107:23:221
49152461417,29sleep00-21swapper/007:22:160
522999287225,42cyclictest7743-21fschecks_time07:30:050
181752590,8sleep0522999cyclictest09:20:290
286592470,10sleep00-21swapper/011:15:310
254222430,3sleep0111rcuc/011:05:350
254222430,3sleep0111rcuc/011:05:350
245182430,8sleep1523099cyclictest12:30:171
522999417,5cyclictest81rcu_preempt10:45:190
286332400,8sleep1523099cyclictest12:40:271
5229993713,7cyclictest3-21ksoftirqd/009:04:250
522999368,13cyclictest2220-21dbus-daemon07:55:250
63022350,9sleep0522999cyclictest07:25:280
56152351,30sleep15710-21hddtemp_smartct11:40:171
522999353,11cyclictest81rcu_preempt07:49:440
323872350,9sleep1523099cyclictest10:00:191
269742351,30sleep026987-21/usr/sbin/munin08:20:140
23452351,4sleep12354-21df_inode08:40:151
523099344,15cyclictest31325-21iostat_ios08:30:221
522999342,20cyclictest17806-21taskset12:10:480
522999341,21cyclictest8077-21grep10:20:240
522999339,13cyclictest6912-21taskset11:44:480
522999338,13cyclictest81rcu_preempt08:09:590
240142331,3sleep06203-21packagekitd12:30:010
523099326,4cyclictest81rcu_preempt12:45:441
522999329,12cyclictest81rcu_preempt12:40:010
522999322,5cyclictest81rcu_preempt10:15:010
5229993211,4cyclictest10976-21timerwakeupswit07:35:320
522999321,14cyclictest0-21swapper/011:50:300
523099316,4cyclictest81rcu_preempt11:11:471
523099316,13cyclictest22682-21runrttasks08:06:221
523099315,4cyclictest81rcu_preempt09:19:261
522999316,12cyclictest0-21swapper/012:17:350
522999311,4cyclictest81rcu_preempt08:14:530
523099306,4cyclictest81rcu_preempt10:09:361
523099305,4cyclictest81rcu_preempt12:21:331
523099305,4cyclictest81rcu_preempt11:03:371
523099304,5cyclictest81rcu_preempt11:19:561
523099304,4cyclictest81rcu_preempt11:45:281
523099304,4cyclictest81rcu_preempt11:23:571
523099304,17cyclictest0-21swapper/109:26:171
523099302,4cyclictest81rcu_preempt07:43:311
523099302,25cyclictest17175-21kworker/u4:012:50:371
523099301,4cyclictest81rcu_preempt10:50:291
523099301,4cyclictest81rcu_preempt09:38:561
523099301,4cyclictest81rcu_preempt09:38:561
523099301,4cyclictest81rcu_preempt09:07:321
523099301,4cyclictest81rcu_preempt08:26:081
522999308,6cyclictest15849-21grep07:50:250
5229993023,4cyclictest22296-21cron09:31:000
522999302,15cyclictest21665-21/usr/sbin/munin10:55:320
522999301,4cyclictest81rcu_preempt12:26:160
522999301,4cyclictest81rcu_preempt08:48:240
522999301,27cyclictest0-21swapper/010:30:520
5229993012,6cyclictest1763-21cut11:30:170
5229993012,5cyclictest7419-21/usr/sbin/munin11:45:200
5229993011,6cyclictest3519-21latency_hist10:10:000
5229993011,5cyclictest30683-21latency_hist08:30:010
522999301,15cyclictest13129-21seq07:41:480
279372301,5sleep10-21swapper/109:45:581
19352301,25sleep11949-21kernelversion11:30:231
89252291,5sleep10-21swapper/110:22:291
523099295,4cyclictest81rcu_preempt11:05:041
523099295,4cyclictest81rcu_preempt11:05:041
523099294,9cyclictest4327-21diskstats08:45:181
523099294,4cyclictest81rcu_preempt10:42:011
523099293,4cyclictest81rcu_preempt08:14:111
523099291,4cyclictest81rcu_preempt10:32:431
523099291,4cyclictest81rcu_preempt10:25:521
523099291,4cyclictest81rcu_preempt09:22:471
523099291,4cyclictest81rcu_preempt07:28:011
523099291,3cyclictest81rcu_preempt07:37:111
523099291,11cyclictest3020-21runrttasks08:23:551
522999294,9cyclictest31024-21diskstats08:30:170
522999293,14cyclictest0-21swapper/009:50:180
522999292,14cyclictest2877-21/usr/sbin/munin08:40:250
522999291,4cyclictest81rcu_preempt12:53:220
5229992911,6cyclictest7374-21runrttasks08:52:240
5229992911,6cyclictest669-21/usr/sbin/munin08:35:230
5229992911,6cyclictest22820-21/usr/sbin/munin11:00:210
5229992911,5cyclictest19574-21ntp_states10:50:300
5229992911,5cyclictest17057-21munin-run09:20:000
522999291,11cyclictest0-21swapper/008:03:090
5229992910,7cyclictest30303-21iostat_ios12:45:230
5229992910,7cyclictest0-21swapper/012:09:000
5229992910,6cyclictest25940-21/usr/sbin/munin08:15:300
47982291,5sleep00-21swapper/011:37:370
319762291,5sleep10-21swapper/109:58:561
11752291,5sleep00-21swapper/010:01:270
72562281,5sleep10-21swapper/108:51:121
70232281,5sleep00-21swapper/010:17:250
53592281,5sleep10-21swapper/110:14:431
523099289,7cyclictest10853-21aten_r4power_cu11:55:121
523099284,3cyclictest81rcu_preempt10:37:351
523099283,4cyclictest81rcu_preempt12:03:171
523099282,4cyclictest81rcu_preempt12:16:231
5230992815,7cyclictest14923-21diskstats12:05:191
523099281,4cyclictest81rcu_preempt08:58:301
5230992811,5cyclictest40950irq/93-i91512:25:191
522999284,8cyclictest32214-21diskstats11:25:210
522999284,13cyclictest2343-21irqbalance10:25:460
522999282,14cyclictest26853-21/usr/sbin/munin11:10:240
522999282,13cyclictest30865-21hddtemp_smartct09:55:190
522999282,13cyclictest19697-21iostat_ios09:25:230
5229992813,4cyclictest81rcu_preempt11:20:010
5229992811,5cyclictest0-21swapper/009:10:420
522999280,4cyclictest81rcu_preempt12:20:030
261812281,5sleep10-21swapper/109:42:101
209512281,5sleep10-21swapper/108:02:551
179892281,5sleep10-21swapper/112:12:421
149632281,5sleep10-21swapper/109:13:041
103692281,5sleep10-21swapper/111:51:441
10102281,3sleep10-21swapper/111:28:291
523099273,4cyclictest81rcu_preempt07:50:021
5230992718,6cyclictest22-21ksoftirqd/107:47:301
523099271,7cyclictest27624-21sh12:39:591
523099271,4cyclictest81rcu_preempt11:37:211
522999279,13cyclictest10150irq/91-xhci_hcd09:35:210
522999279,13cyclictest10150irq/91-xhci_hcd09:35:210
522999274,16cyclictest13047-21diskstats12:00:180
522999272,12cyclictest26913-21df_inode09:45:170
522999271,4cyclictest81rcu_preempt11:58:370
522999271,14cyclictest8672-21grep08:55:280
522999271,14cyclictest14390-21taskset10:35:500
266852271,5sleep10-21swapper/108:19:011
247462271,5sleep00-21swapper/009:40:140
204442271,3sleep10-21swapper/110:55:011
181712271,5sleep10-21swapper/110:45:521
164032271,5sleep00-21swapper/010:41:590
15102271,5sleep10-21swapper/108:36:241
132132271,5sleep00-21swapper/009:09:260
109972271,5sleep10-21swapper/109:00:561
523099265,4cyclictest81rcu_preempt09:33:441
523099262,4cyclictest81rcu_preempt07:55:591
5230992616,6cyclictest65202sleep110:15:311
522999261,12cyclictest28376-21/usr/sbin/munin12:40:210
523099259,12cyclictest0-21swapper/109:50:201
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional