You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-11 - 16:52

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #9, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack9slot4s (updated Sun May 11, 2025 12:59:02)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
262409912551224,25cyclictest14223-21perl08:15:191
262409912021193,6cyclictest0-21swapper/107:25:101
262399911551121,18cyclictest0-21swapper/007:25:080
261282562516,31sleep00-21swapper/007:24:240
257612472427,30sleep10-21swapper/107:20:341
2623999176153,20cyclictest0-21swapper/008:15:190
2623999501,12cyclictest17441-21latency_hist08:25:000
159402480,8sleep12624099cyclictest08:20:191
177782400,8sleep02623999cyclictest08:25:140
135642401,34sleep113643-21latency_hist08:15:011
2624099368,4cyclictest81rcu_preempt10:25:401
122832360,2sleep012281-21ntpdc09:35:260
322272341,5sleep132254-21hddtemp_smartct11:55:211
2624099342,17cyclictest5943-21hddtemp_smartct10:45:191
2623999344,5cyclictest81rcu_preempt12:15:290
2623999341,21cyclictest19910-21grep08:30:230
123302340,9sleep02623999cyclictest11:00:320
2624099332,17cyclictest1889-21memory12:00:251
291012321,27sleep029194-21netstat10:20:290
264442321,27sleep126447-21watchsnmpd08:46:591
2624099325,4cyclictest81rcu_preempt12:46:391
2624099325,4cyclictest81rcu_preempt12:46:391
2624099325,4cyclictest81rcu_preempt11:27:421
2624099324,4cyclictest81rcu_preempt10:42:351
26240993214,5cyclictest40950irq/93-i91509:59:591
2624099321,27cyclictest25253-21smartctl11:35:301
2623999328,4cyclictest0-21swapper/012:10:170
2623999321,20cyclictest18016-21taskset12:44:410
191122323,3sleep019150-21ls11:20:220
2624099316,4cyclictest81rcu_preempt12:10:081
2624099315,4cyclictest81rcu_preempt11:09:581
2624099315,4cyclictest81rcu_preempt10:30:021
2624099314,4cyclictest81rcu_preempt11:50:411
26240993126,3cyclictest22-21ksoftirqd/110:25:001
26240993118,10cyclictest22-21ksoftirqd/109:37:361
2624099311,4cyclictest81rcu_preempt07:56:251
26239993116,11cyclictest22454-21grep08:35:340
26239993112,6cyclictest30667-21apt-get07:35:010
26239993110,7cyclictest7090-21grep09:20:340
165872311,5sleep00-21swapper/009:46:150
3822301,3sleep10-21swapper/109:05:201
3822301,3sleep10-21swapper/109:05:191
266002301,5sleep00-21swapper/008:48:370
2624099307,4cyclictest81rcu_preempt07:38:021
2624099305,4cyclictest81rcu_preempt08:02:091
2624099304,4cyclictest81rcu_preempt09:14:111
2624099304,4cyclictest81rcu_preempt09:14:111
2624099303,4cyclictest81rcu_preempt12:43:371
26240993016,10cyclictest22-21ksoftirqd/111:40:291
2624099301,4cyclictest81rcu_preempt12:22:351
2624099301,17cyclictest0-21swapper/109:45:291
26240993011,6cyclictest30944-21snmpd11:31:021
2624099301,16cyclictest10431-21ntp_states09:30:261
2624099301,11cyclictest220512/usr/sbin/munin08:35:271
2624099301,10cyclictest23921-21unixbench_multi10:05:331
2623999306,12cyclictest27022-21cpu08:50:180
2623999305,4cyclictest81rcu_preempt12:20:040
2623999302,14cyclictest3865-21df10:40:190
26239993020,6cyclictest29905-21cat08:55:330
2623999301,4cyclictest81rcu_preempt08:42:130
26239993012,5cyclictest3098-21python09:10:320
26239993012,5cyclictest3098-21python09:10:320
26239993011,6cyclictest3020-21runrttasks11:46:160
26239993011,6cyclictest2280-21wc07:45:140
185002301,5sleep00-21swapper/009:51:390
15622301,5sleep00-21swapper/009:07:340
15622301,5sleep00-21swapper/009:07:340
114552301,5sleep00-21swapper/008:07:320
91832291,5sleep00-21swapper/009:27:240
315812291,5sleep00-21swapper/010:25:480
295172291,5sleep10-21swapper/111:47:591
2624099295,4cyclictest81rcu_preempt09:02:291
2624099294,4cyclictest81rcu_preempt07:40:311
2624099292,4cyclictest81rcu_preempt12:36:561
2624099292,4cyclictest81rcu_preempt10:58:191
26240992920,5cyclictest3392-21cron10:40:001
26240992920,5cyclictest3392-21cron10:40:001
2624099291,4cyclictest81rcu_preempt12:33:081
2624099291,4cyclictest81rcu_preempt11:19:281
2624099291,4cyclictest81rcu_preempt10:50:061
2623999299,7cyclictest11717-21python12:25:300
2623999298,7cyclictest6204-21gdbus10:13:480
2623999295,8cyclictest12460-21iostat_ios08:10:220
26239992912,5cyclictest2440-21/usr/sbin/munin10:35:290
26239992912,5cyclictest2440-21/usr/sbin/munin10:35:290
26239992911,6cyclictest32370-21kernelversion11:55:240
26239992911,6cyclictest20710-21munin-run10:00:000
2623999291,14cyclictest20148-21wget12:50:140
26239992910,7cyclictest14197-21ntp_states09:40:270
26239992910,6cyclictest13573-21/usr/sbin/munin11:05:240
238792291,5sleep00-21swapper/011:33:030
180432291,5sleep00-21swapper/011:16:020
69982281,4sleep00-21swapper/010:48:200
54652281,5sleep10-21swapper/107:51:061
47142281,5sleep10-21swapper/112:08:111
2624099289,15cyclictest0-21swapper/108:40:341
2624099284,15cyclictest0-21swapper/112:50:141
2624099283,4cyclictest81rcu_preempt10:13:251
2624099283,4cyclictest81rcu_preempt08:56:121
2624099282,5cyclictest28964-21aten_r4power_en07:30:121
2624099282,4cyclictest81rcu_preempt09:44:061
2624099281,7cyclictest81rcu_preempt12:26:361
26240992815,10cyclictest22-21ksoftirqd/111:11:551
2624099281,4cyclictest81rcu_preempt12:15:211
26240992813,4cyclictest22-21ksoftirqd/111:00:181
26240992813,10cyclictest4043-21cat07:50:011
2624099281,23cyclictest17400-21cstates09:50:181
2624099281,14cyclictest27112-21iostat_ios10:15:241
2623999288,6cyclictest13140-21if_eth012:30:210
2623999282,14cyclictest0-21swapper/012:05:480
2623999281,4cyclictest81rcu_preempt07:30:060
2623999281,23cyclictest18602-21fschecks_count12:45:210
2623999281,23cyclictest18602-21fschecks_count12:45:210
26239992810,6cyclictest1200-21cut12:00:140
26239992810,6cyclictest10432-21ntpq09:30:260
220632281,5sleep00-21swapper/011:28:420
191792281,5sleep10-21swapper/108:29:411
54662271,5sleep10-21swapper/109:17:581
285052271,5sleep10-21swapper/108:53:581
2624099274,15cyclictest0-21swapper/109:21:501
26240992718,6cyclictest22-21ksoftirqd/110:03:321
2623999279,5cyclictest4738-21irqstats07:50:260
2623999275,10cyclictest6204-21gdbus11:44:500
2623999274,17cyclictest23055-21diskstats10:05:180
2623999272,12cyclictest4557-21irqstats09:15:230
2623999271,4cyclictest81rcu_preempt10:04:290
2623999271,4cyclictest81rcu_preempt08:01:100
26239992713,8cyclictest7823-21hddtemp_smartct10:50:210
26239992711,5cyclictest2220-21dbus-daemon12:35:210
2623999271,14cyclictest30969-21diskstats09:00:200
26239992710,6cyclictest27893-21runrttasks10:16:400
204012271,5sleep10-21swapper/108:30:351
200722271,5sleep10-21swapper/111:22:271
113542271,5sleep10-21swapper/108:06:281
91632261,5sleep10-21swapper/109:27:111
2623999267,6cyclictest24729-21/usr/sbin/munin11:35:200
2623999261,17cyclictest0-21swapper/011:52:430
2623999261,15cyclictest0-21swapper/010:33:520
26239992610,5cyclictest16296-21seq11:12:240
26239992610,4cyclictest0-21swapper/007:55:170
2623999244,16cyclictest10185-21chrt10:55:300
2623999241,11cyclictest1105-21ntp_states07:40:350
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional