You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-09 - 05:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack3slot3s.osadl.org (updated Fri May 09, 2025 00:43:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
362629212463,57sleep30-21swapper/319:06:573
36266628556,9sleep20-21swapper/219:07:242
36267028369,9sleep10-21swapper/119:07:281
36260128162,13sleep00-21swapper/019:06:340
5532192350,4sleep136-21ksoftirqd/123:42:291
362999993222,7cyclictest383661-21cron20:00:002
362999992924,3cyclictest505222-21ntpq22:50:192
362999992825,2cyclictest431042-21grep21:30:152
362999992824,2cyclictest595213-21pmu-power00:30:202
362999992823,3cyclictest559899-21fschecks_time23:50:132
362999992622,2cyclictest573921-21cut00:05:152
362999992622,2cyclictest499528-21latency_hist22:45:002
362999992622,2cyclictest364900-21idleruntime-cro19:15:002
362999992621,3cyclictest426378-21tail21:25:202
362999992522,2cyclictest362994-21cyclictest22:10:162
362999992521,2cyclictest444566-21cpuspeed_turbos21:45:132
362999992521,2cyclictest421078-21latency_hist21:20:012
362999992514,7cyclictest491656-21sed22:35:252
362999992511,9cyclictest532595-21memory23:20:202
5538492241,2sleep20-21swapper/223:43:362
363005992421,1cyclictest0-21swapper/300:00:353
363005992420,2cyclictest0-21swapper/321:26:593
362999992421,2cyclictest573387-21latency_hist00:05:012
362999992420,3cyclictest541586-21gltestperf23:30:162
362999992418,3cyclictest0-21swapper/223:35:202
362999992417,4cyclictest412908-21/usr/sbin/munin21:10:212
36299999241,20cyclictest404638-21ntpq20:50:162
36299999241,20cyclictest404638-21ntpq20:50:152
36299999241,17cyclictest425161-21ssh21:24:172
5908902231,9sleep0590938-21ntpq00:25:190
5661022231,3sleep00-21swapper/023:55:400
4434802231,9sleep0443494-21ssh21:44:040
363005992310,1cyclictest0-21swapper/321:58:223
362999992319,2cyclictest719-21rs:main2
362999992319,2cyclictest0-21swapper/223:05:002
362999992318,3cyclictest458873-21missed_timers22:00:182
36299999231,18cyclictest531145-21sshd23:19:182
362999992311,8cyclictest509485-21awk22:55:162
362999992311,8cyclictest457916-21/usr/sbin/munin22:00:002
3819822220,2sleep00-21swapper/019:55:110
363005992218,2cyclictest0-21swapper/323:04:143
363005992218,2cyclictest0-21swapper/323:00:003
363005992217,2cyclictest0-21swapper/321:54:403
363005992216,3cyclictest0-21swapper/323:20:183
36300599221,17cyclictest0-21swapper/321:33:093
36300599220,18cyclictest416552-21taskset21:15:033
362999992218,2cyclictest522558-21idleruntime-cro23:10:002
362999992218,2cyclictest440093-21fschecks_count21:40:142
362999992218,2cyclictest411616-21ssh21:10:002
362999992218,2cyclictest0-21swapper/219:55:002
362999992218,2cyclictest0-21swapper/219:20:012
36299999221,7cyclictest372299-21ntpq19:30:182
362999992217,3cyclictest537656-21pmu-power23:25:252
362999992213,6cyclictest582085-21ls00:15:182
36299999220,20cyclictest406342-21uname20:55:182
363005992117,2cyclictest0-21swapper/321:49:583
363005992117,2cyclictest0-21swapper/300:25:163
363005992116,2cyclictest0-21swapper/323:59:493
363005992115,3cyclictest0-21swapper/322:08:473
362999992118,2cyclictest362994-21cyclictest00:40:002
36299999211,7cyclictest557964-21ssh23:47:522
36299999211,7cyclictest501518-21ssh22:45:512
362999992117,2cyclictest0-21swapper/223:55:132
362999992116,3cyclictest390543-21ntp_kernel_err20:15:172
362999992111,8cyclictest0-21swapper/200:25:192
36299999211,17cyclictest466356-21sh22:08:512
36299999211,17cyclictest394523-21ld20:25:192
36299999211,17cyclictest378175-21grep19:45:132
36300599206,2cyclictest0-21swapper/300:08:303
36300599201,8cyclictest539575-21taskset23:27:523
363005992017,1cyclictest0-21swapper/319:15:013
363005992017,1cyclictest0-21swapper/300:24:243
363005992016,2cyclictest0-21swapper/322:37:183
363005992016,2cyclictest0-21swapper/322:21:533
363005992016,2cyclictest0-21swapper/321:25:013
363005992016,2cyclictest0-21swapper/319:35:003
363005992015,2cyclictest0-21swapper/322:29:183
363005992015,2cyclictest0-21swapper/321:36:143
363005992015,2cyclictest0-21swapper/300:18:303
36300599201,14cyclictest555092-21chrt23:45:133
362999992016,2cyclictest376582-21ntp_offset19:40:172
362999992015,3cyclictest449222-21cpuspeed_turbos21:50:162
36299999201,16cyclictest527073-21ssh23:15:002
362999992011,6cyclictest473865-21ssh22:16:022
36299999201,13cyclictest941-21runrttasks00:22:182
362999992010,8cyclictest0-21swapper/222:21:272
362999992010,7cyclictest577613-21taskset00:10:022
362999992010,6cyclictest435611-21head21:35:142
36299999200,17cyclictest941-21runrttasks22:29:222
5635452191,9sleep0563558-21ssh23:54:280
36300599199,6cyclictest581303-21turbostat00:15:003
36300599196,9cyclictest411544-21turbostat21:10:013
36300599195,12cyclictest0-21swapper/323:08:533
36300599194,12cyclictest0-21swapper/322:41:413
363005991916,1cyclictest0-21swapper/319:55:203
363005991915,2cyclictest0-21swapper/323:40:173
363005991915,2cyclictest0-21swapper/323:35:213
363005991915,2cyclictest0-21swapper/322:20:013
363005991915,2cyclictest0-21swapper/320:20:003
363005991915,2cyclictest0-21swapper/300:32:393
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional