You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-10 - 08:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack7slot7s.osadl.org (updated Mon Nov 10, 2025 00:43:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
8912210086,9sleep10-21swapper/119:05:471
710229989,6sleep30-21swapper/319:05:043
922829368,9sleep00-21swapper/019:09:520
919328464,16sleep20-21swapper/219:09:252
9325993231,1cyclictest0-21swapper/122:05:331
932599260,26cyclictest0-21swapper/122:51:411
933199240,23cyclictest0-21swapper/300:26:123
9325992322,1cyclictest0-21swapper/100:26:121
9331992213,7cyclictest11511-21apache223:45:463
933199220,21cyclictest0-21swapper/322:30:053
9329992118,2cyclictest24886-21ntp_kernel_pll_23:45:232
932999210,20cyclictest0-21swapper/219:20:142
9331992018,2cyclictest39-21ksoftirqd/300:21:323
9329992017,2cyclictest27664-21aten2_r7power_p22:40:162
9329992017,2cyclictest27664-21aten2_r7power_p22:40:152
932999200,1cyclictest0-21swapper/222:06:452
932999200,19cyclictest0-21swapper/221:48:052
932199203,10cyclictest224-21systemd-journal22:05:000
9321992017,2cyclictest14604-21sh00:25:000
9329991917,1cyclictest1291-21runrttasks00:05:302
9329991916,2cyclictest14607-21latency_hist00:25:012
932999191,16cyclictest26604-21kworker/2:221:40:162
932599190,18cyclictest0-21swapper/122:30:041
9331991816,2cyclictest39-21ksoftirqd/300:35:183
9329991717,0cyclictest0-21swapper/219:25:272
932999171,16cyclictest32601-21users23:50:302
932999170,15cyclictest21642-21aten2_r7power_v19:35:172
932599171,15cyclictest0-21swapper/123:44:321
932599170,15cyclictest25-21ksoftirqd/122:12:051
932199175,7cyclictest30809-21perf19:54:590
9329991614,2cyclictest1967-21diskmemload23:56:122
9325991613,2cyclictest0-21swapper/121:45:131
932599160,1cyclictest0-21swapper/100:16:021
932599160,1cyclictest0-21swapper/100:16:011
932199161,14cyclictest0-21swapper/021:00:140
933199150,14cyclictest0-21swapper/322:29:513
9329991515,0cyclictest0-21swapper/222:50:152
932999150,2cyclictest0-21swapper/220:50:152
932999150,15cyclictest0-21swapper/222:23:052
932999150,15cyclictest0-21swapper/200:17:062
932999150,15cyclictest0-21swapper/200:17:062
932999150,14cyclictest0-21swapper/220:15:582
9325991510,2cyclictest224-21systemd-journal22:05:001
932599150,5cyclictest61750irq/125-enp2s0-22:37:591
932199156,7cyclictest0-21swapper/021:55:140
932199152,10cyclictest23125-21apache_volume21:25:140
933199146,3cyclictest7744-21snmpd00:34:543
933199145,5cyclictest224-21systemd-journal00:05:013
933199140,2cyclictest0-21swapper/323:10:153
933199140,10cyclictest0-21swapper/323:57:203
932999149,2cyclictest27533-21tr19:45:232
9329991413,1cyclictest37850irq/131-snd_hda21:35:362
9329991411,2cyclictest31025-21cpuspeed_turbos23:05:212
9329991411,2cyclictest28687-21ssh00:11:212
9329991410,3cyclictest5904-21chrt21:12:012
932999140,3cyclictest0-21swapper/223:00:152
932999140,14cyclictest0-21swapper/221:32:012
932199143,5cyclictest0-21swapper/022:25:000
9321991410,2cyclictest19769-21basename23:20:140
932199140,13cyclictest23638-21dump-pmu-power19:40:000
933199139,3cyclictest11499-21apache222:35:073
933199137,6cyclictest11499-21apache222:48:463
933199137,3cyclictest1967-21diskmemload22:50:153
933199136,6cyclictest19475-21grep22:10:213
933199132,4cyclictest19452-21tr00:05:163
9331991311,1cyclictest0-21swapper/323:25:173
933199130,9cyclictest0-21swapper/323:37:553
9329991310,2cyclictest27721-21tr20:55:182
9329991310,2cyclictest27721-21tr20:55:172
9329991310,2cyclictest20240-21wc20:40:002
9329991310,2cyclictest15757-21cron23:39:592
9329991310,2cyclictest13846-21wc23:15:222
9329991310,1cyclictest32067-21needreboot21:55:222
932999130,13cyclictest0-21swapper/200:37:302
932599139,3cyclictest8816-21/usr/sbin/munin00:20:211
932599137,3cyclictest1967-21diskmemload23:15:541
9325991312,1cyclictest0-21swapper/123:58:141
9325991311,1cyclictest1967-21diskmemload21:55:201
932599131,11cyclictest0-21swapper/121:28:341
9325991310,2cyclictest31116-21grep21:00:221
9325991310,2cyclictest10443-21dump-pmu-power21:40:001
932599130,12cyclictest0-21swapper/122:29:511
932199139,3cyclictest30622-21wc00:35:220
932199139,2cyclictest27704-21grep23:25:180
932199139,2cyclictest15071-21cut19:20:160
932199138,4cyclictest13-21ksoftirqd/000:04:340
932199137,1cyclictest17708-21apt-get21:45:130
932199132,3cyclictest32551-21sshd23:50:290
932199131,4cyclictest0-21swapper/022:35:160
9321991310,2cyclictest8213-21grep22:25:290
9321991310,2cyclictest23603-21/usr/sbin/munin23:45:250
9321991310,2cyclictest1291-21runrttasks21:41:480
932199130,12cyclictest20179-21grep00:05:200
932199130,12cyclictest19717-21cron22:35:000
932199130,12cyclictest18252-21munin-plugin-st22:10:010
933199129,2cyclictest6219-21tr20:10:173
933199129,2cyclictest5729-21wc22:00:153
933199129,2cyclictest3504-21uname21:10:223
933199129,2cyclictest28648-21ssh21:53:033
933199129,2cyclictest19247-21ssh23:41:253
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional