You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-03 - 05:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #7

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 100 highest latencies:
System rack7slot7s.osadl.org (updated Wed Dec 03, 2025 00:43:47)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
22333210592,9sleep30-21swapper/319:09:143
2215829481,9sleep10-21swapper/119:06:591
2226328564,6sleep00-21swapper/019:08:190
2222528366,13sleep20-21swapper/219:07:492
2248999310,31cyclictest0-21swapper/122:46:491
2248999310,31cyclictest0-21swapper/122:46:491
22485993028,1cyclictest13-21ksoftirqd/022:55:510
2248599242,2cyclictest141rcu_preempt23:32:530
22485992321,2cyclictest13-21ksoftirqd/022:40:010
22492992221,1cyclictest0-21swapper/223:00:152
22485992216,4cyclictest13-21ksoftirqd/022:28:280
2248599215,16cyclictest13-21ksoftirqd/019:40:150
2248599210,7cyclictest0-21swapper/023:10:140
2248599210,18cyclictest0-21swapper/000:06:250
303882200,1sleep218424-21kworker/2:221:20:032
2249299201,1cyclictest30627-21ssh23:11:532
2249299200,1cyclictest0-21swapper/200:02:582
2248599206,5cyclictest13-21ksoftirqd/023:49:250
2248599205,2cyclictest141rcu_preempt23:37:510
2248599205,1cyclictest141rcu_preempt22:33:410
22492991917,2cyclictest94-21kswapd021:45:122
2248599196,1cyclictest141rcu_preempt23:29:010
22485991916,2cyclictest13-21ksoftirqd/022:50:010
22485991916,2cyclictest13-21ksoftirqd/022:50:010
2248599190,1cyclictest141rcu_preempt22:41:350
22492991815,2cyclictest20278-21grep22:20:152
2249299180,0cyclictest0-21swapper/223:43:382
2248599187,6cyclictest13606-21perf21:05:000
2248599182,2cyclictest141rcu_preempt20:13:570
2248599182,2cyclictest141rcu_preempt00:00:390
2248599180,15cyclictest0-21swapper/021:38:310
2248599180,11cyclictest0-21swapper/021:51:260
2249599172,10cyclictest30967-21apt-get22:50:143
2249299170,2cyclictest0-21swapper/200:27:002
2248599178,1cyclictest141rcu_preempt21:16:280
2248599172,6cyclictest13-21ksoftirqd/000:31:300
22485991710,3cyclictest13-21ksoftirqd/022:20:000
2249599162,10cyclictest9566-21cron19:50:003
22495991613,2cyclictest29402-21sh00:20:273
22495991613,2cyclictest29402-21sh00:20:273
2249599160,16cyclictest0-21swapper/322:28:383
2249299160,16cyclictest0-21swapper/221:31:222
2248999160,2cyclictest0-21swapper/100:35:531
2248599162,1cyclictest141rcu_preempt00:27:220
22485991616,0cyclictest0-21swapper/021:49:540
22485991615,1cyclictest0-21swapper/021:11:270
22485991614,2cyclictest13-21ksoftirqd/022:04:280
2248599160,4cyclictest0-21swapper/021:34:130
2248599160,14cyclictest0-21swapper/019:35:160
22495991513,1cyclictest13007-31frcode00:10:183
2249599150,14cyclictest0-21swapper/321:40:573
22492991515,0cyclictest0-21swapper/223:29:402
22492991515,0cyclictest0-21swapper/222:49:062
22492991515,0cyclictest0-21swapper/222:49:062
2248999154,9cyclictest26956-21apt-get20:25:161
22489991510,3cyclictest9737-21tune2fs20:55:161
2248999150,14cyclictest0-21swapper/122:00:131
2248599152,6cyclictest13-21ksoftirqd/021:55:180
2248599152,10cyclictest13-21ksoftirqd/019:20:160
22485991514,1cyclictest13-21ksoftirqd/023:22:160
22485991511,2cyclictest13202-21wc19:55:190
22485991511,1cyclictest15977-21diskmemload22:50:140
22485991510,2cyclictest13-21ksoftirqd/021:25:430
2248599150,1cyclictest0-21swapper/023:57:360
2248599150,10cyclictest24100-21sendmail20:20:000
2249599149,4cyclictest0-21swapper/323:10:163
2249599144,9cyclictest0-21swapper/323:00:173
2249299143,10cyclictest19167-21ssh21:56:252
22492991413,1cyclictest0-21swapper/200:12:492
22492991412,1cyclictest0-21swapper/222:19:182
2249299140,14cyclictest0-21swapper/223:09:212
2248999148,4cyclictest12826-21aten2_r7power_v23:00:161
22489991413,1cyclictest28392-21apache223:29:231
22489991411,2cyclictest10765-21wc23:20:251
22489991410,3cyclictest2192-21grep23:15:141
22489991410,2cyclictest1766-21grep19:30:261
2248599142,7cyclictest15400-21/usr/sbin/munin00:10:190
2248599140,2cyclictest0-21swapper/023:54:480
2248599140,1cyclictest0-21swapper/022:07:590
115522140,0sleep30-21swapper/321:29:303
2249599139,3cyclictest8992-21cat00:05:223
2249599139,2cyclictest27040-21grep23:55:143
2249599135,5cyclictest0-21swapper/323:35:153
2249599131,11cyclictest558-21lldpd19:53:113
2249599131,0cyclictest0-21swapper/322:37:593
22495991310,2cyclictest5890-21grep23:40:153
22495991310,2cyclictest27711-21head20:25:203
22495991310,2cyclictest23507-21cut21:15:133
22495991310,2cyclictest22805-21cat00:15:273
22495991310,2cyclictest20534-21tune2fs21:35:173
22495991310,2cyclictest18660-21grep20:05:263
22495991310,2cyclictest18249-21tail23:25:253
22495991310,2cyclictest14492-21grep21:30:283
22495991310,2cyclictest14173-21wc22:15:243
2249299139,3cyclictest3080-21wc23:15:182
2249299139,3cyclictest10602-21grep19:50:192
22492991310,2cyclictest9873-21head21:50:202
22492991310,2cyclictest32727-21wc22:50:262
2249299130,12cyclictest17278-21grep22:40:182
2249299130,0cyclictest0-21swapper/223:45:282
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional