You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-02 - 06:49

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #e, slot #6

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TI
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot6s.osadl.org (updated Wed Jul 02, 2025 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
183141699631,47cyclictest0-21swapper/020:10:130
1831420995942,10cyclictest0-21swapper/221:45:012
1831420995942,10cyclictest0-21swapper/221:45:012
1831416995948,4cyclictest0-21swapper/020:20:110
1831420995539,12cyclictest1837726-21apt-key19:45:002
1831420995314,21cyclictest1842088-21/usr/sbin/munin20:10:212
1831420995244,5cyclictest1848824-21apt20:50:012
183142099521,36cyclictest0-21swapper/221:20:012
1831419995042,4cyclictest1857452-21sendmail21:39:591
1831420994935,9cyclictest0-21swapper/223:15:002
1831416994910,4cyclictest0-21swapper/021:55:050
1831420994839,5cyclictest1883904-21apt00:15:012
1831420994839,5cyclictest1883904-21apt00:15:012
1831420994738,5cyclictest1865990-21munin-plugin-st22:30:012
1831420994738,5cyclictest1865990-21munin-plugin-st22:30:012
1831416994741,4cyclictest0-21swapper/021:10:140
1831419994635,7cyclictest1870184-21grep22:50:301
1831419994633,9cyclictest231ktimers/100:10:201
1831419994633,9cyclictest231ktimers/100:10:201
1831419994628,8cyclictest0-21swapper/123:10:001
183141699463,15cyclictest15-21rcuc/022:05:020
1831416994612,31cyclictest0-21swapper/019:15:290
1831419994531,7cyclictest0-21swapper/122:00:281
1831419994530,7cyclictest0-21swapper/119:50:001
183141699451,9cyclictest0-21swapper/021:20:020
1831420994438,4cyclictest0-21swapper/223:25:192
1831420994425,9cyclictest0-21swapper/200:05:002
1831419994433,7cyclictest1865091-21grep22:20:311
1831419994433,6cyclictest0-21swapper/123:50:271
1831419994433,5cyclictest0-21swapper/100:34:591
1831419994429,11cyclictest0-21swapper/121:00:271
1831419994429,11cyclictest0-21swapper/121:00:271
1831416994433,8cyclictest0-21swapper/022:55:000
1831416994428,8cyclictest0-21swapper/020:50:200
183142099431,18cyclictest0-21swapper/219:50:012
1831419994331,8cyclictest0-21swapper/120:10:311
1831419994325,10cyclictest0-21swapper/120:30:231
1831419994323,14cyclictest231ktimers/120:20:151
1831416994337,4cyclictest0-21swapper/019:10:010
1831416994337,4cyclictest0-21swapper/019:10:010
1831416994330,8cyclictest0-21swapper/023:40:330
1831416994325,13cyclictest0-21swapper/000:10:330
1831416994325,13cyclictest0-21swapper/000:10:330
1831420994235,4cyclictest1836731-21munin-plugin-st19:40:002
1831420994230,8cyclictest0-21swapper/220:15:292
1831420994227,11cyclictest311ktimers/221:45:132
183141999429,27cyclictest22-21rcuc/100:15:191
1831419994235,4cyclictest168350irq/127-8000000.ethernet-tx120:20:011
1831419994233,6cyclictest0-21swapper/121:45:291
1831419994232,6cyclictest1837539-21grep19:40:321
1831419994229,8cyclictest0-21swapper/121:40:291
1831419994229,8cyclictest0-21swapper/121:40:291
1831419994229,8cyclictest0-21swapper/121:10:001
1831419994224,14cyclictest231ktimers/123:55:191
1831419994224,14cyclictest231ktimers/123:55:191
183141999421,33cyclictest22-21rcuc/122:10:001
183141699422,35cyclictest0-21swapper/023:25:180
1831416994213,22cyclictest0-21swapper/020:20:010
1831419994128,9cyclictest0-21swapper/121:25:221
1831419994128,8cyclictest1882978-21unixbench_multi00:05:311
1831419994128,8cyclictest0-21swapper/123:30:281
1831419994123,14cyclictest231ktimers/122:25:171
1831419994123,14cyclictest231ktimers/122:25:171
183141999411,2cyclictest0-21swapper/122:15:001
1831416994133,4cyclictest0-21swapper/019:45:020
1831416994128,9cyclictest0-21swapper/019:45:320
1831416994126,12cyclictest131ktimers/000:25:170
183141699411,7cyclictest1862479-21/usr/sbin/munin22:05:290
183141699411,34cyclictest0-21swapper/023:45:260
183141699411,21cyclictest0-21swapper/023:00:270
1831420994030,5cyclictest0-21swapper/222:40:352
1831419994031,4cyclictest0-21swapper/119:35:311
1831419994030,6cyclictest1856469-21grep21:30:291
1831419994014,14cyclictest231ktimers/100:20:131
1831416994034,3cyclictest0-21swapper/023:35:220
1831416994032,5cyclictest20950irq/209-8000000.ethernet21:40:210
1831416994032,5cyclictest20950irq/209-8000000.ethernet21:40:210
1831416994025,11cyclictest0-21swapper/000:05:220
183141699401,27cyclictest0-21swapper/020:44:110
1831420993924,9cyclictest0-21swapper/222:35:322
1831419993926,9cyclictest0-21swapper/120:40:291
1831419993925,8cyclictest0-21swapper/122:35:241
1831419993919,12cyclictest0-21swapper/121:17:011
1831416993931,4cyclictest0-21swapper/019:50:150
1831416993927,8cyclictest0-21swapper/000:30:300
1831416993926,9cyclictest0-21swapper/000:15:290
1831416993923,11cyclictest0-21swapper/021:35:270
1831416993919,6cyclictest1871651-1kworker/0:1H+mmc_complete00:05:000
183141699391,3cyclictest1846975-21awk20:35:320
1831420993827,7cyclictest1887196-21grep00:30:302
183142099381,5cyclictest0-21swapper/200:20:002
183142099381,4cyclictest0-21swapper/222:30:172
183142099381,30cyclictest0-21swapper/222:25:012
1831419993831,4cyclictest0-21swapper/122:45:201
1831419993828,6cyclictest1873676-21munin-plugin-st23:14:591
1831419993828,5cyclictest1835849-21grep19:30:311
183141999382,5cyclictest0-21swapper/120:50:001
1831419993824,8cyclictest0-21swapper/121:50:211
1831419993823,10cyclictest0-21swapper/121:55:221
1831419993822,9cyclictest0-21swapper/121:00:001
183141999381,2cyclictest0-21swapper/121:25:001
1831416993831,4cyclictest20750irq/251-8000000.ethernet00:35:270
1831416993831,4cyclictest19950irq/27-20000000.i2c20:50:010
1831416993828,7cyclictest0-21swapper/023:50:260
1831416993826,7cyclictest131ktimers/022:40:360
1831416993824,9cyclictest0-21swapper/022:30:300
183141699382,32cyclictest1850840-21fschecks_count21:00:200
183141699382,32cyclictest1850840-21fschecks_count21:00:200
1831420993732,4cyclictest14-21rcu_preempt20:00:002
1831420993730,4cyclictest168450irq/135-8000000.ethernet-tx220:05:142
183141999376,23cyclictest1864245-21head22:15:351
1831419993727,5cyclictest0-21swapper/123:15:201
1831419993725,6cyclictest0-21swapper/123:20:221
1831416993728,6cyclictest1874476-21unixbench_singl23:15:280
1831416993727,5cyclictest1856325-21find21:30:280
1831416993724,9cyclictest0-21swapper/022:45:310
183141699371,9cyclictest0-21swapper/022:20:010
1831416993716,6cyclictest0-21swapper/021:54:280
183141699371,34cyclictest0-21swapper/022:20:310
1831420993624,8cyclictest0-21swapper/223:55:342
1831420993624,8cyclictest0-21swapper/223:55:342
1831420993624,8cyclictest0-21swapper/221:35:022
1831420993621,10cyclictest0-21swapper/223:15:292
1831420993616,12cyclictest1852405-21/usr/sbin/munin21:10:202
183141999366,13cyclictest1875667-21/usr/sbin/munin23:25:241
183141999364,23cyclictest1834172-21sed19:20:291
1831419993631,3cyclictest1831413-21cyclictest23:00:001
1831419993629,4cyclictest1847143-21sendmail20:40:001
1831419993626,7cyclictest1882126-21grep00:00:301
1831419993625,8cyclictest0-21swapper/119:10:291
1831419993625,6cyclictest1878776-21head23:40:341
1831419993624,5cyclictest0-21swapper/119:50:291
183141999361,9cyclictest0-21swapper/119:30:011
183141999361,5cyclictest0-21swapper/120:55:001
1831416993630,4cyclictest20950irq/209-8000000.ethernet00:20:260
1831416993622,10cyclictest0-21swapper/023:05:340
183141699361,6cyclictest0-21swapper/022:30:010
183141699361,6cyclictest0-21swapper/022:30:010
1831416993613,4cyclictest0-21swapper/021:25:180
183141699361,33cyclictest0-21swapper/019:10:470
183141699361,30cyclictest1881571-21latency_hist00:00:020
183141699361,30cyclictest1881571-21latency_hist00:00:020
1831420993527,5cyclictest1872837-21munin-plugin-st23:10:002
183142099352,6cyclictest1832564-21users19:15:042
1831420993523,7cyclictest1870256-21grep22:50:292
1831420993522,8cyclictest1835851-21unixbench_singl19:30:302
1831420993522,6cyclictest1850456-21grep20:55:302
1831419993527,5cyclictest0-21swapper/100:39:591
1831419993524,6cyclictest0-21swapper/120:00:291
1831419993524,6cyclictest0-21swapper/120:00:291
1831416993529,4cyclictest20950irq/209-8000000.ethernet22:55:260
1831416993529,4cyclictest20950irq/209-8000000.ethernet19:25:270
1831416993528,4cyclictest20950irq/209-8000000.ethernet22:35:220
1831416993528,4cyclictest20950irq/209-8000000.ethernet19:20:140
1831416993523,9cyclictest1846307-21latency_hist20:35:020
1831416993523,8cyclictest0-21swapper/019:55:320
1831416993522,9cyclictest0-21swapper/020:55:290
183141699351,5cyclictest0-21swapper/021:05:280
183141699350,2cyclictest1841392-21dropbear20:05:210
1831420993426,4cyclictest0-21swapper/221:24:592
1831420993425,5cyclictest0-21swapper/200:15:262
1831420993424,7cyclictest1886141-21phc2sys00:25:272
183142099341,25cyclictest0-21swapper/222:20:012
183141999347,14cyclictest1666-21snmpd19:10:011
183141999347,14cyclictest1666-21snmpd19:10:011
183141999346,20cyclictest1866756-21grep22:30:311
183141999345,17cyclictest1833122-21ptp4l19:15:281
1831419993425,6cyclictest1871875-21grep23:00:301
1831419993424,6cyclictest0-21swapper/122:40:281
1831419993424,5cyclictest0-21swapper/120:05:181
1831419993423,7cyclictest0-21swapper/100:25:311
183141999341,4cyclictest1840198-21munin-run19:59:591
1831416993428,4cyclictest20950irq/209-8000000.ethernet23:30:260
1831416993428,4cyclictest20950irq/209-8000000.ethernet21:20:180
1831416993428,4cyclictest20950irq/209-8000000.ethernet20:00:320
1831416993428,4cyclictest20950irq/209-8000000.ethernet20:00:310
1831416993428,4cyclictest0-21swapper/019:30:300
1831416993426,5cyclictest0-21swapper/019:35:190
1831416993422,8cyclictest0-21swapper/022:10:310
183141699341,30cyclictest1873157-21lsblk23:10:210
183141699341,17cyclictest0-21swapper/020:27:310
1831420993325,4cyclictest0-21swapper/200:05:312
1831420993312,8cyclictest1860270-21/usr/sbin/munin21:55:202
183141699331,6cyclictest1875245-21ptp4l23:20:280
1831420993224,6cyclictest1837874-21/usr/sbin/munin19:45:292
1831420993222,6cyclictest1869391-21grep22:45:302
1831420993221,8cyclictest0-21swapper/221:35:282
1831420993221,7cyclictest0-21swapper/219:20:302
1831420993219,9cyclictest0-21swapper/223:40:002
1831420993213,3cyclictest27350irq/343-s-wl18xx23:50:462
183142099321,28cyclictest1851896-21phc2sys21:05:272
1831420993212,4cyclictest27350irq/343-s-wl18xx22:07:572
183141999328,14cyclictest1877924-21head23:35:331
1831419993220,9cyclictest0-21swapper/121:10:311
1831416993214,4cyclictest0-21swapper/021:48:550
183142099312,5cyclictest1840476-21df_inode20:00:212
183142099312,5cyclictest1840476-21df_inode20:00:212
1831420993124,5cyclictest0-21swapper/223:45:202
1831420993123,5cyclictest0-21swapper/221:00:302
1831420993123,5cyclictest0-21swapper/221:00:302
1831420993122,7cyclictest1832200-21sshd19:11:482
1831420993119,8cyclictest0-21swapper/220:50:282
1831420993119,10cyclictest1666-21snmpd20:25:152
183142099311,28cyclictest0-21swapper/223:00:222
183142099311,24cyclictest0-21swapper/223:45:002
183141999314,19cyclictest1845323-21grep20:25:311
1831419993116,11cyclictest0-21swapper/123:45:241
1831420993019,8cyclictest0-21swapper/222:10:292
1831420993018,8cyclictest0-21swapper/219:30:002
1831420992919,7cyclictest0-21swapper/221:50:202
1831420992915,10cyclictest0-21swapper/220:40:292
1831420992911,4cyclictest27350irq/343-s-wl18xx23:20:572
1831420992819,4cyclictest0-21swapper/223:30:312
1831420992818,6cyclictest1844417-21grep20:20:282
1831420992818,6cyclictest0-21swapper/222:00:292
183142099281,2cyclictest1887497-21apt-get00:35:152
183142099275,12cyclictest1846487-21/usr/sbin/munin20:35:222
183142099271,2cyclictest1855037-21apt-get21:25:172
183142099271,21cyclictest0-21swapper/222:55:052
1831420992615,8cyclictest0-21swapper/220:35:002
1831420992517,4cyclictest0-21swapper/219:10:022
1831420992517,4cyclictest0-21swapper/219:10:012
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional