You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-10-21 - 21:05

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #b, slot #5

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  Phytec

0, Linux 5.4.47-rt28-bsp-yocto-fsl-i.mx8mp-alpha1 (Profile)

Latency plot of shadow in rack #b, slot #5
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q done
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the alll highest latencies:
System rackbslot5s.osadl.org (updated Thu Oct 14, 2021 00:45:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1788476995642,11cyclictest20-21ksoftirqd/119:25:171
1788485994619,6cyclictest0-21swapper/323:15:173
1788485994614,7cyclictest1813306-21munin-run20:20:003
1788471994633,9cyclictest9-21ksoftirqd/019:10:160
178848099452,4cyclictest1808378-21apt-get20:05:142
178848099443,4cyclictest1799702-21apt-get19:40:152
1788471994436,6cyclictest9-21ksoftirqd/021:55:160
178848099432,4cyclictest1794461-21apt-get19:25:182
1788476994335,6cyclictest20-21ksoftirqd/123:45:161
1788480994232,7cyclictest27-21ksoftirqd/222:20:142
178848099422,3cyclictest1822153-21apt-get20:45:142
1788480994215,23cyclictest1942949-21/usr/sbin/munin23:20:252
1788476994211,9cyclictest1821533-21users20:40:271
178848099412,3cyclictest1827339-21apt-get21:00:252
1788480994116,22cyclictest1864482-21cron21:49:002
178848099411,5cyclictest2002965-21apt-get00:35:162
178848099411,5cyclictest1813552-21apt-get20:20:162
1788480994115,23cyclictest1788639-21/usr/sbin/munin19:10:222
178848099411,4cyclictest1934320-21apt-get23:10:192
178848099409,28cyclictest1837819-21ssh21:17:342
178848099408,29cyclictest1873561-21ssh21:59:512
178848099402,3cyclictest1895506-21apt-get22:25:142
1788480994011,27cyclictest1927325-21ssh23:01:072
1788480994010,28cyclictest1834406-21ssh21:14:062
1788480994010,27cyclictest1971086-21ssh23:54:152
1788476994011,16cyclictest1976534-21ntp_states00:00:261
1788485993928,8cyclictest34-21ksoftirqd/322:10:153
1788485993910,5cyclictest0-21swapper/321:25:013
178848099399,27cyclictest1910903-21ssh22:42:302
178848099392,4cyclictest1821122-21ntp_states20:40:242
1788480993920,15cyclictest1999324-21cut00:30:182
178848099391,5cyclictest1829045-21apt-get21:05:182
178848099391,5cyclictest1815261-21/usr/sbin/munin20:25:262
178847699398,10cyclictest1900606-21ntp_kernel_pll_22:30:251
178847699398,10cyclictest1900606-21ntp_kernel_pll_22:30:251
1788476993926,9cyclictest1788643-21apt-get19:10:151
1788476993913,9cyclictest1830524-21diskmemload23:33:051
1788485993829,5cyclictest34-21ksoftirqd/321:35:143
178848099388,27cyclictest0-21swapper/221:21:162
178848099386,2cyclictest0-21swapper/222:59:222
178848099383,4cyclictest1812121-21perl20:15:352
178848099381,5cyclictest1939205-21meminfo23:15:232
178848099381,5cyclictest1796329-21diskstats19:30:162
178848099381,4cyclictest1984503-21ntp_states00:10:302
178848099381,4cyclictest1905096-21/usr/sbin/munin22:35:392
178848099381,4cyclictest1824233-21/usr/sbin/munin20:50:202
178848099381,4cyclictest1824233-21/usr/sbin/munin20:50:202
178848099381,4cyclictest1792109-21users19:15:292
1788476993813,9cyclictest1870954-21users21:55:291
1788476993811,8cyclictest1895673-21df_inode22:25:161
1788471993831,4cyclictest9-21ksoftirqd/000:00:140
178848099377,2cyclictest1848170-21ssh21:30:052
178848099376,2cyclictest1978034-21ssh00:02:292
178848099374,3cyclictest0-21swapper/221:35:292
178848099372,4cyclictest1900517-21memory22:30:232
178848099372,4cyclictest1900517-21memory22:30:232
178848099372,4cyclictest1874025-21apt-get22:00:132
178848099371,5cyclictest1975314-21ssh23:59:502
178848099371,5cyclictest1820195-21munin-run20:40:002
178848099371,5cyclictest1817554-21meminfo20:30:242
178848099371,5cyclictest1805512-21memory19:55:302
178848099371,5cyclictest1805512-21memory19:55:302
178848099371,4cyclictest1964285-21diskstats23:45:182
178848099371,4cyclictest1857394-21irqstats21:40:232
178848099371,4cyclictest1810248-21df_inode20:10:222
178848099371,4cyclictest1806834-21/usr/sbin/munin20:00:192
178848099371,4cyclictest1797659-21apt-get19:35:022
178847699377,9cyclictest1878475-21diskstats22:05:191
178848099364,5cyclictest0-21swapper/221:50:162
178848099362,4cyclictest1847813-21ssh21:29:512
178848099362,3cyclictest1988303-21ntp_states00:15:212
178848099361,5cyclictest1998426-21ssh00:29:192
178848099361,5cyclictest1991947-21if_err_eth100:20:272
178848099361,5cyclictest1933032-21ssh23:08:322
178848099361,5cyclictest1916279-21ssh22:49:222
178848099361,5cyclictest1888873-21ssh22:16:462
178848099361,5cyclictest1884193-21ssh22:11:062
178848099361,4cyclictest1960134-21/usr/sbin/munin23:40:382
178848099361,4cyclictest1878473-21diskstats22:05:182
178848099361,4cyclictest1826160-21memory20:55:232
178848099361,4cyclictest1803302-21df_inode19:50:262
178848099361,3cyclictest1792882-21df19:20:152
178847699367,10cyclictest1790466-21cpuspeed19:15:021
1788476993611,8cyclictest1960132-21sh23:40:151
178848099351,4cyclictest1980249-21irqstats00:05:222
178848099351,4cyclictest1801986-21meminfo19:45:222
1788480993514,18cyclictest27-21ksoftirqd/222:50:192
178847699358,8cyclictest785-21snmpd22:41:551
178847699358,8cyclictest1814125-21meminfo20:20:291
1788476993524,8cyclictest1804936-21apt-get19:55:231
1788476993524,8cyclictest1804936-21apt-get19:55:231
1788476993522,10cyclictest1827339-21apt-get21:00:241
178847699349,9cyclictest1835625-21perl21:15:191
178847699348,9cyclictest1891619-21ssh22:20:251
178847699346,8cyclictest1857709-21ps21:40:271
178847699345,11cyclictest2002509-21ssh00:34:491
1788485993322,6cyclictest1839306-21run-parts21:20:013
178847699338,9cyclictest1980951-21users00:05:301
178847699338,8cyclictest95-1kworker/u9:021:26:071
1788476993316,6cyclictest1810793-21ntp_states20:10:301
178847699327,9cyclictest1991489-21apt-get00:20:191
178847699327,3cyclictest19227852sh22:55:471
178847699325,9cyclictest1840411-21ntp_states21:20:231
1788476993223,7cyclictest1942956-21apt-get23:20:151
1788476993221,8cyclictest1830981-21apt-get21:10:171
1788476993218,9cyclictest20-21ksoftirqd/122:35:291
1788476993212,17cyclictest1986256-21ssh00:13:071
178847199323,24cyclictest1942510-21cron23:20:010
1788471993212,6cyclictest2006380-21cron00:40:000
178847699316,8cyclictest1988163-21meminfo00:15:191
1788476993117,11cyclictest1959178-21ssh23:39:021
178848599302,24cyclictest1987777-21tune2fs00:15:153
1788485993021,6cyclictest34-21ksoftirqd/321:50:113
178847699306,8cyclictest1914892-21ssh22:47:021
1788476993022,6cyclictest1822513-21if_err_can020:45:191
1788476993020,8cyclictest2004005-21timerandwakeup00:35:291
1788476993020,8cyclictest1792741-21apt-get19:20:131
1788476993019,8cyclictest1848252-21apt21:30:161
1788476993019,6cyclictest1801177-21cron19:45:001
1788471993026,2cyclictest0-21swapper/023:20:150
1788485992922,4cyclictest1813641-21sort20:20:223
1788485992921,4cyclictest1933959-21latency_hist23:10:003
1788485992919,5cyclictest1830515-21cron21:10:003
1788485992918,6cyclictest2002610-21cron00:35:003
178847699295,9cyclictest1862345-21users21:45:271
178847699295,8cyclictest1968637-21meminfo23:50:201
1788476992921,6cyclictest1823878-21/usr/sbin/munin20:50:161
1788476992921,6cyclictest1823878-21/usr/sbin/munin20:50:161
1788476992920,6cyclictest1819375-21/usr/sbin/munin20:35:261
1788476992919,7cyclictest1874836-21/usr/sbin/munin22:00:231
1788476992919,7cyclictest1809121-21/usr/sbin/munin20:05:261
1788476992918,8cyclictest1812215-21if_err_can120:15:371
1788476992910,5cyclictest20-21ksoftirqd/123:00:191
1788471992922,4cyclictest34350irq/42-eth023:40:280
1788471992918,6cyclictest1891565-21fschecks_time22:20:240
178848599283,19cyclictest298-21systemd-journal20:00:273
1788485992819,5cyclictest1873702-21basename22:00:013
1788476992819,7cyclictest1801831-21/usr/sbin/munin19:45:201
1788476992819,5cyclictest0-21swapper/121:50:221
1788476992818,7cyclictest1883406-21/usr/sbin/munin22:10:261
1788476992818,7cyclictest1829826-21/usr/sbin/munin21:05:301
1788476992816,9cyclictest1995250-21apt-get00:25:171
1788471992821,4cyclictest34350irq/42-eth023:56:420
1788471992821,4cyclictest34350irq/42-eth000:10:290
1788471992818,5cyclictest1794694-21fschecks_time19:25:240
1788485992720,4cyclictest1830524-21diskmemload22:20:273
1788485992718,6cyclictest34-21ksoftirqd/320:35:163
1788485992718,5cyclictest1998813-21munin-plugin-st00:30:003
1788485992718,5cyclictest1802900-21apt19:50:003
1788485992718,5cyclictest0-21swapper/321:45:183
1788485992717,6cyclictest1821920-21idleruntime-cro20:45:013
1788485992717,5cyclictest1983985-21cut00:10:233
1788485992714,7cyclictest0-21swapper/300:00:003
178847699279,4cyclictest20-21ksoftirqd/119:35:361
178847699278,4cyclictest20-21ksoftirqd/123:10:311
1788476992718,7cyclictest1806662-21apt-get20:00:151
1788476992718,6cyclictest1817647-21/usr/sbin/munin20:30:251
1788476992717,7cyclictest1939376-21open_inodes23:15:261
1788476992717,7cyclictest1796966-21/usr/sbin/munin19:30:241
1788476992717,6cyclictest1803310-21diskstats19:50:271
1788471992722,3cyclictest1921619-21fschecks_count22:55:190
1788471992721,3cyclictest34350irq/42-eth000:30:000
1788471992721,3cyclictest1899450-21sh22:30:000
1788471992719,5cyclictest9-21ksoftirqd/023:50:130
1788471992719,4cyclictest1852852-21fschecks_time21:35:180
1788471992713,3cyclictest9-21ksoftirqd/000:20:170
178847199271,2cyclictest0-21swapper/022:38:060
1788485992618,5cyclictest1960460-21tune2fs23:40:273
1788485992618,5cyclictest1948449-21ls23:25:353
1788485992617,5cyclictest34-21ksoftirqd/323:55:003
1788485992617,5cyclictest1959749-21cron23:40:003
1788485992617,5cyclictest1908070-21cron22:40:003
178847699267,7cyclictest1918765-21ssh22:51:201
1788476992618,6cyclictest1971856-21/usr/sbin/munin23:55:191
1788476992617,6cyclictest1825873-21/usr/sbin/munin20:55:201
1788476992616,7cyclictest1930661-21ssh23:05:201
1788476992615,8cyclictest1853890-21ssh21:35:421
178847199262,5cyclictest1839321-21basename21:20:010
1788471992619,3cyclictest1934543-21fschecks_count23:10:230
178848599253,17cyclictest1848099-21latency_hist21:30:013
178848599252,18cyclictest1792481-21cron19:20:003
1788485992519,4cyclictest1921025-21cut22:55:003
1788485992518,4cyclictest1980471-21ntpq00:05:253
1788485992517,5cyclictest1831429-21cat21:10:243
1788485992516,5cyclictest356-21dbus-daemon22:00:513
178848599251,3cyclictest0-21swapper/323:47:593
1788476992516,6cyclictest1949034-21ssh23:26:191
1788471992521,2cyclictest34350irq/42-eth022:30:160
1788471992521,2cyclictest34350irq/42-eth022:30:160
1788471992519,4cyclictest9-21ksoftirqd/022:15:150
1788471992519,3cyclictest1987808-21fschecks_time00:15:150
1788471992518,4cyclictest1830785-21latency_hist21:10:010
1788471992518,3cyclictest1813338-21latency_hist20:20:000
1788471992516,5cyclictest1806437-21basename20:00:000
1788471992516,5cyclictest1806437-21basename20:00:000
1788485992418,4cyclictest1882250-21basename22:10:003
1788485992418,4cyclictest1793710-21sed19:20:263
1788485992417,5cyclictest34-21ksoftirqd/319:55:043
1788485992417,5cyclictest34-21ksoftirqd/319:55:043
1788485992417,5cyclictest1808997-21sed20:05:233
1788485992417,3cyclictest1913052-21fschecks_time22:45:203
1788485992415,5cyclictest0-21swapper/300:00:293
1788485992414,5cyclictest1828799-21cron21:05:013
178847199247,14cyclictest1865406-21apt21:50:120
178847199242,5cyclictest1844124-21df_inode21:25:150
1788471992418,3cyclictest1827091-21sendmail21:00:000
1788471992417,4cyclictest1925945-21sh23:00:180
1788471992417,4cyclictest1813645-21awk20:20:220
1788471992417,4cyclictest1811582-21pluginstate20:15:010
1788471992416,4cyclictest1955483-21idleruntime-cro23:35:000
178847199241,2cyclictest111rcu_preempt19:15:210
1788485992318,3cyclictest1921369-21/usr/sbin/munin22:55:193
1788485992317,3cyclictest1934178-21sh23:10:043
1788485992317,3cyclictest1908674-21fschecks_count22:40:183
1788485992317,3cyclictest1797937-21latency_hist19:35:013
1788485992316,4cyclictest34-21ksoftirqd/300:37:023
1788485992313,6cyclictest0-21swapper/300:20:343
178847199233,3cyclictest111rcu_preempt21:14:320
178847199232,17cyclictest298-21systemd-journal23:46:250
178847199232,17cyclictest1815487-21grep20:25:170
1788471992319,2cyclictest34350irq/42-eth023:25:240
1788471992318,3cyclictest848-21runrttasks22:50:210
1788471992317,4cyclictest9-21ksoftirqd/000:07:500
1788471992317,3cyclictest1913029-21fschecks_time22:45:200
1788471992317,3cyclictest1848525-21fschecks_time21:30:200
1788471992316,5cyclictest34350irq/42-eth022:00:250
1788471992316,4cyclictest9-21ksoftirqd/020:05:240
178847199231,4cyclictest1930231-21grep23:05:150
1788471992313,6cyclictest1794216-21cron19:25:010
178847199231,2cyclictest111rcu_preempt20:50:170
178847199231,2cyclictest111rcu_preempt20:50:170
1788485992217,3cyclictest34-21ksoftirqd/322:15:223
1788485992217,3cyclictest1810094-21/usr/sbin/munin20:10:213
1788485992216,4cyclictest1800769-21cat19:40:293
1788485992216,3cyclictest18970142sleep322:25:573
1788485992215,4cyclictest1804829-21grep19:55:013
1788485992215,4cyclictest0-21swapper/321:40:193
1788485992214,5cyclictest1852195-21basename21:35:003
1788485992213,5cyclictest1900411-21tr22:30:223
1788485992213,5cyclictest1900411-21tr22:30:223
1788485992213,5cyclictest1799454-21cat19:40:013
1788485992212,6cyclictest1794672-21fschecks_count19:25:243
178847199222,6cyclictest111rcu_preempt22:05:150
1788471992218,2cyclictest34350irq/42-eth021:40:260
1788471992217,3cyclictest9-21ksoftirqd/022:40:080
1788471992217,3cyclictest9-21ksoftirqd/020:45:010
1788471992217,3cyclictest1802167-21processes19:45:260
1788471992216,4cyclictest1828679-21seq21:03:310
1788471992216,3cyclictest1799919-21fschecks_count19:40:180
1788471992215,4cyclictest9-21ksoftirqd/020:35:160
178847199221,2cyclictest111rcu_preempt20:45:130
1788485992116,3cyclictest1815261-21/usr/sbin/munin20:25:143
1788485992115,4cyclictest34-21ksoftirqd/320:55:213
1788485992115,4cyclictest1942949-21/usr/sbin/munin23:20:253
1788485992115,3cyclictest1790205-21latency_hist19:15:013
1788485992113,4cyclictest0-21swapper/320:30:163
1788485992112,5cyclictest1824047-21cut20:50:183
1788485992112,5cyclictest1824047-21cut20:50:183
178847199216,13cyclictest2000239-21ls00:30:260
1788471992116,3cyclictest25750irq/34-mmc120:05:000
1788471992113,5cyclictest34350irq/42-eth022:10:190
178848599209,6cyclictest0-21swapper/320:45:233
1788471992015,3cyclictest9-21ksoftirqd/020:30:260
1788471992013,5cyclictest1796831-21ntpq19:30:220
1788471991913,4cyclictest9-21ksoftirqd/019:35:260
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional