You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-26 - 15:36
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot4.osadl.org (updated Fri Apr 26, 2024 00:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1795447994138,2cyclictest2055950-21fwupd22:36:470
1795447993222,9cyclictest514-21in:imuxsock20:40:000
179544799287,10cyclictest2025587-21taskset22:16:100
1795447992815,12cyclictest2110132-21sh23:15:000
1795447992813,6cyclictest60650irq/128-enp0s29f1:rx-023:54:270
1795458992714,3cyclictest2200623-21kworker/u8:0-rpciod00:20:113
179544799273,12cyclictest2225792-21cat00:30:210
1795447992716,10cyclictest1967962-21gltestperf21:40:130
1795447992715,11cyclictest0-21swapper/022:15:000
179544799266,14cyclictest833-21runrttasks21:22:590
179544799254,10cyclictest2137899-21sh23:30:490
1795447992516,8cyclictest1869149-21munin-run20:20:000
1795447992513,11cyclictest2125664-21fschecks_count23:25:130
179544799244,9cyclictest1933389-21kthreadcore21:15:220
1795447992412,11cyclictest1827270-21sh19:40:000
1795449992320,2cyclictest2182560-21needreboot00:00:201
1795449992319,2cyclictest2208999-21irqcore00:20:151
179544799236,8cyclictest1951057-21sh21:27:390
1795447992312,9cyclictest0-21swapper/020:30:000
1795447992310,11cyclictest0-21swapper/000:29:590
1795449992218,2cyclictest2125110-21idleruntime-cro23:25:001
179544799224,9cyclictest0-21swapper/020:55:180
1795447992219,2cyclictest260-21systemd-journal23:45:000
1795447992210,11cyclictest0-21swapper/023:40:010
1795447992210,11cyclictest0-21swapper/022:45:000
179544799220,5cyclictest1922318-21diskmemload23:22:090
1795458992118,2cyclictest2140103-21cron23:35:013
1795458992117,2cyclictest0-21swapper/323:10:003
179545499214,7cyclictest2209991-21tr00:20:152
1795454992119,1cyclictest1795442-21cyclictest22:20:002
1795449992117,2cyclictest1864652-21irqcore20:15:151
179544799216,7cyclictest1922318-21diskmemload22:25:280
179544799214,8cyclictest1959337-21sh21:34:120
179544799214,12cyclictest1922318-21diskmemload00:12:040
1795447992113,7cyclictest60650irq/128-enp0s29f1:rx-022:30:170
1795447992113,7cyclictest60650irq/128-enp0s29f1:rx-022:30:160
1795447992110,10cyclictest60650irq/128-enp0s29f1:rx-000:08:270
1795458992017,2cyclictest1917864-21irqcore21:05:173
1795454992016,3cyclictest1990151-21systemd-run21:55:002
1795449992015,4cyclictest2220747-21grep00:25:271
1795449992012,7cyclictest3150ktimers/121:46:211
179544799209,10cyclictest0-21swapper/020:30:190
179544799208,11cyclictest260-21systemd-journal00:38:160
179544799207,11cyclictest2035104-21idleruntime-cro22:25:010
179544799204,10cyclictest0-21swapper/021:45:000
1795447992013,6cyclictest60650irq/128-enp0s29f1:rx-023:48:260
1795447992013,6cyclictest60650irq/128-enp0s29f1:rx-021:13:460
1795447992013,6cyclictest60650irq/128-enp0s29f1:rx-000:23:230
179544799201,18cyclictest2088014-21/usr/sbin/munin23:00:150
1795447992011,8cyclictest2066223-21irqcore22:45:200
1795447992010,9cyclictest60650irq/128-enp0s29f1:rx-022:50:200
179544799200,4cyclictest1901426-21cron20:50:000
179544799200,1cyclictest191rcu_preempt22:57:120
179545499196,8cyclictest1901449-21idleruntime-cro20:50:002
179545499196,2cyclictest191rcu_preempt21:10:172
1795454991916,2cyclictest260-21systemd-journal20:28:302
1795454991915,3cyclictest2226083-21kthreadcore00:30:222
1795454991911,1cyclictest191rcu_preempt23:46:052
179544799197,10cyclictest0-21swapper/023:05:200
179544799195,12cyclictest1961556-21tr21:35:210
179544799192,9cyclictest0-21swapper/000:15:150
1795447991914,4cyclictest60650irq/128-enp0s29f1:rx-000:00:520
1795447991913,5cyclictest60650irq/128-enp0s29f1:rx-021:57:570
1795447991911,7cyclictest60650irq/128-enp0s29f1:rx-023:15:180
1795458991814,3cyclictest1811938-21cpuspeed_turbos19:25:133
1795454991814,1cyclictest191rcu_preempt19:10:012
1795454991813,4cyclictest2015457-21kthreadcore22:10:162
1795454991812,1cyclictest191rcu_preempt23:12:002
1795454991812,1cyclictest191rcu_preempt21:40:122
1795454991811,1cyclictest191rcu_preempt22:39:092
1795454991810,1cyclictest191rcu_preempt23:38:172
1795454991810,1cyclictest191rcu_preempt00:06:072
1795449991814,3cyclictest2047255-21ntp_kernel_pll_22:30:241
1795449991814,3cyclictest2047255-21ntp_kernel_pll_22:30:231
1795449991814,2cyclictest2095178-21sh23:05:011
179544799187,9cyclictest1795280-21latency_hist19:10:000
1795447991813,4cyclictest60650irq/128-enp0s29f1:rx-019:15:210
1795447991812,5cyclictest60650irq/128-enp0s29f1:rx-023:58:000
1795447991810,7cyclictest60650irq/128-enp0s29f1:rx-022:00:100
179544799180,7cyclictest1917635-21sort21:05:150
179544799180,2cyclictest0-21swapper/020:10:120
179544799180,1cyclictest1801090-21cat19:15:000
1795458991714,2cyclictest2125112-21sh23:25:003
1795458991713,2cyclictest0-21swapper/300:30:003
179545499179,1cyclictest191rcu_preempt22:50:422
179545499179,1cyclictest191rcu_preempt21:58:342
179545499178,1cyclictest191rcu_preempt22:20:562
179545499175,3cyclictest0-21swapper/222:45:252
179545499173,3cyclictest0-21swapper/221:48:542
179545499173,2cyclictest191rcu_preempt23:16:472
179545499172,3cyclictest1838482-21irqcore19:50:182
1795454991713,2cyclictest2034744-21kworker/u8:1-rpciod22:25:252
1795454991713,1cyclictest191rcu_preempt00:10:192
1795454991710,1cyclictest191rcu_preempt00:29:502
179544999173,2cyclictest191rcu_preempt19:45:001
1795449991714,2cyclictest311ktimers/121:30:491
1795449991714,2cyclictest2955-21cron22:25:011
1795449991712,3cyclictest1891915-21kthreadcore20:40:251
179544999171,13cyclictest2162639-21latency_hist23:50:011
1795447991714,2cyclictest60650irq/128-enp0s29f1:rx-021:50:130
1795447991714,2cyclictest60650irq/128-enp0s29f1:rx-020:05:250
1795447991714,2cyclictest0-21swapper/020:55:010
179544799170,5cyclictest1843333-21tr19:55:110
179544799170,4cyclictest1816814-21munin-plugin-st19:30:000
179544799170,2cyclictest1917109-21latency_hist21:05:010
179544799170,14cyclictest0-21swapper/019:40:120
1795458991614,1cyclictest491ktimers/322:50:273
1795458991613,2cyclictest1896027-21latency_hist20:45:013
1795458991612,3cyclictest1947643-21kthreadcore21:25:223
1795458991612,2cyclictest1811576-21grep19:24:593
1795458991612,2cyclictest0-21swapper/322:10:183
1795458991612,2cyclictest0-21swapper/300:05:093
1795458991611,3cyclictest2238307-21systemd-run00:39:593
1795458991611,3cyclictest2032196-21ntp_kernel_pll_22:20:223
179545899160,14cyclictest1879650-21apt-get20:30:003
179545499169,1cyclictest191rcu_preempt22:56:492
179545499165,2cyclictest41-21ksoftirqd/221:05:172
179545499165,1cyclictest191rcu_preempt00:37:362
179545499165,1cyclictest191rcu_preempt00:01:362
179545499162,3cyclictest1762331-21kworker/u8:0+xprtiod21:35:212
179545499162,3cyclictest0-21swapper/223:07:132
179545499161,6cyclictest391rcuc/221:00:202
1795454991613,2cyclictest2095213-21unin-run23:05:012
1795454991612,1cyclictest191rcu_preempt22:07:252
1795454991612,1cyclictest191rcu_preempt21:30:422
1795454991611,2cyclictest191rcu_preempt23:59:012
1795454991611,1cyclictest191rcu_preempt00:15:312
1795454991610,1cyclictest191rcu_preempt22:00:172
1795454991610,1cyclictest191rcu_preempt21:25:192
179545499160,4cyclictest41-21ksoftirqd/221:15:162
179545499160,4cyclictest1891300-21ps20:40:152
179545499160,4cyclictest0-21swapper/223:31:402
179545499160,10cyclictest2124018-21chrt23:22:572
1795449991613,2cyclictest509-21systemd-logind22:55:001
1795449991613,2cyclictest2102645-21sh23:10:001
1795449991613,2cyclictest2005761-21fschecks_time22:05:161
1795449991612,3cyclictest2072680-21cron22:50:011
1795449991612,3cyclictest1987125-21needreboot21:50:221
1795449991612,3cyclictest1885642-21cpuspeed_turbos20:35:111
179544799166,9cyclictest1869706-21gltestperf20:20:150
179544799163,11cyclictest1891940-21cut20:40:250
1795447991613,2cyclictest60650irq/128-enp0s29f1:rx-019:50:270
1795447991613,2cyclictest60650irq/128-enp0s29f1:rx-019:34:540
179544799160,5cyclictest1852959-21unixbench_multi20:00:280
179544799160,5cyclictest1835843-21grep19:45:210
179544799160,5cyclictest1835843-21grep19:45:210
179545899157,6cyclictest2112789-21tr23:15:193
1795458991513,2cyclictest491ktimers/321:54:543
1795458991512,2cyclictest2080934-21head22:55:163
1795458991512,2cyclictest1967666-21cpuspeed_turbos21:40:103
1795458991512,2cyclictest1856110-21kthreadcore20:05:203
1795458991511,3cyclictest1930578-21gltestperf21:15:173
179545899151,12cyclictest260-21systemd-journal19:45:003
1795458991511,2cyclictest0-21swapper/323:50:013
179545499159,2cyclictest191rcu_preempt19:50:002
179545499159,2cyclictest191rcu_preempt19:50:002
179545499154,1cyclictest191rcu_preempt23:25:162
179545499152,3cyclictest41-21ksoftirqd/221:20:102
1795454991513,1cyclictest2062353-21grep22:40:232
1795454991513,1cyclictest0-21swapper/220:15:012
1795454991512,2cyclictest1869475-21cpuspeed_turbos20:20:122
1795454991510,1cyclictest191rcu_preempt20:05:002
179545499150,6cyclictest2167630-21/usr/sbin/munin23:50:222
179545499150,3cyclictest1922318-21diskmemload23:42:382
179544999159,4cyclictest260-21systemd-journal20:32:401
179544999158,5cyclictest260-21systemd-journal23:15:001
1795449991512,2cyclictest2193964-21irqcore00:10:161
1795449991512,2cyclictest2186291-21gltestperf00:05:131
1795449991511,3cyclictest2083025-21kthreadcore22:55:191
1795449991511,3cyclictest1998106-21cpuspeed_turbos22:00:111
1795449991511,3cyclictest1807663-21kthreadcore19:20:191
1795449991511,2cyclictest0-21swapper/121:45:001
179544799150,5cyclictest1806667-21cpuspeed_turbos19:20:130
179545899148,4cyclictest1821773-21sshd19:31:493
1795458991411,2cyclictest2034744-21kworker/u8:1+xprtiod00:11:333
1795458991411,2cyclictest2012683-21kworker/u8:2+xprtiod22:32:433
1795458991411,2cyclictest2012683-21kworker/u8:2+xprtiod22:32:433
1795458991411,2cyclictest2012683-21kworker/u8:2+xprtiod22:28:293
1795458991411,1cyclictest0-21swapper/323:55:173
1795458991410,2cyclictest0-21swapper/323:35:383
1795458991410,2cyclictest0-21swapper/323:10:263
1795458991410,2cyclictest0-21swapper/322:15:203
1795458991410,2cyclictest0-21swapper/322:00:003
179545499149,1cyclictest191rcu_preempt20:20:002
179545499148,1cyclictest191rcu_preempt20:50:262
179545499148,1cyclictest0-21swapper/220:40:002
179545499142,3cyclictest1825236-21cut19:35:182
179545499141,8cyclictest1811899-21cpuspeed_turbos19:25:132
1795454991412,1cyclictest1828610-21kernelversion19:40:172
1795454991411,2cyclictest391rcuc/220:30:022
1795454991410,3cyclictest1821139-21cut19:30:262
1795454991410,1cyclictest191rcu_preempt20:55:192
179545499140,9cyclictest1804419-21kthreadcore19:15:192
179545499140,4cyclictest0-21swapper/219:20:232
179545499140,3cyclictest1858688-21awk20:10:002
179545499140,3cyclictest1844008-21cut19:55:162
179545499140,3cyclictest0-21swapper/222:30:122
179545499140,3cyclictest0-21swapper/222:30:112
179544999148,5cyclictest2231494-21sed00:35:161
179544999147,5cyclictest2148055-21grep23:40:101
179544999147,5cyclictest1922124-21sshd21:07:451
179544999146,6cyclictest260-21systemd-journal22:30:001
1795449991411,2cyclictest2223665-21cpuspeed_turbos00:30:141
1795449991411,2cyclictest1967228-21cron21:39:591
1795449991411,2cyclictest1854030-21gltestperf20:05:161
1795449991411,2cyclictest1837720-21idleruntime-cro19:50:001
1795449991411,2cyclictest1837720-21idleruntime-cro19:50:001
1795449991410,3cyclictest2141492-21irqcore23:35:181
1795449991410,3cyclictest1901006-21unixbench_singl20:45:301
1795449991410,3cyclictest1859012-21cpuspeed_turbos20:10:111
1795449991410,2cyclictest0-21swapper/123:50:101
1795449991410,2cyclictest0-21swapper/122:35:181
179545899139,3cyclictest2955-21cron23:05:013
179545899139,3cyclictest1976663-21kthreadcore21:45:223
179545899139,3cyclictest1959135-21kworker/u8:3+xprtiod22:04:203
179545899139,2cyclictest0-21swapper/322:45:163
179545899137,4cyclictest497-21dbus-daemon21:25:003
179545899137,4cyclictest2223972-21sed00:30:173
179545899131,5cyclictest1848207-21latency_hist20:00:003
179545899131,2cyclictest20624502gzip22:40:243
1795458991311,2cyclictest2145946-21kworker/u8:0+xprtiod23:54:293
1795458991310,2cyclictest491ktimers/322:35:143
1795458991310,2cyclictest2129685-21kworker/u8:2+xprtiod23:40:543
1795458991310,2cyclictest2034744-21kworker/u8:1+xprtiod00:15:033
1795458991310,2cyclictest1842971-21idleruntime-cro19:55:003
1795458991310,2cyclictest1795442-21cyclictest00:00:103
1795458991310,2cyclictest0-21swapper/319:35:123
179544999139,3cyclictest311ktimers/123:29:201
179544999139,3cyclictest1934406-21ntp_kernel_err21:15:251
179544999139,2cyclictest0-21swapper/121:25:161
179544999139,2cyclictest0-21swapper/121:00:161
179544999139,2cyclictest0-21swapper/120:20:161
179544999139,2cyclictest0-21swapper/119:25:201
179544999138,3cyclictest1879648-21sh20:30:001
179544999137,4cyclictest260-21systemd-journal21:58:491
179544999137,4cyclictest2138415-21sshd23:31:501
179544999137,4cyclictest1938107-21cat21:20:131
179544999136,5cyclictest2110794-21gltestperf23:15:151
1795449991310,2cyclictest1843517-21gltestperf19:55:131
1795449991310,2cyclictest1796060-21cpuspeed_turbos19:10:141
1795449991310,2cyclictest1762331-21kworker/u8:0+xprtiod21:10:431
179545899129,2cyclictest0-21swapper/319:20:003
179545899129,1cyclictest0-21swapper/320:15:193
179545899128,3cyclictest491ktimers/323:26:093
179545899128,3cyclictest1923182-21ssh21:10:153
179545899128,2cyclictest1833097-21cut19:45:173
179545899128,2cyclictest1833097-21cut19:45:163
179545899128,2cyclictest0-21swapper/320:35:203
179545899128,2cyclictest0-21swapper/319:10:283
179545899125,5cyclictest1901288-21seq20:48:223
179545899120,3cyclictest0-21swapper/321:34:053
179545499120,4cyclictest1795546-21base6419:09:592
179544999128,3cyclictest311ktimers/122:19:251
179544999128,3cyclictest0-21swapper/119:20:001
179544999128,2cyclictest1795280-21latency_hist19:10:001
179544999120,3cyclictest2200990-21grep00:15:101
179545899118,2cyclictest1913251-21tr21:00:213
179545899118,2cyclictest0-21swapper/320:15:003
179545899116,4cyclictest1961604-21kthreadcore21:35:203
179545899116,4cyclictest1885359-21sed20:35:013
179545899110,10cyclictest1874409-21latency_hist20:25:013
179544999118,2cyclictest311ktimers/123:57:441
179544999118,1cyclictest0-21swapper/119:35:001
179544999117,2cyclictest0-21swapper/122:10:231
179544999115,4cyclictest260-21systemd-journal20:50:011
179544999115,4cyclictest260-21systemd-journal19:40:001
179545899107,2cyclictest1902072-21gltestperf20:50:153
179544999107,2cyclictest1907330-21idleruntime20:55:161
179544999107,1cyclictest0-21swapper/120:00:121
17954589997,1cyclictest0-21swapper/320:00:153
17954589995,3cyclictest1907019-21cut20:55:123
17954499990,7cyclictest1837729-21latency_hist19:50:011
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional