You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-06-15 - 13:43
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackeslot4.osadl.org (updated Sat Jun 15, 2024 00:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1868906992421,2cyclictest2052371-21fwupd21:52:113
1868903992217,3cyclictest2204241-21latency_hist23:35:002
1868901992118,2cyclictest2227176-21fschecks_count23:50:151
1868906992017,2cyclictest2189084-21sh23:24:593
1868896992016,2cyclictest0-21swapper/000:20:330
1868906991916,2cyclictest2287507-21fschecks_time00:30:173
1868906991916,2cyclictest2117103-21kthreadcore22:35:203
1868903991911,6cyclictest2038263-21sshd21:43:452
1868896991917,1cyclictest1896077-21sh19:40:000
1868896991916,2cyclictest509-21systemd-logind20:40:010
1868896991916,2cyclictest2136796-21sh22:50:000
1868906991815,1cyclictest0-21swapper/321:35:013
1868903991814,3cyclictest2297675-21kthreadcore00:35:182
1868903991814,2cyclictest0-21swapper/223:50:002
1868901991814,2cyclictest514-21in:imuxsock19:50:001
1868896991816,1cyclictest2249441-21ssh00:05:000
1868896991813,3cyclictest1874706-21cut19:15:170
1868896991812,4cyclictest2186702-21grep23:20:310
1868906991714,2cyclictest2197293-21gltestperf23:30:163
1868906991714,2cyclictest2122513-21gltestperf22:40:183
1868906991714,2cyclictest2077935-21kthreadcore22:10:243
1868906991714,2cyclictest1891203-21cpuspeed_turbos19:35:083
1868906991713,3cyclictest2272606-21gltestperf00:20:163
1868906991710,5cyclictest2141478-21ntpq22:50:273
186890399178,6cyclictest260-21systemd-journal20:24:592
1868903991714,2cyclictest2144264-21cron22:54:592
1868903991713,3cyclictest2264992-21uniq00:15:132
1868903991713,2cyclictest0-21swapper/200:15:002
1868903991712,3cyclictest0-21swapper/221:55:312
1868901991714,2cyclictest2054829-21irqcore21:55:211
1868901991714,2cyclictest2051004-21ntp_kernel_pll_21:50:241
1868901991714,2cyclictest1988822-21kthreadcore21:10:191
1868901991712,3cyclictest2189813-21gltestperf23:25:171
1868901991711,4cyclictest260-21systemd-journal19:40:001
1868901991710,5cyclictest260-21systemd-journal19:45:001
1868896991715,1cyclictest2159198-21latency_hist23:05:000
1868896991714,2cyclictest61450irq/129-enp0s29f1:tx-021:30:340
1868896991714,2cyclictest61450irq/129-enp0s29f1:tx-000:13:560
1868896991714,2cyclictest60650irq/128-enp0s29f1:rx-023:12:020
1868896991714,2cyclictest60650irq/128-enp0s29f1:rx-022:30:100
1868896991714,2cyclictest60650irq/128-enp0s29f1:rx-021:54:200
1868896991714,2cyclictest60650irq/128-enp0s29f1:rx-020:55:270
1868896991714,2cyclictest60650irq/128-enp0s29f1:rx-000:39:100
1868896991714,2cyclictest60650irq/128-enp0s29f1:rx-000:06:180
1868896991714,2cyclictest2231215-21ntp_kernel_pll_23:50:260
1868896991714,2cyclictest2174746-21fschecks_count23:15:150
1868896991714,2cyclictest2144579-21/usr/sbin/munin22:55:190
1868896991714,2cyclictest0-21swapper/022:20:010
1868896991713,3cyclictest60650irq/128-enp0s29f1:rx-021:45:130
186890699163,2cyclictest1987303-21diskmemload22:21:113
186890699161,3cyclictest0-21swapper/322:32:283
1868906991613,2cyclictest497-21dbus-daemon19:51:493
1868906991613,2cyclictest2164302-21unixbench_singl23:05:293
1868906991612,2cyclictest0-21swapper/322:45:293
1868906991612,2cyclictest0-21swapper/322:20:013
186890699160,3cyclictest0-21swapper/321:47:103
1868903991613,2cyclictest2196604-21sh23:30:002
1868903991613,2cyclictest0-21swapper/221:05:012
1868903991613,2cyclictest0-21swapper/221:05:002
1868903991612,2cyclictest0-21swapper/221:20:012
1868903991611,3cyclictest0-21swapper/223:40:192
1868901991612,3cyclictest2204845-21gltestperf23:35:161
1868896991614,1cyclictest60650irq/128-enp0s29f1:rx-000:30:410
1868896991614,1cyclictest60650irq/128-enp0s29f1:rx-000:15:410
186889699161,3cyclictest0-21swapper/022:50:150
1868896991613,2cyclictest61450irq/129-enp0s29f1:tx-022:20:250
1868896991613,2cyclictest60650irq/128-enp0s29f1:rx-023:36:170
1868896991613,2cyclictest60650irq/128-enp0s29f1:rx-023:05:050
1868896991613,2cyclictest60650irq/128-enp0s29f1:rx-022:44:580
1868896991613,2cyclictest60650irq/128-enp0s29f1:rx-022:36:250
1868896991613,2cyclictest60650irq/128-enp0s29f1:rx-022:05:180
1868896991613,2cyclictest60650irq/128-enp0s29f1:rx-022:00:140
1868896991613,2cyclictest60650irq/128-enp0s29f1:rx-021:25:180
1868896991613,2cyclictest60650irq/128-enp0s29f1:rx-021:10:080
1868896991613,2cyclictest60650irq/128-enp0s29f1:rx-020:00:140
1868896991613,2cyclictest60650irq/128-enp0s29f1:rx-019:40:210
1868896991613,2cyclictest60650irq/128-enp0s29f1:rx-000:25:180
1868896991613,2cyclictest2055419-21tr21:55:210
1868896991611,4cyclictest1890221-21cron19:30:000
1868896991611,3cyclictest171ktimers/023:45:160
186890699153,1cyclictest1987303-21diskmemload00:00:253
186890699152,2cyclictest191rcu_preempt00:35:183
186890699152,1cyclictest191rcu_preempt23:13:203
1868906991513,1cyclictest0-21swapper/320:15:003
1868906991512,2cyclictest2031929-21cpuspeed_turbos21:40:173
1868906991512,2cyclictest0-21swapper/300:20:003
186890699151,1cyclictest191rcu_preempt23:29:133
186890699150,3cyclictest0-21swapper/321:11:123
186890399158,5cyclictest260-21systemd-journal19:30:002
1868903991512,2cyclictest401ktimers/223:00:142
1868903991512,2cyclictest2287776-21irqcore00:30:212
1868903991512,2cyclictest2114283-21kworker/u8:4-rpciod23:10:202
1868903991512,2cyclictest1933897-21cut20:15:222
1868903991511,2cyclictest0-21swapper/200:30:002
1868903991511,2cyclictest0-21swapper/200:05:132
1868903991510,3cyclictest2068931-21diskmemload22:05:012
1868903991510,3cyclictest2052712-21sshd21:52:452
186890199159,4cyclictest260-21systemd-journal20:47:571
1868901991513,1cyclictest1868892-21cyclictest21:35:201
1868901991513,1cyclictest1868892-21cyclictest21:00:191
1868901991513,1cyclictest1868892-21cyclictest21:00:191
1868901991512,2cyclictest61550irq/131-enp0s29f1:tx-122:31:081
1868901991512,2cyclictest2069401-21cpuspeed_turbos22:05:141
1868901991511,3cyclictest2299517-21nvmesmart_nvme000:35:221
1868901991511,3cyclictest2122343-21fschecks_count22:40:171
1868901991510,3cyclictest0-21swapper/100:25:011
186889699152,2cyclictest191rcu_preempt23:41:240
1868896991514,1cyclictest60650irq/128-enp0s29f1:rx-021:15:310
1868896991513,2cyclictest60650irq/128-enp0s29f1:rx-023:55:050
1868896991513,2cyclictest60650irq/128-enp0s29f1:rx-021:05:220
1868896991513,1cyclictest60650irq/128-enp0s29f1:rx-023:28:550
1868896991512,2cyclictest61450irq/129-enp0s29f1:tx-021:20:170
1868896991512,2cyclictest60650irq/128-enp0s29f1:rx-022:25:420
1868896991512,2cyclictest60650irq/128-enp0s29f1:rx-022:10:130
1868896991512,2cyclictest60650irq/128-enp0s29f1:rx-021:00:350
1868896991512,2cyclictest60650irq/128-enp0s29f1:rx-021:00:340
1868896991512,2cyclictest60650irq/128-enp0s29f1:rx-020:50:130
1868896991512,2cyclictest60650irq/128-enp0s29f1:rx-019:50:110
1868896991510,2cyclictest60650irq/128-enp0s29f1:rx-021:38:500
186889699150,3cyclictest0-21swapper/023:30:150
186889699150,14cyclictest1934887-21turbostat20:19:590
186890699142,1cyclictest1987303-21diskmemload22:26:163
186890699141,3cyclictest2254090-21ntpq00:05:253
1868906991412,1cyclictest0-21swapper/323:00:133
1868906991411,2cyclictest2066538-21unixbench_singl22:00:323
1868906991411,2cyclictest0-21swapper/321:00:003
1868906991411,1cyclictest0-21swapper/300:25:143
186890699140,2cyclictest191rcu_preempt23:15:543
186890699140,2cyclictest191rcu_preempt21:27:183
186890699140,2cyclictest191rcu_preempt21:16:273
186890399149,3cyclictest2006127-21tr21:20:202
186890399148,4cyclictest260-21systemd-journal00:23:202
186890399148,4cyclictest1902370-21cron19:50:002
186890399141,2cyclictest191rcu_preempt23:15:312
1868903991411,2cyclictest2227283-21gltestperf23:50:162
1868903991410,3cyclictest2104336-21unixbench_multi22:25:312
1868903991410,3cyclictest2025090-21fschecks_time21:35:152
1868903991410,2cyclictest0-21swapper/223:00:002
1868903991410,2cyclictest0-21swapper/221:00:002
1868903991410,2cyclictest0-21swapper/220:29:592
186890399140,12cyclictest1890857-21cat19:34:562
186890199145,2cyclictest191rcu_preempt19:59:591
1868901991413,1cyclictest311ktimers/122:45:521
1868901991412,1cyclictest2204126-21sh23:35:001
1868901991411,2cyclictest2257406-21cpuspeed_turbos00:10:151
1868901991411,2cyclictest2221975-21tr23:45:211
1868901991411,2cyclictest1879320-21idleruntime-cro19:20:011
1868901991411,2cyclictest0-21swapper/122:00:251
1868901991410,3cyclictest2212095-21cpuspeed_turbos23:40:161
1868901991410,3cyclictest1937580-21tr20:20:181
1868901991410,2cyclictest0-21swapper/120:15:001
1868896991412,1cyclictest60650irq/128-enp0s29f1:rx-020:40:270
1868896991412,1cyclictest1961757-21gltestperf20:45:160
1868896991411,2cyclictest60650irq/128-enp0s29f1:rx-020:33:480
1868896991411,2cyclictest60650irq/128-enp0s29f1:rx-019:45:460
1868896991411,2cyclictest1928769-21unixbench_singl20:10:280
1868896991410,3cyclictest1939682-21unixbench_multi20:20:290
1868896991410,2cyclictest0-21swapper/019:35:000
186890699139,3cyclictest2212213-21fschecks_count23:40:193
186890699139,3cyclictest1890868-21cron19:35:013
186890699139,3cyclictest0-21swapper/319:15:173
186890699139,2cyclictest2002813-21sort21:20:123
186890699137,4cyclictest2208608-21cat23:35:213
186890699132,2cyclictest191rcu_preempt22:55:573
186890699131,1cyclictest191rcu_preempt23:55:163
1868906991311,1cyclictest0-21swapper/320:00:213
1868906991310,2cyclictest1982452-21cpuspeed_turbos21:05:153
1868906991310,2cyclictest1966375-21turbostat20:54:543
1868906991310,2cyclictest0-21swapper/323:50:163
1868906991310,1cyclictest0-21swapper/319:40:163
186890699130,3cyclictest0-21swapper/321:36:093
186890699130,2cyclictest191rcu_preempt22:06:563
186890699130,1cyclictest0-21swapper/323:45:383
186890399139,3cyclictest2096310-21needreboot22:20:242
186890399139,3cyclictest2069621-21gltestperf22:05:192
186890399139,2cyclictest0-21swapper/223:55:172
186890399139,2cyclictest0-21swapper/223:35:222
186890399139,2cyclictest0-21swapper/222:40:212
186890399139,2cyclictest0-21swapper/221:25:122
186890399135,5cyclictest260-21systemd-journal20:39:592
1868903991311,2cyclictest4050ktimers/221:31:572
1868903991310,2cyclictest2040437-21sort21:45:212
1868903991310,2cyclictest1966696-21cpuspeed_turbos20:50:102
1868903991310,2cyclictest1896073-21idleruntime-cro19:40:002
186890199139,3cyclictest2010420-21gltestperf21:25:171
186890199139,2cyclictest0-21swapper/120:29:591
186890199138,4cyclictest260-21systemd-journal21:45:011
186890199138,4cyclictest1960947-21sshd20:42:451
186890199137,4cyclictest2277351-21sshd00:21:081
186890199136,5cyclictest260-21systemd-journal23:00:001
186890199136,5cyclictest1934703-21systemd20:18:261
1868901991310,2cyclictest311ktimers/121:21:181
1868901991310,2cyclictest2167162-21cpuspeed_turbos23:10:141
1868901991310,2cyclictest2117109-21kthreadcore22:35:201
1868901991310,2cyclictest0-21swapper/100:00:331
186890199130,6cyclictest2144254-21cron22:54:591
1868896991310,2cyclictest1874077-21idleruntime-cro19:15:000
1868896991310,2cyclictest0-21swapper/020:25:170
186890699129,2cyclictest1956682-21irqcore20:40:183
186890699129,2cyclictest1940928-21irqcore20:25:173
186890699129,2cyclictest1933913-21needreboot20:15:223
186890699129,2cyclictest1908836-21irqcore19:55:173
186890699129,2cyclictest0-21swapper/320:10:003
186890699128,3cyclictest1902103-21sshd19:46:473
186890699121,1cyclictest191rcu_preempt21:55:443
186890699120,2cyclictest191rcu_preempt00:10:333
186890699120,10cyclictest1955916-21latency_hist20:40:003
186890399129,2cyclictest514-21in:imuxsock19:51:492
186890399129,2cyclictest1982652-21gltestperf21:05:192
186890399129,2cyclictest1912989-21munin-plugin-st19:59:592
186890399129,2cyclictest0-21swapper/222:45:172
186890399128,2cyclictest2084270-21cpuspeed_turbos22:15:102
186890399128,2cyclictest0-21swapper/223:20:302
186890399128,2cyclictest0-21swapper/223:05:252
186890399128,2cyclictest0-21swapper/220:40:112
186890399128,2cyclictest0-21swapper/200:02:042
186890399127,3cyclictest0-21swapper/221:10:132
186890399126,4cyclictest1879353-21http19:20:022
186890399121,9cyclictest260-21systemd-journal20:09:592
1868903991210,1cyclictest0-21swapper/222:30:092
186890199129,2cyclictest311ktimers/122:25:231
186890199129,2cyclictest2092324-21fschecks_count22:20:151
186890199129,2cyclictest2077145-21gltestperf22:10:191
186890199129,2cyclictest1873641-21unixbench_singl19:10:251
186890199129,2cyclictest0-21swapper/123:00:141
186890199129,2cyclictest0-21swapper/122:20:011
186890199129,2cyclictest0-21swapper/119:50:121
186890199128,3cyclictest311ktimers/100:33:211
186890199128,3cyclictest2017645-21fschecks_count21:30:161
186890199127,4cyclictest1951427-21grep20:35:161
186890199127,4cyclictest0-21swapper/123:10:001
186890199126,4cyclictest2250909-21ssh00:05:201
186890199126,4cyclictest1918193-21cron20:05:001
186890199126,4cyclictest1880664-21sed19:20:181
186890199121,2cyclictest2001293-21ssh21:18:091
186889699128,3cyclictest60650irq/128-enp0s29f1:rx-020:05:100
186889699126,4cyclictest1880173-21df_inode19:20:130
1868896991210,2cyclictest60650irq/128-enp0s29f1:rx-019:55:050
186890699119,1cyclictest0-21swapper/320:30:353
186890699119,1cyclictest0-21swapper/320:20:283
186890699118,2cyclictest1962507-21kthreadcore20:45:203
186890699118,2cyclictest1886380-21kthreadcore19:25:203
186890699110,3cyclictest0-21swapper/319:10:243
186890399118,2cyclictest1961482-21cpuspeed_turbos20:45:132
186890399118,2cyclictest1924259-21cpuspeed_turbos20:10:122
186890399118,2cyclictest0-21swapper/220:30:382
186890399118,2cyclictest0-21swapper/219:45:012
186890399117,2cyclictest0-21swapper/222:10:252
186890399117,2cyclictest0-21swapper/219:20:212
186890399115,4cyclictest2119471-21ssh22:35:362
186890199119,1cyclictest0-21swapper/123:20:171
186890199118,2cyclictest311ktimers/123:57:241
186890199117,3cyclictest1918535-21cpuspeed_turbos20:05:111
186890199116,4cyclictest2265871-21tr00:15:181
186890199110,8cyclictest1891016-21cat19:35:021
186890199110,4cyclictest2042148-21grep21:45:221
186890699100,3cyclictest0-21swapper/319:24:063
186890699100,2cyclictest191rcu_preempt21:00:233
186890699100,2cyclictest191rcu_preempt21:00:223
186890399105,3cyclictest1873869-21sshd19:12:362
186890399104,4cyclictest1913681-21/usr/sbin/munin20:00:192
186890199107,2cyclictest311ktimers/123:15:121
186890199107,2cyclictest1982509-21df21:05:161
186890199107,2cyclictest0-21swapper/120:55:111
186890199106,3cyclictest311ktimers/120:30:161
186890199105,4cyclictest1885621-21grep19:25:171
186890199104,4cyclictest1971614-21sh20:55:011
186890199100,4cyclictest1868811-21latency_hist19:10:001
18688969974,2cyclictest1868984-21cat19:10:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional