You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-04-20 - 14:52
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot4.osadl.org (updated Sat Apr 20, 2024 00:43:44)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
206406099220,21cyclictest2249966-21turbostat21:49:590
2064062992117,2cyclictest2341448-21irqcore22:50:181
2064070992016,2cyclictest0-21swapper/321:55:003
2064065992017,2cyclictest2296551-21irqcore22:20:162
206406299209,3cyclictest2220108-21kworker/u8:1+xprtiod22:22:371
2064062992016,2cyclictest2212628-21turbostat.cron21:25:011
2064062992013,5cyclictest260-21systemd-journal23:27:511
2064070991915,3cyclictest506-21gpm23:10:003
2064070991915,2cyclictest2340886-21/usr/sbin/munin22:50:203
2064065991916,2cyclictest0-21swapper/219:35:002
2064065991915,2cyclictest0-21swapper/222:34:592
2064070991814,3cyclictest1-21systemd23:25:513
2064062991814,2cyclictest0-21swapper/123:45:001
2064062991812,4cyclictest2442745-21grep23:55:211
2064062991811,5cyclictest2258313-21cut21:55:131
2064060991815,2cyclictest516-21rs:main0
2064060991811,6cyclictest2242537-21turbostat21:44:590
2064070991714,2cyclictest516-21rs:main3
2064070991714,2cyclictest491ktimers/323:35:193
2064070991714,2cyclictest2438447-21gltestperf23:55:113
2064070991714,2cyclictest2240055-21unixbench_singl21:40:243
2064070991713,3cyclictest2179597-21cron21:00:003
2064065991714,2cyclictest514-21in:imuxsock21:30:002
2064065991714,2cyclictest2446202-21irqcore00:00:172
2064065991714,2cyclictest2122400-21irqcore20:05:212
2064065991713,3cyclictest2227579-21cron21:35:002
2064062991713,2cyclictest0-21swapper/100:05:011
2064062991712,3cyclictest2191006-21irqcore21:10:161
2064060991714,2cyclictest60650irq/128-enp0s29f1:rx-023:47:520
2064060991714,2cyclictest60650irq/128-enp0s29f1:rx-021:26:520
2064060991713,3cyclictest2408372-21cpuspeed_turbos23:35:140
206407099169,5cyclictest260-21systemd-journal22:10:003
2064070991614,1cyclictest2169866-21irqcore20:50:213
2064070991613,2cyclictest0-21swapper/321:20:013
2064070991613,2cyclictest0-21swapper/320:40:003
2064070991612,3cyclictest2430898-21fschecks_time23:50:143
2064070991612,2cyclictest0-21swapper/322:35:123
2064070991612,2cyclictest0-21swapper/320:00:003
2064065991613,2cyclictest2265694-21latency_hist22:00:002
2064065991612,3cyclictest2404914-21needreboot23:30:212
206406299168,3cyclictest2220108-21kworker/u8:1+xprtiod22:17:281
206406299163,1cyclictest2158657-21grep20:40:011
2064062991613,2cyclictest2324103-21kworker/u8:3-rpciod22:55:171
2064062991613,2cyclictest2324103-21kworker/u8:3-rpciod22:55:161
2064062991613,2cyclictest2306787-21kthreadcore22:25:191
2064062991612,3cyclictest2472256-21cut00:15:211
2064062991612,3cyclictest2137310-21sh20:20:001
2064062991612,2cyclictest0-21swapper/123:14:591
2064062991612,2cyclictest0-21swapper/119:35:151
206406299161,14cyclictest0-21swapper/122:05:231
206406299160,4cyclictest2385080-21ssh23:19:191
2064060991614,2cyclictest60650irq/128-enp0s29f1:rx-020:23:150
2064060991614,1cyclictest60650irq/128-enp0s29f1:rx-023:53:410
2064060991614,1cyclictest60650irq/128-enp0s29f1:rx-023:15:020
2064060991614,1cyclictest60650irq/128-enp0s29f1:rx-023:10:080
2064060991614,1cyclictest60650irq/128-enp0s29f1:rx-000:12:010
2064060991613,2cyclictest61450irq/129-enp0s29f1:tx-023:23:320
2064060991613,2cyclictest61450irq/129-enp0s29f1:tx-022:30:220
2064060991613,2cyclictest61450irq/129-enp0s29f1:tx-000:25:100
2064060991613,2cyclictest60650irq/128-enp0s29f1:rx-023:56:100
2064060991613,2cyclictest60650irq/128-enp0s29f1:rx-023:25:130
2064060991613,2cyclictest60650irq/128-enp0s29f1:rx-022:45:140
2064060991613,2cyclictest60650irq/128-enp0s29f1:rx-022:35:170
2064060991613,2cyclictest60650irq/128-enp0s29f1:rx-022:25:190
2064060991613,2cyclictest60650irq/128-enp0s29f1:rx-022:16:250
2064060991613,2cyclictest60650irq/128-enp0s29f1:rx-021:51:070
2064060991613,2cyclictest60650irq/128-enp0s29f1:rx-020:45:170
2064060991613,2cyclictest60650irq/128-enp0s29f1:rx-000:00:120
2064060991613,2cyclictest171ktimers/022:44:210
2064060991613,2cyclictest0-21swapper/020:55:010
2064060991612,3cyclictest60650irq/128-enp0s29f1:rx-022:55:140
2064060991612,3cyclictest60650irq/128-enp0s29f1:rx-022:55:130
2064060991612,3cyclictest2400840-21cpuspeed_turbos23:30:140
2064060991611,2cyclictest60650irq/128-enp0s29f1:rx-021:37:270
2064060991611,2cyclictest60650irq/128-enp0s29f1:rx-021:20:040
2064070991513,1cyclictest3783561-21kworker/3:2+events_power_efficient23:23:163
2064070991512,2cyclictest2232045-21needreboot21:35:203
2064070991512,2cyclictest2143541-21gltestperf20:25:173
2064070991512,2cyclictest2137696-21latency_hist20:20:003
2064070991511,3cyclictest2281237-21cpuspeed_turbos22:10:113
2064070991511,3cyclictest0-21swapper/321:55:183
2064070991511,3cyclictest0-21swapper/300:10:013
2064070991511,2cyclictest509-21systemd-logind23:35:013
2064070991511,2cyclictest0-21swapper/319:30:003
206406599152,11cyclictest2095426-21munin-run19:40:002
2064065991513,1cyclictest2897824-21kworker/u8:2-flush-259:020:05:002
2064065991513,1cyclictest2897824-21kworker/u8:2-flush-259:020:05:002
2064065991513,1cyclictest2280753-21idleruntime-cro22:10:002
2064065991512,2cyclictest2320726-21kthreadcore22:35:172
2064065991512,2cyclictest2132844-21irqcore20:15:152
2064065991512,2cyclictest2104958-21needreboot19:45:212
2064065991511,3cyclictest2475683-21cron00:20:012
2064065991511,3cyclictest2254856-21grep21:50:242
2064065991510,4cyclictest0-21swapper/222:45:002
206406299159,4cyclictest2333063-21cron22:45:001
206406299158,5cyclictest260-21systemd-journal19:50:001
206406299153,1cyclictest191rcu_preempt23:20:171
206406299151,5cyclictest2190127-21awk21:10:011
206406299151,4cyclictest2257599-21sort21:55:011
2064062991512,2cyclictest2270756-21unixbench_multi22:00:251
2064062991512,2cyclictest2031560-21munin-node21:25:061
2064062991511,2cyclictest0-21swapper/123:50:511
2064062991511,2cyclictest0-21swapper/121:30:141
2064062991511,2cyclictest0-21swapper/119:40:181
2064060991514,1cyclictest60650irq/128-enp0s29f1:rx-022:10:190
2064060991513,2cyclictest60650irq/128-enp0s29f1:rx-019:44:420
2064060991513,1cyclictest60650irq/128-enp0s29f1:rx-022:50:100
2064060991513,1cyclictest60650irq/128-enp0s29f1:rx-022:23:220
2064060991513,1cyclictest60650irq/128-enp0s29f1:rx-021:15:060
2064060991513,1cyclictest60650irq/128-enp0s29f1:rx-020:40:190
2064060991513,1cyclictest60650irq/128-enp0s29f1:rx-019:50:180
2064060991513,1cyclictest60650irq/128-enp0s29f1:rx-000:30:200
2064060991513,1cyclictest60650irq/128-enp0s29f1:rx-000:15:120
2064060991512,2cyclictest60650irq/128-enp0s29f1:rx-023:05:110
2064060991512,2cyclictest60650irq/128-enp0s29f1:rx-022:05:100
2064060991512,2cyclictest60650irq/128-enp0s29f1:rx-021:55:200
2064060991512,2cyclictest60650irq/128-enp0s29f1:rx-021:30:090
2064060991512,2cyclictest60650irq/128-enp0s29f1:rx-021:10:060
2064060991512,2cyclictest60650irq/128-enp0s29f1:rx-020:00:210
2064060991512,2cyclictest60650irq/128-enp0s29f1:rx-020:00:210
2064060991512,2cyclictest60650irq/128-enp0s29f1:rx-000:35:220
2064060991512,2cyclictest60650irq/128-enp0s29f1:rx-000:20:110
2064060991512,2cyclictest60650irq/128-enp0s29f1:rx-000:05:190
2064060991512,2cyclictest2266432-21gltestperf22:00:150
2064060991511,3cyclictest60650irq/128-enp0s29f1:rx-023:00:110
2064070991411,2cyclictest491ktimers/323:10:103
2064070991411,2cyclictest2491332-21gltestperf00:30:143
2064070991411,2cyclictest2486971-21kthreadcore00:25:173
2064070991411,2cyclictest2473096-21timerandwakeup00:15:283
2064070991411,2cyclictest2160345-21kthreadcore20:40:173
2064070991411,2cyclictest0-21swapper/322:25:003
2064070991411,2cyclictest0-21swapper/322:20:003
2064070991410,3cyclictest2303462-21/usr/sbin/munin22:25:223
2064070991410,3cyclictest2116957-21gltestperf20:00:143
2064070991410,3cyclictest2116957-21gltestperf20:00:143
2064070991410,3cyclictest0-21swapper/300:20:203
2064070991410,2cyclictest0-21swapper/319:20:003
2064070991410,2cyclictest0-21swapper/300:35:143
206406599148,4cyclictest2440725-21cut23:55:142
206406599148,4cyclictest2240967-21sshd21:42:022
206406599148,4cyclictest1-21systemd21:10:012
2064065991412,1cyclictest401ktimers/200:27:292
2064065991412,1cyclictest2064052-21cyclictest22:55:012
2064065991411,2cyclictest401ktimers/222:57:152
2064065991411,2cyclictest401ktimers/222:57:142
2064065991411,2cyclictest2266638-21tr22:00:172
2064065991411,2cyclictest2206080-21irqcore21:20:202
2064065991411,2cyclictest2198239-21fschecks_time21:15:132
2064065991411,2cyclictest2144315-21kthreadcore20:25:202
2064065991411,2cyclictest2096191-21sed19:40:202
2064065991411,2cyclictest0-21swapper/221:49:592
2064065991411,2cyclictest0-21swapper/221:15:002
2064065991410,3cyclictest2079730-21sh19:25:002
2064065991410,2cyclictest401ktimers/219:25:122
206406599140,4cyclictest2336665-21grep22:45:212
206406299148,4cyclictest260-21systemd-journal20:21:501
206406299145,8cyclictest0-21swapper/100:40:011
206406299145,3cyclictest2452808-21apt-get00:05:011
2064062991412,2cyclictest311ktimers/123:49:341
2064062991411,2cyclictest509-21systemd-logind20:10:011
2064062991411,2cyclictest2365808-21tr23:05:181
2064062991411,2cyclictest2220108-21kworker/u8:1+xprtiod22:46:451
2064062991410,3cyclictest2412533-21nvmesmart_nvme023:35:251
2064062991410,3cyclictest2106240-21cpuspeed_turbos19:50:191
2064062991410,3cyclictest2069642-21cpuspeed_turbos19:15:131
2064062991410,2cyclictest0-21swapper/122:10:101
2064060991412,2cyclictest60650irq/128-enp0s29f1:rx-020:30:510
2064060991412,2cyclictest60650irq/128-enp0s29f1:rx-020:09:320
2064060991412,1cyclictest60650irq/128-enp0s29f1:rx-021:10:010
2064060991411,2cyclictest60650irq/128-enp0s29f1:rx-020:35:260
2064060991411,2cyclictest60650irq/128-enp0s29f1:rx-019:11:520
206407099139,3cyclictest491ktimers/322:55:103
206407099139,3cyclictest491ktimers/322:55:103
206407099139,3cyclictest23160922sleep322:31:173
206407099139,2cyclictest0-21swapper/322:45:213
206407099139,2cyclictest0-21swapper/321:30:173
206407099138,4cyclictest2424334-21cut23:45:183
2064070991310,2cyclictest2427914-21kworker/u8:1+xprtiod00:11:413
2064070991310,2cyclictest2243244-21gltestperf21:45:183
2064070991310,2cyclictest2110534-21kworker/u8:3+xprtiod21:22:393
206406599139,2cyclictest2340585-21kworker/u8:0-ext4-rsv-conversion23:10:012
206406599137,4cyclictest2371027-21grep23:10:102
2064065991312,1cyclictest401ktimers/223:00:162
2064065991311,1cyclictest0-21swapper/222:29:592
2064065991310,2cyclictest2480635-21unixbench_multi00:20:232
2064065991310,2cyclictest2427914-21kworker/u8:1+xprtiod00:14:352
2064065991310,2cyclictest2340585-21kworker/u8:0+xprtiod23:43:202
2064065991310,2cyclictest2324103-21kworker/u8:3+xprtiod23:39:472
2064065991310,2cyclictest2177170-21kthreadcore20:55:182
2064065991310,2cyclictest2106674-21irqcore19:50:232
2064065991310,2cyclictest0-21swapper/223:15:222
2064065991310,2cyclictest0-21swapper/220:10:212
2064065991310,2cyclictest0-21swapper/200:10:012
206406599130,2cyclictest2281508-21hddtemp_smartct22:10:142
206406299139,3cyclictest2401297-21sed23:30:181
206406299139,2cyclictest0-21swapper/122:30:251
206406299139,2cyclictest0-21swapper/121:15:141
206406299139,2cyclictest0-21swapper/100:25:121
206406299136,5cyclictest260-21systemd-journal00:31:051
206406299133,3cyclictest191rcu_preempt23:00:181
2064062991311,1cyclictest2164644-21irqcore20:45:181
206406299131,10cyclictest2318481-21cpuspeed_turbos22:35:121
2064062991310,2cyclictest61550irq/131-enp0s29f1:tx-100:12:251
2064062991310,1cyclictest0-21swapper/120:00:241
2064062991310,1cyclictest0-21swapper/120:00:241
206406099139,2cyclictest0-21swapper/020:00:010
2064060991311,1cyclictest2105902-21turbostat19:50:000
2064060991310,2cyclictest60650irq/128-enp0s29f1:rx-019:30:210
2064060991310,2cyclictest0-21swapper/021:00:000
206407099129,2cyclictest491ktimers/300:01:593
206407099129,2cyclictest2148592-21cpuspeed_turbos20:30:123
206407099129,1cyclictest0-21swapper/321:25:283
206407099128,3cyclictest2105431-21unixbench_multi19:45:263
206407099127,4cyclictest260-21systemd-journal19:40:513
206407099121,9cyclictest0-21swapper/322:45:003
206407099121,2cyclictest21685372ls20:45:273
2064070991210,2cyclictest2340585-21kworker/u8:0+xprtiod23:40:423
206407099120,3cyclictest2360226-21ssh23:00:243
206407099120,3cyclictest1-21systemd20:15:003
206406599129,2cyclictest61650irq/133-enp0s29f1:tx-222:19:222
206406599129,2cyclictest2153823-21cpuspeed_turbos20:35:132
206406599129,2cyclictest0-21swapper/219:15:002
206406599129,2cyclictest0-21swapper/200:35:012
206406599128,2cyclictest0-21swapper/223:20:142
206406599126,4cyclictest2398743-21sshd23:26:512
2064065991210,1cyclictest0-21swapper/220:40:172
206406599120,3cyclictest0-21swapper/223:46:052
206406299129,2cyclictest2476706-21irqcore00:20:151
206406299129,2cyclictest0-21swapper/120:10:171
206406299128,3cyclictest2111129-21cron19:59:561
206406299128,2cyclictest0-21swapper/119:20:101
206406299126,4cyclictest260-21systemd-journal20:41:511
206406299126,2cyclictest0-21swapper/121:35:121
206406299121,9cyclictest2148199-21latency_hist20:30:001
2064062991210,1cyclictest0-21swapper/119:10:001
206406099128,3cyclictest60650irq/128-enp0s29f1:rx-020:15:100
2064060991210,1cyclictest60650irq/128-enp0s29f1:rx-019:19:230
206407099119,1cyclictest2108578-21kthreadcore19:50:253
206407099118,2cyclictest2180243-21gltestperf21:00:133
206407099118,2cyclictest2064367-21cpuspeed_turbos19:10:143
206407099118,2cyclictest0-21swapper/322:00:123
206407099116,4cyclictest2189919-21sshd21:07:513
206407099116,3cyclictest2089396-21ntp_states19:30:243
206407099116,3cyclictest2078851-21nfs_client19:20:183
206406599119,2cyclictest2427914-21kworker/u8:1+xprtiod00:35:522
206406599118,2cyclictest0-21swapper/221:35:122
206406599118,2cyclictest0-21swapper/220:50:002
206406599117,2cyclictest0-21swapper/220:30:142
206406599116,4cyclictest2184875-21munin-run21:05:002
206406599112,2cyclictest2116540-21cat20:00:002
206406299119,1cyclictest0-21swapper/120:50:251
206406299117,3cyclictest2177027-21kthreadcore20:55:181
206406299117,2cyclictest0-21swapper/121:40:171
206406299112,2cyclictest191rcu_preempt21:00:151
206406299111,3cyclictest0-21swapper/119:10:161
206406099119,2cyclictest60650irq/128-enp0s29f1:rx-020:10:070
206406099116,4cyclictest2143522-21grep20:25:170
206406099115,5cyclictest2074773-21awk19:20:080
206407099105,4cyclictest2093169-21cut19:35:203
206407099100,2cyclictest2126833-21latency_hist20:10:003
206406599108,1cyclictest391rcuc/220:50:242
206406599100,3cyclictest833-21runrttasks19:19:282
206406299108,1cyclictest0-21swapper/119:30:191
206406299105,4cyclictest2149506-21sed20:30:161
206406099107,2cyclictest2180012-21cpuspeed_turbos21:00:110
206406099107,1cyclictest0-21swapper/019:35:260
20640659996,2cyclictest0-21swapper/220:20:132
20640609984,2cyclictest2064118-21cat19:10:000
20640659963,2cyclictest2064089-21latency_hist19:10:002
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional