You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-29 - 18:00
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1.osadl.org (updated Sat Nov 29, 2025 12:45:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
30942930,4chrt0-21swapper/212:16:322
30942930,4chrt0-21swapper/212:16:322
70132870,3sleep30-21swapper/307:41:043
70132870,3sleep30-21swapper/307:41:043
175412860,3sleep00-21swapper/010:40:410
175412860,3sleep00-21swapper/010:40:410
50892800,3sleep00-21swapper/007:36:070
50892800,3sleep00-21swapper/007:36:070
203202750,3sleep220315-21ssh09:25:522
203202750,3sleep220315-21ssh09:25:522
291862670,4sleep30-21swapper/308:41:003
291862670,4sleep30-21swapper/308:41:003
198712640,3chrt0-21swapper/312:35:023
198712640,3chrt0-21swapper/312:35:023
118762610,3sleep00-21swapper/010:32:530
118762610,3sleep00-21swapper/010:32:530
29137996044,10cyclictest7968-21sh09:11:002
29137996044,10cyclictest7968-21sh09:11:002
2913599583,44cyclictest12565-21ssh09:56:020
2913599583,44cyclictest12565-21ssh09:56:020
111502570,3sleep00-21swapper/011:50:500
111502570,3sleep00-21swapper/011:50:500
98652560,2sleep29866-21sshd07:48:162
39962550,3chrt0-21swapper/311:41:133
39962550,3chrt0-21swapper/311:41:133
247012550,3chrt24700-21sed08:26:140
128662540,3sleep00-21swapper/012:27:230
128662540,3sleep00-21swapper/012:27:230
2913799539,4cyclictest131rcu_preempt10:41:002
2913799539,4cyclictest131rcu_preempt10:41:002
287742530,1sleep27616-21diskmemload11:31:562
116192530,2sleep011629-21sh09:54:480
116192530,2sleep011629-21sh09:54:480
296462520,3sleep20-21swapper/207:15:522
296462520,3sleep20-21swapper/207:15:522
2913799523,40cyclictest14746-21ssh11:16:002
2913799523,40cyclictest14746-21ssh11:16:002
29135995216,25cyclictest38850irq/55-eth010:16:020
29135995216,25cyclictest38850irq/55-eth010:16:020
2913599503,41cyclictest542-21snmpd12:36:010
2913599503,41cyclictest542-21snmpd12:36:010
2913599493,12cyclictest131rcu_preempt08:45:590
2913599493,12cyclictest131rcu_preempt08:45:590
285232490,2sleep128529-21ntp_states07:11:091
285232490,2sleep128529-21ntp_states07:11:091
29135994832,10cyclictest6643-21apt-get07:41:000
29135994832,10cyclictest6643-21apt-get07:41:000
29135994831,13cyclictest27068-21apt-get11:30:590
206792480,5sleep333-21ksoftirqd/312:01:023
206792480,5sleep333-21ksoftirqd/312:01:023
2913599477,30cyclictest732-21runrttasks07:45:580
29135994730,11cyclictest38950irq/56-eth011:26:020
29135994730,11cyclictest38950irq/56-eth011:26:020
290792470,3sleep00-21swapper/008:40:530
17752470,2chrt0-21swapper/311:00:053
2913599454,32cyclictest32652-21ssh09:40:580
2913599454,32cyclictest32652-21ssh09:40:580
29135994528,9cyclictest38850irq/55-eth012:31:000
29135994528,9cyclictest38850irq/55-eth012:31:000
2913599452,35cyclictest21027-21ssh10:06:000
2913699445,32cyclictest12311-21ssh09:16:001
2913699445,32cyclictest12311-21ssh09:16:001
2913699445,32cyclictest12311-21ssh09:16:001
290902440,3sleep027646-21ls07:11:160
290902440,3sleep027646-21ls07:11:160
178672440,3sleep20-21swapper/211:57:072
2913799434,3cyclictest131rcu_preempt12:11:002
2913799434,3cyclictest131rcu_preempt12:11:002
70022410,4sleep30-21swapper/312:21:123
70022410,4sleep30-21swapper/312:21:123
213252410,4sleep30-21swapper/308:20:473
213252410,4sleep30-21swapper/308:20:473
36262400,2sleep23846-21unixbench_singl11:01:142
36262400,2sleep23846-21unixbench_singl11:01:142
2913599405,23cyclictest38850irq/55-eth010:26:010
2913599405,23cyclictest38850irq/55-eth010:26:010
2913599402,31cyclictest25381-21ssh10:11:020
2913599402,31cyclictest25381-21ssh10:11:020
188352400,2sleep30-21swapper/311:20:593
188352400,2sleep30-21swapper/311:20:593
119702400,3sleep011969-21grep11:51:060
119702400,3sleep011969-21grep11:51:060
11422390,8sleep333-21ksoftirqd/310:20:553
11422390,8sleep333-21ksoftirqd/310:20:553
29135993725,7cyclictest1292-21rm10:21:010
29135993725,7cyclictest1292-21rm10:21:010
2913699361,27cyclictest2594-21sh11:01:001
2913699361,27cyclictest2594-21sh11:01:001
2913599363,27cyclictest4361-21ssh09:46:000
2913599363,27cyclictest4361-21ssh09:46:000
29135993627,5cyclictest38950irq/56-eth009:20:590
29135993627,5cyclictest38950irq/56-eth009:20:590
2913699351,28cyclictest31337-21ssh11:36:011
126262350,3chrt0-21swapper/007:56:040
126262350,3chrt0-21swapper/007:56:040
95642340,3sleep10-21swapper/110:30:571
95642340,3sleep10-21swapper/110:30:571
34242340,3sleep00-21swapper/011:41:060
34242340,3sleep00-21swapper/011:41:060
2913799345,3cyclictest131rcu_preempt08:51:002
29137993410,4cyclictest131rcu_preempt08:26:012
2913699346,23cyclictest7944-21ssh09:10:591
2913699346,23cyclictest7944-21ssh09:10:591
2913599348,11cyclictest11175-21systemctl07:51:110
209482340,3sleep30-21swapper/310:05:563
29137993310,17cyclictest131rcu_preempt10:11:002
29137993310,17cyclictest131rcu_preempt10:11:002
2913599338,18cyclictest0-21swapper/008:21:000
2913599338,18cyclictest0-21swapper/008:21:000
2913599338,11cyclictest23765-21rm09:29:510
2913599331,18cyclictest21657-21cat08:20:560
133682330,3sleep30-21swapper/311:52:263
133682330,3sleep30-21swapper/311:52:263
132452330,3sleep213247-21cat09:16:112
132452330,3sleep213247-21cat09:16:112
132452330,3sleep213247-21cat09:16:112
2913599328,15cyclictest0-21swapper/010:41:000
2913599328,15cyclictest0-21swapper/010:41:000
2913599326,10cyclictest6751-21cat12:21:070
2913599326,10cyclictest6751-21cat12:21:070
2913599325,20cyclictest38850irq/55-eth012:06:020
2913599325,20cyclictest38850irq/55-eth012:06:020
2913599325,11cyclictest21846-21unixbench-2d12:01:140
2913599325,11cyclictest21846-21unixbench-2d12:01:140
29135993221,9cyclictest7948-21apt-get09:10:590
29135993221,9cyclictest7948-21apt-get09:10:590
29135993219,10cyclictest29776-21apt-get07:15:590
29135993219,10cyclictest29776-21apt-get07:15:590
265142320,3sleep3321rcuc/310:50:573
265142320,3sleep3321rcuc/310:50:573
205612320,3sleep00-21swapper/010:05:320
205612320,3sleep00-21swapper/010:05:320
142052320,3sleep30-21swapper/309:17:133
142052320,3sleep30-21swapper/309:17:133
142052320,3sleep30-21swapper/309:17:133
115132320,4sleep10-21swapper/111:11:121
115132320,4sleep10-21swapper/111:11:121
94332310,2sleep10-21swapper/112:24:271
94332310,2sleep10-21swapper/112:24:271
2913599318,15cyclictest0-21swapper/012:11:000
2913599318,15cyclictest0-21swapper/012:11:000
2913599316,11cyclictest316581idleruntime11:36:040
29135993111,14cyclictest26598-21ssh10:51:020
29135993111,14cyclictest26598-21ssh10:51:020
26702310,2sleep34244-21kworker/3:110:21:233
26702310,2sleep34244-21kworker/3:110:21:233
2913699302,24cyclictest20667-21ssh12:01:001
2913699302,24cyclictest20667-21ssh12:01:001
2913599306,11cyclictest4274-21sh11:02:170
2913599306,11cyclictest4274-21sh11:02:170
2913599304,10cyclictest29899-21ps08:41:100
2913599304,10cyclictest29899-21ps08:41:100
29135993016,12cyclictest2948-21apt-get07:31:000
29135993016,12cyclictest2948-21apt-get07:31:000
138202300,3sleep10-21swapper/110:36:001
138202300,3sleep10-21swapper/110:36:001
77762290,4sleep30-21swapper/310:28:103
77762290,4sleep30-21swapper/310:28:103
2913699292,20cyclictest20787-21ssh12:36:011
2913699292,20cyclictest20787-21ssh12:36:011
2913599299,8cyclictest28035-21rm09:35:280
2913599298,14cyclictest0-21swapper/011:21:010
2913599298,14cyclictest0-21swapper/011:21:010
2913599297,9cyclictest16584-21sh11:56:080
2913599297,10cyclictest28591-21apt-get09:36:020
2913599297,10cyclictest28591-21apt-get09:36:020
2913599295,8cyclictest3023-21ps08:56:090
2913599295,8cyclictest3023-21ps08:56:090
2913599295,18cyclictest29650irq/49-vc410:36:090
2913599286,9cyclictest131rcu_preempt08:51:000
2913599285,16cyclictest1571-21ssh12:16:010
2913599285,16cyclictest1571-21ssh12:16:010
2913599284,16cyclictest30755-21ssh10:56:010
2913599283,10cyclictest25919-21cat08:31:040
29135992812,13cyclictest0-21swapper/011:05:590
29135992812,13cyclictest0-21swapper/011:05:590
163482280,3sleep30-21swapper/311:16:453
163482280,3sleep30-21swapper/311:16:453
2913599272,20cyclictest38850irq/55-eth011:11:020
2913599272,20cyclictest38850irq/55-eth011:11:020
2913599272,20cyclictest38850irq/55-eth009:01:010
2913599272,20cyclictest38850irq/55-eth009:01:010
2913599272,19cyclictest1094-21taskset07:26:000
2913599272,19cyclictest1094-21taskset07:26:000
29135992717,8cyclictest14745-21apt-get11:16:000
29135992717,8cyclictest14745-21apt-get11:16:000
287312270,4sleep128729-21fschecks_count09:36:041
287312270,4sleep128729-21fschecks_count09:36:041
123222270,3sleep30-21swapper/311:11:593
123222270,3sleep30-21swapper/311:11:593
2913799263,3cyclictest131rcu_preempt09:36:022
2913799263,3cyclictest131rcu_preempt09:36:022
2913699262,22cyclictest9070-21needreboot07:46:061
2913599267,7cyclictest12292-21apt-get09:15:590
2913599267,7cyclictest12292-21apt-get09:15:590
2913599267,7cyclictest12292-21apt-get09:15:590
29135992611,12cyclictest0-21swapper/008:06:000
29135992611,12cyclictest0-21swapper/008:06:000
272722260,4sleep30-21swapper/310:12:233
272722260,4sleep30-21swapper/310:12:233
6562250,4sleep10-21swapper/112:14:551
6562250,4sleep10-21swapper/112:14:551
2913799257,10cyclictest131rcu_preempt07:25:592
2913799257,10cyclictest131rcu_preempt07:25:592
2913799253,18cyclictest11370-21ssh11:51:002
2913799253,18cyclictest11370-21ssh11:51:002
2913699254,15cyclictest26152-21apt-get10:50:571
2913699254,15cyclictest26152-21apt-get10:50:571
2913699252,20cyclictest29163-21apt10:15:471
2913699252,20cyclictest29163-21apt10:15:471
2913699251,22cyclictest31822-21wc08:46:101
2913699251,22cyclictest31822-21wc08:46:101
2913599257,14cyclictest29650irq/49-vc410:36:090
2913599257,14cyclictest29650irq/49-vc410:36:090
2913599251,9cyclictest18635-21mailstats08:11:120
2913599251,9cyclictest18635-21mailstats08:11:120
29135992511,10cyclictest14096-21apt-get08:01:020
29135992511,10cyclictest14096-21apt-get08:01:020
140822250,4sleep10-21swapper/111:53:321
140822250,4sleep10-21swapper/111:53:321
85082240,3chrt0-21swapper/307:45:593
2913799243,18cyclictest21383-21rm10:06:052
2913799243,17cyclictest13839-21apt-get10:36:012
2913799243,17cyclictest13839-21apt-get10:36:012
2913799242,19cyclictest5262-21latency_hist10:25:552
2913799242,19cyclictest5262-21latency_hist10:25:552
2913799242,19cyclictest32196-21grep08:46:132
2913799242,19cyclictest32196-21grep08:46:132
2913799242,19cyclictest25686-21tr08:31:022
2913699242,20cyclictest17620-21timerandwakeup09:21:111
2913699242,20cyclictest17620-21timerandwakeup09:21:111
2913699242,19cyclictest7396-21grep11:06:101
2913699242,19cyclictest7396-21grep11:06:101
2913699242,19cyclictest30720-21/usr/sbin/munin10:56:121
2913699242,19cyclictest2538-21fschecks_time08:56:031
2913699242,19cyclictest2538-21fschecks_time08:56:031
2913699242,19cyclictest19580-21wc08:15:511
2913699242,19cyclictest19580-21wc08:15:511
2913599243,10cyclictest21461-21apt-config08:20:490
2913599242,15cyclictest628-21sshd07:21:020
264252240,4sleep30-21swapper/308:31:113
2913799237,4cyclictest0-21swapper/207:51:112
2913799233,17cyclictest13303-21grep09:56:112
2913799233,17cyclictest13303-21grep09:56:112
2913799233,16cyclictest977-21wc07:25:542
2913799232,19cyclictest889-21ntp_kernel_pll_09:41:062
2913799232,19cyclictest889-21ntp_kernel_pll_09:41:062
2913799232,19cyclictest3080-21fschecks_count07:31:012
2913799232,19cyclictest3080-21fschecks_count07:31:012
2913799232,19cyclictest11884-21grep12:26:142
2913799232,19cyclictest11884-21grep12:26:142
2913799232,17cyclictest26513-21ssh10:50:572
2913799232,17cyclictest26513-21ssh10:50:572
29137992314,7cyclictest135-21systemd-journal08:00:462
29137992314,7cyclictest135-21systemd-journal08:00:462
2913799231,19cyclictest29252-21ls08:41:022
2913799231,19cyclictest29252-21ls08:41:022
29137992311,3cyclictest27-21ksoftirqd/211:06:122
29137992311,3cyclictest27-21ksoftirqd/211:06:122
2913699233,7cyclictest21-21ksoftirqd/111:41:071
2913699233,7cyclictest21-21ksoftirqd/111:41:071
2913699233,17cyclictest10903-21apt-get11:50:501
2913699233,17cyclictest10903-21apt-get11:50:501
2913699232,18cyclictest7564-21find07:41:111
2913699232,18cyclictest7564-21find07:41:111
2913699232,18cyclictest5302-21grep09:01:141
2913699232,18cyclictest5302-21grep09:01:141
2913699232,18cyclictest1019-21latency_hist07:25:571
2913699232,17cyclictest15717-21apt-get12:31:011
2913699232,17cyclictest15717-21apt-get12:31:011
2913699231,20cyclictest15177-21unixbench-2d08:01:131
2913699231,20cyclictest15177-21unixbench-2d08:01:131
2913699231,19cyclictest23231-21fschecks_time11:26:051
2913699231,19cyclictest23231-21fschecks_time11:26:051
2913699231,19cyclictest1543-21fschecks_time10:21:041
2913699231,19cyclictest1543-21fschecks_time10:21:041
2913699231,13cyclictest126762df_inode09:56:041
2913699231,13cyclictest126762df_inode09:56:041
2913799223,16cyclictest3787-21cut11:41:092
2913799223,16cyclictest3787-21cut11:41:092
2913799222,18cyclictest25672-21grep09:31:132
2913799222,17cyclictest12211-21wc09:55:472
2913799222,17cyclictest12211-21wc09:55:472
29137992215,5cyclictest22860-21users08:21:132
29137992215,5cyclictest22860-21users08:21:132
29137992213,7cyclictest22595-21ssh12:02:212
29137992213,7cyclictest22595-21ssh12:02:212
29137992212,7cyclictest20950-21latency_hist10:05:562
29137992210,4cyclictest27-21ksoftirqd/210:52:032
29137992210,4cyclictest27-21ksoftirqd/210:52:032
2913699223,17cyclictest29963-21fschecks_count07:16:021
2913699223,17cyclictest29963-21fschecks_count07:16:021
2913699222,18cyclictest24561-21sort09:31:011
2913699222,18cyclictest20718-21fschecks_time09:26:021
2913699222,18cyclictest18040-21grep10:01:131
2913699222,18cyclictest18040-21grep10:01:131
2913699222,18cyclictest1355-21fschecks_time07:26:031
2913699222,18cyclictest1355-21fschecks_time07:26:031
2913699222,18cyclictest13306-21grep07:56:121
2913699222,18cyclictest13306-21grep07:56:121
2913699222,18cyclictest11149-21kernelversion12:26:061
2913699222,18cyclictest11149-21kernelversion12:26:061
2913699221,4cyclictest0-21swapper/110:51:001
2913699221,4cyclictest0-21swapper/110:51:001
29136992214,6cyclictest3557-21ssh12:17:131
29136992214,6cyclictest3557-21ssh12:17:131
29136992214,6cyclictest17945-21sh10:40:581
29136992214,6cyclictest17945-21sh10:40:581
2913699221,19cyclictest25589-21fschecks_count08:31:021
2913699221,18cyclictest9427-21latency_hist10:30:531
2913699221,18cyclictest9427-21latency_hist10:30:531
29136992211,3cyclictest21-21ksoftirqd/111:17:231
29136992211,3cyclictest21-21ksoftirqd/111:17:231
189872220,3sleep30-21swapper/308:11:143
189872220,3sleep30-21swapper/308:11:143
76562210,3sleep10-21swapper/109:10:481
2913799215,11cyclictest0-21swapper/207:36:032
2913799215,11cyclictest0-21swapper/207:36:032
2913799213,16cyclictest5173-21sendmail_mailqu09:46:102
2913799213,16cyclictest5173-21sendmail_mailqu09:46:102
2913799213,16cyclictest4251-21cpuspeed09:01:022
2913799213,16cyclictest4251-21cpuspeed09:01:022
2913799213,16cyclictest11284-21ssh10:31:522
2913799213,16cyclictest11284-21ssh10:31:522
2913799212,17cyclictest7699-21latency_hist09:10:502
2913799212,17cyclictest20696-21wc10:05:472
2913799212,17cyclictest20423-21idleruntime-cro12:35:472
2913799212,17cyclictest20423-21idleruntime-cro12:35:472
2913799212,17cyclictest19520-21awk11:21:072
2913799212,17cyclictest19520-21awk11:21:072
2913799212,17cyclictest16739-21sendmail_mailqu08:06:112
2913799212,17cyclictest16739-21sendmail_mailqu08:06:112
2913799211,18cyclictest7214-21needreboot07:41:072
2913799211,18cyclictest7214-21needreboot07:41:072
2913799211,18cyclictest539-21cron08:15:472
2913799211,18cyclictest539-21cron08:15:472
2913799211,18cyclictest30864-21sed07:16:122
2913799211,18cyclictest30864-21sed07:16:122
2913799211,18cyclictest30784-21grep10:16:152
2913799211,18cyclictest30784-21grep10:16:152
2913799211,18cyclictest2603-21fschecks_time08:56:032
2913799211,18cyclictest2603-21fschecks_time08:56:032
2913799211,18cyclictest21085-21fschecks_time12:36:042
2913799211,18cyclictest21085-21fschecks_time12:36:042
2913799211,18cyclictest14380-21fschecks_time08:01:042
2913799211,18cyclictest14380-21fschecks_time08:01:042
2913799211,18cyclictest11634-21ls11:11:132
2913799211,18cyclictest11634-21ls11:11:132
2913799211,11cyclictest732-21runrttasks08:19:122
2913799211,11cyclictest732-21runrttasks08:19:122
29137992110,3cyclictest27-21ksoftirqd/208:36:072
2913699213,16cyclictest30691-21sed10:16:141
2913699213,16cyclictest30691-21sed10:16:141
2913699213,15cyclictest683-21tune2fs08:51:031
2913699212,17cyclictest21842-21processes10:06:111
2913699212,16cyclictest8522-21grep09:51:041
2913699212,16cyclictest8522-21grep09:51:041
2913699211,17cyclictest24870-21ls12:06:031
2913699211,17cyclictest24870-21ls12:06:031
2913699211,11cyclictest4866-21memory09:46:051
2913699211,11cyclictest4866-21memory09:46:051
2913699211,11cyclictest287692apt-get08:40:531
2913799209,9cyclictest31432-21df11:36:022
2913799207,11cyclictest25465-21ntp_offset12:06:092
2913799207,11cyclictest25465-21ntp_offset12:06:092
2913799203,15cyclictest7042-21mailstats12:21:122
2913799203,15cyclictest7042-21mailstats12:21:122
2913799202,16cyclictest7309-21cpuspeed11:46:032
2913799202,16cyclictest7309-21cpuspeed11:46:032
2913799202,16cyclictest6541-21grep10:26:142
2913799202,16cyclictest6541-21grep10:26:142
2913799201,17cyclictest24263-21grep11:26:162
2913799201,17cyclictest24263-21grep11:26:162
2913699203,15cyclictest4952-21cut07:36:061
2913699203,15cyclictest4952-21cut07:36:061
2913699203,15cyclictest19872-21xargs11:21:121
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional