You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-11 - 02:21
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1.osadl.org (updated Thu Dec 11, 2025 00:45:01)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2911221670,2sleep10-21swapper/121:22:121
2911221670,2sleep10-21swapper/121:22:121
2641821340,4sleep30-21swapper/321:54:213
2641821340,4sleep30-21swapper/321:54:213
1047421270,2chrt0-21swapper/323:21:063
1047421270,2chrt0-21swapper/323:21:063
1237821130,3sleep333-21ksoftirqd/320:56:143
1237821130,3sleep333-21ksoftirqd/320:56:143
2875621030,4sleep30-21swapper/321:21:423
2875621030,4sleep30-21swapper/321:21:423
2420021030,1sleep012-21ksoftirqd/020:01:140
2420021030,1sleep012-21ksoftirqd/020:01:140
18462990,2chrt0-21swapper/000:21:120
18462990,2chrt0-21swapper/000:21:120
201722970,3sleep10-21swapper/100:05:331
201722970,3sleep10-21swapper/100:05:331
130392940,3chrt0-21swapper/021:40:360
130392940,3chrt0-21swapper/021:40:360
83422930,3chrt0-21swapper/122:09:431
83422930,3chrt0-21swapper/122:09:431
142722870,3sleep00-21swapper/022:16:090
142722870,3sleep00-21swapper/022:16:090
298312840,5sleep073-21kswapd023:41:130
298312840,5sleep073-21kswapd023:41:130
40092830,3chrt4001-21ssh21:31:030
40092830,3chrt4001-21ssh21:31:030
269942820,3sleep10-21swapper/121:21:001
269942820,3sleep10-21swapper/121:21:001
58852750,2sleep00-21swapper/019:11:200
306812740,2sleep30-21swapper/300:16:203
306812740,2sleep30-21swapper/300:16:203
103052690,1sleep29493-21/usr/sbin/munin22:11:212
103052690,1sleep29493-21/usr/sbin/munin22:11:212
95322680,3chrt0-21swapper/020:46:220
95322680,3chrt0-21swapper/020:46:220
272522670,3sleep10-21swapper/122:29:361
67672640,2sleep30-21swapper/323:51:123
67672640,2sleep30-21swapper/323:51:123
215642620,3sleep10-21swapper/122:57:211
215642620,3sleep10-21swapper/122:57:211
98832610,3sleep20-21swapper/219:21:212
98832610,3sleep20-21swapper/219:21:212
271532610,2chrt0-21swapper/320:07:193
57722600,2sleep14979-21ls19:11:161
38312590,4chrt0-21swapper/122:39:001
38312590,4chrt0-21swapper/122:39:001
142572590,2chrt0-21swapper/321:01:123
142572590,2chrt0-21swapper/321:01:123
320282560,2sleep20-21swapper/222:34:432
320282560,2sleep20-21swapper/222:34:432
161422550,2sleep30-21swapper/321:06:113
123232540,5chrt0-21swapper/022:13:350
123232540,5chrt0-21swapper/022:13:350
210832530,3sleep00-21swapper/023:31:200
210832530,3sleep00-21swapper/023:31:200
125902530,3sleep10-21swapper/120:56:161
125902530,3sleep10-21swapper/120:56:161
89912520,2sleep29001-21ssh22:44:372
89912520,2sleep29001-21ssh22:44:372
186192520,3sleep00-21swapper/000:03:110
186192520,3sleep00-21swapper/000:03:110
164652520,3chrt0-21swapper/119:41:051
129582500,3chrt0-21swapper/123:22:571
129582500,3chrt0-21swapper/123:22:571
291722480,3sleep30-21swapper/323:40:523
141832480,3sleep10-21swapper/122:16:061
141832480,3sleep10-21swapper/122:16:061
140952480,2chrt0-21swapper/323:58:283
197152460,2sleep30-21swapper/322:21:143
38302450,3sleep30-21swapper/323:47:153
38302450,3sleep30-21swapper/323:47:153
173742440,3sleep30-21swapper/323:27:293
173742440,3sleep30-21swapper/323:27:293
129612440,3sleep00-21swapper/023:56:480
327042430,2sleep231913-21/usr/sbin/munin21:26:202
327042430,2sleep231913-21/usr/sbin/munin21:26:202
58232420,3sleep24979-21ls19:11:172
143222420,3sleep00-21swapper/022:50:250
143222420,3sleep00-21swapper/022:50:250
90442410,2sleep30-21swapper/320:46:173
90442410,2sleep30-21swapper/320:46:173
5512410,3sleep10-21swapper/122:36:031
5512410,3sleep10-21swapper/122:36:031
19532410,3chrt0-21swapper/121:28:071
19532410,3chrt0-21swapper/121:28:071
150172410,3sleep10-21swapper/100:36:201
150172410,3sleep10-21swapper/100:36:201
172732400,2chrt0-21swapper/322:18:493
172732400,2chrt0-21swapper/322:18:493
15242400,3sleep10-21swapper/120:26:181
15242400,3sleep10-21swapper/120:26:181
308132380,2sleep00-21swapper/000:16:210
308132380,2sleep00-21swapper/000:16:210
92552370,2sleep30-21swapper/300:30:193
92552370,2sleep30-21swapper/300:30:193
62422370,4sleep10-21swapper/123:16:141
62422370,4sleep10-21swapper/123:16:141
193682370,2sleep3321rcuc/321:46:173
193682370,2sleep3321rcuc/321:46:173
179432370,3chrt0-21swapper/023:28:190
179432370,3chrt0-21swapper/023:28:190
309232360,3sleep130921-21ssh00:16:221
309232360,3sleep130921-21ssh00:16:221
125632360,3sleep10-21swapper/121:39:521
125632360,3sleep10-21swapper/121:39:521
323952350,8sleep333-21ksoftirqd/323:43:333
323952350,8sleep333-21ksoftirqd/323:43:333
108232350,2sleep110835-21ssh23:55:421
108232350,2sleep110835-21ssh23:55:421
25562340,2sleep30-21swapper/321:29:013
25562340,2sleep30-21swapper/321:29:013
234972340,2sleep223512-21ssh23:34:332
234972340,2sleep223512-21ssh23:34:332
199372340,3sleep20-21swapper/200:05:102
199372340,3sleep20-21swapper/200:05:102
133672340,3sleep30-21swapper/323:23:353
133672340,3sleep30-21swapper/323:23:353
294692330,2chrt0-21swapper/300:15:483
294692330,2chrt0-21swapper/300:15:483
158022330,3sleep30-21swapper/300:36:503
158022330,3sleep30-21swapper/300:36:503
285242320,3sleep30-21swapper/321:56:183
285242320,3sleep30-21swapper/321:56:183
171002320,2sleep30-21swapper/321:44:293
171002320,2sleep30-21swapper/321:44:293
66052310,2sleep00-21swapper/022:41:200
66052310,2sleep00-21swapper/022:41:200
47052310,2sleep00-21swapper/023:14:440
47052310,2sleep00-21swapper/023:14:440
647699304,5cyclictest131rcu_preempt20:26:130
647699304,5cyclictest131rcu_preempt20:26:130
6476993014,12cyclictest151rcuc/020:56:130
6476993014,12cyclictest151rcuc/020:56:130
215112290,4sleep00-21swapper/000:06:210
215112290,4sleep00-21swapper/000:06:210
647699289,3cyclictest12-21ksoftirqd/021:20:220
647699289,3cyclictest12-21ksoftirqd/021:20:220
6476992810,3cyclictest12-21ksoftirqd/022:03:580
6476992810,3cyclictest12-21ksoftirqd/021:06:150
6476992810,2cyclictest12-21ksoftirqd/023:23:020
6476992810,2cyclictest12-21ksoftirqd/023:23:020
287452280,2sleep228755-21ssh21:21:422
287452280,2sleep228755-21ssh21:21:422
226582280,2sleep022670-21rm21:50:460
226582280,2sleep022670-21rm21:50:460
136922280,3sleep00-21swapper/000:35:380
136922280,3sleep00-21swapper/000:35:380
63742270,4sleep30-21swapper/322:06:433
63742270,4sleep30-21swapper/322:06:433
42892270,1sleep329925-21kworker/3:122:39:433
42892270,1sleep329925-21kworker/3:122:39:433
15202270,4chrt12-21ksoftirqd/022:36:180
15202270,4chrt12-21ksoftirqd/022:36:180
135052270,4sleep0151rcuc/019:31:200
135052270,4sleep0151rcuc/019:31:200
647799269,6cyclictest21-21ksoftirqd/121:11:111
647799269,6cyclictest21-21ksoftirqd/121:11:111
647699268,9cyclictest12-21ksoftirqd/021:41:110
647699268,9cyclictest12-21ksoftirqd/021:41:110
6476992614,4cyclictest12-21ksoftirqd/021:54:240
6476992614,4cyclictest12-21ksoftirqd/021:54:240
6476992614,10cyclictest25412-21ssh22:26:440
269012260,23sleep126911-21ssh23:37:291
133822260,2sleep30-21swapper/322:49:033
133822260,2sleep30-21swapper/322:49:033
6478992512,10cyclictest23927-21ps21:51:202
6478992512,10cyclictest23927-21ps21:51:202
647799258,6cyclictest131rcu_preempt20:01:131
647799258,6cyclictest131rcu_preempt20:01:131
6477992516,6cyclictest9043-21ntp_states20:46:171
6477992516,6cyclictest9043-21ntp_states20:46:171
74232240,3sleep30-21swapper/320:41:193
74232240,3sleep30-21swapper/320:41:193
6478992413,9cyclictest17573-21rm21:45:132
6478992413,9cyclictest17573-21rm21:45:132
647799245,10cyclictest21-21ksoftirqd/123:15:591
647799245,10cyclictest21-21ksoftirqd/123:15:591
647799244,4cyclictest131rcu_preempt00:26:231
647799244,4cyclictest131rcu_preempt00:26:231
6476992416,4cyclictest131rcu_preempt19:46:140
6476992416,4cyclictest131rcu_preempt19:46:140
6476992410,7cyclictest12-21ksoftirqd/022:08:180
6476992410,7cyclictest12-21ksoftirqd/022:08:180
37282240,3sleep10-21swapper/123:47:041
37282240,3sleep10-21swapper/123:47:041
103862240,2sleep30-21swapper/322:11:213
103862240,2sleep30-21swapper/322:11:213
647899233,5cyclictest27-21ksoftirqd/222:06:202
647899233,5cyclictest27-21ksoftirqd/222:06:202
647899232,2cyclictest131rcu_preempt00:06:242
647899232,2cyclictest131rcu_preempt00:06:242
6478992312,9cyclictest29433-21ssh21:56:522
6478992312,9cyclictest29433-21ssh21:56:522
6478992312,9cyclictest14271-21/usr/sbin/munin22:16:092
6478992312,9cyclictest14271-21/usr/sbin/munin22:16:092
647899231,19cyclictest10244-21find20:51:112
647899231,19cyclictest10244-21find20:51:112
647899231,18cyclictest10954-21ls19:26:112
647799238,11cyclictest131rcu_preempt19:31:121
647799238,11cyclictest131rcu_preempt19:31:121
647799232,3cyclictest131rcu_preempt21:06:141
6477992317,4cyclictest0-21swapper/122:41:191
6477992317,4cyclictest0-21swapper/122:41:191
6476992318,3cyclictest29337-1kworker/0:0H20:41:140
6476992318,3cyclictest29337-1kworker/0:0H20:41:140
647699231,19cyclictest4198-21find21:31:120
647699231,19cyclictest4198-21find21:31:120
647699231,17cyclictest3037-21cut20:31:120
218012231,2sleep20-21swapper/221:15:232
218012231,2sleep20-21swapper/221:15:232
201912230,3sleep30-21swapper/321:12:543
201912230,3sleep30-21swapper/321:12:543
169372230,3sleep10-21swapper/122:18:191
169372230,3sleep10-21swapper/122:18:191
647899228,3cyclictest131rcu_preempt20:11:132
647899228,11cyclictest27-21ksoftirqd/219:41:142
647899228,11cyclictest27-21ksoftirqd/219:41:142
647899222,4cyclictest131rcu_preempt19:36:102
6478992215,5cyclictest22378-1kworker/u9:100:16:122
6478992215,5cyclictest22378-1kworker/u9:100:16:122
647899221,16cyclictest9938-21find00:31:102
647899221,16cyclictest9938-21find00:31:102
647899221,16cyclictest22826-21ls21:50:582
647899221,16cyclictest22826-21ls21:50:582
647899221,12cyclictest5264-21grep22:41:012
647899221,12cyclictest5264-21grep22:41:012
647799227,4cyclictest131rcu_preempt22:01:131
647799225,9cyclictest21-21ksoftirqd/121:46:491
647799225,9cyclictest21-21ksoftirqd/121:46:491
647799224,6cyclictest21-21ksoftirqd/121:01:121
647799224,6cyclictest21-21ksoftirqd/121:01:121
647799222,3cyclictest131rcu_preempt23:41:121
647799222,3cyclictest131rcu_preempt23:41:121
647799222,16cyclictest4783-21ls21:31:171
647799222,16cyclictest4783-21ls21:31:171
647799221,18cyclictest30546-21ls20:16:221
647799221,18cyclictest30546-21ls20:16:221
647699222,12cyclictest10445-21sh23:55:060
647699222,12cyclictest10445-21sh23:55:060
6476992216,4cyclictest12-21ksoftirqd/000:36:130
6476992216,4cyclictest12-21ksoftirqd/000:36:130
6476992215,5cyclictest12-21ksoftirqd/000:26:230
6476992215,5cyclictest12-21ksoftirqd/000:26:230
6476992215,4cyclictest12-21ksoftirqd/020:11:120
6476992213,6cyclictest1952-21cpuspeed23:46:090
6476992213,6cyclictest1952-21cpuspeed23:46:090
6476992212,8cyclictest31221-21ssh21:25:280
6476992212,8cyclictest31221-21ssh21:25:280
6476992212,8cyclictest29200-21fschecks_time23:06:130
6476992212,8cyclictest29200-21fschecks_time23:06:130
647699221,18cyclictest16183-21ls22:51:230
647699221,18cyclictest16183-21ls22:51:230
647699221,11cyclictest135-21systemd-journal20:40:570
647699221,11cyclictest135-21systemd-journal20:40:570
278742220,2sleep3321rcuc/322:30:343
647899212,3cyclictest0-21swapper/223:46:112
647899212,3cyclictest0-21swapper/223:46:112
647899211,16cyclictest24820-21switchtime22:26:212
647899210,18cyclictest25089-21ls20:01:242
647899210,18cyclictest25089-21ls20:01:242
647899210,16cyclictest6954-21ls23:16:232
647899210,16cyclictest6954-21ls23:16:232
647799214,3cyclictest131rcu_preempt00:25:431
647799214,3cyclictest131rcu_preempt00:25:431
647799213,4cyclictest131rcu_preempt20:51:231
647799213,4cyclictest131rcu_preempt20:51:231
6477992115,4cyclictest21-21ksoftirqd/122:00:591
6477992115,4cyclictest21-21ksoftirqd/122:00:591
6477992114,4cyclictest21-21ksoftirqd/120:33:381
6477992111,7cyclictest22819-21ps19:56:191
647799211,17cyclictest14022-21ls21:41:131
647799211,17cyclictest14022-21ls21:41:131
647799211,14cyclictest32010-21ls20:21:131
647799211,14cyclictest32010-21ls20:21:131
6477992110,3cyclictest188432ssh23:29:381
6477992110,3cyclictest188432ssh23:29:381
647699212,11cyclictest732-21runrttasks23:04:040
647699212,11cyclictest732-21runrttasks23:04:040
6476992113,6cyclictest22988-21ssh22:59:340
6476992113,6cyclictest22988-21ssh22:59:340
647699211,18cyclictest23507-21ls22:25:590
647699211,14cyclictest7416-21ls19:16:110
647699211,14cyclictest7416-21ls19:16:110
55882214,12sleep30-21swapper/319:11:143
220812210,3sleep10-21swapper/123:32:261
220812210,3sleep10-21swapper/123:32:261
647899203,8cyclictest27-21ksoftirqd/223:56:112
6478992013,5cyclictest774-21ssh23:44:452
6478992013,5cyclictest774-21ssh23:44:452
6478992012,6cyclictest32017-21rm23:09:062
6478992012,6cyclictest32017-21rm23:09:062
6478992012,6cyclictest20667-21ssh22:21:432
6478992012,5cyclictest17533-21diskmemload23:01:172
6478992012,5cyclictest17533-21diskmemload23:01:172
6478992011,6cyclictest2691-21sh00:21:212
6478992011,6cyclictest2691-21sh00:21:212
647799208,3cyclictest21-21ksoftirqd/123:10:201
647799208,3cyclictest21-21ksoftirqd/123:10:201
647799207,3cyclictest21-21ksoftirqd/123:04:331
647799207,3cyclictest21-21ksoftirqd/123:04:331
647799203,2cyclictest131rcu_preempt19:16:131
647799203,2cyclictest131rcu_preempt19:16:131
647799201,4cyclictest131rcu_preempt00:08:551
647799201,4cyclictest131rcu_preempt00:08:551
6477992014,4cyclictest0-21swapper/122:21:261
6477992013,4cyclictest21-21ksoftirqd/120:36:121
6477992013,4cyclictest21-21ksoftirqd/120:36:121
647799201,17cyclictest10037-21ls19:21:231
647799201,17cyclictest10037-21ls19:21:231
647799201,11cyclictest21038-21rt-features19:51:161
647799201,11cyclictest21038-21rt-features19:51:161
647799201,10cyclictest27501-21latency_hist20:10:591
647699208,4cyclictest6367-21ssh23:16:150
647699208,4cyclictest6367-21ssh23:16:150
647699202,9cyclictest19241-21users21:11:230
647699202,9cyclictest19241-21users21:11:230
647699202,10cyclictest25540-21grep00:11:120
647699202,10cyclictest25540-21grep00:11:120
647699201,16cyclictest16836-21idleruntime19:41:140
647699201,16cyclictest16836-21idleruntime19:41:140
647699201,10cyclictest24893-21/usr/sbin/munin23:36:100
6476992010,7cyclictest29715-21fschecks_count20:16:120
6476992010,7cyclictest29715-21fschecks_count20:16:120
40842200,2sleep30-21swapper/323:13:513
40842200,2sleep30-21swapper/323:13:513
647899191,5cyclictest26490-21ntpq20:06:162
647899191,4cyclictest131rcu_preempt20:56:132
647899191,4cyclictest131rcu_preempt20:56:132
6478991914,3cyclictest14620-1kworker/2:2H21:31:152
6478991914,3cyclictest14620-1kworker/2:2H21:31:152
647899191,3cyclictest0-21swapper/220:16:122
647899191,3cyclictest0-21swapper/220:16:122
647899191,13cyclictest25538-21ssh00:11:122
647899191,13cyclictest25538-21ssh00:11:122
6478991910,7cyclictest23133-21netstat21:16:162
6478991910,7cyclictest23133-21netstat21:16:162
6478991910,6cyclictest0-21swapper/221:06:042
6478991910,6cyclictest0-21swapper/221:06:042
647799196,4cyclictest21-21ksoftirqd/100:12:081
647799196,4cyclictest21-21ksoftirqd/100:12:081
647799196,3cyclictest21-21ksoftirqd/119:26:111
647799192,2cyclictest131rcu_preempt19:46:211
647799192,2cyclictest131rcu_preempt19:46:211
647799192,2cyclictest131rcu_preempt19:41:221
647799192,2cyclictest131rcu_preempt19:41:221
647799192,2cyclictest131rcu_preempt00:35:471
647799192,2cyclictest131rcu_preempt00:35:471
6477991914,3cyclictest21-21ksoftirqd/122:51:161
6477991914,3cyclictest21-21ksoftirqd/122:51:161
647799191,2cyclictest131rcu_preempt20:46:011
647799191,2cyclictest131rcu_preempt20:46:011
647799191,15cyclictest11943-21needreboot23:56:141
6477991910,2cyclictest73-21kswapd020:11:111
647699192,3cyclictest131rcu_preempt20:21:100
647699192,3cyclictest131rcu_preempt20:21:100
647699191,8cyclictest25870-21/usr/sbin/munin20:06:210
6476991912,5cyclictest12-21ksoftirqd/022:31:110
6476991912,5cyclictest12-21ksoftirqd/022:31:110
6476991912,4cyclictest12-21ksoftirqd/019:56:130
647699191,10cyclictest28176-21irqstats21:56:140
647699191,10cyclictest28176-21irqstats21:56:140
6479991812,4cyclictest0-21swapper/323:06:243
6479991812,4cyclictest0-21swapper/323:06:243
647999181,13cyclictest0-21swapper/320:36:023
647999181,10cyclictest0-21swapper/321:36:103
647999181,10cyclictest0-21swapper/321:36:103
647899189,7cyclictest23309-21users19:56:232
647899186,3cyclictest131rcu_preempt19:51:092
647899186,3cyclictest131rcu_preempt19:51:092
647899181,15cyclictest8800-21latency_hist19:20:582
647899181,15cyclictest8800-21latency_hist19:20:582
6478991811,5cyclictest30596-21kworker/u8:423:14:122
6478991811,5cyclictest30596-21kworker/u8:423:14:122
647899181,13cyclictest8395-21sh23:52:012
647899181,13cyclictest8395-21sh23:52:012
647899181,12cyclictest4634-21sh22:06:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional