You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-22 - 16:03
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1.osadl.org (updated Mon Dec 22, 2025 12:45:04)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1937621730,2sleep10-21swapper/112:35:251
1937621730,2sleep10-21swapper/112:35:251
1937621730,2sleep10-21swapper/112:35:251
323652970,3sleep20-21swapper/211:41:572
323652970,3sleep20-21swapper/211:41:572
15982970,2chrt1597-21/usr/sbin/munin11:10:280
15982970,2chrt1597-21/usr/sbin/munin11:10:280
94742920,3chrt9473-21cat07:30:280
94742920,3chrt9473-21cat07:30:280
202492910,2chrt0-21swapper/209:16:502
202492910,2chrt0-21swapper/209:16:502
202492910,2chrt0-21swapper/209:16:502
32312890,3chrt0-21swapper/009:32:550
32312890,3chrt0-21swapper/009:32:550
183522860,3sleep00-21swapper/009:15:190
183522860,3sleep00-21swapper/009:15:190
183522860,3sleep00-21swapper/009:15:190
139842850,3chrt0-21swapper/010:16:520
139842850,3chrt0-21swapper/010:16:520
304512810,3chrt0-21swapper/008:30:110
304512810,3chrt0-21swapper/008:30:110
3952710,4sleep30-21swapper/311:42:383
3952710,4sleep30-21swapper/311:42:383
94452690,4sleep30-21swapper/311:51:183
94452690,4sleep30-21swapper/311:51:183
36162690,3sleep30-21swapper/308:45:123
36162690,3sleep30-21swapper/308:45:123
264482670,3chrt26443-21ssh10:30:180
264482670,3chrt26443-21ssh10:30:180
104092660,3chrt0-21swapper/011:20:170
104092660,3chrt0-21swapper/011:20:170
104092660,3chrt0-21swapper/011:20:170
269272640,3sleep30-21swapper/308:20:173
82272620,3sleep00-21swapper/011:50:230
82272620,3sleep00-21swapper/011:50:230
20612620,4sleep20-21swapper/212:18:102
20612620,4sleep20-21swapper/212:18:102
46302610,3sleep00-21swapper/010:07:270
46302610,3sleep00-21swapper/010:07:270
127572600,2sleep20-21swapper/207:40:222
93732590,3sleep30-21swapper/310:12:163
106422590,3sleep20-21swapper/210:14:042
61212570,4chrt0-21swapper/210:09:342
61212570,4chrt0-21swapper/210:09:342
241022550,2chrt24103-21/usr/sbin/munin11:00:262
8792540,3sleep00-21swapper/010:36:370
8792540,3sleep00-21swapper/010:36:370
287972530,3chrt0-21swapper/011:38:490
287972530,3chrt0-21swapper/011:38:490
200962510,2sleep020107-21sh12:02:170
200962510,2sleep020107-21sh12:02:170
171822480,2sleep10-21swapper/112:33:281
171822480,2sleep10-21swapper/112:33:281
101172470,3sleep010118-21fschecks_count09:40:190
101172470,3sleep010118-21fschecks_count09:40:190
175642460,2sleep30-21swapper/310:20:283
175642460,2sleep30-21swapper/310:20:283
142922450,4chrt0-21swapper/110:17:171
142922450,4chrt0-21swapper/110:17:171
48982440,3chrt0-21swapper/111:46:541
48982440,3chrt0-21swapper/111:46:541
40972440,2sleep24211-21sort10:40:212
40972440,2sleep24211-21sort10:40:212
232882440,4sleep00-21swapper/009:53:410
232882440,4sleep00-21swapper/009:53:410
128132440,3sleep30-21swapper/310:49:423
128132440,3sleep30-21swapper/310:49:423
27582420,3chrt2729-21ssh11:45:200
27582420,3chrt2729-21ssh11:45:200
103682420,3sleep00-21swapper/012:25:550
103682420,3sleep00-21swapper/012:25:550
103682420,3sleep00-21swapper/012:25:550
82662410,2sleep08295-21latency_hist07:30:090
82662410,2sleep08295-21latency_hist07:30:090
236482410,2sleep30-21swapper/312:06:573
236482410,2sleep30-21swapper/312:06:573
126772410,4sleep10-21swapper/109:10:061
126772410,4sleep10-21swapper/109:10:061
307052400,3sleep00-21swapper/011:07:230
307052400,3sleep00-21swapper/011:07:230
203032390,4sleep012-21ksoftirqd/011:30:180
203032390,4sleep012-21ksoftirqd/011:30:180
291312380,2sleep2261rcuc/212:12:392
231499381,7cyclictest23287-21cat12:40:050
231499381,7cyclictest23287-21cat12:40:050
231499381,7cyclictest23287-21cat12:40:050
146702380,3sleep10-21swapper/109:10:581
146702380,3sleep10-21swapper/109:10:581
311802370,3sleep00-21swapper/012:15:140
195212370,2sleep119535-21ssh12:01:301
195212370,2sleep119535-21ssh12:01:301
156052370,4chrt0-21swapper/309:45:243
156052370,4chrt0-21swapper/309:45:243
231499367,26cyclictest16723-21seq11:25:470
231499361,7cyclictest5628-21memory09:35:180
231499361,7cyclictest5628-21memory09:35:180
220452360,2sleep122123-21processes08:05:251
220452360,2sleep122123-21processes08:05:251
182522360,3sleep10-21swapper/109:48:271
182522360,3sleep10-21swapper/109:48:271
34862350,4sleep03484-21sort12:20:050
34862350,4sleep03484-21sort12:20:050
310492350,4sleep20-21swapper/208:30:212
231499350,8cyclictest13902-21latency_hist07:45:110
231499350,7cyclictest695-21runrttasks10:53:370
231499350,7cyclictest695-21runrttasks10:53:370
195022350,2sleep219533-21latency_hist09:50:102
195022350,2sleep219533-21latency_hist09:50:102
144202350,3sleep30-21swapper/307:45:223
144202350,3sleep30-21swapper/307:45:223
278632340,4sleep30-21swapper/309:25:193
231499342,6cyclictest10141-21cat09:00:260
231499341,9cyclictest30996-21sh09:28:140
231499341,8cyclictest19614-21ssh09:50:130
231499341,8cyclictest19614-21ssh09:50:130
231499341,8cyclictest11424-21/usr/sbin/munin11:20:270
231499341,8cyclictest11424-21/usr/sbin/munin11:20:270
231499341,4cyclictest8953-21cron09:00:050
231499341,4cyclictest8953-21cron09:00:050
231499340,5cyclictest18354-21chrt10:21:100
231499340,5cyclictest18354-21chrt10:21:100
94202330,2sleep19431-21ssh11:51:171
94202330,2sleep19431-21ssh11:51:171
234042330,3chrt0-21swapper/209:53:512
234042330,3chrt0-21swapper/209:53:512
231499332,9cyclictest18803-21unixbench_multi07:55:270
231499332,9cyclictest18803-21unixbench_multi07:55:270
231499331,9cyclictest26227-21ssh09:55:540
231499331,9cyclictest26227-21ssh09:55:540
231499331,9cyclictest26227-21ssh09:55:540
231499331,9cyclictest17810-21ssh09:15:110
231499331,9cyclictest17810-21ssh09:15:110
231499331,9cyclictest14593-21ssh11:56:300
231499331,9cyclictest14593-21ssh11:56:300
231499331,9cyclictest11486-21irqstats09:05:190
231499331,9cyclictest11486-21irqstats09:05:190
231499331,8cyclictest25324-21grep08:15:190
231499331,8cyclictest25324-21grep08:15:190
231499331,8cyclictest25324-21grep08:15:190
231499331,6cyclictest8702-21/usr/sbin/munin10:45:190
231499331,6cyclictest8702-21/usr/sbin/munin10:45:190
231499331,6cyclictest6357-21sh12:22:170
231499331,6cyclictest6357-21sh12:22:170
231499331,4cyclictest13975-21perl12:30:210
231499331,4cyclictest13975-21perl12:30:210
231499331,29cyclictest19332-21switchtime10:55:280
231499331,29cyclictest19332-21switchtime10:55:280
231499330,7cyclictest25831-21chrt10:29:500
231499330,7cyclictest25831-21chrt10:29:500
231499330,7cyclictest25831-21chrt10:29:500
231499330,5cyclictest30602-21wc11:40:220
231499330,5cyclictest30602-21wc11:40:220
231499330,4cyclictest26302sleep007:14:370
231499330,4cyclictest26302sleep007:14:370
37032320,2sleep13720-21ssh12:20:151
37032320,2sleep13720-21ssh12:20:151
231499329,21cyclictest695-21runrttasks10:40:290
231499329,21cyclictest695-21runrttasks10:40:290
231499323,4cyclictest19871-21gltestperf08:00:190
231499323,4cyclictest19871-21gltestperf08:00:190
231499322,8cyclictest31659-21sed08:30:270
231499321,8cyclictest6773-21/usr/sbin/munin10:10:190
231499321,8cyclictest30288-21processes10:00:250
231499321,8cyclictest30288-21processes10:00:250
231499321,8cyclictest30288-21processes10:00:250
231499321,8cyclictest22722-21taskset08:07:190
231499321,8cyclictest22722-21taskset08:07:190
231499321,7cyclictest7260-21date08:55:110
231499321,7cyclictest7260-21date08:55:110
231499321,4cyclictest5741-21grep07:20:270
231499321,4cyclictest5741-21grep07:20:270
231499321,4cyclictest25521-21ssh09:22:230
231499318,21cyclictest116082chrt07:36:420
231499318,21cyclictest116082chrt07:36:420
231499318,21cyclictest116082chrt07:36:420
231499312,7cyclictest2901-21/usr/sbin/munin07:15:290
231499312,7cyclictest2901-21/usr/sbin/munin07:15:290
231499311,8cyclictest23870-21ntpq08:10:240
231499311,8cyclictest23870-21ntpq08:10:240
231499311,8cyclictest23412-21fschecks_time11:00:180
231499311,8cyclictest2168-21sed08:40:200
231499311,8cyclictest2168-21sed08:40:200
231499311,7cyclictest611-21awk08:35:250
231499311,7cyclictest611-21awk08:35:250
231499311,7cyclictest611-21awk08:35:250
231499311,7cyclictest16562-21taskset07:50:250
231499311,7cyclictest16562-21taskset07:50:250
231499311,6cyclictest23244-21sh12:06:240
231499311,6cyclictest23244-21sh12:06:240
231499311,4cyclictest27584-21ntp_states08:20:240
231499307,20cyclictest13986-21awk07:45:180
231499307,20cyclictest13986-21awk07:45:180
231499301,6cyclictest4345-21ntpq08:45:230
231499301,6cyclictest4345-21ntpq08:45:230
231499301,6cyclictest4345-21ntpq08:45:230
185502300,2sleep218721-21/usr/sbin/munin10:55:202
185502300,2sleep218721-21/usr/sbin/munin10:55:202
166902290,2sleep30-21swapper/310:53:153
166902290,2sleep30-21swapper/310:53:153
16442285,15sleep10-21swapper/107:10:251
16442285,15sleep10-21swapper/107:10:251
2316992516,7cyclictest23412-21ssh11:33:152
2316992516,7cyclictest23412-21ssh11:33:152
315712241,3sleep20-21swapper/210:01:392
315712241,3sleep20-21swapper/210:01:392
315712241,3sleep20-21swapper/210:01:392
2316992419,3cyclictest3181-21ssh11:12:242
2316992419,3cyclictest3181-21ssh11:12:242
294072230,3sleep316503-21kworker/3:111:39:413
294072230,3sleep316503-21kworker/3:111:39:413
25082230,2sleep227-21ksoftirqd/211:45:142
25082230,2sleep227-21ksoftirqd/211:45:142
2315992316,5cyclictest21474-21ssh09:18:331
2315992316,5cyclictest21474-21ssh09:18:331
2315992316,5cyclictest21474-21ssh09:18:331
314922220,3sleep30-21swapper/310:01:313
314922220,3sleep30-21swapper/310:01:313
314922220,3sleep30-21swapper/310:01:313
246312220,3chrt0-21swapper/111:34:571
246312220,3chrt0-21swapper/111:34:571
2316992212,8cyclictest17027-21ssh11:26:132
231599229,3cyclictest21-21ksoftirqd/109:55:291
231599229,3cyclictest21-21ksoftirqd/109:55:291
231599229,3cyclictest21-21ksoftirqd/109:55:291
2315992215,5cyclictest5916-21sh10:09:171
2315992215,5cyclictest5916-21sh10:09:171
2315992213,7cyclictest12657-21diskmemload10:28:411
2315992213,7cyclictest12657-21diskmemload10:28:411
2315992213,7cyclictest12657-21diskmemload10:28:411
169522220,2sleep30-21swapper/311:59:463
169522220,2sleep30-21swapper/311:59:463
10112221,16sleep30-21swapper/307:10:183
10112221,16sleep30-21swapper/307:10:183
231699218,5cyclictest27-21ksoftirqd/210:21:362
231699218,5cyclictest27-21ksoftirqd/210:21:362
2316992114,5cyclictest27863-21ssh09:58:142
2316992114,5cyclictest27863-21ssh09:58:142
2316992114,5cyclictest27863-21ssh09:58:142
2316992114,5cyclictest27-21ksoftirqd/210:29:042
2316992114,5cyclictest27-21ksoftirqd/210:29:042
2316992114,5cyclictest27-21ksoftirqd/210:29:042
2316992114,4cyclictest27-21ksoftirqd/210:20:172
2316992113,6cyclictest12657-21diskmemload12:23:552
2316992113,6cyclictest12657-21diskmemload12:23:552
231699211,18cyclictest9370-21grep09:00:182
231699211,17cyclictest13393-21ls12:30:052
231699211,17cyclictest13393-21ls12:30:052
231699211,17cyclictest13393-21ls12:30:052
231599215,3cyclictest131rcu_preempt10:35:211
231599215,3cyclictest131rcu_preempt10:35:211
231599213,9cyclictest131rcu_preempt12:07:251
231599213,9cyclictest131rcu_preempt12:07:251
231599211,4cyclictest131rcu_preempt12:28:041
231599211,4cyclictest131rcu_preempt12:28:041
231599211,4cyclictest131rcu_preempt12:28:041
2315992114,5cyclictest17502-21ssh11:26:521
2315992111,5cyclictest12711-1kworker/u9:009:31:101
2315992111,5cyclictest12711-1kworker/u9:009:31:101
22722214,10sleep20-21swapper/207:10:382
22722214,10sleep20-21swapper/207:10:382
73182200,3chrt0-21swapper/112:23:351
73182200,3chrt0-21swapper/112:23:351
231799201,16cyclictest0-21swapper/311:00:173
231799201,13cyclictest0-21swapper/308:55:183
231799201,13cyclictest0-21swapper/308:55:183
231699209,3cyclictest27-21ksoftirqd/209:30:272
231699209,3cyclictest27-21ksoftirqd/209:30:272
231699201,9cyclictest7611-21cat11:50:172
231699201,9cyclictest7611-21cat11:50:172
2316992012,6cyclictest12657-21diskmemload12:02:162
2316992012,6cyclictest12657-21diskmemload12:02:162
2316992012,3cyclictest0-21swapper/211:18:182
2316992012,3cyclictest0-21swapper/211:18:182
2316992012,3cyclictest0-21swapper/211:18:182
2316992011,7cyclictest5805-21ntpq09:35:212
2316992011,7cyclictest5805-21ntpq09:35:212
2316992011,7cyclictest4887-21users08:45:282
2316992011,7cyclictest4887-21users08:45:282
2316992011,7cyclictest4887-21users08:45:282
231699201,10cyclictest695-21runrttasks08:13:322
231699201,10cyclictest695-21runrttasks08:13:322
231599209,9cyclictest10987-21sh10:14:341
231599208,10cyclictest17351-21ssh10:54:121
231599208,10cyclictest17351-21ssh10:54:121
231599204,4cyclictest131rcu_preempt10:59:451
231599204,4cyclictest131rcu_preempt10:59:451
231599201,3cyclictest0-21swapper/109:40:061
231599201,3cyclictest0-21swapper/109:40:061
2315992013,5cyclictest9373-21processes10:45:261
2315992013,5cyclictest9373-21processes10:45:261
2315992013,5cyclictest2800-21/usr/sbin/munin08:40:271
2315992013,5cyclictest2800-21/usr/sbin/munin08:40:271
231599201,2cyclictest131rcu_preempt10:00:211
231599201,2cyclictest131rcu_preempt10:00:211
231599201,2cyclictest131rcu_preempt10:00:211
231599201,17cyclictest15940-21cut07:50:181
231599201,17cyclictest15940-21cut07:50:181
231599201,14cyclictest5980-21missed_timers11:15:221
231599201,14cyclictest5980-21missed_timers11:15:221
231599201,14cyclictest5980-21missed_timers11:15:221
231599201,12cyclictest21218-21ls08:05:091
231599201,12cyclictest21218-21ls08:05:091
231799191,14cyclictest0-21swapper/309:25:093
231799191,13cyclictest0-21swapper/309:55:173
231799191,13cyclictest0-21swapper/309:55:173
231799191,13cyclictest0-21swapper/309:55:173
231799191,13cyclictest0-21swapper/308:35:213
231799191,13cyclictest0-21swapper/308:35:213
231799191,13cyclictest0-21swapper/308:35:213
231799191,13cyclictest0-21swapper/308:15:173
231799191,13cyclictest0-21swapper/308:15:173
231799191,13cyclictest0-21swapper/308:15:173
231799191,12cyclictest0-21swapper/310:40:193
231799191,12cyclictest0-21swapper/310:40:193
231699199,8cyclictest30068-21seq10:33:552
231699199,8cyclictest30068-21seq10:33:552
231699191,9cyclictest6699-21grep08:50:292
231699191,9cyclictest6699-21grep08:50:292
231699191,12cyclictest11662-21find10:15:162
231599192,4cyclictest131rcu_preempt08:00:051
231599192,4cyclictest131rcu_preempt08:00:051
231599192,2cyclictest131rcu_preempt11:44:421
231599192,2cyclictest131rcu_preempt11:44:421
231599192,2cyclictest131rcu_preempt11:05:211
231599192,2cyclictest131rcu_preempt11:05:211
231599191,3cyclictest131rcu_preempt11:55:241
231599191,3cyclictest131rcu_preempt11:55:241
231599191,2cyclictest131rcu_preempt09:44:271
231599191,2cyclictest131rcu_preempt09:44:271
231599191,2cyclictest131rcu_preempt08:10:241
231599191,2cyclictest131rcu_preempt08:10:241
231599191,12cyclictest9023-21ls07:30:221
231599191,12cyclictest9023-21ls07:30:221
229092190,2sleep30-21swapper/311:32:343
229092190,2sleep30-21swapper/311:32:343
231799181,14cyclictest0-21swapper/311:20:143
231799181,14cyclictest0-21swapper/311:20:143
231799181,14cyclictest0-21swapper/311:20:143
231799181,12cyclictest0-21swapper/312:05:053
231799181,12cyclictest0-21swapper/312:05:053
231799181,11cyclictest0-21swapper/312:35:113
231799181,11cyclictest0-21swapper/312:35:113
231799181,11cyclictest0-21swapper/309:10:093
231799181,11cyclictest0-21swapper/309:10:093
231699189,7cyclictest27577-21ntp_states08:20:242
231699186,10cyclictest28034-21sort11:05:172
231699186,10cyclictest28034-21sort11:05:172
231699186,10cyclictest15075-21grep07:45:282
231699186,10cyclictest15075-21grep07:45:282
231699186,10cyclictest12144-21sed09:05:272
231699186,10cyclictest12144-21sed09:05:272
231699184,12cyclictest32404-21find08:35:192
231699184,12cyclictest32404-21find08:35:192
231699184,12cyclictest32404-21find08:35:192
231699181,9cyclictest8937-21latency07:30:212
231699181,9cyclictest8937-21latency07:30:212
231699181,9cyclictest695-21runrttasks08:16:272
231699181,9cyclictest695-21runrttasks08:16:272
231699181,9cyclictest695-21runrttasks08:16:272
231699181,9cyclictest695-21runrttasks08:04:282
231699181,9cyclictest695-21runrttasks08:04:282
231699181,5cyclictest0-21swapper/209:29:592
2316991813,3cyclictest27-21ksoftirqd/210:45:252
2316991813,3cyclictest27-21ksoftirqd/210:45:252
2316991812,3cyclictest27-21ksoftirqd/212:31:182
2316991812,3cyclictest27-21ksoftirqd/212:31:182
231699181,15cyclictest6809-21fschecks_time07:25:182
231699181,15cyclictest6809-21fschecks_time07:25:182
2316991811,5cyclictest10490-21kworker/u8:209:12:082
2316991811,5cyclictest10490-21kworker/u8:209:12:082
231699181,14cyclictest13871-21irqstats10:50:202
231699181,14cyclictest13871-21irqstats10:50:202
231699181,11cyclictest12616-21ls11:55:172
231699181,11cyclictest12616-21ls11:55:172
2316991810,6cyclictest24118-21sshd12:07:362
2316991810,6cyclictest24118-21sshd12:07:362
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional