You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-09 - 05:57
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1.osadl.org (updated Fri May 09, 2025 00:44:48)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
213726630,2sleep012-21ksoftirqd/019:11:390
213726630,2sleep012-21ksoftirqd/019:11:390
2622021540,2sleep10-21swapper/120:16:341
2622021540,2sleep10-21swapper/120:16:341
2947721420,3sleep00-21swapper/023:57:040
2947721420,3sleep00-21swapper/023:57:040
1830321340,4sleep00-21swapper/023:45:230
1830321340,4sleep00-21swapper/023:45:230
1842421270,3sleep00-21swapper/000:23:570
1842421270,3sleep00-21swapper/000:23:570
2601221060,2sleep10-21swapper/100:34:031
2098221030,3sleep10-21swapper/122:33:021
542321010,3sleep121-21ksoftirqd/100:06:541
542321010,3sleep121-21ksoftirqd/100:06:541
184821010,2chrt0-21swapper/222:11:282
68292960,4chrt0-21swapper/300:09:413
68292960,4chrt0-21swapper/300:09:413
207712940,3sleep10-21swapper/120:01:351
146812930,3sleep30-21swapper/321:11:333
146812930,3sleep30-21swapper/321:11:333
124952930,3chrt0-21swapper/000:16:380
124952930,3chrt0-21swapper/000:16:380
98712890,2sleep30-21swapper/321:01:123
98712890,2sleep30-21swapper/321:01:123
203242800,4sleep30-21swapper/322:31:553
98302790,3sleep00-21swapper/000:13:080
98302790,3sleep00-21swapper/000:13:080
229552790,3sleep10-21swapper/123:12:321
229552790,3sleep10-21swapper/123:12:321
189302790,3sleep00-21swapper/021:53:220
189302790,3sleep00-21swapper/021:53:220
19592770,2sleep30-21swapper/322:11:293
234699655,48cyclictest29605-21ssh22:06:262
234699655,48cyclictest29605-21ssh22:06:262
185232650,3sleep30-21swapper/323:45:463
185232650,3sleep30-21swapper/323:45:463
121452640,3chrt0-21swapper/319:41:153
82242620,2chrt0-21swapper/021:41:280
82242620,2chrt0-21swapper/021:41:280
37542600,4chrt0-21swapper/323:27:413
37542600,4chrt0-21swapper/323:27:413
2344995836,18cyclictest38650irq/55-eth019:21:230
2344995836,18cyclictest38650irq/55-eth019:21:230
2344995826,23cyclictest38650irq/55-eth000:06:250
2344995826,23cyclictest38650irq/55-eth000:06:250
9282570,3sleep30-21swapper/323:25:203
9282570,3sleep30-21swapper/323:25:203
177002570,3chrt0-21swapper/300:22:353
177002570,3chrt0-21swapper/300:22:353
54612560,3chrt0-21swapper/322:52:373
54612560,3chrt0-21swapper/322:52:373
259372560,3chrt0-21swapper/322:01:333
259372560,3chrt0-21swapper/322:01:333
173012540,3sleep30-21swapper/323:06:313
173012540,3sleep30-21swapper/323:06:313
101812530,4sleep333-21ksoftirqd/323:36:173
101812530,4sleep333-21ksoftirqd/323:36:173
95262520,3sleep30-21swapper/300:12:193
95262520,3sleep30-21swapper/300:12:193
242552520,2sleep024377-21/usr/sbin/munin22:36:390
242552520,2sleep024377-21/usr/sbin/munin22:36:390
234699527,7cyclictest131rcu_preempt22:21:222
234699527,7cyclictest131rcu_preempt22:21:222
216342520,3sleep30-21swapper/323:11:303
216342520,3sleep30-21swapper/323:11:303
234499514,11cyclictest131rcu_preempt23:51:240
234499514,11cyclictest131rcu_preempt23:51:240
19292510,2sleep00-21swapper/023:26:310
19292510,2sleep00-21swapper/023:26:310
63872500,3sleep30-21swapper/322:16:323
63872500,3sleep30-21swapper/322:16:323
2347995013,30cyclictest0-21swapper/320:51:233
2347995013,30cyclictest0-21swapper/320:51:233
207822490,3sleep00-21swapper/022:32:400
119492490,3chrt0-21swapper/323:37:103
119492490,3chrt0-21swapper/323:37:103
295682480,4sleep30-21swapper/322:43:043
295682480,4sleep30-21swapper/322:43:043
234699481,40cyclictest5803-21sh22:16:262
234699481,40cyclictest5803-21sh22:16:262
234499488,29cyclictest38650irq/55-eth021:31:250
234499488,29cyclictest38650irq/55-eth021:31:250
234699474,35cyclictest15902-21ssh00:21:252
234699474,35cyclictest15902-21ssh00:21:252
234699471,38cyclictest7954-21ssh22:56:252
234699471,38cyclictest7954-21ssh22:56:252
2344994713,4cyclictest12937-21find21:06:370
2344994713,4cyclictest12937-21find21:06:370
95622460,3sleep20-21swapper/223:35:242
95622460,3sleep20-21swapper/223:35:242
80762460,4sleep30-21swapper/321:41:273
80762460,4sleep30-21swapper/321:41:273
322142460,3sleep30-21swapper/322:08:523
322142460,3sleep30-21swapper/322:08:523
234499461,39cyclictest1469-21ssh22:11:240
234499461,12cyclictest27497-21sh22:41:260
234499461,12cyclictest27497-21sh22:41:260
179222460,3sleep10-21swapper/123:44:431
179222460,3sleep10-21swapper/123:44:431
267772450,3sleep30-21swapper/323:54:463
267772450,3sleep30-21swapper/323:54:463
247362450,4sleep00-21swapper/021:22:230
247362450,4sleep00-21swapper/021:22:230
2344994514,11cyclictest19514-21find21:16:360
2344994514,11cyclictest19514-21find21:16:360
151342450,2sleep215133-21ssh22:26:322
151342450,2sleep215133-21ssh22:26:322
91722440,2sleep00-21swapper/023:34:470
91722440,2sleep00-21swapper/023:34:470
234599446,36cyclictest32322-21ssh00:01:271
234599446,36cyclictest32322-21ssh00:01:271
2344994413,9cyclictest11340-21find19:36:370
183302440,3sleep00-21swapper/023:07:050
183302440,3sleep00-21swapper/023:07:050
75262430,3chrt0-21swapper/123:31:571
75262430,3chrt0-21swapper/123:31:571
3842430,2sleep30-21swapper/322:46:383
3842430,2sleep30-21swapper/322:46:383
2344994324,15cyclictest2986-21apt-get19:16:260
13262430,3sleep10-21swapper/123:26:031
13262430,3sleep10-21swapper/123:26:031
43432420,4sleep03584-21/usr/sbin/munin21:36:340
302232420,3sleep00-21swapper/020:26:370
234699423,3cyclictest131rcu_preempt21:26:292
234699423,3cyclictest131rcu_preempt21:26:292
234599421,36cyclictest31739-21ssh21:31:261
234599421,36cyclictest31739-21ssh21:31:261
234599421,36cyclictest12311-21sshd19:41:231
234599421,36cyclictest12311-21sshd19:41:231
223622420,3chrt0-21swapper/100:29:001
223622420,3chrt0-21swapper/100:29:001
124382420,3chrt73-21kswapd023:01:271
124382420,3chrt73-21kswapd023:01:271
325152410,4chrt0-21swapper/321:31:353
325152410,4chrt0-21swapper/321:31:353
289322410,2sleep1201rcuc/123:56:371
289322410,2sleep1201rcuc/123:56:371
234599414,33cyclictest16748-21ssh23:06:251
234599414,33cyclictest16748-21ssh23:06:251
234599411,34cyclictest7913-21ssh21:41:251
234599411,34cyclictest7913-21ssh21:41:251
234499419,8cyclictest13326-21find23:01:370
234499419,8cyclictest13326-21find23:01:370
234499418,17cyclictest15574-21find22:26:380
234499418,17cyclictest15574-21find22:26:380
2344994114,22cyclictest0-21swapper/019:31:260
2344994114,22cyclictest0-21swapper/019:31:260
234699406,29cyclictest3605-21ssh21:36:242
234499407,17cyclictest29454-21cat22:06:190
234499407,17cyclictest29454-21cat22:06:190
234499404,22cyclictest38650irq/55-eth021:46:250
2344994011,12cyclictest22724-21ssh23:12:070
2344994011,12cyclictest22724-21ssh23:12:070
106902400,3sleep30-21swapper/321:01:333
106902400,3sleep30-21swapper/321:01:333
310442390,3sleep10-21swapper/122:06:531
310442390,3sleep10-21swapper/122:06:531
234799395,13cyclictest0-21swapper/323:01:253
234799395,13cyclictest0-21swapper/323:01:253
234699392,26cyclictest25442-21ssh23:16:232
234699392,26cyclictest25442-21ssh23:16:232
234499399,12cyclictest28313-21ssh23:19:220
234499399,12cyclictest28313-21ssh23:19:220
2344993928,8cyclictest8251-21apt-get20:56:270
2344993928,8cyclictest8251-21apt-get20:56:270
2346993827,6cyclictest7913-21sh00:11:262
2346993827,6cyclictest7913-21sh00:11:262
234499389,8cyclictest312-21find22:46:380
234499389,8cyclictest312-21find22:46:380
2344993823,12cyclictest2637-21cat20:41:210
2344993813,10cyclictest9888-21latency_hist22:21:130
2344993813,10cyclictest9888-21latency_hist22:21:130
2344993810,11cyclictest22525-21ssh00:29:180
2344993810,11cyclictest22525-21ssh00:29:180
234599372,29cyclictest27484-21sh22:41:241
234599372,29cyclictest27484-21sh22:41:241
234499379,11cyclictest9990-21taskset22:58:000
234499379,11cyclictest9990-21taskset22:58:000
234499374,22cyclictest38650irq/55-eth000:01:270
234499374,22cyclictest38650irq/55-eth000:01:270
2344993723,10cyclictest0-21swapper/021:11:250
2344993723,10cyclictest0-21swapper/021:11:250
2344993711,12cyclictest30842-21find23:21:410
2344993711,12cyclictest30842-21find23:21:410
220712370,3chrt0-21swapper/123:49:261
220712370,3chrt0-21swapper/123:49:261
17852370,2chrt22295-21munin-node23:26:271
17852370,2chrt22295-21munin-node23:26:271
13552370,2sleep1201rcuc/119:11:271
13552370,2sleep1201rcuc/119:11:271
234699365,19cyclictest22295-21munin-node19:16:262
2344993610,11cyclictest29264-21sh21:27:400
2344993610,11cyclictest29264-21sh21:27:400
110062360,2sleep20-21swapper/223:36:332
110062360,2sleep20-21swapper/223:36:332
282682350,4sleep30-21swapper/323:19:183
282682350,4sleep30-21swapper/323:19:183
2346993515,10cyclictest131rcu_preempt23:11:252
2346993515,10cyclictest131rcu_preempt23:11:252
234499359,12cyclictest29447-21sh00:39:050
234499352,27cyclictest10386-21ssh23:36:260
234499352,27cyclictest10386-21ssh23:36:260
2344993520,10cyclictest0-21swapper/019:46:250
2344993510,9cyclictest21222-21chrt20:02:500
2344993510,10cyclictest5376-21ssh22:52:290
2344993510,10cyclictest5376-21ssh22:52:290
89212340,2sleep18930-21ssh22:19:321
89212340,2sleep18930-21ssh22:19:321
234499349,8cyclictest20162-21awk23:46:370
234499349,8cyclictest20162-21awk23:46:370
234499348,12cyclictest3136-21sed20:41:290
234499348,12cyclictest3136-21sed20:41:290
234799335,20cyclictest0-21swapper/319:41:233
234799335,20cyclictest0-21swapper/319:41:233
2346993315,13cyclictest2343-21cyclictest22:51:252
2346993315,13cyclictest2343-21cyclictest22:51:252
2346993314,14cyclictest27-21ksoftirqd/221:51:292
2346993314,14cyclictest27-21ksoftirqd/221:51:292
234499337,11cyclictest11373-21taskset21:02:180
234499337,11cyclictest11373-21taskset21:02:180
219392330,3sleep20-21swapper/223:49:112
219392330,3sleep20-21swapper/223:49:112
131842330,3sleep30-21swapper/322:24:323
131842330,3sleep30-21swapper/322:24:323
38112320,4sleep30-21swapper/319:16:363
234699325,20cyclictest27-21ksoftirqd/200:26:262
234699325,20cyclictest27-21ksoftirqd/200:26:262
2345993221,7cyclictest201rcuc/122:01:211
234499329,9cyclictest23597-21awk20:11:250
234499329,9cyclictest23597-21awk20:11:250
234499328,8cyclictest23672-21apt00:31:240
234499328,10cyclictest16095-21apt-get19:51:220
234499328,10cyclictest10914-21proc_pri22:21:330
234499328,10cyclictest10914-21proc_pri22:21:330
234499327,10cyclictest73-21kswapd020:36:250
234799317,16cyclictest0-21swapper/323:56:263
234799317,16cyclictest0-21swapper/323:56:263
234599311,22cyclictest18492-21ssh21:16:251
234599311,22cyclictest18492-21ssh21:16:251
234499319,7cyclictest13370-21cat19:41:360
234499319,7cyclictest13370-21cat19:41:360
234499317,8cyclictest7490-21awk20:51:350
234499317,8cyclictest7490-21awk20:51:350
234499317,8cyclictest26139-21ntpq20:16:320
234499317,8cyclictest26139-21ntpq20:16:320
234499317,10cyclictest4864-21basename20:46:300
234499317,10cyclictest4864-21basename20:46:300
234499317,10cyclictest27795-21latency20:21:270
234499317,10cyclictest27795-21latency20:21:270
2344993111,8cyclictest31561-21taskset22:07:470
2344993111,8cyclictest31561-21taskset22:07:470
2344993110,8cyclictest22369-21ssh21:56:520
2344993110,8cyclictest22369-21ssh21:56:520
214822310,3sleep30-21swapper/323:48:243
214822310,3sleep30-21swapper/323:48:243
39312300,3sleep173-21kswapd019:16:381
234799304,19cyclictest0-21swapper/320:41:213
234699302,22cyclictest12255-21ssh23:01:252
234699302,22cyclictest12255-21ssh23:01:252
234599303,18cyclictest63642sleep120:51:231
234599303,18cyclictest63642sleep120:51:231
257792290,3sleep30-21swapper/322:38:463
257792290,3sleep30-21swapper/322:38:463
234599299,9cyclictest21-21ksoftirqd/119:32:151
234599299,9cyclictest21-21ksoftirqd/119:32:151
234599291,6cyclictest21-21ksoftirqd/122:56:281
234599291,6cyclictest21-21ksoftirqd/122:56:281
234799285,16cyclictest0-21swapper/319:21:233
234799285,16cyclictest0-21swapper/319:21:233
2346992816,10cyclictest31775-21ssh23:22:442
2346992816,10cyclictest31775-21ssh23:22:442
234599284,16cyclictest15885-21ssh00:21:241
234599284,16cyclictest15885-21ssh00:21:241
234599283,5cyclictest21-21ksoftirqd/120:26:291
2344992811,13cyclictest0-21swapper/019:56:240
2344992811,13cyclictest0-21swapper/019:56:240
294542270,3sleep10-21swapper/123:21:121
294542270,3sleep10-21swapper/123:21:121
234699274,17cyclictest27-21ksoftirqd/219:56:242
234699274,17cyclictest27-21ksoftirqd/219:56:242
2346992718,4cyclictest0-21swapper/222:46:252
2346992718,4cyclictest0-21swapper/222:46:252
234599276,9cyclictest21-21ksoftirqd/121:30:441
234599276,9cyclictest21-21ksoftirqd/121:30:441
234599272,20cyclictest11724-21ssh00:16:281
234599272,20cyclictest11724-21ssh00:16:281
234599272,17cyclictest23573-21ssh23:51:251
234599272,17cyclictest23573-21ssh23:51:251
234599271,6cyclictest21-21ksoftirqd/119:26:281
234599271,6cyclictest21-21ksoftirqd/119:26:281
309292260,2sleep2261rcuc/222:45:242
309292260,2sleep2261rcuc/222:45:242
234799266,15cyclictest0-21swapper/322:01:213
234699263,19cyclictest27-21ksoftirqd/221:01:252
234699263,19cyclictest27-21ksoftirqd/221:01:252
2346992619,4cyclictest27-21ksoftirqd/221:46:282
234599266,8cyclictest21-21ksoftirqd/119:36:281
2345992619,4cyclictest201rcuc/100:36:511
234599261,5cyclictest21-21ksoftirqd/123:36:281
234599261,5cyclictest21-21ksoftirqd/123:36:281
2345992614,3cyclictest131rcu_preempt20:00:361
2345992614,3cyclictest131rcu_preempt20:00:361
234499264,7cyclictest0-21swapper/020:06:330
234499264,7cyclictest0-21swapper/020:06:330
251542250,4sleep30-21swapper/322:01:173
234599258,6cyclictest21-21ksoftirqd/121:39:101
234599257,4cyclictest21-21ksoftirqd/121:51:131
234599257,4cyclictest131rcu_preempt22:36:271
234599257,4cyclictest131rcu_preempt22:36:271
234599256,9cyclictest21-21ksoftirqd/122:30:291
234599256,9cyclictest21-21ksoftirqd/122:30:291
2344992513,9cyclictest73-21kswapd020:31:280
2344992513,9cyclictest73-21kswapd020:31:280
10332254,13sleep20-21swapper/219:11:212
10332254,13sleep20-21swapper/219:11:212
234799245,12cyclictest0-21swapper/321:36:263
234699242,17cyclictest25354-21chrt22:01:282
234699242,17cyclictest25354-21chrt22:01:282
2346992415,7cyclictest22833-21ssh21:57:432
2346992415,7cyclictest22833-21ssh21:57:432
2346992412,10cyclictest26649-21ssh23:54:342
2346992412,10cyclictest26649-21ssh23:54:342
2346992411,11cyclictest7600-21cron00:11:122
2346992411,11cyclictest7600-21cron00:11:122
234599249,7cyclictest21-21ksoftirqd/122:01:291
234599249,7cyclictest21-21ksoftirqd/122:01:291
234599247,9cyclictest21-21ksoftirqd/120:10:511
234599247,9cyclictest21-21ksoftirqd/120:10:511
234599247,7cyclictest21-21ksoftirqd/122:16:211
234599247,7cyclictest21-21ksoftirqd/120:35:111
234599247,7cyclictest21-21ksoftirqd/120:35:111
234599247,5cyclictest21-21ksoftirqd/120:24:231
234599247,5cyclictest21-21ksoftirqd/120:24:231
234599246,8cyclictest21-21ksoftirqd/122:25:101
234599246,8cyclictest21-21ksoftirqd/122:25:101
234599246,8cyclictest21-21ksoftirqd/119:53:291
234599244,9cyclictest21-21ksoftirqd/121:56:251
234599244,4cyclictest21-21ksoftirqd/120:46:291
234599244,4cyclictest21-21ksoftirqd/120:46:291
234599243,7cyclictest21-21ksoftirqd/122:46:281
234599243,7cyclictest21-21ksoftirqd/122:46:281
2345992418,4cyclictest201rcuc/100:16:071
2345992418,4cyclictest201rcuc/100:16:071
2345992412,9cyclictest25611-21ssh21:23:551
2345992412,9cyclictest25611-21ssh21:23:551
234499244,8cyclictest6693-21/usr/sbin/munin19:26:330
234499244,8cyclictest6693-21/usr/sbin/munin19:26:330
117262240,4sleep30-21swapper/300:16:283
117262240,4sleep30-21swapper/300:16:283
234699237,5cyclictest131rcu_preempt21:46:182
234699237,5cyclictest131rcu_preempt21:46:182
2346992312,4cyclictest0-21swapper/200:01:432
2346992312,4cyclictest0-21swapper/200:01:432
234699231,17cyclictest2095-21cut23:26:312
234699231,17cyclictest2095-21cut23:26:312
234599236,6cyclictest21-21ksoftirqd/121:13:021
234599236,6cyclictest21-21ksoftirqd/121:13:021
234599235,10cyclictest21-21ksoftirqd/120:41:231
234599235,10cyclictest21-21ksoftirqd/120:41:231
234599234,10cyclictest21-21ksoftirqd/120:36:341
234599234,10cyclictest21-21ksoftirqd/120:36:341
234599233,6cyclictest21-21ksoftirqd/121:51:371
234599233,6cyclictest21-21ksoftirqd/121:51:371
234599233,3cyclictest131rcu_preempt20:11:251
234599233,3cyclictest131rcu_preempt20:11:251
45312220,4sleep30-21swapper/320:46:263
45312220,4sleep30-21swapper/320:46:263
234799225,14cyclictest0-21swapper/321:51:273
234799225,14cyclictest0-21swapper/321:51:273
234799224,12cyclictest0-21swapper/322:56:253
234799224,12cyclictest0-21swapper/322:56:253
234699221,18cyclictest16180-21ls19:51:252
234699221,16cyclictest16732-21find23:06:252
234699221,16cyclictest16732-21find23:06:252
234599226,7cyclictest21-21ksoftirqd/119:48:351
234599223,9cyclictest4235-21strings22:51:311
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional