You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-22 - 07:30
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1.osadl.org (updated Sat Nov 22, 2025 00:44:59)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
833224140,3sleep30-21swapper/319:10:553
833224140,3sleep30-21swapper/319:10:553
1121221440,2sleep20-21swapper/223:54:062
312812960,5sleep30-21swapper/321:56:153
312812960,5sleep30-21swapper/321:56:153
168402860,3sleep30-21swapper/300:38:443
168402860,3sleep30-21swapper/300:38:443
187872850,3sleep30-21swapper/322:18:053
187872850,3sleep30-21swapper/322:18:053
202552830,4sleep30-21swapper/323:28:553
202552830,4sleep30-21swapper/323:28:553
141312820,2sleep20-21swapper/222:13:102
141312820,2sleep20-21swapper/222:13:102
926899794,59cyclictest15862-21ssh21:40:522
926899794,59cyclictest15862-21ssh21:40:522
926899794,59cyclictest15862-21ssh21:40:522
317332770,2chrt0-21swapper/100:16:071
317332770,2chrt0-21swapper/100:16:071
271412720,7sleep333-21ksoftirqd/320:01:023
30872690,3sleep30-21swapper/300:21:033
30872690,3sleep30-21swapper/300:21:033
124322670,3chrt12391-21ssh23:20:531
124322670,3chrt12391-21ssh23:20:531
124232670,3sleep011923-21/usr/sbin/munin22:46:000
124232670,3sleep011923-21/usr/sbin/munin22:46:000
8862640,3chrt0-21swapper/123:08:191
8862640,3chrt0-21swapper/123:08:191
219752630,4chrt0-21swapper/321:11:383
219752630,4chrt0-21swapper/321:11:383
90482620,3chrt9265-21cyclictest23:16:122
90482620,3chrt9265-21cyclictest23:16:122
248002620,3chrt0-21swapper/122:59:271
248002620,3chrt0-21swapper/122:59:271
61972610,3sleep00-21swapper/022:05:160
61972610,3sleep00-21swapper/022:05:160
47802590,3sleep00-21swapper/023:12:040
47802590,3sleep00-21swapper/023:12:040
926899564,42cyclictest10027-21ssh00:30:522
926899564,42cyclictest10027-21ssh00:30:522
926699551,2cyclictest0-21swapper/000:30:530
926699551,2cyclictest0-21swapper/000:30:530
926699542,12cyclictest0-21swapper/020:15:510
926699542,12cyclictest0-21swapper/020:15:510
311122540,4sleep30-21swapper/320:11:033
311122540,4sleep30-21swapper/320:11:033
9268995230,13cyclictest131rcu_preempt21:50:522
9268995230,13cyclictest131rcu_preempt21:50:522
926799524,9cyclictest131rcu_preempt20:15:511
926799524,9cyclictest131rcu_preempt20:15:511
926699523,21cyclictest12-21ksoftirqd/022:30:520
926699523,21cyclictest12-21ksoftirqd/022:30:520
926699521,3cyclictest0-21swapper/020:30:510
926699521,3cyclictest0-21swapper/020:30:510
68072520,4sleep00-21swapper/022:05:530
68072520,4sleep00-21swapper/022:05:530
926699511,2cyclictest0-21swapper/019:45:520
926699511,2cyclictest0-21swapper/019:45:520
926699502,41cyclictest25944-21apt-get23:00:530
926699502,41cyclictest25944-21apt-get23:00:530
269782500,2sleep226988-21ssh21:17:132
269782500,2sleep226988-21ssh21:17:132
15502500,3chrt0-21swapper/120:20:571
15502500,3chrt0-21swapper/120:20:571
128132500,2sleep20-21swapper/221:36:372
128132500,2sleep20-21swapper/221:36:372
9268994926,12cyclictest27-21ksoftirqd/221:25:532
9268994926,12cyclictest27-21ksoftirqd/221:25:532
926699495,37cyclictest16683-21ssh22:50:550
926699495,37cyclictest16683-21ssh22:50:550
926699491,43cyclictest0-21swapper/019:40:550
926699491,43cyclictest0-21swapper/019:40:550
323152490,3chrt0-21swapper/200:17:132
323152490,3chrt0-21swapper/200:17:132
268892490,2sleep00-21swapper/020:01:000
926799482,36cyclictest20352-21ssh21:10:521
926799482,36cyclictest20352-21ssh21:10:521
926699484,33cyclictest26095-21ssh00:10:530
926699484,33cyclictest26095-21ssh00:10:530
926699481,42cyclictest0-21swapper/021:00:520
926699481,42cyclictest0-21swapper/021:00:520
127142480,3sleep20-21swapper/220:50:522
127142480,3sleep20-21swapper/220:50:522
926699472,36cyclictest0-21swapper/022:10:520
926699472,36cyclictest0-21swapper/022:10:520
926699471,41cyclictest26409-21apt-get23:35:530
926699471,2cyclictest0-21swapper/021:25:540
926699471,2cyclictest0-21swapper/021:25:540
926699470,8cyclictest28268-21ssh21:53:470
926699470,8cyclictest28268-21ssh21:53:470
158042470,2sleep115914-21missed_timers19:30:591
926799463,6cyclictest131rcu_preempt23:40:541
926799463,6cyclictest131rcu_preempt23:40:541
926699460,3cyclictest30566-21apt-get23:05:510
926699460,3cyclictest30566-21apt-get23:05:510
92262460,3sleep30-21swapper/300:29:393
92262460,3sleep30-21swapper/300:29:393
120142460,3sleep10-21swapper/119:20:571
120142460,3sleep10-21swapper/119:20:571
926699454,35cyclictest0-21swapper/000:25:510
926699454,35cyclictest0-21swapper/000:25:510
926699452,38cyclictest0-21swapper/000:20:510
926699452,38cyclictest0-21swapper/000:20:510
926699451,39cyclictest0-21swapper/022:35:520
89502450,2sleep00-21swapper/019:11:050
89502450,2sleep00-21swapper/019:11:050
288522450,2sleep20-21swapper/222:29:022
165642450,3sleep30-21swapper/300:00:183
165642450,3sleep30-21swapper/300:00:183
926699442,3cyclictest14101-21apt-get00:35:530
926699442,3cyclictest14101-21apt-get00:35:530
926699442,38cyclictest0-21swapper/020:25:510
926699442,38cyclictest0-21swapper/020:25:510
926699441,39cyclictest0-21swapper/000:15:530
926699441,39cyclictest0-21swapper/000:15:530
232072440,3sleep10-21swapper/122:22:421
183602440,3chrt0-21swapper/019:36:030
926699431,4cyclictest1437-21apt-get20:20:550
926699431,4cyclictest1437-21apt-get20:20:550
926699431,4cyclictest0-21swapper/022:55:520
926699431,4cyclictest0-21swapper/022:55:520
926699431,2cyclictest0-21swapper/023:15:510
926699431,2cyclictest0-21swapper/023:15:510
325392430,3sleep20-21swapper/223:07:092
325392430,3sleep20-21swapper/223:07:092
225452430,11sleep012-21ksoftirqd/000:06:010
191122430,3sleep119111-21ssh00:01:561
191122430,3sleep119111-21ssh00:01:561
11632430,8sleep333-21ksoftirqd/300:19:393
11632430,8sleep333-21ksoftirqd/300:19:393
926899423,5cyclictest131rcu_preempt20:25:512
926899423,5cyclictest131rcu_preempt20:25:512
926899422,31cyclictest21741-21ssh23:30:512
926899422,31cyclictest21741-21ssh23:30:512
56222420,3sleep20-21swapper/200:25:122
56222420,3sleep20-21swapper/200:25:122
9268994125,8cyclictest131rcu_preempt20:55:532
9268994125,8cyclictest131rcu_preempt20:55:532
926699413,24cyclictest38850irq/55-eth023:50:510
926699412,35cyclictest0-21swapper/019:15:520
926699412,35cyclictest0-21swapper/019:15:520
926699411,36cyclictest0-21swapper/020:55:510
926699411,36cyclictest0-21swapper/020:55:510
926699411,10cyclictest0-21swapper/023:40:540
926699411,10cyclictest0-21swapper/023:40:540
154282410,3sleep30-21swapper/320:56:033
154282410,3sleep30-21swapper/320:56:033
9268994019,9cyclictest131rcu_preempt20:05:522
9268994019,9cyclictest131rcu_preempt20:05:522
926799404,5cyclictest131rcu_preempt22:30:521
926799404,5cyclictest131rcu_preempt22:30:521
926799402,33cyclictest29879-21ssh21:55:541
926799402,33cyclictest29879-21ssh21:55:541
926699404,24cyclictest38850irq/55-eth023:30:510
926699404,24cyclictest38850irq/55-eth023:30:510
926699401,36cyclictest0-21swapper/019:20:530
926699401,36cyclictest0-21swapper/019:20:530
926699401,35cyclictest0-21swapper/000:00:510
926699401,35cyclictest0-21swapper/000:00:510
926699401,2cyclictest0-21swapper/021:45:520
926699401,2cyclictest0-21swapper/021:45:520
248022400,3sleep10-21swapper/121:50:301
248022400,3sleep10-21swapper/121:50:301
926699391,3cyclictest25667-21apt-get22:25:520
926699391,34cyclictest0-21swapper/021:05:510
926699391,34cyclictest0-21swapper/021:05:510
926699391,33cyclictest0-21swapper/020:05:520
926699391,33cyclictest0-21swapper/020:05:520
164722390,2sleep116519-21ssh21:40:591
164722390,2sleep116519-21ssh21:40:591
164722390,2sleep116519-21ssh21:40:591
926799384,31cyclictest14122-21ssh00:35:541
926799384,31cyclictest14122-21ssh00:35:541
926799384,30cyclictest15840-21latency_hist21:40:501
926699381,33cyclictest0-21swapper/022:15:520
926699381,33cyclictest0-21swapper/022:15:520
80252380,2sleep27780-21ls19:10:502
80252380,2sleep27780-21ls19:10:502
174162380,2sleep217467-21ssh22:51:052
174162380,2sleep217467-21ssh22:51:052
926799378,18cyclictest21-21ksoftirqd/123:35:531
926799375,20cyclictest21860-21ssh00:05:531
926799373,31cyclictest25002-21ssh21:15:541
926799373,31cyclictest25002-21ssh21:15:541
926799373,28cyclictest7650-21ssh23:15:511
926799373,28cyclictest7650-21ssh23:15:511
9267993717,17cyclictest21-21ksoftirqd/119:35:511
926699374,23cyclictest38850irq/55-eth021:10:520
926699374,23cyclictest38850irq/55-eth021:10:520
926699371,33cyclictest19055-21sh21:43:540
926699371,33cyclictest19055-21sh21:43:540
926699371,33cyclictest19055-21sh21:43:540
926699371,2cyclictest0-21swapper/021:20:530
926699371,2cyclictest0-21swapper/021:20:530
926699370,7cyclictest11631-21chrt20:46:030
92212370,3sleep00-21swapper/020:40:530
92212370,3sleep00-21swapper/020:40:530
219452370,2sleep30-21swapper/319:46:033
219452370,2sleep30-21swapper/319:46:033
926799366,16cyclictest2493-21ssh22:35:521
926699362,31cyclictest11194-21ssh21:35:540
926699362,31cyclictest11194-21ssh21:35:540
926699362,2cyclictest7303-21apt-get22:40:520
926699362,2cyclictest7303-21apt-get22:40:520
926699361,32cyclictest18492-21rm23:26:220
926699361,32cyclictest18492-21rm23:26:220
926699361,2cyclictest0-21swapper/019:50:520
926699361,2cyclictest0-21swapper/019:50:520
136722360,3sleep30-21swapper/321:37:563
136722360,3sleep30-21swapper/321:37:563
926699353,29cyclictest12545-21apt-get23:55:500
926699353,29cyclictest12545-21apt-get23:55:500
926699351,4cyclictest29950-21cpu21:55:560
926699351,4cyclictest29950-21cpu21:55:560
293932350,2sleep229406-21ssh23:38:522
200092350,2sleep120051-21latency_hist21:10:411
200092350,2sleep120051-21latency_hist21:10:411
170742350,2sleep10-21swapper/123:25:551
170742350,2sleep10-21swapper/123:25:551
106192350,2sleep10-21swapper/122:09:561
106192350,2sleep10-21swapper/122:09:561
926899346,20cyclictest27-21ksoftirqd/221:45:512
926899346,20cyclictest27-21ksoftirqd/221:45:512
926699342,2cyclictest3160-21apt-get23:45:530
926699342,2cyclictest3160-21apt-get23:45:530
926699341,5cyclictest12511-21fschecks_count23:20:540
926699341,5cyclictest12511-21fschecks_count23:20:540
926699341,4cyclictest13563-21apt-get19:25:540
926699341,4cyclictest13563-21apt-get19:25:540
926699341,30cyclictest6505-21sort21:30:530
926699341,30cyclictest6505-21sort21:30:530
926699340,3cyclictest20851-21apt-get22:20:520
9268993316,11cyclictest6234-21kworker/2:022:05:512
9268993316,11cyclictest6234-21kworker/2:022:05:512
926699331,2cyclictest0-21swapper/020:54:470
926699331,2cyclictest0-21swapper/020:54:470
926699331,21cyclictest732-21runrttasks21:18:430
926699331,21cyclictest732-21runrttasks21:18:430
926699331,10cyclictest31713-21seq20:15:210
926699331,10cyclictest31713-21seq20:15:210
926699330,9cyclictest542-21snmpd19:55:580
926699330,9cyclictest542-21snmpd19:55:580
298772330,3sleep10-21swapper/123:05:101
298772330,3sleep10-21swapper/123:05:101
111992330,4chrt0-21swapper/323:19:303
111992330,4chrt0-21swapper/323:19:303
9267993212,17cyclictest21-21ksoftirqd/123:45:531
9267993212,17cyclictest21-21ksoftirqd/123:45:531
926699321,8cyclictest8001-21find20:36:060
926799312,24cyclictest12543-21ssh23:55:491
926799312,24cyclictest12543-21ssh23:55:491
926799312,22cyclictest29647-21ssh21:20:531
926799312,22cyclictest29647-21ssh21:20:531
926699311,2cyclictest0-21swapper/019:30:530
230832310,3chrt0-21swapper/221:13:222
230832310,3chrt0-21swapper/221:13:222
230192310,2sleep30-21swapper/321:47:513
230192310,2sleep30-21swapper/321:47:513
926899302,23cyclictest25969-21ssh23:00:552
926899302,23cyclictest25969-21ssh23:00:552
926899301,22cyclictest16569-21latency_hist22:50:492
926899301,22cyclictest16569-21latency_hist22:50:492
926899301,21cyclictest0-21swapper/220:20:552
926899301,21cyclictest0-21swapper/220:20:552
9267993015,10cyclictest9265-21cyclictest00:10:521
9267993015,10cyclictest9265-21cyclictest00:10:521
71942300,2sleep27238-21http23:15:412
71942300,2sleep27238-21http23:15:412
320442300,4sleep30-21swapper/320:15:523
320442300,4sleep30-21swapper/320:15:523
112442300,3chrt0-21swapper/123:54:091
97762290,3chrt0-21swapper/322:08:373
97762290,3chrt0-21swapper/322:08:373
926899293,22cyclictest27-21ksoftirqd/221:55:542
926899293,22cyclictest27-21ksoftirqd/221:55:542
926799293,19cyclictest7015-21cat20:35:481
926799293,19cyclictest7015-21cat20:35:481
926899271,21cyclictest6546-21ssh21:30:552
926899271,21cyclictest6546-21ssh21:30:552
38482270,3sleep10-21swapper/100:21:501
38482270,3sleep10-21swapper/100:21:501
139152270,2sleep113773-21latency_hist00:35:441
139152270,2sleep113773-21latency_hist00:35:441
926899262,20cyclictest14121-21sh00:35:542
926899262,20cyclictest14121-21sh00:35:542
926899262,19cyclictest6133-21sh00:25:512
926899262,19cyclictest6133-21sh00:25:512
269332260,2sleep333-21ksoftirqd/300:11:033
269332260,2sleep333-21ksoftirqd/300:11:033
129482260,3sleep20-21swapper/223:55:552
129482260,3sleep20-21swapper/223:55:552
10302260,3sleep30-21swapper/322:34:013
10302260,3sleep30-21swapper/322:34:013
9672250,3chrt0-21swapper/222:33:562
9672250,3chrt0-21swapper/222:33:562
926999253,15cyclictest0-21swapper/322:50:493
926999253,15cyclictest0-21swapper/322:50:493
238992250,4chrt0-21swapper/222:23:432
9269992418,4cyclictest321rcuc/300:04:113
9269992418,4cyclictest321rcuc/300:04:113
9269992415,6cyclictest321rcuc/322:58:423
9269992415,6cyclictest321rcuc/322:58:423
926899242,16cyclictest3173-21sh23:45:532
926899242,16cyclictest3173-21sh23:45:532
9268992413,7cyclictest0-21swapper/221:05:492
926799241,18cyclictest20021-21diskmemload22:25:531
184682240,2sleep20-21swapper/200:01:052
184682240,2sleep20-21swapper/200:01:052
926899234,14cyclictest27-21ksoftirqd/222:40:522
926899234,14cyclictest27-21ksoftirqd/222:40:522
926899232,16cyclictest0-21swapper/219:20:542
926899232,16cyclictest0-21swapper/219:20:542
253812230,1sleep1201rcuc/121:50:551
253812230,1sleep1201rcuc/121:50:551
9269992216,4cyclictest321rcuc/322:03:443
9269992216,4cyclictest321rcuc/322:03:443
926899229,11cyclictest29965-21fw_packets21:20:562
926899229,11cyclictest29965-21fw_packets21:20:562
926899223,3cyclictest131rcu_preempt19:30:542
926899221,18cyclictest32393-21latency20:15:562
926899221,18cyclictest32393-21latency20:15:562
926899221,12cyclictest30182-21latency_hist00:15:402
926899221,12cyclictest30182-21latency_hist00:15:402
926899220,20cyclictest20620-21ls22:20:432
926899220,20cyclictest20620-21ls22:20:432
926799222,15cyclictest16408-21apt-get21:00:521
926799222,15cyclictest16408-21apt-get21:00:521
926799221,17cyclictest16690-21ssh22:50:561
926799221,17cyclictest16690-21ssh22:50:561
76902220,3sleep30-21swapper/320:36:023
101502220,4sleep30-21swapper/323:52:303
9269992116,3cyclictest321rcuc/322:24:383
9269992112,6cyclictest0-21swapper/323:14:173
9269992112,6cyclictest0-21swapper/323:14:173
926899216,13cyclictest23021-21sed00:06:052
926899215,12cyclictest6818-21kworker/2:020:35:542
926899211,14cyclictest10849-21cat20:45:492
926799212,10cyclictest15689-21unixbench_singl20:56:051
926799212,10cyclictest15689-21unixbench_singl20:56:051
9267992113,6cyclictest0-21swapper/122:45:351
9267992113,6cyclictest0-21swapper/122:45:351
926799211,18cyclictest2628-21ls22:00:581
926799211,18cyclictest2628-21ls22:00:581
926799211,17cyclictest13866-21ls19:25:571
926799211,17cyclictest13866-21ls19:25:571
926799211,16cyclictest7328-21apt-key23:15:421
926799211,16cyclictest7328-21apt-key23:15:421
9267992110,8cyclictest21-21ksoftirqd/122:50:491
9267992110,8cyclictest21-21ksoftirqd/122:50:491
31932210,3sleep20-21swapper/222:36:012
26412211,3sleep30-21swapper/323:45:243
26412211,3sleep30-21swapper/323:45:243
926999203,13cyclictest0-21swapper/320:35:483
9269992013,5cyclictest0-21swapper/323:46:243
9269992013,5cyclictest0-21swapper/323:46:243
9269992013,5cyclictest0-21swapper/321:25:373
9269992013,5cyclictest0-21swapper/321:25:373
9269992012,6cyclictest0-21swapper/322:43:513
9269992012,6cyclictest0-21swapper/322:43:513
926899206,8cyclictest27-21ksoftirqd/220:30:512
926899206,8cyclictest27-21ksoftirqd/220:30:512
9268992014,4cyclictest0-21swapper/222:55:522
9268992014,4cyclictest0-21swapper/222:55:522
9268992013,5cyclictest30588-21grep20:10:552
9268992013,5cyclictest30588-21grep20:10:552
926799209,8cyclictest21-21ksoftirqd/121:30:541
926799209,8cyclictest21-21ksoftirqd/121:30:541
926799201,15cyclictest20518-21latency_hist22:20:491
926799201,15cyclictest20518-21latency_hist22:20:491
85042209,6sleep17401-1kworker/u9:119:10:581
85042209,6sleep17401-1kworker/u9:119:10:581
79822200,3sleep30-21swapper/321:31:193
79822200,3sleep30-21swapper/321:31:193
926999193,10cyclictest0-21swapper/319:50:543
926999193,10cyclictest0-21swapper/319:50:543
9269991914,3cyclictest321rcuc/323:34:183
9269991914,3cyclictest321rcuc/323:34:183
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional