You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 23:48
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1.osadl.org (updated Wed Nov 19, 2025 12:44:57)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2208921590,3sleep20-21swapper/210:42:052
2889021540,2sleep10-21swapper/110:50:561
2889021540,2sleep10-21swapper/110:50:561
782121340,2sleep07823-21cron07:20:380
3248421150,3sleep30-21swapper/311:36:443
3211621080,2sleep20-21swapper/212:18:222
3211621080,2sleep20-21swapper/212:18:222
1930921050,4sleep333-21ksoftirqd/309:56:053
1930921050,4sleep333-21ksoftirqd/309:56:053
108322930,3sleep30-21swapper/312:31:463
108322930,3sleep30-21swapper/312:31:463
108192920,4chrt0-21swapper/212:31:452
108192920,4chrt0-21swapper/212:31:452
114322870,3sleep00-21swapper/010:29:130
114322870,3sleep00-21swapper/010:29:130
130212800,3sleep30-21swapper/309:48:333
130212800,3sleep30-21swapper/309:48:333
255752760,3sleep00-21swapper/010:46:050
255752760,3sleep00-21swapper/010:46:050
38872680,3sleep00-21swapper/011:42:110
38872680,3sleep00-21swapper/011:42:110
136802680,3chrt0-21swapper/211:11:592
136802680,3chrt0-21swapper/211:11:592
107552680,1sleep2261rcuc/209:45:552
107552680,1sleep2261rcuc/209:45:552
95202670,2chrt0-21swapper/308:50:543
18162670,2sleep20-21swapper/208:30:562
18162670,2sleep20-21swapper/208:30:562
184672620,3sleep30-21swapper/312:00:553
184672620,3sleep30-21swapper/312:00:553
161882580,3sleep30-21swapper/307:40:583
161882580,3sleep30-21swapper/307:40:583
128912570,4sleep00-21swapper/012:35:400
128912570,4sleep00-21swapper/012:35:400
5591995631,13cyclictest21-21ksoftirqd/111:40:501
5591995631,13cyclictest21-21ksoftirqd/111:40:501
56312540,1sleep22518-1kworker/u9:009:39:212
56312540,1sleep22518-1kworker/u9:009:39:212
30992530,3chrt0-21swapper/012:22:060
30992530,3chrt0-21swapper/012:22:060
209842520,3sleep20-21swapper/207:55:442
103662520,3sleep30-21swapper/307:25:553
103662520,3sleep30-21swapper/307:25:553
87652510,3chrt0-21swapper/212:30:162
87652510,3chrt0-21swapper/212:30:162
559199515,36cyclictest10627-21ssh09:45:531
559199515,36cyclictest10627-21ssh09:45:531
325232510,2sleep132527-21ssh10:15:051
325232510,2sleep132527-21ssh10:15:051
131962510,3sleep20-21swapper/209:00:582
558999502,37cyclictest23187-21ssh11:25:510
558999502,37cyclictest23187-21ssh11:25:510
323382500,2sleep10-21swapper/108:25:531
323382500,2sleep10-21swapper/108:25:531
213962500,4chrt0-21swapper/309:16:063
559299491,35cyclictest27401-21ssh09:25:512
559299491,35cyclictest27401-21ssh09:25:512
5591994922,23cyclictest21-21ksoftirqd/107:55:511
5591994922,23cyclictest21-21ksoftirqd/107:55:511
209252480,3sleep30-21swapper/310:40:583
271292470,4sleep027124-21tail08:10:570
215262470,4sleep30-21swapper/311:22:573
559199464,34cyclictest31195-21ssh09:30:511
559199464,34cyclictest31195-21ssh09:30:511
558999464,35cyclictest20583-21ssh09:15:510
28002460,3sleep10-21swapper/110:58:521
277682460,3chrt0-21swapper/212:12:362
277682460,3chrt0-21swapper/212:12:362
559199453,34cyclictest2738-21ssh09:35:511
559199453,34cyclictest2738-21ssh09:35:511
43912450,3sleep20-21swapper/208:36:022
23192450,2sleep10-21swapper/108:31:031
23192450,2sleep10-21swapper/108:31:031
5591994426,11cyclictest21-21ksoftirqd/111:15:511
5591994426,11cyclictest21-21ksoftirqd/111:15:511
40692440,3chrt0-21swapper/212:23:572
40692440,3chrt0-21swapper/212:23:572
111112440,3sleep00-21swapper/011:09:440
111112440,3sleep00-21swapper/011:09:440
559299422,32cyclictest28405-21ssh10:50:522
559299422,32cyclictest28405-21ssh10:50:522
5591994225,8cyclictest21-21ksoftirqd/112:25:511
5591994225,8cyclictest21-21ksoftirqd/112:25:511
297712420,3sleep00-21swapper/008:18:120
297712420,3sleep00-21swapper/008:18:120
559299418,10cyclictest27-21ksoftirqd/211:35:532
559199411,31cyclictest0-21swapper/111:05:541
559199411,31cyclictest0-21swapper/111:05:541
285382410,2sleep128539-21sh10:09:301
285382410,2sleep128539-21sh10:09:301
266942410,3sleep30-21swapper/310:48:053
266942410,3sleep30-21swapper/310:48:053
157682410,3sleep20-21swapper/211:57:042
559299402,32cyclictest3992-21ssh11:00:532
559299402,32cyclictest3992-21ssh11:00:532
5591994018,13cyclictest21-21ksoftirqd/112:10:511
5591994018,13cyclictest21-21ksoftirqd/112:10:511
559199392,25cyclictest16182-21diskmemload12:00:511
559199392,25cyclictest16182-21diskmemload12:00:511
199452390,2sleep00-21swapper/011:20:570
177712390,3sleep25588-21cyclictest09:55:272
177712390,3sleep25588-21cyclictest09:55:272
121092390,2chrt0-21swapper/011:53:000
559199381,28cyclictest0-21swapper/110:00:501
559199381,28cyclictest0-21swapper/110:00:501
274522380,3sleep00-21swapper/009:25:520
274522380,3sleep00-21swapper/009:25:520
98302360,3chrt0-21swapper/209:44:502
98302360,3chrt0-21swapper/209:44:502
558999362,20cyclictest30939-21ssh11:35:520
55422360,2sleep24129-21ls07:11:082
76222350,4sleep10-21swapper/108:45:541
76222350,4sleep10-21swapper/108:45:541
559199356,19cyclictest8626-21ssh10:25:541
559199356,19cyclictest8626-21ssh10:25:541
559199354,16cyclictest21899-21ssh12:05:491
559199354,16cyclictest21899-21ssh12:05:491
559199352,29cyclictest9668-21latency_hist07:25:461
559199352,29cyclictest9668-21latency_hist07:25:461
26122350,3sleep00-21swapper/010:17:080
233992350,3sleep20-21swapper/209:19:552
89182340,2sleep08928-21ssh11:49:320
313982340,2sleep031409-21ssh12:17:090
313982340,2sleep031409-21ssh12:17:090
273052340,3chrt0-21swapper/209:25:452
273052340,3chrt0-21swapper/209:25:452
5591993325,7cyclictest24108-21fschecks_count09:20:541
5591993325,7cyclictest24108-21fschecks_count09:20:541
559199332,24cyclictest0-21swapper/111:10:521
559199332,24cyclictest0-21swapper/111:10:521
558999338,18cyclictest38850irq/55-eth009:10:500
558999338,18cyclictest38850irq/55-eth009:10:500
320502330,3sleep00-21swapper/009:31:030
320502330,3sleep00-21swapper/009:31:030
82752320,11sleep227-21ksoftirqd/210:25:372
82752320,11sleep227-21ksoftirqd/210:25:372
559199322,24cyclictest0-21swapper/107:15:521
327042320,2sleep10-21swapper/112:19:311
327042320,2sleep10-21swapper/112:19:311
110642320,2sleep10-21swapper/111:51:091
89382310,1sleep227-21ksoftirqd/211:49:342
559199314,3cyclictest131rcu_preempt08:05:501
558999315,17cyclictest21926-21ssh10:00:510
558999315,17cyclictest21926-21ssh10:00:510
558999312,6cyclictest12-21ksoftirqd/009:50:540
558999312,6cyclictest12-21ksoftirqd/009:50:540
558999312,4cyclictest0-21swapper/011:00:540
558999312,4cyclictest0-21swapper/011:00:540
203352310,2sleep012-21ksoftirqd/012:03:130
203352310,2sleep012-21ksoftirqd/012:03:130
5592993013,8cyclictest0-21swapper/211:50:532
559299301,22cyclictest12438-21ssh10:30:502
559299301,22cyclictest12438-21ssh10:30:502
559299301,20cyclictest27-21ksoftirqd/208:25:492
559299301,20cyclictest27-21ksoftirqd/208:25:492
5592993010,7cyclictest27-21ksoftirqd/208:54:542
559199302,21cyclictest28404-21chrt08:15:521
559199302,21cyclictest28404-21chrt08:15:521
558999309,10cyclictest12-21ksoftirqd/008:54:540
558999308,9cyclictest12-21ksoftirqd/009:35:550
558999308,9cyclictest12-21ksoftirqd/009:35:550
558999297,15cyclictest0-21swapper/012:25:510
558999297,15cyclictest0-21swapper/012:25:510
558999293,23cyclictest13923-21ssh11:55:510
558999292,24cyclictest0-21swapper/007:35:530
41022290,2sleep14116-21ssh12:24:011
41022290,2sleep14116-21ssh12:24:011
256172290,3sleep30-21swapper/312:10:403
256172290,3sleep30-21swapper/312:10:403
180872290,3sleep10-21swapper/110:37:181
180872290,3sleep10-21swapper/110:37:181
174302290,3chrt0-21swapper/207:45:512
559299283,11cyclictest27-21ksoftirqd/210:10:512
559299283,11cyclictest27-21ksoftirqd/210:10:512
5592992811,8cyclictest27-21ksoftirqd/210:19:422
5591992814,5cyclictest21-21ksoftirqd/108:10:491
157712280,2sleep115781-21sh09:51:251
157712280,2sleep115781-21sh09:51:251
139092280,2sleep10-21swapper/110:31:351
139092280,2sleep10-21swapper/110:31:351
559399271,20cyclictest11862-21taskset07:30:533
559299272,19cyclictest26582-21runrttasks08:10:502
559199273,18cyclictest31108-21ssh09:30:471
559199273,18cyclictest31108-21ssh09:30:471
558999279,3cyclictest131rcu_preempt11:18:460
558999279,3cyclictest131rcu_preempt11:18:460
558999278,9cyclictest12-21ksoftirqd/008:56:560
558999278,9cyclictest12-21ksoftirqd/008:56:560
558999274,6cyclictest12-21ksoftirqd/008:06:010
558999273,20cyclictest29410-21ssh10:10:510
558999273,20cyclictest29410-21ssh10:10:510
558999270,6cyclictest12-21ksoftirqd/007:40:520
558999270,6cyclictest12-21ksoftirqd/007:40:520
306682270,3sleep00-21swapper/011:35:400
306682270,3sleep00-21swapper/011:35:400
275512270,3chrt0-21swapper/111:30:551
275512270,3chrt0-21swapper/111:30:551
16072270,3sleep10-21swapper/111:40:191
559299262,18cyclictest13207-21ssh12:35:532
559299262,18cyclictest13207-21ssh12:35:532
5592992610,11cyclictest261rcuc/210:55:512
559199262,19cyclictest6632-21ssh11:46:021
558999267,12cyclictest12-21ksoftirqd/012:35:520
558999267,12cyclictest12-21ksoftirqd/012:35:520
5589992619,5cyclictest151rcuc/009:45:530
5589992619,5cyclictest151rcuc/009:45:530
5589992613,11cyclictest8189-21sh09:41:340
5589992613,11cyclictest8189-21sh09:41:340
5589992610,8cyclictest12-21ksoftirqd/007:47:360
558999260,8cyclictest12-21ksoftirqd/007:55:530
558999260,8cyclictest12-21ksoftirqd/007:55:530
558999260,6cyclictest12-21ksoftirqd/007:30:550
43372264,14sleep00-21swapper/007:10:460
559399253,16cyclictest0-21swapper/310:30:503
559399253,16cyclictest0-21swapper/310:30:503
559299257,3cyclictest131rcu_preempt07:30:522
559299253,16cyclictest2122-21ssh11:40:512
559299253,16cyclictest2122-21ssh11:40:512
559299251,18cyclictest732-21runrttasks11:15:532
559299251,18cyclictest732-21runrttasks11:15:532
559299251,11cyclictest17089-21kworker/2:109:55:562
559299251,11cyclictest17089-21kworker/2:109:55:562
559199252,16cyclictest24308-21ssh10:45:511
559199252,16cyclictest24308-21ssh10:45:511
558999258,9cyclictest12-21ksoftirqd/009:01:290
558999254,6cyclictest12-21ksoftirqd/008:01:050
558999252,16cyclictest25772-21rm10:05:520
558999252,16cyclictest25772-21rm10:05:520
5589992517,5cyclictest12-21ksoftirqd/008:30:520
5589992517,5cyclictest12-21ksoftirqd/008:30:520
558999251,21cyclictest0-21swapper/007:20:500
558999251,21cyclictest0-21swapper/007:20:500
558999251,18cyclictest12-21ksoftirqd/007:25:520
558999251,18cyclictest12-21ksoftirqd/007:25:520
36422250,2sleep30-21swapper/310:18:593
559399244,16cyclictest0-21swapper/310:35:543
559399244,16cyclictest0-21swapper/310:35:543
559299245,3cyclictest131rcu_preempt09:05:512
559299245,3cyclictest131rcu_preempt09:05:512
5592992411,5cyclictest131rcu_preempt09:33:112
5592992411,5cyclictest131rcu_preempt09:33:112
5592992411,10cyclictest27-21ksoftirqd/207:35:512
558999248,3cyclictest131rcu_preempt11:11:200
558999248,3cyclictest131rcu_preempt11:11:200
558999248,10cyclictest12-21ksoftirqd/008:23:560
558999248,10cyclictest12-21ksoftirqd/008:23:560
558999247,7cyclictest12-21ksoftirqd/010:36:380
558999247,7cyclictest12-21ksoftirqd/010:36:380
558999246,5cyclictest12-21ksoftirqd/009:55:580
558999246,5cyclictest12-21ksoftirqd/009:55:580
558999242,18cyclictest542-21snmpd08:40:540
558999242,18cyclictest542-21snmpd08:40:540
558999242,18cyclictest24088-21ssh09:20:540
558999242,18cyclictest24088-21ssh09:20:540
558999242,10cyclictest25858-21rm12:10:520
558999242,10cyclictest25858-21rm12:10:520
558999241,3cyclictest0-21swapper/008:35:500
5589992410,5cyclictest12-21ksoftirqd/010:32:270
5589992410,5cyclictest12-21ksoftirqd/010:32:270
559399235,14cyclictest0-21swapper/309:30:473
559399235,14cyclictest0-21swapper/309:30:473
559299237,13cyclictest131rcu_preempt10:05:522
559299237,13cyclictest131rcu_preempt10:05:522
559299234,3cyclictest131rcu_preempt10:03:222
559299234,3cyclictest131rcu_preempt10:03:222
5592992316,5cyclictest30256-21/usr/sbin/munin08:20:542
5592992316,5cyclictest30256-21/usr/sbin/munin08:20:542
5592992310,10cyclictest131rcu_preempt10:35:542
5592992310,10cyclictest131rcu_preempt10:35:542
558999239,12cyclictest0-21swapper/008:46:330
558999239,12cyclictest0-21swapper/008:46:330
558999239,12cyclictest0-21swapper/008:27:440
558999239,12cyclictest0-21swapper/008:27:440
558999238,4cyclictest12-21ksoftirqd/010:55:060
558999238,4cyclictest12-21ksoftirqd/010:55:060
558999236,7cyclictest12-21ksoftirqd/012:10:460
558999236,7cyclictest12-21ksoftirqd/012:10:460
559299221,8cyclictest131rcu_preempt07:25:382
559299221,8cyclictest131rcu_preempt07:25:382
5592992213,7cyclictest24236-21sshd08:02:132
559199221,18cyclictest14137-21grep11:55:541
559199221,17cyclictest9919-21rm12:31:001
559199221,17cyclictest9919-21rm12:31:001
559199221,17cyclictest16509-21ssh09:10:521
559199221,17cyclictest16509-21ssh09:10:521
558999228,6cyclictest12-21ksoftirqd/010:45:430
558999227,12cyclictest0-21swapper/010:20:510
558999227,12cyclictest0-21swapper/010:20:510
49892225,9sleep10-21swapper/107:10:591
193532220,2sleep20-21swapper/212:01:062
193532220,2sleep20-21swapper/212:01:062
158012220,3chrt0-21swapper/207:40:532
158012220,3chrt0-21swapper/207:40:532
559399213,14cyclictest0-21swapper/312:15:523
559399213,14cyclictest0-21swapper/312:15:523
5592992115,4cyclictest11106-21fschecks_time08:55:542
5592992115,4cyclictest11106-21fschecks_time08:55:542
5592992112,7cyclictest26987-21ssh10:48:372
5592992112,7cyclictest26987-21ssh10:48:372
559299211,15cyclictest28830-21latency08:15:572
559299211,15cyclictest28830-21latency08:15:572
559199213,12cyclictest21-21ksoftirqd/111:00:531
559199213,12cyclictest21-21ksoftirqd/111:00:531
5591992117,3cyclictest137792sleep107:35:551
558999218,11cyclictest0-21swapper/007:51:190
558999217,8cyclictest12-21ksoftirqd/009:06:480
558999217,8cyclictest12-21ksoftirqd/009:06:480
558999213,6cyclictest12-21ksoftirqd/010:58:410
177142210,3sleep30-21swapper/311:17:393
177142210,3sleep30-21swapper/311:17:393
80262200,4sleep30-21swapper/309:41:143
80262200,4sleep30-21swapper/309:41:143
559399204,10cyclictest0-21swapper/310:25:543
559399204,10cyclictest0-21swapper/310:25:543
559299205,8cyclictest27-21ksoftirqd/208:46:222
559299205,8cyclictest27-21ksoftirqd/208:46:222
559299205,7cyclictest131rcu_preempt11:26:132
559299205,7cyclictest131rcu_preempt11:26:132
559299204,6cyclictest27-21ksoftirqd/208:06:362
5592992012,6cyclictest29045-21ssh11:32:392
5592992012,6cyclictest29045-21ssh11:32:392
559199209,9cyclictest21309-21ssh11:22:301
5591992013,5cyclictest9462-21sshd07:23:121
5591992013,5cyclictest9462-21sshd07:23:121
5591992011,6cyclictest0-21swapper/107:50:431
559199201,14cyclictest24247-21timerandwakeup11:26:071
559199201,14cyclictest24247-21timerandwakeup11:26:071
5591992010,8cyclictest1589-21cat10:16:011
559399193,11cyclictest0-21swapper/310:55:523
559399192,11cyclictest0-21swapper/308:25:513
559399192,11cyclictest0-21swapper/308:25:513
559399192,11cyclictest0-21swapper/307:50:503
559399191,13cyclictest0-21swapper/312:25:573
559399191,13cyclictest0-21swapper/312:25:573
559399191,13cyclictest0-21swapper/311:41:033
559399191,13cyclictest0-21swapper/311:41:033
559399191,13cyclictest0-21swapper/310:05:443
559399191,13cyclictest0-21swapper/310:05:443
559299197,10cyclictest131rcu_preempt07:55:542
559299197,10cyclictest131rcu_preempt07:55:542
559299196,6cyclictest131rcu_preempt12:05:472
559299196,6cyclictest131rcu_preempt12:05:472
559299191,7cyclictest0-21swapper/207:20:392
5592991914,3cyclictest27-21ksoftirqd/209:11:002
5592991914,3cyclictest27-21ksoftirqd/209:11:002
5592991913,4cyclictest0-21swapper/208:45:452
5592991910,2cyclictest131rcu_preempt11:24:562
559199197,10cyclictest30438-21uniq08:20:531
559199197,10cyclictest30438-21uniq08:20:531
559199196,10cyclictest14810-21fschecks_time09:05:541
559199196,10cyclictest14810-21fschecks_time09:05:541
559199193,11cyclictest21-21ksoftirqd/110:40:501
5591991912,5cyclictest12752-21/usr/sbin/munin09:00:581
5591991911,6cyclictest6740-21sshd08:44:121
5591991911,6cyclictest6740-21sshd08:44:121
167852191,3sleep30-21swapper/311:59:033
56982180,3sleep30-21swapper/308:40:583
56982180,3sleep30-21swapper/308:40:583
559399184,10cyclictest0-21swapper/309:05:513
559399184,10cyclictest0-21swapper/309:05:513
559399183,10cyclictest0-21swapper/309:10:523
559399183,10cyclictest0-21swapper/309:10:523
559399182,11cyclictest0-21swapper/312:20:553
559399182,11cyclictest0-21swapper/312:20:553
559399182,11cyclictest0-21swapper/308:55:533
559399182,11cyclictest0-21swapper/308:55:533
559399182,11cyclictest0-21swapper/308:45:463
559399182,11cyclictest0-21swapper/308:45:463
559399182,11cyclictest0-21swapper/308:00:543
559399182,11cyclictest0-21swapper/307:55:523
559399182,11cyclictest0-21swapper/307:55:523
559399182,10cyclictest0-21swapper/308:35:513
559399181,13cyclictest0-21swapper/310:20:513
559399181,13cyclictest0-21swapper/310:20:513
559399181,13cyclictest0-21swapper/307:45:563
559399181,12cyclictest0-21swapper/308:15:513
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional