You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-01 - 13:48
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack4slot1.osadl.org (updated Mon Dec 01, 2025 12:45:00)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2627424750,1sleep012-21ksoftirqd/007:11:180
2601523020,4chrt0-21swapper/207:11:162
2740321510,3sleep00-21swapper/010:58:360
2740321510,3sleep00-21swapper/010:58:360
3178121120,3sleep20-21swapper/212:21:122
3178121120,3sleep20-21swapper/212:21:122
1655521120,3sleep216554-21ssh11:25:502
1655521120,3sleep216554-21ssh11:25:502
270762990,2chrt0-21swapper/108:41:121
270762990,2chrt0-21swapper/108:41:121
199262980,3chrt0-21swapper/111:29:111
199262980,3chrt0-21swapper/111:29:111
68612930,2chrt0-21swapper/211:13:082
68612930,2chrt0-21swapper/211:13:082
82622920,4sleep20-21swapper/209:15:532
82622920,4sleep20-21swapper/209:15:532
26340998911,66cyclictest8608-21ssh10:36:022
26340998911,66cyclictest8608-21ssh10:36:022
140262880,3sleep10-21swapper/109:22:401
140262880,3sleep10-21swapper/109:22:401
185442780,3sleep10-21swapper/110:46:511
185442780,3sleep10-21swapper/110:46:511
226682770,4sleep10-21swapper/110:13:391
226682770,4sleep10-21swapper/110:13:391
128562730,4chrt12852-21grep10:41:081
128562730,4chrt12852-21grep10:41:081
11522700,3sleep10-21swapper/111:06:121
2634099653,41cyclictest7501-21ssh12:31:032
2634099653,41cyclictest7501-21ssh12:31:032
318222640,3chrt0-21swapper/109:45:581
21712600,6sleep012-21ksoftirqd/009:01:130
21712600,6sleep012-21ksoftirqd/009:01:130
26339995816,16cyclictest21-21ksoftirqd/108:51:001
26339995816,16cyclictest21-21ksoftirqd/108:51:001
217902580,6chrt21792-21grep08:26:141
166822580,4sleep00-21swapper/010:45:540
166822580,4sleep00-21swapper/010:45:540
63822550,3sleep10-21swapper/109:11:301
63822550,3sleep10-21swapper/109:11:301
26338995517,8cyclictest131rcu_preempt07:36:000
2633999546,17cyclictest21-21ksoftirqd/108:06:051
26338995328,16cyclictest38850irq/55-eth011:11:020
26338995328,16cyclictest38850irq/55-eth011:11:020
235362530,4sleep10-21swapper/110:53:181
235362530,4sleep10-21swapper/110:53:181
223892530,3sleep10-21swapper/111:31:161
223892530,3sleep10-21swapper/111:31:161
38602520,2sleep30-21swapper/311:49:143
26339995141,5cyclictest732-21runrttasks11:46:021
296192490,2sleep10-21swapper/111:01:101
296192490,2sleep10-21swapper/111:01:101
2634099481,32cyclictest16883-21ssh10:46:022
2634099481,32cyclictest16883-21ssh10:46:022
2633899474,29cyclictest38850irq/55-eth010:21:020
236432470,3sleep30-21swapper/308:31:143
34582450,2sleep30-21swapper/312:26:083
34582450,2sleep30-21swapper/312:26:083
205312450,3sleep20-21swapper/211:30:212
205312450,3sleep20-21swapper/211:30:212
14232450,3sleep30-21swapper/312:24:073
14232450,3sleep30-21swapper/312:24:073
300652440,4sleep10-21swapper/110:21:511
283672440,3sleep20-21swapper/212:17:392
283672440,3sleep20-21swapper/212:17:392
2633899447,10cyclictest12-21ksoftirqd/008:51:000
2633899447,10cyclictest12-21ksoftirqd/008:51:000
2633899444,24cyclictest0-21swapper/010:36:020
2633899444,24cyclictest0-21swapper/010:36:020
235802440,3sleep30-21swapper/311:33:273
235802440,3sleep30-21swapper/311:33:273
223252440,2sleep022341-21grep08:30:520
116822440,2sleep30-21swapper/309:20:413
42212430,4sleep20-21swapper/210:30:532
26340994325,9cyclictest27-21ksoftirqd/208:41:012
26340994325,9cyclictest27-21ksoftirqd/208:41:012
191732430,2sleep30-21swapper/310:09:483
191732430,2sleep30-21swapper/310:09:483
318702420,3sleep0151rcuc/008:56:020
318702420,3sleep0151rcuc/008:56:020
2634099423,30cyclictest21143-21ssh10:51:012
2634099423,30cyclictest21143-21ssh10:51:012
2633999423,33cyclictest16845-21latency_hist11:25:581
2633999423,33cyclictest16845-21latency_hist11:25:581
2633999413,31cyclictest20129-21cat10:10:581
2633999413,31cyclictest20129-21cat10:10:581
2633999403,28cyclictest24169-21ssh10:16:021
2633999403,28cyclictest24169-21ssh10:16:021
2633899404,22cyclictest38850irq/55-eth012:31:020
2633899404,22cyclictest38850irq/55-eth012:31:020
2634199392,28cyclictest0-21swapper/310:36:023
2634199392,28cyclictest0-21swapper/310:36:023
2633999391,33cyclictest0-21swapper/112:06:021
2633999391,27cyclictest25144-21rm10:56:021
2633999391,27cyclictest25144-21rm10:56:021
163162390,4sleep10-21swapper/112:05:041
163162390,4sleep10-21swapper/112:05:041
271022380,2sleep30-21swapper/309:40:063
271022380,2sleep30-21swapper/309:40:063
2633999384,13cyclictest21-21ksoftirqd/107:16:001
2633999384,13cyclictest21-21ksoftirqd/107:16:001
2633999381,27cyclictest23712-21ssh09:36:011
2633999381,27cyclictest23712-21ssh09:36:011
2633899381,30cyclictest26157-21sh12:16:030
2633899381,30cyclictest26157-21sh12:16:030
26340993724,8cyclictest0-21swapper/207:21:002
26340993724,8cyclictest0-21swapper/207:21:002
2633999373,27cyclictest4435-21runrttasks10:31:031
2633999373,27cyclictest4435-21runrttasks10:31:031
2633999364,28cyclictest3721-21ssh09:51:011
2634099351,26cyclictest11954-21ssh10:01:002
2634099351,26cyclictest11954-21ssh10:01:002
2633899351,27cyclictest29475-21ssh11:41:030
119542350,2sleep011960-21sh12:00:300
119542350,2sleep011960-21sh12:00:300
2634099340,28cyclictest21144-21ssh11:31:032
2634099340,28cyclictest21144-21ssh11:31:032
2633899342,21cyclictest38850irq/55-eth008:00:590
76112330,2sleep07616-21sh11:53:550
76112330,2sleep07616-21sh11:53:550
2634099331,25cyclictest1100-21ssh11:46:032
2633999339,16cyclictest0-21swapper/111:16:021
2633999339,16cyclictest0-21swapper/111:16:021
2633999333,23cyclictest4002-21apt-get10:30:581
61262320,2sleep30-21swapper/310:32:023
61262320,2sleep30-21swapper/310:32:023
306372320,3sleep333-21ksoftirqd/310:22:543
2634099322,24cyclictest29484-21seq11:41:042
2633899320,3cyclictest131rcu_preempt10:16:130
2633899320,3cyclictest131rcu_preempt10:16:130
2633999314,21cyclictest25237-21ssh11:36:011
2633999314,21cyclictest25237-21ssh11:36:011
250742310,3sleep30-21swapper/308:36:163
125982310,4sleep112597-21ssh12:36:111
125982310,4sleep112597-21ssh12:36:111
271512300,2sleep20-21swapper/210:58:012
271512300,2sleep20-21swapper/210:58:012
2633999302,21cyclictest2903-21rm12:26:021
2633999302,21cyclictest2903-21rm12:26:021
2633999302,21cyclictest1214-21chrt09:01:021
2633999302,21cyclictest1214-21chrt09:01:021
2633899303,19cyclictest31896-21ssh09:46:030
2633899303,19cyclictest31896-21ssh09:46:030
26338993011,11cyclictest38850irq/55-eth010:01:000
26338993011,11cyclictest38850irq/55-eth010:01:000
151042300,3sleep20-21swapper/208:11:002
151042300,3sleep20-21swapper/208:11:002
2633999294,19cyclictest21665-21ssh12:11:021
2633999294,19cyclictest21665-21ssh12:11:021
2633999293,22cyclictest11966-21ssh10:01:021
2633999293,22cyclictest11966-21ssh10:01:021
2633999292,21cyclictest31900-21rm09:46:031
2633999292,21cyclictest31900-21rm09:46:031
26339992913,13cyclictest26158-21rm12:16:031
26339992913,13cyclictest26158-21rm12:16:031
232852290,4sleep30-21swapper/310:14:503
232852290,4sleep30-21swapper/310:14:503
288562280,1sleep228992-21/usr/sbin/munin09:41:142
288562280,1sleep228992-21/usr/sbin/munin09:41:142
2633999283,19cyclictest21-21ksoftirqd/107:26:021
2633899282,20cyclictest1107-21ssh11:46:030
2633999269,8cyclictest21-21ksoftirqd/111:51:401
2633999269,8cyclictest21-21ksoftirqd/111:51:401
2633999268,6cyclictest21-21ksoftirqd/111:11:021
2633999268,6cyclictest21-21ksoftirqd/111:11:021
2633999264,18cyclictest8791-21ssh11:56:011
2633999264,18cyclictest8791-21ssh11:56:011
2633899262,19cyclictest7703-21ssh09:56:020
2633899262,19cyclictest7703-21ssh09:56:020
2633899261,20cyclictest4331-21latency_hist10:30:580
2633899261,20cyclictest16894-21ssh10:46:030
2633899261,20cyclictest16894-21ssh10:46:030
257262264,15sleep125743-21ntp_states07:11:121
32542250,2sleep30-21swapper/309:50:393
32542250,2sleep30-21swapper/309:50:393
2633899252,15cyclictest151rcuc/007:21:000
2633899252,15cyclictest151rcuc/007:21:000
2633899251,19cyclictest25241-21seq11:36:010
2633899251,19cyclictest25241-21seq11:36:010
2634099243,15cyclictest17067-21sort10:06:192
2634099243,15cyclictest17067-21sort10:06:192
26340992411,9cyclictest131rcu_preempt12:06:022
2633999241,9cyclictest21-21ksoftirqd/108:21:031
2633999241,9cyclictest21-21ksoftirqd/108:21:031
2633999241,19cyclictest2857-21ls07:36:111
2633899243,17cyclictest11874-21ssh12:36:030
2633899243,17cyclictest11874-21ssh12:36:030
2633899242,17cyclictest21139-21taskset11:31:030
2633899242,17cyclictest21139-21taskset11:31:030
2633899242,16cyclictest2902-21sh12:26:020
2633899242,16cyclictest2902-21sh12:26:020
2633899241,20cyclictest5752-21ls07:46:040
2633899241,20cyclictest5752-21ls07:46:040
2633899240,5cyclictest131rcu_preempt10:06:190
2633899240,5cyclictest131rcu_preempt10:06:190
159952240,3sleep30-21swapper/312:04:343
159952240,3sleep30-21swapper/312:04:343
7042230,2sleep2261rcuc/207:31:092
2634099233,5cyclictest25296-21ssh10:16:142
2634099233,5cyclictest25296-21ssh10:16:142
2634099232,15cyclictest9379-21apt-get07:56:032
2634099232,15cyclictest9379-21apt-get07:56:032
2634099231,17cyclictest28392-21awk10:21:002
2633999231,19cyclictest7949-21ls12:31:071
2633999231,19cyclictest7949-21ls12:31:071
2633999231,17cyclictest3722-21chrt09:06:121
2633999231,17cyclictest3722-21chrt09:06:121
2633899238,12cyclictest21148-21sh10:51:020
2633899238,12cyclictest21148-21sh10:51:020
2633899231,10cyclictest15865-21ps08:11:100
2633899231,10cyclictest15865-21ps08:11:100
175742230,2sleep30-21swapper/309:26:393
175742230,2sleep30-21swapper/309:26:393
175742230,2sleep30-21swapper/309:26:393
154152230,3sleep00-21swapper/012:03:420
154152230,3sleep00-21swapper/012:03:420
141142230,2sleep3321rcuc/311:21:293
141142230,2sleep3321rcuc/311:21:293
2634099223,17cyclictest0-21swapper/208:16:032
2634099222,17cyclictest8793-21sh11:56:012
2634099222,17cyclictest8793-21sh11:56:012
2634099222,16cyclictest8527-21ssh11:16:032
2634099222,16cyclictest8527-21ssh11:16:032
2633999222,13cyclictest30797-21ssh12:21:021
2633999222,13cyclictest30797-21ssh12:21:021
2633999221,15cyclictest9092-21taskset09:16:101
2633899223,10cyclictest19146-21fschecks_time08:21:050
2633899223,10cyclictest19146-21fschecks_time08:21:050
26338992212,3cyclictest12-21ksoftirqd/008:16:150
2633899221,18cyclictest5428-21ls07:45:480
2633899221,18cyclictest5428-21ls07:45:480
2633899221,17cyclictest3734-21ssh09:51:020
2633899221,14cyclictest5486-21ls10:31:160
2633899221,14cyclictest5486-21ls10:31:160
2633899221,10cyclictest23011-21grep08:31:070
261712226,10sleep30-21swapper/307:11:183
324452210,2sleep00-21swapper/007:30:550
2634199211,14cyclictest0-21swapper/309:51:003
2634099217,4cyclictest131rcu_preempt11:06:042
2634099212,14cyclictest23717-21ssh09:36:022
2634099212,14cyclictest23717-21ssh09:36:022
2633999213,13cyclictest732-21runrttasks07:31:021
26339992113,6cyclictest7630-21df_inode07:51:031
26339992113,6cyclictest7630-21df_inode07:51:031
26338992110,8cyclictest5947-21switchtime09:11:130
26338992110,8cyclictest5947-21switchtime09:11:130
2634199201,13cyclictest0-21swapper/307:26:043
2634099206,12cyclictest20628-21grep09:31:112
2634099206,12cyclictest20628-21grep09:31:112
2634099205,3cyclictest131rcu_preempt07:16:142
2634099205,3cyclictest131rcu_preempt07:16:142
2634099204,9cyclictest0-21swapper/207:26:072
2634099203,6cyclictest20703-21ssh10:11:072
2634099203,6cyclictest20703-21ssh10:11:072
2634099203,4cyclictest131rcu_preempt07:36:012
2634099202,14cyclictest13125-21apt-get08:06:052
26340992014,4cyclictest12853-21cut10:41:082
26340992014,4cyclictest12853-21cut10:41:082
26340992014,3cyclictest27-21ksoftirqd/207:51:022
26340992014,3cyclictest27-21ksoftirqd/207:51:022
26340992013,5cyclictest13246-21ssh12:36:232
26340992013,5cyclictest13246-21ssh12:36:232
2634099200,18cyclictest0-21swapper/209:21:022
2634099200,18cyclictest0-21swapper/209:21:022
2634099200,15cyclictest17163-21ls09:26:132
2634099200,15cyclictest17163-21ls09:26:132
2634099200,15cyclictest17163-21ls09:26:132
2633999208,7cyclictest21-21ksoftirqd/110:00:341
2633999208,7cyclictest21-21ksoftirqd/110:00:341
2633999205,7cyclictest21-21ksoftirqd/107:57:411
2633999205,7cyclictest21-21ksoftirqd/107:57:411
2633999205,12cyclictest19889-21ssh09:31:011
2633999205,12cyclictest19889-21ssh09:31:011
26339992013,5cyclictest0-21swapper/108:36:051
26339992011,7cyclictest23466-21cat08:31:131
2633999201,15cyclictest28377-21ssh09:41:071
2633899205,6cyclictest24953-21unixbench_singl09:36:150
2633899205,6cyclictest24953-21unixbench_singl09:36:150
2633899204,9cyclictest0-21swapper/012:22:290
2633899204,9cyclictest0-21swapper/012:22:290
2633899202,5cyclictest27817-21latency_hist08:45:490
2633899202,5cyclictest27817-21latency_hist08:45:490
2633899202,5cyclictest13125-21apt-get08:06:050
2633899201,8cyclictest12864-21fschecks_count11:21:030
2633899201,8cyclictest12864-21fschecks_count11:21:030
2633899201,3cyclictest131rcu_preempt11:16:020
2633899201,3cyclictest131rcu_preempt11:16:020
26338992013,5cyclictest0-21swapper/011:26:160
26338992013,5cyclictest0-21swapper/011:26:160
26338992012,6cyclictest9950irq/48-mmc011:06:240
2633899201,16cyclictest9333-21latency_hist07:55:560
2633899201,16cyclictest21653-21ssh12:11:010
2633899201,16cyclictest21653-21ssh12:11:010
2633899201,11cyclictest28472-21needreboot09:41:080
2633899201,11cyclictest28472-21needreboot09:41:080
2633899201,11cyclictest11461-21sh09:20:120
2633899201,10cyclictest21671-21sh10:11:380
2633899201,10cyclictest21671-21sh10:11:380
2634199192,12cyclictest0-21swapper/308:21:033
2634199192,12cyclictest0-21swapper/308:21:033
2634199192,11cyclictest0-21swapper/311:01:043
2634199192,11cyclictest0-21swapper/311:01:043
2634199192,11cyclictest0-21swapper/307:41:023
2634199192,11cyclictest0-21swapper/307:41:023
2634199191,13cyclictest0-21swapper/310:26:063
2634199191,13cyclictest0-21swapper/309:41:133
2634199191,13cyclictest0-21swapper/309:41:133
2634199191,13cyclictest0-21swapper/308:56:043
2634199191,13cyclictest0-21swapper/308:56:043
2634099199,8cyclictest12512-21apt-get12:01:002
2634099199,8cyclictest12512-21apt-get12:01:002
2634099198,9cyclictest31867-21sort08:56:022
2634099198,9cyclictest31867-21sort08:56:022
2634099198,9cyclictest2892-21apt-get12:26:022
2634099198,9cyclictest2892-21apt-get12:26:022
2634099197,10cyclictest28061-21apt-get08:46:042
2634099197,10cyclictest28061-21apt-get08:46:042
2634099195,3cyclictest131rcu_preempt09:51:542
26340991912,5cyclictest4665-21diskmemload09:56:052
26340991912,5cyclictest4665-21diskmemload09:56:052
2634099191,16cyclictest4008-21ls09:06:152
2634099191,16cyclictest4008-21ls09:06:152
2634099191,13cyclictest4686-21switchtime07:41:122
2634099191,13cyclictest4686-21switchtime07:41:122
2634099191,10cyclictest732-21runrttasks10:31:142
2634099191,10cyclictest732-21runrttasks10:31:142
2633999198,5cyclictest21-21ksoftirqd/110:40:491
2633999198,5cyclictest21-21ksoftirqd/110:40:491
2633999197,3cyclictest131rcu_preempt07:21:001
2633999197,3cyclictest131rcu_preempt07:21:001
2633999196,4cyclictest21-21ksoftirqd/108:01:061
2633999193,6cyclictest21-21ksoftirqd/108:16:021
26339991912,5cyclictest30170-21ssh11:41:111
26339991912,4cyclictest131rcu_preempt09:26:021
26339991912,4cyclictest131rcu_preempt09:26:021
26339991912,4cyclictest131rcu_preempt09:26:021
2633999191,13cyclictest28514-21latency08:46:091
2633999191,13cyclictest28514-21latency08:46:091
2633999191,12cyclictest4078-21cut07:41:041
2633999191,12cyclictest4078-21cut07:41:041
2633899199,3cyclictest131rcu_preempt09:31:000
2633899199,3cyclictest131rcu_preempt09:31:000
2633899192,9cyclictest28948-21ssh11:01:020
2633899192,9cyclictest28948-21ssh11:01:020
2633899191,8cyclictest25999-21ls08:40:490
2633899191,7cyclictest29103-21timerwakeupswit08:46:160
2633899191,7cyclictest29103-21timerwakeupswit08:46:160
26338991913,4cyclictest38850irq/55-eth009:30:170
26338991913,4cyclictest38850irq/55-eth009:30:170
26338991913,4cyclictest38850irq/55-eth009:30:170
2634199183,10cyclictest0-21swapper/308:41:013
2634199183,10cyclictest0-21swapper/308:41:013
2634199182,12cyclictest0-21swapper/311:11:033
2634199182,12cyclictest0-21swapper/311:11:033
2634199182,11cyclictest0-21swapper/310:51:023
2634199182,11cyclictest0-21swapper/310:51:023
2634199182,11cyclictest0-21swapper/310:46:033
2634199182,11cyclictest0-21swapper/310:46:033
2634199182,11cyclictest0-21swapper/309:06:053
2634199182,11cyclictest0-21swapper/309:06:053
2634199182,11cyclictest0-21swapper/308:51:023
2634199182,11cyclictest0-21swapper/308:51:023
2634199181,14cyclictest0-21swapper/309:56:043
2634199181,14cyclictest0-21swapper/309:56:043
2634199181,13cyclictest0-21swapper/307:51:003
2634199181,13cyclictest0-21swapper/307:51:003
2634199181,13cyclictest0-21swapper/307:25:513
2634199181,13cyclictest0-21swapper/307:25:513
2634199181,12cyclictest0-21swapper/312:11:043
2634199181,12cyclictest0-21swapper/312:11:043
2634199181,12cyclictest0-21swapper/308:30:523
2634199181,11cyclictest0-21swapper/310:56:043
2634199181,11cyclictest0-21swapper/310:56:043
2634099189,7cyclictest31521-21cron08:55:492
2634099189,7cyclictest31521-21cron08:55:492
2634099182,13cyclictest21347-21ntp_states08:26:092
2634099181,9cyclictest732-21runrttasks07:47:062
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional