You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-05-23 - 16:54

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot4s.osadl.org (updated Sat May 23, 2026 12:45:12)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
456899629,13cyclictest12-21ksoftirqd/011:55:130
4588995822,23cyclictest35966irq/58-eth010:35:133
458899495,26cyclictest35966irq/58-eth009:45:143
458099482,26cyclictest24514-21ssh11:15:132
458899473,26cyclictest35966irq/58-eth008:55:133
457299471,34cyclictest11496-21ssh10:50:121
456899467,12cyclictest12-21ksoftirqd/012:25:130
456899455,9cyclictest12-21ksoftirqd/012:05:130
458099431,31cyclictest27-21ksoftirqd/210:30:152
458899425,23cyclictest35966irq/58-eth011:30:133
458899422,25cyclictest35966irq/58-eth010:15:143
457299422,34cyclictest13309-21ssh09:40:121
458099392,31cyclictest4004-21ssh11:39:062
458099392,13cyclictest27-21ksoftirqd/211:55:132
4580993912,15cyclictest131rcu_preempt10:10:132
457299399,18cyclictest21-21ksoftirqd/110:05:141
458099382,31cyclictest15029-21sh12:38:062
458099382,31cyclictest15029-21sh12:38:062
458899375,19cyclictest35966irq/58-eth010:20:143
458899361,22cyclictest35966irq/58-eth012:10:123
458099362,27cyclictest19968-21ssh11:46:232
456899362,24cyclictest20656-21ssh10:55:130
456899362,23cyclictest23309-21ssh09:10:130
458899350,21cyclictest35966irq/58-eth009:30:123
457299355,19cyclictest21-21ksoftirqd/107:30:121
458899342,18cyclictest35966irq/58-eth011:05:123
458099347,16cyclictest4559-21cyclictest11:10:132
457299340,31cyclictest9273-21ssh12:00:131
458899332,19cyclictest35966irq/58-eth009:40:123
4580993320,8cyclictest261rcuc/211:31:132
4580993310,16cyclictest131rcu_preempt12:25:132
456899331,25cyclictest23491-21ssh11:50:130
458899321,23cyclictest35966irq/58-eth011:00:123
458899321,23cyclictest35966irq/58-eth011:00:123
458899321,16cyclictest35966irq/58-eth008:10:133
458099322,21cyclictest27346irq/40-dwc2_hso09:25:142
458099322,18cyclictest27346irq/40-dwc2_hso11:25:132
457299321,27cyclictest25570-21sh10:40:121
457299321,19cyclictest16484-21ssh10:35:121
458899311,21cyclictest0-21swapper/311:10:133
458099314,16cyclictest27346irq/40-dwc2_hso09:40:122
456899312,18cyclictest151rcuc/011:35:130
458099303,21cyclictest0-21swapper/212:15:062
458099301,21cyclictest131rcu_preempt08:40:142
458099301,21cyclictest131rcu_preempt08:40:142
457299305,22cyclictest22511-21ssh09:45:141
457299301,23cyclictest32441-21ssh09:15:141
456899308,7cyclictest131rcu_preempt09:20:130
458099293,18cyclictest31599-21rm09:50:142
458099293,18cyclictest27346irq/40-dwc2_hso07:20:142
458099292,7cyclictest131rcu_preempt08:10:132
457299292,23cyclictest21-21ksoftirqd/108:20:131
457299292,22cyclictest27350-21ssh12:10:121
4568992922,3cyclictest0-21swapper/011:10:130
458899282,15cyclictest35966irq/58-eth012:35:133
458899282,15cyclictest35966irq/58-eth012:35:133
458099283,20cyclictest0-21swapper/212:04:062
458099282,19cyclictest5089-21ssh11:40:072
457299287,11cyclictest21-21ksoftirqd/111:35:131
457299282,18cyclictest14319-21ssh11:45:131
4572992820,6cyclictest21-21ksoftirqd/107:45:141
456899284,20cyclictest13174-21ssh12:20:140
456899283,21cyclictest6495-21ssh11:05:130
456899282,20cyclictest12-21ksoftirqd/008:10:130
456899281,25cyclictest3207-21ssh10:10:140
458099273,16cyclictest27346irq/40-dwc2_hso08:50:132
458099272,18cyclictest0-21swapper/210:50:142
458099272,10cyclictest0-21swapper/207:50:142
458099271,18cyclictest23925-21find07:30:122
457299271,19cyclictest11683-21kworker/1:011:10:331
456899274,19cyclictest17542-21ssh10:00:140
456899273,15cyclictest9251-21ssh12:00:120
456899272,19cyclictest11953-21ssh12:19:060
456899271,18cyclictest25577-21ssh10:40:120
458899263,17cyclictest0-21swapper/312:34:063
458899263,15cyclictest0-21swapper/312:19:063
4588992610,11cyclictest35966irq/58-eth009:25:153
456899265,18cyclictest21489-21ssh10:20:150
456899262,5cyclictest12-21ksoftirqd/010:35:140
458899253,14cyclictest35966irq/58-eth009:50:133
458099257,12cyclictest0-21swapper/209:30:122
458099252,15cyclictest27346irq/40-dwc2_hso08:45:132
457299252,16cyclictest21-21ksoftirqd/109:20:121
4572992513,4cyclictest21-21ksoftirqd/111:05:121
457299251,11cyclictest29523-21kworker/1:011:55:011
456899252,21cyclictest27412-21ssh09:30:130
4568992514,3cyclictest12-21ksoftirqd/007:35:290
4568992513,4cyclictest12-21ksoftirqd/012:39:160
4568992513,4cyclictest12-21ksoftirqd/012:39:160
458899243,14cyclictest0-21swapper/311:39:063
458899243,12cyclictest35966irq/58-eth008:45:133
458099244,15cyclictest27346irq/40-dwc2_hso10:05:132
458099242,16cyclictest27346irq/40-dwc2_hso09:20:132
458099242,15cyclictest0-21swapper/208:20:052
4580992413,9cyclictest4050-21ssh11:02:402
4580992413,9cyclictest4050-21ssh11:02:402
458099241,13cyclictest12369-21ssh10:15:152
457299242,19cyclictest31354-21ssh12:30:141
457299241,17cyclictest22217-21ssh12:25:131
4572992410,8cyclictest30601-21ssh10:25:141
456899243,2cyclictest131rcu_preempt08:05:140
456899243,2cyclictest131rcu_preempt08:05:140
456899241,19cyclictest8384-21sh09:55:150
4568992411,3cyclictest12-21ksoftirqd/011:25:220
458099233,15cyclictest27346irq/40-dwc2_hso08:30:132
458099232,6cyclictest0-21swapper/211:53:062
458099232,18cyclictest0-21swapper/208:25:142
458099231,9cyclictest13439-21grep09:00:022
457299238,8cyclictest0-21swapper/111:01:271
457299238,8cyclictest0-21swapper/111:01:271
457299233,13cyclictest22896-21diskmemload09:55:141
4572992313,6cyclictest0-21swapper/108:20:021
4572992310,3cyclictest131rcu_preempt12:20:141
456899234,5cyclictest131rcu_preempt10:30:210
456899233,18cyclictest12-21ksoftirqd/011:30:140
4568992313,3cyclictest12-21ksoftirqd/012:30:230
456899231,2cyclictest131rcu_preempt10:45:280
456899231,19cyclictest22512-21ssh09:45:140
458099225,7cyclictest7472-21kthreadcore09:35:242
458099223,13cyclictest27346irq/40-dwc2_hso07:15:132
458099222,15cyclictest27346irq/40-dwc2_hso09:45:132
458099222,15cyclictest0-21swapper/211:21:072
458099221,9cyclictest14652-21ssh12:20:202
458099221,8cyclictest28726-21/usr/sbin/munin07:35:372
4580992211,9cyclictest0-21swapper/209:15:292
458099221,17cyclictest25452-21grep08:05:172
458099221,17cyclictest25452-21grep08:05:172
458099221,11cyclictest17052-21apt-get10:00:042
457299225,10cyclictest0-21swapper/111:15:121
457299222,16cyclictest3876-21apt-get08:50:121
457299222,16cyclictest18306-21ssh12:05:141
457299221,3cyclictest131rcu_preempt10:00:241
4572992212,5cyclictest4559-21cyclictest08:05:151
4572992212,5cyclictest4559-21cyclictest08:05:151
456899229,4cyclictest12-21ksoftirqd/011:20:150
456899228,11cyclictest12740-21ps10:15:190
456899224,4cyclictest131rcu_preempt09:50:140
456899222,16cyclictest0-21swapper/011:46:060
4568992215,3cyclictest12-21ksoftirqd/011:01:110
4568992215,3cyclictest12-21ksoftirqd/011:01:110
4568992211,3cyclictest0-21swapper/009:36:490
458899213,13cyclictest0-21swapper/312:00:143
458899213,12cyclictest0-21swapper/311:46:233
458899213,10cyclictest35966irq/58-eth009:10:133
458899212,14cyclictest35966irq/58-eth010:45:153
4588992111,6cyclictest35966irq/58-eth011:40:133
458099213,9cyclictest6124-21fschecks_count07:45:172
458099213,10cyclictest10801-21tr09:55:232
458099212,15cyclictest0-21swapper/210:25:142
458099212,12cyclictest27346irq/40-dwc2_hso07:55:132
458099211,11cyclictest19302-21kthreadcore10:35:252
4572992114,4cyclictest0-21swapper/108:40:021
4572992113,4cyclictest0-21swapper/111:40:121
457299211,16cyclictest1147-21find11:20:121
457299211,14cyclictest7873-21ssh09:35:281
457299211,10cyclictest12612-21ssh10:15:181
457299210,18cyclictest19407-21ls07:55:291
456899218,3cyclictest131rcu_preempt09:05:110
456899215,11cyclictest13298-21apt-get09:40:120
4568992114,4cyclictest151rcuc/009:15:120
4568992113,4cyclictest12-21ksoftirqd/010:25:170
4568992111,3cyclictest12-21ksoftirqd/012:10:190
4568992111,2cyclictest131rcu_preempt07:30:130
4568992111,2cyclictest12-21ksoftirqd/009:25:030
4568992110,4cyclictest12-21ksoftirqd/007:50:280
4568992110,3cyclictest12-21ksoftirqd/010:05:250
458899209,8cyclictest0-21swapper/310:57:253
458899209,6cyclictest0-21swapper/311:57:423
458899203,12cyclictest0-21swapper/311:22:063
458099204,3cyclictest131rcu_preempt07:40:142
458099203,8cyclictest23595-21kthreadcore08:35:222
458099201,9cyclictest26011-21ps10:40:192
458099201,16cyclictest25133-21ssh10:20:272
458099201,11cyclictest3047-21kthreadcore10:45:222
458099201,10cyclictest22088-21kthreadcore10:55:202
458099201,10cyclictest20383-21kthreadcore09:05:202
457299207,10cyclictest16309-21cat10:35:021
457299202,15cyclictest7175-21timerandwakeup10:10:301
4572992014,3cyclictest21-21ksoftirqd/111:30:131
457299201,15cyclictest31607-21ssh09:50:141
457299201,14cyclictest18305-21apt-get09:05:121
457299201,10cyclictest0-21swapper/111:27:581
457299200,16cyclictest15915-21timerandwakeup08:25:281
456899202,6cyclictest131rcu_preempt08:00:140
456899201,8cyclictest9070-21switchtime11:40:270
4568992014,3cyclictest12-21ksoftirqd/007:20:010
4568992013,4cyclictest12-21ksoftirqd/011:15:200
4568992012,4cyclictest12-21ksoftirqd/009:00:150
4568992012,4cyclictest12-21ksoftirqd/008:15:200
458899199,7cyclictest35966irq/58-eth010:00:283
458899193,11cyclictest0-21swapper/311:27:063
458899192,14cyclictest0-21swapper/307:10:123
458899192,11cyclictest0-21swapper/311:50:063
458899192,11cyclictest0-21swapper/310:50:043
4588991911,4cyclictest0-21swapper/311:18:283
4588991910,6cyclictest0-21swapper/309:15:303
458099197,10cyclictest32436-21kthreadcore12:30:202
458099193,13cyclictest21297-21kthreadcore08:00:202
458099192,8cyclictest27453-21df_inode12:10:132
458099192,8cyclictest19536-21irqcore07:25:192
458099192,3cyclictest0-21swapper/211:05:132
458099191,9cyclictest2672-21fschecks_time08:15:162
458099191,9cyclictest13651-21fschecks_time09:00:152
458099191,11cyclictest23701-21ps09:10:192
458099191,10cyclictest8564-21head07:10:282
458099191,10cyclictest18587-21idleruntime12:05:172
457299190,9cyclictest0-21swapper/107:25:261
457299190,7cyclictest7957-21/usr/sbin/munin12:35:121
457299190,7cyclictest7957-21/usr/sbin/munin12:35:121
457299190,13cyclictest22299-21timerandwakeup09:25:301
456899199,2cyclictest12-21ksoftirqd/008:30:010
456899199,2cyclictest12-21ksoftirqd/007:55:220
456899197,3cyclictest12-21ksoftirqd/007:40:150
456899191,5cyclictest12-21ksoftirqd/010:53:470
4568991913,2cyclictest12-21ksoftirqd/008:35:220
456899190,16cyclictest14446-21ls07:20:160
458899189,6cyclictest0-21swapper/309:21:103
458899188,8cyclictest0-21swapper/310:05:333
458899188,8cyclictest0-21swapper/309:38:573
458899183,10cyclictest0-21swapper/312:09:063
458899182,10cyclictest0-21swapper/308:50:143
458899182,10cyclictest0-21swapper/308:40:143
458899182,10cyclictest0-21swapper/308:40:143
458899182,10cyclictest0-21swapper/308:35:153
458899182,10cyclictest0-21swapper/308:30:143
458899182,10cyclictest0-21swapper/308:00:143
458899182,10cyclictest0-21swapper/307:35:143
458899182,10cyclictest0-21swapper/307:30:133
458899181,13cyclictest0-21swapper/310:25:163
458899181,13cyclictest0-21swapper/309:10:013
458899181,12cyclictest0-21swapper/308:05:293
458899181,12cyclictest0-21swapper/308:05:293
4588991810,6cyclictest0-21swapper/312:22:443
4588991810,6cyclictest0-21swapper/310:43:413
458899180,13cyclictest0-21swapper/309:55:133
457299186,8cyclictest21471-21apt-get10:20:141
457299183,10cyclictest131rcu_preempt11:55:141
457299182,9cyclictest6121-21sh12:15:211
457299181,7cyclictest20804-21irqcore08:00:181
457299181,6cyclictest29241-21irqrtprio07:35:321
457299181,2cyclictest131rcu_preempt09:13:081
4572991812,4cyclictest0-21swapper/110:45:141
4572991812,4cyclictest0-21swapper/109:30:141
4572991812,4cyclictest0-21swapper/107:15:171
4572991811,4cyclictest0-21swapper/109:00:151
4572991810,5cyclictest24539-21switchtime10:55:281
457299180,2cyclictest131rcu_preempt07:20:211
457299180,12cyclictest15425-21ls07:55:021
456899185,10cyclictest12-21ksoftirqd/008:40:140
456899185,10cyclictest12-21ksoftirqd/008:40:140
456899182,11cyclictest0-21swapper/007:45:140
4568991812,3cyclictest12-21ksoftirqd/008:45:160
4568991812,2cyclictest12-21ksoftirqd/008:30:270
458899179,6cyclictest0-21swapper/312:28:253
458899179,6cyclictest0-21swapper/310:31:243
458899179,6cyclictest0-21swapper/310:12:063
458899172,10cyclictest0-21swapper/309:00:133
458899171,10cyclictest0-21swapper/307:40:143
457299179,6cyclictest1-21systemd08:40:441
457299179,6cyclictest1-21systemd08:40:441
457299174,7cyclictest0-21swapper/108:30:131
457299173,3cyclictest131rcu_preempt08:45:141
457299171,3cyclictest131rcu_preempt08:10:161
457299171,13cyclictest1478-21sed07:40:181
457299170,10cyclictest8677-21find08:55:131
456899171,8cyclictest7569-21fschecks_time08:20:150
4568991712,3cyclictest12-21ksoftirqd/007:10:190
4568991711,3cyclictest12-21ksoftirqd/008:55:220
4568991710,4cyclictest12-21ksoftirqd/007:25:170
456899170,10cyclictest3984-21cut08:50:150
458899162,7cyclictest0-21swapper/307:50:133
458099164,5cyclictest4586-21latency_hist07:10:022
4572991611,3cyclictest0-21swapper/107:10:141
4588991510,3cyclictest0-21swapper/307:20:073
458899141,7cyclictest0-21swapper/308:25:223
457299147,5cyclictest4659-21sed07:10:021
456899140,11cyclictest4587-21latency_hist07:10:020
458899138,3cyclictest0-21swapper/307:57:483
458899133,6cyclictest0-21swapper/307:15:143
458899127,3cyclictest0-21swapper/307:25:253
458899122,6cyclictest0-21swapper/308:20:133
458899122,6cyclictest0-21swapper/307:45:143
458899111,7cyclictest0-21swapper/308:15:133
45889991,5cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional