You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-30 - 11:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Thu Jul 30, 2026 00:45:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
6744996924,8cyclictest131rcu_preempt19:20:130
674499632,48cyclictest15084-21sshd23:10:140
6744995926,25cyclictest12-21ksoftirqd/019:55:130
675599582,43cyclictest27421-21ssh22:05:101
6744995523,12cyclictest131rcu_preempt20:45:150
674499535,37cyclictest7871-21ssh00:00:140
675599511,40cyclictest25343-21ssh21:10:151
675599492,42cyclictest0-21swapper/119:40:111
674499492,40cyclictest16908-21ssh00:05:170
675599481,40cyclictest10745-21apt-get20:55:121
674499473,8cyclictest131rcu_preempt19:30:120
674499472,32cyclictest618-21sshd23:20:130
674499463,35cyclictest29408-21ssh00:30:130
6775994528,11cyclictest35966irq/58-eth021:20:133
675599451,39cyclictest0-21swapper/122:40:141
675599441,32cyclictest995966ssh23:05:301
677599434,22cyclictest35966irq/58-eth023:45:133
675599438,23cyclictest20463-21ssh00:25:111
675599431,38cyclictest0-21swapper/119:25:111
675599431,36cyclictest0-21swapper/120:40:151
675599431,35cyclictest1038-21apt-get20:45:131
674499432,7cyclictest131rcu_preempt21:40:130
675599422,36cyclictest10888-21ssh21:20:121
675599422,29cyclictest4253-21ssh23:40:091
675599420,38cyclictest0-21swapper/120:50:131
677599413,23cyclictest35966irq/58-eth022:45:123
676499411,8cyclictest1838-21acpi21:15:112
675599412,37cyclictest0-21swapper/120:00:121
675599412,36cyclictest13282-21apt-get23:45:121
675599411,37cyclictest18681-21ssh23:30:111
675599411,35cyclictest16898-21apt-get00:05:171
675599411,2cyclictest0-21swapper/119:45:111
6744994125,10cyclictest16644-21ssh22:35:140
674499411,31cyclictest22025-21ssh22:20:140
676499401,13cyclictest9691-21sh23:25:122
675599401,3cyclictest0-21swapper/119:55:141
675599401,34cyclictest18338-21apt-get22:00:131
675599401,30cyclictest14688-21ssh21:40:141
675599401,27cyclictest24914-21diskmemload21:26:101
675599400,2cyclictest5563-21ssh21:35:101
674499402,29cyclictest20329-21ssh22:55:160
674499402,29cyclictest20329-21ssh22:55:160
676499391,8cyclictest31161-21kthreadcore21:30:202
676499391,15cyclictest20020-21fschecks_count21:25:162
675599392,28cyclictest9693-21ssh23:25:131
675599391,34cyclictest26226-21apt-get19:30:121
674499391,5cyclictest131rcu_preempt00:20:130
676499382,12cyclictest27726-21ssh23:35:142
676499381,9cyclictest25903-21memory20:00:212
676499380,11cyclictest15061-21missed_timers00:20:232
675599381,34cyclictest0-21swapper/123:50:121
675599381,30cyclictest22019-21apt-get22:20:131
675599380,23cyclictest5603-21ssh21:15:261
676499371,18cyclictest11210-21ssh22:50:142
676499371,14cyclictest19817-21kthreadcore22:00:192
675599372,33cyclictest0-21swapper/121:45:131
675599371,3cyclictest0-21swapper/120:15:131
675599371,33cyclictest7606-21apt-get22:30:131
675599371,33cyclictest0-21swapper/100:35:131
675599371,21cyclictest0-21swapper/122:12:101
675599370,25cyclictest4415-21kthreadcore00:15:221
6744993724,11cyclictest12-21ksoftirqd/022:50:120
677599363,23cyclictest35966irq/58-eth000:25:113
677599363,22cyclictest35966irq/58-eth021:45:123
676499368,4cyclictest32276-21switchtime20:40:302
676499362,11cyclictest29478-21ssh23:00:152
676499362,10cyclictest22255-21sshd23:50:142
676499361,12cyclictest31061-21ssh22:25:132
676499360,14cyclictest2181-21ssh00:15:022
675599361,31cyclictest2429-21grep19:35:291
675599361,2cyclictest12982-21apt-get19:50:151
675599361,2cyclictest0-21swapper/121:55:111
675599361,29cyclictest27480-21apt-get20:05:131
674499360,30cyclictest19840-21ssh21:25:120
677599353,19cyclictest35966irq/58-eth023:40:093
677599352,23cyclictest35966irq/58-eth022:20:153
676499351,9cyclictest3043-21kthreadcore23:20:212
676499351,14cyclictest135-21systemd-journal19:15:012
675599351,31cyclictest3492-21kthreadcore23:20:211
675599351,2cyclictest0-21swapper/119:10:111
675599351,29cyclictest15114-21wc23:10:141
675599351,29cyclictest14407-21cut22:15:191
675599351,29cyclictest0-21swapper/123:00:141
675599351,22cyclictest24892-21ssh22:39:391
675599351,22cyclictest0-21swapper/100:11:561
675599350,2cyclictest0-21swapper/121:00:121
675599350,2cyclictest0-21swapper/121:00:111
674499355,22cyclictest12-21ksoftirqd/020:25:130
674499352,25cyclictest25816-21ssh00:10:130
677599342,21cyclictest35966irq/58-eth022:35:143
676499341,9cyclictest6903-21grep19:40:262
676499341,9cyclictest19444-21ssh22:35:232
676499341,7cyclictest2243-21grep19:35:282
676499341,15cyclictest24429-21unixbench_multi22:55:332
676499341,15cyclictest24429-21unixbench_multi22:55:322
675599342,28cyclictest29039-21latency_hist23:00:031
675599342,28cyclictest29039-21latency_hist23:00:021
675599341,3cyclictest0-21swapper/120:30:131
675599341,30cyclictest0-21swapper/122:45:151
675599341,2cyclictest0-21swapper/121:50:091
675599341,29cyclictest1315-21cut23:55:231
675599341,22cyclictest3401-21ssh23:39:271
675599340,2cyclictest0-21swapper/120:10:131
675599340,22cyclictest4447-21ssh00:33:381
676499338,4cyclictest0-21swapper/219:30:312
676499331,14cyclictest21213-21kthreadcore00:25:192
676499331,13cyclictest18836-21latency_hist20:30:012
675599331,2cyclictest11526-21apt-get00:20:141
675599331,28cyclictest11519-21ntp_192.168.11400:00:271
675599330,29cyclictest11183-21ssh22:50:121
677599321,20cyclictest35966irq/58-eth022:25:123
676499322,11cyclictest0-21swapper/220:35:152
676499321,9cyclictest20424-21kthreadcore19:55:222
676499321,8cyclictest11425-21missed_timers00:00:272
676499321,14cyclictest21045-21apt-key19:25:012
676499321,14cyclictest14419-21ssh21:40:022
676499321,13cyclictest9500-21fschecks_time21:55:152
676499321,12cyclictest17312-21cat22:15:302
676499321,10cyclictest15442-21kthreadcore19:50:222
675599321,2cyclictest14238-21apt-get20:25:121
675599321,27cyclictest27694-21ntp_states23:15:241
675599321,27cyclictest19880-21ntpq19:20:241
675599321,27cyclictest13059-21grep20:20:281
677599315,16cyclictest35966irq/58-eth023:00:153
677599312,19cyclictest35966irq/58-eth022:15:123
677599311,19cyclictest35966irq/58-eth000:30:143
676499319,7cyclictest0-21swapper/200:30:032
676499312,21cyclictest27346irq/40-dwc2_hso21:00:122
676499312,21cyclictest27346irq/40-dwc2_hso21:00:122
676499311,25cyclictest23991-21ssh23:15:102
675599312,23cyclictest31033-21apt-get22:25:131
675599311,25cyclictest5390-21ls21:35:021
674499311,22cyclictest27412-21ssh22:05:100
676499302,18cyclictest27-21ksoftirqd/221:50:102
676499302,11cyclictest0-21swapper/222:10:122
676499301,9cyclictest9772-21ntp_states23:05:282
675599301,24cyclictest20578-21basename21:05:171
675599301,24cyclictest11635-21/usr/sbin/munin19:15:261
675599300,26cyclictest27408-21sendmail_mailtr20:35:301
674499302,22cyclictest30158-21ssh23:54:130
6744993013,7cyclictest131rcu_preempt22:25:120
677599292,16cyclictest0-21swapper/322:05:103
677599292,14cyclictest35966irq/58-eth022:40:143
676499293,21cyclictest0-21swapper/200:06:132
676499292,18cyclictest10919-21ssh21:20:142
677599281,15cyclictest35966irq/58-eth019:20:123
676499281,7cyclictest9871-21/usr/sbin/munin00:35:302
674499282,18cyclictest7621-21sh22:30:140
6775992722,3cyclictest321rcuc/321:30:323
677599272,22cyclictest35966irq/58-eth019:25:133
677599271,16cyclictest35966irq/58-eth000:20:123
676499274,6cyclictest0-21swapper/223:30:132
676499274,14cyclictest27346irq/40-dwc2_hso22:05:122
676499273,16cyclictest27346irq/40-dwc2_hso21:05:142
676499272,18cyclictest2447-21ssh00:15:142
676499272,17cyclictest31338-21ssh23:55:162
676499272,16cyclictest0-21swapper/219:15:152
676499272,10cyclictest0-21swapper/220:20:132
674499275,14cyclictest12-21ksoftirqd/020:10:130
677599269,4cyclictest35966irq/58-eth020:20:123
677599264,12cyclictest35966irq/58-eth022:10:133
677599263,15cyclictest0-21swapper/321:55:133
677599263,15cyclictest0-21swapper/321:50:133
6775992613,8cyclictest35966irq/58-eth023:55:173
676499264,18cyclictest27346irq/40-dwc2_hso23:40:112
676499262,19cyclictest0-21swapper/222:40:152
675599261,21cyclictest6816-21cat19:10:011
674499262,18cyclictest366-21/usr/sbin/munin21:50:120
677599254,12cyclictest35966irq/58-eth000:15:153
677599253,17cyclictest35966irq/58-eth021:35:113
677599253,14cyclictest0-21swapper/300:39:523
677599252,15cyclictest0-21swapper/323:54:133
677599252,13cyclictest35966irq/58-eth022:00:143
677599251,18cyclictest0-21swapper/323:10:143
676499251,5cyclictest25336-21rm21:10:142
676499243,14cyclictest27346irq/40-dwc2_hso21:45:142
676499241,21cyclictest0-21swapper/223:45:122
674499242,16cyclictest4042-21ssh22:10:120
6775992317,4cyclictest321rcuc/319:10:123
677599231,13cyclictest0-21swapper/321:15:113
6775992310,8cyclictest35966irq/58-eth000:05:183
676499232,6cyclictest0-21swapper/222:45:122
6764992313,7cyclictest24914-21diskmemload22:30:112
674499232,16cyclictest6059-21ssh23:05:140
677599226,10cyclictest35966irq/58-eth022:30:143
6775992216,4cyclictest321rcuc/320:00:113
6775992215,4cyclictest321rcuc/319:15:253
676499221,9cyclictest1885566ssh21:40:302
6764992211,2cyclictest0-21swapper/223:10:132
674499226,12cyclictest151rcuc/021:15:110
674499222,15cyclictest2197-21ssh22:45:140
6744992210,7cyclictest18341-21ssh22:00:110
677599213,11cyclictest0-21swapper/323:30:113
6775992116,3cyclictest321rcuc/323:35:043
6775992116,3cyclictest321rcuc/321:41:123
6775992115,3cyclictest321rcuc/300:13:063
676499212,13cyclictest27346irq/40-dwc2_hso20:30:132
676499212,10cyclictest854-21sshd19:25:122
676499211,17cyclictest32377-21fschecks_count20:10:152
676499211,11cyclictest8310-21grep20:15:282
676499211,10cyclictest23617-21cut22:20:212
674499219,6cyclictest0-21swapper/019:50:140
674499219,10cyclictest6423-21sed00:15:300
674499213,11cyclictest5996-21ssh00:35:140
674499211,11cyclictest9280-21/usr/sbin/munin21:55:300
6775992015,3cyclictest321rcuc/321:10:213
6775992015,3cyclictest321rcuc/300:00:153
6775992014,4cyclictest321rcuc/323:27:003
6775992012,6cyclictest0-21swapper/323:10:013
676499202,9cyclictest2977-21kthreadcore20:45:222
676499201,15cyclictest32173-21cat20:10:012
676499201,10cyclictest7562-21kthreadcore20:50:202
674499208,8cyclictest12-21ksoftirqd/021:10:130
674499205,12cyclictest131rcu_preempt19:25:210
674499204,10cyclictest131rcu_preempt20:55:120
674499203,13cyclictest0-21swapper/023:45:120
674499201,9cyclictest30029-21/usr/sbin/munin23:00:200
674499201,7cyclictest131rcu_preempt22:40:240
674499201,4cyclictest25578-1kworker/u9:223:41:320
6744992012,6cyclictest1-21systemd20:15:470
6744992012,6cyclictest0-21swapper/023:30:260
6744992012,5cyclictest0-21swapper/021:05:310
674499201,17cyclictest19205-21timerandwakeup21:00:280
674499201,17cyclictest19205-21timerandwakeup21:00:270
674499201,15cyclictest23763-21ls20:35:010
674499201,14cyclictest23945-21sh00:25:220
674499200,4cyclictest131rcu_preempt19:10:280
677599193,9cyclictest0-21swapper/323:21:233
677599192,11cyclictest0-21swapper/319:30:053
677599191,9cyclictest0-21swapper/320:15:123
6775991914,3cyclictest321rcuc/322:55:003
6775991914,3cyclictest321rcuc/320:40:223
6775991913,4cyclictest321rcuc/319:35:133
677599191,13cyclictest35966irq/58-eth021:25:133
677599191,13cyclictest0-21swapper/319:55:163
677599191,12cyclictest0-21swapper/320:10:283
676499193,13cyclictest10748-21/usr/sbin/munin20:55:132
676499191,15cyclictest8325-21grep19:45:142
674499197,9cyclictest9710-21grep21:35:280
674499192,11cyclictest28931-21ssh21:30:130
6744991913,4cyclictest0-21swapper/021:45:300
674499191,2cyclictest131rcu_preempt23:25:220
674499191,15cyclictest14145-21latency_hist20:25:010
674499191,14cyclictest30811-21apt-get23:55:020
677599183,10cyclictest0-21swapper/323:18:133
677599182,10cyclictest0-21swapper/321:05:143
677599182,10cyclictest0-21swapper/321:00:133
677599182,10cyclictest0-21swapper/321:00:133
677599182,10cyclictest0-21swapper/320:45:143
677599182,10cyclictest0-21swapper/320:25:133
677599182,10cyclictest0-21swapper/319:50:153
677599182,10cyclictest0-21swapper/319:45:123
6775991812,4cyclictest0-21swapper/322:59:263
6775991812,4cyclictest0-21swapper/322:59:253
677599181,13cyclictest0-21swapper/320:30:283
677599181,13cyclictest0-21swapper/319:40:283
674499181,9cyclictest23834-21/usr/sbin/munin20:35:310
674499181,6cyclictest27165-21apt-get20:05:040
674499181,2cyclictest131rcu_preempt21:20:330
6744991812,4cyclictest0-21swapper/020:50:270
6744991812,4cyclictest0-21swapper/020:00:270
674499181,13cyclictest27368-21kthreadcore23:15:210
6775991711,4cyclictest0-21swapper/320:50:153
6775991711,4cyclictest0-21swapper/320:05:163
674499174,10cyclictest2193-21switchtime19:35:280
674499171,9cyclictest12598-21kthreadcore19:15:220
674499171,14cyclictest28138-21irqcore23:35:180
674499170,2cyclictest131rcu_preempt22:15:130
6775991611,3cyclictest321rcuc/320:35:233
674499161,13cyclictest10608-21kthreadcore19:45:210
674499160,6cyclictest3530-21fschecks_count19:40:150
677599158,5cyclictest321rcuc/320:55:113
676499151,12cyclictest6728-21cyclictest19:10:012
674499157,5cyclictest0-21swapper/020:40:230
674499131,9cyclictest6766-21latency_hist19:10:020
67759991,6cyclictest0-21swapper/319:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional