You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-24 - 07:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Tue Mar 24, 2026 00:45:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2312299573,30cyclictest25654-21ssh21:55:122
2312299481,33cyclictest20695-21ssh23:40:142
2313099474,25cyclictest28366irq/58-eth021:45:153
23114994724,12cyclictest131rcu_preempt23:10:101
2310699474,5cyclictest131rcu_preempt20:00:130
2311499465,34cyclictest18874-21ssh22:45:151
2311499463,33cyclictest17120-21ssh23:20:131
23130994527,11cyclictest28366irq/58-eth021:55:133
2313099446,25cyclictest28366irq/58-eth023:35:113
23114994432,7cyclictest131rcu_preempt20:05:121
2313099434,21cyclictest28366irq/58-eth022:20:123
2310699434,31cyclictest4375-21ssh22:55:090
23130994218,19cyclictest28366irq/58-eth021:15:153
2311499413,7cyclictest131rcu_preempt00:05:151
2311499411,27cyclictest8975-21diskmemload00:35:151
2311499403,9cyclictest131rcu_preempt23:05:131
2313099393,22cyclictest28366irq/58-eth022:10:143
2312299393,19cyclictest18260-21ssh21:15:152
2313099375,21cyclictest28366irq/58-eth019:30:123
2313099373,18cyclictest28366irq/58-eth022:00:133
2312299372,29cyclictest13150-21ssh21:46:222
23122993710,22cyclictest15400-21ssh22:41:232
23114993714,18cyclictest24368-21ssh00:00:121
2310699372,29cyclictest29675-21ssh23:45:130
2312299367,10cyclictest1056-21kworker/2:100:10:152
2311499362,8cyclictest131rcu_preempt23:30:121
2310699361,22cyclictest13482-21ssh00:30:110
2313099353,20cyclictest28366irq/58-eth023:05:133
2313099352,21cyclictest28366irq/58-eth000:15:143
2313099352,20cyclictest28366irq/58-eth023:55:143
2311499357,18cyclictest9748-21ssh22:40:121
2311499351,24cyclictest21-21ksoftirqd/122:35:131
2313099343,20cyclictest28366irq/58-eth023:00:143
2313099342,23cyclictest28366irq/58-eth000:10:143
2312299342,19cyclictest21412-21ssh22:27:242
2310699344,19cyclictest13003-21ssh21:30:150
23106993417,10cyclictest8975-21diskmemload22:47:230
2313099337,16cyclictest0-21swapper/323:10:123
2312299332,18cyclictest21514-21ssh23:04:242
2311499331,23cyclictest73-21kswapd000:30:121
2310699331,28cyclictest3970-21ssh21:25:130
2310699331,28cyclictest3970-21ssh21:25:120
2310699330,27cyclictest6329-21ssh23:50:130
2313099322,17cyclictest28366irq/58-eth021:35:133
2312299325,16cyclictest22346irq/40-dwc2_hso19:10:092
23122993220,8cyclictest27-21ksoftirqd/223:10:232
2311499324,19cyclictest11732-21rm23:35:131
2311499323,24cyclictest26180-21ssh23:25:131
23114993224,5cyclictest0-21swapper/121:10:161
23106993219,7cyclictest20089-21kworker/u8:121:55:120
23106993213,8cyclictest12-21ksoftirqd/020:35:150
23106993213,8cyclictest12-21ksoftirqd/020:35:150
2312299319,15cyclictest0-21swapper/220:10:052
2312299319,15cyclictest0-21swapper/220:10:042
2312299313,11cyclictest12878-21ssh22:23:002
2311499312,20cyclictest201rcuc/100:10:151
23114993121,6cyclictest131rcu_preempt21:35:141
2313099306,10cyclictest28366irq/58-eth023:15:123
2313099302,20cyclictest28366irq/58-eth021:00:133
2311499303,4cyclictest131rcu_preempt21:40:121
2311499302,19cyclictest13388-21ssh23:00:131
2310699302,22cyclictest7172-21ssh23:14:230
23106993013,13cyclictest16567-21ssh21:50:130
2310699301,18cyclictest27942-21diskmemload22:50:150
2313099298,13cyclictest28366irq/58-eth021:40:133
2313099293,16cyclictest28366irq/58-eth022:05:113
2313099292,14cyclictest28366irq/58-eth022:30:113
23130992915,5cyclictest28366irq/58-eth021:10:133
2312299293,21cyclictest0-21swapper/200:01:242
2312299283,20cyclictest0-21swapper/222:48:232
2312299282,17cyclictest22346irq/40-dwc2_hso22:15:152
2312299282,17cyclictest22346irq/40-dwc2_hso22:15:142
2312299282,11cyclictest0-21swapper/223:08:182
2312299282,11cyclictest0-21swapper/220:45:152
2312299281,6cyclictest22548-21ssh00:35:152
2312299281,10cyclictest1928066ssh21:50:212
23122992810,15cyclictest0-21swapper/221:25:152
23122992810,15cyclictest0-21swapper/221:25:152
2311499284,20cyclictest201rcuc/121:25:141
2311499284,20cyclictest201rcuc/121:25:131
2311499282,22cyclictest8975-21diskmemload22:28:001
2311499282,21cyclictest2477-21ssh22:16:231
2311499282,21cyclictest2477-21ssh22:16:231
2310699282,19cyclictest2185-21rm21:40:230
2313099273,16cyclictest0-21swapper/322:27:243
2312299272,15cyclictest20356-21ssh22:10:142
2312299271,17cyclictest2763-21apt-get23:30:132
2311499275,11cyclictest633-21Async12:11:531
23114992713,7cyclictest0-21swapper/119:50:111
2310699272,19cyclictest8975-21diskmemload22:33:230
2313099264,14cyclictest0-21swapper/322:15:153
2313099264,14cyclictest0-21swapper/322:15:143
2313099263,12cyclictest0-21swapper/320:05:123
2313099262,13cyclictest28366irq/58-eth000:00:113
23130992615,7cyclictest28366irq/58-eth023:25:133
2312299262,19cyclictest13344-21sh23:53:072
2312299262,16cyclictest22346irq/40-dwc2_hso19:15:142
2312299262,10cyclictest0-21swapper/219:55:132
2312299261,14cyclictest11734-21ssh23:35:122
2311499263,17cyclictest73-21kswapd022:10:151
2311499262,18cyclictest449-21ssh21:58:231
2310699264,4cyclictest131rcu_preempt00:35:150
2310699262,17cyclictest5951-21ssh22:20:140
2313099253,14cyclictest28366irq/58-eth020:30:143
2312299253,15cyclictest22346irq/40-dwc2_hso23:20:142
2312299252,8cyclictest4198-21ssh21:41:232
2312299252,16cyclictest22346irq/40-dwc2_hso21:20:132
2312299251,8cyclictest3287-21grep20:30:162
2312299251,21cyclictest9683-21ls21:10:192
2312299251,15cyclictest8108-21ssh23:15:142
2311499253,13cyclictest21-21ksoftirqd/119:10:101
2310699253,13cyclictest28032-21ssh00:20:120
2310699252,17cyclictest15342-21ssh23:55:150
23106992514,3cyclictest0-21swapper/000:05:150
23130992419,3cyclictest321rcuc/320:50:263
23130992414,8cyclictest0-21swapper/322:56:273
2312299243,11cyclictest22346irq/40-dwc2_hso20:20:142
2312299242,6cyclictest0-21swapper/222:50:052
2312299241,9cyclictest471-21ls22:35:012
2312299241,15cyclictest11323-21ssh22:05:122
2312299241,11cyclictest3227966ssh00:20:312
2311499242,18cyclictest13977-21ssh21:47:231
23114992410,12cyclictest21-21ksoftirqd/121:30:141
2313099233,15cyclictest0-21swapper/322:39:223
2313099233,14cyclictest28366irq/58-eth022:50:153
2313099232,15cyclictest0-21swapper/322:45:163
23130992318,3cyclictest321rcuc/323:20:123
23130992314,6cyclictest0-21swapper/321:50:313
2312299236,6cyclictest27-21ksoftirqd/223:55:132
2312299232,6cyclictest0-21swapper/222:04:232
2312299232,16cyclictest0-21swapper/200:28:362
2312299232,13cyclictest22346irq/40-dwc2_hso20:50:152
23122992312,5cyclictest27041-21ssh21:35:472
23122992311,5cyclictest20141-21ssh00:32:412
23122992310,10cyclictest0-21swapper/220:05:182
2310699237,13cyclictest22037-21ssh21:35:130
2310699231,17cyclictest18258-21sh21:15:150
2313099222,10cyclictest28366irq/58-eth000:30:113
23130992210,5cyclictest0-21swapper/300:35:153
2312299229,4cyclictest4204-21latency_hist21:05:012
2312299222,11cyclictest9970-21grep20:00:232
23122992211,9cyclictest7534-21ssh22:37:512
2312299221,10cyclictest1248-21fschecks_time00:05:172
2311499225,12cyclictest21-21ksoftirqd/100:24:231
2311499223,11cyclictest6323-21ssh23:50:131
2311499222,14cyclictest9453-21memory22:20:231
2311499222,14cyclictest20196-21ntp_192.168.11421:50:231
2310699224,11cyclictest2776-21ssh23:30:130
2310699223,14cyclictest8975-21diskmemload22:00:140
2310699221,9cyclictest6794-21cut00:25:230
23106992213,5cyclictest0-21swapper/021:20:140
2310699221,16cyclictest20104-21ls19:40:160
23106992210,3cyclictest8975-21diskmemload23:05:450
23130992115,4cyclictest321rcuc/323:54:543
2312299216,12cyclictest16296-21kthreadcore21:30:242
2312299215,7cyclictest29716-21wc19:50:162
2312299212,14cyclictest27-21ksoftirqd/219:30:122
23122992111,7cyclictest2944-21cron20:30:012
2312299211,17cyclictest24610-21latency_hist19:45:022
2312299211,16cyclictest924-21snmpd22:55:122
2312299211,15cyclictest810-21ls23:45:232
2312299211,12cyclictest21380-21fschecks_time20:15:172
2312299211,11cyclictest19238-21fschecks_time00:15:172
2312299211,10cyclictest29368-21latency_hist19:50:012
2311499219,9cyclictest20683-21apt-get23:40:141
23114992113,5cyclictest27704-21ssh22:30:251
23114992111,7cyclictest11257-21cron20:05:011
2310699219,8cyclictest0-21swapper/021:10:130
2310699211,16cyclictest21217-21switchtime20:45:300
2310699211,15cyclictest567-21ls19:20:140
2313099203,12cyclictest0-21swapper/323:43:253
2313099203,11cyclictest0-21swapper/319:15:143
2313099203,10cyclictest28366irq/58-eth020:15:143
23130992015,3cyclictest321rcuc/321:26:263
23130992015,3cyclictest321rcuc/321:26:253
23130992014,4cyclictest0-21swapper/300:26:253
23130992014,4cyclictest0-21swapper/300:21:263
23130992013,5cyclictest0-21swapper/321:21:073
23130992013,4cyclictest0-21swapper/323:30:123
2313099201,11cyclictest28366irq/58-eth022:40:133
2312299203,12cyclictest22346irq/40-dwc2_hso19:35:142
23122992014,4cyclictest0-21swapper/223:25:142
2312299201,11cyclictest1527-21kthreadcore19:20:192
2312299201,10cyclictest5635-21irqcore19:25:172
2311499205,12cyclictest27924-21ssh22:50:121
2311499202,3cyclictest131rcu_preempt23:45:151
2311499201,2cyclictest131rcu_preempt20:40:101
23114992012,5cyclictest0-21swapper/122:00:181
23114992011,5cyclictest8971-21latency_hist21:10:031
2311499201,15cyclictest22378-21apt-get20:50:141
2311499200,17cyclictest18396-21cut21:15:171
2310699208,3cyclictest131rcu_preempt00:10:270
23106992012,5cyclictest15301-21ntp_192.168.11423:35:240
2310699201,16cyclictest14963-21sed22:05:220
2310699201,13cyclictest18988-21sh00:15:140
2313099193,11cyclictest0-21swapper/323:49:233
23130991914,3cyclictest321rcuc/319:10:143
23130991913,4cyclictest0-21swapper/321:31:263
23130991913,4cyclictest0-21swapper/300:05:313
2313099191,13cyclictest0-21swapper/321:05:233
2313099191,12cyclictest0-21swapper/320:40:203
2312299196,6cyclictest28008-21cut20:55:192
2312299191,9cyclictest8039-21cpuspeed20:35:162
2312299191,9cyclictest8039-21cpuspeed20:35:152
2312299191,9cyclictest4529-21fschecks_count21:05:152
2312299191,15cyclictest12968-21fschecks_time20:40:142
2311499198,8cyclictest1-21systemd23:58:081
2311499197,9cyclictest31019-21kthreadcore19:15:221
2311499192,14cyclictest28631-21kthreadcore20:55:211
2311499191,9cyclictest697-21fschecks_time19:20:151
2311499191,9cyclictest131rcu_preempt00:16:211
23114991912,4cyclictest0-21swapper/100:25:141
2310699198,8cyclictest817-1kworker/u9:022:10:300
2310699194,10cyclictest0-21swapper/019:45:150
2310699191,9cyclictest31732-21ssh22:15:220
2310699191,9cyclictest31732-21ssh22:15:210
2310699191,2cyclictest131rcu_preempt20:55:220
23106991912,4cyclictest4300-21missed_timers22:35:240
23106991911,5cyclictest15393-21kthreadcore23:00:230
23130991812,4cyclictest321rcuc/320:00:243
23130991812,4cyclictest0-21swapper/320:55:263
2312299181,8cyclictest23143-21latency_hist19:10:022
2311499186,9cyclictest3498-21sh21:25:021
2311499181,15cyclictest11882-21kthreadcore19:30:181
23114991811,5cyclictest0-21swapper/120:30:251
2311499181,14cyclictest21305-21cut20:15:171
2311499181,13cyclictest5087-21irqstats22:55:171
2311499180,10cyclictest3084-21latency_hist20:30:021
2310699189,5cyclictest10517-21kthreadcore22:40:200
2310699188,8cyclictest848-21tail19:50:280
2310699186,9cyclictest8943-21awk21:05:320
2310699186,9cyclictest10595-21sh23:15:210
2310699182,11cyclictest0-21swapper/020:05:140
2310699182,11cyclictest0-21swapper/019:25:130
23106991811,4cyclictest6491-21idleruntime-cro20:00:010
23106991811,4cyclictest23640-21cron22:30:020
2310699181,14cyclictest17116-21apt-get23:20:140
2310699181,13cyclictest8280-21irqstats21:45:180
2310699181,13cyclictest26178-21sh23:25:120
2313099172,9cyclictest0-21swapper/320:10:053
2313099172,9cyclictest0-21swapper/320:10:043
2313099172,9cyclictest0-21swapper/319:45:153
23130991712,3cyclictest0-21swapper/320:25:283
2313099171,13cyclictest0-21swapper/319:55:163
2313099171,10cyclictest0-21swapper/320:35:303
2313099171,10cyclictest0-21swapper/320:35:293
2311499177,8cyclictest17801-21cut20:45:181
2311499173,3cyclictest131rcu_preempt20:10:051
2311499173,3cyclictest131rcu_preempt20:10:051
2311499171,8cyclictest14918-21/usr/sbin/munin19:35:301
2311499171,8cyclictest131rcu_preempt19:25:201
2311499171,5cyclictest8575-21irqrtprio20:35:201
2311499171,5cyclictest8575-21irqrtprio20:35:191
23114991711,4cyclictest8110-21cpu23:15:141
2311499171,13cyclictest13296-21kthreadcore22:05:191
2310699179,5cyclictest22618-21kthreadcore20:15:210
2310699178,7cyclictest32200-21perl21:00:160
2310699173,11cyclictest26202-21fschecks_count20:20:170
2310699172,12cyclictest25478-21ssh00:00:190
2310699172,10cyclictest0-21swapper/020:30:150
2310699171,13cyclictest25800-21kthreadcore19:10:180
23106991710,5cyclictest0-21swapper/019:20:000
2313099169,5cyclictest0-21swapper/319:27:243
23130991611,3cyclictest0-21swapper/319:20:133
2311499164,9cyclictest25715-21kthreadcore19:45:211
2311499161,13cyclictest5101-21kthreadcore19:55:221
2311499161,10cyclictest29145-21kthreadcore20:20:231
2310699169,4cyclictest12619-21kthreadcore19:30:200
2310699166,7cyclictest13624-21cut20:40:170
2310699163,10cyclictest1040-21kthreadcore20:25:220
2310699162,11cyclictest19564-21sendmail19:40:010
2310699162,10cyclictest0-21swapper/023:40:140
2310699161,10cyclictest22776-21irqcore20:50:200
2313099152,9cyclictest0-21swapper/320:20:143
2313099152,9cyclictest0-21swapper/319:35:143
23130991510,3cyclictest0-21swapper/319:50:113
2311499151,10cyclictest32129-21cut21:00:171
2310699152,9cyclictest23050-21latency_hist19:10:020
2310699152,10cyclictest16414-21apt-get20:10:140
2310699152,10cyclictest16414-21apt-get20:10:130
2313099149,3cyclictest0-21swapper/319:40:033
2311499141,10cyclictest23142-21latency_hist19:10:011
2313099138,3cyclictest0-21swapper/320:45:043
2313099101,6cyclictest0-21swapper/319:10:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional