You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-27 - 16:07

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #4

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot4s.osadl.org (updated Sat Jun 27, 2026 12:45:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
755999602,41cyclictest11511-21ssh11:25:150
758299581,38cyclictest32416-21ssh10:25:152
757199562,35cyclictest14157-21ssh10:15:131
759899483,30cyclictest35966irq/58-eth011:20:143
757199482,39cyclictest18115-21ssh10:35:131
758299473,23cyclictest25690-21diskmemload10:40:122
7582994630,9cyclictest27-21ksoftirqd/209:55:122
759899443,23cyclictest35966irq/58-eth009:30:113
7582994420,11cyclictest27-21ksoftirqd/210:35:132
759899432,25cyclictest35966irq/58-eth009:10:153
758299421,19cyclictest20520-21ssh11:30:142
7559994234,6cyclictest32399-21apt-get10:25:150
7559994232,7cyclictest13000-21apt-get10:50:140
7559994222,15cyclictest0-21swapper/008:45:130
7559994216,11cyclictest28599-21ssh10:57:370
7559994215,10cyclictest3093-21/usr/sbin/munin12:30:320
7559994214,11cyclictest28772-21if_eth010:05:170
7559994212,12cyclictest24198-21df_inode11:50:130
758299413,28cyclictest16787-21ssh11:10:132
7559994133,5cyclictest18097-21apt-get10:35:120
7559994126,8cyclictest0-21swapper/009:20:140
7559994116,11cyclictest10635-21ssh09:19:110
7559994113,12cyclictest26333-21/usr/sbin/munin09:10:180
7559994113,11cyclictest25896-21tr10:20:190
7559994112,8cyclictest24649-21/usr/sbin/munin09:45:240
759899401,25cyclictest35966irq/58-eth012:00:143
7559994029,7cyclictest0-21swapper/011:45:140
7559994021,14cyclictest0-21swapper/009:00:130
7559994015,8cyclictest25690-21diskmemload12:15:130
7559994015,8cyclictest20589-21cpu11:30:160
7559994015,10cyclictest19986-21ssh10:16:370
7559994015,10cyclictest1858-21ssh10:43:020
7559994014,9cyclictest15105-21fschecks_count08:25:160
7559994014,9cyclictest13480-21ssh12:36:200
7559994014,8cyclictest0-21swapper/009:30:230
7559994014,11cyclictest3108-21ssh12:14:000
7559994013,9cyclictest25927-21ssh12:25:260
7559994013,10cyclictest22148-21apt-get07:25:120
757199391,32cyclictest752-21ssh11:55:131
7559993929,7cyclictest9778-21ssh12:00:120
7559993916,10cyclictest15023-21ssh09:39:460
7559993915,8cyclictest15108-21ssh10:31:560
7559993914,9cyclictest26479-21irqstats11:15:180
7559993914,12cyclictest592-21sed11:35:240
7559993913,10cyclictest25316-21cat09:25:300
7559993913,10cyclictest24362-21ssh12:06:200
759899383,21cyclictest35966irq/58-eth011:15:123
759899382,21cyclictest35966irq/58-eth009:15:123
757199381,30cyclictest15128-21ssh11:45:141
755999389,13cyclictest12458-21latency_hist07:15:020
7559993829,6cyclictest6112-21apt-get11:40:140
7559993816,8cyclictest25690-21diskmemload10:47:280
7559993815,8cyclictest24156-21ssh09:44:520
7559993815,7cyclictest22437-21ssh11:11:300
7559993814,14cyclictest25690-21diskmemload12:21:570
7559993813,9cyclictest14201-21grep08:55:220
7559993813,9cyclictest14201-21grep08:55:210
7559993812,9cyclictest2454-21apt-get11:20:140
7559993812,7cyclictest8760-21ntp_states10:10:250
7559993812,11cyclictest14414-21grep07:50:180
7559993812,10cyclictest8886-21ssh11:59:270
759899372,21cyclictest35966irq/58-eth010:55:143
758299372,27cyclictest29330-21usage_EthcTask09:45:302
758299372,18cyclictest19999-21usage_Anybus09:40:292
757199374,27cyclictest12998-21ssh10:50:131
7559993723,10cyclictest28217-21apt-get08:05:130
7559993717,12cyclictest25690-21diskmemload11:02:020
7559993713,10cyclictest14935-21tr07:15:210
7559993712,8cyclictest19500-21diskstats10:00:150
7559993712,12cyclictest13919-21ssh09:55:230
7559993712,10cyclictest13050-21ssh11:06:000
7559993711,6cyclictest25690-21diskmemload09:50:140
758299362,28cyclictest25690-21diskmemload10:23:352
7582993614,8cyclictest0-21swapper/208:35:152
7559993622,12cyclictest0-21swapper/007:45:120
7559993613,8cyclictest19213-21irqstats07:55:190
7559993613,7cyclictest7562-21ntp_192.168.11407:40:240
7559993612,11cyclictest9164-21tr08:50:230
758299353,9cyclictest0-21swapper/210:18:352
758299352,16cyclictest10928-21cat09:35:292
757199352,22cyclictest31074-21ssh11:00:141
7559993517,14cyclictest0-21swapper/008:00:120
7559993516,15cyclictest0-21swapper/007:35:150
7559993512,8cyclictest0-21swapper/009:05:310
7559993512,10cyclictest21337-21cat07:20:290
757199340,29cyclictest24084-21sh11:50:111
7559993423,7cyclictest5359-21apt-get08:15:120
7559993423,7cyclictest5359-21apt-get08:15:120
7559993418,13cyclictest0-21swapper/008:30:150
759899333,16cyclictest35966irq/58-eth012:25:133
7598993317,11cyclictest35966irq/58-eth010:30:143
758299322,25cyclictest29533-21ssh11:35:132
7559993222,6cyclictest0-21swapper/008:35:140
758299312,24cyclictest8856-21ssh10:45:342
758299312,21cyclictest15441-21ntp_192.168.11409:20:272
757199312,25cyclictest4354-21sh12:15:131
757199312,23cyclictest6139-21ssh11:40:151
757199311,24cyclictest20850-21ssh09:25:141
758299302,18cyclictest27346irq/40-dwc2_hso07:55:132
757199302,20cyclictest1690-21getconf09:30:271
755999309,7cyclictest0-21swapper/008:40:160
759899293,19cyclictest0-21swapper/309:35:293
759899293,17cyclictest0-21swapper/309:57:363
758299292,9cyclictest0-21swapper/211:17:352
7582992924,3cyclictest27-21ksoftirqd/207:25:132
758299292,21cyclictest27-21ksoftirqd/208:25:132
757199292,22cyclictest32385-21ssh11:17:361
757199292,19cyclictest7972-21ssh12:35:151
7571992914,6cyclictest21-21ksoftirqd/111:05:441
755999298,11cyclictest11897-21kthreadcore08:20:210
755999296,10cyclictest26939-21/usr/sbin/munin07:30:280
759899283,17cyclictest0-21swapper/309:45:313
759899283,16cyclictest0-21swapper/309:40:293
759899283,16cyclictest0-21swapper/309:23:353
758299282,10cyclictest24307-21ssh10:00:342
758299281,20cyclictest24100-21ssh11:50:122
7571992820,6cyclictest21-21ksoftirqd/109:05:141
7571992812,5cyclictest21-21ksoftirqd/112:00:181
757199281,20cyclictest248066ssh11:20:161
7571992811,9cyclictest21-21ksoftirqd/109:44:191
7571992811,5cyclictest21-21ksoftirqd/109:45:201
758299273,19cyclictest0-21swapper/209:53:352
758299272,19cyclictest0-21swapper/209:00:122
758299272,19cyclictest0-21swapper/208:50:152
758299272,15cyclictest27346irq/40-dwc2_hso12:05:152
7582992712,9cyclictest4356-21ssh12:15:132
757199274,5cyclictest21-21ksoftirqd/112:20:281
757199272,20cyclictest28446-21ssh10:05:141
757199272,19cyclictest27668-21ssh10:04:351
757199272,19cyclictest11793-21ssh09:20:151
757199271,22cyclictest20526-21sh11:30:141
7559992718,7cyclictest493-21sshd08:10:060
758299264,16cyclictest0-21swapper/208:05:122
758299263,18cyclictest0-21swapper/210:58:352
7582992611,8cyclictest27-21ksoftirqd/207:50:122
757199269,4cyclictest21-21ksoftirqd/112:05:151
757199268,12cyclictest21-21ksoftirqd/109:36:351
759899258,10cyclictest35966irq/58-eth010:15:143
759899256,15cyclictest35966irq/58-eth009:00:133
7598992513,8cyclictest35966irq/58-eth011:40:143
758299254,16cyclictest0-21swapper/211:40:352
758299252,18cyclictest27346irq/40-dwc2_hso11:05:142
7582992513,5cyclictest6071-21ssh09:34:532
7582992511,6cyclictest29005-21sh10:05:182
758299251,15cyclictest27346irq/40-dwc2_hso08:20:132
758299250,12cyclictest29531-21ls09:10:252
757199259,4cyclictest21-21ksoftirqd/107:35:151
757199258,8cyclictest21-21ksoftirqd/111:36:131
757199258,5cyclictest21-21ksoftirqd/110:11:011
757199256,12cyclictest131rcu_preempt09:58:021
757199255,4cyclictest131rcu_preempt12:10:201
757199251,19cyclictest23305-21ssh10:20:131
759899243,18cyclictest35966irq/58-eth011:35:133
7582992418,4cyclictest12400-21ssh11:25:212
7582992416,5cyclictest9417-21ssh12:00:022
7582992412,4cyclictest21598-21latency_hist10:55:012
7582992412,10cyclictest27-21ksoftirqd/212:10:132
758299241,14cyclictest20849-21sh09:25:142
757199249,4cyclictest21-21ksoftirqd/111:11:211
757199248,8cyclictest29493-21apt-get08:40:131
757199248,8cyclictest21-21ksoftirqd/109:50:351
757199248,7cyclictest21-21ksoftirqd/112:25:031
757199248,3cyclictest131rcu_preempt08:45:171
757199244,9cyclictest131rcu_preempt09:00:231
757199244,5cyclictest21-21ksoftirqd/110:25:191
757199243,5cyclictest21-21ksoftirqd/111:25:241
757199242,16cyclictest31323-21ssh12:30:141
7571992413,6cyclictest0-21swapper/107:30:121
757199241,18cyclictest9068-21ssh10:30:131
7571992411,4cyclictest21-21ksoftirqd/110:43:031
7559992412,8cyclictest7688-21latency_hist07:10:020
759899233,11cyclictest35966irq/58-eth012:15:133
7598992313,6cyclictest0-21swapper/310:00:133
758299237,6cyclictest27-21ksoftirqd/210:10:192
758299235,9cyclictest11057-21kthreadcore07:45:202
758299234,16cyclictest1642-21cron08:45:022
758299232,6cyclictest0-21swapper/211:48:352
7582992312,9cyclictest6334-21ssh11:03:512
757199239,8cyclictest21-21ksoftirqd/109:15:481
757199237,4cyclictest597-21df08:10:151
757199232,3cyclictest131rcu_preempt09:10:251
757199231,5cyclictest0-21swapper/107:15:161
7571992310,3cyclictest131rcu_preempt07:40:241
759899229,9cyclictest35966irq/58-eth012:35:153
759899226,12cyclictest35966irq/58-eth008:35:143
759899224,11cyclictest35966irq/58-eth012:30:163
759899223,13cyclictest35966irq/58-eth010:25:153
759899222,13cyclictest35966irq/58-eth009:25:143
758299223,16cyclictest0-21swapper/207:20:152
758299222,12cyclictest2902-21kthreadcore08:10:222
758299222,11cyclictest5582-21cut11:20:252
758299221,18cyclictest9803-21ssh12:00:142
758299221,18cyclictest2669-21ssh09:15:132
7582992211,7cyclictest26811-21usage_EthcTask12:25:322
758299221,16cyclictest7502-21grep07:40:232
758299221,15cyclictest27346irq/40-dwc2_hso08:45:142
757199229,3cyclictest131rcu_preempt08:30:211
757199228,4cyclictest21-21ksoftirqd/107:55:301
757199227,7cyclictest131rcu_preempt10:47:331
757199225,3cyclictest21-21ksoftirqd/108:35:251
757199222,4cyclictest21-21ksoftirqd/108:00:351
757199221,5cyclictest21-21ksoftirqd/108:15:241
757199221,5cyclictest21-21ksoftirqd/108:15:241
7571992211,4cyclictest21-21ksoftirqd/107:20:211
7571992210,4cyclictest21-21ksoftirqd/110:56:041
759899213,12cyclictest0-21swapper/311:49:353
758299213,9cyclictest9707-21kthreadcore12:35:222
758299213,14cyclictest27346irq/40-dwc2_hso08:15:132
758299213,14cyclictest27346irq/40-dwc2_hso08:15:132
758299211,17cyclictest12853-21mailstats10:30:262
758299211,10cyclictest3058-21head12:30:312
757199217,7cyclictest21-21ksoftirqd/108:05:271
757199210,8cyclictest10489-21/usr/sbin/munin08:50:301
759899209,8cyclictest0-21swapper/312:13:513
759899208,8cyclictest35966irq/58-eth011:05:153
759899207,7cyclictest35966irq/58-eth010:40:143
759899204,11cyclictest0-21swapper/309:50:313
759899201,13cyclictest0-21swapper/307:30:223
7598992010,8cyclictest0-21swapper/312:21:413
758299203,9cyclictest13064-21kthreadcore08:55:192
758299203,9cyclictest13064-21kthreadcore08:55:192
758299203,14cyclictest17319-21timerwakeupswit12:20:282
758299202,9cyclictest28577-21kthreadcore07:30:212
758299202,15cyclictest27232-21unixbench_multi08:00:282
758299201,9cyclictest579-21kthreadcore07:35:212
758299201,15cyclictest19905-21fschecks_count08:30:182
758299201,11cyclictest25765-21ssh09:10:022
758299201,10cyclictest17139-21latency_hist07:20:012
757199208,3cyclictest131rcu_preempt07:10:201
757199207,7cyclictest21-21ksoftirqd/107:45:181
757199203,8cyclictest21-21ksoftirqd/108:25:171
757199202,10cyclictest21-21ksoftirqd/107:25:181
757199201,4cyclictest0-21swapper/107:50:171
757199201,3cyclictest131rcu_preempt08:55:171
757199201,3cyclictest131rcu_preempt08:55:171
759899199,8cyclictest0-21swapper/312:06:343
759899199,8cyclictest0-21swapper/311:33:423
759899193,11cyclictest0-21swapper/311:11:353
759899193,11cyclictest0-21swapper/310:12:353
759899193,11cyclictest0-21swapper/310:05:353
7598991911,6cyclictest0-21swapper/310:45:093
759899191,13cyclictest0-21swapper/308:50:183
758299191,9cyclictest8302-21irqrtprio07:10:212
758299191,15cyclictest7635-21cat07:10:022
757199191,3cyclictest131rcu_preempt08:20:141
759899184,9cyclictest0-21swapper/310:50:133
759899184,4cyclictest0-21swapper/311:50:113
759899183,11cyclictest0-21swapper/310:35:353
759899183,11cyclictest0-21swapper/310:20:353
759899183,10cyclictest0-21swapper/311:29:353
759899182,10cyclictest0-21swapper/308:55:133
759899182,10cyclictest0-21swapper/308:55:133
759899182,10cyclictest0-21swapper/308:20:143
759899182,10cyclictest0-21swapper/308:15:153
759899182,10cyclictest0-21swapper/308:15:143
759899181,9cyclictest0-21swapper/307:40:033
759899181,13cyclictest0-21swapper/307:50:023
759899172,10cyclictest0-21swapper/311:00:143
759899172,10cyclictest0-21swapper/308:05:153
759899171,12cyclictest0-21swapper/311:55:293
759899162,10cyclictest0-21swapper/307:50:133
759899161,10cyclictest0-21swapper/309:05:193
757199169,5cyclictest21-21ksoftirqd/107:10:021
759899155,4cyclictest0-21swapper/308:40:143
759899152,9cyclictest0-21swapper/307:55:153
759899152,8cyclictest0-21swapper/308:25:143
759899151,11cyclictest0-21swapper/307:15:023
759899149,3cyclictest0-21swapper/308:30:273
759899149,3cyclictest0-21swapper/308:00:223
759899141,8cyclictest0-21swapper/307:25:133
759899141,7cyclictest0-21swapper/308:10:133
759899125,5cyclictest0-21swapper/307:15:223
759899123,7cyclictest0-21swapper/307:20:163
759899122,7cyclictest0-21swapper/308:45:173
759899121,5cyclictest0-21swapper/307:40:183
759899111,7cyclictest0-21swapper/307:10:023
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional