You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-24 - 22:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack7slot3s.osadl.org (updated Mon Nov 24, 2025 12:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2588521800,5sleep11258099cyclictest10:00:381
2588521800,5sleep11258099cyclictest10:00:381
2411221260,2sleep3381ktimersoftd/307:40:283
1257999931,39cyclictest10285-21grep10:25:390
150022890,5sleep314981-21ssh12:05:363
1257999882,41cyclictest0-21swapper/010:50:250
81912830,2sleep00-21swapper/008:25:330
12579998016,45cyclictest28320-21unixbench_singl07:50:350
16672780,2sleep10-21swapper/109:25:281
1257999783,56cyclictest0-21swapper/007:10:100
25672770,5sleep01257999cyclictest08:10:280
107602710,6sleep10-21swapper/107:05:181
201642690,1sleep20-21swapper/209:00:252
12579996821,32cyclictest394-21sh12:30:360
1258199658,5cyclictest7850irq/56-dwc_otg_07:50:352
1258199654,13cyclictest7850irq/56-dwc_otg_07:35:222
1258199652,6cyclictest7850irq/56-dwc_otg_12:25:342
1258199651,8cyclictest10287-21head10:25:392
1258199642,19cyclictest7850irq/56-dwc_otg_12:20:212
1257999642,47cyclictest0-21swapper/009:10:370
270222630,2sleep20-21swapper/211:36:002
62912620,2sleep10-21swapper/111:55:071
1258199628,4cyclictest0-21swapper/211:42:182
1258199628,4cyclictest0-21swapper/211:42:182
12581996222,3cyclictest7850irq/56-dwc_otg_11:10:112
12581996222,3cyclictest7850irq/56-dwc_otg_11:10:112
1257999622,53cyclictest0-21swapper/012:36:150
1257999621,45cyclictest1231-21tail08:05:370
1258199617,4cyclictest7850irq/56-dwc_otg_08:35:192
12581996119,5cyclictest17650-21sh12:10:282
1257999612,47cyclictest15327-21sshd11:20:280
1257999612,41cyclictest0-21swapper/011:10:230
1257999612,41cyclictest0-21swapper/011:10:230
1257999611,44cyclictest0-21swapper/008:00:250
1257999611,38cyclictest27504-21fschecks_time09:15:250
1257999611,38cyclictest0-21swapper/007:30:250
1258199606,10cyclictest7850irq/56-dwc_otg_10:20:072
1258199603,4cyclictest7750irq/56-dwc_otg12:00:232
1257999602,44cyclictest0-21swapper/009:50:260
1257999602,42cyclictest22698-21cat07:35:340
1257999601,7cyclictest0-21swapper/009:30:330
1257999601,7cyclictest0-21swapper/009:30:330
1257999593,50cyclictest0-21swapper/009:00:180
1257999593,35cyclictest0-21swapper/010:20:070
1257999591,41cyclictest0-21swapper/009:55:220
1258199585,4cyclictest0-21swapper/210:10:292
1258199582,14cyclictest0-21swapper/209:50:132
1258099583,9cyclictest7750irq/56-dwc_otg10:55:201
1257999588,34cyclictest0-21swapper/008:35:210
1257999586,12cyclictest0-21swapper/007:45:200
1257999583,39cyclictest18582-21unixbench_multi12:10:380
1257999582,6cyclictest1035-21latency_hist09:25:080
1257999582,41cyclictest0-21swapper/010:30:360
1258199573,4cyclictest0-21swapper/210:50:372
1258199571,15cyclictest17494-21runrttasks09:49:552
1257999571,45cyclictest14377-21memory12:05:280
1257999571,42cyclictest21700-21tr12:15:310
1257999571,39cyclictest28926-21grep12:25:340
1258299561,6cyclictest7950irq/56-dwc_otg_09:40:203
1258199561,14cyclictest0-21swapper/211:35:072
12579995637,9cyclictest23527-21diskmemload11:00:070
12579995636,5cyclictest7850irq/56-dwc_otg_11:40:460
12579995636,5cyclictest7850irq/56-dwc_otg_11:40:460
1257999562,41cyclictest16861-21ssh10:35:330
1257999561,43cyclictest15223-21/usr/sbin/munin07:15:340
1257999561,40cyclictest7064-21ssh11:55:290
1258199557,14cyclictest7850irq/56-dwc_otg_10:20:272
1258199552,11cyclictest0-21swapper/209:25:402
1257999551,40cyclictest8108-21wget09:35:240
1257999551,39cyclictest0-21swapper/008:45:210
1257999551,38cyclictest25273-21modprobe11:35:070
1258199548,10cyclictest0-21swapper/211:49:002
1258199541,6cyclictest11662-21fschecks_time11:15:252
1257999541,40cyclictest0-21swapper/008:15:280
1257999541,40cyclictest0-21swapper/008:15:280
12579995413,30cyclictest32625-21load11:45:300
22602530,1sleep10-21swapper/111:48:591
1258199531,14cyclictest0-21swapper/209:55:252
12579995318,6cyclictest0-21swapper/007:40:180
1257999531,40cyclictest0-21swapper/011:10:070
1257999531,37cyclictest19205-21sed08:55:360
1257999531,34cyclictest0-21swapper/010:10:290
1258199523,6cyclictest7750irq/56-dwc_otg09:15:322
1258199522,9cyclictest0-21swapper/211:03:182
1258199521,6cyclictest7920-21apt-get11:10:072
1257999525,33cyclictest16579-21aten_rbpower_po08:50:190
1257999525,33cyclictest16579-21aten_rbpower_po08:50:190
1257999525,30cyclictest0-21swapper/008:30:330
12582995124,13cyclictest26421-21ssh10:50:183
1257999514,35cyclictest23223-21sort10:45:260
1257999514,35cyclictest23223-21sort10:45:260
1257999511,34cyclictest18821-21irqstats11:25:300
1257999511,34cyclictest0-21swapper/009:41:460
1257999511,11cyclictest0-21swapper/012:20:190
1258199503,17cyclictest0-21swapper/207:10:102
1257999502,36cyclictest0-21swapper/011:03:180
1257999501,37cyclictest19376-21/usr/sbin/munin10:40:320
1257999501,35cyclictest24582-21acpi10:00:190
1257999501,35cyclictest24582-21acpi10:00:190
1257999501,34cyclictest0-21swapper/009:49:220
1258099491,7cyclictest0-21swapper/108:50:171
1258099491,7cyclictest0-21swapper/108:50:171
1257999491,39cyclictest0-21swapper/011:15:260
1257999491,38cyclictest0-21swapper/010:10:070
1257999491,36cyclictest19205-21grep07:25:370
1257999491,35cyclictest0-21swapper/011:50:180
1257999481,38cyclictest0-21swapper/012:04:240
1258199477,8cyclictest0-21swapper/212:15:202
1258199477,25cyclictest0-21swapper/208:20:222
1257999471,32cyclictest8247-21taskset10:23:520
1258199461,5cyclictest7850irq/56-dwc_otg_08:25:192
1258099467,25cyclictest0-21swapper/109:10:221
12580994616,19cyclictest0-21swapper/109:35:281
1257999461,35cyclictest0-21swapper/008:20:280
12579994612,29cyclictest0-21swapper/007:20:190
1258199457,25cyclictest0-21swapper/207:25:222
1258199453,30cyclictest7750irq/56-dwc_otg10:10:072
1258099454,17cyclictest11-21rcu_preempt10:25:191
1258099452,23cyclictest0-21swapper/107:10:191
12579994518,13cyclictest7950irq/56-dwc_otg_09:10:070
1257999451,15cyclictest0-21swapper/011:35:230
1257999451,13cyclictest0-21swapper/009:25:300
1258299447,22cyclictest0-21swapper/307:15:213
1258199447,23cyclictest0-21swapper/210:40:132
1258099447,25cyclictest0-21swapper/107:40:211
1257999441,31cyclictest31331-21snmpd07:59:230
12582994316,12cyclictest23527-21diskmemload10:00:083
12582994315,5cyclictest16317-21latency_hist08:50:083
1258299431,27cyclictest27967-21ssh10:05:223
1258199437,24cyclictest0-21swapper/211:20:212
1258199436,23cyclictest0-21swapper/207:15:212
1258099434,9cyclictest7750irq/56-dwc_otg10:20:191
12580994317,8cyclictest0-21swapper/112:05:331
12580994310,25cyclictest0-21swapper/110:45:211
12580994310,25cyclictest0-21swapper/110:45:211
121102432,35sleep00-21swapper/007:07:320
308892420,2sleep30-21swapper/309:20:273
1258299427,24cyclictest0-21swapper/307:20:193
1258199427,25cyclictest0-21swapper/209:20:212
1258199427,20cyclictest0-21swapper/208:50:192
1258199427,20cyclictest0-21swapper/208:50:192
12580994217,6cyclictest31708-21idleruntime-cro11:45:081
12580994217,6cyclictest31708-21idleruntime-cro11:45:081
1257999421,33cyclictest13643-21ps08:40:350
1258099417,24cyclictest0-21swapper/112:30:201
1258099417,22cyclictest0-21swapper/107:30:131
279132400,2sleep30-21swapper/311:38:353
1258199406,22cyclictest0-21swapper/210:00:222
1258199406,22cyclictest0-21swapper/210:00:222
1258099401,8cyclictest0-21swapper/107:20:191
1258299397,23cyclictest0-21swapper/307:30:213
1258199397,20cyclictest0-21swapper/207:30:142
1258099397,27cyclictest7950irq/56-dwc_otg_12:10:211
12582993818,5cyclictest1120-21runrttasks09:28:113
12582993813,7cyclictest126422chrt07:10:213
1258299381,2cyclictest0-21swapper/311:05:223
12580993821,4cyclictest20150-21ntp_offset10:40:321
1258099381,7cyclictest21546-21fschecks_time09:55:261
100772380,1sleep10-21swapper/111:11:481
100772380,1sleep10-21swapper/111:11:481
12582993713,5cyclictest19056-21grep07:25:353
1258199376,21cyclictest7750irq/56-dwc_otg11:55:082
1258099372,10cyclictest7750irq/56-dwc_otg08:10:191
12580993710,15cyclictest0-21swapper/112:35:291
310052360,3sleep131007-21cat09:20:291
242692360,11sleep3381ktimersoftd/312:20:053
12582993615,5cyclictest7750irq/56-dwc_otg09:45:393
12582993610,5cyclictest1055-21timerandwakeup08:05:353
1258199366,15cyclictest0-21swapper/210:35:332
1258199364,19cyclictest7750irq/56-dwc_otg08:57:302
1258099361,25cyclictest0-21swapper/109:50:221
1258299356,4cyclictest7950irq/56-dwc_otg_10:32:273
1258299356,20cyclictest18330-21ssh11:25:223
1258299355,17cyclictest8427-21ssh09:35:283
12582993517,5cyclictest2243-21sh11:48:573
12582993516,5cyclictest381ktimersoftd/310:40:313
1258199357,6cyclictest1120-21runrttasks09:36:592
1258199354,21cyclictest7850irq/56-dwc_otg_08:50:082
1258199354,20cyclictest23527-21diskmemload12:38:112
1258199351,6cyclictest29968-21wget10:55:212
1258099356,6cyclictest0-21swapper/111:30:291
1258099353,22cyclictest0-21swapper/112:17:561
12580993520,4cyclictest23-21ksoftirqd/111:05:371
12580993516,5cyclictest30801-21aten_rbpower_en08:00:211
1258299346,3cyclictest0-21swapper/311:57:133
12582993415,5cyclictest29947-21aten_rbpower_en10:55:213
12582993413,7cyclictest0-21swapper/312:01:443
1258299341,2cyclictest0-21swapper/309:14:543
1258299341,19cyclictest0-21swapper/311:10:123
1258299341,19cyclictest0-21swapper/311:10:123
12582993411,6cyclictest308612chrt11:42:253
12582993411,6cyclictest308612chrt11:42:253
1258199346,14cyclictest7850irq/56-dwc_otg_11:55:202
1258199345,14cyclictest7850irq/56-dwc_otg_09:10:372
1258199342,22cyclictest29854-21ls07:55:332
12581993411,4cyclictest0-21swapper/210:45:342
12581993411,4cyclictest0-21swapper/210:45:342
1258099348,17cyclictest0-21swapper/110:54:041
1258099347,7cyclictest7750irq/56-dwc_otg11:20:071
1258099345,6cyclictest7750irq/56-dwc_otg11:40:081
1258099342,17cyclictest0-21swapper/110:05:321
12580993416,5cyclictest26844-21sshd07:49:271
12580993411,12cyclictest10731-21latency_hist08:35:081
12580993410,5cyclictest7750irq/56-dwc_otg10:15:201
12580993410,5cyclictest0-21swapper/109:40:341
1258299331,7cyclictest1795150rm10:37:153
1258199338,12cyclictest0-21swapper/209:30:382
1258199338,12cyclictest0-21swapper/209:30:382
1258199333,21cyclictest7850irq/56-dwc_otg_11:25:332
12581993314,11cyclictest23527-21diskmemload12:07:322
12581993311,5cyclictest0-21swapper/207:45:232
1258099336,4cyclictest7750irq/56-dwc_otg08:20:261
1258099335,6cyclictest29717-21seq12:26:261
1258099334,7cyclictest7750irq/56-dwc_otg09:05:241
1258099334,4cyclictest7850irq/56-dwc_otg_11:55:201
12580993322,7cyclictest4798-21sshd09:30:271
12580993322,7cyclictest4798-21sshd09:30:271
1258099331,22cyclictest15641-21timerandwakeup08:45:391
12580993311,16cyclictest0-21swapper/112:00:191
1258299328,7cyclictest0-21swapper/310:14:043
1258299328,3cyclictest0-21swapper/312:32:493
1258299327,15cyclictest0-21swapper/308:55:203
1258299326,6cyclictest27720-21kworker/u8:408:20:363
1258299326,5cyclictest7950irq/56-dwc_otg_08:50:233
1258299326,5cyclictest7950irq/56-dwc_otg_08:50:233
1258299326,14cyclictest0-21swapper/311:50:213
1258299321,23cyclictest18035-21ssh11:25:073
12582993210,3cyclictest0-21swapper/310:20:233
1258199323,17cyclictest0-21swapper/210:30:222
1258199322,20cyclictest7750irq/56-dwc_otg08:18:102
1258199322,20cyclictest7750irq/56-dwc_otg08:18:102
1258199321,24cyclictest0-21swapper/207:44:102
1258099327,15cyclictest0-21swapper/108:40:231
1258099325,6cyclictest7950irq/56-dwc_otg_10:10:241
1258099322,13cyclictest7850irq/56-dwc_otg_07:55:241
12580993215,5cyclictest28629-21run-parts07:54:071
12580993211,4cyclictest0-21swapper/112:22:461
1258099321,13cyclictest7750irq/56-dwc_otg09:15:361
118702324,18sleep30-21swapper/307:05:373
1258299313,4cyclictest7950irq/56-dwc_otg_10:05:083
1258299313,4cyclictest7950irq/56-dwc_otg_10:05:083
12582993115,12cyclictest7950irq/56-dwc_otg_10:15:243
1258299311,27cyclictest7950irq/56-dwc_otg_12:35:203
1258199316,14cyclictest0-21swapper/209:05:222
1258199312,18cyclictest0-21swapper/208:30:342
1258199312,10cyclictest7850irq/56-dwc_otg_07:20:272
12581993116,4cyclictest1681-21cron08:10:072
12581993114,4cyclictest1120-21runrttasks12:30:552
1258099315,4cyclictest7850irq/56-dwc_otg_11:20:211
1258099312,20cyclictest0-21swapper/108:17:491
1258099312,20cyclictest0-21swapper/108:17:491
1258099311,18cyclictest7950irq/56-dwc_otg_09:45:201
12580993111,4cyclictest0-21swapper/110:30:321
12580993110,4cyclictest0-21swapper/110:39:081
1258299305,5cyclictest37-21rcuc/312:10:293
1258299305,5cyclictest0-21swapper/311:30:293
12582993023,5cyclictest1967-21aten_rbpower_cu08:10:203
12582993022,5cyclictest31331-21snmpd11:00:093
1258299301,6cyclictest7950irq/56-dwc_otg_08:35:083
1258299301,6cyclictest7950irq/56-dwc_otg_08:25:253
12582993014,10cyclictest381ktimersoftd/309:50:263
12582993013,6cyclictest24274-21sh10:45:483
12582993013,6cyclictest24274-21sh10:45:483
1258299301,17cyclictest0-21swapper/310:25:303
12582993010,6cyclictest21289-21uptime09:00:453
1258199302,5cyclictest0-21swapper/208:40:212
12581993010,4cyclictest0-21swapper/209:40:362
1258099307,14cyclictest0-21swapper/107:35:211
1258099303,5cyclictest7950irq/56-dwc_otg_09:00:351
1258099303,5cyclictest7750irq/56-dwc_otg11:00:271
1258099302,13cyclictest0-21swapper/108:26:441
12580993019,8cyclictest21652-21cron11:30:071
1258099301,8cyclictest18514-21sed07:25:281
1258299296,10cyclictest0-21swapper/312:29:403
1258299293,6cyclictest22850-21unixbench_multi07:35:353
1258299292,3cyclictest0-21swapper/309:17:033
1258299292,10cyclictest7950irq/56-dwc_otg_08:35:353
12582992917,6cyclictest381ktimersoftd/309:05:163
1258299291,14cyclictest0-21swapper/308:00:083
12582992910,4cyclictest0-21swapper/311:15:133
1258199292,18cyclictest7850irq/56-dwc_otg_08:02:162
1258199291,12cyclictest7850irq/56-dwc_otg_08:10:172
12580992920,6cyclictest18607-21fschecks_time08:55:251
1258299287,8cyclictest7950irq/56-dwc_otg_12:20:233
1258299282,5cyclictest7750irq/56-dwc_otg09:30:373
1258299282,5cyclictest7750irq/56-dwc_otg09:30:373
12582992810,13cyclictest271582sleep307:50:183
1258099287,19cyclictest0-21swapper/108:35:211
1258099281,21cyclictest470-21idleruntime08:05:271
1258299276,12cyclictest0-21swapper/307:45:203
1258299272,10cyclictest7950irq/56-dwc_otg_08:05:083
1258299271,6cyclictest4911-21sed08:15:363
1258299271,6cyclictest4911-21sed08:15:363
1258299271,4cyclictest7950irq/56-dwc_otg_08:40:243
12580992710,5cyclictest14978-21/usr/sbin/munin07:15:291
121422274,16sleep20-21swapper/207:07:592
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional