You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-12-01 - 22:00

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Mon Dec 01, 2025 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
207292990,2sleep0101ktimersoftd/012:25:110
262692920,8sleep0101ktimersoftd/009:30:120
325872870,2sleep30-21swapper/311:10:393
32362870,2sleep10-21swapper/112:00:531
32362870,2sleep10-21swapper/112:00:531
5152830,2sleep10-21swapper/107:05:311
1990997655,11cyclictest0-21swapper/010:15:240
199799721,59cyclictest28192-21ssh11:50:352
1990997215,30cyclictest13-21rcuc/009:45:240
199099701,33cyclictest0-21swapper/008:13:240
199799696,44cyclictest7950irq/56-dwc_otg_08:10:322
1990996943,7cyclictest7750irq/56-dwc_otg10:25:240
199099692,12cyclictest7850irq/56-dwc_otg_11:30:240
199799673,48cyclictest7750irq/56-dwc_otg09:23:402
199099677,53cyclictest0-21swapper/009:20:250
1990996718,27cyclictest0-21swapper/007:10:230
199799664,46cyclictest0-21swapper/210:21:402
199799661,44cyclictest1164-21tune2fs09:40:302
199799661,41cyclictest7850irq/56-dwc_otg_07:20:372
304092650,2sleep30-21swapper/311:54:193
199899653,45cyclictest3892-21aten_rbpower_po07:15:243
199799657,34cyclictest7850irq/56-dwc_otg_10:10:362
199799651,53cyclictest29237-21/usr/sbin/munin12:35:332
199799641,4cyclictest7850irq/56-dwc_otg_10:35:402
199799641,3cyclictest0-21swapper/209:15:232
314642630,2sleep30-21swapper/309:36:513
199899639,34cyclictest7950irq/56-dwc_otg_10:10:363
199899633,49cyclictest0-21swapper/310:30:243
199799636,47cyclictest7950irq/56-dwc_otg_10:02:352
199799634,45cyclictest7850irq/56-dwc_otg_12:15:412
199799632,3cyclictest0-21swapper/211:35:112
199799631,5cyclictest3593-21latency_hist10:30:112
199799631,45cyclictest0-21swapper/209:04:242
199799631,45cyclictest0-21swapper/209:04:242
1994996318,5cyclictest7950irq/56-dwc_otg_11:25:231
199099635,8cyclictest9-21ksoftirqd/008:45:250
1990996343,7cyclictest13060-21idleruntime-cro09:10:100
199899621,4cyclictest7750irq/56-dwc_otg09:45:403
199899621,43cyclictest24212-21fschecks_time08:10:303
199799626,43cyclictest7950irq/56-dwc_otg_11:00:362
199799621,50cyclictest28326-21latency_hist12:35:122
199799621,50cyclictest28326-21latency_hist12:35:122
1990996235,7cyclictest0-21swapper/012:05:240
199899611,38cyclictest0-21swapper/308:50:503
199799617,38cyclictest4804-21expr07:15:372
199799614,36cyclictest500-21ls09:40:112
199799614,32cyclictest7850irq/56-dwc_otg_07:10:262
199799612,48cyclictest0-21swapper/211:15:292
199899601,47cyclictest61-21kswapd010:55:243
199899601,47cyclictest11342-21chrt11:25:493
199899601,40cyclictest6562-21/usr/sbin/munin07:20:363
199899601,36cyclictest0-21swapper/311:15:233
199799607,34cyclictest7750irq/56-dwc_otg11:25:312
199799606,38cyclictest0-21swapper/209:30:282
199799603,41cyclictest0-21swapper/209:45:412
199799602,44cyclictest0-21swapper/212:27:212
199799601,47cyclictest20092-21ssh12:24:032
199799601,44cyclictest6794-21open_files08:50:502
199799601,44cyclictest0-21swapper/207:46:412
1994996016,18cyclictest13023-21kworker/u8:210:15:241
1990996028,16cyclictest13-21rcuc/008:40:220
1990996027,8cyclictest0-21swapper/008:35:250
199899591,43cyclictest24858-21seq09:26:463
199899591,42cyclictest3594-21latency_hist10:30:113
199899591,38cyclictest16497-21basename09:15:253
199799591,3cyclictest0-21swapper/210:55:382
199899584,33cyclictest7950irq/56-dwc_otg_11:00:373
199899582,43cyclictest7950irq/56-dwc_otg_12:24:053
199899581,43cyclictest7879-21sh10:35:323
199899581,34cyclictest14866-21wget07:45:313
199799582,40cyclictest9520-21cut12:10:252
199799581,41cyclictest7950irq/56-dwc_otg_07:39:132
1990995840,11cyclictest9953-21missed_timers07:30:330
199899571,46cyclictest29274-21grep12:35:333
199899571,40cyclictest0-21swapper/310:00:373
199899571,37cyclictest7950irq/56-dwc_otg_10:52:193
199899571,2cyclictest0-21swapper/308:05:113
1998995711,36cyclictest12042-1kworker/u9:211:38:433
199799578,32cyclictest0-21swapper/211:11:312
199799576,38cyclictest7950irq/56-dwc_otg_11:08:542
199799575,40cyclictest7950irq/56-dwc_otg_12:10:042
199799572,41cyclictest7950irq/56-dwc_otg_11:56:092
199799571,5cyclictest0-21swapper/210:30:252
199799571,49cyclictest13023-21kworker/u8:209:26:452
199799571,44cyclictest17972-21taskset07:51:562
199799571,44cyclictest0-21swapper/210:45:022
199799571,42cyclictest3623-21rm12:01:412
199799571,42cyclictest3623-21rm12:01:412
199799571,33cyclictest21987-21grep08:05:112
199899567,34cyclictest16278-21ssh10:46:593
199899567,34cyclictest16278-21ssh10:46:593
199899562,35cyclictest0-21swapper/308:15:463
199899561,43cyclictest13292-21ssh09:10:153
199799565,35cyclictest7950irq/56-dwc_otg_08:05:332
199799562,40cyclictest2447-21tr08:40:212
199799561,41cyclictest7950irq/56-dwc_otg_09:51:092
199499563,11cyclictest0-21swapper/108:40:221
1990995627,4cyclictest6309-21fschecks_count08:50:340
199099562,28cyclictest13412-21touch11:30:120
199099561,32cyclictest868-21ssh09:40:260
199099561,30cyclictest4945-21grep07:15:380
199899554,38cyclictest7950irq/56-dwc_otg_09:40:293
199899552,43cyclictest3810-21sh12:02:033
199899552,43cyclictest3810-21sh12:02:033
1998995517,33cyclictest28268-21cron12:35:113
1998995517,33cyclictest28268-21cron12:35:113
199899551,41cyclictest29688-21ssh11:06:283
199899551,3cyclictest0-21swapper/309:30:303
199799553,40cyclictest0-21swapper/210:07:122
199799552,3cyclictest7950irq/56-dwc_otg_07:57:352
199799551,6cyclictest0-21swapper/210:54:562
199799551,4cyclictest0-21swapper/211:39:052
199799551,44cyclictest16444-21ssh10:47:212
199799551,44cyclictest16444-21ssh10:47:212
199099552,30cyclictest0-21swapper/008:05:280
199899541,43cyclictest0-21swapper/307:28:003
199899541,42cyclictest7950irq/56-dwc_otg_07:10:263
199899541,40cyclictest22302-21df_inode08:05:313
199899541,3cyclictest7950irq/56-dwc_otg_12:10:083
199899541,3cyclictest7750irq/56-dwc_otg11:45:093
199899541,39cyclictest23263-21rm12:27:403
199899541,29cyclictest0-21swapper/312:15:393
199799541,50cyclictest25859-21grep08:15:262
199799541,44cyclictest27043-21ssh10:16:292
199799541,43cyclictest13091-21ssh09:56:382
199799541,3cyclictest0-21swapper/211:45:092
199499546,16cyclictest0-21swapper/111:05:251
1990995439,5cyclictest5068-21ssh10:30:440
1990995427,5cyclictest2375-21http11:15:110
199899531,48cyclictest8970-21unixbench_multi08:55:383
199899531,3cyclictest0-21swapper/310:42:113
199899531,37cyclictest9570-21aten_rbpower_po12:10:263
199799532,3cyclictest7950irq/56-dwc_otg_09:14:302
199799532,39cyclictest7950irq/56-dwc_otg_07:41:272
199799531,48cyclictest8968-21unixbench_multi08:55:382
199099531,41cyclictest0-21swapper/010:10:170
199899521,39cyclictest14325-21ssh09:59:223
199799521,41cyclictest0-21swapper/208:45:252
1990995225,13cyclictest0-21swapper/007:35:290
199099522,40cyclictest0-21swapper/010:04:100
1990995219,30cyclictest9-21ksoftirqd/007:25:300
199099521,33cyclictest11369-21cat09:05:110
199099521,33cyclictest11369-21cat09:05:110
199099521,28cyclictest13-21rcuc/011:25:110
199899515,34cyclictest7950irq/56-dwc_otg_10:15:373
199899511,3cyclictest0-21swapper/311:33:513
199899511,39cyclictest20766-21ssh10:08:083
199899511,35cyclictest0-21swapper/309:54:123
199899511,34cyclictest7950irq/56-dwc_otg_11:47:493
199799511,4cyclictest0-21swapper/211:20:162
199799511,3cyclictest7950irq/56-dwc_otg_11:45:462
1997995110,38cyclictest29937-21missed_timers08:25:322
1997995110,38cyclictest29937-21missed_timers08:25:322
1994995115,24cyclictest0-21swapper/109:15:201
199099511,28cyclictest19604-21sh10:05:420
255872500,7sleep07750irq/56-dwc_otg11:45:420
199899503,44cyclictest739-21aten_rbpower_vo08:35:293
199899502,36cyclictest32211-21ssh10:24:543
199899501,45cyclictest21793-21ssh09:22:483
199899501,43cyclictest0-21swapper/307:40:233
199899501,3cyclictest0-21swapper/311:56:293
199899501,3cyclictest0-21swapper/308:42:403
199899501,39cyclictest0-21swapper/307:36:323
199099505,16cyclictest13981-21timerwakeupswit07:40:380
1990995022,15cyclictest17812-21grep07:50:400
1990995018,7cyclictest0-21swapper/012:15:250
199099501,29cyclictest6055-21latency_hist11:20:110
14392503,12sleep00-21swapper/007:06:320
199899491,5cyclictest6252-21ssh11:20:163
199899491,5cyclictest0-21swapper/307:50:173
199899491,44cyclictest342-21dpkg08:35:113
199899491,37cyclictest0-21swapper/308:26:473
199899491,37cyclictest0-21swapper/308:26:473
199899491,34cyclictest0-21swapper/307:57:333
199899491,2cyclictest0-21swapper/309:04:433
199899491,2cyclictest0-21swapper/309:04:433
199899491,2cyclictest0-21swapper/308:45:453
199799492,44cyclictest737-21wget08:35:292
199799492,43cyclictest9882-21grep07:30:322
199499498,6cyclictest7950irq/56-dwc_otg_09:00:241
199499498,6cyclictest7950irq/56-dwc_otg_09:00:241
1990994923,6cyclictest7850irq/56-dwc_otg_08:25:390
1990994923,6cyclictest7850irq/56-dwc_otg_08:25:390
199099492,22cyclictest8311-21ssh10:35:390
199099492,22cyclictest0-21swapper/010:21:060
1990994921,4cyclictest9-21ksoftirqd/009:15:110
1990994920,14cyclictest9-21ksoftirqd/007:20:220
1990994918,17cyclictest28316-21sh12:35:120
1990994918,17cyclictest28316-21sh12:35:120
199099491,33cyclictest0-21swapper/007:55:410
199099491,29cyclictest26333-21/usr/sbin/munin09:30:320
199799482,42cyclictest7586-21wget07:25:262
199799481,5cyclictest28044-21kernelversion08:20:322
199799481,44cyclictest343-21latency_hist08:35:112
199499485,11cyclictest7850irq/56-dwc_otg_11:20:221
199099487,25cyclictest0-21swapper/010:50:250
199099486,32cyclictest0-21swapper/010:45:390
199099486,32cyclictest0-21swapper/010:45:390
1990994831,5cyclictest7850irq/56-dwc_otg_08:30:310
1990994829,6cyclictest7850irq/56-dwc_otg_12:35:280
199899472,33cyclictest7950irq/56-dwc_otg_07:31:203
199799471,6cyclictest11522-21basename09:05:272
199499479,13cyclictest0-21swapper/112:35:241
199099472,30cyclictest0-21swapper/011:55:380
1990994720,14cyclictest31595-21ssh09:37:090
199099471,4cyclictest14686-21apt10:00:110
199899462,41cyclictest28694-21grep08:20:403
199899461,42cyclictest11535-21tr09:05:273
199099467,27cyclictest14485-21kworker/0:111:00:180
199099467,16cyclictest0-21swapper/007:50:110
1990994625,6cyclictest7750irq/56-dwc_otg12:15:110
199099461,7cyclictest22168-21ssh11:40:470
1990994615,14cyclictest1808-21/usr/sbin/munin12:00:370
1990994615,14cyclictest1808-21/usr/sbin/munin12:00:370
1990994614,23cyclictest0-21swapper/012:25:140
1990994613,16cyclictest8199-21/usr/sbin/munin09:50:290
199499457,3cyclictest7750irq/56-dwc_otg07:20:171
199499457,24cyclictest0-21swapper/107:30:251
1990994523,5cyclictest0-21swapper/008:55:250
199099452,29cyclictest0-21swapper/008:15:310
1990994520,18cyclictest0-21swapper/011:05:250
199499447,25cyclictest0-21swapper/109:05:271
199499446,24cyclictest19723-21ssh09:20:251
199499441,7cyclictest7850irq/56-dwc_otg_07:35:221
1994994410,5cyclictest0-21swapper/109:10:221
199099441,23cyclictest0-21swapper/009:15:390
199499433,24cyclictest7950irq/56-dwc_otg_10:46:331
199499433,24cyclictest7950irq/56-dwc_otg_10:46:331
1990994320,4cyclictest7750irq/56-dwc_otg10:40:440
199099431,24cyclictest0-21swapper/011:40:110
199499426,24cyclictest0-21swapper/108:10:241
199499423,6cyclictest7750irq/56-dwc_otg11:50:271
199499422,31cyclictest7950irq/56-dwc_otg_09:25:231
199499422,12cyclictest0-21swapper/108:20:251
1990994218,5cyclictest27984-21ssh11:50:330
199499418,16cyclictest21-21rcuc/109:40:231
1990994119,10cyclictest0-21swapper/010:55:410
199499407,5cyclictest7850irq/56-dwc_otg_10:35:281
199499404,7cyclictest17449-21awk12:20:281
1994994022,6cyclictest23865-21sh10:58:421
1994994013,4cyclictest0-21swapper/110:00:401
199099406,24cyclictest0-21swapper/008:00:230
199499397,23cyclictest0-21swapper/108:00:231
199499396,23cyclictest13040-21diskmemload10:20:251
1994993918,4cyclictest30991-21/usr/sbin/munin11:55:391
1990993925,4cyclictest0-21swapper/008:24:580
199499388,6cyclictest23-21ksoftirqd/112:15:301
199499386,5cyclictest0-21swapper/112:10:341
199499383,2cyclictest0-21swapper/109:55:241
1994993817,5cyclictest23-21ksoftirqd/110:55:101
1994993815,5cyclictest9175-21latency_hist12:10:111
1994993815,13cyclictest0-21swapper/112:32:311
1994993815,13cyclictest0-21swapper/112:32:311
1994993813,9cyclictest22926-21sh11:42:331
199499377,6cyclictest7950irq/56-dwc_otg_07:55:301
199499376,22cyclictest61-21kswapd008:35:171
1994993714,16cyclictest0-21swapper/111:15:181
199499371,16cyclictest7750irq/56-dwc_otg11:05:121
10112375,19sleep31095-21threads07:05:463
199499365,5cyclictest0-21swapper/109:50:251
199499363,5cyclictest7750irq/56-dwc_otg10:40:281
1994993615,4cyclictest26796-21if_eth009:30:301
1994993611,5cyclictest7950irq/56-dwc_otg_12:25:331
1994993610,16cyclictest0-21swapper/111:30:261
199499355,20cyclictest32694-21ssh10:25:261
1994993521,4cyclictest17186-21sh11:35:211
199499352,12cyclictest0-21swapper/107:10:261
199499351,5cyclictest2427250ssh11:45:221
1994993512,4cyclictest19689-21unixbench_multi10:05:421
199499345,7cyclictest16403-21latency_hist07:50:111
199499344,4cyclictest21-21rcuc/108:15:381
199499342,7cyclictest17849-21sed07:50:401
199499342,7cyclictest0-21swapper/110:30:261
199499342,5cyclictest30028-21fschecks_time09:35:281
199499342,16cyclictest8180-21fschecks_time08:55:281
1994993417,10cyclictest21-21rcuc/107:25:371
1994993415,6cyclictest4323-21runrttasks09:45:251
199499341,16cyclictest25163-21latency_hist10:15:121
199499338,4cyclictest0-21swapper/111:10:401
1994993310,10cyclictest7750irq/56-dwc_otg07:15:381
199499327,6cyclictest23-21ksoftirqd/108:45:341
199499322,16cyclictest7775-21ls08:55:121
199499317,5cyclictest7750irq/56-dwc_otg08:05:331
1994993120,7cyclictest221ktimersoftd/108:25:241
1994993120,7cyclictest221ktimersoftd/108:25:241
1994992910,11cyclictest23-21ksoftirqd/108:30:231
15552277,13sleep20-21swapper/207:08:042
199499261,4cyclictest7750irq/56-dwc_otg07:40:331
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional