You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-30 - 16:16

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack7slot3s.osadl.org (updated Tue Dec 30, 2025 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3179899836,22cyclictest0-21swapper/009:00:240
271872740,3sleep127186-21grep08:25:321
271872740,3sleep127186-21grep08:25:321
31798996710,22cyclictest0-21swapper/008:10:240
3179899657,35cyclictest16917-21grep09:25:390
3180199621,17cyclictest7950irq/56-dwc_otg_10:10:253
31798995731,4cyclictest0-21swapper/011:35:270
31798995626,28cyclictest101ktimersoftd/007:40:220
3179899552,30cyclictest20461-21sed08:05:430
31798995521,5cyclictest0-21swapper/012:35:210
31798995514,22cyclictest5421-21fschecks_count10:25:300
3179899551,33cyclictest28001-21grep11:25:380
31798995510,25cyclictest25209-21fschecks_count09:50:290
31798995510,25cyclictest25209-21fschecks_count09:50:290
3180099546,20cyclictest7950irq/56-dwc_otg_10:35:172
31798995430,14cyclictest0-21swapper/007:30:100
3179899542,25cyclictest0-21swapper/008:05:100
3180099535,6cyclictest0-21swapper/208:40:232
3179899532,37cyclictest0-21swapper/010:45:230
3179899531,48cyclictest0-21swapper/010:00:240
3180099523,9cyclictest0-21swapper/210:10:242
3179899522,29cyclictest0-21swapper/011:55:290
3179899522,14cyclictest0-21swapper/008:40:110
31798995221,3cyclictest0-21swapper/011:20:100
31798995128,7cyclictest7850irq/56-dwc_otg_09:30:390
31798995121,9cyclictest20602-21grep09:35:390
3179899511,30cyclictest4710-21idleruntime-cro08:55:100
3179899511,28cyclictest0-21swapper/010:30:280
3179899511,28cyclictest0-21swapper/010:30:280
31798995111,4cyclictest473-21unixbench_multi07:10:390
3180099504,19cyclictest7950irq/56-dwc_otg_09:10:362
3180099501,8cyclictest7850irq/56-dwc_otg_08:45:242
31798995024,17cyclictest9365-21ls10:35:300
3179899501,27cyclictest2256-21cat08:45:380
31798995011,22cyclictest11617-21wc12:10:390
3179999492,12cyclictest7750irq/56-dwc_otg12:20:231
31798994929,13cyclictest0-21swapper/012:15:330
31798994927,5cyclictest14306-21aten_rbpower_cu10:50:250
31798994926,5cyclictest0-21swapper/012:20:260
3179899492,31cyclictest0-21swapper/007:45:240
31798994922,8cyclictest10096-21cron09:10:100
31798994921,13cyclictest0-21swapper/010:10:280
3179899491,29cyclictest3634-21fschecks_time10:20:280
3180099487,25cyclictest0-21swapper/212:05:262
3180099487,25cyclictest0-21swapper/212:05:262
3180099486,25cyclictest0-21swapper/207:55:162
31798994823,5cyclictest23529-21missed_timers08:15:330
3179899482,25cyclictest0-21swapper/010:05:260
31798994821,5cyclictest15947-21fschecks_count07:55:270
31798994821,10cyclictest7950irq/56-dwc_otg_08:25:250
31798994821,10cyclictest7950irq/56-dwc_otg_08:25:250
3179899481,4cyclictest1644-21latency_hist11:45:110
3180099477,25cyclictest0-21swapper/212:00:252
3180099477,25cyclictest0-21swapper/212:00:252
3179899473,41cyclictest0-21swapper/012:00:220
3179899473,41cyclictest0-21swapper/012:00:220
31798994722,12cyclictest0-21swapper/010:40:280
31798994722,12cyclictest0-21swapper/008:30:350
31798994713,15cyclictest28781-21cut11:30:250
314122473,34sleep00-21swapper/007:08:470
3180099467,25cyclictest0-21swapper/211:50:252
3179899468,36cyclictest101ktimersoftd/007:51:190
31798994631,4cyclictest7750irq/56-dwc_otg12:30:250
31798994618,19cyclictest0-21swapper/009:22:460
3180199457,24cyclictest0-21swapper/312:20:253
3180099457,24cyclictest0-21swapper/212:25:252
3180099457,24cyclictest0-21swapper/207:10:242
3180099456,25cyclictest0-21swapper/209:45:152
31800994517,5cyclictest12778-21fschecks_count12:15:282
31799994520,13cyclictest16370-21fschecks_time10:55:291
3179999451,3cyclictest7850irq/56-dwc_otg_07:20:221
3179899455,19cyclictest13287-21sshd09:15:390
3179899455,19cyclictest13287-21sshd09:15:390
3179899452,21cyclictest0-21swapper/008:55:220
31798994522,14cyclictest23874-21/usr/sbin/munin09:45:340
31798994520,4cyclictest4436-21grep11:50:380
31798994520,4cyclictest26982-21/usr/sbin/munin09:55:270
31798994517,14cyclictest17306-21users12:25:400
3179899451,28cyclictest7719-21sed07:30:400
3179899451,28cyclictest32043-21cut08:40:260
3180099447,25cyclictest0-21swapper/212:35:242
31798994423,9cyclictest0-21swapper/010:15:300
3179899442,19cyclictest0-21swapper/007:35:280
31798994417,16cyclictest3211-21diskstats07:20:250
31798994413,23cyclictest0-21swapper/012:06:080
31798994413,23cyclictest0-21swapper/012:06:080
3180199437,22cyclictest0-21swapper/311:30:253
3180199436,24cyclictest0-21swapper/308:30:233
3179999437,23cyclictest0-21swapper/111:05:241
3179999437,22cyclictest0-21swapper/111:30:251
3179999436,4cyclictest0-21swapper/110:10:251
3179899436,24cyclictest0-21swapper/009:40:250
3180199427,24cyclictest0-21swapper/311:10:233
3180199427,23cyclictest0-21swapper/309:45:163
3180199427,23cyclictest0-21swapper/307:35:243
3180099427,22cyclictest0-21swapper/210:25:262
3180099426,21cyclictest0-21swapper/211:20:242
3180099421,7cyclictest0-21swapper/208:30:212
31799994213,6cyclictest23-21ksoftirqd/110:20:101
31798994220,16cyclictest10872-21latency09:10:300
3179899421,4cyclictest0-21swapper/008:20:300
3180099417,22cyclictest0-21swapper/210:20:242
31800994117,6cyclictest20112-21/usr/sbin/munin09:35:332
3179999418,17cyclictest22987-21find09:45:111
3179999416,22cyclictest0-21swapper/110:35:241
31799994119,5cyclictest23-21ksoftirqd/108:00:401
31799994114,6cyclictest7750irq/56-dwc_otg10:50:391
3179899412,27cyclictest0-21swapper/011:06:420
3179899412,20cyclictest0-21swapper/007:15:280
3179899412,20cyclictest0-21swapper/007:15:280
31798994120,3cyclictest0-21swapper/011:10:360
23492410,6sleep13179999cyclictest11:45:331
323882400,5sleep33180199cyclictest07:10:313
3180199407,24cyclictest0-21swapper/307:40:243
3179899401,15cyclictest0-21swapper/011:00:310
3180199395,16cyclictest7950irq/56-dwc_otg_09:35:113
3180099396,24cyclictest0-21swapper/209:55:242
3180099391,26cyclictest23194-21cut08:15:292
3179999393,4cyclictest0-21swapper/107:10:211
3179999392,14cyclictest27431-21tr09:55:331
31798993929,4cyclictest7850irq/56-dwc_otg_11:45:380
31798993920,4cyclictest7850irq/56-dwc_otg_10:55:230
249252390,5sleep23180099cyclictest11:20:102
3180199384,6cyclictest7950irq/56-dwc_otg_08:10:283
31801993816,6cyclictest39-21ksoftirqd/307:45:263
3180099388,24cyclictest0-21swapper/211:00:252
3180099381,26cyclictest30399-21cut08:35:302
3180099381,26cyclictest21825-21ls11:10:282
3180099381,23cyclictest8829-21grep09:05:282
3179999382,15cyclictest7850irq/56-dwc_otg_10:45:281
3179999381,5cyclictest4447-21unixbench_multi10:20:391
302642385,20sleep229971-21/usr/sbin/munin07:05:292
133012381,5sleep10-21swapper/112:15:371
3180199371,5cyclictest32650-21cut11:40:293
31801993714,12cyclictest128-21systemd-journal07:18:273
31801993714,12cyclictest128-21systemd-journal07:18:273
3180199371,23cyclictest20258-21ls11:05:323
3179999371,11cyclictest19027-21grep12:30:391
31798993718,11cyclictest0-21swapper/011:22:220
3180099363,4cyclictest21340-21latency_hist11:10:112
31799993612,7cyclictest128-21systemd-journal09:48:561
171182360,1sleep10-21swapper/107:58:231
3180199352,5cyclictest0-21swapper/311:55:233
3180199351,3cyclictest7950irq/56-dwc_otg_09:55:233
3180099359,4cyclictest7750irq/56-dwc_otg09:25:392
3180099357,25cyclictest0-21swapper/210:50:252
3180099357,15cyclictest0-21swapper/210:30:252
3180099357,15cyclictest0-21swapper/210:30:252
31800993515,5cyclictest2863950wget10:00:252
31799993519,5cyclictest24143-21switchtime11:15:391
3179999351,24cyclictest21294-21ls08:10:261
31799993510,5cyclictest23-21ksoftirqd/110:00:331
3180199343,16cyclictest7950irq/56-dwc_otg_10:55:293
3180199342,22cyclictest0-21swapper/309:20:113
3180199342,22cyclictest0-21swapper/309:20:113
31801993410,14cyclictest7950irq/56-dwc_otg_08:30:113
31801993410,14cyclictest7950irq/56-dwc_otg_08:30:113
3180099349,5cyclictest7750irq/56-dwc_otg07:50:102
3180099348,6cyclictest25866-21unixbench_multi08:20:382
3180099345,13cyclictest7750irq/56-dwc_otg12:10:252
3180099344,7cyclictest0-21swapper/210:15:242
3180099344,18cyclictest5333-21basename11:55:242
3180099343,7cyclictest4877-21aten_rbpower_po07:25:242
31800993425,6cyclictest301ktimersoftd/211:25:222
3180099342,19cyclictest0-21swapper/209:22:212
31800993416,9cyclictest19439-21seq12:34:392
3179999349,4cyclictest13250-21unixbench_multi09:15:391
3179999349,4cyclictest13250-21unixbench_multi09:15:391
3179999346,15cyclictest16919-21tail09:25:391
3179999346,13cyclictest7950irq/56-dwc_otg_12:15:111
3179999344,15cyclictest7950irq/56-dwc_otg_09:50:401
3179999344,15cyclictest7950irq/56-dwc_otg_09:50:401
3179999343,15cyclictest21-21rcuc/111:04:241
3179999342,15cyclictest7850irq/56-dwc_otg_12:04:401
3179999342,15cyclictest7850irq/56-dwc_otg_12:04:401
3179999341,3cyclictest7950irq/56-dwc_otg_11:35:231
3179999341,3cyclictest0-21swapper/111:55:391
3180199336,6cyclictest7950irq/56-dwc_otg_08:00:103
3180199333,15cyclictest7950irq/56-dwc_otg_10:25:363
31801993317,3cyclictest0-21swapper/310:15:293
3180199331,6cyclictest7750irq/56-dwc_otg12:30:393
31801993311,4cyclictest7850irq/56-dwc_otg_10:45:233
3180099331,5cyclictest12220-21grep09:15:252
3180099331,5cyclictest12220-21grep09:15:252
31800993314,5cyclictest7152-21load07:30:322
3180099331,3cyclictest7750irq/56-dwc_otg07:40:102
3179999336,4cyclictest7750irq/56-dwc_otg12:35:271
31799993325,5cyclictest26670-21sh11:25:101
31799993324,7cyclictest221ktimersoftd/108:35:241
3179999332,19cyclictest21-21rcuc/107:50:251
3179999331,17cyclictest0-21swapper/107:15:301
3179999331,17cyclictest0-21swapper/107:15:301
146702331,2sleep30-21swapper/307:50:373
3180199328,15cyclictest39-21ksoftirqd/311:15:283
3180199326,14cyclictest0-21swapper/310:35:243
3180199326,14cyclictest0-21swapper/310:00:243
3180199325,17cyclictest11354-21ntpq10:40:353
3180199325,14cyclictest7950irq/56-dwc_otg_07:20:303
31801993226,4cyclictest26100-21ls11:20:383
3180199321,25cyclictest16105-21grep09:25:293
3180099326,12cyclictest7950irq/56-dwc_otg_09:40:312
3180099325,6cyclictest7950irq/56-dwc_otg_09:50:262
3180099325,6cyclictest7950irq/56-dwc_otg_09:50:262
3180099324,6cyclictest6997-21fschecks_count09:00:282
3180099323,8cyclictest0-21swapper/211:35:222
3180099321,5cyclictest0-21swapper/209:35:112
3179999325,13cyclictest7950irq/56-dwc_otg_11:10:241
3179999324,15cyclictest7950irq/56-dwc_otg_09:20:271
3179999324,15cyclictest7850irq/56-dwc_otg_07:30:101
31799993211,4cyclictest23-21ksoftirqd/112:25:301
3179999321,13cyclictest3470-21fschecks_time08:50:301
274622320,1sleep30-21swapper/311:25:313
3180199318,5cyclictest7750irq/56-dwc_otg12:40:113
3180199315,6cyclictest7950irq/56-dwc_otg_09:20:233
3180199315,5cyclictest7950irq/56-dwc_otg_12:15:393
3180199313,4cyclictest7750irq/56-dwc_otg08:15:373
3180199313,16cyclictest7950irq/56-dwc_otg_08:50:293
3180199313,11cyclictest7950irq/56-dwc_otg_10:20:363
3180199312,15cyclictest32548-21missed_timers08:40:323
3180199311,17cyclictest3652-21cpuspeed11:50:263
3180199311,17cyclictest14630-21ls10:50:293
31801993110,5cyclictest1125-21runrttasks09:45:043
3180099315,15cyclictest28959-21fschecks_count11:30:282
3180099312,5cyclictest0-21swapper/211:00:112
3180099311,8cyclictest6052-21unixbench_multi08:55:372
3180099311,5cyclictest20491-21unixbench_singl08:05:432
3179999317,16cyclictest0-21swapper/107:30:251
3179999316,8cyclictest7850irq/56-dwc_otg_08:55:261
3179999316,8cyclictest7850irq/56-dwc_otg_07:40:101
3179999316,5cyclictest8073-21tail10:30:391
3179999316,5cyclictest8073-21tail10:30:391
3179999315,6cyclictest23-21ksoftirqd/110:25:381
3179999314,6cyclictest7950irq/56-dwc_otg_11:30:111
3179999312,14cyclictest7750irq/56-dwc_otg08:40:371
31799993117,5cyclictest18105-21/usr/sbin/munin09:30:321
31799993112,5cyclictest1125-21runrttasks10:44:581
312062312,22sleep30-21swapper/307:06:073
3180199308,15cyclictest20999-21sh09:40:103
3180199305,16cyclictest8009-21sshd10:30:393
3180199305,16cyclictest8009-21sshd10:30:393
3180199303,13cyclictest7950irq/56-dwc_otg_09:00:393
3180199302,12cyclictest7950irq/56-dwc_otg_09:09:113
3180199301,15cyclictest7342-21fschecks_count12:00:283
3180199301,15cyclictest7342-21fschecks_count12:00:283
3180199301,12cyclictest7950irq/56-dwc_otg_08:05:403
3180099308,10cyclictest7950irq/56-dwc_otg_10:45:222
3180099306,6cyclictest7950irq/56-dwc_otg_08:25:392
3180099306,6cyclictest7950irq/56-dwc_otg_08:25:392
3180099302,6cyclictest7850irq/56-dwc_otg_10:10:102
3180099302,19cyclictest0-21swapper/208:50:152
3180099301,27cyclictest2537-21ls11:45:372
3179999304,15cyclictest7950irq/56-dwc_otg_08:19:381
3179999303,10cyclictest7750irq/56-dwc_otg08:05:261
31799993016,5cyclictest12499-21/usr/sbin/munin07:45:311
31799993013,10cyclictest8840-21grep09:05:281
31799993012,15cyclictest0-21swapper/109:10:241
3180199297,7cyclictest9450irq/48-DMA10:44:553
3180199297,12cyclictest7950irq/56-dwc_otg_11:00:263
3180199295,6cyclictest11425-21/usr/sbin/munin12:10:363
3180199294,18cyclictest3278-21latency_hist11:50:113
3180199293,5cyclictest0-21swapper/309:50:233
3180199293,5cyclictest0-21swapper/309:50:233
31801992923,4cyclictest9221-21fschecks_time12:05:303
31801992923,4cyclictest9221-21fschecks_time12:05:303
31801992914,4cyclictest8160-21latency_hist07:35:103
3180199291,20cyclictest0-21swapper/311:39:393
3180199291,13cyclictest5031-21fschecks_count07:25:273
3180099291,6cyclictest7850irq/56-dwc_otg_07:55:112
3180099291,5cyclictest7850irq/56-dwc_otg_11:45:112
3180099291,5cyclictest112450snmpd08:03:362
3180099291,18cyclictest0-21swapper/207:43:582
3179999293,6cyclictest0-21swapper/107:40:221
3179999293,13cyclictest7950irq/56-dwc_otg_11:40:291
3179999292,4cyclictest7750irq/56-dwc_otg12:05:241
3179999292,4cyclictest7750irq/56-dwc_otg12:05:241
3179999291,9cyclictest7950irq/56-dwc_otg_11:50:231
3179999291,9cyclictest21006-21munin-run09:40:101
3179999291,7cyclictest7950irq/56-dwc_otg_10:10:101
31799992911,9cyclictest7850irq/56-dwc_otg_08:24:211
3180199284,14cyclictest0-21swapper/312:26:243
3180199284,12cyclictest7950irq/56-dwc_otg_10:05:333
3180199282,6cyclictest7950irq/56-dwc_otg_09:10:393
3180199281,20cyclictest0-21swapper/308:55:163
3180199281,19cyclictest0-21swapper/308:20:363
3180199281,14cyclictest1966-21grep08:45:333
3180099283,12cyclictest7950irq/56-dwc_otg_07:24:262
3180099282,7cyclictest0-21swapper/210:40:202
3180099281,3cyclictest7850irq/56-dwc_otg_07:20:112
3180099281,3cyclictest7850irq/56-dwc_otg_07:20:112
3179999283,12cyclictest7850irq/56-dwc_otg_08:47:211
3179999281,12cyclictest7750irq/56-dwc_otg09:00:301
3180099274,6cyclictest7950irq/56-dwc_otg_08:13:552
3180099274,13cyclictest7950irq/56-dwc_otg_12:20:402
3180199266,11cyclictest17734-21diskstats08:00:273
312292267,13sleep10-21swapper/107:06:251
3179999254,4cyclictest28920-21missed_timers08:30:311
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional