You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-01 - 18:45

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Tue Sep 01, 2026 12:43:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2745621700,5sleep12123499cyclictest12:20:051
2123399938,63cyclictest13-21rcuc/009:30:380
21235999119,10cyclictest17966-21proc_pri09:30:402
21234999119,26cyclictest17917-21ssh09:30:391
21233998916,22cyclictest7750irq/56-dwc_otg09:20:380
21233998729,12cyclictest0-21swapper/007:45:260
21233998729,12cyclictest0-21swapper/007:45:260
2123599815,11cyclictest0-21swapper/209:20:392
21233997821,40cyclictest29617-21tail07:30:420
59902770,2sleep00-21swapper/012:32:320
2123399751,44cyclictest0-21swapper/012:15:250
21233997243,7cyclictest7950irq/56-dwc_otg_12:00:250
245082700,2sleep10-21swapper/110:56:561
21234997016,24cyclictest9556-21ntp_offset09:20:391
197352690,2sleep20-21swapper/210:50:452
21233996821,27cyclictest0-21swapper/009:25:270
21233996517,26cyclictest7260-21/usr/sbin/munin08:00:330
21233996510,30cyclictest0-21swapper/008:45:400
2123499643,8cyclictest0-21swapper/110:05:261
21233996410,31cyclictest19850-21wget08:35:310
146382630,2sleep20-21swapper/208:20:332
63352620,5sleep12123499cyclictest10:35:341
2123599625,14cyclictest7950irq/56-dwc_otg_10:40:372
2123599625,14cyclictest7950irq/56-dwc_otg_10:40:372
2123599619,7cyclictest7950irq/56-dwc_otg_11:00:412
21235996110,4cyclictest0-21swapper/209:35:292
2123399612,42cyclictest15115-21ps10:45:380
2123399611,4cyclictest31601-21aten_rbpower_po11:45:290
2123599601,19cyclictest0-21swapper/212:38:242
21233996033,5cyclictest31684-21modprobe12:25:140
2123399594,10cyclictest0-21swapper/008:50:280
21233995914,31cyclictest21632-21modprobe10:15:140
21236995812,23cyclictest7950irq/56-dwc_otg_09:20:383
2123599575,12cyclictest7850irq/56-dwc_otg_12:03:512
2123499571,7cyclictest0-21swapper/109:35:261
2123399576,31cyclictest32335-21latency_hist09:10:140
21233995733,5cyclictest7850irq/56-dwc_otg_10:30:310
2123399573,34cyclictest0-21swapper/012:10:140
2123399572,25cyclictest0-21swapper/011:05:150
21233995716,20cyclictest6225-21cpuspeed10:35:310
2123399571,43cyclictest0-21swapper/007:30:140
2123399571,34cyclictest18385-21/usr/sbin/munin10:50:350
2123399571,31cyclictest0-21swapper/007:35:270
2123399564,27cyclictest21831-21cat07:10:340
21233995632,5cyclictest7950irq/56-dwc_otg_11:35:140
2123399561,39cyclictest15034-21aten_rbpower_cu11:25:270
21233995610,23cyclictest0-21swapper/011:45:140
295692550,2sleep30-21swapper/312:21:243
2123399555,32cyclictest101ktimersoftd/008:30:430
21233995532,4cyclictest0-21swapper/011:15:280
21233995515,11cyclictest7950irq/56-dwc_otg_11:10:300
2123399551,40cyclictest8529-21grep11:55:430
2123399551,16cyclictest30180-21ls09:45:360
2123599532,15cyclictest0-21swapper/209:42:352
21234995313,10cyclictest7950irq/56-dwc_otg_09:55:271
21233995329,5cyclictest7950irq/56-dwc_otg_10:10:140
2123699526,12cyclictest7750irq/56-dwc_otg09:30:403
21234995224,12cyclictest0-21swapper/110:10:251
21233995231,5cyclictest7750irq/56-dwc_otg09:10:300
2123399521,30cyclictest6186-21awk09:55:360
2123399521,30cyclictest22670-21aten_rbpower_vo10:55:290
2123399521,26cyclictest10918-21ntpq10:40:370
2123399521,26cyclictest10918-21ntpq10:40:370
2123699518,27cyclictest2514-21ssh11:10:283
2123599518,12cyclictest7850irq/56-dwc_otg_10:15:142
2123499515,10cyclictest0-21swapper/107:40:251
21234995111,24cyclictest11794-21ssh11:20:401
21234995110,27cyclictest927-21grep09:50:141
2123399511,26cyclictest4025-21wc07:50:390
21235995032,6cyclictest7950irq/56-dwc_otg_10:50:142
21235995012,9cyclictest7750irq/56-dwc_otg07:45:262
21235995012,9cyclictest7750irq/56-dwc_otg07:45:262
2123499502,12cyclictest0-21swapper/110:50:271
21233995034,5cyclictest25741-21fschecks_time09:40:300
2123399502,25cyclictest12105-21unixbench_multi11:20:420
2123399501,26cyclictest4592-21wget09:15:280
2123399501,11cyclictest3449-21wc12:30:140
2123499493,5cyclictest7850irq/56-dwc_otg_09:40:231
21234994928,4cyclictest7750irq/56-dwc_otg09:25:351
21234994912,7cyclictest25208-21perl07:20:331
21234994910,25cyclictest32331-21diskmemload09:14:111
2123399492,28cyclictest0-21swapper/010:20:230
2123399492,24cyclictest0-21swapper/010:00:320
2123399491,34cyclictest658-21cat07:40:400
2123399491,32cyclictest0-21swapper/009:37:590
2123599483,14cyclictest14986-21ssh11:25:252
2123499489,21cyclictest12797-21ssh10:04:121
2123499484,27cyclictest26912-21entropy11:00:311
21234994824,17cyclictest7750irq/56-dwc_otg10:32:001
21234994813,21cyclictest31902-21ssh10:26:241
21234994811,14cyclictest10151-21apt-get10:40:251
21234994811,14cyclictest10151-21apt-get10:40:251
21233994831,4cyclictest23120-21ssh10:15:420
2123399482,31cyclictest0-21swapper/008:16:220
2123399482,31cyclictest0-21swapper/008:16:220
21233994817,18cyclictest9-21ksoftirqd/008:20:250
2123399481,30cyclictest1725-21irqstats09:50:350
2123399481,30cyclictest1725-21irqstats09:50:350
2123499478,23cyclictest0-21swapper/107:30:281
2123499477,23cyclictest0-21swapper/107:35:291
2123499476,22cyclictest29579-21ssh12:21:251
21234994710,18cyclictest6560-21ssh09:17:001
2123399473,31cyclictest0-21swapper/012:35:320
21233994725,15cyclictest0-21swapper/009:04:340
2123399471,29cyclictest11190-21cat08:10:350
21233994711,26cyclictest13-21rcuc/012:15:140
2123399471,10cyclictest24230-21unixbench_singl07:15:420
209442473,11sleep00-21swapper/007:09:590
2123599467,5cyclictest26196-21switchtime08:50:402
2123499468,11cyclictest12497-21aten_rbpower_vo08:15:301
2123499468,11cyclictest12497-21aten_rbpower_vo08:15:301
2123499467,24cyclictest0-21swapper/112:30:281
2123499462,9cyclictest7950irq/56-dwc_otg_09:00:251
21234994616,15cyclictest7850irq/56-dwc_otg_07:20:141
21233994634,5cyclictest0-21swapper/007:55:260
2123699457,24cyclictest0-21swapper/308:50:193
2123699457,24cyclictest0-21swapper/308:15:293
2123699457,24cyclictest0-21swapper/308:15:293
2123699456,25cyclictest0-21swapper/307:30:283
2123599457,23cyclictest0-21swapper/208:45:192
2123599456,12cyclictest1613-21idleruntime09:50:332
2123599456,12cyclictest1613-21idleruntime09:50:332
2123499457,26cyclictest0-21swapper/108:10:191
2123499457,25cyclictest0-21swapper/108:25:191
2123499456,20cyclictest7950irq/56-dwc_otg_10:15:311
2123499455,3cyclictest0-21swapper/111:10:151
21234994514,16cyclictest11730-21sh12:00:321
2123399452,5cyclictest0-21swapper/011:50:370
2123699447,22cyclictest0-21swapper/308:00:283
21236994424,5cyclictest12917-21aten_rbpower_cu09:25:293
2123599446,23cyclictest0-21swapper/207:40:272
2123499449,19cyclictest22791-21ssh11:35:011
2123499448,20cyclictest10011-21ssh11:58:181
2123499447,25cyclictest0-21swapper/111:45:281
2123499445,25cyclictest60-21kswapd012:35:251
2123499443,28cyclictest17854-21ssh11:28:321
21234994414,9cyclictest7950irq/56-dwc_otg_11:35:251
21234994413,23cyclictest15758-21ssh10:46:031
2123499441,10cyclictest23725-21fschecks_time08:45:301
21233994423,6cyclictest7750irq/56-dwc_otg11:05:440
2123399442,20cyclictest0-21swapper/010:30:140
2123699437,25cyclictest0-21swapper/307:25:273
2123699436,24cyclictest0-21swapper/309:15:283
2123599436,25cyclictest0-21swapper/209:45:282
21235994327,9cyclictest17100-21ssh12:06:112
21235994313,7cyclictest24514-21grep11:35:432
2123499437,23cyclictest0-21swapper/108:05:271
2123499437,19cyclictest1671-21ssh12:27:101
2123499436,12cyclictest6072-21ssh11:14:531
21234994321,5cyclictest7750irq/56-dwc_otg07:25:411
2123499431,26cyclictest1703-21grep07:45:321
2123499431,26cyclictest1703-21grep07:45:321
21234994311,16cyclictest3564-21ssh09:52:371
21234994311,16cyclictest3564-21ssh09:52:371
21233994317,5cyclictest24364-21grep11:35:420
21236994220,11cyclictest0-21swapper/307:40:243
2123499424,7cyclictest6634-21ssh11:15:271
2123499421,26cyclictest0-21swapper/108:50:191
21234994212,14cyclictest28323-21/usr/sbin/munin11:40:411
21234994211,20cyclictest17291-21ssh12:06:321
2123399422,18cyclictest0-21swapper/008:05:300
21233994217,14cyclictest7950irq/56-dwc_otg_09:00:140
2123699417,22cyclictest0-21swapper/309:05:293
2123699414,5cyclictest29503-21ssh09:45:263
2123599416,23cyclictest0-21swapper/209:55:212
2123499419,17cyclictest22796-21runrttasks07:15:001
2123499417,21cyclictest32331-21diskmemload11:54:581
2123499416,21cyclictest21824-21runrttasks12:12:191
2123499415,21cyclictest27637-21ssh10:21:091
2123399411,25cyclictest22416-21switchtime08:40:400
190852410,2sleep219089-21unixbench_singl08:30:432
222822401,33sleep30-21swapper/309:35:423
2123699406,20cyclictest19664-21ssh12:10:283
21236994026,12cyclictest7950irq/56-dwc_otg_11:35:263
2123599406,24cyclictest0-21swapper/208:10:272
2123599402,10cyclictest32317-21kworker/u8:210:35:282
21234994013,7cyclictest3333-21aten_rbpower_en07:50:281
21233994020,9cyclictest0-21swapper/007:21:260
2123399401,28cyclictest0-21swapper/008:25:400
276822390,3sleep227684-21latency08:55:342
2123699396,20cyclictest3198-21ssh11:50:283
2123599396,23cyclictest0-21swapper/210:30:262
2123599393,15cyclictest0-21swapper/209:30:142
2123599392,3cyclictest7950irq/56-dwc_otg_07:10:342
2123499391,4cyclictest7950irq/56-dwc_otg_08:00:271
21234993911,8cyclictest14219-21apt-get08:20:251
298792380,1sleep20-21swapper/211:42:542
216002380,14sleep17950irq/56-dwc_otg_08:40:281
2123699387,21cyclictest0-21swapper/308:55:283
2123699387,20cyclictest32003-21ls07:40:153
2123699381,9cyclictest0-21swapper/311:05:263
21236993817,5cyclictest28844-21sh09:44:413
2123699381,25cyclictest9676-21ls08:05:403
2123599387,24cyclictest0-21swapper/210:25:282
2123599387,22cyclictest0-21swapper/210:15:272
21235993818,10cyclictest301ktimersoftd/210:20:432
2123599381,23cyclictest17871-21ls08:30:142
2123599381,17cyclictest3694-21irqstats11:50:342
2123499383,22cyclictest19150-21chrt08:31:061
2123699375,9cyclictest14998-21ssh11:25:253
2123699373,11cyclictest7950irq/56-dwc_otg_11:05:153
2123699372,19cyclictest0-21swapper/309:52:553
2123699372,19cyclictest0-21swapper/309:52:553
2123599377,5cyclictest28440-21ls07:30:152
2123599371,18cyclictest11252-21perl11:20:322
21235993710,5cyclictest8518-21sh11:55:432
21235993710,13cyclictest31825-21fschecks_time11:45:322
2123499372,23cyclictest32001-21taskset09:06:231
21234993713,7cyclictest27159-21apt-get08:55:271
2123699366,5cyclictest7850irq/56-dwc_otg_10:55:323
21235993619,7cyclictest22566-21aten_rbpower_cu10:55:282
21235993614,14cyclictest7381-21/usr/sbin/munin11:15:372
2123499362,9cyclictest0-21swapper/107:59:231
2123699357,5cyclictest7850irq/56-dwc_otg_11:20:143
2123699351,27cyclictest0-21swapper/312:28:153
2123699351,23cyclictest4332-21ls12:30:363
2123599351,24cyclictest0-21swapper/212:25:282
2123699349,14cyclictest7750irq/56-dwc_otg11:59:153
2123699347,24cyclictest0-21swapper/308:40:273
2123699346,15cyclictest0-21swapper/307:50:283
2123699345,21cyclictest7950irq/56-dwc_otg_08:45:193
2123699345,20cyclictest15556-21ssh12:05:283
2123699342,19cyclictest0-21swapper/312:01:253
21236993421,4cyclictest15042-21sh10:45:363
21236993416,9cyclictest32331-21diskmemload11:43:443
2123699341,24cyclictest6587-21latency10:35:363
2123699341,18cyclictest28200-21ssh10:22:113
2123699341,16cyclictest2163050ssh10:15:143
2123599347,11cyclictest0-21swapper/208:05:302
2123599341,19cyclictest13527-21ssh10:05:152
21234993417,13cyclictest20161-21if_wlan008:35:351
2123699337,14cyclictest0-21swapper/308:35:293
2123699332,9cyclictest7750irq/56-dwc_otg10:35:143
2123699332,12cyclictest7950irq/56-dwc_otg_12:15:263
2123699331,22cyclictest5590-21latency07:55:323
21236993311,4cyclictest0-21swapper/310:00:313
21236993310,5cyclictest14476-21ntp_offset10:05:373
2123599339,12cyclictest20094-21ssh11:30:422
2123599334,5cyclictest7750irq/56-dwc_otg10:05:422
198562330,5sleep32123699cyclictest10:50:583
284872320,1sleep20-21swapper/207:30:252
2123699327,5cyclictest7750irq/56-dwc_otg11:50:143
2123699321,4cyclictest5013-21apt-get09:55:213
2123699321,19cyclictest21496-21cut07:10:303
21235993218,5cyclictest1180-21runrttasks09:11:552
2123699319,4cyclictest0-21swapper/311:30:433
2123699317,13cyclictest0-21swapper/311:20:193
2123699315,13cyclictest7950irq/56-dwc_otg_07:20:343
21236993111,13cyclictest24145-21/usr/sbin/munin07:15:423
2123599319,15cyclictest0-21swapper/211:05:162
2123599319,11cyclictest0-21swapper/212:23:422
2123599318,3cyclictest0-21swapper/212:19:092
2123599312,7cyclictest32337-21apt09:10:142
2123599312,6cyclictest0-21swapper/212:30:432
21235993113,12cyclictest7850irq/56-dwc_otg_07:50:372
21235993113,12cyclictest19765-21aten_rbpower_cu08:35:302
2123699306,6cyclictest37-21rcuc/309:00:403
2123699306,15cyclictest0-21swapper/307:45:283
2123699306,15cyclictest0-21swapper/307:45:283
2123699302,6cyclictest8597-21taskset12:35:383
2123699301,22cyclictest4301-21ls09:15:153
21236993010,4cyclictest10850-21fschecks_count08:10:313
2123599305,4cyclictest7850irq/56-dwc_otg_09:15:282
2123599304,13cyclictest7850irq/56-dwc_otg_12:10:362
2123599303,14cyclictest0-21swapper/207:20:212
2123599302,5cyclictest7750irq/56-dwc_otg07:35:292
21235993015,4cyclictest7950irq/56-dwc_otg_07:55:262
21235993010,4cyclictest0-21swapper/211:11:072
2123699292,5cyclictest21919-21/usr/sbin/munin10:15:283
2123699291,22cyclictest7950irq/56-dwc_otg_08:25:263
2123599292,7cyclictest31-21ksoftirqd/208:00:322
206422290,22sleep10-21swapper/107:06:041
2123699286,4cyclictest0-21swapper/310:42:103
2123699286,4cyclictest0-21swapper/310:42:103
2123699282,18cyclictest0-21swapper/308:24:153
2123699281,7cyclictest30539-21ssh10:25:333
2123599286,10cyclictest7850irq/56-dwc_otg_07:15:432
2123599285,10cyclictest7750irq/56-dwc_otg09:00:332
2123599283,17cyclictest0-21swapper/208:41:142
2123599283,10cyclictest7750irq/56-dwc_otg08:15:412
2123599283,10cyclictest7750irq/56-dwc_otg08:15:412
21236992717,7cyclictest18019-21aten_rbpower_po08:30:293
201582270,6sleep20-21swapper/207:05:392
207262263,14sleep30-21swapper/307:07:113
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional