You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-09-04 - 20:28

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Fri Sep 04, 2026 12:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3089821700,4sleep31562199cyclictest10:30:373
15618991053,80cyclictest0-21swapper/012:15:390
2922921020,3sleep10-21swapper/110:29:181
2922921020,3sleep10-21swapper/110:29:181
15619999039,24cyclictest20607-21/usr/sbin/munin12:15:361
53822890,2sleep10-21swapper/111:56:501
1561899886,73cyclictest2413-21kworker/u8:411:30:230
141362830,2sleep20-21swapper/211:29:012
15572790,5sleep11561999cyclictest09:18:291
214662780,6sleep11561999cyclictest09:41:031
159492760,5sleep01561899cyclictest09:35:300
159492760,5sleep01561899cyclictest09:35:300
324492730,6sleep31562199cyclictest12:30:263
1561999719,23cyclictest13156-21sh10:10:331
15621997027,22cyclictest7950irq/56-dwc_otg_12:15:373
1561999704,11cyclictest354-21ssh09:55:281
1561999704,11cyclictest354-21ssh09:55:281
219722690,2sleep00-21swapper/010:59:200
15619996929,11cyclictest22275-21memory10:20:371
1561899693,55cyclictest0-21swapper/012:30:360
1561999681,15cyclictest11038-21ssh11:25:271
295162670,2sleep30-21swapper/310:29:483
295162670,2sleep30-21swapper/310:29:483
15619996737,16cyclictest31134-21timerandwakeup07:50:411
15620996618,29cyclictest301ktimersoftd/212:15:372
15619996628,18cyclictest13690-21ssh10:49:161
15619996625,15cyclictest12556-21fschecks_time08:30:331
15619996517,26cyclictest7750irq/56-dwc_otg11:15:251
1561899652,45cyclictest27011-21ssh11:05:290
302442640,4sleep121-21rcuc/109:51:221
15619996427,20cyclictest7750irq/56-dwc_otg08:25:291
15619996427,20cyclictest7750irq/56-dwc_otg08:25:291
15618996425,6cyclictest7950irq/56-dwc_otg_09:45:260
212122630,1sleep30-21swapper/311:36:393
212122630,1sleep30-21swapper/311:36:393
15619996333,14cyclictest26309-21sshd07:39:571
15619996333,14cyclictest26309-21sshd07:39:571
15619996331,22cyclictest8211-21ssh10:04:451
15619996331,14cyclictest7356-21ssh10:40:431
15619996320,13cyclictest23010-21cpuspeed11:00:291
1561899631,46cyclictest23012-21aten_rbpower_cu07:30:280
15619996226,27cyclictest17093-21rm11:31:401
15619996226,21cyclictest1724-21sh11:52:411
1561899622,44cyclictest2445-21grep08:00:430
1561899622,34cyclictest0-21swapper/011:10:370
15619996132,11cyclictest31234-21ssh11:49:541
15619996127,12cyclictest3307-21fschecks_time09:20:331
15619996124,13cyclictest23807-21grep09:00:361
15619996120,24cyclictest0-21swapper/107:55:371
1561899613,51cyclictest0-21swapper/007:20:270
1561899611,44cyclictest7281-21munin-run12:00:150
15618996113,5cyclictest19686-21fschecks_count11:35:330
15618996113,5cyclictest19686-21fschecks_count11:35:330
15619996036,12cyclictest28593-21df_inode07:45:301
15619996027,23cyclictest19629-21ssh10:17:211
15619996022,19cyclictest7950irq/56-dwc_otg_12:31:091
15619996010,5cyclictest0-21swapper/112:20:281
1561899604,50cyclictest0-21swapper/012:25:160
1561899604,39cyclictest0-21swapper/010:00:290
15618996041,11cyclictest101ktimersoftd/010:05:420
15618996040,5cyclictest101ktimersoftd/010:20:160
1561899601,14cyclictest0-21swapper/011:40:300
15619995933,11cyclictest22161-21ssh11:38:231
15619995933,11cyclictest22161-21ssh11:38:231
15619995930,18cyclictest8060-21sshd08:16:561
15619995924,22cyclictest11933-21ssh10:08:481
15619995921,19cyclictest30114-21ssh11:09:081
1561899599,39cyclictest9-21ksoftirqd/011:20:360
1561899599,39cyclictest9-21ksoftirqd/011:20:360
1561899598,38cyclictest26532-21diskmemload09:40:370
1561899596,27cyclictest0-21swapper/008:40:280
15618995925,26cyclictest32159-21ls07:55:270
1561899592,42cyclictest0-21swapper/009:59:560
1561899592,42cyclictest0-21swapper/009:59:560
15619995827,13cyclictest7750irq/56-dwc_otg07:45:151
15619995823,12cyclictest24764-21grep09:45:341
1561899588,38cyclictest15163-21unixbench_singl08:35:450
1561899586,27cyclictest0-21swapper/008:45:210
1561899586,21cyclictest28494-21ssh09:50:210
15618995836,5cyclictest7950irq/56-dwc_otg_11:45:430
15621995718,7cyclictest7950irq/56-dwc_otg_12:05:283
15621995718,7cyclictest7950irq/56-dwc_otg_12:05:283
15619995730,12cyclictest6770-21ssh12:38:081
15619995729,12cyclictest32671-21ssh11:11:151
15619995728,13cyclictest9012-21ssh12:00:541
15619995727,17cyclictest15633-21/usr/sbin/munin07:10:411
15619995726,22cyclictest1454-21chrt10:35:021
15619995724,21cyclictest18242-21ssh12:12:421
15619995723,14cyclictest16203-21systemd-journal10:50:391
15619995716,26cyclictest7850irq/56-dwc_otg_09:31:531
1561899577,47cyclictest0-21swapper/010:20:290
15618995726,4cyclictest7950irq/56-dwc_otg_11:15:270
1561899572,44cyclictest32575-21grep11:50:410
1561899572,42cyclictest0-21swapper/007:50:150
1561899571,40cyclictest0-21swapper/008:05:330
15619995627,19cyclictest10179-21ssh09:28:371
15619995625,20cyclictest13525-21sh12:06:361
15619995625,20cyclictest13525-21sh12:06:361
15619995526,19cyclictest19034-21ssh09:39:141
15619995526,19cyclictest19034-21ssh09:39:141
15619995525,13cyclictest30059-21ssh12:26:301
15619995522,11cyclictest0-21swapper/111:40:361
15619995521,13cyclictest4262-21ssh10:37:291
15619995518,20cyclictest0-21swapper/110:57:581
15619995517,16cyclictest10552-21taskset11:25:011
15619995517,16cyclictest10552-21taskset11:25:011
1561899551,44cyclictest16169-21fschecks_time12:10:320
15618995511,31cyclictest14958-21fschecks_time10:50:360
72082540,1sleep00-21swapper/012:38:560
15619995424,11cyclictest20639-21taskset07:21:441
15619995423,13cyclictest21197-21latency_hist08:55:161
15619995419,12cyclictest28754-21chrt09:11:441
1561899542,40cyclictest0-21swapper/009:19:400
1561899541,42cyclictest0-21swapper/011:00:330
1561899541,42cyclictest0-21swapper/010:35:430
1561899541,40cyclictest12706-21unixbench_singl09:30:420
1561899541,39cyclictest0-21swapper/009:00:270
1561899541,36cyclictest12760-21timerandwakeup12:05:410
1561899541,36cyclictest12760-21timerandwakeup12:05:410
1561899541,35cyclictest32108-21ssh12:30:150
1562199539,11cyclictest7950irq/56-dwc_otg_11:20:293
1562199539,11cyclictest7950irq/56-dwc_otg_11:20:293
15621995340,11cyclictest60-21kswapd012:20:283
15619995320,18cyclictest0-21swapper/108:45:401
1561899532,41cyclictest10483-21aten_rbpower_po08:25:290
1561899532,41cyclictest10483-21aten_rbpower_po08:25:290
1561899532,39cyclictest0-21swapper/007:35:410
1561899532,39cyclictest0-21swapper/007:35:410
1561899531,40cyclictest0-21swapper/009:05:270
1562199525,5cyclictest7950irq/56-dwc_otg_07:40:293
15621995214,12cyclictest7850irq/56-dwc_otg_09:40:263
15619995242,8cyclictest17454-21apt-get07:15:271
1561999523,6cyclictest7750irq/56-dwc_otg07:25:291
1561899524,40cyclictest9-21ksoftirqd/008:30:430
15618995233,13cyclictest101ktimersoftd/007:50:410
15618995228,7cyclictest9-21ksoftirqd/011:25:430
15618995217,22cyclictest0-21swapper/010:42:580
15618995211,25cyclictest0-21swapper/009:25:430
15619995122,21cyclictest17070-21runrttasks08:41:161
15619995120,9cyclictest0-21swapper/109:09:171
1561899518,31cyclictest0-21swapper/010:45:270
1561899511,41cyclictest0-21swapper/008:23:200
1561899511,38cyclictest30459-21fschecks_time10:30:310
1561899511,38cyclictest0-21swapper/012:00:310
129792510,19sleep27950irq/56-dwc_otg_10:47:592
15619995027,11cyclictest24518-21chrt07:35:041
15619995018,15cyclictest7750irq/56-dwc_otg08:36:171
1561899501,40cyclictest9-21ksoftirqd/010:26:350
1561899501,40cyclictest9-21ksoftirqd/010:26:350
1561899501,39cyclictest13200-21basename10:10:340
15618995011,4cyclictest26904-21aten_rbpower_cu09:10:280
1562199492,15cyclictest7950irq/56-dwc_otg_07:20:283
1562099496,18cyclictest0-21swapper/212:25:222
15619994933,14cyclictest3166-21/usr/sbin/munin08:05:341
15619994923,11cyclictest2777-21runrttasks08:03:311
15619994916,14cyclictest0-21swapper/108:22:401
15619994915,13cyclictest0-21swapper/108:11:521
1561899497,32cyclictest0-21swapper/008:15:280
1561899494,8cyclictest0-21swapper/007:40:290
15618994923,6cyclictest0-21swapper/009:20:430
1562099481,35cyclictest7950irq/56-dwc_otg_08:50:332
1562099477,25cyclictest0-21swapper/208:20:282
1561899471,35cyclictest0-21swapper/007:15:430
1562099467,25cyclictest0-21swapper/211:00:282
1562099467,25cyclictest0-21swapper/209:05:292
15618994629,9cyclictest62212sleep008:11:300
1562199456,25cyclictest0-21swapper/311:40:303
1562099457,23cyclictest0-21swapper/207:15:292
1561899452,31cyclictest0-21swapper/008:55:270
15618994523,14cyclictest101ktimersoftd/008:55:160
55572440,5sleep35536-21missed_timers08:10:363
15619994429,12cyclictest21729-21/usr/sbin/munin08:55:321
152492442,35sleep00-21swapper/007:08:590
152492442,35sleep00-21swapper/007:08:590
1562199437,26cyclictest0-21swapper/308:25:283
1562199437,26cyclictest0-21swapper/308:25:283
1562199431,5cyclictest7950irq/56-dwc_otg_09:10:273
15618994326,11cyclictest101ktimersoftd/007:27:130
1561899431,32cyclictest0-21swapper/007:15:150
279562420,5sleep21562099cyclictest11:45:322
1562199427,25cyclictest0-21swapper/309:00:283
1562199426,25cyclictest0-21swapper/308:00:293
1562199423,5cyclictest0-21swapper/308:55:263
1562099425,5cyclictest0-21swapper/211:10:312
93762410,1sleep30-21swapper/309:27:123
1562199417,25cyclictest0-21swapper/309:50:213
1562199416,4cyclictest7950irq/56-dwc_otg_07:45:273
15621994118,13cyclictest1180-21runrttasks10:00:163
15621994118,13cyclictest1180-21runrttasks10:00:163
1562099416,23cyclictest1180-21runrttasks08:15:272
261082400,1sleep30-21swapper/309:46:453
1562199409,6cyclictest39-21ksoftirqd/310:20:163
1562199407,24cyclictest0-21swapper/311:30:213
1562199407,21cyclictest0-21swapper/307:55:263
1562099408,17cyclictest17217-1kworker/u9:112:00:302
158322390,2sleep315826-21ssh12:10:253
1562199397,25cyclictest0-21swapper/307:50:283
1562099391,28cyclictest9144-21grep10:05:312
1562199388,3cyclictest0-21swapper/311:55:293
15621993819,4cyclictest37-21rcuc/308:30:413
1562099382,9cyclictest7850irq/56-dwc_otg_10:30:252
1562099382,23cyclictest7750irq/56-dwc_otg11:15:412
15620993818,10cyclictest3214-21fschecks_count09:20:322
1562099381,28cyclictest16001-21ls07:10:332
15620993810,5cyclictest28414-21aten_rbpower_cu07:45:282
316042370,1sleep30-21swapper/311:50:163
15621993721,6cyclictest3147550ssh11:10:383
15621993717,5cyclictest37-21rcuc/310:20:293
15621993715,12cyclictest1365750ssh09:32:163
1562199371,4cyclictest0-21swapper/310:10:333
1562099378,5cyclictest7750irq/56-dwc_otg11:55:162
1562099377,21cyclictest0-21swapper/212:35:272
1562099375,14cyclictest7850irq/56-dwc_otg_07:35:352
1562099375,14cyclictest7850irq/56-dwc_otg_07:35:352
1562099373,5cyclictest7750irq/56-dwc_otg12:10:252
1562099371,24cyclictest13129-21timerandwakeup08:30:422
1562199366,7cyclictest7950irq/56-dwc_otg_10:55:163
1562199366,10cyclictest37-21rcuc/310:40:443
15621993619,4cyclictest18865-21sh09:38:563
15621993619,4cyclictest18865-21sh09:38:563
1562099363,25cyclictest31-21ksoftirqd/211:35:272
1562099363,25cyclictest31-21ksoftirqd/211:35:272
1562099362,6cyclictest7850irq/56-dwc_otg_11:35:172
1562099362,5cyclictest7950irq/56-dwc_otg_10:55:272
15620993613,5cyclictest26532-21diskmemload09:35:392
15620993613,5cyclictest26532-21diskmemload09:35:392
145382367,12sleep20-21swapper/207:05:412
145382367,12sleep20-21swapper/207:05:412
1562199357,14cyclictest0-21swapper/310:55:293
1562199356,6cyclictest7850irq/56-dwc_otg_08:20:413
1562199356,14cyclictest0-21swapper/308:45:213
1562199356,13cyclictest39-21ksoftirqd/312:35:423
1562199352,4cyclictest0-21swapper/310:37:123
1562199352,10cyclictest0-21swapper/311:00:273
15621993518,4cyclictest3849-21sh09:20:403
1562099357,13cyclictest22888-21ls07:30:162
1562099355,3cyclictest7850irq/56-dwc_otg_07:55:402
1562099353,3cyclictest7750irq/56-dwc_otg09:55:332
1562099353,3cyclictest7750irq/56-dwc_otg09:55:332
15620993518,4cyclictest15557-21switchtime10:50:452
1562099351,24cyclictest0-21swapper/209:25:402
1562199347,6cyclictest7950irq/56-dwc_otg_10:45:303
1562199346,14cyclictest0-21swapper/308:35:303
1562199343,14cyclictest7950irq/56-dwc_otg_11:15:303
15621993417,8cyclictest7950irq/56-dwc_otg_11:05:283
1562199341,16cyclictest31862-21basename11:50:323
1562099348,11cyclictest7850irq/56-dwc_otg_09:40:412
1562099345,13cyclictest7950irq/56-dwc_otg_08:05:162
1562099343,16cyclictest7950irq/56-dwc_otg_08:30:172
1562099341,7cyclictest7950irq/56-dwc_otg_08:45:422
1562099341,3cyclictest0-21swapper/211:43:002
1562099341,27cyclictest22939-21ls09:00:162
1562199337,14cyclictest0-21swapper/307:30:283
1562199336,5cyclictest7950irq/56-dwc_otg_08:05:273
1562199335,19cyclictest4629-21ssh10:00:293
1562199332,8cyclictest0-21swapper/312:00:273
1562199331,7cyclictest9170-21sort10:05:313
1562199331,21cyclictest2668-21ls09:20:163
1562099338,13cyclictest0-21swapper/210:10:382
1562099337,4cyclictest0-21swapper/211:55:402
1562099337,4cyclictest0-21swapper/210:27:572
1562099337,4cyclictest0-21swapper/210:27:572
1562099337,12cyclictest7850irq/56-dwc_otg_08:35:402
1562099335,15cyclictest7950irq/56-dwc_otg_12:20:432
1562099333,14cyclictest7950irq/56-dwc_otg_08:30:172
1562099333,14cyclictest7850irq/56-dwc_otg_10:00:302
1562099333,11cyclictest7950irq/56-dwc_otg_09:48:252
1562099332,20cyclictest0-21swapper/209:15:382
1562099332,17cyclictest0-21swapper/208:45:162
1562099331,21cyclictest17787-21ls10:15:352
1562199327,15cyclictest0-21swapper/308:40:283
1562199326,14cyclictest0-21swapper/307:35:273
1562199326,14cyclictest0-21swapper/307:35:273
15621993212,4cyclictest22878-21latency_hist07:30:163
1562199321,20cyclictest25299-21ls09:05:333
15621993210,6cyclictest16222-21runrttasks07:10:363
1562099329,4cyclictest0-21swapper/211:20:372
1562099329,4cyclictest0-21swapper/211:20:372
1562099326,6cyclictest7750irq/56-dwc_otg09:50:352
1562099325,12cyclictest7950irq/56-dwc_otg_07:30:422
1562099324,24cyclictest7950irq/56-dwc_otg_08:05:292
1562099324,10cyclictest7950irq/56-dwc_otg_10:40:332
1562099323,6cyclictest7750irq/56-dwc_otg10:35:362
1562099322,4cyclictest0-21swapper/211:05:562
1562099321,6cyclictest2198650ssh10:20:332
143742327,14sleep313798-21/usr/sbin/munin07:05:383
143742327,14sleep313798-21/usr/sbin/munin07:05:383
1562099311,6cyclictest7850irq/56-dwc_otg_09:00:252
1562099311,14cyclictest7750irq/56-dwc_otg09:30:392
1562199308,10cyclictest0-21swapper/312:25:193
1562199307,5cyclictest0-21swapper/311:25:273
1562099308,5cyclictest31-21ksoftirqd/212:30:422
1562099307,12cyclictest0-21swapper/212:08:002
1562099307,12cyclictest0-21swapper/212:08:002
1562099305,6cyclictest7950irq/56-dwc_otg_07:20:322
1562099302,18cyclictest0-21swapper/207:44:262
15620993012,11cyclictest26839-21/usr/sbin/munin09:10:432
15621992923,4cyclictest17513-21cut07:15:293
1562199291,9cyclictest7850irq/56-dwc_otg_08:50:433
1562199291,4cyclictest7950irq/56-dwc_otg_08:15:373
15620992923,4cyclictest5574-21ls08:10:362
1562099283,6cyclictest7750irq/56-dwc_otg07:50:282
137932264,15sleep10-21swapper/107:05:241
137932264,15sleep10-21swapper/107:05:241
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional