You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-08 - 17:36

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Mon Dec 08, 2025 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1354225660,5sleep30-21swapper/307:05:263
15347999322,54cyclictest29597-21wc12:20:450
15347999120,57cyclictest10923-21processes08:25:370
15347999118,56cyclictest22370-21idleruntime-cro07:30:140
1534799896,58cyclictest0-21swapper/007:20:270
15347998827,46cyclictest0-21swapper/012:00:270
15347998827,44cyclictest20947-21ssh10:35:440
15347998824,47cyclictest26786-21missed_timers07:40:370
15347998822,52cyclictest24498-21wget07:35:300
15347998325,47cyclictest4716-21ssh10:13:060
15347998324,40cyclictest1576-21awk10:55:260
15347998323,46cyclictest27351-21sh11:32:220
15347998323,43cyclictest4246-21fschecks_count09:25:340
138842830,3sleep00-21swapper/011:13:340
217462820,5sleep01534799cyclictest08:55:390
15347998225,39cyclictest8083-21latency_hist08:20:150
15347998225,39cyclictest8083-21latency_hist08:20:150
15347998223,38cyclictest21005-21aten_rbpower_po09:50:290
15347998125,41cyclictest22944-21tune2fs07:30:320
15347998123,42cyclictest29305-21ssh09:14:250
15347998022,43cyclictest28811-21taskset10:46:590
149882800,2sleep20-21swapper/209:40:452
15347997923,44cyclictest19011-21ssh11:20:370
15347997922,39cyclictest0-21swapper/009:20:150
15347997921,44cyclictest1280-21chrt10:55:150
15347997827,31cyclictest15496-21ls11:15:360
15347997823,42cyclictest0-21swapper/008:15:150
15347997823,40cyclictest5701-21ssh12:32:110
15347997822,43cyclictest24341-21runrttasks12:15:040
15347997820,48cyclictest24982-21sshd10:41:240
15347997820,39cyclictest14980-21taskset08:36:180
15347997819,47cyclictest9809-21chrt12:37:520
15347997815,48cyclictest0-21swapper/009:39:250
15347997721,39cyclictest0-21swapper/009:30:440
15347997719,45cyclictest0-21swapper/011:09:350
15347997625,41cyclictest990-21sh10:07:280
15347997619,39cyclictest0-21swapper/010:24:280
15347997617,45cyclictest0-21swapper/009:22:420
15347997616,43cyclictest0-21swapper/008:50:120
15347997520,40cyclictest0-21swapper/008:09:530
15347997519,45cyclictest18972-21ssh12:06:280
15347997519,40cyclictest7267-21ssh10:15:570
15347997514,42cyclictest0-21swapper/009:03:520
15347997427,28cyclictest5632-21ssh11:00:410
15347997422,35cyclictest24860-21ssh11:29:500
15347997421,39cyclictest8875-21grep08:20:350
15347997421,39cyclictest31290-21rm11:38:030
15347997421,39cyclictest24740-21ssh12:15:250
15347997421,39cyclictest17299-21ssh09:45:190
15347997421,39cyclictest0-21swapper/007:46:130
278782730,2sleep20-21swapper/210:00:302
15347997326,31cyclictest17030-21chrt08:44:230
15347997319,41cyclictest0-21swapper/011:52:140
15347997319,41cyclictest0-21swapper/011:52:140
15347997316,42cyclictest0-21swapper/012:26:290
15349997217,35cyclictest12820-21sh11:58:002
15347997220,39cyclictest16248-21dpkg10:30:270
15347997219,39cyclictest0-21swapper/009:42:130
15347997219,39cyclictest0-21swapper/009:06:150
15347997218,40cyclictest0-21swapper/011:44:040
15347997121,39cyclictest0-21swapper/007:51:070
15347997121,39cyclictest0-21swapper/007:51:070
15347997120,39cyclictest0-21swapper/011:46:330
15347997120,12cyclictest29585-21ssh10:01:460
15347997118,40cyclictest0-21swapper/007:12:350
15347997117,38cyclictest0-21swapper/010:27:160
15347997117,38cyclictest0-21swapper/010:27:160
15347997025,33cyclictest1660-21ntp_kernel_pll_08:00:360
15347997024,26cyclictest0-21swapper/007:56:230
15347997017,41cyclictest27247-21ssh09:59:230
15347997017,41cyclictest27247-21ssh09:59:230
147902700,1sleep30-21swapper/310:27:363
147902700,1sleep30-21swapper/310:27:363
15347996923,33cyclictest0-21swapper/008:31:220
15347996920,40cyclictest0-21swapper/011:57:550
15348996843,16cyclictest1120-21runrttasks08:55:281
15347996814,42cyclictest0-21swapper/008:52:380
1534999675,44cyclictest20890-21/usr/sbin/munin09:50:412
1534999673,53cyclictest4574-21memory09:25:382
1534999673,51cyclictest9496-21users08:20:442
1534999672,47cyclictest0-21swapper/207:30:262
15347996722,32cyclictest0-21swapper/007:17:270
151822670,3sleep20-21swapper/207:10:152
239112660,2sleep10-21swapper/109:54:451
15349996516,9cyclictest7950irq/56-dwc_otg_12:00:272
1534999641,51cyclictest10073-21ssh12:38:232
1535099633,13cyclictest7750irq/56-dwc_otg08:45:283
1534999633,42cyclictest7950irq/56-dwc_otg_10:55:382
1534999631,47cyclictest30505-21ssh10:50:202
1534999631,47cyclictest0-21swapper/207:25:332
1534999622,41cyclictest7950irq/56-dwc_otg_10:10:302
1534999621,48cyclictest0-21swapper/211:18:092
1534999601,44cyclictest7850irq/56-dwc_otg_10:44:212
15349996014,40cyclictest24869-21aten_rbpower_po12:15:302
1534999601,42cyclictest7950irq/56-dwc_otg_12:35:162
1534999595,40cyclictest7950irq/56-dwc_otg_11:35:442
1534999594,39cyclictest7950irq/56-dwc_otg_10:30:362
1534999593,45cyclictest7850irq/56-dwc_otg_08:04:222
1534999593,38cyclictest7950irq/56-dwc_otg_08:55:312
1534999592,39cyclictest7850irq/56-dwc_otg_12:15:152
1534999591,5cyclictest20562-21apt07:25:142
1534999591,4cyclictest24411-21apt-get07:35:282
1534999591,45cyclictest21404-21ssh10:36:132
1534999591,38cyclictest0-21swapper/211:34:022
1534999585,37cyclictest7950irq/56-dwc_otg_09:59:072
1534999585,37cyclictest7950irq/56-dwc_otg_09:59:072
1534999583,41cyclictest7950irq/56-dwc_otg_09:20:252
1534999577,38cyclictest7750irq/56-dwc_otg10:21:452
1534999572,9cyclictest0-21swapper/211:50:292
1534999572,9cyclictest0-21swapper/211:50:292
1534999572,7cyclictest0-21swapper/207:55:292
15349995718,37cyclictest23657-21processes09:00:422
1534999571,5cyclictest19647-21ssh09:48:102
1534999571,4cyclictest8383-21ssh10:19:172
1534999571,4cyclictest13239-21ssh09:39:492
1534999571,4cyclictest0-21swapper/212:22:152
185182560,2sleep30-21swapper/310:32:553
1535099562,13cyclictest23467-21ssh10:40:283
1534999561,4cyclictest6156-21apt-get08:15:172
1534999561,4cyclictest6156-21apt-get08:15:172
1534999561,4cyclictest29330-21ssh09:14:292
1534999561,37cyclictest0-21swapper/211:07:102
1534999556,34cyclictest0-21swapper/211:20:412
1534999551,5cyclictest8778-21ssh09:31:072
1534999551,4cyclictest2948-21ssh12:29:242
1534999551,39cyclictest0-21swapper/212:09:422
1534999551,37cyclictest0-21swapper/211:40:402
1534999545,38cyclictest0-21swapper/211:48:492
1534999542,38cyclictest27301-21grep07:40:442
1534999541,4cyclictest13566-21ssh11:12:502
1534999541,45cyclictest14859-21ssh10:27:452
1534999541,45cyclictest14859-21ssh10:27:452
1534999541,40cyclictest0-21swapper/211:03:592
1534999531,4cyclictest7750irq/56-dwc_otg08:05:412
1534999531,41cyclictest0-21swapper/211:28:392
1534899537,30cyclictest8192-21ssh11:05:281
1534999521,4cyclictest17188-21apt-get07:15:252
1534999521,48cyclictest29875-21aten_rbpower_en07:50:302
1534999521,48cyclictest29875-21aten_rbpower_en07:50:302
1534999521,47cyclictest27313-21grep10:45:352
150282523,12sleep00-21swapper/007:09:370
1534999511,3cyclictest0-21swapper/207:50:162
1534999501,5cyclictest0-21swapper/208:45:212
1534999501,4cyclictest7950irq/56-dwc_otg_09:20:062
15348995012,15cyclictest0-21swapper/112:05:301
1535099492,7cyclictest0-21swapper/309:40:293
1534999492,38cyclictest0-21swapper/208:39:272
1534999491,44cyclictest24977-21tune2fs09:05:322
1534999491,3cyclictest0-21swapper/210:05:282
1534999491,3cyclictest0-21swapper/208:44:242
1534999491,3cyclictest0-21swapper/208:25:442
1534899497,6cyclictest7750irq/56-dwc_otg08:05:281
1534899495,5cyclictest0-21swapper/108:45:201
1534999482,43cyclictest13528-21latency_hist08:35:142
1534999481,44cyclictest5764-21head08:10:422
1534999472,4cyclictest15390-21apt-get07:10:262
1534999472,42cyclictest19504-21fschecks_count08:50:332
1534899477,25cyclictest0-21swapper/108:30:301
1534899474,6cyclictest2469-21ssh10:10:281
15348994715,9cyclictest0-21swapper/112:00:271
1534899469,24cyclictest0-21swapper/108:15:201
1534899469,24cyclictest0-21swapper/108:15:201
310082451,37sleep30-21swapper/310:05:153
15348994525,5cyclictest1751350wget09:45:301
1534899447,27cyclictest0-21swapper/111:45:291
15348994322,5cyclictest31679-21fschecks_time10:05:351
1534899431,8cyclictest0-21swapper/110:45:281
1535099421,5cyclictest7750irq/56-dwc_otg12:20:303
1534899427,25cyclictest0-21swapper/111:20:291
1534899427,23cyclictest0-21swapper/108:10:261
1534899421,8cyclictest0-21swapper/110:00:291
189452410,5sleep31535099cyclictest09:46:343
1534899417,22cyclictest0-21swapper/109:35:291
1534899416,23cyclictest0-21swapper/111:15:281
1534899416,23cyclictest0-21swapper/108:25:261
1534899415,19cyclictest0-21swapper/112:20:291
1535099407,24cyclictest0-21swapper/310:50:293
1535099407,23cyclictest0-21swapper/309:10:283
1535099404,8cyclictest0-21swapper/311:10:273
1535099403,12cyclictest24457-21ssh09:55:293
1535099403,12cyclictest24457-21ssh09:55:293
1535099402,9cyclictest7750irq/56-dwc_otg07:20:263
1534899406,5cyclictest24784-21taskset12:15:281
15348994021,16cyclictest21-21rcuc/109:00:281
15348994020,5cyclictest746950wget09:30:331
78552390,1sleep30-21swapper/312:35:343
15350993926,9cyclictest0-21swapper/307:40:333
15350993915,6cyclictest18363-21/usr/sbin/munin11:20:373
1535099391,4cyclictest7950irq/56-dwc_otg_10:35:283
1534899397,22cyclictest0-21swapper/111:50:291
1534899397,22cyclictest0-21swapper/111:50:291
1534899396,23cyclictest0-21swapper/109:20:291
1534899394,6cyclictest1880-21fschecks_count10:55:341
15350993817,4cyclictest26231-21diskmemload12:05:153
15350993816,6cyclictest15635-21aten_rbpower_po08:40:293
16692370,2sleep30-21swapper/312:26:483
15350993730,5cyclictest381ktimersoftd/307:35:283
15350993717,4cyclictest10532-21if_eth008:25:313
15350993714,6cyclictest11114-21cut09:35:353
1534899377,5cyclictest16245-21ssh10:30:271
1534899376,10cyclictest0-21swapper/107:45:281
15348993717,5cyclictest12184-21sh11:56:281
1534899371,21cyclictest1305-21grep08:00:321
1535099364,14cyclictest7950irq/56-dwc_otg_11:05:333
15350993625,5cyclictest381ktimersoftd/308:12:303
15350993618,9cyclictest28072-21sh11:33:593
1534899365,19cyclictest23480-21ssh10:40:301
1534899364,3cyclictest7950irq/56-dwc_otg_07:10:331
1534899363,22cyclictest5634-21missed_timers11:00:421
15348993612,7cyclictest4424-21grep12:30:381
1535099359,13cyclictest0-21swapper/310:50:153
1535099358,6cyclictest7950irq/56-dwc_otg_11:45:213
1535099357,25cyclictest0-21swapper/307:25:293
1535099355,5cyclictest7950irq/56-dwc_otg_12:30:283
1535099353,20cyclictest7750irq/56-dwc_otg12:14:253
1535099353,15cyclictest7950irq/56-dwc_otg_09:30:153
15350993513,6cyclictest1008-21aten_rbpower_cu08:00:283
15350993513,6cyclictest0-21swapper/311:40:443
15350993513,4cyclictest12529-21/usr/sbin/munin08:30:383
1534899357,6cyclictest7850irq/56-dwc_otg_10:15:471
1534899357,4cyclictest9954-21processes10:20:411
1534899357,15cyclictest0-21swapper/108:35:281
1534899356,17cyclictest0-21swapper/107:50:291
1534899356,17cyclictest0-21swapper/107:50:291
1534899351,4cyclictest1358-21rm12:26:091
15348993511,6cyclictest7950irq/56-dwc_otg_11:28:121
1535099347,24cyclictest0-21swapper/307:55:313
1535099346,14cyclictest7950irq/56-dwc_otg_07:10:293
1535099343,22cyclictest0-21swapper/308:15:283
1535099343,22cyclictest0-21swapper/308:15:283
1535099342,7cyclictest22121-21grep09:50:443
1535099342,3cyclictest7950irq/56-dwc_otg_09:05:273
15350993419,5cyclictest39-21ksoftirqd/310:10:143
15350993418,12cyclictest7950irq/56-dwc_otg_09:20:383
1535099341,5cyclictest39-21ksoftirqd/308:20:323
1535099341,16cyclictest7950irq/56-dwc_otg_09:35:163
1534899347,12cyclictest0-21swapper/111:12:481
1534899344,4cyclictest7750irq/56-dwc_otg11:30:281
15348993411,12cyclictest0-21swapper/109:19:101
101912340,8sleep37950irq/56-dwc_otg_11:55:083
101912340,8sleep37950irq/56-dwc_otg_11:55:083
30772331,28sleep10-21swapper/111:44:571
1535099337,21cyclictest0-21swapper/310:20:283
1535099332,20cyclictest7850irq/56-dwc_otg_10:10:373
1535099332,15cyclictest7950irq/56-dwc_otg_08:55:373
1535099331,6cyclictest29808-21proc_pri11:35:423
1535099331,19cyclictest7850irq/56-dwc_otg_10:58:153
1534899339,6cyclictest7950irq/56-dwc_otg_08:40:251
1534899335,6cyclictest7850irq/56-dwc_otg_12:10:391
1534899333,6cyclictest7750irq/56-dwc_otg11:35:351
1534899331,21cyclictest22774-21ls07:30:301
1534899331,19cyclictest19550-21fschecks_time08:50:341
1535099327,15cyclictest0-21swapper/310:15:313
1535099326,15cyclictest0-21swapper/311:00:323
1535099324,14cyclictest7850irq/56-dwc_otg_12:16:093
1535099324,13cyclictest7850irq/56-dwc_otg_08:35:423
1535099321,22cyclictest30030-21ssh09:15:293
1534899328,4cyclictest0-21swapper/109:43:321
1534899328,15cyclictest7950irq/56-dwc_otg_10:00:151
1534899328,15cyclictest7950irq/56-dwc_otg_10:00:151
1534899322,15cyclictest0-21swapper/110:50:291
1534899322,10cyclictest0-21swapper/110:28:451
1534899322,10cyclictest0-21swapper/110:28:451
1534899321,5cyclictest7750irq/56-dwc_otg10:35:321
147542324,16sleep10-21swapper/107:06:011
1535099317,5cyclictest39-21ksoftirqd/307:46:093
1535099317,14cyclictest0-21swapper/308:05:303
1535099315,6cyclictest20823-21latency_hist08:55:153
1535099315,12cyclictest7950irq/56-dwc_otg_09:00:323
1535099312,20cyclictest0-21swapper/311:56:403
1535099312,19cyclictest0-21swapper/312:06:453
1535099311,14cyclictest7950irq/56-dwc_otg_11:15:343
1534899313,8cyclictest7750irq/56-dwc_otg07:15:411
15348993120,4cyclictest4490-21latency09:25:371
1535099307,14cyclictest0-21swapper/307:15:273
1535099304,9cyclictest37-21rcuc/307:30:263
1535099301,9cyclictest29876-21wget07:50:303
1535099301,9cyclictest29876-21wget07:50:303
1535099301,15cyclictest7950irq/56-dwc_otg_11:30:033
1534899308,5cyclictest7750irq/56-dwc_otg09:10:391
1534899306,5cyclictest7750irq/56-dwc_otg07:40:281
1534899301,3cyclictest0-21swapper/112:35:341
1534899292,4cyclictest25841-21users09:05:441
1534899291,18cyclictest0-21swapper/107:23:521
15348992910,12cyclictest0-21swapper/107:35:341
1534899283,7cyclictest0-21swapper/108:25:101
1534899282,18cyclictest0-21swapper/107:57:131
1534899262,4cyclictest21782-21ls07:25:421
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional