You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-07-04 - 18:32

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sat Jul 04, 2026 12:43:36)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2351321840,3sleep21336199cyclictest11:05:182
1953421710,3sleep21336199cyclictest12:17:332
13354991022,73cyclictest26969-21grep09:50:410
1335499927,25cyclictest0-21swapper/007:35:250
1336299877,12cyclictest0-21swapper/308:30:263
1335499862,7cyclictest13677-21fschecks_time07:10:310
1335999842,55cyclictest9937-21apt-get10:10:251
77612820,1sleep30-21swapper/311:25:163
141752820,4sleep01335499cyclictest11:32:140
1335499828,61cyclictest0-21swapper/011:50:120
13354998211,48cyclictest5240-21sed09:25:330
52192800,6sleep18050irq/56-dwc_otg_12:00:331
254272800,5sleep11335999cyclictest11:45:391
1335499796,27cyclictest0-21swapper/009:20:240
1335499796,27cyclictest0-21swapper/009:20:240
13354997922,5cyclictest8050irq/56-dwc_otg_10:10:130
1335999781,55cyclictest21103-21/usr/sbin/munin09:00:291
13354997824,4cyclictest8050irq/56-dwc_otg_11:25:120
1335999771,42cyclictest24699-21grep11:05:411
1335499771,17cyclictest27820-21taskset07:50:120
179372760,5sleep01335499cyclictest08:50:340
179372760,5sleep01335499cyclictest08:50:340
1335499751,19cyclictest21914-21fschecks_count09:45:300
1335999741,44cyclictest1448-21/usr/sbin/munin11:55:411
1335999741,42cyclictest17569-21fw_packets12:15:311
13354997211,5cyclictest22042-21grep09:00:410
13354997116,6cyclictest18361-21unixbench-2d12:15:400
13354997115,15cyclictest4168-21sh12:38:210
13354997115,15cyclictest4168-21sh12:38:210
1335999701,56cyclictest31172-21ssh11:52:541
1335999701,53cyclictest13391-21apt-get07:10:251
1335499703,17cyclictest0-21swapper/008:30:400
13354997013,14cyclictest0-21swapper/009:12:020
292512690,1sleep00-21swapper/010:33:080
1335999691,46cyclictest3888-21users10:40:431
1335499698,5cyclictest7950irq/56-dwc_otg_09:45:130
13354996914,15cyclictest19422-21fschecks_time08:55:300
1335499691,22cyclictest32302-21/usr/sbin/munin08:00:340
1335499691,22cyclictest32302-21/usr/sbin/munin08:00:340
1335499691,20cyclictest125-21systemd-journal10:03:200
232772680,2sleep10-21swapper/107:35:361
13362996814,18cyclictest7850irq/56-dwc_otg07:30:263
1335499684,5cyclictest16353-21unixbench-2d07:15:410
13354996819,8cyclictest7850irq/56-dwc_otg11:55:530
13354996817,5cyclictest8050irq/56-dwc_otg_10:35:420
13354996817,5cyclictest8050irq/56-dwc_otg_10:35:420
1335499681,17cyclictest22748-21ssh12:20:500
1335499681,15cyclictest16903-21idleruntime-cro09:40:130
1335999671,54cyclictest15989-21/usr/sbin/munin07:15:361
1335999671,50cyclictest26465-21netstat09:50:351
1335999671,47cyclictest31650-21apt-get08:00:261
1335999671,47cyclictest31650-21apt-get08:00:261
1335999671,46cyclictest21837-21ssh11:02:531
1335999671,45cyclictest12570-21apt-get12:10:151
1335999671,43cyclictest31280-21irqstats10:35:341
1335999671,43cyclictest31280-21irqstats10:35:341
13354996726,6cyclictest0-21swapper/010:20:230
1335499672,17cyclictest0-21swapper/011:25:310
1335499671,6cyclictest29605-21munin-plugin-st07:55:130
1335499671,19cyclictest26309-21ssh12:25:380
182552660,2sleep30-21swapper/309:40:373
1335999661,54cyclictest27913-21ssh09:14:201
1335999661,49cyclictest15124-21sshd08:43:571
1335999661,44cyclictest22921-21ssh12:21:081
13354996615,5cyclictest0-21swapper/012:10:010
1335499661,20cyclictest0-21swapper/011:00:310
13354996610,5cyclictest8050irq/56-dwc_otg_08:25:120
179052650,2sleep20-21swapper/207:20:362
179052650,2sleep20-21swapper/207:20:362
1335999656,44cyclictest7850irq/56-dwc_otg11:25:291
1335999652,47cyclictest21540-21sh09:45:191
1335999651,52cyclictest7595-21ssh09:28:391
1335999651,52cyclictest7421-21ssh11:25:041
1335999651,45cyclictest19910-21taskset10:21:151
1335999651,43cyclictest29833-21/usr/sbin/munin07:55:241
1335999651,43cyclictest22382-21rm11:42:021
1335999651,43cyclictest1371-21apt-get12:35:241
1335999651,43cyclictest1371-21apt-get12:35:241
13354996512,9cyclictest13024-21runrttasks10:52:000
1335499651,21cyclictest0-21swapper/012:04:420
1335499651,18cyclictest0-21swapper/009:15:160
13354996510,14cyclictest10688-21ntp_kernel_pll_10:10:340
44542640,2sleep20-21swapper/212:00:142
13361996416,16cyclictest7850irq/56-dwc_otg12:10:252
1335999641,53cyclictest11428-21sshd08:32:581
1335999641,45cyclictest29555-21ssh11:11:351
1335499642,18cyclictest29994-21fschecks_time12:30:300
1335499642,13cyclictest29831-21ssh11:50:400
13354996414,12cyclictest0-21swapper/011:08:090
1335499641,31cyclictest0-21swapper/010:30:070
13354996412,14cyclictest0-21swapper/007:58:150
1335499641,19cyclictest0-21swapper/011:39:040
1335499641,19cyclictest0-21swapper/011:39:040
1335499641,16cyclictest0-21swapper/011:00:120
1335499641,14cyclictest0-21swapper/010:18:310
13361996322,20cyclictest0-21swapper/207:15:242
1335999631,41cyclictest31039-21ssh09:17:261
1335999631,40cyclictest30378-21ssh10:35:111
1335999631,39cyclictest12762-21/usr/sbin/munin11:30:351
1335999631,38cyclictest0-21swapper/111:17:041
1335499632,19cyclictest0-21swapper/011:20:000
1335499631,19cyclictest31547-21chrt09:56:160
1335499631,18cyclictest0-21swapper/009:31:330
1335499631,16cyclictest0-21swapper/010:45:250
1335499631,16cyclictest0-21swapper/010:45:250
13362996212,19cyclictest7850irq/56-dwc_otg11:00:263
1336199624,12cyclictest14102-21ssh10:15:242
13354996211,16cyclictest0-21swapper/011:14:080
13362996124,6cyclictest8050irq/56-dwc_otg_10:35:273
13362996124,6cyclictest8050irq/56-dwc_otg_10:35:273
1335999611,51cyclictest19285-21ssh11:38:581
1335999611,51cyclictest19285-21ssh11:38:581
1335999611,49cyclictest727-21runrttasks12:10:011
1335999611,48cyclictest28257-21/usr/sbin/munin07:50:271
1335999611,45cyclictest20437-21taskset08:57:451
1335999611,39cyclictest12910-21/usr/sbin/munin08:35:391
1335999611,18cyclictest0-21swapper/109:30:261
1335499612,13cyclictest22442-21rm11:42:090
1335499611,15cyclictest0-21swapper/012:14:470
1335999601,51cyclictest374-1kworker/u9:209:42:151
1335999601,48cyclictest8398-21ssh10:46:191
1335999601,48cyclictest8398-21ssh10:46:191
1335999601,42cyclictest31673-21sh09:56:301
1335999601,41cyclictest374-1kworker/u9:209:22:421
1335999601,41cyclictest374-1kworker/u9:209:22:421
1335999601,35cyclictest0-21swapper/110:15:401
1335999601,33cyclictest3182950ssh12:32:141
13354996019,2cyclictest0-21swapper/007:25:310
13354996013,12cyclictest0-21swapper/007:21:130
13354996013,12cyclictest0-21swapper/007:21:130
1335999595,43cyclictest27914-21latency_hist07:50:131
1335999591,42cyclictest24260-21diskmemload10:04:261
1335999591,40cyclictest10946-21ssh10:50:211
13354995914,13cyclictest0-21swapper/008:11:180
13354995912,10cyclictest0-21swapper/010:43:130
13362995817,18cyclictest7950irq/56-dwc_otg_09:50:223
1335999581,44cyclictest9505-21ssh10:10:021
1335999581,37cyclictest17598-21ssh10:57:441
1335999581,37cyclictest16472-21ssh09:39:271
1335499581,19cyclictest0-21swapper/008:36:090
1335999571,44cyclictest0-21swapper/110:28:531
1336299567,9cyclictest7950irq/56-dwc_otg_08:35:243
1335999561,40cyclictest29196-21ssh12:29:591
1336299553,7cyclictest8050irq/56-dwc_otg_11:30:263
1336299553,4cyclictest0-21swapper/310:20:243
1335999551,33cyclictest0-21swapper/108:30:111
1336299542,9cyclictest7850irq/56-dwc_otg11:55:263
13361995411,3cyclictest7850irq/56-dwc_otg07:35:252
1335999541,41cyclictest20161-21chrt07:26:141
1335999541,38cyclictest2429-21chrt08:08:291
1335999541,37cyclictest3647-21smartpower08:10:371
1335499541,31cyclictest516-21in:imuxsock08:10:120
1335499541,18cyclictest0-21swapper/008:27:540
1335999531,48cyclictest17839-21/usr/sbin/munin08:50:331
1335999531,48cyclictest17839-21/usr/sbin/munin08:50:331
1335999531,38cyclictest59142sleep108:16:581
1335999521,36cyclictest0-21swapper/107:23:231
1335999521,36cyclictest0-21swapper/107:23:231
13354995218,18cyclictest0-21swapper/008:15:240
1336199516,7cyclictest8050irq/56-dwc_otg_09:10:242
1335999512,36cyclictest8015-21munin-run08:25:121
1335999511,38cyclictest0-21swapper/108:49:491
1335999511,38cyclictest0-21swapper/108:49:491
1336199508,10cyclictest7950irq/56-dwc_otg_11:50:242
1335999503,43cyclictest20959-21fschecks_count07:30:291
1335499509,13cyclictest0-21swapper/007:30:260
13354995028,4cyclictest23444-21wc09:05:380
1335999491,45cyclictest24336-21apt-get07:40:261
1336299482,37cyclictest8050irq/56-dwc_otg_12:40:143
1336299482,37cyclictest8050irq/56-dwc_otg_12:40:143
1335999481,38cyclictest0-21swapper/109:08:501
1336299475,4cyclictest0-21swapper/310:05:243
1336199477,25cyclictest15187-21ssh10:55:272
270952462,39sleep20-21swapper/210:30:332
1336299467,25cyclictest0-21swapper/308:45:263
1336299467,25cyclictest0-21swapper/308:45:263
1336299467,25cyclictest0-21swapper/307:40:263
1336199468,25cyclictest0-21swapper/209:25:262
1336199463,7cyclictest8050irq/56-dwc_otg_08:10:232
1335499461,26cyclictest15246-21munin-run08:45:130
128262462,12sleep00-21swapper/007:06:480
13362994512,15cyclictest8050irq/56-dwc_otg_11:05:243
1336199457,25cyclictest0-21swapper/207:45:172
13361994523,11cyclictest301ktimersoftd/209:45:192
1336299447,23cyclictest0-21swapper/307:15:173
1336199447,22cyclictest0-21swapper/208:00:262
1336199447,22cyclictest0-21swapper/208:00:262
1336199443,5cyclictest0-21swapper/209:05:262
13361994413,20cyclictest7868-21ssh11:25:262
1335499446,25cyclictest0-21swapper/007:40:260
13362994313,15cyclictest8050irq/56-dwc_otg_09:30:123
287102420,2sleep20-21swapper/209:53:402
1336299421,25cyclictest8050irq/56-dwc_otg_09:35:363
1336199427,23cyclictest0-21swapper/210:40:182
13361994226,9cyclictest301ktimersoftd/211:45:252
1336199422,18cyclictest7950irq/56-dwc_otg_09:40:292
1336299416,18cyclictest8050irq/56-dwc_otg_10:50:183
1336199416,19cyclictest0-21swapper/212:32:162
1336199413,4cyclictest7950irq/56-dwc_otg_11:35:262
1336199413,4cyclictest7950irq/56-dwc_otg_11:35:262
13361994115,5cyclictest31-21ksoftirqd/211:30:312
13361994111,6cyclictest8050irq/56-dwc_otg_12:35:242
13361994111,6cyclictest8050irq/56-dwc_otg_12:35:242
1335499416,22cyclictest0-21swapper/008:45:260
1335499416,22cyclictest0-21swapper/008:45:260
13362993920,6cyclictest14255-21tail12:10:413
13362993912,21cyclictest8050irq/56-dwc_otg_12:00:233
1336199396,24cyclictest0-21swapper/209:30:262
1336199396,23cyclictest0-21swapper/211:15:182
1336199392,23cyclictest0-21swapper/208:05:262
1336199391,5cyclictest0-21swapper/211:10:242
13361993914,15cyclictest0-21swapper/210:45:272
13361993914,15cyclictest0-21swapper/210:45:272
1336299387,23cyclictest0-21swapper/307:45:263
1336299384,13cyclictest0-21swapper/309:45:183
13362993816,12cyclictest24260-21diskmemload12:28:083
1336299378,23cyclictest0-21swapper/309:20:263
1336299378,23cyclictest0-21swapper/309:20:263
1336299377,25cyclictest0-21swapper/307:55:273
1336299373,19cyclictest8050irq/56-dwc_otg_12:30:303
1336199372,6cyclictest4613-21/usr/sbin/munin08:15:412
1336199371,6cyclictest2022150ssh11:00:382
13361993715,5cyclictest4181-21modprobe12:00:122
13361993714,8cyclictest7950irq/56-dwc_otg_09:00:282
1336299363,25cyclictest8050irq/56-dwc_otg_11:50:243
1336299361,6cyclictest0-21swapper/310:45:253
1336299361,6cyclictest0-21swapper/310:45:253
1336199366,7cyclictest0-21swapper/210:20:272
1336199364,23cyclictest9957-21ssh10:10:252
1336199361,6cyclictest23117-21missed_timers10:25:372
247602350,1sleep30-21swapper/310:27:313
1336299355,5cyclictest7850irq/56-dwc_otg09:30:413
1336299355,18cyclictest8050irq/56-dwc_otg_10:40:283
1336299353,5cyclictest5754-21unixbench_singl08:15:413
1336299352,7cyclictest8050irq/56-dwc_otg_11:00:123
13362993518,4cyclictest6731-21fschecks_time08:20:313
1336299351,3cyclictest7950irq/56-dwc_otg_12:05:243
1336199356,5cyclictest31-21ksoftirqd/207:10:252
1336199353,7cyclictest30803-21grep10:35:282
1336199353,7cyclictest30803-21grep10:35:282
1336199352,10cyclictest29-21rcuc/212:10:132
13361993515,5cyclictest31-21ksoftirqd/208:55:362
1336199351,24cyclictest24233-21ssh11:45:122
1336199351,18cyclictest0-21swapper/207:40:382
1336199351,17cyclictest15388-21ls08:45:132
1336199351,14cyclictest31-21ksoftirqd/208:35:392
1336299349,4cyclictest8050irq/56-dwc_otg_11:49:453
1336299343,7cyclictest3123-21fschecks_time08:10:293
1336299342,7cyclictest304-21grep08:00:403
1336299342,7cyclictest304-21grep08:00:403
1336299341,21cyclictest17572-21ls12:15:313
1336199349,8cyclictest7850irq/56-dwc_otg08:30:342
1336199348,14cyclictest29-21rcuc/210:50:232
125482340,1sleep10-21swapper/107:05:391
1336299338,6cyclictest8050irq/56-dwc_otg_11:37:303
1336299338,6cyclictest8050irq/56-dwc_otg_11:37:303
1336299338,4cyclictest8050irq/56-dwc_otg_07:25:413
1336299337,6cyclictest23665-21grep07:35:403
1336299337,4cyclictest0-21swapper/310:16:403
1336299335,5cyclictest8050irq/56-dwc_otg_11:20:403
1336299333,19cyclictest24326-21latency_hist11:45:133
1336299333,14cyclictest20837-1kworker/u9:011:10:303
1336299332,7cyclictest20826-21latency_hist09:00:133
1336299331,4cyclictest26597-21ssh10:30:253
13362993310,4cyclictest0-21swapper/310:10:443
1336199338,8cyclictest7850irq/56-dwc_otg09:55:392
1336199337,14cyclictest0-21swapper/207:50:252
1336199336,14cyclictest0-21swapper/208:25:262
1336199333,8cyclictest8050irq/56-dwc_otg_11:20:342
13361993326,5cyclictest6509-21aten_rbpower_po08:20:282
1336199331,22cyclictest30205-21idleruntime07:55:312
1336299327,6cyclictest7950irq/56-dwc_otg_07:50:313
1336299326,5cyclictest8050irq/56-dwc_otg_07:20:293
1336299326,5cyclictest8050irq/56-dwc_otg_07:20:293
1336299323,16cyclictest8050irq/56-dwc_otg_08:25:293
1336299322,11cyclictest2556650ssh09:10:393
1336299321,3cyclictest0-21swapper/311:16:333
1336199324,14cyclictest29-21rcuc/212:23:572
13361993227,4cyclictest31-21ksoftirqd/209:15:352
1336199321,9cyclictest26051-21chrt12:25:342
1336199321,24cyclictest6845-21timerandwakeup10:05:392
1336299314,13cyclictest8050irq/56-dwc_otg_09:05:303
1336299314,10cyclictest8050irq/56-dwc_otg_10:03:233
1336299311,7cyclictest1329-21grep08:05:303
1336199317,6cyclictest7850irq/56-dwc_otg09:35:352
1336199316,6cyclictest1671-21aten_rbpower_en10:00:242
13361993125,4cyclictest0-21swapper/209:20:292
13361993125,4cyclictest0-21swapper/209:20:292
1336299308,4cyclictest0-21swapper/312:22:403
1336299304,13cyclictest8050irq/56-dwc_otg_09:55:373
1336299302,6cyclictest0-21swapper/307:10:393
1336299302,11cyclictest18085-21rt-features08:50:373
1336299302,11cyclictest18085-21rt-features08:50:373
1336299301,6cyclictest29929-21sed09:15:393
1336199308,5cyclictest7950irq/56-dwc_otg_07:25:242
1336199305,11cyclictest0-21swapper/207:30:402
1336199302,21cyclictest0-21swapper/208:51:592
1336199302,21cyclictest0-21swapper/208:51:592
122812305,8sleep20-21swapper/207:05:372
1336199281,16cyclictest15718-21cut08:45:302
1336199281,16cyclictest15718-21cut08:45:302
1336299272,14cyclictest8050irq/56-dwc_otg_08:40:213
1336299271,7cyclictest0-21swapper/309:03:283
127452263,15sleep30-21swapper/307:05:463
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional