You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-08-12 - 14:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Wed Aug 12, 2026 00:43:45)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
528121610,2sleep2354699cyclictest22:17:582
3546991216,86cyclictest10941-21sort21:45:252
35449912060,25cyclictest0-21swapper/020:05:040
35449912060,25cyclictest0-21swapper/020:05:040
35449911238,26cyclictest0-21swapper/020:35:200
3546991105,80cyclictest0-21swapper/221:50:292
3544991109,83cyclictest0-21swapper/021:35:110
3544991109,83cyclictest0-21swapper/021:35:110
35479910547,22cyclictest27937-21ssh22:05:333
3546991052,61cyclictest27994-21ls22:05:332
35479910437,45cyclictest11069-21ntpq21:45:283
3544991046,67cyclictest0-21swapper/000:10:260
3546991003,64cyclictest0-21swapper/200:10:042
3546991003,64cyclictest0-21swapper/200:10:042
3545999921,18cyclictest11027-21nfs4_client21:45:261
3547999726,49cyclictest15445-21timerandwakeup21:50:293
3544999739,42cyclictest0-21swapper/022:25:210
354499966,47cyclictest0-21swapper/022:05:320
354499953,75cyclictest7950irq/56-dwc_otg_00:00:340
3544999529,51cyclictest15297-21kworker/0:000:10:050
3544999529,51cyclictest15297-21kworker/0:000:10:050
354599943,8cyclictest0-21swapper/121:50:291
354599933,12cyclictest27937-21ssh22:05:331
354699923,64cyclictest1249-21ntp_states00:10:282
218812920,2sleep20-21swapper/221:59:142
354599913,13cyclictest0-21swapper/100:10:281
3544999119,39cyclictest0-21swapper/023:45:170
3544999065,7cyclictest0-21swapper/021:49:500
354699893,65cyclictest3989-21ls22:55:252
3546998925,58cyclictest23299-21latency_hist20:05:052
3546998925,58cyclictest23299-21latency_hist20:05:052
354699882,55cyclictest11378-21ssh22:25:232
3544998731,26cyclictest0-21swapper/020:05:060
354499863,78cyclictest0-21swapper/021:50:300
354499862,66cyclictest20048-21sh23:15:200
354499855,36cyclictest0-21swapper/022:55:270
354699833,53cyclictest0-21swapper/200:00:332
354499834,40cyclictest0-21swapper/022:15:160
3547998144,21cyclictest3996-21ssh22:55:253
354699813,73cyclictest23591-21apt-config20:05:062
354499811,37cyclictest0-21swapper/023:55:170
3547997931,30cyclictest0-21swapper/323:45:093
354699796,48cyclictest7950irq/56-dwc_otg_23:15:212
354799782,54cyclictest23580-21modprobe20:05:063
354699782,51cyclictest7750irq/56-dwc_otg23:45:072
354499787,44cyclictest15688-21ssh23:10:150
354499775,11cyclictest0-21swapper/020:10:160
243972770,2sleep324393-21ls22:40:293
3544997636,15cyclictest7750irq/56-dwc_otg20:40:150
3547997524,28cyclictest32712-21cut00:10:053
3547997524,28cyclictest32712-21cut00:10:053
354799746,48cyclictest7750irq/56-dwc_otg20:05:053
354799746,48cyclictest7750irq/56-dwc_otg20:05:053
354499748,53cyclictest0-21swapper/021:05:190
3547997311,37cyclictest7950irq/56-dwc_otg_00:00:343
354599733,48cyclictest7750irq/56-dwc_otg00:10:051
354599733,48cyclictest7750irq/56-dwc_otg00:10:051
354499733,37cyclictest0-21swapper/020:30:160
354699729,49cyclictest18935-21rt-features19:50:272
354699729,49cyclictest18935-21rt-features19:50:272
3544997216,23cyclictest0-21swapper/020:50:180
354699719,40cyclictest7950irq/56-dwc_otg_21:40:172
354499711,22cyclictest26223-21latency_hist21:25:040
354499711,22cyclictest26223-21latency_hist21:25:040
3547996921,18cyclictest0-21swapper/322:25:203
354699695,51cyclictest7950irq/56-dwc_otg_21:25:322
354499694,56cyclictest15899-21kworker/0:219:55:290
354699674,46cyclictest7850irq/56-dwc_otg_19:10:202
3545996723,28cyclictest0-21swapper/122:25:241
3545996714,31cyclictest23353-21apt-config20:05:041
3545996714,31cyclictest23353-21apt-config20:05:041
354699667,41cyclictest7950irq/56-dwc_otg_00:00:032
354699666,46cyclictest7950irq/56-dwc_otg_21:39:532
354699666,46cyclictest7950irq/56-dwc_otg_21:39:532
3547996517,23cyclictest11902-21fschecks_time23:05:233
354699659,45cyclictest7850irq/56-dwc_otg_21:35:032
354699659,45cyclictest7850irq/56-dwc_otg_21:35:032
354599659,18cyclictest7750irq/56-dwc_otg22:55:281
3544996531,16cyclictest0-21swapper/021:10:150
3544996524,21cyclictest31964-21grep20:25:340
354799646,40cyclictest7950irq/56-dwc_otg_23:15:213
3547996416,27cyclictest4752-21sed20:40:293
354699643,48cyclictest7950irq/56-dwc_otg_20:30:032
354699641,4cyclictest7850irq/56-dwc_otg_20:30:222
3544996426,18cyclictest30666-21ssh21:30:190
3544996426,18cyclictest30666-21ssh21:30:190
354499642,30cyclictest0-21swapper/023:50:160
354699638,36cyclictest7950irq/56-dwc_otg_23:40:282
354699636,44cyclictest7950irq/56-dwc_otg_21:16:412
354699633,46cyclictest14446-21diskmemload21:11:322
354699632,41cyclictest0-21swapper/219:45:252
354699631,7cyclictest5167-21latency00:15:252
354699631,51cyclictest8942-21ssh23:00:552
354699631,4cyclictest7850irq/56-dwc_otg_20:35:372
354699631,42cyclictest4941-21unixbench_multi20:40:312
354499637,17cyclictest0-21swapper/019:30:090
3544996311,36cyclictest29005-21latency_hist20:20:040
354599623,29cyclictest7750irq/56-dwc_otg20:05:061
354499622,31cyclictest1211350ssh23:05:260
354699612,8cyclictest12715-21latency_hist21:05:042
354699611,5cyclictest16788-21ssh23:10:322
3545996129,18cyclictest7750irq/56-dwc_otg00:00:381
354499618,30cyclictest13-21rcuc/019:15:180
354499615,28cyclictest13-21rcuc/022:00:170
354499612,33cyclictest9552-21fschecks_time20:55:220
3547996029,25cyclictest1253-21ssh00:10:283
354699602,43cyclictest7950irq/56-dwc_otg_22:20:142
354699601,4cyclictest7950irq/56-dwc_otg_21:24:442
354699601,4cyclictest7950irq/56-dwc_otg_21:24:442
354499602,39cyclictest0-21swapper/021:00:170
354499602,39cyclictest0-21swapper/020:45:300
3547995920,21cyclictest8660-21ssh23:40:263
354699585,4cyclictest7850irq/56-dwc_otg_22:50:152
354599583,10cyclictest0-21swapper/123:45:071
354499586,13cyclictest0-21swapper/023:25:170
3544995814,29cyclictest14579-21grep19:40:180
354499581,41cyclictest17960-21grep00:30:280
354499581,36cyclictest8824-21ssh00:20:150
354499581,34cyclictest24521-21unixbench_multi22:40:300
254712580,2sleep30-21swapper/323:20:553
354699576,39cyclictest7950irq/56-dwc_otg_19:26:152
354699571,43cyclictest0-21swapper/223:52:002
354699571,42cyclictest30733-21cat20:25:042
354499572,39cyclictest0-21swapper/000:25:230
3547995620,21cyclictest6441-21grep19:15:333
3547995619,23cyclictest24556-21ssh21:21:463
3547995619,23cyclictest24556-21ssh21:21:463
3547995619,21cyclictest16666-21perl19:45:213
3547995618,14cyclictest7706-21/usr/sbin/munin22:20:293
354699563,43cyclictest7950irq/56-dwc_otg_19:20:542
354699562,6cyclictest279792sleep220:15:302
354699561,6cyclictest16751-21ssh22:31:152
354699561,4cyclictest7750irq/56-dwc_otg00:20:212
354599563,5cyclictest7850irq/56-dwc_otg_19:10:151
3544995622,17cyclictest32504-21grep22:50:290
3544995620,16cyclictest7850irq/56-dwc_otg_19:45:210
354499561,33cyclictest18976-21ssh21:15:210
189512560,1sleep10-21swapper/119:50:281
189512560,1sleep10-21swapper/119:50:281
3547995519,14cyclictest16093-21idleruntime-cro19:45:033
3547995518,13cyclictest23543-21ps22:00:303
354699552,4cyclictest7850irq/56-dwc_otg_00:40:032
354699552,49cyclictest26116-21ntpq20:10:282
354699552,46cyclictest0-21swapper/222:03:072
354699552,46cyclictest0-21swapper/219:15:342
354699551,5cyclictest8154-21ntp_states20:50:282
354699551,4cyclictest0-21swapper/223:31:572
354699551,45cyclictest0-21swapper/223:05:302
354499552,6cyclictest3567-21munin-run23:35:040
3544995522,18cyclictest0-21swapper/021:40:080
3544995521,21cyclictest0-21swapper/019:35:310
354499551,32cyclictest26534-21/usr/sbin/munin21:25:230
354499551,30cyclictest29099-21wget20:20:180
9822540,2sleep10-21swapper/122:12:451
3547995418,16cyclictest18113-21/usr/sbin/munin19:50:243
3547995418,16cyclictest18113-21/usr/sbin/munin19:50:243
3547995417,13cyclictest4280-21ssh21:37:093
3547995417,13cyclictest4280-21ssh21:37:093
3547995416,25cyclictest2839-21ps20:35:323
354699542,46cyclictest12746-21grep19:35:182
354699541,5cyclictest0-21swapper/222:45:452
354699541,4cyclictest0-21swapper/219:42:422
354699541,48cyclictest10971-21grep21:00:042
354699541,3cyclictest0-21swapper/222:43:552
3545995416,16cyclictest7750irq/56-dwc_otg23:15:211
354499541,32cyclictest8580-21grep23:00:300
354699536,35cyclictest7950irq/56-dwc_otg_23:24:132
354699533,38cyclictest7950irq/56-dwc_otg_22:11:022
354699532,8cyclictest11425-21latency19:30:242
354699532,7cyclictest5647-21/usr/sbin/munin20:45:312
354699531,47cyclictest13127-21fschecks_count21:05:232
3544995318,18cyclictest19267-21grep19:50:310
3544995318,18cyclictest19267-21grep19:50:310
108022530,12sleep37850irq/56-dwc_otg_19:30:043
3547995216,23cyclictest28541-21ssh21:26:543
3547995215,14cyclictest0-21swapper/322:15:233
3547995213,24cyclictest17830-21ssh21:13:593
354699521,5cyclictest0-21swapper/219:55:162
354499522,27cyclictest0-21swapper/022:35:180
3547995114,12cyclictest16471-21ntp_states23:10:283
3547995113,22cyclictest7950irq/56-dwc_otg_20:30:193
3547995113,13cyclictest32652-21ssh21:32:043
3547995113,13cyclictest32652-21ssh21:32:043
354699513,39cyclictest7950irq/56-dwc_otg_23:38:002
354699511,5cyclictest28082-21taskset23:25:162
3545995126,6cyclictest31087-21/usr/sbin/munin21:30:251
3545995126,6cyclictest31087-21/usr/sbin/munin21:30:251
354499512,27cyclictest24676-21ntp_kernel_err23:20:240
354499511,24cyclictest0-21swapper/022:10:160
264402510,5sleep1354599cyclictest23:22:421
354799509,27cyclictest0-21swapper/319:55:183
3547995013,13cyclictest10986-21ssh23:04:403
3547995012,24cyclictest7950irq/56-dwc_otg_19:40:043
3546995017,31cyclictest13756-21/usr/sbin/munin00:25:272
354699501,5cyclictest21000-21ssh22:36:272
354699501,5cyclictest0-21swapper/200:31:412
354599508,26cyclictest0-21swapper/119:40:171
3544995026,5cyclictest15035-21/usr/sbin/munin22:30:250
354499502,27cyclictest0-21swapper/019:25:250
354499501,34cyclictest7353-21latency22:20:230
3547994914,15cyclictest11558-21ssh00:23:073
3547994913,23cyclictest20498-21ssh21:16:313
354599498,24cyclictest0-21swapper/119:30:091
354599491,5cyclictest0-21swapper/120:15:161
354499496,26cyclictest4196-21cut23:35:210
354499492,27cyclictest0-21swapper/019:14:160
3544994914,25cyclictest0-21swapper/021:55:140
3547994816,12cyclictest19171-21memory21:55:253
3547994815,22cyclictest7178-21seq19:20:103
3547994815,17cyclictest7950irq/56-dwc_otg_21:40:113
3547994814,19cyclictest2900-21rm22:54:593
3547994812,22cyclictest19899-21sh23:54:423
29602483,36sleep00-21swapper/019:05:560
354799475,11cyclictest7950irq/56-dwc_otg_19:10:163
3547994713,12cyclictest14297-21sh00:25:343
3547994712,20cyclictest1180-21runrttasks20:53:343
354599478,27cyclictest0-21swapper/120:45:181
354799464,25cyclictest7950irq/56-dwc_otg_00:37:003
3547994614,12cyclictest17911-21ssh22:33:233
3547994610,21cyclictest1180-21runrttasks21:03:493
354599463,8cyclictest7850irq/56-dwc_otg_22:35:151
354499467,20cyclictest0-21swapper/000:35:310
354499462,26cyclictest0-21swapper/019:20:240
3544994621,12cyclictest0-21swapper/022:45:150
3547994512,23cyclictest31374-21ssh23:29:113
3547994512,13cyclictest31630-21sh22:10:253
3547994512,11cyclictest0-21swapper/300:15:323
354499445,22cyclictest9-21ksoftirqd/023:40:080
354499441,18cyclictest4718-21grep00:15:190
3547994316,12cyclictest18854-21ssh00:31:203
3547994311,19cyclictest30667-21cut20:25:033
3547994311,18cyclictest1180-21runrttasks20:29:583
3547994311,17cyclictest14446-21diskmemload23:37:293
354599437,21cyclictest0-21swapper/122:45:171
354599431,12cyclictest0-21swapper/121:55:151
354799429,19cyclictest14134-21runrttasks21:06:173
3547994217,11cyclictest26954-21chrt20:14:343
3547994215,18cyclictest21282-21ssh22:36:583
3547994214,20cyclictest22680-21ssh23:57:133
3547994213,19cyclictest1411-21rm23:31:093
3547994213,17cyclictest10707-21chrt20:58:403
354599425,9cyclictest15104-21ssh21:10:231
354599422,25cyclictest22872-21ssh21:20:211
354599422,25cyclictest22872-21ssh21:20:211
3547994129,9cyclictest5660-21apt-get20:45:163
3547994124,14cyclictest10887-21apt-get19:30:103
3547994112,11cyclictest30692-21sh22:49:033
354599418,21cyclictest0-21swapper/121:40:111
354599412,17cyclictest7750irq/56-dwc_otg20:30:271
3545994111,18cyclictest23-21ksoftirqd/120:20:151
354599408,13cyclictest7850irq/56-dwc_otg_20:25:251
354599407,21cyclictest0-21swapper/120:35:201
354599406,23cyclictest2104-21ssh21:35:101
354599406,23cyclictest2104-21ssh21:35:101
354599398,4cyclictest0-21swapper/100:25:151
354599398,15cyclictest7950irq/56-dwc_otg_20:55:041
354599394,4cyclictest7850irq/56-dwc_otg_00:20:261
354599393,20cyclictest0-21swapper/123:30:141
3545993916,5cyclictest10841-21date19:30:041
3545993912,20cyclictest23-21ksoftirqd/120:11:151
354599387,16cyclictest7950irq/56-dwc_otg_21:15:341
354599382,20cyclictest11842-21ssh23:45:041
3547993725,8cyclictest27262-21apt-get20:15:163
354599378,19cyclictest0-21swapper/123:25:241
354599373,12cyclictest0-21swapper/122:40:301
354599372,4cyclictest7408-21ssh23:00:141
3545993715,8cyclictest0-21swapper/100:35:151
354599371,24cyclictest17197-21switchtime19:45:301
354599369,5cyclictest16357-21users22:30:341
354599368,6cyclictest7750irq/56-dwc_otg00:30:211
3545993620,5cyclictest15968-21fschecks_count23:10:221
354599356,6cyclictest7950irq/56-dwc_otg_21:25:201
354599351,6cyclictest7750irq/56-dwc_otg21:00:231
354599351,4cyclictest7750irq/56-dwc_otg20:55:151
354599346,6cyclictest7950irq/56-dwc_otg_22:55:041
3545993424,7cyclictest7432-21fschecks_count19:20:211
354599341,7cyclictest838450ssh00:20:031
3545993412,18cyclictest0-21swapper/119:55:161
354599341,10cyclictest14593-21sh21:10:041
354599337,5cyclictest7750irq/56-dwc_otg22:20:031
354599333,8cyclictest12824-21aten_rbpower_vo19:35:181
354599331,4cyclictest0-21swapper/123:05:151
354599331,10cyclictest5467-21/usr/sbin/munin19:15:181
3545993310,10cyclictest0-21swapper/123:55:411
354599328,7cyclictest7750irq/56-dwc_otg23:35:281
354599322,6cyclictest22712-21ssh22:00:181
354599321,14cyclictest6818-21wget22:20:171
354599317,15cyclictest0-21swapper/123:50:171
354599312,17cyclictest3880-21wget20:40:151
29322313,21sleep30-21swapper/319:05:343
29362305,14sleep20-21swapper/219:05:382
18042290,4sleep11807-21aten_rbpower_po19:05:151
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional