You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-03 - 11:53

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Wed Jun 03, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2649523760,3sleep10-21swapper/119:05:231
96421780,5sleep12821799cyclictest23:40:301
1773121710,1sleep20-21swapper/222:43:142
447121080,2sleep14472-21ntpq21:10:321
447121080,2sleep14472-21ntpq21:10:321
28217991053,12cyclictest7850irq/56-dwc_otg23:35:191
483821010,5sleep22821999cyclictest00:25:082
28211999531,34cyclictest0-21swapper/000:10:220
28211998615,29cyclictest0-21swapper/019:10:130
28211998317,54cyclictest0-21swapper/023:35:190
309932820,1sleep10-21swapper/122:59:351
2821199826,62cyclictest0-21swapper/020:20:210
210742820,5sleep28050irq/56-dwc_otg_22:46:362
2821199802,23cyclictest28155-21latency_hist23:35:080
28211997920,15cyclictest0-21swapper/021:37:190
2821799783,25cyclictest16006-21runrttasks23:20:201
28211997825,38cyclictest0-21swapper/000:25:260
28211997731,22cyclictest0-21swapper/000:00:210
28211997729,5cyclictest7950irq/56-dwc_otg_20:00:070
2821199771,30cyclictest7269-21awk23:10:070
2821199761,26cyclictest20401-21grep20:25:320
2821199761,26cyclictest20401-21grep20:25:320
28211997520,13cyclictest0-21swapper/019:40:090
28211997518,4cyclictest29745-21fschecks_time21:40:250
48812740,2sleep20-21swapper/223:05:482
28211997424,4cyclictest7950irq/56-dwc_otg_20:50:340
2821199741,34cyclictest0-21swapper/022:45:200
105082730,3sleep10-21swapper/122:35:071
2821199722,23cyclictest0-21swapper/022:13:560
28228997112,17cyclictest7950irq/56-dwc_otg_21:00:203
28211997145,7cyclictest8050irq/56-dwc_otg_21:10:210
28211997145,7cyclictest8050irq/56-dwc_otg_21:10:210
28211997126,4cyclictest7950irq/56-dwc_otg_21:26:160
28211997122,4cyclictest8050irq/56-dwc_otg_21:10:080
28211997117,4cyclictest24047-21runrttasks00:08:420
28211997117,4cyclictest24047-21runrttasks00:08:420
2821199711,22cyclictest5668-21runrttasks22:28:530
2821199711,22cyclictest5668-21runrttasks22:28:530
28211997015,13cyclictest0-21swapper/021:15:160
157412700,2sleep10-21swapper/121:24:171
28211996916,17cyclictest0-21swapper/021:56:390
28211996916,17cyclictest0-21swapper/021:56:390
28211996820,4cyclictest141-21systemd-journal23:29:370
28211996819,5cyclictest1857-21ssh21:45:290
2821199681,22cyclictest16272-21ssh21:25:080
2821199681,21cyclictest30719-21ssh22:59:050
2821199681,15cyclictest0-21swapper/020:45:330
2821199672,19cyclictest0-21swapper/020:00:280
2821199671,23cyclictest0-21swapper/023:46:160
2821199671,23cyclictest0-21swapper/023:46:160
2821199671,20cyclictest0-21swapper/023:01:500
2821199671,20cyclictest0-21swapper/023:01:500
2821199665,17cyclictest3211-21diskmemload22:42:540
2821199665,14cyclictest0-21swapper/023:53:100
2821199661,22cyclictest0-21swapper/021:01:330
2821199661,20cyclictest0-21swapper/019:20:150
140172660,19sleep28050irq/56-dwc_otg_23:55:372
2821799654,13cyclictest7850irq/56-dwc_otg20:05:221
28211996527,8cyclictest0-21swapper/021:50:220
28211996521,4cyclictest7950irq/56-dwc_otg_00:30:330
2821199651,27cyclictest0-21swapper/023:57:300
2822899643,14cyclictest7850irq/56-dwc_otg22:35:193
2821199641,32cyclictest15766-21latency_hist23:20:090
2821199641,32cyclictest15766-21latency_hist23:20:090
28211996413,13cyclictest0-21swapper/022:50:490
2822899639,12cyclictest2550-21ssh22:25:193
2822899639,12cyclictest2550-21ssh22:25:193
28211996332,8cyclictest7950irq/56-dwc_otg_00:15:240
28211996319,4cyclictest7950irq/56-dwc_otg_19:31:330
28211996211,17cyclictest0-21swapper/000:36:350
146702620,2sleep10-21swapper/123:18:321
146702620,2sleep10-21swapper/123:18:321
28211996116,28cyclictest0-21swapper/019:45:210
2821199611,32cyclictest7218-21ssh22:30:280
2821199611,23cyclictest0-21swapper/021:31:450
239032610,1sleep20-21swapper/221:33:362
2821199602,21cyclictest0-21swapper/022:15:370
2822899592,18cyclictest0-21swapper/321:35:193
2821199591,35cyclictest26114-21grep20:40:360
2821199591,30cyclictest11511-21latency_hist23:15:080
2821799575,11cyclictest7850irq/56-dwc_otg23:30:191
2821799574,15cyclictest7950irq/56-dwc_otg_22:20:201
2821199571,34cyclictest0-21swapper/020:15:210
2821199571,30cyclictest1094-21ssh00:20:240
2821199571,23cyclictest4149-21ssh23:45:090
2821199562,40cyclictest0-21swapper/020:30:370
2821199561,29cyclictest24420-21grep20:35:360
177572560,3sleep10-21swapper/120:20:091
301182540,2sleep20-21swapper/200:15:342
2821199541,31cyclictest15246-21unixbench_multi22:00:370
2821199541,28cyclictest20005-21wc23:25:090
2821199532,30cyclictest31232-21chrt20:55:300
28211995221,21cyclictest0-21swapper/022:24:330
37632510,1sleep30-21swapper/323:44:383
2821199502,22cyclictest0-21swapper/022:05:300
110582500,2sleep00-21swapper/022:35:220
2822899495,8cyclictest7950irq/56-dwc_otg_00:35:223
28211994927,5cyclictest7648-21awk19:50:280
2822899486,31cyclictest8050irq/56-dwc_otg_00:05:263
2822899486,31cyclictest8050irq/56-dwc_otg_00:05:263
28228994813,13cyclictest13518-21grep20:05:393
28217994814,6cyclictest8562-21idleruntime-cro19:55:071
2822899477,26cyclictest0-21swapper/319:25:123
2822899477,25cyclictest0-21swapper/323:05:133
2822899477,25cyclictest0-21swapper/320:55:193
2822899476,28cyclictest29398-21apt-get21:40:183
2821799472,38cyclictest8050irq/56-dwc_otg_20:45:301
2822899467,25cyclictest0-21swapper/320:10:213
2822899467,24cyclictest0-21swapper/319:30:183
2822899464,9cyclictest0-21swapper/321:10:213
2822899464,9cyclictest0-21swapper/321:10:213
2821799467,27cyclictest0-21swapper/119:20:171
279082462,36sleep00-21swapper/019:10:010
2822899458,19cyclictest21716-21ssh22:09:183
2822899453,25cyclictest8050irq/56-dwc_otg_20:50:293
28228994510,12cyclictest17647-21ssh22:04:383
2821799456,29cyclictest29420-21sh21:40:211
2821199452,22cyclictest0-21swapper/020:10:240
2821199451,26cyclictest0-21swapper/019:25:130
248452450,5sleep22821999cyclictest23:30:282
2822899447,22cyclictest0-21swapper/300:30:233
28228994411,12cyclictest5140-21/usr/sbin/munin00:25:393
28228994410,6cyclictest39-21ksoftirqd/322:50:373
28219994423,11cyclictest28544-21cut23:35:212
28217994420,5cyclictest19253-21aten_rbpower_cu22:45:221
28211994427,7cyclictest7950irq/56-dwc_otg_19:20:080
28211994427,7cyclictest7950irq/56-dwc_otg_19:20:080
28211994412,17cyclictest4943-21modprobe19:45:070
28211994412,17cyclictest4943-21modprobe19:45:070
2822899436,25cyclictest10332-21rm23:13:113
2822899436,20cyclictest7850irq/56-dwc_otg22:23:433
28228994321,9cyclictest8050irq/56-dwc_otg_00:11:223
28228994317,12cyclictest7850irq/56-dwc_otg22:15:313
28228994311,7cyclictest18780-21apt-get22:45:123
28228994310,18cyclictest8272-21/usr/sbin/munin21:15:253
146892430,2sleep30-21swapper/323:18:343
146892430,2sleep30-21swapper/323:18:343
2822899429,18cyclictest24879-21ssh22:12:273
2822899429,14cyclictest19648-21ssh00:03:143
2822899427,21cyclictest12707-21kworker/u8:319:10:203
2822899425,3cyclictest0-21swapper/323:50:213
28228994216,14cyclictest8050irq/56-dwc_otg_23:21:103
28228994216,14cyclictest8050irq/56-dwc_otg_21:56:333
28228994216,14cyclictest8050irq/56-dwc_otg_21:56:333
2821799427,25cyclictest0-21swapper/100:25:221
2821799425,26cyclictest7950irq/56-dwc_otg_21:30:341
2821799424,7cyclictest7950irq/56-dwc_otg_00:30:221
2822899418,12cyclictest22278-21ssh23:27:053
2822899417,5cyclictest7850irq/56-dwc_otg23:00:083
2822899412,9cyclictest8050irq/56-dwc_otg_21:50:203
28228994115,17cyclictest381ktimersoftd/323:35:203
28228994112,10cyclictest13423-21ls21:20:363
2821799415,19cyclictest8050irq/56-dwc_otg_19:35:081
28211994116,16cyclictest0-21swapper/020:05:120
124102410,1sleep20-21swapper/221:57:562
124102410,1sleep20-21swapper/221:57:562
75752401,4sleep10-21swapper/123:10:191
2821999409,13cyclictest0-21swapper/219:40:142
2821999409,13cyclictest0-21swapper/219:40:142
2821999407,23cyclictest0-21swapper/223:45:202
2821999407,23cyclictest0-21swapper/223:45:202
175492401,3sleep117651-21ssh21:25:351
2822899399,12cyclictest1449-21df21:45:233
2822899398,12cyclictest18399-21chrt21:26:293
2822899394,13cyclictest3934-21ls19:40:243
2822899394,13cyclictest3934-21ls19:40:243
28228993911,17cyclictest2668-21ssh23:04:303
28228993911,17cyclictest2668-21ssh23:04:303
2822899391,10cyclictest8050irq/56-dwc_otg_20:00:253
2821999395,22cyclictest19139-21grep22:05:332
2821799397,22cyclictest0-21swapper/123:55:221
28217993931,5cyclictest8472-21latency_hist23:50:091
28217993923,14cyclictest5823-21fschecks_time21:50:271
46822380,5sleep12821799cyclictest22:27:061
46822380,5sleep12821799cyclictest22:27:061
2822899387,18cyclictest8050irq/56-dwc_otg_19:40:083
2822899384,12cyclictest30994-21grep19:20:203
28228993814,7cyclictest7850irq/56-dwc_otg22:31:533
28228993814,6cyclictest381ktimersoftd/300:20:223
28228993812,12cyclictest6004-21ssh23:46:093
28228993812,12cyclictest6004-21ssh23:46:093
28228993810,12cyclictest31552-21ssh00:17:173
28219993813,17cyclictest0-21swapper/223:00:262
28219993813,17cyclictest0-21swapper/223:00:262
28217993811,5cyclictest1133-21runrttasks00:15:271
28217993810,12cyclictest0-21swapper/123:25:191
2822899377,20cyclictest0-21swapper/322:40:213
2822899376,16cyclictest24567-21ssh21:34:463
2822899371,26cyclictest21807-21ls20:30:273
2821999379,19cyclictest15762-21kworker/2:123:50:202
2821999372,30cyclictest8050irq/56-dwc_otg_20:00:192
28219993712,6cyclictest23839-21unixbench_singl22:10:372
2821799378,17cyclictest1707-21wget21:05:221
28217993717,4cyclictest1133-21runrttasks00:09:591
28217993717,4cyclictest1133-21runrttasks00:09:591
28217993714,4cyclictest14448-21fschecks_count20:10:241
2822899362,12cyclictest15048-21ssh23:57:233
28219993616,7cyclictest0-21swapper/221:50:312
28219993615,4cyclictest12404-21sh21:20:232
28219993614,4cyclictest29141-21sensors19:10:332
2821999361,18cyclictest7956-21ssh21:15:142
2821799369,7cyclictest7850irq/56-dwc_otg22:10:241
2821799365,22cyclictest30242-21latency_hist20:55:081
28217993610,8cyclictest0-21swapper/121:45:211
2822899358,4cyclictest7950irq/56-dwc_otg_20:25:293
2822899358,4cyclictest7950irq/56-dwc_otg_20:25:293
2822899356,13cyclictest8050irq/56-dwc_otg_20:47:453
2822899351,4cyclictest8050irq/56-dwc_otg_20:15:193
28228993511,5cyclictest0-21swapper/321:10:083
28228993511,13cyclictest8050irq/56-dwc_otg_23:32:183
2821999359,4cyclictest0-21swapper/223:10:372
2821999357,4cyclictest8050irq/56-dwc_otg_23:40:252
2821999356,22cyclictest0-21swapper/220:25:212
2821999356,22cyclictest0-21swapper/220:25:212
2821999354,22cyclictest31-21ksoftirqd/221:00:202
2821999351,5cyclictest5944-21ssh00:25:322
2821999351,24cyclictest24170-21timerandwakeup20:35:342
2821999351,24cyclictest15199-21timerwakeupswit20:10:342
28219993511,17cyclictest8050irq/56-dwc_otg_21:40:192
2821799358,6cyclictest7850irq/56-dwc_otg22:05:081
2821799352,6cyclictest7850irq/56-dwc_otg00:00:261
28217993518,5cyclictest5467-21sh23:06:521
2821799351,3cyclictest0-21swapper/122:15:261
2821799351,19cyclictest0-21swapper/123:46:581
2822899346,15cyclictest0-21swapper/319:50:203
2821999348,13cyclictest7950irq/56-dwc_otg_22:15:322
2821999347,22cyclictest0-21swapper/221:05:222
2821999347,21cyclictest0-21swapper/219:45:212
2821999345,13cyclictest17787-21latency_hist20:20:092
2821999342,21cyclictest8050irq/56-dwc_otg_22:20:352
2821999341,7cyclictest17245-21idleruntime-cro00:40:082
2821999341,22cyclictest17299-21ls00:00:262
28219993410,5cyclictest7950irq/56-dwc_otg_20:40:192
2821799342,18cyclictest23476-21basename22:50:231
2821799342,11cyclictest9735-21ssh21:55:181
2821799342,11cyclictest9735-21ssh21:55:181
2821799341,26cyclictest0-21swapper/123:52:331
2822899332,10cyclictest0-21swapper/320:42:323
2822899331,14cyclictest7950irq/56-dwc_otg_20:35:293
2821999337,14cyclictest0-21swapper/219:25:132
2821999335,6cyclictest7950irq/56-dwc_otg_23:20:262
2821999331,6cyclictest0-21swapper/222:00:202
28219993310,3cyclictest0-21swapper/200:08:522
28219993310,3cyclictest0-21swapper/200:08:522
2821799338,4cyclictest0-21swapper/121:15:261
2821799338,11cyclictest7950irq/56-dwc_otg_00:20:311
2821799336,13cyclictest8050irq/56-dwc_otg_20:20:351
2821799335,12cyclictest8050irq/56-dwc_otg_00:35:231
2821799333,5cyclictest10800-21/usr/sbin/munin22:35:331
2821799332,4cyclictest0-21swapper/119:40:161
2821799332,4cyclictest0-21swapper/119:40:161
2821799331,7cyclictest7850irq/56-dwc_otg22:05:331
2821799331,25cyclictest12172-21latency_hist20:05:081
28217993310,10cyclictest0-21swapper/123:03:261
28217993310,10cyclictest0-21swapper/123:03:261
2822899329,4cyclictest10341-21chrt20:00:023
2821999327,7cyclictest8050irq/56-dwc_otg_23:25:252
2821999327,4cyclictest0-21swapper/221:25:592
2821999327,19cyclictest0-21swapper/222:50:212
2821999323,5cyclictest7850irq/56-dwc_otg00:10:372
2821999323,5cyclictest1310250ssh00:35:082
2821999323,15cyclictest7950irq/56-dwc_otg_22:25:212
2821999323,15cyclictest7950irq/56-dwc_otg_22:25:212
2821999322,24cyclictest7950irq/56-dwc_otg_22:34:232
28219993215,5cyclictest11303-21if_wlan022:35:262
28219993211,6cyclictest1793350wget20:20:222
2821799326,5cyclictest7850irq/56-dwc_otg19:45:241
2821799325,5cyclictest7850irq/56-dwc_otg21:00:331
2821799323,6cyclictest7850irq/56-dwc_otg19:10:301
28217993216,5cyclictest27496-21sh00:12:281
2822899318,12cyclictest7950irq/56-dwc_otg_19:17:083
2822899318,12cyclictest7950irq/56-dwc_otg_19:17:083
2822899316,15cyclictest0-21swapper/319:45:213
2822899310,13cyclictest19196-21runrttasks20:22:373
2821999317,5cyclictest7950irq/56-dwc_otg_21:35:212
2821999316,7cyclictest8050irq/56-dwc_otg_22:55:302
2821999314,14cyclictest8050irq/56-dwc_otg_20:05:282
2821999312,8cyclictest12091-21fschecks_time23:15:242
2821999312,8cyclictest12091-21fschecks_time23:15:242
2821999312,4cyclictest7850irq/56-dwc_otg21:10:342
2821999312,4cyclictest7850irq/56-dwc_otg21:10:342
28219993113,5cyclictest3061-21basename19:40:082
2821799317,5cyclictest7850irq/56-dwc_otg22:40:291
28217993117,4cyclictest20783-21unixbench_singl20:25:351
28217993117,4cyclictest20783-21unixbench_singl20:25:351
2821799311,20cyclictest0-21swapper/120:37:231
294842300,2sleep20-21swapper/220:50:332
28219993016,4cyclictest26847-21/usr/sbin/munin20:45:332
2821999301,20cyclictest0-21swapper/219:20:192
28217993014,5cyclictest1133-21runrttasks20:58:511
2821799301,12cyclictest7850irq/56-dwc_otg21:39:041
2821999296,12cyclictest7850irq/56-dwc_otg19:20:082
2821999296,12cyclictest7850irq/56-dwc_otg19:20:082
2821999292,12cyclictest7950irq/56-dwc_otg_21:45:102
2821799294,4cyclictest7850irq/56-dwc_otg19:55:191
2821999282,3cyclictest0-21swapper/219:30:192
2821999281,4cyclictest0-21swapper/220:30:202
2821799282,4cyclictest25263-21df20:40:231
28217992819,6cyclictest628-21latency_hist19:30:081
277182283,15sleep30-21swapper/319:07:333
27412281,5sleep10-21swapper/119:36:261
2821999272,15cyclictest0-21swapper/220:55:242
2821999271,17cyclictest0-21swapper/219:59:142
2821799275,4cyclictest7950irq/56-dwc_otg_20:30:331
2821799272,14cyclictest0-21swapper/119:17:191
2821799272,14cyclictest0-21swapper/119:17:191
268422275,8sleep20-21swapper/219:05:292
2821999261,4cyclictest7315-21cut19:50:242
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional