You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-06-11 - 19:18

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rack7slot3s.osadl.org (updated Thu Jun 11, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2287621730,4sleep00-21swapper/007:05:140
1975121580,5sleep02462899cyclictest10:06:300
130952970,2sleep20-21swapper/210:36:352
156202890,1sleep00-21swapper/011:18:000
135762800,5sleep22463099cyclictest10:00:122
35032780,6sleep09-21ksoftirqd/011:41:150
35032780,6sleep09-21ksoftirqd/011:41:150
305302770,2sleep10-21swapper/107:25:181
89392750,3sleep08941-21kernelversion07:55:200
24628997416,33cyclictest15446-21idleruntime-cro08:15:010
24628997416,33cyclictest15446-21idleruntime-cro08:15:010
24629997321,10cyclictest8050irq/56-dwc_otg_12:00:141
24628997132,6cyclictest0-21swapper/007:45:120
2462899713,23cyclictest0-21swapper/011:55:100
2462899713,23cyclictest0-21swapper/011:55:100
24628997114,21cyclictest28026-21sh07:20:010
2462899701,28cyclictest0-21swapper/010:25:120
166102700,1sleep10-21swapper/112:36:031
24630996831,9cyclictest7950irq/56-dwc_otg_11:15:132
2462899685,27cyclictest0-21swapper/008:20:120
24630996627,22cyclictest12704-21ssh11:52:432
24630996612,19cyclictest8050irq/56-dwc_otg_07:30:132
24630996529,19cyclictest28862-21memory10:55:212
24628996514,30cyclictest29614-21sh11:35:010
24630996427,21cyclictest8058-21sh11:09:422
24630996426,25cyclictest26248-21rm10:52:112
24630996423,13cyclictest6203-21sort09:50:282
24630996327,20cyclictest1230-21sh12:18:062
24630996324,17cyclictest1741-21rm10:24:082
2462899633,35cyclictest0-21swapper/012:35:000
24628996325,18cyclictest26955-21runrttasks08:45:170
24628996314,25cyclictest727-21runrttasks11:25:170
88112620,5sleep12462999cyclictest09:54:441
2463199626,8cyclictest0-21swapper/307:25:113
24630996225,16cyclictest8913-21if_wlan007:55:192
24630996221,15cyclictest27201-21grep07:15:252
24628996214,25cyclictest26349-21sh10:15:110
24628996214,25cyclictest26349-21sh10:15:110
24630996129,10cyclictest6430-21munin-run07:50:012
24630996128,12cyclictest26570-21ssh09:36:262
24630996127,19cyclictest6828-21ssh09:14:162
24630996125,12cyclictest0-21swapper/211:15:012
24630996125,12cyclictest0-21swapper/211:15:012
24630996121,21cyclictest7950irq/56-dwc_otg_10:10:112
24628996141,6cyclictest7950irq/56-dwc_otg_09:45:280
2462899613,30cyclictest0-21swapper/010:00:010
2462899613,30cyclictest0-21swapper/010:00:010
2462899611,38cyclictest3144-21tr07:40:140
24630996027,13cyclictest8045-21ssh11:46:582
24630996025,12cyclictest14647-21tune2fs11:55:162
24630996025,12cyclictest14647-21tune2fs11:55:162
24630996024,20cyclictest9397-21ssh10:32:352
24630996023,21cyclictest2357250rm11:26:592
24628996033,7cyclictest7950irq/56-dwc_otg_10:50:190
2462899603,25cyclictest0-21swapper/009:20:110
24628996019,21cyclictest7332-21if_eth010:30:180
24630995927,13cyclictest29256-21ssh10:18:102
24630995927,13cyclictest29256-21ssh10:18:102
24630995926,11cyclictest28942-21ssh12:12:042
24628995929,3cyclictest0-21swapper/012:10:220
2462899592,33cyclictest1356-21head11:00:300
2462899592,27cyclictest464550ssh09:10:280
2462899591,38cyclictest6345-21unixbench_singl09:50:290
24630995827,11cyclictest5072-21ssh11:44:062
24630995827,11cyclictest5072-21ssh11:44:062
24630995825,23cyclictest17121-21ssh10:41:112
24630995825,23cyclictest17121-21ssh10:41:112
24630995825,12cyclictest32431-21/usr/sbin/munin09:00:192
24630995824,22cyclictest16978-21chrt08:16:472
2462899582,35cyclictest9-21ksoftirqd/008:35:290
24630995738,13cyclictest28956-21apt-get09:40:102
24630995729,17cyclictest21804-21ssh09:30:342
24630995728,12cyclictest4164-21ssh12:20:532
24630995726,10cyclictest6986-21perl07:50:192
24630995724,21cyclictest679-21seq11:38:452
2462899572,39cyclictest24829-21/usr/sbin/munin08:40:210
2462899571,32cyclictest31272-21ssh12:15:180
2462899571,30cyclictest17460-21unixbench_singl09:25:290
24630995628,14cyclictest25830-21uptime07:10:292
24630995626,13cyclictest9606-21ssh09:16:392
24630995626,11cyclictest24642-21rm12:06:472
24630995625,23cyclictest17405-21ssh12:37:282
24630995623,22cyclictest3407-21ssh09:47:442
24630995623,21cyclictest21266-21ssh10:09:092
24630995619,12cyclictest8648-21sh12:26:302
24630995619,12cyclictest8648-21sh12:26:302
2462899565,31cyclictest18917-21latency_hist12:40:010
24628995629,7cyclictest7950irq/56-dwc_otg_12:25:280
24628995629,7cyclictest7950irq/56-dwc_otg_12:25:280
24628995629,14cyclictest13-21rcuc/011:45:130
24628995629,12cyclictest7850irq/56-dwc_otg11:50:280
24628995626,6cyclictest20909-21unixbench_multi10:45:280
24628995619,20cyclictest21234-21sh09:30:230
2462899561,44cyclictest0-21swapper/010:40:040
2462899561,44cyclictest0-21swapper/010:40:040
24630995525,19cyclictest24229-21chrt08:35:392
24630995524,11cyclictest14409-21ssh09:22:332
24630995523,11cyclictest4532-21ssh10:26:322
24630995521,14cyclictest21015-21rm10:45:312
24630995520,15cyclictest19986-21ssh12:00:532
2462999556,13cyclictest7850irq/56-dwc_otg11:05:131
24628995534,6cyclictest2390-21grep07:35:270
24628995530,11cyclictest7950irq/56-dwc_otg_12:20:230
2462899552,27cyclictest0-21swapper/009:05:000
24630995428,11cyclictest14418-21ssh12:34:362
24630995427,17cyclictest3112-21ssh11:03:302
24630995422,12cyclictest2874-21chrt09:08:472
24630995419,10cyclictest8050irq/56-dwc_otg_08:55:142
24630995419,10cyclictest8050irq/56-dwc_otg_08:55:142
24630995418,11cyclictest8050irq/56-dwc_otg_09:25:282
24630995324,8cyclictest19407-21apt-get08:25:112
24630995318,22cyclictest0-21swapper/208:41:072
24630995318,10cyclictest0-21swapper/208:06:202
24628995325,14cyclictest7850irq/56-dwc_otg11:10:210
24628995325,14cyclictest7850irq/56-dwc_otg11:10:210
24628995313,24cyclictest25273-21latency09:35:210
2462899531,31cyclictest28942-21/usr/sbin/munin09:40:240
2462899531,30cyclictest19439-21aten_rbpower_cu08:25:130
24630995226,15cyclictest21658-21iwlist08:30:192
24630995221,13cyclictest28254-21/usr/sbin/munin07:20:202
24630995221,13cyclictest28254-21/usr/sbin/munin07:20:202
24630995220,12cyclictest18658-21ssh11:20:522
2462999525,6cyclictest8050irq/56-dwc_otg_09:00:131
2462999522,23cyclictest7950irq/56-dwc_otg_11:35:131
2462899522,29cyclictest0-21swapper/009:15:410
24628995214,35cyclictest101ktimersoftd/010:35:290
24630995123,12cyclictest28150-21taskset08:49:352
24630995120,13cyclictest10956-21ssh09:56:022
24630995120,13cyclictest10956-21ssh09:56:022
2462899516,20cyclictest17658-21latency_hist10:05:010
24628995130,5cyclictest28529-21aten_rbpower_po08:50:150
2463199505,8cyclictest8050irq/56-dwc_otg_08:40:143
24630995025,12cyclictest28062-21rm11:32:202
24630995023,12cyclictest2865-21sed07:40:012
2462899506,7cyclictest7950irq/56-dwc_otg_11:05:130
2462899502,37cyclictest0-21swapper/012:10:010
2462899501,26cyclictest29081-21ps10:55:250
2463199493,11cyclictest0-21swapper/309:05:123
24630994920,14cyclictest0-21swapper/207:42:042
2462999492,11cyclictest5911-21ssh11:45:111
2462899493,26cyclictest9-21ksoftirqd/008:00:290
24628994924,15cyclictest0-21swapper/007:25:210
24628994916,4cyclictest8050irq/56-dwc_otg_10:10:130
2462899491,26cyclictest30833-21grep10:20:140
24630994818,12cyclictest31331-21chrt07:26:202
2462899482,28cyclictest0-21swapper/011:36:450
24628994819,14cyclictest28324-21wget07:20:130
24628994819,14cyclictest28324-21wget07:20:130
119282480,2sleep10-21swapper/111:51:181
2462999478,25cyclictest0-21swapper/108:20:061
2462999477,24cyclictest0-21swapper/108:10:131
2462999477,24cyclictest0-21swapper/108:10:131
24629994738,7cyclictest21-21rcuc/110:55:121
2462899471,6cyclictest7950irq/56-dwc_otg_07:30:110
24628994715,22cyclictest0-21swapper/008:57:540
24628994715,22cyclictest0-21swapper/008:57:540
2462899471,23cyclictest0-21swapper/008:20:010
2463199466,7cyclictest0-21swapper/310:50:113
24630994618,9cyclictest0-21swapper/208:52:002
2462899461,29cyclictest12541-21/usr/sbin/munin08:05:200
24630994530,12cyclictest18546-21sed08:20:282
2462999457,26cyclictest0-21swapper/108:35:141
2462999454,9cyclictest0-21swapper/107:30:121
2462999454,4cyclictest0-21swapper/110:05:131
2462999451,5cyclictest0-21swapper/108:30:131
24628994525,5cyclictest9-21ksoftirqd/007:10:020
2462899452,23cyclictest0-21swapper/009:07:100
2462899451,4cyclictest19931-21ssh12:00:470
2463199447,23cyclictest0-21swapper/308:20:053
24630994429,12cyclictest14917-21timerwakeupswit08:10:272
24630994429,12cyclictest14917-21timerwakeupswit08:10:272
24630994420,12cyclictest11717-21chrt08:03:432
24312440,5sleep32463199cyclictest10:25:013
24629994318,5cyclictest11516-21fschecks_time10:35:191
2462899435,22cyclictest0-21swapper/007:50:210
2462899432,23cyclictest0-21swapper/008:32:140
2462899431,24cyclictest13-21rcuc/011:20:260
2463199427,25cyclictest0-21swapper/307:35:133
24631994220,4cyclictest30391-21latency_hist10:20:013
24631994220,4cyclictest30391-21latency_hist10:20:013
2462999423,18cyclictest0-21swapper/109:10:131
24629994218,14cyclictest2994-21diskmemload11:55:121
24629994218,14cyclictest2994-21diskmemload11:55:121
2462999421,6cyclictest0-21swapper/111:25:131
2463199416,20cyclictest7950irq/56-dwc_otg_09:40:163
186432410,2sleep30-21swapper/312:39:433
294912401,4sleep329616-21users08:50:303
2463199401,25cyclictest0-21swapper/308:15:133
24631994010,9cyclictest12018-21aten_rbpower_vo09:20:143
2462999404,6cyclictest8050irq/56-dwc_otg_10:25:131
2463199397,12cyclictest7950irq/56-dwc_otg_11:55:133
2463199397,12cyclictest7950irq/56-dwc_otg_11:55:133
24631993921,6cyclictest5552-21ntp_states07:45:243
2463199391,8cyclictest7950irq/56-dwc_otg_08:05:113
24631993914,9cyclictest0-21swapper/311:00:023
2462999395,7cyclictest0-21swapper/112:05:151
2463199385,16cyclictest7950irq/56-dwc_otg_11:35:273
24629993818,8cyclictest221ktimersoftd/110:20:041
2462999381,5cyclictest30908-21ssh12:15:121
24629993815,4cyclictest13607-21/usr/sbin/munin11:15:231
24629993811,14cyclictest0-21swapper/109:15:191
62042370,1sleep3381ktimersoftd/311:06:283
2463199378,6cyclictest7950irq/56-dwc_otg_10:10:003
2463199377,13cyclictest7950irq/56-dwc_otg_10:45:023
2463199371,16cyclictest0-21swapper/309:12:513
2462999375,19cyclictest21-21rcuc/109:30:511
2462999374,14cyclictest7850irq/56-dwc_otg08:30:011
24629993716,6cyclictest0-21swapper/110:30:121
24629993712,13cyclictest23-21ksoftirqd/110:15:191
24629993712,13cyclictest23-21ksoftirqd/110:15:191
119642370,5sleep32463199cyclictest09:57:483
119642370,5sleep32463199cyclictest09:57:483
2463199361,25cyclictest25529-21switchtime07:10:263
2462999368,5cyclictest7850irq/56-dwc_otg09:20:281
2462999368,13cyclictest8050irq/56-dwc_otg_09:45:001
24629993611,11cyclictest0-21swapper/109:47:581
24629993610,4cyclictest0-21swapper/110:40:261
24629993610,4cyclictest0-21swapper/110:40:261
2463199354,17cyclictest2331-21latency_hist12:20:013
2463199352,23cyclictest0-21swapper/309:00:143
2463199351,3cyclictest7950irq/56-dwc_otg_11:50:193
2463199351,22cyclictest6182-21uniq11:45:163
2463199351,11cyclictest5684-21memory09:50:213
2462999357,3cyclictest0-21swapper/110:45:261
2462999355,7cyclictest7850irq/56-dwc_otg08:50:191
2462999355,6cyclictest7850irq/56-dwc_otg12:20:291
2462999355,14cyclictest8050irq/56-dwc_otg_10:05:001
2462999354,6cyclictest25945-21grep09:35:301
24629993524,7cyclictest8703-21fschecks_count07:55:171
24629993522,5cyclictest23-21ksoftirqd/107:10:231
24629993519,4cyclictest23-21ksoftirqd/112:15:011
24629993517,6cyclictest10741-21uname08:00:211
2462999351,24cyclictest12562-21latency08:05:201
143632350,2sleep30-21swapper/311:15:503
2463199348,6cyclictest7950irq/56-dwc_otg_09:15:163
2463199348,6cyclictest7850irq/56-dwc_otg08:30:283
2463199348,5cyclictest7850irq/56-dwc_otg08:02:503
2463199346,6cyclictest7950irq/56-dwc_otg_07:20:263
2463199346,6cyclictest7950irq/56-dwc_otg_07:20:263
2463199345,6cyclictest0-21swapper/311:30:203
2463199344,15cyclictest8050irq/56-dwc_otg_07:30:183
2463199343,9cyclictest7141-21fschecks_time12:25:183
2463199343,9cyclictest7141-21fschecks_time12:25:183
2463199342,24cyclictest7275-21ls10:30:183
2463199341,21cyclictest3264-21missed_timers12:20:223
2463199341,16cyclictest7850irq/56-dwc_otg09:30:163
2462999349,11cyclictest7850irq/56-dwc_otg07:45:271
2462999348,14cyclictest8050irq/56-dwc_otg_07:50:291
2462999347,14cyclictest8050irq/56-dwc_otg_07:15:241
2462999343,13cyclictest8050irq/56-dwc_otg_09:05:271
2462999342,6cyclictest0-21swapper/110:10:271
2462999342,18cyclictest0-21swapper/112:30:251
24629993421,4cyclictest8050irq/56-dwc_otg_11:12:421
24629993421,4cyclictest8050irq/56-dwc_otg_11:12:421
2462999341,17cyclictest221ktimersoftd/109:25:211
24629993411,11cyclictest0-21swapper/111:05:011
2463199336,12cyclictest7950irq/56-dwc_otg_08:35:213
2463199335,14cyclictest8050irq/56-dwc_otg_09:35:153
2463199335,11cyclictest7950irq/56-dwc_otg_12:00:303
2463199334,16cyclictest15438-21ssh10:40:163
24631993315,12cyclictest3983-21kworker/u8:411:10:133
24631993315,12cyclictest3983-21kworker/u8:411:10:133
24631993311,5cyclictest0-21swapper/311:20:293
2462999335,11cyclictest23-21ksoftirqd/110:50:111
2463199325,6cyclictest7850irq/56-dwc_otg07:50:163
2463199325,15cyclictest0-21swapper/308:45:133
2463199325,12cyclictest8050irq/56-dwc_otg_09:45:183
2463199323,6cyclictest28231-21ssh10:55:123
2463199322,6cyclictest7950irq/56-dwc_otg_09:25:153
2463199322,20cyclictest0-21swapper/312:33:053
2463199321,20cyclictest8458-21/usr/sbin/munin07:55:223
24631993211,5cyclictest15061-1kworker/u9:210:05:013
2462999327,4cyclictest0-21swapper/111:23:201
2462999327,23cyclictest0-21swapper/108:55:131
2462999327,23cyclictest0-21swapper/108:55:131
2462999327,19cyclictest0-21swapper/111:31:281
2462999322,15cyclictest0-21swapper/107:35:511
2462999321,13cyclictest7850irq/56-dwc_otg09:55:261
2462999321,13cyclictest7850irq/56-dwc_otg09:55:261
2463199318,14cyclictest0-21swapper/311:40:133
2463199318,14cyclictest0-21swapper/311:40:133
2463199317,15cyclictest0-21swapper/310:35:143
2463199317,14cyclictest0-21swapper/310:25:183
2463199314,4cyclictest7850irq/56-dwc_otg12:10:203
2463199314,14cyclictest8050irq/56-dwc_otg_08:10:213
2463199314,14cyclictest8050irq/56-dwc_otg_08:10:213
2463199311,20cyclictest23234-21ssh12:05:213
24631993110,4cyclictest0-21swapper/310:10:163
2462999317,6cyclictest7950irq/56-dwc_otg_11:40:171
2462999317,6cyclictest7950irq/56-dwc_otg_11:40:171
2462999315,15cyclictest8050irq/56-dwc_otg_08:16:111
2462999312,7cyclictest0-21swapper/108:40:141
2462999312,15cyclictest8050irq/56-dwc_otg_07:40:211
24629993110,10cyclictest0-21swapper/112:27:241
24629993110,10cyclictest0-21swapper/112:27:241
2463199303,11cyclictest7950irq/56-dwc_otg_11:25:463
2463199302,14cyclictest8050irq/56-dwc_otg_08:55:233
2463199302,14cyclictest8050irq/56-dwc_otg_08:55:233
2463199301,14cyclictest8050irq/56-dwc_otg_10:50:013
2462999306,9cyclictest7850irq/56-dwc_otg07:20:141
2462999306,9cyclictest7850irq/56-dwc_otg07:20:141
243092303,21sleep30-21swapper/307:09:313
2463199296,13cyclictest8050irq/56-dwc_otg_07:15:283
2462999297,12cyclictest8050irq/56-dwc_otg_08:45:221
2411122910,13sleep20-21swapper/207:06:552
2463199281,12cyclictest8050irq/56-dwc_otg_07:40:253
242242283,18sleep10-21swapper/107:08:241
2463199261,5cyclictest37-21rcuc/308:25:283
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional