You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-03-16 - 05:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Mon Mar 16, 2026 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2389821400,5sleep14896-21cyclictest23:55:051
302021180,5sleep0490399cyclictest23:29:560
302021180,5sleep0490399cyclictest23:29:560
4903991151,61cyclictest0-21swapper/023:05:100
49039910211,52cyclictest0-21swapper/019:10:060
256802930,1sleep20-21swapper/223:17:102
197292840,5sleep3490899cyclictest21:14:403
21852830,2sleep20-21swapper/222:10:192
303072810,2sleep10-21swapper/100:01:571
490799795,36cyclictest7950irq/56-dwc_otg_20:45:102
281962730,3sleep028197-21diskmemload21:24:310
490899726,17cyclictest0-21swapper/323:05:103
490399713,31cyclictest0-21swapper/023:50:150
490399701,21cyclictest0-21swapper/021:50:170
99682690,1sleep30-21swapper/322:58:513
22772680,2sleep30-21swapper/300:07:273
490699663,4cyclictest0-21swapper/122:05:151
490799654,13cyclictest7850irq/56-dwc_otg_23:30:152
490799653,13cyclictest0-21swapper/220:00:162
4903996314,29cyclictest24127-21unixbench_singl20:00:330
289202630,6sleep3490899cyclictest23:20:333
490399622,40cyclictest11512-21head19:25:320
4903996215,26cyclictest27399-21unixbench_singl22:00:330
4907996118,24cyclictest29004-21missed_timers20:15:262
4903996112,29cyclictest4526-21if_wlan000:10:220
490399602,33cyclictest29007-21aten_rbpower_vo21:25:190
490399602,33cyclictest29007-21aten_rbpower_vo21:25:190
44332603,49sleep20-21swapper/219:07:272
490399593,45cyclictest25066-21kworker/0:222:15:220
490399591,42cyclictest23599-21grep23:15:180
490399591,41cyclictest3885-21unixbench-2d22:50:330
4906995818,16cyclictest0-21swapper/120:00:161
490399582,7cyclictest0-21swapper/022:45:130
4903995820,6cyclictest0-21swapper/023:10:150
490799574,10cyclictest7850irq/56-dwc_otg_23:35:182
4907995717,21cyclictest23964-21ssh22:35:332
490399572,37cyclictest0-21swapper/023:30:320
490399571,33cyclictest0-21swapper/021:30:150
4903995713,27cyclictest9-21ksoftirqd/023:20:210
4907995618,10cyclictest16474-21ssh00:25:162
490699565,9cyclictest6103-21ssh22:15:161
490699562,8cyclictest0-21swapper/123:00:151
490399564,8cyclictest0-21swapper/023:35:190
4903995618,25cyclictest30380-21aten_rbpower_vo22:05:180
490799553,8cyclictest0-21swapper/220:25:172
4903995536,11cyclictest505-21rs:main0
490399552,9cyclictest9-21ksoftirqd/000:25:150
4903995525,11cyclictest7950irq/56-dwc_otg_20:00:040
490399552,24cyclictest0-21swapper/020:45:170
490399551,35cyclictest25111-21fschecks_time20:05:210
4907995424,24cyclictest4909-21taskset19:10:062
4907995417,18cyclictest23545-21ssh21:55:592
4907995417,18cyclictest23545-21ssh21:55:592
490699543,5cyclictest7750irq/56-dwc_otg19:30:171
490399542,28cyclictest8860-21sh21:40:040
490399542,28cyclictest4956-21latency_hist20:40:050
490399542,19cyclictest12109-21ls23:40:220
490399541,33cyclictest0-21swapper/022:10:210
169002540,2sleep30-21swapper/322:27:433
4907995314,11cyclictest12282-21egrep19:30:192
490399536,30cyclictest0-21swapper/019:50:290
490399536,24cyclictest25831-21ls22:00:050
490399536,24cyclictest25831-21ls22:00:050
490399532,29cyclictest13-21rcuc/000:30:310
490399532,28cyclictest9-21ksoftirqd/020:55:130
490399532,28cyclictest9-21ksoftirqd/020:55:130
490399532,16cyclictest0-21swapper/023:45:250
4907995215,20cyclictest28027-21ssh22:01:372
490399526,23cyclictest9-21ksoftirqd/000:05:220
490399522,27cyclictest0-21swapper/000:20:260
490399521,8cyclictest10826-21latency_hist23:00:050
490399521,4cyclictest20945-21grep21:15:240
490399521,35cyclictest14776-21runrttasks23:05:040
4903995213,22cyclictest8663-21aten_rbpower_vo19:20:170
490799519,6cyclictest7850irq/56-dwc_otg_21:45:172
490799514,7cyclictest7950irq/56-dwc_otg_21:05:172
4907995122,17cyclictest24985-21idleruntime21:20:212
4907995117,19cyclictest22410-21rm00:31:262
4907995115,21cyclictest11290-21/usr/sbin/munin22:20:322
4907995115,18cyclictest8349-21ssh21:39:112
4907995113,11cyclictest24272-21munin-run21:20:042
490699516,11cyclictest1958-21ssh22:10:161
490699512,12cyclictest7750irq/56-dwc_otg20:40:141
490399519,7cyclictest712-21unixbench_singl20:25:330
4903995136,5cyclictest7850irq/56-dwc_otg_00:35:330
490399513,26cyclictest26816-21aten_rbpower_en22:40:180
4903995121,7cyclictest13896-21idleruntime-cro21:05:040
490399511,34cyclictest0-21swapper/022:35:180
490399511,21cyclictest17114-21grep21:10:310
4907995012,21cyclictest7850irq/56-dwc_otg_20:55:292
4907995012,21cyclictest7850irq/56-dwc_otg_20:55:292
4907995011,22cyclictest21019-21ssh22:32:432
490399509,20cyclictest0-21swapper/020:15:160
490399502,30cyclictest0-21swapper/022:20:260
490399502,30cyclictest0-21swapper/019:20:050
4903995022,8cyclictest12448-21fschecks_time19:30:210
4907994912,21cyclictest18156-21ssh21:50:232
4907994912,13cyclictest12066-21fschecks_time23:40:212
4907994912,13cyclictest10280-21sed19:25:052
4907994911,26cyclictest7750irq/56-dwc_otg20:20:302
4907994911,26cyclictest7750irq/56-dwc_otg20:20:302
490399499,27cyclictest31790-21ls20:25:050
490399499,27cyclictest31790-21ls20:25:050
490399491,30cyclictest0-21swapper/020:30:250
490399491,28cyclictest8842-21aten_rbpower_vo20:50:190
490399491,26cyclictest19377-21ssh22:30:300
490899483,31cyclictest0-21swapper/319:10:063
4907994816,13cyclictest30365-21ssh21:25:512
4907994816,13cyclictest30365-21ssh21:25:512
4907994810,13cyclictest6911-21aten_rbpower_en22:55:182
4903994815,16cyclictest24067-21/usr/sbin/munin23:55:190
4903994815,16cyclictest24067-21/usr/sbin/munin23:55:190
490399481,31cyclictest13-21rcuc/019:40:230
490399481,26cyclictest28365-21grep00:00:200
490899477,25cyclictest0-21swapper/321:00:153
490899477,25cyclictest0-21swapper/319:45:173
4907994716,11cyclictest2423-21ssh21:31:162
490399471,7cyclictest18334-21latency_hist22:30:050
490399471,35cyclictest0-21swapper/000:15:260
43482473,35sleep00-21swapper/019:06:200
178352470,5sleep2490799cyclictest22:29:272
54552460,19sleep17950irq/56-dwc_otg_21:35:231
490899467,24cyclictest0-21swapper/319:50:183
490899467,23cyclictest0-21swapper/322:00:183
4907994619,16cyclictest14827-21ssh00:22:412
4907994617,19cyclictest7170-21timerandwakeup22:15:312
4907994611,13cyclictest19051-21ssh23:49:022
490699467,25cyclictest0-21swapper/120:15:181
490699467,24cyclictest0-21swapper/122:45:101
262062460,4sleep1221ktimersoftd/122:39:411
490799457,24cyclictest0-21swapper/220:40:162
490799452,6cyclictest7950irq/56-dwc_otg_19:25:172
4907994515,11cyclictest2198350ssh23:51:512
4907994514,13cyclictest27592-21/usr/sbin/munin23:20:312
4907994513,13cyclictest30176-21ssh22:44:232
4907994513,11cyclictest20676-21chrt19:51:512
4907994512,13cyclictest1019-21ssh23:26:162
4907994512,13cyclictest1019-21ssh23:26:162
4907994511,16cyclictest26712-21ssh00:36:342
4907994510,21cyclictest7850irq/56-dwc_otg_23:06:332
490699452,6cyclictest3208050ssh23:25:211
490699452,6cyclictest3208050ssh23:25:211
4906994519,3cyclictest0-21swapper/119:25:051
490399453,8cyclictest7950irq/56-dwc_otg_19:35:220
490399453,8cyclictest7950irq/56-dwc_otg_19:35:220
490399451,12cyclictest0-21swapper/021:45:210
490899446,8cyclictest0-21swapper/319:30:163
4908994412,6cyclictest0-21swapper/321:55:233
4908994412,6cyclictest0-21swapper/321:55:233
490799443,11cyclictest15709-21diskmemload22:50:182
4907994415,19cyclictest21701-21sh23:12:272
4907994414,12cyclictest13253-21cut21:45:052
4907994413,16cyclictest18302-21ssh21:12:102
4907994411,18cyclictest149-21systemd-journal21:01:162
4907994410,12cyclictest9031-21grep20:50:222
490699447,24cyclictest0-21swapper/120:45:091
490699447,24cyclictest0-21swapper/100:20:181
490699443,5cyclictest0-21swapper/123:30:151
4906994420,5cyclictest20406-21aten_rbpower_vo21:15:171
4903994414,22cyclictest0-21swapper/020:44:570
490899433,22cyclictest7850irq/56-dwc_otg_19:25:323
490799439,12cyclictest0-21swapper/219:35:202
490799439,12cyclictest0-21swapper/219:35:202
490799435,25cyclictest10983-21ssh23:00:172
4907994313,12cyclictest1253-21ssh00:05:362
4907994312,11cyclictest27831-21ssh00:00:002
4907994312,11cyclictest27831-21ssh00:00:002
4907994311,16cyclictest11735-21ssh00:19:352
4907994310,20cyclictest15709-21diskmemload22:49:592
490699437,23cyclictest0-21swapper/122:50:181
490899427,25cyclictest0-21swapper/300:25:173
490899427,22cyclictest0-21swapper/319:35:183
490899427,22cyclictest0-21swapper/319:35:183
490899422,25cyclictest0-21swapper/319:55:173
490799429,19cyclictest7950irq/56-dwc_otg_22:10:042
490799428,11cyclictest28502-21grep00:00:232
490799427,22cyclictest0-21swapper/219:15:172
490699427,25cyclictest0-21swapper/119:25:181
490399422,24cyclictest0-21swapper/019:45:090
490699416,22cyclictest0-21swapper/100:35:101
490699415,8cyclictest0-21swapper/121:05:161
490699414,5cyclictest21-21rcuc/122:30:051
490799405,11cyclictest0-21swapper/200:13:592
490699402,23cyclictest28661-21ssh21:25:051
4906994019,8cyclictest32438-21aten_rbpower_po00:05:181
4906994019,4cyclictest21-21rcuc/121:25:221
4906994019,4cyclictest21-21rcuc/121:25:221
490399409,19cyclictest15763-21cat21:10:040
4903994013,13cyclictest0-21swapper/020:12:170
490899399,19cyclictest27315-21grep22:40:243
490899397,23cyclictest0-21swapper/323:55:183
490899397,23cyclictest0-21swapper/323:55:183
490899394,10cyclictest7950irq/56-dwc_otg_00:25:053
490799399,11cyclictest2853-21seq20:33:512
490699394,12cyclictest0-21swapper/121:30:161
4906993918,4cyclictest21-21rcuc/122:44:111
490699391,22cyclictest0-21swapper/100:15:161
490899387,24cyclictest0-21swapper/300:10:173
490699382,19cyclictest24609-21ssh23:55:241
490699382,19cyclictest24609-21ssh23:55:241
4906993820,5cyclictest18573-21aten_rbpower_en22:30:181
4906993815,5cyclictest19694-21fschecks_time19:50:221
4906993815,5cyclictest15709-21diskmemload00:10:271
490899371,19cyclictest11360-21rm23:40:043
4907993712,13cyclictest4681-21chrt20:39:092
490699379,11cyclictest23-21ksoftirqd/122:21:101
490699375,15cyclictest7950irq/56-dwc_otg_00:30:051
490699374,19cyclictest0-21swapper/120:50:181
490699374,18cyclictest7950irq/56-dwc_otg_19:10:281
490699373,21cyclictest0-21swapper/122:55:171
4906993722,6cyclictest21-21rcuc/123:20:271
4906993712,12cyclictest221ktimersoftd/120:20:231
4906993712,12cyclictest221ktimersoftd/120:20:231
130962370,1sleep30-21swapper/322:23:203
490899363,21cyclictest0-21swapper/300:00:083
4908993616,5cyclictest9103-21sh22:18:363
490899361,26cyclictest5766-21latency_hist22:15:043
490899361,21cyclictest12271-21ls21:00:053
490899361,21cyclictest12271-21ls21:00:053
4906993610,16cyclictest15484-21sh23:45:041
44742363,25sleep10-21swapper/119:07:591
228282360,5sleep1490699cyclictest00:32:131
490899359,4cyclictest7950irq/56-dwc_otg_20:30:043
4908993516,12cyclictest17398-21latency_hist19:45:043
490899351,26cyclictest997-21cut21:30:213
4908993512,5cyclictest18092-21sh21:50:223
4908993510,14cyclictest0-21swapper/323:40:193
490799358,11cyclictest26371-21awk20:10:042
490699357,6cyclictest7750irq/56-dwc_otg21:10:181
490699352,5cyclictest7950irq/56-dwc_otg_19:35:171
490699352,5cyclictest7950irq/56-dwc_otg_19:35:171
490699352,18cyclictest7950irq/56-dwc_otg_21:00:161
4906993519,9cyclictest17027-21sh21:49:281
4906993513,7cyclictest3290-21aten_rbpower_po20:35:181
490699351,20cyclictest17520-21ls19:45:051
4906993510,8cyclictest23-21ksoftirqd/120:55:261
4906993510,8cyclictest23-21ksoftirqd/120:55:261
4906993510,5cyclictest0-21swapper/123:35:261
490899346,18cyclictest0-21swapper/323:30:173
4908993416,5cyclictest9997-21sh21:40:283
490899341,4cyclictest7950irq/56-dwc_otg_00:35:173
490899341,25cyclictest3031-21uniq22:50:223
490799344,9cyclictest0-21swapper/220:10:302
490699349,6cyclictest7750irq/56-dwc_otg19:50:041
490699343,4cyclictest7750irq/56-dwc_otg23:45:181
4906993424,8cyclictest7950irq/56-dwc_otg_21:59:221
4906993424,8cyclictest7950irq/56-dwc_otg_21:59:221
4906993412,6cyclictest7750irq/56-dwc_otg21:50:201
490899339,4cyclictest0-21swapper/321:24:503
490899337,24cyclictest0-21swapper/320:35:183
490899335,19cyclictest20622-21ssh00:30:173
490899331,4cyclictest7950irq/56-dwc_otg_20:45:163
4908993314,10cyclictest7446-21sh19:15:273
490899331,3cyclictest7950irq/56-dwc_otg_20:05:173
490799337,14cyclictest17039-21runrttasks19:41:312
490799337,12cyclictest0-21swapper/219:59:142
490799336,17cyclictest0-21swapper/219:45:562
490699336,7cyclictest7950irq/56-dwc_otg_23:10:221
490699335,5cyclictest7750irq/56-dwc_otg23:05:191
490699332,8cyclictest26210-21ssh22:00:191
490699332,8cyclictest21-21rcuc/120:10:041
490699331,12cyclictest21-21rcuc/123:15:191
490899328,10cyclictest7950irq/56-dwc_otg_21:05:313
490899327,23cyclictest0-21swapper/320:20:183
490899327,23cyclictest0-21swapper/320:20:183
490899322,8cyclictest12019-21munin-run00:20:043
490899321,4cyclictest7950irq/56-dwc_otg_21:15:143
4908993212,4cyclictest1460-21cut20:30:193
490899321,21cyclictest0-21swapper/320:00:193
490899321,12cyclictest7950irq/56-dwc_otg_21:48:573
4908993210,11cyclictest0-21swapper/321:39:403
490699327,15cyclictest0-21swapper/121:40:171
490699326,9cyclictest21415-21fschecks_count19:55:211
490699324,10cyclictest221ktimersoftd/120:15:041
490699323,14cyclictest9450irq/48-DMA09:12:251
310492320,2sleep30-21swapper/321:27:043
310492320,2sleep30-21swapper/321:27:043
490899317,14cyclictest0-21swapper/323:10:183
490899316,15cyclictest0-21swapper/320:50:183
490899311,3cyclictest7950irq/56-dwc_otg_19:20:163
4908993113,10cyclictest22884-21ssh22:35:193
490899311,11cyclictest0-21swapper/322:09:183
4908993111,12cyclictest18525-21ssh22:30:163
490899307,12cyclictest0-21swapper/322:45:263
490899301,18cyclictest7950irq/56-dwc_otg_20:15:243
490699306,9cyclictest0-21swapper/119:15:271
490699301,4cyclictest7750irq/56-dwc_otg20:30:341
490899298,11cyclictest0-21swapper/323:47:323
4908992925,2cyclictest0-21swapper/323:50:153
490899291,14cyclictest7750irq/56-dwc_otg23:04:193
4908992910,9cyclictest0-21swapper/323:25:203
4908992910,9cyclictest0-21swapper/323:25:203
265852291,2sleep30-21swapper/320:10:173
490899289,4cyclictest0-21swapper/323:15:503
490899271,4cyclictest7850irq/56-dwc_otg_20:40:223
44762273,15sleep30-21swapper/319:08:013
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional