You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-04-26 - 18:27

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #7, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.

Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack7slot3s.osadl.org (updated Sun Apr 26, 2026 12:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
274972950,3sleep127503-21timerandwakeup16:46:181
276342900,2sleep10-21swapper/121:01:041
198592830,4sleep20-21swapper/216:26:122
44852730,2sleep02113199cyclictest20:32:120
2113199737,32cyclictest0-21swapper/018:21:010
2113199737,32cyclictest0-21swapper/018:21:010
312162690,2sleep00-21swapper/020:26:040
296312690,2sleep0101ktimersoftd/021:41:290
126002680,1sleep30-21swapper/320:03:003
126002680,1sleep30-21swapper/320:03:003
21131996728,27cyclictest7850irq/56-dwc_otg_18:35:490
2113199656,36cyclictest0-21swapper/019:56:030
21131996423,21cyclictest19547-21idleruntime-cro21:30:500
21131996423,21cyclictest19547-21idleruntime-cro21:30:500
9812630,2sleep30-21swapper/321:07:183
9812630,2sleep30-21swapper/321:07:183
2113199631,17cyclictest9-21ksoftirqd/021:11:030
313542620,6sleep22113399cyclictest19:07:042
21131996228,10cyclictest0-21swapper/018:51:070
210432620,2sleep30-21swapper/320:52:163
134862620,6sleep02113199cyclictest20:04:380
134862620,6sleep02113199cyclictest20:04:380
2113399615,10cyclictest0-21swapper/219:26:012
2113199618,36cyclictest7850irq/56-dwc_otg_18:26:030
21131996123,14cyclictest0-21swapper/018:56:010
2113199611,8cyclictest953-21cron19:10:490
21131996116,24cyclictest18408-21fschecks_time19:31:070
21131996116,24cyclictest18408-21fschecks_time19:31:070
75162600,1sleep20-21swapper/219:17:302
21131996011,28cyclictest21274-21aten_rbpower_vo16:31:040
21131996011,28cyclictest21274-21aten_rbpower_vo16:31:040
2113399599,24cyclictest0-21swapper/218:56:012
21131995944,2cyclictest0-21swapper/019:51:010
2113199581,35cyclictest4369-21grep17:11:180
2113199572,10cyclictest32706-21grep21:46:070
2113199572,10cyclictest32706-21grep21:46:070
21131995717,37cyclictest23136-21latency_hist20:55:500
21131995638,11cyclictest7850irq/56-dwc_otg_21:53:210
21131995632,4cyclictest2934-21awk21:10:500
21131995632,4cyclictest2934-21awk21:10:500
2113199561,6cyclictest0-21swapper/017:51:010
2113199561,37cyclictest25726-21ls16:41:160
2113399552,13cyclictest4688-21ls21:51:112
21131995533,5cyclictest7950irq/56-dwc_otg_19:26:040
21131995530,5cyclictest7850irq/56-dwc_otg_18:01:180
21131995524,5cyclictest7850irq/56-dwc_otg_20:46:110
2113199552,36cyclictest5577-21unixbench_singl18:36:200
21131995510,27cyclictest13-21rcuc/020:25:490
21131995510,22cyclictest13-21rcuc/021:56:150
21131995434,11cyclictest0-21swapper/021:05:490
21131995426,7cyclictest7850irq/56-dwc_otg_18:46:120
21131995421,15cyclictest20532-21latency21:31:100
21131995417,18cyclictest13-21rcuc/017:20:500
2113199541,36cyclictest12477-21proc_pri21:21:160
21131995334,4cyclictest7950irq/56-dwc_otg_19:40:490
21131995325,3cyclictest0-21swapper/019:15:550
2113199531,30cyclictest23936-21unixbench-2d20:16:180
2113199531,30cyclictest23254-21aten_rbpower_cu18:06:040
2113299524,10cyclictest0-21swapper/118:21:011
2113299524,10cyclictest0-21swapper/118:21:011
21131995231,5cyclictest7850irq/56-dwc_otg_18:41:170
21131995229,6cyclictest7850irq/56-dwc_otg_17:06:060
2113199522,26cyclictest5195-21apt-config19:15:500
2113199521,26cyclictest10946-21sh20:41:010
2113499515,8cyclictest7950irq/56-dwc_otg_20:45:493
21133995115,3cyclictest7750irq/56-dwc_otg17:05:562
21131995131,5cyclictest7850irq/56-dwc_otg_17:41:140
2113199512,27cyclictest28498-21grep16:51:030
21131995114,25cyclictest11102-21/usr/sbin/munin17:31:110
2113199511,28cyclictest27498-21ntpq18:16:120
259962500,4sleep325974-21ssh19:01:093
2113199509,37cyclictest13-21rcuc/019:21:000
2113199504,30cyclictest7365-21ls17:21:100
21131995018,17cyclictest31133-21timerandwakeup16:56:150
21131995017,15cyclictest28077-21latency_hist21:40:500
2113399491,36cyclictest27430-21ls18:16:112
21131994928,4cyclictest7950irq/56-dwc_otg_20:06:130
21131994918,3cyclictest0-21swapper/020:11:040
2113199491,32cyclictest32673-21ntpq17:01:130
2113399487,26cyclictest0-21swapper/221:30:552
21131994826,4cyclictest7850irq/56-dwc_otg_19:02:590
21131994824,12cyclictest0-21swapper/020:39:200
21131994824,12cyclictest0-21swapper/020:39:200
2113199481,29cyclictest0-21swapper/016:36:090
2113399478,11cyclictest0-21swapper/219:11:012
2113299471,5cyclictest0-21swapper/118:51:021
2113499466,7cyclictest7850irq/56-dwc_otg_16:31:003
2113499466,7cyclictest7850irq/56-dwc_otg_16:31:003
2113499461,5cyclictest7750irq/56-dwc_otg19:06:023
21131994621,13cyclictest0-21swapper/017:56:160
21131994617,13cyclictest0-21swapper/017:36:180
2113199461,28cyclictest8161-21cat21:16:110
2113199452,33cyclictest0-21swapper/019:47:360
2113199452,23cyclictest0-21swapper/020:56:590
205582451,35sleep00-21swapper/016:27:040
2113499441,3cyclictest7950irq/56-dwc_otg_20:31:003
21132994428,4cyclictest5148-21aten_rbpower_po17:16:041
2113199443,22cyclictest0-21swapper/019:41:170
21134994312,17cyclictest7950irq/56-dwc_otg_18:51:023
21132994319,5cyclictest908-21runrttasks21:38:261
21132994319,12cyclictest7950irq/56-dwc_otg_19:58:451
2113199431,26cyclictest0-21swapper/017:27:320
21131994310,20cyclictest22384-21kworker/0:116:48:270
2113499422,8cyclictest7750irq/56-dwc_otg18:31:013
2113499422,5cyclictest917-21snmpd21:31:013
2113399426,5cyclictest7850irq/56-dwc_otg_20:11:042
2113399424,6cyclictest0-21swapper/217:46:022
21132994220,6cyclictest11084-21aten_rbpower_vo20:41:051
2113299421,8cyclictest7950irq/56-dwc_otg_16:45:491
2113199421,21cyclictest13-21rcuc/017:46:180
2113499412,6cyclictest7850irq/56-dwc_otg_21:26:033
2113499412,6cyclictest7850irq/56-dwc_otg_21:26:033
2113499411,23cyclictest0-21swapper/316:51:023
2113399417,25cyclictest0-21swapper/216:46:052
2113299416,20cyclictest7950irq/56-dwc_otg_17:40:491
2113299412,20cyclictest15353-21grep17:41:181
21132994115,5cyclictest31906-21fschecks_time21:06:071
21132994115,5cyclictest31906-21fschecks_time21:06:071
2113299411,18cyclictest0-21swapper/121:41:181
21134994018,5cyclictest908-21runrttasks18:46:013
21133994017,5cyclictest23065-21aten_rbpower_vo16:36:052
21133994014,7cyclictest0-21swapper/220:32:122
2113299406,7cyclictest21-21rcuc/117:46:041
2113299406,6cyclictest7950irq/56-dwc_otg_19:26:111
2113299403,4cyclictest0-21swapper/118:31:011
21132994018,5cyclictest26942-21aten_rbpower_vo18:16:041
21132994016,11cyclictest21268-21latency_hist18:00:501
21132994014,13cyclictest91550lldpd16:32:241
21132994014,13cyclictest91550lldpd16:32:241
2113499397,8cyclictest39-21ksoftirqd/319:26:013
2113499394,19cyclictest7950irq/56-dwc_otg_21:55:503
21134993928,4cyclictest7850irq/56-dwc_otg_20:06:153
2113499391,21cyclictest0-21swapper/321:41:183
2113299396,17cyclictest7950irq/56-dwc_otg_17:30:491
2113299392,4cyclictest7950irq/56-dwc_otg_18:41:011
21132993922,5cyclictest2400-21fschecks_count19:51:071
21132993920,7cyclictest8207-21idleruntime-cro18:40:501
21132993917,8cyclictest7950irq/56-dwc_otg_21:26:031
21132993917,8cyclictest7950irq/56-dwc_otg_21:26:031
21131993913,14cyclictest25326-21fschecks_time18:11:080
2113499384,28cyclictest7950irq/56-dwc_otg_20:11:023
2113499383,29cyclictest7950irq/56-dwc_otg_21:21:043
2113499381,22cyclictest20209-21ls17:56:103
2113399385,21cyclictest28244-21ssh21:41:042
21133993818,12cyclictest2630-21seq21:48:532
21133993818,12cyclictest2630-21seq21:48:532
2113299387,5cyclictest7850irq/56-dwc_otg_19:35:501
2113299387,5cyclictest7850irq/56-dwc_otg_19:35:501
2113299384,4cyclictest7750irq/56-dwc_otg18:56:161
2113299382,5cyclictest23-21ksoftirqd/118:26:071
2113299382,4cyclictest7750irq/56-dwc_otg19:11:011
21132993823,5cyclictest21-21rcuc/121:11:111
2113299382,14cyclictest7940-21timerwakeupswit20:36:171
2113299382,14cyclictest7940-21timerwakeupswit20:36:171
2113299381,9cyclictest988150ssh19:21:031
21132993817,5cyclictest21-21rcuc/121:51:031
21132993815,11cyclictest26714-21unixbench-2d19:01:181
21132993812,8cyclictest7531-21aten_rbpower_cu21:16:031
21132993812,5cyclictest23-21ksoftirqd/120:26:151
2113299381,19cyclictest3524-21grep20:31:141
21132993811,22cyclictest7950irq/56-dwc_otg_19:41:011
21134993717,4cyclictest37-21rcuc/318:59:063
2113399376,6cyclictest7750irq/56-dwc_otg18:35:502
21133993713,7cyclictest0-21swapper/220:26:182
2113399371,20cyclictest0-21swapper/219:25:502
21133993711,13cyclictest18892-21unixbench_multi17:51:182
2113299373,6cyclictest23-21ksoftirqd/121:31:011
2113299372,6cyclictest0-21swapper/121:46:191
2113299372,6cyclictest0-21swapper/121:46:191
2113299372,5cyclictest1518950ssh18:48:151
21132993719,4cyclictest20867-21sh20:51:561
2113299371,6cyclictest0-21swapper/120:47:541
21132993716,18cyclictest7950irq/56-dwc_otg_19:06:021
21132993715,14cyclictest21-21rcuc/121:56:421
21132993714,11cyclictest7175-21fschecks_time17:21:061
21132993713,5cyclictest21-21rcuc/116:36:011
38132361,30sleep20-21swapper/217:11:112
2113499365,13cyclictest37-21rcuc/319:41:013
21134993617,5cyclictest24242-21/usr/sbin/munin20:56:153
21134993615,8cyclictest9351-21http19:20:503
2113399363,20cyclictest7750irq/56-dwc_otg16:45:412
2113399363,18cyclictest7750irq/56-dwc_otg18:22:302
2113399363,18cyclictest7750irq/56-dwc_otg18:22:302
21133993619,5cyclictest219150wget19:51:042
21133993618,12cyclictest31591-21ssh21:06:032
21133993618,12cyclictest31591-21ssh21:06:032
2113399361,4cyclictest7850irq/56-dwc_otg_16:51:032
2113399361,20cyclictest15526-21ssh21:25:502
2113299367,5cyclictest0-21swapper/121:25:501
2113299363,13cyclictest7950irq/56-dwc_otg_17:06:131
2113299362,4cyclictest0-21swapper/119:16:021
2113299362,10cyclictest0-21swapper/116:55:491
21132993617,6cyclictest19946-21sh20:11:201
2113299361,5cyclictest23-21ksoftirqd/120:01:021
2113299361,5cyclictest23-21ksoftirqd/120:01:021
21132993612,11cyclictest0-21swapper/119:49:551
2113499356,6cyclictest7950irq/56-dwc_otg_18:40:513
2113499355,10cyclictest7950irq/56-dwc_otg_21:11:143
21134993517,5cyclictest28252-21seq18:19:193
2113499351,2cyclictest0-21swapper/316:46:053
21134993511,6cyclictest5433-1kworker/u9:020:46:073
2113399357,15cyclictest0-21swapper/216:56:022
2113399356,9cyclictest0-21swapper/218:43:572
21133993522,9cyclictest12142-21latency_hist22:00:502
21133993518,10cyclictest9656-21ssh21:17:512
2113399351,6cyclictest0-21swapper/218:39:132
2113299356,14cyclictest7950irq/56-dwc_otg_20:06:191
21132993516,4cyclictest23509-21fschecks_time18:06:071
21132993514,4cyclictest12035-21sh17:35:491
21132993514,4cyclictest0-21swapper/118:11:131
53632340,12sleep37950irq/56-dwc_otg_19:15:503
2113499347,6cyclictest39-21ksoftirqd/321:50:503
2113499347,6cyclictest39-21ksoftirqd/321:50:503
2113499347,22cyclictest0-21swapper/319:21:023
2113499346,5cyclictest7850irq/56-dwc_otg_17:16:073
2113499342,8cyclictest7959-21grep17:21:183
2113499342,4cyclictest0-21swapper/319:35:503
2113499342,4cyclictest0-21swapper/319:35:503
21134993423,8cyclictest0-21swapper/317:11:083
21134993421,4cyclictest30539-21idleruntime16:56:073
2113499341,7cyclictest16121-21aten_rbpower_vo17:46:043
2113499341,22cyclictest2397-21uniq19:51:073
21134993410,16cyclictest0-21swapper/321:40:423
2113399348,12cyclictest7850irq/56-dwc_otg_18:51:052
2113399347,24cyclictest0-21swapper/218:26:032
2113299343,28cyclictest7950irq/56-dwc_otg_20:56:021
2113299341,5cyclictest0-21swapper/117:13:131
21132993410,13cyclictest0-21swapper/120:24:551
306722330,3sleep330673-21/usr/sbin/munin19:46:063
275792330,3sleep221130-21cyclictest19:02:412
2113499337,24cyclictest0-21swapper/317:36:043
2113499333,25cyclictest7950irq/56-dwc_otg_20:26:013
21134993316,6cyclictest37-21rcuc/318:15:103
21134993316,6cyclictest27556-21aten_rbpower_po21:01:033
21134993315,10cyclictest8103-21head20:36:203
21134993315,10cyclictest8103-21head20:36:203
2113499331,22cyclictest9136-21switchtime21:56:163
2113399339,16cyclictest7850irq/56-dwc_otg_19:46:062
2113399337,14cyclictest0-21swapper/217:26:042
2113399335,19cyclictest10365-21ssh20:01:012
2113399335,19cyclictest10365-21ssh20:01:012
2113399333,23cyclictest7750irq/56-dwc_otg18:50:322
2113399332,4cyclictest7850irq/56-dwc_otg_17:16:012
2113399331,7cyclictest21189-21wget16:31:032
2113399331,7cyclictest21189-21wget16:31:032
2113399331,4cyclictest0-21swapper/218:06:032
2113399331,22cyclictest16697-21switchtime21:26:162
2113399331,22cyclictest16697-21switchtime21:26:162
2113399331,22cyclictest0-21swapper/217:31:012
2113399331,10cyclictest7850irq/56-dwc_otg_20:51:052
21132993315,10cyclictest23406-21missed_timers20:16:121
2113499325,4cyclictest7750irq/56-dwc_otg18:26:103
2113499323,14cyclictest7950irq/56-dwc_otg_17:45:503
2113499321,5cyclictest24798-21ssh20:17:403
2113499321,4cyclictest7850irq/56-dwc_otg_21:20:503
2113499321,23cyclictest7950irq/56-dwc_otg_16:36:043
2113499321,20cyclictest8858-21uniq18:41:073
2113399325,5cyclictest7750irq/56-dwc_otg19:31:002
2113399325,5cyclictest7750irq/56-dwc_otg19:31:002
2113399324,19cyclictest7750irq/56-dwc_otg20:16:102
21133993217,8cyclictest24705-21ssh20:56:252
2113399321,22cyclictest0-21swapper/217:44:422
2113399321,22cyclictest0-21swapper/217:22:492
2113399321,20cyclictest21686-21fschecks_count18:01:072
21133993210,7cyclictest6544-21uniq19:56:072
2113299329,5cyclictest0-21swapper/119:36:091
2113299327,23cyclictest0-21swapper/117:01:041
2113299326,5cyclictest18080-21fschecks_count17:51:071
2113499317,14cyclictest0-21swapper/317:01:043
2113499316,5cyclictest39-21ksoftirqd/318:21:023
2113499316,5cyclictest39-21ksoftirqd/318:21:023
2113499316,14cyclictest0-21swapper/319:56:033
2113499311,12cyclictest381ktimersoftd/316:41:163
2113399317,14cyclictest0-21swapper/221:10:552
2113399314,18cyclictest0-21swapper/219:36:042
2113399314,13cyclictest7950irq/56-dwc_otg_18:11:112
2113399312,10cyclictest29-21rcuc/220:06:162
2113399311,7cyclictest16810-21ssh20:47:052
2113399311,21cyclictest0-21swapper/221:04:532
2113399311,18cyclictest32343-21ls17:01:092
21133993111,11cyclictest32003-21diskmemload20:25:362
2113299313,6cyclictest917-21snmpd18:03:221
2113499305,9cyclictest7950irq/56-dwc_otg_20:23:373
2113399307,12cyclictest297832sleep219:45:162
2113399305,4cyclictest7750irq/56-dwc_otg20:36:082
2113399305,4cyclictest7750irq/56-dwc_otg20:36:082
2113399305,15cyclictest19866-21fschecks_count17:56:062
2113399304,12cyclictest24104-21ssh21:36:032
2113399303,9cyclictest0-21swapper/220:42:152
21132993016,4cyclictest21-21rcuc/116:59:121
2113499295,3cyclictest0-21swapper/319:37:353
2113499292,6cyclictest2142-21ntp_kernel_pll_17:06:113
2113499291,18cyclictest0-21swapper/318:09:533
2113499291,15cyclictest10233-21latency_hist17:30:503
207302294,14sleep10-21swapper/116:29:111
2113499282,16cyclictest12038-21latency_hist17:35:503
2113399284,14cyclictest12267-21/usr/sbin/munin17:36:182
201212285,9sleep30-21swapper/316:26:183
2113499271,8cyclictest7950irq/56-dwc_otg_18:02:043
2113499261,5cyclictest0-21swapper/317:51:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional