You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-31 - 14:56

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Wed Dec 31, 2025 12:43:57)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
384990122650,2sleep13849893-21ntpq07:55:201
384990122650,2sleep13849893-21ntpq07:55:201
3795544212563,57sleep30-21swapper/307:07:463
399249921150,2sleep10-21swapper/110:10:191
40246252910,3sleep0221rcuc/010:40:240
379556328859,9sleep20-21swapper/207:08:002
379557727857,10sleep10-21swapper/107:08:111
379536427655,15sleep00-21swapper/007:05:290
3795824992724,1cyclictest0-21swapper/308:00:013
3795824992724,1cyclictest0-21swapper/308:00:003
3795824992522,1cyclictest0-21swapper/310:30:253
3795824992522,1cyclictest0-21swapper/309:35:013
3795824992219,1cyclictest0-21swapper/308:40:113
3795824992219,1cyclictest0-21swapper/307:45:093
3795824992211,3cyclictest39937962sleep310:15:113
379582499220,15cyclictest0-21swapper/308:04:053
3795824992119,1cyclictest0-21swapper/308:10:113
3795824992118,1cyclictest0-21swapper/312:10:143
3795824992117,2cyclictest0-21swapper/307:21:403
379582499209,8cyclictest3964826-21taskset09:45:163
3795824992017,1cyclictest0-21swapper/307:55:013
3795824992017,1cyclictest0-21swapper/307:30:153
379582499201,10cyclictest3978012-21taskset10:00:123
379582499201,10cyclictest3978012-21taskset10:00:113
39306642198,7sleep03930809-21cut09:15:170
388656721913,2sleep20-21swapper/208:30:212
3795824991916,1cyclictest0-21swapper/312:25:203
3795824991916,1cyclictest0-21swapper/310:35:153
3795824991916,1cyclictest0-21swapper/309:05:153
3795824991916,1cyclictest0-21swapper/307:35:243
379582499191,14cyclictest3956783-21chrt09:40:113
379582499191,14cyclictest0-21swapper/311:45:123
379582499191,14cyclictest0-21swapper/307:25:123
379582499191,13cyclictest0-21swapper/308:30:123
379582499191,11cyclictest3919237-21taskset09:05:013
38968752189,5sleep13896988-21head08:40:191
38825732181,4sleep13882592-21apt-get08:30:131
3795824991816,1cyclictest0-21swapper/307:40:143
3795824991815,1cyclictest0-21swapper/312:30:173
3795824991815,1cyclictest0-21swapper/312:20:013
3795824991815,1cyclictest0-21swapper/311:30:133
379582499181,15cyclictest0-21swapper/309:15:123
379582499181,13cyclictest4083744-21chrt11:40:123
379582499181,13cyclictest0-21swapper/309:10:153
379582499181,10cyclictest4000965-21taskset10:20:163
379582499177,6cyclictest4062143-21taskset11:18:563
379582499172,10cyclictest4008555-21taskset10:25:203
3795824991715,1cyclictest0-21swapper/307:15:013
3795824991714,1cyclictest0-21swapper/310:50:153
3795824991714,1cyclictest0-21swapper/310:45:123
3795824991713,2cyclictest0-21swapper/311:10:133
3795824991713,2cyclictest0-21swapper/308:05:183
3795824991712,2cyclictest38672062sleep308:15:163
379582499171,13cyclictest0-21swapper/308:50:003
379582499171,12cyclictest4104967-21turbostat12:00:003
379582499171,10cyclictest4040272-21taskset10:55:233
379582499170,14cyclictest0-21swapper/309:30:003
379582499170,13cyclictest0-21swapper/311:20:203
379582499170,13cyclictest0-21swapper/309:35:193
40932922160,1sleep20-21swapper/211:45:212
40625142161,10sleep24062526-21cat11:20:122
38498442167,5sleep0151ktimers/007:55:200
38498442167,5sleep0151ktimers/007:55:200
379582499161,9cyclictest4072774-21taskset11:27:183
379582499161,3cyclictest0-21swapper/307:10:113
3795824991613,1cyclictest0-21swapper/310:40:163
3795824991612,2cyclictest0-21swapper/308:55:233
379582499161,11cyclictest3877066-21sleep08:25:003
379582499160,13cyclictest0-21swapper/312:37:253
379582499160,13cyclictest0-21swapper/312:05:173
379582499160,13cyclictest0-21swapper/312:05:173
379582499160,13cyclictest0-21swapper/312:00:143
379582499160,13cyclictest0-21swapper/310:10:163
379582499160,13cyclictest0-21swapper/309:50:213
379582499160,13cyclictest0-21swapper/309:20:253
379582499160,12cyclictest0-21swapper/311:35:223
40137542151,9sleep14013805-21ntpq10:30:201
39926542151,10sleep03992769-21gzip10:10:230
38980072150,1sleep20-21swapper/208:44:412
379582499151,1cyclictest0-21swapper/312:20:123
379582499151,12cyclictest0-21swapper/308:35:163
379582499150,13cyclictest0-21swapper/311:00:133
379582499150,12cyclictest0-21swapper/311:50:013
379582499150,12cyclictest0-21swapper/310:05:163
379582499150,10cyclictest3977701-21chrt09:59:553
41461152140,1sleep10-21swapper/112:35:221
41115222141,8sleep14113185-21cut12:05:171
41115222141,8sleep14113185-21cut12:05:161
39659062140,1sleep10-21swapper/109:45:201
39606912141,9sleep23960736-21ld09:40:192
39308102140,3sleep10-21swapper/109:15:161
39091242140,6sleep03909344-21cat08:55:160
38633802140,8sleep13865122-21kthreadcore08:10:161
38405542140,2sleep10-21swapper/107:50:131
379582499141,9cyclictest3908436-21chrt08:53:473
379582499141,8cyclictest4046726-21taskset11:05:013
3795824991411,1cyclictest0-21swapper/308:25:133
41162232131,7sleep14116261-21kthreadcore12:10:171
39977752131,3sleep00-21swapper/010:15:210
39456932131,8sleep13945717-21cut09:30:011
38726172131,2sleep20-21swapper/208:20:152
38193682131,1sleep20-21swapper/207:30:132
41366962121,6sleep04136734-21basename12:30:000
40790202121,6sleep04079069-21kthreadcore11:35:170
39493352121,7sleep03949839-21needreboot09:30:190
39143792120,8sleep23914529-21grep09:00:162
40475262111,6sleep24049133-21kthreadcore11:05:182
40403292111,6sleep04040414-21ps10:55:260
40085132111,4sleep24008553-21ntpq10:25:202
39354072110,1sleep20-21swapper/209:20:112
39261182111,2sleep10-21swapper/109:10:201
39097222111,6sleep23911354-21kthreadcore08:55:182
39093452110,7sleep13909721-21cat08:55:171
38621442110,3sleep00-21swapper/008:10:160
38010222111,5sleep13801152-21munin-node07:15:011
41388702101,2sleep10-21swapper/112:30:171
41264472101,5sleep14126528-21tr12:20:111
41258662101,4sleep24125894-21munin-run12:20:012
41234392101,4sleep04124701-21memory12:15:170
41153952101,4sleep04115443-21munin-run12:10:010
41153952101,4sleep04115443-21munin-run12:10:000
40837222101,4sleep24083741-21apt-get11:40:122
40737552101,4sleep14073825-21irqstats11:30:171
40638242101,5sleep04065458-21grep11:20:180
40522902101,4sleep04052499-21hddtemp_smartct11:10:160
40189302101,4sleep24019018-21ntpq10:35:182
40139092103,1sleep0171rcu_preempt10:30:230
40095352101,5sleep14009590-21wc10:30:001
40088422101,5sleep04008925-21expr10:25:270
39726622101,3sleep13972169-21apt-get09:55:021
39570622101,5sleep13957289-21basename09:40:141
39408462101,4sleep03940969-21diskstats09:25:150
38878842101,4sleep13888087-21diskstats08:35:131
38415222101,5sleep23842603-21kthreadcore07:50:182
38246062101,4sleep03824625-21apt-get07:35:120
38140722101,4sleep13813574-21apt-get07:25:021
38073972101,5sleep23807507-21uname07:15:212
4147007291,3sleep0151ktimers/012:38:380
4103944291,3sleep04103991-21ntpq11:55:210
4094243291,4sleep04094472-21cat11:50:000
4089252291,4sleep14089348-21cpuspeed_turbos11:45:121
4078451290,1sleep10-21swapper/111:35:021
4062800291,4sleep14062924-21perl11:20:151
4051999291,3sleep14052018-21apt-get11:10:131
4041504291,3sleep24041523-21apt-get11:00:112
4036848291,4sleep24037783-21grep10:55:192
4032032291,4sleep14033718-21cut10:50:161
4020332291,4sleep24020471-21grep10:40:112
4016687291,4sleep04018425-21tr10:35:170
4003007291,4sleep24003055-21ntpq10:20:202
3977053291,4sleep03977329-21sed09:55:290
3973300291,4sleep23974841-21grep09:55:192
3935698291,4sleep13935907-21hddtemp_smartct09:20:141
3902130291,4sleep23902271-21ntpq08:45:202
3877385291,3sleep03877588-21diskstats08:25:140
3860620291,4sleep03860682-21cat08:05:260
3855395291,4sleep13855453-21cat08:00:261
3851649291,4sleep03853051-21tr08:00:170
3834055291,4sleep23834101-21ntpq07:40:202
3830516290,2sleep10-21swapper/107:40:151
3829365290,3sleep13828939-31frcode07:35:371
3827550291,4sleep23828458-21wc07:35:182
3797208292,3sleep23798867-21kthreadcore07:10:162
4146906280,1sleep20-21swapper/212:37:202
4141015282,3sleep24141105-21wc12:30:222
4132224281,3sleep20-21swapper/212:25:182
4109381280,1sleep00-21swapper/012:00:230
4109233280,3sleep24109287-21ntpq12:00:212
4105254281,3sleep14105437-21cstates12:00:121
4073826281,3sleep24075070-21kthreadcore11:30:182
4058837281,3sleep04060559-21kthreadcore11:15:170
4047256281,3sleep14047524-21kthreadcore11:05:171
4036245280,3sleep14035748-21apt-get10:55:021
4025637281,3sleep14025830-21diskstats10:45:121
4010401281,3sleep20-21swapper/210:30:172
3993990281,3sleep13993797-21/usr/sbin/munin10:15:131
3979923281,3sleep03981682-21kthreadcore10:00:170
3979923281,3sleep03981682-21kthreadcore10:00:170
3977990280,2sleep20-21swapper/210:00:122
3977990280,2sleep20-21swapper/210:00:112
3967185280,3sleep00-21swapper/009:50:110
3944645281,3sleep23944693-21ntpq09:25:202
3925193281,3sleep23925289-21tune2fs09:10:182
3908536281,3sleep03908630-21latency_hist08:55:000
3903287281,3sleep13903438-21latency_hist08:50:001
3892735280,1sleep00-21swapper/008:39:210
3892543280,1sleep20-21swapper/208:36:542
3835323281,4sleep03835479-21cstates07:45:100
3819664281,3sleep13819876-21smartctl07:30:171
3817962280,4sleep20-21swapper/207:25:192
3813209280,1sleep20-21swapper/207:20:322
3808288281,3sleep03808305-21latency_hist07:20:000
4103993270,2sleep10-21swapper/111:55:221
4094797271,3sleep14094776-21/usr/sbin/munin11:50:141
4088694270,3sleep14089007-21latency_hist11:45:001
4082527270,1sleep20-21swapper/211:35:242
4072850270,1sleep10-21swapper/111:28:171
4068002270,3sleep20-21swapper/211:25:142
4057540271,3sleep14057700-21gltestperf11:15:151
4031341271,2sleep00-21swapper/010:50:150
4026113270,3sleep0221rcuc/010:45:150
4020310270,2sleep14020329-21apt-get10:40:101
4003864270,1sleep00-21swapper/010:23:140
3999056270,2sleep13999075-21apt-get10:20:101
3998612270,1sleep20-21swapper/210:17:592
3988101270,1sleep10-21swapper/110:07:561
3984513270,3sleep23986178-21cut10:05:162
3966670271,3sleep2800-21snmpd09:46:492
3952036271,2sleep03953082-21kthreadcore09:35:170
3950648270,1sleep10-21swapper/109:31:281
3946527271,2sleep23947595-21cut09:30:172
3940366270,1sleep00-21swapper/009:24:220
3929592270,1sleep00-21swapper/009:11:110
3920209271,2sleep10-21swapper/109:05:151
3919637270,2sleep00-21swapper/009:05:110
3913882271,2sleep03908583-21kworker/0:1+events09:00:010
3880830271,2sleep13881259-21strings08:25:191
3866821270,2sleep00-21swapper/008:15:120
3856346270,2sleep20-21swapper/208:05:132
3855762270,1sleep20-21swapper/208:00:552
3850714270,1sleep20-21swapper/207:59:032
3850714270,1sleep20-21swapper/207:59:022
3818386270,2sleep00-21swapper/007:25:260
4136954260,1sleep00-21swapper/012:30:090
4136388260,4sleep10-21swapper/112:26:091
4131159260,1sleep20-21swapper/212:21:122
4131156260,1sleep00-21swapper/012:21:100
4125822260,1sleep10-21swapper/112:19:321
4120375260,4sleep00-21swapper/012:11:480
4115648260,1sleep20-21swapper/212:10:062
4104641260,1sleep20-21swapper/211:56:072
4099735260,1sleep00-21swapper/011:54:580
4098797260,2sleep24094776-21/usr/sbin/munin11:50:222
4088380260,4sleep00-21swapper/011:41:030
4078059260,4sleep00-21swapper/011:33:130
4072879260,3sleep00-21swapper/011:28:390
4062104260,1sleep20-21swapper/211:18:282
4052021260,2sleep20-21swapper/211:10:142
4043313260,2sleep00-21swapper/011:00:180
4041794261,2sleep10-21swapper/111:00:151
4025831260,3sleep2421rcuc/210:45:122
4019837260,4sleep10-21swapper/110:37:391
3988551260,2sleep20-21swapper/210:10:142
3988062260,1sleep00-21swapper/010:07:270
3971821260,1sleep20-21swapper/209:50:452
3967206260,2sleep10-21swapper/109:50:121
3966678260,1sleep00-21swapper/009:46:550
3961020260,2sleep00-21swapper/009:40:250
3955785260,3sleep10-21swapper/109:35:261
3919785260,2sleep20-21swapper/209:05:122
3915649261,2sleep10-21swapper/109:00:181
3908473260,4sleep20-21swapper/208:54:152
3908209260,1sleep10-21swapper/108:50:531
3887452260,1sleep00-21swapper/008:33:570
3868060261,2sleep10-21swapper/108:15:171
3860426261,2sleep13860529-21processes08:05:231
3821426260,3sleep03823121-21latency07:30:200
3813453260,4sleep10-21swapper/107:23:391
3810656261,2sleep00-21swapper/007:20:170
3805500260,3sleep13807116-21cat07:15:161
3800991260,1sleep00-21swapper/007:14:400
4113187250,1sleep20-21swapper/212:05:182
4113187250,1sleep20-21swapper/212:05:172
4051115251,1sleep00-21swapper/011:05:280
3934428250,2sleep20-21swapper/209:15:262
3902326250,1sleep00-21swapper/008:45:220
3897100250,2sleep00-21swapper/008:40:220
3875281250,1sleep00-21swapper/008:20:180
3872461250,1sleep10-21swapper/108:20:151
3840011250,1sleep20-21swapper/207:46:282
3837029250,2sleep10-21swapper/107:45:151
3833877250,2sleep00-21swapper/007:40:170
4031058240,1sleep20-21swapper/210:50:132
3977984240,1sleep10-21swapper/110:00:071
3977984240,1sleep10-21swapper/110:00:061
3951435240,1sleep20-21swapper/209:35:022
3877299240,1sleep20-21swapper/208:25:122
3867102240,1sleep20-21swapper/208:15:142
3861313240,1sleep20-21swapper/208:10:012
3840889240,1sleep00-21swapper/007:50:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional