You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-14 - 16:19

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Sun Dec 14, 2025 12:43:54)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3600703212259,58sleep30-21swapper/307:08:333
360071528555,6sleep00-21swapper/007:08:420
360067928468,10sleep20-21swapper/207:08:162
360048628064,9sleep10-21swapper/107:05:471
3600951992926,1cyclictest0-21swapper/308:43:293
3600951992522,1cyclictest0-21swapper/311:10:143
36231492241,11sleep23623195-21ntpq07:25:242
3600951992421,1cyclictest0-21swapper/311:30:113
3600951992421,1cyclictest0-21swapper/308:05:013
3600951992419,2cyclictest0-21swapper/307:55:013
3600951992320,1cyclictest0-21swapper/307:38:403
360095199227,12cyclictest15850irq/127-ahci[0000:00:17.0]08:50:113
3600951992219,1cyclictest0-21swapper/309:25:003
3600951992219,1cyclictest0-21swapper/308:00:003
39422022211,11sleep13942352-21seq12:30:111
360095199212,12cyclictest3935709-21taskset12:20:173
3600951992118,1cyclictest0-21swapper/309:15:003
3600951992117,2cyclictest0-21swapper/308:15:003
360095199211,16cyclictest3757470-21taskset09:35:093
3600951992015,2cyclictest0-21swapper/311:25:003
360095199201,14cyclictest0-21swapper/308:25:123
3600951991916,1cyclictest0-21swapper/308:15:203
3600951991916,1cyclictest0-21swapper/307:40:183
3600951991916,1cyclictest0-21swapper/307:28:303
3600951991913,3cyclictest3640794-21turbostat07:49:553
38644712181,12sleep23866184-21cut11:15:162
360095199184,7cyclictest3857891-21turbostat11:10:013
360095199183,12cyclictest0-21swapper/308:05:243
360095199182,11cyclictest3921290-21taskset12:10:123
3600951991815,1cyclictest0-21swapper/310:50:163
3600951991815,1cyclictest0-21swapper/310:40:193
3600951991815,1cyclictest0-21swapper/309:50:163
3600951991815,1cyclictest0-21swapper/308:45:143
360095199181,10cyclictest3878172-21taskset11:25:183
37722952171,7sleep13772546-21unixbench_singl09:45:251
375754421711,3sleep23757761-21/usr/sbin/munin09:35:122
360095199172,10cyclictest3951476-21taskset12:35:213
3600951991715,1cyclictest0-21swapper/312:25:163
3600951991714,1cyclictest0-21swapper/309:45:153
3600951991713,2cyclictest0-21swapper/312:00:253
3600951991713,2cyclictest0-21swapper/307:31:293
3600951991713,2cyclictest0-21swapper/307:14:593
360095199171,13cyclictest0-21swapper/312:15:123
360095199171,12cyclictest3810176-21turbostat10:25:003
360095199171,12cyclictest3720073-21chrt09:00:093
360095199171,10cyclictest3799813-21taskset10:15:013
360095199171,10cyclictest3793140-21taskset10:05:183
360095199171,10cyclictest3729383-21taskset09:05:193
360095199171,10cyclictest3617727-21taskset07:20:183
360095199170,10cyclictest3831374-21taskset10:45:073
36925732161,10sleep13692623-21ntpq08:30:181
360095199161,9cyclictest3762761-21taskset09:40:063
3600951991613,2cyclictest0-21swapper/312:00:013
3600951991613,1cyclictest0-21swapper/309:35:013
3600951991613,1cyclictest0-21swapper/308:20:263
360095199161,13cyclictest0-21swapper/312:05:153
360095199161,12cyclictest0-21swapper/311:40:223
360095199161,12cyclictest0-21swapper/309:15:163
360095199161,12cyclictest0-21swapper/307:15:193
360095199161,10cyclictest3815421-21taskset10:30:003
360095199161,10cyclictest3750703-21taskset09:25:193
360095199160,13cyclictest0-21swapper/312:30:193
360095199160,13cyclictest0-21swapper/311:50:173
360095199160,13cyclictest0-21swapper/308:35:223
360095199160,13cyclictest0-21swapper/308:30:113
360095199160,11cyclictest0-21swapper/310:55:123
39479512158,3sleep13948354-21cut12:35:161
39111152151,4sleep23911325-21hddtemp_smartct12:00:142
38856622151,9sleep13887374-21grep11:35:161
38476182151,9sleep23847101-21apt-get11:00:022
38091952151,10sleep13809244-21ntpq10:20:201
37999552152,4sleep13800067-21cpuspeed_turbos10:15:121
37469632155,6sleep03747072-21uniq09:25:120
360095199151,9cyclictest3899977-21taskset11:48:433
360095199151,9cyclictest3847643-21taskset11:00:053
360095199151,12cyclictest3788900-21turbostat10:05:013
360095199151,10cyclictest3817810-21chrt10:30:163
360095199151,10cyclictest3782441-21taskset09:55:163
360095199150,12cyclictest0-21swapper/311:35:113
360095199150,12cyclictest0-21swapper/310:35:233
360095199150,12cyclictest0-21swapper/310:15:143
38017952141,8sleep23803598-21cut10:15:172
37732102141,2sleep10-21swapper/109:50:111
37203512144,6sleep13720561-21idleruntime09:00:131
360095199141,8cyclictest3714768-21taskset08:55:013
360095199141,10cyclictest3867430-21chrt11:15:213
38683402131,8sleep13868502-21gpg-connect-age11:20:001
38319542131,3sleep13832913-21cut10:45:181
38314092131,8sleep23831670-21/usr/sbin/munin10:45:152
37529132131,7sleep23753987-21kthreadcore09:30:162
37524652131,8sleep10-21swapper/109:30:151
36061382131,8sleep13606550-21latency_hist07:15:011
39426862121,4sleep03942763-21kernelversion12:30:140
39014802121,7sleep13903205-21cut11:50:151
38901532121,7sleep23890214-21forks11:40:142
38266932121,6sleep23827638-21kthreadcore10:40:132
38057492121,7sleep00-21swapper/010:20:150
37952342121,7sleep13796513-21kthreadcore10:10:141
37807432121,7sleep23782435-21latency09:55:162
37311812121,3sleep23731868-21kthreadcore09:10:162
37098202120,1sleep10-21swapper/108:50:141
36787092121,7sleep03680096-21kthreadcore08:20:150
36477312121,7sleep23648332-21kthreadcore07:50:172
36415942121,7sleep13642133-21cat07:45:141
38899402114,3sleep10-21swapper/111:40:131
38846612111,4sleep03884680-21apt-get11:35:110
37719052111,5sleep23771943-21ntpq09:45:182
37086552111,5sleep13708899-21users08:45:261
36926252111,4sleep23692679-21ntpq08:30:192
36873772111,5sleep03687431-21ntpq08:25:210
36864062111,6sleep13687217-21cat08:25:181
36555422111,4sleep03655591-21ntpq07:55:200
36250012111,5sleep03625214-21hddtemp_smartct07:30:150
39157752101,5sleep03915955-21cat12:05:000
38873752101,4sleep23888476-21memory11:35:162
38834152101,4sleep13883461-21ntpq11:30:191
38426912103,3sleep10-21swapper/110:55:161
38408672101,4sleep23840913-21ntpq10:50:192
38208442101,4sleep03820863-21apt-get10:35:110
38156582101,5sleep23815814-21cstates10:30:112
37879232101,4sleep13787955-21ntpq10:00:211
37630492101,5sleep13763156-21cut09:40:131
37624812101,5sleep03762533-21tr09:40:000
37253442101,3sleep23725363-21apt-get09:05:112
36942882101,4sleep03694404-21irqstats08:35:150
36889982101,6sleep03689153-21/usr/sbin/munin08:30:130
36778522100,6sleep03678025-21latency_hist08:20:000
36717452101,4sleep03671962-21users08:10:260
36626002101,2sleep10-21swapper/108:05:141
36570892101,5sleep03657118-21sort08:00:120
36352922101,3sleep03635272-31find07:38:310
36346452101,5sleep23634881-21grep07:35:252
36303112101,4sleep13630568-21df07:35:121
36092362100,6sleep03610040-21cut07:15:150
36065512101,5sleep23608104-21grep07:15:022
3933163291,4sleep03934923-21grep12:20:160
3931998290,1sleep10-21swapper/112:20:121
3931065290,1sleep10-21swapper/112:15:281
3899352291,4sleep23899385-21mailstats11:45:232
3874169291,3sleep23874188-21apt-get11:25:102
3858697291,4sleep23859642-21kthreadcore11:10:162
3845935291,3sleep03846076-21ntpq10:55:180
3836859291,3sleep03836365-21apt-get10:50:020
3815931290,1sleep00-21swapper/010:30:140
3810509291,4sleep03810713-21diskstats10:25:130
3784511291,4sleep03785420-21grep10:00:150
3773232291,4sleep23773488-21df09:50:122
3766620291,3sleep23766709-21ntpq09:40:182
3714780291,3sleep03714799-21apt-get08:55:110
3700463291,4sleep00-21swapper/008:40:160
3697881290,4sleep23697937-21ntp_states08:35:212
3672332291,4sleep13672700-21apt-config08:15:001
3646295291,3sleep13646313-21apt-get07:50:121
3636466291,4sleep03638145-21cut07:40:170
3619184291,3sleep03619203-21apt-get07:25:130
3610042291,4sleep13611802-21kthreadcore07:15:171
3606046291,4sleep21-21systemd07:14:282
3946205281,2sleep20-21swapper/212:30:192
3937502281,3sleep23937870-21kthreadcore12:25:152
3935613281,2sleep20-21swapper/212:20:172
3925067281,3sleep13925180-21ntpq12:10:181
3921824281,3sleep23922476-21cat12:10:152
3910543281,3sleep13910796-21latency_hist12:00:011
3904950281,2sleep0800-21snmpd11:50:250
3890436281,3sleep03890647-21kthreadcore11:40:150
3878552281,3sleep1321rcuc/111:25:251
3872878281,2sleep00-21swapper/011:20:200
3869504281,3sleep13871016-21kthreadcore11:20:171
3848222281,3sleep03849506-21kthreadcore11:00:160
3842159281,3sleep23842416-21df10:55:132
3814330281,4sleep23810424-21/usr/sbin/munin10:25:192
3789661281,3sleep23789811-21kthreadcore10:05:152
3788890280,1sleep00-21swapper/010:05:010
3761485280,3sleep13761526-21ntpq09:35:191
3751276280,1sleep10-21swapper/109:25:251
3729197283,2sleep10-21swapper/109:05:181
3703946281,4sleep13703947-21sshd08:43:291
3702974281,3sleep23703063-21ntpq08:40:172
3694405281,3sleep10-21swapper/108:35:161
3667938281,3sleep23669395-21/usr/sbin/munin08:10:162
3662854281,3sleep23664525-21kthreadcore08:05:142
3656812280,3sleep20-21swapper/208:00:102
3655593280,3sleep13655645-21ntp_states07:55:201
3628743281,3sleep23628780-21ntpq07:30:212
3618563281,3sleep13618564-21sshd07:20:301
3614136280,4sleep20-21swapper/207:20:152
3606063281,3sleep03207559-21gdbus07:14:310
3926454270,1sleep00-21swapper/012:15:110
3920961270,1sleep00-21swapper/012:09:260
3917693271,2sleep10-21swapper/112:05:181
3912731271,3sleep1331ktimers/112:00:161
3910391270,1sleep20-21swapper/211:58:062
3903206271,2sleep20-21swapper/211:50:162
3900063270,1sleep10-21swapper/111:49:511
3879427270,2sleep00-21swapper/011:30:100
3868075270,1sleep00-21swapper/011:16:380
3857747270,1sleep20-21swapper/211:08:282
3857632270,1sleep00-21swapper/011:06:560
3849509271,3sleep13851322-21cut11:00:171
3840749271,2sleep13840866-21ntpq10:50:191
3821122271,3sleep23821150-21grep10:35:132
3815636270,2sleep10-21swapper/110:30:101
3815169270,1sleep10-21swapper/110:27:271
3810011270,1sleep20-21swapper/210:23:062
3799933270,2sleep00-21swapper/010:15:110
3777859270,1sleep00-21swapper/009:50:540
3764120271,3sleep03765855-21kthreadcore09:40:160
3751311270,1sleep20-21swapper/209:25:382
3745482271,2sleep20-21swapper/209:20:222
3742913271,3sleep13744687-21kthreadcore09:20:171
3737789270,3sleep20-21swapper/209:15:172
3736145270,1sleep00-21swapper/009:15:060
3735481270,1sleep10-21swapper/109:13:241
3720051270,2sleep20-21swapper/209:00:092
3720044270,1sleep00-21swapper/009:00:040
3711008271,2sleep00-21swapper/008:50:180
3687952270,4sleep20-21swapper/208:25:272
3674909271,2sleep10-21swapper/108:15:161
3645341271,3sleep23641046-21/usr/sbin/munin07:45:232
3641044270,1sleep00-21swapper/007:45:090
3624709270,2sleep10-21swapper/107:30:121
3619757271,2sleep13621058-21kthreadcore07:25:181
3613969270,2sleep00-21swapper/007:20:130
3952104260,4sleep00-21swapper/012:36:230
3948356261,2sleep20-21swapper/212:35:172
3941729260,4sleep10-21swapper/112:27:341
3937244260,3sleep03937278-21fschecks_count12:25:130
3926477261,2sleep20-21swapper/212:15:132
3925199261,2sleep00-21swapper/012:10:210
3919455261,2sleep23919959-21strings12:05:192
3909560261,1sleep00-21swapper/011:55:190
3897728261,2sleep03898918-21memory11:45:170
3871019261,2sleep20-21swapper/211:20:182
3863015260,4sleep00-21swapper/011:13:550
3862011261,2sleep10-21swapper/111:10:191
3857746260,1sleep10-21swapper/111:08:261
3836134260,4sleep00-21swapper/010:47:140
3796514261,2sleep20-21swapper/210:10:162
3789142260,3sleep10-21swapper/110:05:061
3788202260,2sleep20-21swapper/210:00:262
3741444260,3sleep00-21swapper/009:20:120
3736721261,2sleep10-21swapper/109:15:151
3729665261,2sleep03725345-21/usr/sbin/munin09:05:240
3719557260,1sleep20-21swapper/208:57:222
3715684261,2sleep10-21swapper/108:55:161
3713427260,3sleep20-21swapper/208:50:192
3705898261,2sleep00-21swapper/008:45:170
3678425260,2sleep20-21swapper/208:20:142
3672705260,2sleep20-21swapper/208:15:012
3666949260,1sleep00-21swapper/008:08:300
3661778260,1sleep10-21swapper/108:04:301
3655286260,2sleep23655478-21strings07:55:172
3651121260,4sleep00-21swapper/007:53:190
3640525260,1sleep20-21swapper/207:41:432
3878229250,1sleep00-21swapper/011:25:210
3821755250,2sleep13823440-21kthreadcore10:35:161
3782675250,1sleep10-21swapper/109:55:201
3778717250,1sleep00-21swapper/009:55:110
3769793250,1sleep00-21swapper/009:45:170
3756051250,2sleep00-21swapper/009:30:220
3730908250,1sleep00-21swapper/009:10:140
3708989250,3sleep20-21swapper/208:46:282
3678221250,1sleep10-21swapper/108:20:121
3635927250,2sleep10-21swapper/107:40:141
3879557240,1sleep20-21swapper/211:30:122
3826411240,1sleep00-21swapper/010:40:120
3825873240,1sleep10-21swapper/110:40:011
3794480240,1sleep00-21swapper/010:10:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional