You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2026-01-12 - 06:44

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack3slot3s.osadl.org (updated Mon Jan 12, 2026 00:43:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
368482622490,1sleep13684833-21needreboot20:35:201
3589353212765,57sleep30-21swapper/319:09:293
358931927954,19sleep00-21swapper/019:09:020
358917527963,10sleep20-21swapper/219:07:122
358906227863,9sleep10-21swapper/119:05:471
3589504992522,1cyclictest0-21swapper/300:00:013
3589504992521,2cyclictest0-21swapper/323:40:003
358950499231,18cyclictest3902754-21sleep00:05:013
3589504992219,1cyclictest0-21swapper/319:20:153
3589504992218,2cyclictest0-21swapper/323:00:013
3589504992217,2cyclictest0-21swapper/323:44:593
3589504992119,1cyclictest0-21swapper/322:25:003
3589504992119,1cyclictest0-21swapper/320:25:153
3589504992118,1cyclictest0-21swapper/322:35:233
37239712206,4sleep13724957-21cut21:15:171
358950499207,7cyclictest3886634-21taskset23:46:563
3589504991916,1cyclictest0-21swapper/320:33:303
3589504991916,1cyclictest0-21swapper/320:15:173
3589504991915,2cyclictest0-21swapper/322:45:283
3589504991915,2cyclictest0-21swapper/300:20:003
358950499191,14cyclictest0-21swapper/321:50:093
358950499191,14cyclictest0-21swapper/320:10:343
358950499191,13cyclictest3855700-21chrt23:20:113
358950499191,13cyclictest0-21swapper/321:55:093
358950499188,7cyclictest15850irq/127-ahci[0000:00:17.0]20:44:553
358950499185,8cyclictest3708581-21taskset21:00:143
358950499182,11cyclictest3637835-21taskset19:50:213
358950499182,10cyclictest3904830-21taskset00:05:163
358950499182,10cyclictest3742767-21taskset21:30:173
3589504991816,1cyclictest0-21swapper/300:20:143
3589504991814,2cyclictest0-21swapper/320:50:013
358950499181,13cyclictest0-21swapper/323:00:133
358950499181,10cyclictest3735932-21taskset21:25:163
358950499180,13cyclictest0-21swapper/320:35:133
358950499172,10cyclictest3630726-21taskset19:45:163
3589504991714,1cyclictest0-21swapper/323:25:183
3589504991714,1cyclictest0-21swapper/320:50:163
358950499171,3cyclictest0-21swapper/320:06:493
3589504991713,2cyclictest0-21swapper/321:35:273
3589504991713,2cyclictest0-21swapper/321:20:003
3589504991713,2cyclictest0-21swapper/319:40:153
358950499171,14cyclictest0-21swapper/323:50:113
358950499171,13cyclictest3934467-21taskset00:34:463
358950499171,13cyclictest0-21swapper/300:35:103
358950499171,10cyclictest3648577-21taskset20:00:273
358950499171,10cyclictest3621452-21taskset19:35:183
358950499170,15cyclictest0-21swapper/323:10:253
358950499170,11cyclictest3733333-21chrt21:21:313
358950499170,10cyclictest3789166-21taskset22:15:183
358950499170,10cyclictest3749871-21chrt21:40:073
38973462161,10sleep01-21systemd00:00:000
37617672161,10sleep03763491-21grep21:50:150
36380962161,10sleep03638344-21grep19:50:270
36181992161,8sleep23618322-21diskstats19:35:142
358950499162,9cyclictest3775720-21taskset22:02:313
358950499161,9cyclictest3866135-21taskset23:30:013
358950499161,9cyclictest3866135-21taskset23:30:013
3589504991614,1cyclictest0-21swapper/319:25:143
3589504991613,1cyclictest0-21swapper/319:55:163
3589504991613,1cyclictest0-21swapper/319:10:173
3589504991612,2cyclictest0-21swapper/323:05:173
358950499161,12cyclictest0-21swapper/322:05:223
358950499161,11cyclictest0-21swapper/320:55:213
358950499160,13cyclictest0-21swapper/322:50:133
358950499160,13cyclictest0-21swapper/322:10:263
358950499160,13cyclictest0-21swapper/300:10:193
358950499160,12cyclictest0-21swapper/322:30:203
358950499160,10cyclictest3665206-21taskset20:20:073
358950499160,10cyclictest3665206-21taskset20:20:063
39300532151,9sleep23930807-21grep00:30:172
37286842151,9sleep03728906-21cat21:20:130
37082642151,8sleep23708577-21kthreadcore21:00:142
37074352151,5sleep03707448-21turbostat.cron21:00:010
36657712151,9sleep23666491-21cut20:20:152
36657712151,9sleep23666491-21cut20:20:152
36028852151,9sleep13604411-21tr19:20:181
358950499151,9cyclictest3855044-21taskset23:15:383
358950499151,9cyclictest3797425-21taskset22:25:123
358950499151,9cyclictest3717664-21taskset21:06:563
358950499151,10cyclictest3929261-21sleep00:30:003
358950499150,12cyclictest0-21swapper/322:40:093
358950499150,12cyclictest0-21swapper/321:45:143
358950499150,12cyclictest0-21swapper/321:10:113
358950499150,12cyclictest0-21swapper/319:15:153
358950499150,12cyclictest0-21swapper/319:15:143
358950499141,11cyclictest0-21swapper/319:35:013
38556782132,5sleep23855697-21apt-get23:20:112
38188772131,7sleep13819000-21diskstats22:45:151
37827052131,7sleep23784422-21kthreadcore22:10:172
36563022131,8sleep03658057-21cut20:10:190
36525452131,8sleep13653303-21sed20:05:191
36335212131,8sleep23633526-21cron19:50:012
38753602121,7sleep03875404-21collect223:35:200
38446282121,8sleep1800-21snmpd23:05:431
38299222121,7sleep23830060-21kthreadcore22:55:142
37440222122,7sleep0800-21snmpd21:34:160
36844382121,7sleep03684820-21needreboot20:35:200
36236012121,7sleep03623790-21gltestperf19:40:120
35902612124,4sleep13591298-21cut19:10:151
39029822110,2sleep10-21swapper/100:05:101
38877252111,6sleep13889211-21kthreadcore23:50:161
38544192111,5sleep13854468-21ntpq23:15:201
38491652111,5sleep03849211-21ntpq23:10:190
38192812110,6sleep03820290-21tr22:45:180
37761762111,4sleep13776195-21apt-get22:05:111
37721712111,6sleep13773886-21kthreadcore22:00:181
37168082111,4sleep13716896-21ntpq21:05:191
36022812111,4sleep03602300-21apt-get19:20:130
39286152101,4sleep23928858-21users00:25:272
39121932101,4sleep03912244-21ntpq00:10:190
39030042100,1sleep20-21swapper/200:05:122
39016652102,4sleep03901712-21ntpq00:00:210
38661492101,4sleep13866168-21apt-get23:30:121
38661492101,4sleep13866168-21apt-get23:30:121
38609362101,4sleep03861192-21df23:25:140
38559692101,4sleep13856178-21hddtemp_smartct23:20:141
38455242101,4sleep23845732-21hddtemp_smartct23:10:132
38288712101,4sleep13828926-21munin-run22:55:011
38252982101,6sleep23827022-21kthreadcore22:50:172
38177662101,5sleep13818008-21sed22:40:261
38029822101,5sleep03803142-21grep22:30:150
38014982100,2sleep10-21swapper/122:25:231
38014482101,4sleep03801497-21modprobe22:25:220
37852072101,4sleep03785347-21ntpq22:10:180
36765852101,5sleep23678238-21tr20:30:172
36744962101,4sleep13674546-21ntp_states20:25:231
36498942101,5sleep23650044-21interrupts20:05:162
36337822101,4sleep13633801-21apt-get19:50:111
36324732101,4sleep13632530-21ntpq19:45:191
36272782101,4sleep23627325-21ntpq19:40:192
36167552101,4sleep13616801-21ntpq19:30:201
35898572101,4sleep23589980-21diskstats19:10:122
3897818291,4sleep13897941-21diskstats00:00:141
3880671291,4sleep03880719-21ntpq23:40:210
3865265291,4sleep23860913-21/usr/sbin/munin23:25:272
3842050291,4sleep03843764-21sed23:05:170
3840493291,4sleep23840568-21grep23:05:162
3833488291,4sleep03833535-21awk22:55:200
3828005291,4sleep03828123-21sendmail_mailst22:50:230
3808778291,4sleep23809259-21grep22:35:182
3793271293,3sleep03795001-21kthreadcore22:20:160
3781951291,4sleep13782704-21tr22:10:161
3765278291,4sleep23765281-21sshd21:53:452
3753881291,4sleep13753934-21ntp_states21:40:221
3744344290,3sleep00-21swapper/021:35:120
3733892291,3sleep23734145-21df21:25:122
3732920291,4sleep13732925-21sensors_temp21:20:261
3696638291,4sleep23696694-21apt-config20:50:012
3686164291,4sleep03686184-21dpkg20:40:130
3645882291,4sleep13647651-21cut20:00:171
3622474291,4sleep03622659-21users19:35:280
3611657291,4sleep13611773-21sendmail_mailqu19:25:231
3590191291,4sleep03590260-21irqstats19:10:150
3939619280,1sleep10-21swapper/100:38:461
3935315281,3sleep03936297-21tr00:35:160
3929507281,3sleep13929510-21apt-get00:30:121
3928539281,3sleep13928614-21/usr/sbin/munin00:25:271
3918752281,3sleep23918769-21apt00:20:112
3917774280,1sleep10-21swapper/100:15:251
3908875281,3sleep13910491-21kthreadcore00:10:161
3887136281,2sleep00-21swapper/023:50:110
3881894280,1sleep00-21swapper/023:45:060
3881307280,2sleep20-21swapper/223:40:472
3851002281,3sleep03851717-21kthreadcore23:15:160
3823661281,3sleep20-21swapper/222:50:012
3818436280,1sleep20-21swapper/222:44:512
3806561281,3sleep13806674-21ntpq22:30:191
3802286280,1sleep20-21swapper/222:28:122
3791861281,3sleep23791880-21apt-get22:20:112
3769762280,1sleep20-21swapper/221:55:222
3766905281,3sleep03767698-21kthreadcore21:55:160
3713524281,3sleep23714645-21grep21:05:162
3707224280,2sleep10-21swapper/120:57:211
3698350281,3sleep23700048-21kthreadcore20:50:162
3697187281,3sleep13697226-21cpu20:50:121
3690878280,1sleep10-21swapper/120:42:011
3675709281,3sleep13675712-21apt-get20:30:121
3674445281,4sleep03674495-21collect220:25:220
3663980281,3sleep13664037-21ntp_states20:15:211
3663761281,3sleep23663883-21netstat20:15:182
3654696281,2sleep23654715-21apt-get20:10:122
3639315281,3sleep23639355-21grep19:55:142
3632901280,1sleep00-21swapper/019:45:260
3617410280,1sleep20-21swapper/219:30:292
3596950280,2sleep00-21swapper/019:15:120
3596950280,2sleep00-21swapper/019:15:120
3594888280,1sleep20-21swapper/219:15:022
3594888280,1sleep20-21swapper/219:15:012
3919898271,3sleep03921595-21tr00:20:170
3891417271,3sleep20-21swapper/223:50:252
3882456271,2sleep20-21swapper/223:45:162
3882287271,3sleep10-21swapper/123:45:151
3876354270,1sleep20-21swapper/223:39:432
3857479271,3sleep03859280-21kthreadcore23:20:170
3834702270,1sleep10-21swapper/123:00:101
3813012271,3sleep13813013-21sshd22:37:451
3812835270,1sleep00-21swapper/022:35:320
3806679271,1sleep20-21swapper/222:30:202
3796834270,1sleep10-21swapper/122:24:261
3770388270,4sleep10-21swapper/121:56:491
3764467270,4sleep13764511-21pluginstate21:50:201
3755088271,2sleep23755100-21apt-get21:45:112
3749133270,1sleep10-21swapper/121:37:401
3739082270,1sleep20-21swapper/221:30:092
3733870270,2sleep10-21swapper/121:25:101
3723402271,1sleep00-21swapper/021:15:110
3722895270,1sleep10-21swapper/121:11:521
3717217271,3sleep00-21swapper/021:05:260
3712003271,2sleep03712074-21find21:00:250
3691647270,2sleep10-21swapper/120:45:111
3690148270,4sleep20-21swapper/220:40:232
3681478271,2sleep20-21swapper/220:35:182
3675160270,1sleep20-21swapper/220:26:362
3660656271,2sleep00-21swapper/020:15:160
3659572270,1sleep10-21swapper/120:13:481
3648532271,3sleep23648576-21wc20:00:262
3639554270,3sleep03639682-21cut19:55:160
3623311270,2sleep10-21swapper/119:40:101
3612214270,1sleep00-21swapper/019:25:580
3934236260,1sleep00-21swapper/000:31:510
3923370260,1sleep10-21swapper/100:21:061
3917485260,2sleep00-21swapper/000:15:200
3903273261,2sleep00-21swapper/000:05:140
3902315260,4sleep20-21swapper/200:00:312
3893200261,2sleep10-21swapper/123:55:171
3892651260,3sleep20-21swapper/223:55:142
3880727261,2sleep13880772-21/usr/sbin/munin23:40:221
3875406260,2sleep10-21swapper/123:35:211
3870843260,4sleep00-21swapper/023:31:310
3870843260,4sleep00-21swapper/023:31:310
3855225260,1sleep20-21swapper/223:17:502
3849825260,4sleep10-21swapper/123:10:321
3834983261,2sleep00-21swapper/023:00:130
3834704260,1sleep20-21swapper/223:00:112
3834088262,2sleep13589498-21cyclictest22:55:491
3815381261,2sleep00-21swapper/022:40:170
3790416260,2sleep00-21swapper/022:15:200
3787515261,2sleep20-21swapper/222:15:182
3787171260,2sleep10-21swapper/122:15:171
3775916261,2sleep20-21swapper/222:05:002
3771484260,2sleep03772167-21kthreadcore22:00:170
3759743260,1sleep10-21swapper/121:45:311
3754746260,1sleep20-21swapper/221:44:082
3750481260,2sleep00-21swapper/021:40:170
3739658261,2sleep10-21swapper/121:30:161
3733486260,4sleep20-21swapper/221:23:272
3723394260,1sleep20-21swapper/221:15:062
3722873260,1sleep20-21swapper/221:11:372
3722858260,4sleep00-21swapper/021:11:270
3707392260,1sleep20-21swapper/220:59:312
3702026260,1sleep00-21swapper/020:53:030
3696434260,1sleep00-21swapper/020:47:260
3680349260,1sleep00-21swapper/020:31:180
3670169260,1sleep00-21swapper/020:24:340
3670169260,1sleep00-21swapper/020:24:340
3665209261,2sleep10-21swapper/120:20:101
3665209261,2sleep10-21swapper/120:20:101
3649110260,1sleep00-21swapper/020:03:370
3643823260,1sleep10-21swapper/119:58:131
3637779261,2sleep20-21swapper/219:50:212
3617539260,1sleep00-21swapper/019:32:030
3608120261,2sleep20-21swapper/219:25:162
3596972261,2sleep10-21swapper/119:15:131
3596972261,2sleep10-21swapper/119:15:131
3938049250,1sleep20-21swapper/200:35:182
3924187250,1sleep00-21swapper/000:25:020
3917582250,1sleep20-21swapper/200:15:212
3912540250,1sleep20-21swapper/200:10:252
3776198250,1sleep20-21swapper/222:05:122
3737620250,1sleep00-21swapper/021:25:170
3649568250,1sleep00-21swapper/020:05:140
3622751250,4sleep10-21swapper/119:36:291
3606358250,1sleep20-21swapper/219:20:232
3865887240,1sleep20-21swapper/223:30:012
3865887240,1sleep20-21swapper/223:30:002
3861193240,1sleep10-21swapper/123:25:141
3776170240,1sleep00-21swapper/022:05:060
3754870240,1sleep00-21swapper/021:45:020
3744242240,1sleep20-21swapper/221:35:022
3707450240,1sleep10-21swapper/121:00:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional