You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-16 - 21:42

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Tue Dec 16, 2025 12:43:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3481783211761,52sleep30-21swapper/307:06:343
36308362910,2sleep0221rcuc/009:25:250
348177628549,10sleep10-21swapper/107:06:291
347718228353,9sleep20-21swapper/207:05:062
348191728160,15sleep00-21swapper/007:08:160
3482183993220,8cyclictest3631558-21sleep09:30:003
35784652300,2sleep1321rcuc/108:35:261
3482183992521,2cyclictest0-21swapper/310:20:173
348218399241,19cyclictest3620692-21taskset09:15:273
348218399230,8cyclictest3615185-21taskset09:10:253
348218399230,19cyclictest3552431-21chrt08:14:433
348218399221,17cyclictest3546404-21taskset08:05:213
348218399221,10cyclictest3637069-21turbostat09:34:593
3482183992118,1cyclictest0-21swapper/309:45:253
3482183992118,1cyclictest0-21swapper/308:50:003
3482183992118,1cyclictest0-21swapper/308:15:133
3482183992118,1cyclictest0-21swapper/307:25:143
3482183992017,1cyclictest0-21swapper/307:38:163
348218399201,16cyclictest0-21swapper/310:19:593
348218399201,15cyclictest0-21swapper/309:00:113
348218399201,15cyclictest0-21swapper/308:00:123
3482183991916,1cyclictest0-21swapper/311:35:173
3482183991916,1cyclictest0-21swapper/307:13:553
348218399191,14cyclictest3768496-21chrt11:40:113
348218399191,14cyclictest3768496-21chrt11:40:113
348218399191,11cyclictest0-21swapper/312:10:183
348218399189,5cyclictest3815238-21turbostat12:24:593
348218399184,7cyclictest3583943-21taskset08:42:053
3482183991815,1cyclictest0-21swapper/311:05:173
3482183991815,1cyclictest0-21swapper/310:40:163
3482183991815,1cyclictest0-21swapper/310:15:003
3482183991815,1cyclictest0-21swapper/309:45:003
3482183991814,2cyclictest0-21swapper/311:50:013
3482183991814,2cyclictest0-21swapper/309:55:253
3482183991814,2cyclictest0-21swapper/308:30:173
348218399181,13cyclictest0-21swapper/311:00:103
348218399181,13cyclictest0-21swapper/310:05:113
348218399181,13cyclictest0-21swapper/309:05:133
36634262171,10sleep23663445-21apt-get10:00:122
35317832171,5sleep23531802-21apt-get07:55:112
3482183991714,1cyclictest0-21swapper/311:10:153
3482183991714,1cyclictest0-21swapper/309:50:233
3482183991713,2cyclictest0-21swapper/311:20:143
348218399171,13cyclictest0-21swapper/308:50:123
348218399171,13cyclictest0-21swapper/307:25:003
348218399171,12cyclictest0-21swapper/311:15:133
348218399171,12cyclictest0-21swapper/307:55:123
348218399171,10cyclictest3719612-21taskset10:50:183
348218399170,11cyclictest3830868-21chrt12:39:393
348218399170,10cyclictest3562642-21taskset08:21:093
348218399170,10cyclictest3521283-21taskset07:45:023
36054212161,7sleep23605432-21latency_hist09:05:002
348218399169,4cyclictest0-21swapper/311:50:123
348218399161,9cyclictest3721308-21taskset10:55:113
348218399161,9cyclictest3573952-21taskset08:35:113
348218399161,9cyclictest3563196-21taskset08:25:003
3482183991613,1cyclictest0-21swapper/310:35:233
3482183991611,3cyclictest0-21swapper/312:25:123
348218399161,12cyclictest3757754-21sleep11:30:003
348218399161,11cyclictest3600186-21turbostat09:00:003
348218399160,13cyclictest0-21swapper/307:50:103
37742382151,10sleep13774921-21cut11:45:161
37257022151,9sleep23725947-21users10:55:272
37145642150,2sleep10-21swapper/110:45:201
36934512151,9sleep13693567-21gcc10:25:191
35684622151,9sleep03568528-21grep08:30:120
35542392151,9sleep13556032-21cut08:15:181
348218399151,9cyclictest3715237-21taskset10:45:363
348218399151,9cyclictest3693887-21taskset10:25:243
348218399151,8cyclictest3789073-21turbostat12:00:013
348218399151,2cyclictest0-21swapper/310:30:123
3482183991512,1cyclictest0-21swapper/307:30:123
3482183991511,3cyclictest0-21swapper/312:05:123
348218399151,10cyclictest3761799-21chrt11:30:173
348218399150,13cyclictest0-21swapper/312:00:143
348218399150,13cyclictest0-21swapper/309:20:123
348218399150,12cyclictest0-21swapper/312:15:213
348218399150,11cyclictest0-21swapper/307:40:183
38276342144,6sleep23829441-21kthreadcore12:35:142
38217182141,8sleep13823447-21tr12:30:171
36989672141,8sleep23699095-21readlink10:30:222
35585152141,8sleep13558817-21grep08:20:161
35303682141,9sleep23530509-21ntpq07:50:182
35154512141,8sleep21-21systemd07:37:562
348218399141,9cyclictest3489626-21chrt07:15:123
348218399141,10cyclictest3663448-21chrt10:00:123
348218399140,12cyclictest0-21swapper/312:30:143
37324022137,2sleep10-21swapper/111:05:161
36589942133,6sleep03660696-21kthreadcore09:55:160
36360422131,8sleep03636094-21ld09:30:200
35588182131,7sleep23560479-21kthreadcore08:20:172
348218399131,9cyclictest3637329-21chrt09:35:113
38247542121,7sleep23824818-21pmu-power12:30:222
38206922121,5sleep03820711-21apt-get12:30:120
37945682121,5sleep13794587-21apt-get12:05:121
37319072121,6sleep23732063-21cstates11:05:132
37271232121,7sleep13727363-21cut11:00:141
37006562121,7sleep03701361-21kthreadcore10:35:170
36484662121,6sleep13650118-21kthreadcore09:45:151
35959882121,7sleep03597673-21tr08:55:170
35068572121,7sleep13508612-21kthreadcore07:30:171
38000872111,5sleep23800237-21grep12:10:132
37671722111,5sleep03767213-21ntpq11:35:200
37253792111,4sleep13725434-21ntp_states10:55:221
37106342111,2sleep20-21swapper/210:45:132
36743902111,5sleep23674461-21irqstats10:10:142
36535322110,6sleep03654749-21kthreadcore09:50:160
35884212111,5sleep13588468-21ntpq08:45:211
35370422112,4sleep23537061-21apt-get08:00:132
35165262111,6sleep03516622-21kthreadcore07:40:150
38194572101,4sleep23819507-21ntpq12:25:202
38102902101,4sleep13810546-21df12:20:111
38053172101,4sleep23805419-21perl12:15:152
38050582101,3sleep13805316-21df_inode12:15:131
37952692101,5sleep23796906-21grep12:05:172
37791862101,4sleep03779399-21hddtemp_smartct11:50:150
37475862101,3sleep23747605-21apt-get11:20:112
37304652101,5sleep03730600-21awk11:00:170
37158452101,3sleep03715864-21apt-get10:50:110
37059682100,2sleep20-21swapper/210:40:152
36988642101,4sleep03698915-21ntpq10:30:190
36606972101,4sleep13661984-21memory09:55:171
36530442101,4sleep13653248-21diskstats09:50:131
36048322101,4sleep03605049-21users09:00:250
35899632101,5sleep03590017-21/usr/sbin/munin08:50:110
35883622101,4sleep03588419-21ntpq08:45:190
35794742101,4sleep03579684-21hddtemp_smartct08:40:140
35791802101,4sleep13579199-21apt-get08:40:101
35681822101,4sleep03568201-21munin-run08:30:000
35671432101,4sleep13567195-21ntpq08:25:201
35305592101,4sleep03530613-21ntpq07:50:210
35052072101,5sleep0248-21systemd-journal07:30:000
34863422101,4sleep03486393-21ntpq07:10:200
3793200291,4sleep13793293-21ntpq12:00:181
3789619291,4sleep03789733-21cut12:00:140
3777672291,4sleep23777726-21ld11:45:212
3768474290,3sleep20-21swapper/211:40:122
3768474290,3sleep20-21swapper/211:40:112
3762872290,1sleep10-21swapper/111:33:581
3756679291,4sleep20-21swapper/211:25:212
3747753291,3sleep03747876-21diskstats11:20:140
3737426294,2sleep2421rcuc/211:10:132
3716596291,4sleep13718280-21grep10:50:161
3684374293,2sleep23684393-21apt-get10:20:122
3675744291,4sleep03677569-21cut10:10:160
3669898291,4sleep23671627-21cut10:05:162
3648025291,4sleep23648048-21/usr/sbin/munin09:45:122
3636095291,4sleep13636153-21ntpq09:30:201
3609626291,3sleep23609673-21ntpq09:05:212
3577884291,4sleep23577930-21ld08:35:192
3552720291,3sleep00-21swapper/008:15:130
3549191291,4sleep03550998-21cut08:10:170
3546265291,3sleep13546311-21ntpq08:05:201
3521362290,5sleep23521581-21sh07:45:142
3509526291,4sleep23509618-21proc_pri07:30:222
3505471291,2sleep00-21swapper/007:30:130
3487491291,4sleep03488352-21cat07:15:020
3820195280,1sleep00-21swapper/012:27:020
3815996281,3sleep13816237-21kthreadcore12:25:151
3815209280,1sleep20-21swapper/212:24:572
3814179281,3sleep03814230-21ntpq12:20:180
3801457281,3sleep13801847-21kthreadcore12:10:171
3789337280,2sleep20-21swapper/212:00:112
3784804281,3sleep13786472-21kthreadcore11:55:181
3784104280,3sleep03784123-21apt-get11:55:120
3742900281,3sleep00-21swapper/011:15:160
3738661281,3sleep13740426-21cut11:10:161
3703102280,4sleep20-21swapper/210:35:182
3689299280,2sleep03482176-21cyclictest10:24:260
3684090281,3sleep23684111-21munin-run10:19:592
3642833281,3sleep13643046-21hddtemp_smartct09:40:151
3637601281,3sleep13637815-21smartctl09:35:131
3632029280,3sleep23631541-21apt-get09:30:012
3626120280,1sleep20-21swapper/209:22:562
3620187280,1sleep20-21swapper/209:15:232
3616962281,3sleep03618642-21cut09:15:160
3615691280,2sleep20-21swapper/209:12:502
3610896281,2sleep00-21swapper/009:10:110
3608753281,3sleep03609520-21missed_timers09:05:190
3604376280,1sleep10-21swapper/109:00:181
3600023280,1sleep10-21swapper/108:58:151
3579784281,3sleep23581237-21kthreadcore08:40:162
3556822281,3sleep23556912-21ps08:15:232
3520335281,3sleep23520498-21/usr/sbin/munin07:40:242
3516629281,3sleep13518279-21kthreadcore07:40:151
3494976280,1sleep20-21swapper/207:20:122
3494954281,2sleep13494973-21apt-get07:20:111
3486401281,3sleep13486450-21ntpq07:10:211
3784101270,1sleep20-21swapper/211:55:112
3778895270,2sleep10-21swapper/111:50:121
3770348271,2sleep10-21swapper/111:40:191
3770348271,2sleep10-21swapper/111:40:181
3757985271,2sleep03758004-21apt-get11:30:120
3751617270,1sleep10-21swapper/111:20:221
3747263270,1sleep20-21swapper/211:19:322
3737707271,2sleep00-21swapper/011:10:150
3727364271,2sleep20-21swapper/211:00:152
3693568271,2sleep23693621-21ntpq10:25:202
3683396271,3sleep13683560-21timerwakeupswit10:15:251
3679125271,3sleep03679302-21expr10:15:120
3678629270,1sleep10-21swapper/110:12:221
3673386270,1sleep10-21swapper/110:06:561
3644620271,2sleep00-21swapper/009:40:180
3642542271,2sleep23642561-21apt-get09:40:112
3642248270,4sleep20-21swapper/209:39:562
3631519270,1sleep20-21swapper/209:29:582
3626167270,1sleep10-21swapper/109:23:311
3606068271,3sleep10-21swapper/109:05:161
3591669271,2sleep10-21swapper/108:50:161
3590234271,3sleep23590405-21cut08:50:132
3573330270,1sleep10-21swapper/108:34:131
3547211271,1sleep2171rcu_preempt08:09:542
3541925270,4sleep10-21swapper/108:03:561
3535852270,3sleep00-21swapper/007:55:210
3526114270,1sleep10-21swapper/107:48:041
3521285270,1sleep00-21swapper/007:45:030
3511982271,2sleep00-21swapper/007:35:170
3505030270,1sleep20-21swapper/207:27:502
3486688271,2sleep20-21swapper/207:10:262
3826491261,2sleep10-21swapper/112:35:131
3800370261,2sleep00-21swapper/012:10:160
3799013261,2sleep00-21swapper/012:05:260
3779477261,2sleep20-21swapper/211:50:172
3778297260,1sleep00-21swapper/011:45:470
3773401260,1sleep00-21swapper/011:44:470
3773401260,1sleep00-21swapper/011:44:460
3762732260,1sleep20-21swapper/211:32:092
3756284260,2sleep13756672-21strings11:25:191
3754483261,2sleep00-21swapper/011:25:180
3721882261,2sleep00-21swapper/010:55:170
3720516260,1sleep20-21swapper/210:51:492
3710083260,4sleep10-21swapper/110:41:381
3701364261,2sleep10-21swapper/110:35:181
3693677260,3sleep00-21swapper/010:25:230
3672686261,1sleep00-21swapper/010:05:220
3668341260,1sleep00-21swapper/010:04:190
3656548260,2sleep23656858-21strings09:50:182
3647750260,2sleep00-21swapper/009:45:100
3641963260,4sleep00-21swapper/009:36:170
3616119260,4sleep10-21swapper/109:15:061
3611960261,2sleep10-21swapper/109:10:161
3573202260,4sleep20-21swapper/208:32:392
3562644260,1sleep00-21swapper/008:21:100
3552148260,1sleep20-21swapper/208:11:072
3542839261,2sleep00-21swapper/008:05:150
3541715260,1sleep00-21swapper/008:01:150
3515425260,4sleep10-21swapper/107:37:341
3494311260,1sleep20-21swapper/207:15:322
3805590250,2sleep00-21swapper/012:15:160
3767854250,3sleep20-21swapper/211:35:462
3763241250,1sleep10-21swapper/111:35:131
3742400250,1sleep10-21swapper/111:15:141
3714425250,2sleep00-21swapper/010:45:190
3689234250,1sleep10-21swapper/110:23:401
3662648250,1sleep20-21swapper/209:55:262
3621338250,3sleep00-21swapper/009:20:100
3600100250,3sleep20-21swapper/208:59:122
3578209251,1sleep00-21swapper/008:35:260
3532133250,1sleep10-21swapper/107:55:141
3526806250,1sleep10-21swapper/107:50:141
3504216250,1sleep10-21swapper/107:25:211
3498816250,1sleep00-21swapper/007:20:180
3488353250,1sleep10-21swapper/107:15:031
3825666240,1sleep00-21swapper/012:35:000
3732184240,1sleep00-21swapper/011:05:150
3705687240,1sleep00-21swapper/010:40:130
3694580240,1sleep10-21swapper/110:30:001
3663727240,1sleep10-21swapper/110:00:151
3626549240,1sleep10-21swapper/109:25:121
3584176240,1sleep20-21swapper/208:45:002
3562993240,1sleep20-21swapper/208:25:002
3547344240,1sleep10-21swapper/108:10:021
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional