You are here: Home / Technical Services / OSADL QA Farm Real-time / 
2025-12-28 - 05:20

OSADL QA Farm on Real-time of Mainline Linux

About - Hardware - CPUs - Benchmarks - Graphics - Benchmarks - Kernels - Boards/Distros - Latency monitoring - Latency plots - System data - Profiles - Compare - Awards

Default latency plot of shadow in rack #3, slot #3

Rack #0/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #1/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #2/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #3/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #4/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #5/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #6/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #7/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #8/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #9/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #a/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #b/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #c/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #d/#0 #1 #2 #3 #4 #5 #6 #7 #8 -
Rack #e/#0 #1 #2 #3 #4 #5 #6 #7 #8 - Rack #f/#0 #1 #2 #3 #4 #5 #6 #7 #8 
Special  All - All RT - Optimization - Ethernet - Thumbnails - Next
  TQ-Systems
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack3slot3s.osadl.org (updated Sun Dec 28, 2025 00:43:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
3414940212762,60sleep30-21swapper/319:09:353
341478829058,10sleep20-21swapper/219:07:372
341470528470,9sleep10-21swapper/119:06:361
341474827159,6sleep00-21swapper/019:07:070
3415086992421,1cyclictest0-21swapper/320:10:143
3415086992421,1cyclictest0-21swapper/319:40:153
3415086992320,1cyclictest0-21swapper/320:15:153
3415086992320,1cyclictest0-21swapper/320:15:143
3415086992320,1cyclictest0-21swapper/300:15:153
341508699231,18cyclictest3548486-21chrt21:13:083
341508699220,18cyclictest3458719-21chrt19:46:293
341508699217,7cyclictest3737782-21taskset00:13:203
3415086992117,2cyclictest0-21swapper/322:15:143
3415086992018,1cyclictest0-21swapper/319:20:233
3415086992017,1cyclictest0-21swapper/321:40:253
3415086992017,1cyclictest0-21swapper/320:00:213
3415086992016,1cyclictest0-21swapper/321:59:283
35334712191,9sleep23534690-21cut21:00:172
3415086991916,1cyclictest0-21swapper/322:25:193
3415086991916,1cyclictest0-21swapper/321:25:173
3415086991916,1cyclictest0-21swapper/300:25:193
3415086991915,2cyclictest0-21swapper/322:09:593
341508699191,14cyclictest0-21swapper/321:20:153
341508699191,14cyclictest0-21swapper/321:20:143
341508699191,14cyclictest0-21swapper/319:55:123
341508699191,14cyclictest0-21swapper/300:00:123
341508699191,13cyclictest0-21swapper/323:00:103
341508699191,13cyclictest0-21swapper/319:35:113
341508699182,10cyclictest3658515-21taskset22:56:443
3415086991816,1cyclictest0-21swapper/323:20:163
3415086991815,1cyclictest0-21swapper/323:45:203
3415086991815,1cyclictest0-21swapper/322:45:203
3415086991815,1cyclictest0-21swapper/300:20:153
3415086991814,2cyclictest0-21swapper/319:20:013
3415086991814,2cyclictest0-21swapper/319:20:003
3415086991814,2cyclictest0-21swapper/300:35:173
341508699181,14cyclictest0-21swapper/320:35:253
341508699181,13cyclictest0-21swapper/322:30:123
341508699177,6cyclictest3532397-21taskset20:57:163
3415086991714,1cyclictest0-21swapper/320:25:193
3415086991714,1cyclictest0-21swapper/319:50:163
341508699171,3cyclictest0-21swapper/323:15:123
3415086991713,2cyclictest0-21swapper/323:30:163
341508699171,12cyclictest3637779-21turbostat22:40:003
341508699171,12cyclictest0-21swapper/321:50:143
341508699171,12cyclictest0-21swapper/321:35:153
341508699171,12cyclictest0-21swapper/321:15:113
341508699171,12cyclictest0-21swapper/319:10:193
341508699170,10cyclictest3685679-21chrt23:25:073
37265302161,5sleep23726579-21ntpq00:00:212
341508699162,9cyclictest3492976-21taskset20:20:163
341508699161,9cyclictest3727746-21chrt00:05:073
341508699161,9cyclictest3648530-21taskset22:50:083
341508699161,9cyclictest3638025-21taskset22:40:023
341508699161,9cyclictest3517445-21taskset20:45:133
341508699161,9cyclictest3501441-21taskset20:30:063
3415086991614,1cyclictest0-21swapper/321:05:173
3415086991613,1cyclictest0-21swapper/321:00:173
3415086991612,2cyclictest0-21swapper/322:20:203
341508699161,13cyclictest0-21swapper/322:00:143
341508699161,12cyclictest3527435-21sleep20:55:003
341508699160,13cyclictest0-21swapper/323:55:183
341508699160,13cyclictest0-21swapper/323:50:183
341508699160,13cyclictest0-21swapper/323:40:113
341508699160,13cyclictest0-21swapper/321:30:123
341508699160,13cyclictest0-21swapper/320:40:143
341508699160,13cyclictest0-21swapper/300:30:213
341508699160,12cyclictest3479946-21turbostat20:09:593
341508699160,12cyclictest0-21swapper/323:10:173
341508699160,10cyclictest3700878-21taskset23:36:333
36567872152,8sleep03657642-21sed22:55:190
35842052151,10sleep00-21swapper/021:45:200
3415086991512,1cyclictest0-21swapper/321:45:163
3415086991512,1cyclictest0-21swapper/319:30:013
3415086991511,2cyclictest0-21swapper/322:10:203
341508699150,12cyclictest0-21swapper/319:30:213
37015022141,9sleep23701720-21df_inode23:40:132
35801392141,9sleep13580367-21sed21:45:011
341508699140,12cyclictest0-21swapper/323:05:113
35436142131,8sleep23543615-21/usr/sbin/munin21:10:112
35292262131,8sleep13530959-21kthreadcore20:55:181
35238482131,7sleep13525547-21kthreadcore20:50:181
34234512131,7sleep13425087-21kthreadcore19:15:181
34234512131,7sleep13425087-21kthreadcore19:15:181
37316082121,5sleep03731689-21ntpq00:05:200
36386062120,8sleep23639437-21cut22:40:182
36220972121,7sleep03622139-21systemd22:25:000
36071842120,7sleep03608591-21grep22:10:180
35491762121,7sleep03549274-21fschecks_time21:15:130
35009662121,7sleep03500967-21sshd20:27:160
34755352121,7sleep03476777-21tr20:05:190
34484962121,4sleep23448501-21cron19:40:002
34385462121,7sleep23438634-21fschecks_time19:30:142
37431062111,6sleep03743107-21sshd00:19:270
36106382111,4sleep13610691-21ntp_states22:10:231
35948742111,5sleep13594928-21ntpq21:55:231
35704752111,6sleep03570510-21kthreadcore21:35:170
35648422111,5sleep13564966-21diskstats21:30:131
35072432111,5sleep03508640-21kthreadcore20:35:170
34949552111,5sleep23494982-21ntpq20:20:202
34803552111,5sleep23480484-21diskstats20:10:152
34702692110,7sleep13471031-21kthreadcore20:00:201
34527942111,5sleep03452840-21ntpq19:40:230
37594432101,4sleep03759546-21perl00:35:140
37526622101,4sleep13752704-21ntpq00:25:211
37224732101,4sleep13722491-21apt-get00:00:131
37158592101,4sleep13715910-21ntpq23:50:181
37106192101,4sleep23710657-21ntpq23:45:232
36959442100,5sleep244-21ksoftirqd/223:35:002
36791842101,4sleep03679240-21ntp_states23:15:230
36577572101,5sleep23657803-21ntpq22:55:202
36419072101,4sleep13642020-21ntpq22:40:201
36239882101,5sleep23625718-21cut22:25:182
36054002101,4sleep23605454-21ntpq22:05:222
35863282100,6sleep23587932-21kthreadcore21:50:192
35701942101,5sleep23570391-21grep21:35:152
35378472101,5sleep03537870-21munin-run21:05:000
35224272100,1sleep00-21swapper/020:50:090
34672012101,4sleep13468259-21memory19:55:181
34580272101,4sleep03458068-21ntpq19:45:230
34475472101,4sleep03447601-21ntpq19:35:210
34437892101,4sleep13443999-21hddtemp_smartct19:35:141
3754403291,4sleep23754474-21irqstats00:30:162
3738756291,4sleep13740198-21grep00:15:151
3722156291,4sleep13722222-21sendmail-msp00:00:001
3721466291,4sleep03721539-21awk23:55:260
3721130291,4sleep23721168-21ntpq23:55:202
3715912291,5sleep23715963-21ld23:50:192
3701042290,1sleep20-21swapper/223:38:382
3691791291,4sleep03693430-21tr23:30:170
3681115291,4sleep13682732-21kthreadcore23:20:161
3669825291,3sleep13669844-21apt-get23:10:101
3659076291,4sleep03659305-21expr23:00:110
3645902291,3sleep03647112-21memory22:45:170
3641154291,4sleep03641906-21wc22:40:190
3612120291,4sleep03612127-21entropy22:15:140
3611834291,3sleep13611853-21apt-get22:15:111
3601122291,4sleep03601171-21/usr/sbin/munin22:05:010
3591448291,5sleep03592803-21kthreadcore21:55:180
3590029291,4sleep03590249-21users21:50:300
3580400291,3sleep13580419-21apt-get21:45:131
3565250291,4sleep03566588-21cut21:30:170
3563421291,4sleep23563472-21/usr/sbin/munin21:25:222
3563397291,4sleep13563420-21gcc21:25:201
3554483291,4sleep23554530-21fschecks_count21:20:172
3554483291,4sleep23554530-21fschecks_count21:20:162
3542330290,1sleep00-21swapper/021:05:190
3538660291,4sleep23538689-21fschecks_count21:05:132
3533400291,4sleep13533470-21irqstats21:00:161
3508641291,4sleep13510420-21grep20:35:181
3501451291,3sleep23501470-21apt-get20:30:142
3489713291,3sleep23489739-21ntpq20:15:202
3489713291,3sleep23489739-21ntpq20:15:192
3489572291,4sleep13489711-21netstat20:15:181
3489572291,4sleep13489711-21netstat20:15:181
3480767291,4sleep13481979-21grep20:10:171
3476778291,4sleep13478484-21cut20:05:201
3459539291,4sleep23459640-21tune2fs19:50:152
3453445290,1sleep10-21swapper/119:40:481
3438287290,1sleep10-21swapper/119:30:141
3433593291,4sleep03434486-21grep19:25:150
3425090291,3sleep23426360-21memory19:15:192
3425090291,3sleep23426360-21memory19:15:192
3415565291,4sleep03415776-21hddtemp_smartct19:10:160
3415271291,3sleep13415290-21apt-get19:10:131
3759716281,3sleep23760395-21kthreadcore00:35:162
3758007281,3sleep13758092-21ps00:30:231
3752953280,4sleep23753056-21sshd00:25:262
3748647281,2sleep00-21swapper/000:25:120
3747731281,3sleep23747988-21unixbench_singl00:20:262
3737567280,1sleep20-21swapper/200:10:362
3716310281,4sleep03716527-21users23:50:260
3696713281,3sleep13696810-21ps23:35:151
3685688280,3sleep20-21swapper/223:25:152
3680411281,2sleep00-21swapper/023:20:110
3679505281,3sleep13675114-21/usr/sbin/munin23:15:281
3679080281,2sleep23679128-21ntpq23:15:212
3666020281,3sleep13667751-21kthreadcore23:05:181
3664322280,1sleep00-21swapper/023:05:130
3654193280,1sleep10-21swapper/122:55:161
3653345280,1sleep10-21swapper/122:52:461
3636856281,4sleep03636918-21pmu-power22:35:230
3633377281,3sleep23634249-21kthreadcore22:35:162
3627139281,3sleep13627140-21sshd22:27:411
3590506280,1sleep10-21swapper/121:53:391
3554205281,3sleep03554224-21apt-get21:20:150
3554205281,3sleep03554224-21apt-get21:20:140
3552757281,3sleep13552898-21ntpq21:15:181
3543639281,3sleep03543897-21df_inode21:10:120
3539232280,4sleep13540867-21cut21:05:161
3532538280,1sleep00-21swapper/020:59:020
3516706284,2sleep0800-21snmpd20:42:080
3495949281,3sleep10-21swapper/120:25:001
3490045280,2sleep03490117-21find20:15:260
3490045280,2sleep03490117-21find20:15:250
3484936280,1sleep00-21swapper/020:11:480
3479042281,3sleep23479126-21proc_pri20:05:252
3437281280,1sleep10-21swapper/119:25:241
3420367281,4sleep03421195-21tr19:15:020
3420367281,4sleep03421195-21tr19:15:010
3729322271,2sleep20-21swapper/200:05:182
3728323271,2sleep10-21swapper/100:05:171
3706664270,1sleep00-21swapper/023:45:080
3702003272,2sleep13702864-21grep23:40:161
3695858270,1sleep10-21swapper/123:33:551
3685165270,1sleep20-21swapper/223:21:452
3670930270,3sleep23672613-21grep23:10:152
3667753271,3sleep23668194-21needreboot23:05:192
3663456271,3sleep23663667-21users23:00:252
3663347271,3sleep10-21swapper/123:00:241
3653208270,1sleep00-21swapper/022:51:010
3648826271,3sleep20-21swapper/222:50:152
3647113271,3sleep13643283-21/usr/sbin/munin22:45:171
3643282270,2sleep20-21swapper/222:45:102
3632550271,3sleep23632570-21turbostat22:35:012
3632352270,1sleep00-21swapper/022:32:290
3622357270,2sleep00-21swapper/022:25:120
3611463270,1sleep20-21swapper/222:13:362
3595150271,3sleep20-21swapper/221:55:282
3585065270,1sleep20-21swapper/221:46:142
3555175271,3sleep13556817-21grep21:20:191
3555175271,3sleep13556817-21grep21:20:191
3549445271,2sleep20-21swapper/221:15:152
3521941270,1sleep00-21swapper/020:47:080
3517218270,3sleep23517444-21expr20:45:132
3517196270,2sleep10-21swapper/120:45:121
3511940270,2sleep10-21swapper/120:40:121
3500377270,3sleep23500463-21pool-udisksd20:25:252
3495729270,3sleep03495730-21sshd20:22:160
3459258270,2sleep00-21swapper/019:50:130
3458922270,1sleep10-21swapper/119:49:051
3453773270,3sleep20-21swapper/219:44:572
3442986270,1sleep00-21swapper/019:31:490
3764134260,3sleep13764378-21latency_hist00:40:001
3755729260,2sleep00-21swapper/000:30:180
3748242260,1sleep00-21swapper/000:23:130
3743068260,1sleep20-21swapper/200:18:572
3733630261,2sleep03735255-21kthreadcore00:10:180
3727145260,4sleep00-21swapper/000:00:360
3710769260,2sleep13710886-21pool-udisksd23:45:251
3690347260,4sleep10-21swapper/123:25:461
3618122261,2sleep13619175-21kthreadcore22:20:211
3616505260,1sleep20-21swapper/222:15:402
3601953260,2sleep10-21swapper/122:05:171
3601029260,1sleep20-21swapper/222:03:502
3600874260,1sleep10-21swapper/122:01:531
3580030260,1sleep20-21swapper/221:43:372
3575173260,1sleep00-21swapper/021:40:110
3574202260,2sleep10-21swapper/121:35:271
3569435260,1sleep20-21swapper/221:32:272
3564382260,1sleep00-21swapper/021:29:360
3548489260,1sleep10-21swapper/121:13:111
3527954261,2sleep20-21swapper/220:55:152
3525548261,2sleep20-21swapper/220:50:192
3512913260,3sleep2421rcuc/220:40:162
3510760261,2sleep244-21ksoftirqd/220:35:232
3506223260,1sleep10-21swapper/120:32:471
3506051260,1sleep00-21swapper/020:30:370
3474496260,1sleep00-21swapper/020:02:260
3471035261,2sleep20-21swapper/220:00:212
3460267261,2sleep10-21swapper/119:50:181
3458772260,1sleep20-21swapper/219:47:092
3432676260,4sleep10-21swapper/119:23:521
3427830261,2sleep00-21swapper/019:20:150
3420304261,2sleep20-21swapper/219:14:522
3743687250,1sleep10-21swapper/100:20:131
3733229250,1sleep10-21swapper/100:10:151
3705281250,1sleep00-21swapper/023:40:200
3700562250,1sleep00-21swapper/023:35:260
3636656250,1sleep10-21swapper/122:35:191
3631326250,1sleep10-21swapper/122:30:181
3621186250,1sleep20-21swapper/222:20:262
3601837250,1sleep00-21swapper/022:05:160
3500294250,1sleep10-21swapper/120:25:241
3465551250,2sleep00-21swapper/019:55:170
3437991250,1sleep20-21swapper/219:29:542
3431640250,1sleep20-21swapper/219:20:202
3685969240,1sleep00-21swapper/023:25:160
3669724240,1sleep00-21swapper/023:10:010
3464840240,1sleep20-21swapper/219:55:152
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional