You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-12 - 12:47
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot8.osadl.org (updated Sat Jul 12, 2025 00:43:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
32118812910,1sleep10-21swapper/120:50:171
30819842360,2chrt34-21ksoftirqd/119:05:161
3086677993114,12cyclictest3324590-21idleruntime22:05:162
3086676992723,2cyclictest3561480-21systemd-journal19:25:001
3086677992617,7cyclictest3477874-21kthreadcore23:40:152
3086681992522,1cyclictest0-21swapper/321:55:003
3086677992518,6cyclictest0-21swapper/221:50:012
3086677992515,8cyclictest3453032-21irqcore23:25:152
3086677992515,8cyclictest0-21swapper/219:55:002
3086677992416,7cyclictest3524632-21/usr/sbin/munin00:10:182
3086677992416,7cyclictest3222925-21sendmail21:00:002
3086676992420,2cyclictest0-21swapper/121:55:001
3086671992421,2cyclictest3331861-21idleruntime-cro22:09:590
3086671992420,2cyclictest0-21swapper/019:15:000
3086677992320,2cyclictest761-21cups-browsed20:20:002
3086677992316,6cyclictest0-21swapper/223:20:002
3086677992314,8cyclictest3415764-21kthreadcore23:00:172
3086677992314,7cyclictest3388794-21cpuspeed_turbos22:45:132
308667799230,13cyclictest3411239-21taskset22:57:212
308667799230,12cyclictest3426621-21taskset23:05:592
308667799225,12cyclictest769-21runrttasks19:43:162
308667799221,10cyclictest3149418-21taskset19:56:152
308667799220,17cyclictest3346942-21sh22:17:352
308667799220,17cyclictest3100599-21taskset19:16:012
308667799220,12cyclictest3562748-21taskset00:31:282
308667799220,12cyclictest3490602-21taskset23:46:232
308667799220,12cyclictest3314385-21runrttasks21:56:422
308667799220,12cyclictest3210523-21taskset20:47:182
308667799220,12cyclictest3161793-21taskset20:08:172
308667799220,12cyclictest3106975-21taskset19:24:172
308667799220,11cyclictest769-21runrttasks22:11:302
308667799220,11cyclictest3300203-21taskset21:50:062
308667799220,11cyclictest3267045-21taskset21:28:362
308667799220,11cyclictest3259484-21taskset21:24:402
3086676992219,2cyclictest0-21swapper/121:45:001
3086676992219,2cyclictest0-21swapper/119:20:001
3086676992218,2cyclictest0-21swapper/119:15:011
3086671992219,2cyclictest3086666-21cyclictest21:30:150
3086671992219,2cyclictest0-21swapper/020:55:000
3086681992118,1cyclictest0-21swapper/323:00:003
3086681992118,1cyclictest0-21swapper/300:04:593
3086677992111,8cyclictest3245408-21kthreadcore21:15:182
308667799210,16cyclictest3498549-21taskset23:51:132
308667799210,16cyclictest3466771-21sh23:31:472
308667799210,12cyclictest769-21runrttasks22:29:132
308667799210,12cyclictest3236080-21diskmemload21:10:152
308667799210,11cyclictest3571630-21taskset00:38:192
308667799210,11cyclictest3571630-21taskset00:38:182
308667799210,11cyclictest3555740-21sh00:28:392
308667799210,11cyclictest3523798-21taskset00:08:552
308667799210,11cyclictest3449758-21awk23:20:222
308667799210,11cyclictest3434641-21taskset23:11:072
308667799210,11cyclictest3403263-21seq22:52:222
308667799210,11cyclictest3387929-21runrttasks22:44:122
308667799210,11cyclictest3290012-21taskset21:40:492
308667799210,11cyclictest3273776-21taskset21:30:382
308667799210,11cyclictest3235334-21chrt21:09:132
308667799210,11cyclictest3137031-21seq19:45:282
308667799210,11cyclictest3092461-21taskset19:11:382
3086676992118,2cyclictest3468230-21systemd-run23:35:001
3086671992118,2cyclictest3315961-21sendmail22:00:000
3086671992118,2cyclictest3279082-21grep21:35:210
3086671992114,6cyclictest3385740-21ntp_kernel_pll_22:40:230
3086671992113,6cyclictest3561480-21systemd-journal00:35:000
3086681992017,1cyclictest0-21swapper/321:15:243
308667799204,7cyclictest769-21runrttasks22:30:542
3086677992017,2cyclictest1-21systemd22:05:002
308667799200,12cyclictest3235405-21diskmemload00:02:152
308667799200,11cyclictest3204433-21taskset20:42:082
308667799200,11cyclictest3192105-21taskset20:30:432
308667799200,11cyclictest3180187-21taskset20:24:172
308667799200,11cyclictest3125201-21taskset19:38:402
308667799200,10cyclictest769-21runrttasks23:59:102
308667799200,10cyclictest3540939-21sh00:20:142
308667799200,10cyclictest3380109-21diskmemload22:39:342
308667799200,10cyclictest3277481-21chrt21:35:202
308667799200,10cyclictest3167972-21taskset20:14:212
308667799200,10cyclictest1-21systemd00:20:002
3086676992017,2cyclictest3428759-21/usr/sbin/munin23:10:251
3086676992017,2cyclictest3297762-21sed21:45:251
3086676992017,2cyclictest3283859-21idleruntime-cro21:40:011
3086671992017,2cyclictest3210822-21latency_hist20:50:000
3086671992017,2cyclictest3137563-21systemd-run19:50:010
3086681991916,1cyclictest0-21swapper/300:25:003
3086681991915,2cyclictest3204680-21turbostat20:45:003
3086677991916,2cyclictest0-21swapper/220:40:002
3086677991912,6cyclictest3114692-21kthreadcore19:30:162
3086677991911,5cyclictest0-21swapper/222:20:142
308667799190,9cyclictest769-21runrttasks23:39:422
308667799190,9cyclictest3229039-21taskset21:01:442
308667799190,11cyclictest3155820-21seq20:04:542
308667799190,10cyclictest3107277-21seq19:25:032
308667699198,9cyclictest3561480-21systemd-journal22:14:541
3086676991916,2cyclictest3572752-21sendmail00:40:001
3086676991916,2cyclictest3572752-21sendmail00:40:001
3086676991916,2cyclictest3186363-21/usr/sbin/munin20:30:001
308667199199,7cyclictest1-21systemd23:59:590
3086671991915,3cyclictest514-21systemd-logind20:40:000
3086671991915,3cyclictest3404932-21cpuspeed_turbos22:55:140
3086681991815,1cyclictest0-21swapper/322:40:253
3086681991815,1cyclictest0-21swapper/300:40:013
3086681991815,1cyclictest0-21swapper/300:40:003
3086681991815,1cyclictest0-21swapper/300:00:013
308667799189,8cyclictest3216433-21unixbench_singl20:50:252
3086676991815,2cyclictest3513473-21ssh00:00:201
3086676991815,2cyclictest3452322-21systemd-run23:25:001
3086676991815,2cyclictest0-21swapper/122:10:001
3086671991815,2cyclictest3421211-21irqcore23:05:160
3086671991814,2cyclictest3373994-21kthreadcore22:35:180
3086671991813,3cyclictest514-21systemd-logind22:50:000
3086671991811,6cyclictest64450irq/159-eno2:rx-021:10:110
308667199181,15cyclictest3323922-21turbostat22:04:590
3086681991714,1cyclictest0-21swapper/320:40:003
3086681991714,1cyclictest0-21swapper/319:55:003
3086676991714,2cyclictest0-21swapper/122:20:001
3086676991714,2cyclictest0-21swapper/100:19:591
3086676991713,3cyclictest522-21in:imuxsock21:25:001
3086676991713,3cyclictest3372900-21fschecks_time22:35:151
3086676991713,3cyclictest3112225-21ntp_kernel_pll_19:25:211
3086671991714,2cyclictest3267863-21latency_hist21:30:020
3086671991714,2cyclictest0-21swapper/023:40:000
3086671991714,2cyclictest0-21swapper/020:00:000
3086671991713,3cyclictest3481534-21ntp_kernel_err23:40:190
3086671991713,2cyclictest0-21swapper/019:35:000
3086671991713,2cyclictest0-21swapper/000:30:000
3086681991613,1cyclictest0-21swapper/323:30:003
3086681991613,1cyclictest0-21swapper/323:00:193
3086681991613,1cyclictest0-21swapper/322:45:153
3086681991613,1cyclictest0-21swapper/321:40:003
3086681991613,1cyclictest0-21swapper/320:05:003
3086681991613,1cyclictest0-21swapper/300:18:423
308667799168,7cyclictest3180730-21grep20:25:142
3086676991613,2cyclictest3143225-21grep19:50:251
3086676991613,1cyclictest0-21swapper/123:00:171
3086676991612,3cyclictest3505030-21kthreadcore23:55:191
3086676991611,3cyclictest1-21systemd22:03:241
308667699160,5cyclictest3308721-21irqcore21:55:201
3086671991613,2cyclictest3444351-21sendmail23:20:010
3086671991613,2cyclictest3401817-21needreboot22:50:220
3086671991613,2cyclictest3284320-21tr21:40:130
3086671991613,2cyclictest1343558-21kworker/u8:1+events_unbound19:30:010
3086671991612,3cyclictest3340365-21/usr/sbin/munin22:15:210
3086671991612,2cyclictest3173255-21ntp_kernel_pll_20:15:200
3086681991513,1cyclictest0-21swapper/322:10:003
3086681991512,1cyclictest0-21swapper/323:45:203
3086681991512,1cyclictest0-21swapper/323:15:123
3086681991512,1cyclictest0-21swapper/322:25:123
3086681991512,1cyclictest0-21swapper/321:57:383
3086681991512,1cyclictest0-21swapper/321:00:163
3086681991512,1cyclictest0-21swapper/320:50:033
3086681991512,1cyclictest0-21swapper/320:20:253
3086681991512,1cyclictest0-21swapper/319:30:153
3086681991512,1cyclictest0-21swapper/319:20:183
3086681991512,1cyclictest0-21swapper/319:15:013
3086681991511,1cyclictest0-21swapper/320:12:033
308667699159,2cyclictest191rcu_preempt23:35:291
3086676991512,2cyclictest3466429-21kworker/u8:0-rpciod00:35:011
3086676991512,2cyclictest3244256-21sed21:15:171
3086676991512,2cyclictest3235530-21latency_hist21:10:001
3086676991512,2cyclictest3004-21cron20:35:011
3086676991511,3cyclictest1-21systemd20:42:251
3086676991510,3cyclictest3426811-21sshd23:06:241
308667699150,3cyclictest3210727-21turbostat.cron20:49:551
308667199157,6cyclictest3561480-21systemd-journal23:35:000
3086671991512,2cyclictest3568823-21kthreadcore00:35:190
3086671991512,2cyclictest3568823-21kthreadcore00:35:190
3086671991512,2cyclictest3494799-21kthreadcore23:50:190
3086671991512,2cyclictest3365877-21kthreadcore22:30:180
3086671991512,2cyclictest3323949-21kworker/0:2+kacpid23:20:170
3086671991512,2cyclictest3293180-21irqrtprio21:45:160
3086671991512,2cyclictest3223027-21apt-key21:00:000
3086671991511,3cyclictest3524804-21cut00:10:140
3086671991511,3cyclictest3349384-21irqrtprio22:20:170
3086671991511,3cyclictest3191947-21unixbench-2d20:30:240
3086671991511,2cyclictest0-21swapper/021:05:150
3086681991411,1cyclictest0-21swapper/323:44:243
3086681991411,1cyclictest0-21swapper/322:50:223
3086681991411,1cyclictest0-21swapper/322:30:163
3086681991411,1cyclictest0-21swapper/322:20:133
3086681991411,1cyclictest0-21swapper/322:15:253
3086681991411,1cyclictest0-21swapper/322:10:013
3086681991411,1cyclictest0-21swapper/322:00:183
3086681991411,1cyclictest0-21swapper/321:45:173
3086681991411,1cyclictest0-21swapper/320:45:143
3086681991411,1cyclictest0-21swapper/320:15:243
3086681991411,1cyclictest0-21swapper/300:30:203
308667699149,4cyclictest3174127-21cron20:20:001
308667699142,2cyclictest471rcuog/223:45:001
3086676991411,2cyclictest3410241-21timerwakeupswit22:55:251
3086676991411,2cyclictest3382824-21kthreadcore22:40:211
3086676991411,2cyclictest3372510-21latency_hist22:35:011
3086676991411,2cyclictest3240333-21kthreadcore21:10:181
3086676991411,2cyclictest3118807-21unixbench_singl19:30:231
3086676991411,2cyclictest0-21swapper/121:00:001
3086676991410,3cyclictest769-21runrttasks22:47:561
308667199148,4cyclictest3516843-21grep00:05:130
3086671991411,2cyclictest3508232-21kworker/u8:1-rpciod00:00:200
3086671991411,2cyclictest3323949-21kworker/0:2+kacpid00:15:360
3086671991411,2cyclictest3303744-21kthreadcore21:50:210
3086671991411,2cyclictest3245109-21irqrtprio21:15:180
3086671991411,2cyclictest3223565-21cpuspeed_turbos21:00:150
3086671991410,3cyclictest3330504-21kworker/u8:0-nfsiod22:25:160
3086671991410,3cyclictest3175441-21irqrtprio20:20:200
3086671991410,3cyclictest3163350-21irqrtprio20:10:160
3086671991410,3cyclictest3138791-21irqrtprio19:50:170
3086671991410,3cyclictest3096622-21kthreadcore19:15:190
308668199139,1cyclictest0-21swapper/322:35:133
3086681991311,1cyclictest0-21swapper/323:20:183
3086681991311,1cyclictest0-21swapper/321:20:233
3086681991310,1cyclictest0-21swapper/323:50:193
3086681991310,1cyclictest0-21swapper/323:35:253
3086681991310,1cyclictest0-21swapper/323:30:163
3086681991310,1cyclictest0-21swapper/323:07:563
3086681991310,1cyclictest0-21swapper/321:40:043
3086681991310,1cyclictest0-21swapper/321:25:033
3086681991310,1cyclictest0-21swapper/321:13:373
3086681991310,1cyclictest0-21swapper/321:00:003
3086681991310,1cyclictest0-21swapper/320:30:163
3086681991310,1cyclictest0-21swapper/320:25:173
3086681991310,1cyclictest0-21swapper/320:05:233
3086681991310,1cyclictest0-21swapper/319:30:003
3086681991310,1cyclictest0-21swapper/319:10:133
3086681991310,1cyclictest0-21swapper/300:26:383
3086681991310,1cyclictest0-21swapper/300:05:393
308667699139,3cyclictest3264607-21kthreadcore21:25:211
308667699139,3cyclictest3174527-21cpuspeed_turbos20:20:151
308667699139,3cyclictest3124909-21unixbench_singl19:35:251
308667699139,2cyclictest0-21swapper/100:25:171
308667699136,6cyclictest3486534-21tr23:45:161
3086676991310,2cyclictest508-21polkitd00:25:011
3086676991310,2cyclictest3495506-21kthreadcore23:50:191
3086676991310,2cyclictest3457546-21tr23:25:191
3086676991310,2cyclictest3400083-21kthreadcore22:50:201
3086676991310,2cyclictest3270926-21kthreadcore21:30:181
3086676991310,2cyclictest3165086-21kthreadcore20:10:181
3086676991310,2cyclictest3152072-21kthreadcore20:00:181
3086676991310,2cyclictest0-21swapper/122:25:161
3086676991310,1cyclictest0-21swapper/100:05:151
308667699130,11cyclictest277-21plymouthd19:50:011
308667199139,3cyclictest3122368-21kthreadcore19:35:200
308667199139,2cyclictest0-21swapper/020:40:210
308667199133,3cyclictest471rcuog/221:20:160
308667199132,10cyclictest3435300-21ssh23:12:370
308667199131,3cyclictest3417545-21kthreadcore23:00:180
3086671991311,1cyclictest3323949-21kworker/0:2+kacpid23:28:410
3086671991311,1cyclictest3323949-21kworker/0:2+kacpid22:10:470
3086671991311,1cyclictest3074309-21kworker/0:1+kacpid20:05:300
3086671991311,1cyclictest3074309-21kworker/0:1+kacpid19:41:490
3086671991311,1cyclictest3074309-21kworker/0:1+kacpid19:20:120
3086671991310,2cyclictest3323949-21kworker/0:2+kacpid23:47:310
3086671991310,2cyclictest3074309-21kworker/0:1+kacpid20:04:380
308667199130,12cyclictest64450irq/159-eno2:rx-000:20:180
308668199129,1cyclictest0-21swapper/321:31:443
308668199129,1cyclictest0-21swapper/319:59:043
308668199129,1cyclictest0-21swapper/319:49:243
308668199129,1cyclictest0-21swapper/319:40:243
308668199129,1cyclictest0-21swapper/300:10:183
3086681991210,1cyclictest0-21swapper/323:10:253
308667799120,10cyclictest3086705-21latency_hist19:10:002
308667699129,2cyclictest3527895-21kthreadcore00:10:191
308667699129,2cyclictest3235405-21diskmemload22:20:181
308667699129,2cyclictest0-21swapper/123:20:011
308667699121,1cyclictest31289021kthreadcore19:40:221
308667199129,2cyclictest3180771-21fschecks_time20:25:140
308668199111,2cyclictest31194632sleep319:35:133
308667699118,2cyclictest3227025-21kthreadcore21:00:201
308667699118,2cyclictest0-21swapper/120:05:191
308667699118,2cyclictest0-21swapper/119:55:171
308667699117,3cyclictest3194870-21kthreadcore20:35:181
308667199119,1cyclictest3086808-21latency_hist19:10:010
308668199108,1cyclictest0-21swapper/321:05:183
3086551280,2sleep30-21swapper/319:09:563
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional