You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-09 - 05:45
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rackeslot8.osadl.org (updated Fri May 09, 2025 00:43:55)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
382096821050,2sleep00-21swapper/023:45:280
37345962350,1sleep00-21swapper/022:55:170
3410923992925,3cyclictest3474730-21sh20:00:012
3410921992825,2cyclictest3871939-21cron00:20:011
3410926992614,1cyclictest0-21swapper/322:45:213
3410926992413,1cyclictest0-21swapper/322:10:153
3410926992219,1cyclictest0-21swapper/300:05:003
3410926992118,1cyclictest0-21swapper/320:20:013
3410917992118,2cyclictest3577983-21idleruntime-cro21:20:010
3410917992114,5cyclictest0-21swapper/022:50:000
3410926992017,1cyclictest0-21swapper/322:05:003
3410926992017,1cyclictest0-21swapper/319:30:003
3410926992015,2cyclictest3757661-21sleep23:10:003
3410923992017,2cyclictest0-21swapper/223:15:012
3410921992017,2cyclictest0-21swapper/120:40:001
3410921992016,2cyclictest0-21swapper/122:00:001
3410917992017,2cyclictest0-21swapper/022:20:000
3410917992012,7cyclictest3681940-21grep22:20:190
34237572190,1chrt0-21swapper/319:15:193
3410923991916,2cyclictest0-21swapper/221:35:002
3410923991915,2cyclictest769-21runrttasks21:52:212
3410923991915,2cyclictest0-21swapper/223:10:002
3410921991915,3cyclictest3474796-21systemctl20:00:011
3410921991915,2cyclictest0-21swapper/122:45:011
3410921991915,2cyclictest0-21swapper/100:00:001
3410917991916,2cyclictest761-21cups-browsed23:25:000
3410917991916,2cyclictest500-21avahi-daemon19:50:010
3410917991916,2cyclictest3636687-21irqrtprio21:55:150
3410926991815,1cyclictest0-21swapper/300:30:013
3410923991815,2cyclictest3855715-21systemd-run00:10:012
3410923991815,2cyclictest3700978-21latency_hist22:35:002
3410923991815,2cyclictest3644029-21cpuspeed_turbos22:00:152
3410923991815,2cyclictest0-21swapper/222:20:012
3410923991814,3cyclictest3717120-21ls22:45:012
3410923991813,3cyclictest3620758-21kthreadcore21:45:192
3410921991815,2cyclictest3904770-21latency_hist00:40:001
3410921991815,2cyclictest3823073-21cron23:50:001
3410921991815,2cyclictest3651892-21apt-config22:05:001
3410921991815,2cyclictest3493566-21latency_hist20:15:001
3410921991815,2cyclictest0-21swapper/119:20:001
3410921991814,3cyclictest3853287-21timerandwakeup00:05:241
3410921991814,2cyclictest0-21swapper/123:45:001
3410921991814,2cyclictest0-21swapper/123:30:251
3410921991814,2cyclictest0-21swapper/120:30:001
3410921991813,4cyclictest3666082-21grep22:10:241
3410921991810,6cyclictest3561480-21systemd-journal21:15:011
3410917991815,2cyclictest0-21swapper/021:15:000
3410926991714,1cyclictest0-21swapper/322:30:013
3410926991714,1cyclictest0-21swapper/321:39:593
3410926991714,1cyclictest0-21swapper/320:05:003
3410926991713,2cyclictest3904647-21turbostat00:40:013
3410923991713,3cyclictest3449136-21ntp_kernel_pll_19:35:212
3410923991713,2cyclictest0-21swapper/219:45:252
3410921991714,2cyclictest761-21cups-browsed00:15:001
3410921991714,2cyclictest3847568-21latency_hist00:05:001
3410921991714,2cyclictest3417141-21latency_hist19:15:021
3410921991714,2cyclictest1-21systemd22:18:401
3410921991714,2cyclictest0-21swapper/123:40:001
3410921991713,3cyclictest3746460-21kthreadcore23:00:191
3410921991713,2cyclictest3437518-21kworker/u8:2-nfsiod23:00:001
3410921991712,3cyclictest1-21systemd20:02:401
341092199170,15cyclictest3686468-21cut22:25:181
3410917991714,2cyclictest3831536-21/usr/sbin/munin23:55:210
3410917991714,2cyclictest3518304-21systemd-run20:35:010
3410917991714,2cyclictest3474783-21systemd-run20:00:010
3410917991714,2cyclictest0-21swapper/023:20:010
3410917991714,2cyclictest0-21swapper/022:15:010
3410917991713,3cyclictest3717130-21cut22:45:010
3410917991713,3cyclictest3652657-21cut22:05:150
3410917991713,2cyclictest0-21swapper/022:05:000
3410917991713,2cyclictest0-21swapper/000:35:010
3410917991711,5cyclictest3749851-21/usr/sbin/munin23:05:160
3410926991613,1cyclictest0-21swapper/323:55:223
3410926991613,1cyclictest0-21swapper/323:20:183
3410926991613,1cyclictest0-21swapper/322:55:253
3410926991613,1cyclictest0-21swapper/322:15:443
3410926991613,1cyclictest0-21swapper/322:07:133
3410926991613,1cyclictest0-21swapper/320:52:233
3410926991613,1cyclictest0-21swapper/319:56:193
3410926991613,1cyclictest0-21swapper/319:40:163
3410926991613,1cyclictest0-21swapper/319:25:013
3410926991613,1cyclictest0-21swapper/300:20:013
3410923991613,2cyclictest3725386-21latency_hist22:50:002
3410923991613,2cyclictest3668644-21kworker/u8:4-nfsiod22:20:202
3410923991613,2cyclictest3643619-21latency_hist22:00:012
3410923991613,2cyclictest3373125-21kworker/u8:3+rpciod21:12:292
3410923991613,2cyclictest0-21swapper/223:39:592
3410923991612,3cyclictest3690127-21ntp_kernel_pll_22:25:222
3410923991612,3cyclictest3525090-21gltestperf20:40:142
3410923991612,2cyclictest769-21runrttasks21:25:222
3410923991612,2cyclictest3863780-21cut00:15:002
3410923991612,2cyclictest3576184-21kworker/u8:0-nfsiod21:35:182
3410923991612,2cyclictest0-21swapper/223:40:232
3410921991613,2cyclictest3880449-21/usr/sbin/munin00:25:141
3410921991613,2cyclictest3583599-21proc_pri21:20:231
3410921991613,2cyclictest3554665-21systemctl21:00:241
3410921991613,2cyclictest3438291-21cut19:30:151
3410921991613,2cyclictest3437041-21threads19:25:241
3410921991613,2cyclictest277-21plymouthd21:10:001
3410921991612,3cyclictest3500217-21fschecks_time20:20:111
3410921991610,4cyclictest3549225-21ls21:00:001
3410917991613,2cyclictest3742652-21irqcore23:00:150
3410917991613,2cyclictest3690757-21runrttasks22:25:380
3410917991613,2cyclictest3628416-21irqcore21:50:190
3410917991613,2cyclictest3619440-21cut21:45:140
3410917991613,2cyclictest3584335-21runrttasks21:21:050
3410917991612,3cyclictest522-21in:imuxsock21:30:010
3410926991513,1cyclictest0-21swapper/323:50:243
3410926991513,1cyclictest0-21swapper/321:40:153
3410926991513,1cyclictest0-21swapper/321:04:423
3410926991513,1cyclictest0-21swapper/320:55:133
3410926991512,1cyclictest0-21swapper/323:45:463
3410926991512,1cyclictest0-21swapper/323:41:163
3410926991512,1cyclictest0-21swapper/323:30:123
3410926991512,1cyclictest0-21swapper/323:15:123
3410926991512,1cyclictest0-21swapper/323:10:353
3410926991512,1cyclictest0-21swapper/323:00:423
3410926991512,1cyclictest0-21swapper/322:50:283
3410926991512,1cyclictest0-21swapper/322:35:163
3410926991512,1cyclictest0-21swapper/321:55:433
3410926991512,1cyclictest0-21swapper/321:46:033
3410926991512,1cyclictest0-21swapper/321:27:063
3410926991512,1cyclictest0-21swapper/321:20:253
3410926991512,1cyclictest0-21swapper/321:16:273
3410926991512,1cyclictest0-21swapper/320:46:453
3410926991512,1cyclictest0-21swapper/320:40:193
3410926991512,1cyclictest0-21swapper/320:25:543
3410926991512,1cyclictest0-21swapper/320:10:263
3410926991512,1cyclictest0-21swapper/319:50:173
3410926991512,1cyclictest0-21swapper/319:46:483
3410926991512,1cyclictest0-21swapper/300:30:173
3410926991512,1cyclictest0-21swapper/300:10:293
3410926991512,1cyclictest0-21swapper/300:05:463
341092399157,6cyclictest3561480-21systemd-journal22:55:012
3410923991512,2cyclictest3836702-21ntp_kernel_pll_23:55:222
3410923991512,2cyclictest3733949-21gltestperf22:55:152
3410923991512,2cyclictest3583909-21unixbench-2d21:20:262
3410923991512,2cyclictest0-21swapper/219:15:012
3410923991511,3cyclictest3888284-21sh00:30:012
3410923991511,3cyclictest3792656-21kthreadcore23:30:192
3410923991511,3cyclictest3779409-21ntp_kernel_pll_23:20:262
3410923991511,3cyclictest3468576-21systemd-run19:55:002
3410923991511,2cyclictest3436899-21sendmail_mailqu19:25:232
3410923991511,2cyclictest0-21swapper/223:45:012
341092199159,4cyclictest501-21dbus-daemon23:05:231
341092199156,6cyclictest1-21systemd21:20:001
3410921991513,1cyclictest3459433-21kthreadcore19:45:201
3410921991512,2cyclictest3877403-21head00:20:201
3410921991512,2cyclictest3823764-21gltestperf23:50:131
3410921991512,2cyclictest3693469-21gltestperf22:30:151
3410921991512,2cyclictest3684657-21latency_hist22:25:001
3410921991512,2cyclictest3540066-21kthreadcore20:50:161
3410921991512,2cyclictest3517488-21processes20:30:231
3410921991512,2cyclictest3495182-21kthreadcore20:15:171
3410921991511,3cyclictest522-21in:imuxsock00:35:011
3410921991511,3cyclictest3778501-21kthreadcore23:20:231
3410921991511,3cyclictest3770822-21kthreadcore23:15:171
3410921991511,3cyclictest3462618-21/usr/sbin/munin19:50:161
3410921991511,2cyclictest0-21swapper/121:35:001
341091799153,10cyclictest3886167-21ssh00:25:270
3410917991512,2cyclictest3410912-21cyclictest21:30:140
3410917991512,2cyclictest3404524-21kworker/0:0+kacpid20:10:160
3410917991512,2cyclictest0-21swapper/020:50:130
3410917991512,2cyclictest0-21swapper/000:35:210
3410917991511,3cyclictest3852993-21ntp_kernel_pll_00:05:200
3410917991511,3cyclictest3814614-21sh23:44:590
3410917991511,3cyclictest3727648-21kthreadcore22:50:190
3410917991511,3cyclictest3694008-21irqcore22:30:160
3410917991511,3cyclictest3453259-21kthreadcore19:40:170
3410917991511,2cyclictest3438306-21tail19:30:150
3410917991510,3cyclictest3615307-21kthreadcore21:40:180
3410926991412,1cyclictest0-21swapper/323:35:053
3410926991412,1cyclictest0-21swapper/323:27:343
3410926991412,1cyclictest0-21swapper/322:40:483
3410926991412,1cyclictest0-21swapper/322:30:403
3410926991412,1cyclictest0-21swapper/321:50:043
3410926991412,1cyclictest0-21swapper/321:05:273
3410926991412,1cyclictest0-21swapper/320:35:223
3410926991412,1cyclictest0-21swapper/320:20:013
3410926991412,1cyclictest0-21swapper/320:05:053
3410926991412,1cyclictest0-21swapper/319:30:033
3410926991412,1cyclictest0-21swapper/300:20:343
3410926991411,1cyclictest0-21swapper/322:20:013
3410926991411,1cyclictest0-21swapper/321:30:103
3410926991411,1cyclictest0-21swapper/320:30:053
3410926991411,1cyclictest0-21swapper/319:35:323
3410926991411,1cyclictest0-21swapper/319:10:063
341092399149,3cyclictest3877868-21ssh00:20:242
341092399149,3cyclictest3486348-21wc20:05:212
341092399147,5cyclictest3492666-21ntpq20:10:222
341092399146,6cyclictest3898630-21tr00:35:182
3410923991412,1cyclictest3561589-21diskmemload22:37:012
3410923991411,2cyclictest431ktimers/221:17:002
3410923991411,2cyclictest3537452-21gltestperf20:50:132
3410923991411,2cyclictest3410912-21cyclictest22:05:192
3410923991411,2cyclictest0-21swapper/223:15:182
3410923991410,3cyclictest3787923-21unixbench-2d23:25:302
3410923991410,2cyclictest0-21swapper/221:10:002
341092199148,5cyclictest1-21systemd19:25:011
341092199148,5cyclictest0-21swapper/123:25:171
3410921991411,2cyclictest3763553-21grep23:10:251
3410921991411,2cyclictest3722494-21needreboot22:45:201
3410921991411,2cyclictest3628467-21irqcore21:50:191
3410921991411,2cyclictest3592709-21ssh21:26:331
3410921991411,2cyclictest3444585-21irqcore19:35:181
3410921991410,3cyclictest277-21plymouthd21:40:241
3410921991410,2cyclictest0-21swapper/122:35:181
341091799149,4cyclictest3798768-21grep23:35:140
341091799148,4cyclictest1-21systemd20:45:010
3410917991411,2cyclictest3860795-21kthreadcore00:10:210
3410917991411,2cyclictest3610852-21kworker/0:0+kacpid00:20:130
3410917991411,2cyclictest3561589-21diskmemload22:35:180
3410917991411,2cyclictest3493199-21kworker/0:2+kacpid21:35:140
3410917991411,2cyclictest3404524-21kworker/0:0+kacpid19:16:470
3410917991411,2cyclictest1-21systemd20:36:400
3410917991411,2cyclictest0-21swapper/020:15:230
3410917991410,3cyclictest3867719-21kthreadcore00:15:200
3410917991410,3cyclictest3845276-21unixbench-2d00:00:270
3410917991410,3cyclictest3544186-21irqrtprio20:55:180
3410917991410,3cyclictest3430996-21grep19:20:200
3410917991410,2cyclictest0-21swapper/023:30:000
3410926991311,1cyclictest0-21swapper/321:10:033
341092399139,2cyclictest0-21swapper/220:45:152
341092399139,2cyclictest0-21swapper/220:15:132
341092399139,2cyclictest0-21swapper/219:20:162
341092399138,4cyclictest3666297-21sshd22:10:392
3410923991310,2cyclictest431ktimers/220:59:132
3410923991310,2cyclictest3893656-21missed_timers00:30:202
3410923991310,2cyclictest3866291-21kthreadcore00:15:212
3410923991310,2cyclictest3826515-21kthreadcore23:50:162
3410923991310,2cyclictest3504084-21kthreadcore20:20:152
3410923991310,2cyclictest3476655-21kthreadcore20:00:162
3410923991310,2cyclictest3424848-21unixbench_multi19:15:242
341092199139,3cyclictest3603174-21fschecks_time21:35:151
341092199139,3cyclictest3486764-21timerandwakeup20:05:251
3410921991311,1cyclictest0-21swapper/121:45:171
3410921991310,2cyclictest769-21runrttasks22:05:031
3410921991310,2cyclictest3526451-21kthreadcore20:40:171
341091799139,3cyclictest0-21swapper/023:10:220
341091799139,2cyclictest0-21swapper/019:10:170
3410917991311,1cyclictest3610852-21kworker/0:0+kacpid23:51:060
3410917991311,1cyclictest3610852-21kworker/0:0+kacpid23:30:330
3410917991311,1cyclictest3493199-21kworker/0:2+kacpid21:06:070
3410917991311,1cyclictest3493199-21kworker/0:2+kacpid20:25:210
3410917991311,1cyclictest3404524-21kworker/0:0+kacpid20:05:270
3410917991311,1cyclictest3404524-21kworker/0:0+kacpid20:01:090
3410917991311,1cyclictest3404524-21kworker/0:0+kacpid19:50:310
3410917991311,1cyclictest3404524-21kworker/0:0+kacpid19:28:020
3410917991310,2cyclictest3550543-21irqrtprio21:00:180
3410917991310,2cyclictest3501371-21kthreadcore20:20:150
3410917991310,2cyclictest3444317-21cut19:35:160
341091799130,13cyclictest0-21swapper/020:45:240
34105232130,2sleep30-21swapper/319:06:503
341092399129,2cyclictest3742689-21irqcore23:00:162
341092399129,2cyclictest3614777-21kthreadcore21:40:182
341092399129,2cyclictest3561480-21systemd-journal00:03:082
341092399129,2cyclictest3518271-21cron20:35:002
341092399126,4cyclictest3511221-21ntpq20:25:222
341092399126,4cyclictest3442995-21ntpq19:30:222
341092199127,3cyclictest37255562chrt22:50:131
3410921991210,1cyclictest0-21swapper/119:44:591
34049142121,5sleep10-21swapper/119:05:121
341092399119,1cyclictest0-21swapper/221:00:242
341092399116,4cyclictest3523398-21memory20:35:162
341092399110,10cyclictest3455142-21sed19:40:172
341092199110,1cyclictest0-21swapper/120:45:501
34109239997,1cyclictest0-21swapper/219:10:012
3405307291,5sleep03405605-21sed19:05:150
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional