You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-10-19 - 02:45
Click here to display the system's profile data.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rackeslot8.osadl.org (updated Sat Oct 18, 2025 12:43:56)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
34146292800,1sleep30-21swapper/311:59:403
31480942490,2sleep23148096-21munin-plugin-st09:15:012
33425292450,1chrt3342524-21df11:15:171
2989362992725,1cyclictest3003741-21munin-plugin-st07:20:001
2989358992721,5cyclictest522-21in:imuxsock09:30:560
2989358992721,5cyclictest3078262-21sh08:20:010
2989358992620,5cyclictest3317778-21apt11:00:010
2989362992522,2cyclictest761-21cups-browsed12:25:001
2989358992416,3cyclictest3229434-21cpuspeed_turbos10:05:150
2989368992320,1cyclictest0-21swapper/310:35:013
2989358992318,4cyclictest0-21swapper/010:40:010
2989358992318,4cyclictest0-21swapper/009:20:000
2989358992316,2cyclictest3100201-21kthreadcore08:35:190
2989358992316,2cyclictest3061467-21kthreadcore08:05:200
2989358992316,2cyclictest3022461-21turbostat.cron07:34:590
2989368992219,1cyclictest0-21swapper/310:40:003
2989365992219,2cyclictest3192056-21kthreadcore09:40:182
2989365992217,4cyclictest522-21in:imuxsock07:15:012
2989362992219,2cyclictest3078246-21idleruntime-cro08:20:011
2989358992214,2cyclictest3253774-21fschecks_time10:20:170
2989362992117,3cyclictest3007218-21kthreadcore07:20:201
2989358992115,5cyclictest3046845-21unixbench_singl07:50:250
2989358992115,1cyclictest0-21swapper/009:44:590
2989358992114,2cyclictest547-21gdbus09:05:240
2989358992114,2cyclictest3198478-21kthreadcore09:45:180
2989358992113,3cyclictest3156665-21cpuspeed_turbos09:20:140
2989358992112,3cyclictest3422911-21cron12:05:000
2989365992017,2cyclictest500-21avahi-daemon07:55:012
2989362992017,1cyclictest3245091-21cron10:15:001
2989362992015,3cyclictest3350554-21grep11:20:141
33789712190,1chrt0-21swapper/311:35:233
2989368991916,1cyclictest0-21swapper/310:50:213
2989365991916,2cyclictest3084673-21cron08:25:002
2989365991916,2cyclictest0-21swapper/210:25:012
2989362991917,1cyclictest3028777-21sendmail07:40:001
2989362991916,2cyclictest3314391-21kthreadcore10:55:231
2989362991915,2cyclictest3278263-21irqcore10:35:191
2989362991914,3cyclictest3358666-21cpuspeed_turbos11:25:131
32457092180,1sleep30-21swapper/310:15:143
32457092180,1sleep30-21swapper/310:15:143
2989368991815,1cyclictest0-21swapper/312:40:013
2989368991815,1cyclictest0-21swapper/311:34:593
2989368991815,1cyclictest0-21swapper/309:50:003
2989368991815,1cyclictest0-21swapper/308:15:003
2989365991813,4cyclictest3350004-21cron11:20:012
2989362991815,2cyclictest3455734-21fschecks_count12:25:171
2989358991811,2cyclictest0-21swapper/009:10:260
2989368991713,1cyclictest0-21swapper/307:55:193
2989368991712,2cyclictest3090820-21turbostat08:30:013
2989365991714,2cyclictest3428244-21needreboot12:05:232
2989365991713,3cyclictest3096072-21ntp_kernel_err08:30:242
2989365991713,3cyclictest1-21systemd12:04:432
2989362991714,2cyclictest3022607-21latency_hist07:35:011
2989362991713,3cyclictest3391258-21fschecks_time11:45:131
2989362991713,3cyclictest3250368-21missed_timers10:15:201
2989362991713,3cyclictest3250368-21missed_timers10:15:191
2989362991712,3cyclictest3382739-21grep11:40:001
2989362991712,3cyclictest3338044-21kthreadcore11:10:211
2989358991712,4cyclictest3380634-21kworker/u8:3-xprtiod11:55:220
2989368991614,1cyclictest0-21swapper/311:30:013
2989368991613,1cyclictest0-21swapper/312:20:013
2989368991613,1cyclictest0-21swapper/312:20:003
2989368991613,1cyclictest0-21swapper/311:15:003
2989368991613,1cyclictest0-21swapper/310:05:003
2989368991613,1cyclictest0-21swapper/309:20:153
2989368991613,1cyclictest0-21swapper/307:50:203
298936599168,6cyclictest501-21dbus-daemon12:40:002
2989365991613,2cyclictest508-21polkitd11:30:002
2989365991613,2cyclictest3466619-21kthreadcore12:30:182
2989365991613,2cyclictest3294220-21gltestperf10:45:142
2989365991613,2cyclictest3285990-21cpuspeed_turbos10:40:152
2989365991613,2cyclictest3263030-21tr10:25:172
2989365991613,2cyclictest3231659-21kthreadcore10:05:192
2989365991613,2cyclictest3127835-21systemd-run09:00:002
2989365991613,2cyclictest0-21swapper/207:50:002
2989365991612,3cyclictest3380465-21unixbench_singl11:35:282
2989365991612,3cyclictest3315692-21unixbench_singl10:55:292
2989365991612,2cyclictest3277398-21sh10:35:012
2989365991610,5cyclictest3350881-21sed11:20:172
2989362991614,1cyclictest3471283-21idleruntime-cro12:35:001
2989362991613,2cyclictest3221522-21fschecks_time10:00:151
2989362991613,2cyclictest3103528-21cpuspeed_turbos08:40:141
2989362991613,2cyclictest3035691-21irqcore07:45:171
2989362991613,2cyclictest0-21swapper/110:30:261
2989362991613,2cyclictest0-21swapper/109:30:001
2989362991612,3cyclictest3302311-21head10:50:161
2989362991612,3cyclictest3204810-21sh09:50:001
2989362991612,3cyclictest3079085-21gltestperf08:20:161
2989362991612,3cyclictest3065050-21ntp_kernel_err08:05:231
2989358991614,1cyclictest0-21swapper/009:30:010
2989358991613,2cyclictest3439495-21cpuspeed_turbos12:15:140
2989358991613,2cyclictest3439495-21cpuspeed_turbos12:15:140
2989358991613,2cyclictest3374724-21latency_hist11:35:010
2989358991613,2cyclictest3263101-21kthreadcore10:25:170
2989358991613,2cyclictest3059739-21apt-key08:05:000
2989358991612,3cyclictest3463815-21fschecks_count12:30:150
2989358991612,3cyclictest3237902-21irqcore10:10:170
2989358991612,2cyclictest3309882-21latency_hist10:55:000
2989358991612,2cyclictest0-21swapper/011:25:210
2989368991512,1cyclictest0-21swapper/311:45:163
2989368991512,1cyclictest0-21swapper/310:40:333
2989368991512,1cyclictest0-21swapper/309:55:163
2989368991512,1cyclictest0-21swapper/309:50:203
2989368991512,1cyclictest0-21swapper/309:05:263
2989368991512,1cyclictest0-21swapper/309:02:273
2989368991512,1cyclictest0-21swapper/308:55:223
2989368991512,1cyclictest0-21swapper/308:50:043
2989368991512,1cyclictest0-21swapper/308:45:153
2989368991511,1cyclictest0-21swapper/309:25:183
2989368991511,1cyclictest0-21swapper/308:20:003
2989365991512,2cyclictest3414861-21sh12:00:002
2989365991512,2cyclictest3333883-21cron11:10:012
2989365991512,2cyclictest3303841-21kthreadcore10:50:182
2989365991512,2cyclictest3221768-21grep10:00:172
2989365991512,2cyclictest3180004-21sshd09:33:432
2989365991512,2cyclictest3115063-21unixbench_singl08:45:252
2989365991511,3cyclictest3383263-21grep11:40:162
2989365991511,2cyclictest0-21swapper/209:25:012
298936299159,4cyclictest3422909-21cron12:05:001
298936299157,2cyclictest3212797-21munin-run09:55:001
2989362991512,2cyclictest3439212-21latency_hist12:15:011
2989362991512,2cyclictest3184662-21kthreadcore09:35:181
2989362991512,2cyclictest3173015-21cpuspeed_turbos09:30:141
2989362991512,2cyclictest3047854-21gltestperf07:55:151
2989362991512,2cyclictest3011334-21cut07:25:171
2989362991512,2cyclictest0-21swapper/110:45:161
2989362991512,2cyclictest0-21swapper/110:05:261
2989362991511,3cyclictest3115865-21cpuspeed_turbos08:50:141
2989362991511,2cyclictest0-21swapper/112:00:001
2989362991511,2cyclictest0-21swapper/109:20:261
2989362991511,2cyclictest0-21swapper/109:15:131
2989362991510,3cyclictest3326742-21ps11:05:191
298936299150,6cyclictest1-21systemd07:15:021
2989358991512,2cyclictest3473350-21sh12:35:190
2989358991512,2cyclictest3294397-21irqcore10:45:160
2989358991512,2cyclictest3272637-21kthreadcore10:30:210
2989358991512,2cyclictest3097004-21kworker/0:0+kacpi_notify11:39:380
2989358991512,2cyclictest3097004-21kworker/0:0+kacpid11:46:010
2989358991512,2cyclictest3097004-21kworker/0:0+kacpid11:06:080
2989358991512,2cyclictest3038169-21kthreadcore07:45:200
2989358991512,2cyclictest3000782-21kthreadcore07:15:180
2989358991511,3cyclictest3071055-21missed_timers08:10:200
2989358991510,4cyclictest3186169-21grep09:35:220
2989358991510,3cyclictest3387032-21kthreadcore11:40:210
2989358991510,3cyclictest3213266-21cpuspeed_turbos09:55:150
2989358991510,3cyclictest3033994-21needreboot07:40:220
2989368991412,1cyclictest0-21swapper/311:00:193
2989368991412,1cyclictest0-21swapper/309:20:013
2989368991411,1cyclictest0-21swapper/312:20:123
2989368991411,1cyclictest0-21swapper/312:10:213
2989368991411,1cyclictest0-21swapper/312:00:183
2989368991411,1cyclictest0-21swapper/311:50:243
2989368991411,1cyclictest0-21swapper/311:40:173
2989368991411,1cyclictest0-21swapper/311:20:173
2989368991411,1cyclictest0-21swapper/310:45:183
2989368991411,1cyclictest0-21swapper/310:10:183
2989368991411,1cyclictest0-21swapper/310:10:003
2989368991411,1cyclictest0-21swapper/309:40:003
2989368991411,1cyclictest0-21swapper/309:10:153
2989368991411,1cyclictest0-21swapper/308:40:203
2989368991411,1cyclictest0-21swapper/308:38:493
2989368991411,1cyclictest0-21swapper/308:10:003
2989368991411,1cyclictest0-21swapper/307:35:193
2989368991411,1cyclictest0-21swapper/307:30:413
2989368991411,1cyclictest0-21swapper/307:20:253
298936599148,4cyclictest3818356-21systemd-journal10:36:422
2989365991412,1cyclictest0-21swapper/207:40:002
2989365991411,2cyclictest3457534-21kthreadcore12:25:212
2989365991411,2cyclictest3439497-21cpuspeed_turbos12:15:152
2989365991411,2cyclictest3439497-21cpuspeed_turbos12:15:142
2989365991411,2cyclictest3431598-21fschecks_time12:10:172
2989365991411,2cyclictest3391380-21gltestperf11:45:142
2989365991411,2cyclictest3242590-21sendmail_mailst10:10:242
2989365991411,2cyclictest3148726-21gltestperf09:15:152
2989365991411,2cyclictest0-21swapper/208:45:002
2989365991411,2cyclictest0-21swapper/208:20:012
2989365991411,2cyclictest0-21swapper/208:00:002
2989365991411,2cyclictest0-21swapper/207:25:012
2989365991410,3cyclictest3085272-21gltestperf08:25:172
2989365991410,2cyclictest3323735-21ssh11:00:282
2989365991410,2cyclictest0-21swapper/208:35:172
2989365991410,2cyclictest0-21swapper/207:15:152
298936299148,4cyclictest501-21dbus-daemon08:35:001
2989362991411,2cyclictest3439525-21cpuspeed_turbos12:15:141
2989362991411,2cyclictest3439525-21cpuspeed_turbos12:15:141
2989362991411,2cyclictest3402070-21kthreadcore11:50:211
2989362991411,2cyclictest3140759-21gltestperf09:10:161
2989362991411,2cyclictest0-21swapper/111:00:161
2989362991411,2cyclictest0-21swapper/110:25:141
2989362991410,3cyclictest3258631-21ntp_offset10:20:241
2989362991410,3cyclictest3217718-21kthreadcore09:55:211
2989362991410,3cyclictest3004-21cron08:05:001
2989362991410,2cyclictest3097474-21fschecks_time08:35:151
2989362991410,2cyclictest0-21swapper/111:40:141
298935899149,3cyclictest3103563-21cpuspeed_turbos08:40:150
298935899147,5cyclictest3818356-21systemd-journal11:15:000
2989358991412,1cyclictest3345371-21kthreadcore11:15:210
2989358991412,1cyclictest3097004-21kworker/0:0+kacpid10:01:490
2989358991411,2cyclictest3402449-21kthreadcore11:50:210
2989358991411,2cyclictest3355410-21cut11:20:210
2989358991411,2cyclictest3318305-21tr11:00:160
2989358991411,2cyclictest3116095-21head08:50:150
2989358991411,2cyclictest3097004-21kworker/0:0+kacpid10:42:350
2989358991411,2cyclictest0-21swapper/009:50:160
298935899141,11cyclictest2989559-21apt-get07:10:150
2989358991410,3cyclictest3251760-21cron10:17:010
2989358991410,3cyclictest3251760-21cron10:17:000
2989358991410,2cyclictest0-21swapper/008:20:170
2989368991311,1cyclictest0-21swapper/309:40:193
2989368991311,1cyclictest0-21swapper/307:40:233
2989368991311,1cyclictest0-21swapper/307:25:183
2989368991310,1cyclictest0-21swapper/312:30:183
2989368991310,1cyclictest0-21swapper/311:07:043
2989368991310,1cyclictest0-21swapper/310:55:273
2989368991310,1cyclictest0-21swapper/310:25:163
2989368991310,1cyclictest0-21swapper/310:20:213
2989368991310,1cyclictest0-21swapper/309:30:183
2989368991310,1cyclictest0-21swapper/308:20:193
2989368991310,1cyclictest0-21swapper/307:45:163
298936599139,3cyclictest3202130-21ntp_kernel_pll_09:45:202
298936599139,3cyclictest0-21swapper/209:35:142
298936599139,2cyclictest0-21swapper/208:10:232
298936599137,4cyclictest3818356-21systemd-journal10:00:002
2989365991311,1cyclictest0-21swapper/212:25:002
2989365991310,2cyclictest514-21systemd-logind09:30:012
2989365991310,2cyclictest3368942-21kthreadcore11:30:182
2989365991310,2cyclictest3245674-21fschecks_count10:15:152
2989365991310,2cyclictest3245674-21fschecks_count10:15:142
2989365991310,2cyclictest3118323-21kthreadcore08:50:172
2989365991310,2cyclictest3060218-21cpuspeed_turbos08:05:152
2989365991310,2cyclictest3029389-21gltestperf07:40:182
2989365991310,2cyclictest3022489-21latency_hist07:35:012
298936299139,2cyclictest3090813-21cron08:30:011
298936299137,4cyclictest3818356-21systemd-journal09:05:011
298936299137,4cyclictest3818356-21systemd-journal07:40:421
298936299136,5cyclictest3818356-21systemd-journal08:50:001
2989362991311,1cyclictest0-21swapper/109:05:141
2989362991310,2cyclictest0-21swapper/110:40:321
298936299130,1cyclictest0-21swapper/112:05:161
298935899139,3cyclictest3085041-21cpuspeed_turbos08:25:140
298935899138,3cyclictest3013938-21kthreadcore07:25:190
298935899137,4cyclictest3028191-21awk07:35:250
2989358991310,2cyclictest3455755-21fschecks_count12:25:170
2989358991310,2cyclictest3452186-21kthreadcore12:20:190
2989358991310,2cyclictest3432745-21ssh12:10:190
2989358991310,2cyclictest3427119-21kthreadcore12:05:210
2989358991310,2cyclictest3131013-21kthreadcore09:00:220
2989358991310,2cyclictest3113404-21kthreadcore08:45:190
2989358991310,2cyclictest3092622-21kthreadcore08:30:210
2989358991310,2cyclictest3047969-21sed07:55:170
298936899129,1cyclictest0-21swapper/312:25:223
298936899129,1cyclictest0-21swapper/312:05:193
298936899129,1cyclictest0-21swapper/307:20:013
298936599129,2cyclictest3334614-21gltestperf11:10:172
298936599129,1cyclictest0-21swapper/209:05:002
298936599128,3cyclictest0-21swapper/207:25:162
298936599128,2cyclictest0-21swapper/209:05:152
298936599127,4cyclictest3053640-21sshd08:00:102
2989365991210,1cyclictest0-21swapper/209:55:002
298936299129,2cyclictest3472135-21sh12:35:181
298936299129,2cyclictest3041621-21fschecks_time07:50:161
298936299128,3cyclictest3370063-21kthreadcore11:30:181
298936299127,4cyclictest3126927-21ntpq08:55:221
298936299126,4cyclictest501-21dbus-daemon09:40:231
2989362991210,1cyclictest0-21swapper/108:10:431
298935899129,2cyclictest0-21swapper/008:55:250
298935899129,2cyclictest0-21swapper/007:20:200
298936899119,1cyclictest0-21swapper/308:00:133
298936899118,1cyclictest0-21swapper/308:30:243
298936899118,1cyclictest0-21swapper/307:15:003
298936899116,3cyclictest3348324-21taskset11:16:153
298936599119,1cyclictest3140072-21diskmemload11:50:052
298936599104,4cyclictest2989493-21cut07:10:012
298936299106,2cyclictest2989256-21latency_hist07:10:011
2983423290,4sleep30-21swapper/307:05:133
29893589981,5cyclictest2989256-21latency_hist07:10:010
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional