You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-09-19 - 22:51
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the 376 highest latencies:
System rackeslot2.osadl.org (updated Thu Sep 19, 2024 12:43:58)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
333665521000,1sleep13336644-21fschecks_time12:00:161
31634822880,2sleep30-21swapper/309:30:293
31409272310,2chrt0-21swapper/309:15:103
2420791290,4ptp4l175090irq/125-enp1s0-TxRx-011:42:163
32295622260,2sleep30-21swapper/310:30:013
2420791250,3ptp4l175090irq/125-enp1s0-TxRx-008:20:223
299055099245,9cyclictest0-21swapper/211:00:012
2420791240,6ptp4l175090irq/125-enp1s0-TxRx-012:05:003
2420791240,4ptp4l175090irq/125-enp1s0-TxRx-010:40:213
2420791240,3ptp4l175090irq/125-enp1s0-TxRx-010:10:273
2420791240,2ptp4l0-21swapper/308:15:023
2990549992320,2cyclictest0-21swapper/108:50:411
2990549992315,7cyclictest3150075-21kthreadcore09:20:191
2420791230,5ptp4l175090irq/125-enp1s0-TxRx-010:51:173
2420791230,3ptp4l175090irq/125-enp1s0-TxRx-010:35:133
2420791230,2ptp4l175090irq/125-enp1s0-TxRx-007:21:263
2420791230,2ptp4l0-21swapper/311:35:003
2420791230,2ptp4l0-21swapper/309:50:193
333504922218,2sleep20-21swapper/211:55:292
331240422218,2sleep20-21swapper/211:40:142
2420791220,6ptp4l175090irq/125-enp1s0-TxRx-011:37:073
2420791220,4ptp4l175090irq/125-enp1s0-TxRx-011:09:293
2420791220,4ptp4l175090irq/125-enp1s0-TxRx-011:09:293
2420791220,4ptp4l175090irq/125-enp1s0-TxRx-007:15:043
2420791220,4ptp4l175090irq/125-enp1s0-TxRx-007:15:043
2420791220,3ptp4l175090irq/125-enp1s0-TxRx-011:00:123
2420791220,3ptp4l175090irq/125-enp1s0-TxRx-008:55:023
2420791220,2ptp4l175090irq/125-enp1s0-TxRx-010:55:203
2420791220,2ptp4l0-21swapper/307:25:123
2420791220,1ptp4l0-21swapper/311:11:303
2420791220,1ptp4l0-21swapper/309:40:203
2420791220,11ptp4l175090irq/125-enp1s0-TxRx-011:26:243
338291022118,2sleep20-21swapper/212:39:062
337695922118,2sleep20-21swapper/212:33:132
335907122118,2sleep20-21swapper/212:18:102
332365522118,2sleep20-21swapper/211:49:362
330734022118,2sleep20-21swapper/211:35:172
330565722118,2sleep20-21swapper/211:30:402
329985622118,2sleep20-21swapper/211:26:522
326960122118,2sleep20-21swapper/211:00:232
324654822117,2sleep20-21swapper/210:40:272
300996122117,2sleep20-21swapper/207:20:282
299602922118,2sleep20-21swapper/207:10:502
2990550992119,1cyclictest0-21swapper/212:25:012
2990548992119,1cyclictest3170020-21awk09:40:000
2990548992119,1cyclictest3170020-21awk09:40:000
2990548992118,2cyclictest3241326-21grep10:40:010
2420791210,4ptp4l175090irq/125-enp1s0-TxRx-012:40:003
2420791210,4ptp4l175090irq/125-enp1s0-TxRx-012:15:093
2420791210,4ptp4l175090irq/125-enp1s0-TxRx-009:58:113
2420791210,3ptp4l175090irq/125-enp1s0-TxRx-010:20:243
2420791210,2ptp4l0-21swapper/309:25:213
328223022017,2sleep20-21swapper/211:12:092
327633422017,2sleep20-21swapper/211:07:082
327633422017,2sleep20-21swapper/211:07:082
325282622017,2sleep20-21swapper/210:47:482
324095822017,2sleep20-21swapper/210:36:502
323522922017,2sleep20-21swapper/210:34:012
322331022017,2sleep20-21swapper/210:22:282
321159322017,2sleep20-21swapper/210:13:092
320562422017,2sleep20-21swapper/210:06:322
319970622017,2sleep20-21swapper/210:01:192
319351122017,2sleep20-21swapper/209:56:062
317622222017,2sleep20-21swapper/209:45:042
317566322017,2sleep20-21swapper/209:41:512
316394122017,2sleep20-21swapper/209:33:022
315202822017,2sleep20-21swapper/209:21:072
314640722017,2sleep20-21swapper/209:19:272
313453522017,2sleep20-21swapper/209:07:412
312853522017,2sleep20-21swapper/209:01:112
311672622017,2sleep20-21swapper/208:50:532
310564722017,2sleep20-21swapper/208:45:042
309381622017,2sleep20-21swapper/208:35:062
308158122017,2sleep20-21swapper/208:23:402
307550122017,2sleep20-21swapper/208:16:232
306977222017,2sleep20-21swapper/208:13:062
306388422017,2sleep20-21swapper/208:07:412
305785622017,2sleep20-21swapper/208:00:322
304673822017,2sleep20-21swapper/207:55:072
304623622017,2sleep20-21swapper/207:52:382
304007822017,2sleep20-21swapper/207:47:252
303398122017,2sleep20-21swapper/207:43:042
2990549992017,2cyclictest3187805-21apt-key09:55:001
2420791200,7ptp4l175090irq/125-enp1s0-TxRx-012:29:563
2420791200,6ptp4l175090irq/125-enp1s0-TxRx-012:10:123
2420791200,5ptp4l175090irq/125-enp1s0-TxRx-010:06:323
2420791200,5ptp4l175090irq/125-enp1s0-TxRx-008:15:073
2420791200,5ptp4l175090irq/125-enp1s0-TxRx-007:55:553
2420791200,5ptp4l175090irq/125-enp1s0-TxRx-007:39:363
2420791200,4ptp4l175090irq/125-enp1s0-TxRx-008:40:243
2420791200,3ptp4l175090irq/125-enp1s0-TxRx-011:57:473
2420791200,3ptp4l175090irq/125-enp1s0-TxRx-011:24:373
2420791200,3ptp4l175090irq/125-enp1s0-TxRx-007:43:133
2420791200,3ptp4l175090irq/125-enp1s0-TxRx-007:31:183
2420791200,2ptp4l0-21swapper/311:45:243
2420791200,2ptp4l0-21swapper/309:06:343
2990549991916,2cyclictest3288386-21sendmail11:20:001
2990549991916,2cyclictest1611-21systemd-logind08:20:011
2990549991915,3cyclictest3335903-21sh12:00:011
2990549991915,3cyclictest3193084-21mailstats09:55:231
2990549991915,3cyclictest1608-21in:imuxsock07:20:001
2990549991915,3cyclictest1608-21in:imuxsock07:20:001
2990548991913,5cyclictest1583-1atopacctd09:10:200
299054899190,1cyclictest0-21swapper/007:45:350
2420791190,7ptp4l175090irq/125-enp1s0-TxRx-009:00:123
2420791190,5ptp4l175090irq/125-enp1s0-TxRx-011:16:033
2420791190,5ptp4l175090irq/125-enp1s0-TxRx-010:49:013
2420791190,5ptp4l175090irq/125-enp1s0-TxRx-009:24:123
2420791190,4ptp4l175090irq/125-enp1s0-TxRx-012:32:053
2420791190,4ptp4l175090irq/125-enp1s0-TxRx-008:50:423
2420791190,4ptp4l175090irq/125-enp1s0-TxRx-008:25:263
2420791190,2ptp4l175090irq/125-enp1s0-TxRx-012:05:193
2420791190,2ptp4l0-21swapper/310:30:103
2420791190,2ptp4l0-21swapper/310:00:173
2420791190,2ptp4l0-21swapper/309:11:593
2990549991815,2cyclictest3158373-21systemd-run09:30:021
2990549991815,2cyclictest0-21swapper/107:15:021
2990548991815,2cyclictest3064173-21systemd-run08:10:010
2990548991815,2cyclictest2997121-21latency_hist07:15:020
2420791180,9ptp4l175090irq/125-enp1s0-TxRx-008:35:063
2420791180,6ptp4l175090irq/125-enp1s0-TxRx-012:20:113
2420791180,4ptp4l175090irq/125-enp1s0-TxRx-008:49:463
2420791180,4ptp4l175090irq/125-enp1s0-TxRx-007:11:433
2420791180,3ptp4l175090irq/125-enp1s0-TxRx-007:50:243
2420791180,3ptp4l175090irq/125-enp1s0-TxRx-007:47:463
2420791180,2ptp4l0-21swapper/310:20:003
2420791180,2ptp4l0-21swapper/310:20:003
2420791180,2ptp4l0-21swapper/308:05:263
2990550991714,1cyclictest0-21swapper/210:20:012
2990550991714,1cyclictest0-21swapper/210:20:002
2990549991714,2cyclictest3358446-21mailstats12:15:221
2990549991714,2cyclictest3347505-21systemd-run12:10:001
2990549991714,2cyclictest3105390-21systemd-run08:45:011
2990549991714,2cyclictest3067070-21kthreadcore08:10:201
2990549991714,2cyclictest2990543-21cyclictest09:10:171
2990549991714,2cyclictest1611-21systemd-logind08:35:021
2990549991713,3cyclictest3098077-21kthreadcore08:35:201
2990549991713,3cyclictest1-21systemd10:40:481
2990548991714,2cyclictest3347713-21munin-plugin-st12:10:020
2990548991714,2cyclictest1611-21systemd-logind12:40:000
2990548991714,2cyclictest1583-1atopacctd12:35:000
2990548991713,3cyclictest3142239-21kthreadcore09:15:260
2990548991713,3cyclictest1611-21systemd-logind11:30:010
2990548991713,3cyclictest0-21swapper/008:45:010
2659998170,4rtkit-daemon1887-21snmpd11:07:130
2659998170,4rtkit-daemon1887-21snmpd11:07:130
2420791170,4ptp4l175090irq/125-enp1s0-TxRx-009:48:343
2420791170,2ptp4l0-21swapper/309:39:553
2420791170,2ptp4l0-21swapper/309:39:553
2420791170,2ptp4l0-21swapper/308:30:103
2420791170,10ptp4l175090irq/125-enp1s0-TxRx-011:52:353
299055099161,5cyclictest0-21swapper/212:00:122
2990550991614,1cyclictest0-21swapper/209:10:122
299054999169,6cyclictest3009909-21unixbench_singl07:20:271
2990549991613,2cyclictest3353442-21systemctl12:15:011
2990549991613,2cyclictest3325483-21kthreadcore11:50:201
2990549991613,2cyclictest3323701-21latency_hist11:50:011
2990549991613,2cyclictest3294228-21cron11:25:001
2990549991613,2cyclictest3267663-21kthreadcore11:00:201
2990549991613,2cyclictest3057653-21timerwakeupswit08:00:241
2990549991613,2cyclictest3028333-21/usr/sbin/munin07:40:011
2990549991613,2cyclictest2990543-21cyclictest10:55:011
2990549991613,2cyclictest0-21swapper/111:45:011
2990549991613,2cyclictest0-21swapper/107:45:021
2990549991612,3cyclictest3200923-21fschecks_count10:05:161
2990549991612,3cyclictest3014936-21kthreadcore07:25:201
2990548991613,2cyclictest3183653-21kthreadcore09:50:200
2990548991613,2cyclictest3164208-21systemd-run09:35:000
2990548991613,2cyclictest3007926-21kthreadcore07:20:190
2420791160,5ptp4l175090irq/125-enp1s0-TxRx-008:00:203
2420791160,5ptp4l175090irq/125-enp1s0-TxRx-007:05:113
299055099158,2cyclictest0-21swapper/209:25:112
299055099155,5cyclictest0-21swapper/208:30:022
2990550991513,1cyclictest0-21swapper/211:20:002
2990550991513,1cyclictest0-21swapper/207:30:002
2990550991513,1cyclictest0-21swapper/207:15:122
2990550991513,1cyclictest0-21swapper/207:15:122
2990550991512,1cyclictest0-21swapper/208:55:202
2990550991512,1cyclictest0-21swapper/208:44:562
2990549991512,2cyclictest3241255-21cut10:40:001
2990549991512,2cyclictest3170162-21latency_hist09:40:011
2990549991512,2cyclictest3170162-21latency_hist09:40:011
2990549991512,2cyclictest3130506-21kthreadcore09:05:221
2990549991512,2cyclictest3128849-21latency_hist09:05:011
2990549991512,2cyclictest1869-21runrttasks07:45:261
2990549991511,3cyclictest3143722-21kthreadcore09:15:261
2990549991511,3cyclictest3110269-21phc2sys08:45:211
2990549991511,3cyclictest3043719-21kthreadcore07:50:191
299054899155,6cyclictest0-21swapper/010:42:440
299054899155,6cyclictest0-21swapper/010:04:010
299054899155,6cyclictest0-21swapper/008:33:150
2990548991513,1cyclictest3323845-21cut11:50:020
2990548991512,2cyclictest3282469-21latency_hist11:15:010
2990548991512,2cyclictest3270701-21cut11:05:000
2990548991512,2cyclictest3216918-21mailstats10:15:220
2990548991512,2cyclictest3216918-21mailstats10:15:220
2990548991512,2cyclictest3151856-21unixbench-2d09:20:260
2990548991512,2cyclictest3099406-21perf08:40:010
2990548991512,2cyclictest3052310-21cron08:00:010
2990548991512,2cyclictest3016355-21apt-config07:30:000
2990548991512,2cyclictest0-21swapper/012:15:000
2990548991512,2cyclictest0-21swapper/010:00:010
299054899151,12cyclictest3288356-21apt11:20:000
2990548991511,2cyclictest0-21swapper/009:50:010
299055099146,3cyclictest0-21swapper/212:25:122
299055099143,6cyclictest0-21swapper/209:40:012
299055099143,6cyclictest0-21swapper/209:40:012
2990550991411,1cyclictest0-21swapper/211:20:242
299054999146,7cyclictest3272601-21tr11:05:181
299054999146,7cyclictest3272601-21tr11:05:181
2990549991412,1cyclictest3081704-21latency_hist08:25:011
2990549991411,2cyclictest3381621-21kthreadcore12:35:201
2990549991411,2cyclictest3366812-21cut12:25:181
2990549991411,2cyclictest3294956-21cstates11:25:151
2990549991411,2cyclictest3248981-21kthreadcore10:45:191
2990549991411,2cyclictest3207112-21grep10:10:211
2990549991411,2cyclictest3170509-21aten_repower_vo09:40:131
2990549991411,2cyclictest3060201-21kthreadcore08:05:191
2990549991411,2cyclictest1583-1atopacctd10:00:131
2990549991411,2cyclictest0-21swapper/108:30:011
2990549991410,3cyclictest3281628-21processes11:10:251
2990549991410,3cyclictest3227069-21kthreadcore10:25:201
2990549991410,3cyclictest3214340-21kthreadcore10:15:171
2990549991410,3cyclictest3214340-21kthreadcore10:15:171
2990549991410,3cyclictest3118686-21kthreadcore08:55:171
2990549991410,3cyclictest1583-1atopacctd11:30:171
299054899148,5cyclictest3288705-21apt-get11:20:110
299054899148,5cyclictest3057919-21sshd08:01:190
299054899145,5cyclictest0-21swapper/008:21:450
299054899144,6cyclictest0-21swapper/008:49:300
299054899144,6cyclictest0-21swapper/007:17:040
299054899144,6cyclictest0-21swapper/007:17:040
2990548991411,2cyclictest3363181-21kthreadcore12:20:190
2990548991411,2cyclictest3260571-21kthreadcore10:55:190
2990548991411,2cyclictest3225678-21kthreadcore10:25:190
2990548991411,2cyclictest3170376-21aten_repower_cu09:40:120
2990548991411,2cyclictest3153969-21kthreadcore09:25:190
2990548991411,2cyclictest3113233-21kthreadcore08:50:360
2990548991411,2cyclictest26598-21rtkit-daemon12:02:140
2990548991411,2cyclictest26598-21rtkit-daemon10:07:550
2990548991411,2cyclictest0-21swapper/010:35:020
2990548991410,3cyclictest3354009-21cpuspeed_turbos12:15:120
2990548991410,3cyclictest3069952-21latency_hist08:15:000
2990550991310,2cyclictest0-21swapper/212:05:162
2990550991310,2cyclictest0-21swapper/207:35:182
2990550991310,1cyclictest0-21swapper/208:30:202
299054999139,3cyclictest3361396-21kthreadcore12:20:181
299054999139,3cyclictest3259481-21cpuspeed_turbos10:55:141
299054999139,3cyclictest3051462-21processes07:55:261
299054999139,3cyclictest3018106-21kthreadcore07:30:181
2990549991310,2cyclictest3372150-21fschecks_count12:30:161
2990549991310,2cyclictest3230002-21cpuspeed_turbos10:30:131
2990549991310,2cyclictest3220657-21kthreadcore10:20:201
2990549991310,2cyclictest3177768-21kthreadcore09:45:181
2990549991310,2cyclictest3159851-21kthreadcore09:30:191
299054999130,4cyclictest3307637-21ps11:35:171
299054899138,4cyclictest3028920-21cstates07:40:140
299054899137,5cyclictest0-21swapper/007:50:190
299054899136,6cyclictest3313141-21grep11:40:180
2990548991310,2cyclictest3334153-21kthreadcore11:55:210
2990548991310,2cyclictest3310477-21kthreadcore11:35:190
2990548991310,2cyclictest3252082-21pmu-power10:45:230
2990548991310,2cyclictest3219296-21kthreadcore10:20:190
2990548991310,2cyclictest3209201-21kthreadcore10:10:230
2990548991310,2cyclictest3133503-21kthreadcore09:05:240
2990548991310,2cyclictest3126067-21kthreadcore09:00:200
2990548991310,2cyclictest3119145-21kthreadcore08:55:170
2990548991310,2cyclictest3083176-21irqrtprio08:25:180
2990548991310,2cyclictest3075354-21sed08:15:250
2990548991310,2cyclictest3025490-21kthreadcore07:35:210
299055099128,2cyclictest0-21swapper/209:50:122
299055099127,3cyclictest0-21swapper/210:50:122
2990550991210,1cyclictest0-21swapper/207:30:192
299055099120,3cyclictest0-21swapper/212:13:442
299054899129,2cyclictest3326763-21kthreadcore11:50:210
299054899129,2cyclictest3255681-21kthreadcore10:50:190
299054899129,2cyclictest1562-21gmain07:10:020
299054899124,7cyclictest3365794-21aten2.4-expect12:25:120
299054899123,5cyclictest258-21systemd-journal11:30:020
299055099119,1cyclictest0-21swapper/210:25:182
299055099115,3cyclictest0-21swapper/211:50:122
299054999118,2cyclictest0-21swapper/107:10:021
299054899115,4cyclictest3016820-21apt07:30:110
29905509997,1cyclictest0-21swapper/207:10:022
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional