You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-11-19 - 11:33
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Wed Nov 19, 2025 00:43:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
278459226150,7sleep60-21swapper/603:19:146
278834725747,7sleep50-21swapper/503:20:095
278833725646,7sleep40-21swapper/403:20:004
2788913995551,3cyclictest2074657-21kworker/u16:4+flush-8:003:43:455
278892299520,50cyclictest3136126-21latency_hist06:13:457
2788909994644,2cyclictest3146593-21kworker/u16:1+events_unbound06:39:154
2788909994139,2cyclictest2904058-21kworker/u16:3+flush-8:004:19:164
278892299390,37cyclictest3439310-21latency_hist08:43:447
278892299390,37cyclictest3439310-21latency_hist08:43:447
27883492393,31sleep70-21swapper/703:20:117
2788918993836,2cyclictest3237137-21kworker/u16:2+events_unbound07:24:086
2788913993836,2cyclictest2732658-21kworker/u16:2+events_unbound03:29:045
2788909993634,2cyclictest3257158-21kworker/u16:0+events_unbound07:11:044
2788909993634,2cyclictest2954418-21kworker/u16:0+events_unbound04:55:564
2788922993529,5cyclictest3206777-21latency_hist06:48:457
2788913993432,2cyclictest2984360-21kworker/u16:1+events_unbound05:09:005
2788909993432,2cyclictest3186381-21kworker/u16:3+events_unbound06:57:434
2788909993432,2cyclictest2904058-21kworker/u16:3+events_unbound04:35:484
2788909993331,2cyclictest3075554-21kworker/u16:3+events_unbound06:18:124
2788909993331,2cyclictest2732658-21kworker/u16:2+flush-8:003:28:454
2788909993230,2cyclictest3075529-21kworker/u16:1+events_unbound05:55:244
2788909993230,2cyclictest3004647-21kworker/u16:2+flush-8:005:12:004
2788909993230,2cyclictest2843211-21kworker/u16:0+events_unbound03:53:164
2788909993230,2cyclictest2732658-21kworker/u16:2+events_unbound03:39:554
2788909993228,3cyclictest3439295-21updatedb08:43:394
2788909993228,3cyclictest3439295-21updatedb08:43:394
2788913993129,2cyclictest3257779-21kworker/u16:1+flush-8:007:18:445
2788909993129,2cyclictest3338936-21kworker/u16:1+events_unbound08:07:214
2788909993129,2cyclictest3126074-21kworker/u16:1+events_unbound06:11:124
2788909993129,2cyclictest3115957-21kworker/u16:2+events_unbound06:23:174
2788909993129,2cyclictest3075554-21kworker/u16:3+flush-8:005:44:124
2788909993129,2cyclictest3075554-21kworker/u16:3+events_unbound05:51:304
2788909993129,2cyclictest2904058-21kworker/u16:3+events_unbound05:15:484
2788909993129,2cyclictest2074657-21kworker/u16:4+flush-8:003:32:204
2788909993028,2cyclictest3237137-21kworker/u16:2+flush-8:007:37:554
2788909993028,2cyclictest3227014-21kworker/u16:0+events_unbound07:04:204
2788909993028,2cyclictest3198285-21kworker/u16:4+flush-8:006:51:244
2788909993028,2cyclictest3186381-21kworker/u16:3+flush-8:007:01:124
2788909993028,2cyclictest3186381-21kworker/u16:3+events_unbound07:17:454
2788909993028,2cyclictest3105829-21kworker/u16:0+events_unbound06:02:544
2788909993028,2cyclictest3004647-21kworker/u16:2+events_unbound05:05:444
2788909993028,2cyclictest2954418-21kworker/u16:0+events_unbound04:51:404
2788909993028,2cyclictest2944167-21kworker/u16:1+events_unbound04:42:194
2788909993028,2cyclictest2904058-21kworker/u16:3+events_unbound05:03:254
2788909993028,2cyclictest2904058-21kworker/u16:3+events_unbound05:03:254
2788909993028,2cyclictest2903596-21kworker/u16:1+events_unbound04:16:164
2788909993028,2cyclictest2853362-21kworker/u16:3+events_unbound04:03:164
2788922992924,4cyclictest3196654-21latency_hist06:43:457
2788909992927,2cyclictest3338936-21kworker/u16:1+events_unbound08:00:234
2788909992927,2cyclictest3338936-21kworker/u16:1+events_unbound08:00:234
2788909992927,2cyclictest3267942-21kworker/u16:0+events_unbound07:31:594
2788909992927,2cyclictest3186381-21kworker/u16:3+events_unbound06:37:044
2788909992927,2cyclictest2954418-21kworker/u16:0+events_unbound04:47:284
2788909992927,2cyclictest2904058-21kworker/u16:3+events_unbound04:28:444
2788909992926,2cyclictest3115957-21kworker/u16:2+flush-8:006:43:594
2788922992827,0cyclictest0-21swapper/704:14:137
278892299280,26cyclictest3065462-21latency_hist05:38:457
2788909992826,2cyclictest3428405-21kworker/u16:1+events_unbound08:35:244
2788909992826,2cyclictest3408681-21kworker/u16:3+events_unbound08:48:004
2788909992826,2cyclictest3338936-21kworker/u16:1+flush-8:008:11:084
2788909992826,2cyclictest3176232-21kworker/u16:4+flush-8:006:31:524
2788909992826,2cyclictest3075554-21kworker/u16:3+events_unbound06:07:374
2788909992826,2cyclictest3035126-21kworker/u16:1+events_unbound05:27:284
2788913992725,2cyclictest3237137-21kworker/u16:2+flush-8:007:08:445
2788909992725,2cyclictest3408681-21kworker/u16:3+events_unbound08:25:244
2788909992725,2cyclictest3267942-21kworker/u16:0+events_unbound08:32:494
2788909992725,2cyclictest3237137-21kworker/u16:2+events_unbound07:41:284
2788909992725,2cyclictest3237137-21kworker/u16:2+events_unbound07:28:284
2788909992725,2cyclictest3009893-21kworker/u16:4+events_unbound05:31:404
2788909992725,2cyclictest2954418-21kworker/u16:0+flush-8:005:19:204
2788909992725,2cyclictest2924200-21kworker/u16:0+events_unbound04:30:044
2788909992725,2cyclictest2873565-21kworker/u16:2+flush-8:004:05:134
2788909992724,2cyclictest2833115-21kworker/u16:1+events_unbound03:53:554
2788922992620,5cyclictest589235-21systemd-journal04:57:217
2788913992623,2cyclictest3267942-21kworker/u16:0+flush-8:007:34:035
2788909992624,2cyclictest3378699-21kworker/u16:2+events_unbound08:16:154
2788909992624,2cyclictest3328546-21kworker/u16:3+events_unbound07:51:594
2788909992624,2cyclictest3105829-21kworker/u16:0+events_unbound06:26:444
2788909992624,2cyclictest3065290-21kworker/u16:2+events_unbound05:38:204
2788909992624,2cyclictest2873565-21kworker/u16:2+events_unbound04:13:164
2788909992624,2cyclictest2823026-21kworker/u16:0+events_unbound03:37:164
2788909992623,2cyclictest3408681-21kworker/u16:3+events_unbound08:48:494
2788909992623,2cyclictest3267942-21kworker/u16:0+flush-8:007:53:474
2788922992522,2cyclictest3025246-21cat05:18:457
2788913992521,3cyclictest2853437-21kworker/u16:4+flush-8:004:14:175
2788909992523,2cyclictest3009893-21kworker/u16:4+events_unbound05:40:594
2788909992523,2cyclictest2833115-21kworker/u16:1+events_unbound03:45:174
2788922992419,4cyclictest2944345-21latency_hist04:38:457
2788922992317,5cyclictest589235-21systemd-journal05:53:457
2788922992317,5cyclictest3055389-21latency_hist05:33:457
2788913992321,2cyclictest2954418-21kworker/u16:0+events_unbound04:44:145
2788909992321,2cyclictest3338936-21kworker/u16:1+flush-8:008:20:454
2788909992321,2cyclictest3277790-21kworker/u16:4+events_unbound07:47:174
2788909992321,2cyclictest3257779-21kworker/u16:1+events_unbound07:20:374
2788913992220,2cyclictest3408681-21kworker/u16:3+events_unbound08:29:035
2788913992220,2cyclictest3277790-21kworker/u16:4+flush-8:007:39:115
2788922992116,4cyclictest3307996-21latency_hist07:38:457
2788918992117,3cyclictest3105829-21kworker/u16:0+flush-8:006:29:126
2788922992015,4cyclictest3429188-21latency_hist08:38:447
2788913992018,2cyclictest3267942-21kworker/u16:0+events_unbound07:19:135
32411822190,1sleep60-21swapper/607:04:046
2788922991913,5cyclictest3378658-21latency_hist08:13:457
278892299190,18cyclictest3237168-21latency_hist07:03:447
2788922991817,0cyclictest0-21swapper/705:58:557
2788922991813,4cyclictest589235-21systemd-journal08:31:207
2788922991813,4cyclictest589235-21systemd-journal06:33:397
2788922991813,4cyclictest2853396-21latency_hist03:53:467
278892299180,17cyclictest0-21swapper/707:54:117
278892299180,17cyclictest0-21swapper/703:39:107
2788922991714,2cyclictest0-21swapper/704:23:457
2788922991711,5cyclictest589235-21systemd-journal05:28:457
2788918991715,2cyclictest3257779-21kworker/u16:1+events_unbound07:22:226
2788918991715,2cyclictest2853362-21kworker/u16:3+events_unbound04:07:556
2788918991714,2cyclictest3009893-21kworker/u16:4+flush-8:005:39:166
278891399170,16cyclictest0-21swapper/508:19:105
2788922991613,2cyclictest4175659-21snmpd05:03:517
2788922991613,2cyclictest4175659-21snmpd04:38:507
278892299160,14cyclictest589235-21systemd-journal06:08:457
278892299160,14cyclictest3297913-21sh07:33:457
278892299160,14cyclictest3287792-21wc07:28:457
278892299160,14cyclictest0-21swapper/707:49:117
278891899160,15cyclictest3002236-21kthreadcore05:04:086
2788922991514,0cyclictest0-21swapper/706:24:107
278892299150,14cyclictest0-21swapper/705:54:107
2788918991513,2cyclictest3277790-21kworker/u16:4+events_unbound07:43:256
2788918991513,1cyclictest3251302-21kthreadcore07:09:056
2788918991513,1cyclictest2945031-21cstates04:38:586
2788913991513,2cyclictest3009893-21kworker/u16:4+flush-8:005:39:125
2788913991511,3cyclictest3186381-21kworker/u16:3+flush-8:006:49:045
278892299148,5cyclictest2974625-21latency_hist04:53:457
278892299146,7cyclictest27904-21gmain05:44:117
2788922991411,2cyclictest0-21swapper/708:43:457
278892299140,1cyclictest0-21swapper/703:34:137
278892299140,12cyclictest589235-21systemd-journal05:21:217
2788918991414,0cyclictest0-21swapper/604:59:056
2788918991414,0cyclictest0-21swapper/604:59:056
2788918991413,1cyclictest3157927-21cat06:24:046
2788918991412,2cyclictest3035126-21kworker/u16:1+events_unbound05:31:056
2788918991412,2cyclictest2904058-21kworker/u16:3+events_unbound05:26:436
2788918991412,1cyclictest3205862-21ntpq06:44:136
2788918991412,1cyclictest3099812-21kthreadcore05:54:056
2788913991412,2cyclictest2904058-21kworker/u16:3+events_unbound04:34:145
2788913991412,2cyclictest2853437-21kworker/u16:4+flush-8:003:59:165
2788913991412,1cyclictest2971886-21kthreadcore04:49:085
2788913991410,3cyclictest3237137-21kworker/u16:2+flush-8:007:09:125
278891399140,1cyclictest0-21swapper/504:39:125
278892299139,3cyclictest4175659-21snmpd08:26:437
278892299139,3cyclictest0-21swapper/703:28:467
278892299138,4cyclictest921-21dbus-daemon07:48:447
2788922991310,2cyclictest0-21swapper/707:18:457
2788918991312,1cyclictest3432963-21kthreadcore08:39:056
2788918991312,1cyclictest3432963-21kthreadcore08:39:056
2788918991312,1cyclictest3386643-21cut08:14:076
2788918991312,1cyclictest3369306-21expr08:09:006
2788918991312,1cyclictest3260084-21kthreadcore07:14:056
2788918991312,1cyclictest3218044-21smartctl06:54:026
2788918991312,1cyclictest3209094-21cut06:49:046
2788918991312,1cyclictest2954840-21/usr/sbin/munin04:44:006
2788918991312,1cyclictest2867796-21kthreadcore03:59:076
2788918991311,1cyclictest2933351-21ntpq04:29:156
2788918991311,1cyclictest2892817-21ntpq04:09:096
2788918991310,2cyclictest2943399-21ntpq04:34:116
2788913991311,2cyclictest3277790-21kworker/u16:4+events_unbound07:49:035
2788913991311,2cyclictest3267942-21kworker/u16:0+events_unbound07:54:025
278892299129,2cyclictest4175659-21snmpd08:22:377
278892299129,2cyclictest4175659-21snmpd07:11:157
278892299129,2cyclictest4175659-21snmpd07:05:117
278892299129,2cyclictest4175659-21snmpd06:16:197
278892299129,2cyclictest0-21swapper/703:48:457
278892299128,3cyclictest27642-21gmain04:44:077
278892299127,4cyclictest2883699-21latency_hist04:08:457
278892299126,5cyclictest2798960-21latency_hist03:28:457
278892299124,7cyclictest27924-21gmain04:09:077
2788922991211,0cyclictest0-21swapper/708:49:407
278892299120,11cyclictest4175659-21snmpd08:14:287
278891899122,6cyclictest86-21ksoftirqd/607:59:026
278891899122,6cyclictest86-21ksoftirqd/607:59:026
278891899122,6cyclictest86-21ksoftirqd/604:19:086
278891899122,6cyclictest86-21ksoftirqd/603:24:036
278891899122,5cyclictest86-21ksoftirqd/603:49:096
2788918991212,0cyclictest0-21swapper/603:34:176
2788918991211,1cyclictest841rcuc/607:49:066
2788918991211,1cyclictest3403645-21kthreadcore08:24:086
2788918991211,1cyclictest3367097-21kthreadcore08:04:066
2788918991211,1cyclictest3327650-21wc07:44:146
2788918991211,1cyclictest3146248-21sd-close06:18:446
2788918991211,1cyclictest3125143-21ntpq06:04:126
2788918991211,1cyclictest3026833-21cut05:19:056
2788918991211,1cyclictest2825659-21kthreadcore03:39:046
2788918991211,1cyclictest0-21swapper/607:54:056
2788918991210,1cyclictest3296893-21ntpq07:29:116
2788918991210,1cyclictest3024137-21ntpq05:14:116
2788918991210,1cyclictest0-21swapper/608:29:076
2788918991210,1cyclictest0-21swapper/605:59:156
278891899120,5cyclictest2836477-21kthreadcore03:44:076
278891899120,12cyclictest0-21swapper/603:57:556
2788913991211,1cyclictest0-21swapper/508:49:135
2788913991211,1cyclictest0-21swapper/506:06:045
2788913991211,1cyclictest0-21swapper/504:11:085
2788913991210,2cyclictest3146593-21kworker/u16:1+flush-8:006:34:595
278891399120,1cyclictest0-21swapper/505:48:435
278891399120,12cyclictest0-21swapper/505:30:455
278891399120,11cyclictest0-21swapper/505:04:035
278891399120,0cyclictest0-21swapper/508:16:545
278892299118,2cyclictest4175659-21snmpd07:15:217
278892299118,2cyclictest4175659-21snmpd05:38:547
278892299118,2cyclictest4175659-21snmpd04:30:327
278892299118,2cyclictest4175659-21snmpd04:02:257
278892299117,3cyclictest984852-40tracker-miner-f06:19:107
278892299117,3cyclictest4175659-21snmpd05:02:157
278892299117,3cyclictest4175659-21snmpd05:02:157
278892299117,3cyclictest3009607-21kthreadcore05:09:087
278892299117,3cyclictest27204-21gmain06:49:117
278892299115,4cyclictest2858251-21grep03:54:057
278892299114,6cyclictest0-21swapper/706:53:537
278892299110,2cyclictest3185496-21cat06:34:107
278892299110,2cyclictest2904330-21/usr/sbin/munin04:19:117
278892299110,10cyclictest4175659-21snmpd07:40:367
278891899119,2cyclictest2873565-21kworker/u16:2+events_unbound04:27:006
278891899119,1cyclictest3086181-21cpuspeed_turbos05:48:586
278891899119,1cyclictest0-21swapper/608:49:096
278891899119,1cyclictest0-21swapper/608:44:056
278891899118,2cyclictest2975533-21df04:54:036
278891899111,6cyclictest86-21ksoftirqd/605:09:066
278891899111,6cyclictest86-21ksoftirqd/604:49:066
2788918991110,1cyclictest3177857-21irqcore06:34:036
2788918991110,1cyclictest2894199-21/usr/sbin/munin04:14:016
2788918991110,1cyclictest0-21swapper/608:20:346
2788918991110,1cyclictest0-21swapper/607:01:446
2788918991110,1cyclictest0-21swapper/603:28:496
278891899110,5cyclictest3077749-21cut05:44:056
278891899110,4cyclictest3300976-21kthreadcore07:34:046
278891899110,4cyclictest3149823-21cut06:19:056
278891899110,11cyclictest0-21swapper/608:33:486
278891399119,2cyclictest3237137-21kworker/u16:2+events_unbound07:24:225
278891399118,2cyclictest3139374-21kthreadcore06:14:065
2788913991110,1cyclictest0-21swapper/508:34:245
2788913991110,1cyclictest0-21swapper/507:28:595
2788913991110,1cyclictest0-21swapper/506:57:375
2788913991110,1cyclictest0-21swapper/506:42:285
2788913991110,1cyclictest0-21swapper/506:02:355
278891399110,11cyclictest0-21swapper/508:42:365
278891399110,11cyclictest0-21swapper/508:42:365
278891399110,11cyclictest0-21swapper/506:23:055
278891399110,11cyclictest0-21swapper/504:27:415
278891399110,0cyclictest0-21swapper/505:00:105
278891399110,0cyclictest0-21swapper/505:00:105
278891399110,0cyclictest0-21swapper/503:52:045
278891399110,0cyclictest0-21swapper/503:47:585
278892299100,4cyclictest4175659-21snmpd07:59:047
278892299100,4cyclictest4175659-21snmpd07:59:047
278891899109,1cyclictest0-21swapper/606:39:006
278891899109,1cyclictest0-21swapper/606:09:056
278891899108,1cyclictest3056530-21smartctl05:34:016
278891399109,1cyclictest0-21swapper/508:25:165
278891399109,1cyclictest0-21swapper/508:06:585
278891399109,1cyclictest0-21swapper/506:32:235
278891399109,1cyclictest0-21swapper/506:09:325
278891399109,1cyclictest0-21swapper/503:26:315
278891399108,2cyclictest3338936-21kworker/u16:1+flush-8:008:09:585
27889229996,2cyclictest0-21swapper/708:03:567
27889139998,1cyclictest0-21swapper/506:24:495
27889139994,4cyclictest3086852-21cat05:49:015
27889139994,4cyclictest2933902-21grep04:29:185
27889139994,4cyclictest2855027-21tail03:54:035
27889139987,1cyclictest0-21swapper/504:20:175
27889139987,1cyclictest0-21swapper/504:07:495
27889139987,1cyclictest0-21swapper/503:35:405
27889139986,1cyclictest0-21swapper/506:59:085
27889139986,1cyclictest0-21swapper/505:54:045
27889139986,1cyclictest0-21swapper/505:19:085
27889139983,4cyclictest3350619-21cut07:59:035
27889139983,4cyclictest3350619-21cut07:59:035
27889139983,4cyclictest3320461-21cut07:44:035
27889139983,4cyclictest3057512-21grep05:34:035
27889139980,7cyclictest3040650-21grep05:24:055
27889139980,7cyclictest0-21swapper/506:48:055
27889139974,2cyclictest2977574-21kthreadcore04:54:085
27889139973,3cyclictest3015789-21cat05:13:595
27889139972,4cyclictest3440087-21cat08:43:585
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional