You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-07-04 - 12:10
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack0slot2.osadl.org (updated Fri Jul 04, 2025 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
300964825141,7sleep50-21swapper/503:31:325
300952325117,30sleep70-21swapper/703:29:497
300950224130,8sleep40-21swapper/403:29:304
301005799380,36cyclictest3168456-21latency_hist04:47:597
300959923828,7sleep60-21swapper/603:30:536
3010057993533,1cyclictest3488055-21sed07:22:597
3010057993533,1cyclictest3488055-21sed07:22:597
3010056993432,2cyclictest3570379-21kworker/u16:2+events_unbound08:34:096
3010056993432,2cyclictest3241327-21kworker/u16:4+events_unbound06:18:246
3010056993330,2cyclictest2932650-21kworker/u16:2+events_unbound04:27:186
301005799320,30cyclictest643-21systemd-journal05:57:597
301005799320,30cyclictest3075690-21latency_hist04:02:597
3010056993230,2cyclictest3621028-21kworker/u16:4+events_unbound08:27:096
3010056993230,2cyclictest3301448-21kworker/u16:2+events_unbound07:06:056
3010056993230,2cyclictest2932650-21kworker/u16:2+events_unbound05:02:096
3010056993230,2cyclictest2932650-21kworker/u16:2+events_unbound05:02:096
3010057993129,1cyclictest3034661-21sed03:43:007
3010056993129,2cyclictest3271267-21kworker/u16:0+events_unbound06:53:096
3010056993129,2cyclictest3260194-21kworker/u16:6+events_unbound05:39:096
3010056993129,2cyclictest2998018-21kworker/u16:3+events_unbound05:29:296
3010056993129,2cyclictest2998018-21kworker/u16:3+events_unbound03:54:146
3010056993128,2cyclictest3363974-21kworker/u16:1+events_unbound07:33:096
3010050993128,2cyclictest2998018-21kworker/u16:3+flush-8:003:33:154
301005799300,28cyclictest3673585-21systemd-run08:52:587
3010056993028,2cyclictest3559163-21kworker/u16:0+flush-8:008:17:086
3010056993028,2cyclictest3455669-21kworker/u16:4+events_unbound07:17:096
3010056993028,2cyclictest3271267-21kworker/u16:0+events_unbound06:45:536
3010056992927,2cyclictest3260194-21kworker/u16:6+events_unbound05:45:096
3010056992927,2cyclictest3260194-21kworker/u16:6+events_unbound05:45:096
3010056992927,2cyclictest3198624-21kworker/u16:0+events_unbound05:20:086
3010056992927,2cyclictest2932650-21kworker/u16:2+events_unbound05:03:086
3010056992926,2cyclictest3198624-21kworker/u16:0+events_unbound05:23:096
3010050992927,2cyclictest3168150-21kworker/u16:1+flush-8:004:48:144
3010056992826,2cyclictest3528213-21kworker/u16:5+events_unbound07:47:096
3010056992826,2cyclictest3301448-21kworker/u16:2+events_unbound06:09:386
3010056992725,2cyclictest3621028-21kworker/u16:4+events_unbound08:29:106
3010056992725,2cyclictest3363974-21kworker/u16:1+events_unbound07:59:186
3010056992725,2cyclictest3301448-21kworker/u16:2+events_unbound07:19:096
3010056992725,2cyclictest3301448-21kworker/u16:2+events_unbound07:19:096
3010056992725,2cyclictest3301448-21kworker/u16:2+events_unbound06:27:496
3010056992725,2cyclictest3168150-21kworker/u16:1+events_unbound04:44:186
3010056992725,2cyclictest2998018-21kworker/u16:3+events_unbound05:10:416
3010056992624,2cyclictest3673586-21kworker/u16:2+events_unbound08:55:006
3010056992624,2cyclictest3673586-21kworker/u16:2+events_unbound08:55:006
3010056992624,2cyclictest3280787-21kworker/u16:1+events_unbound05:52:016
3010056992624,2cyclictest3198624-21kworker/u16:0+flush-8:005:14:016
3010056992624,2cyclictest3019283-21kworker/u16:1+events_unbound03:38:096
3010056992624,2cyclictest2932650-21kworker/u16:2+events_unbound03:48:096
3010056992623,2cyclictest3446266-21kworker/u16:0+flush-8:007:08:106
3010052992622,3cyclictest3241327-21kworker/u16:4+flush-8:006:23:245
3010050992624,2cyclictest3570379-21kworker/u16:2+flush-8:008:23:154
3010050992624,2cyclictest3241327-21kworker/u16:4+events_unbound05:23:194
3010057992524,0cyclictest0-21swapper/708:23:247
301005799250,23cyclictest3096245-21latency_hist04:12:597
301005799250,23cyclictest3096245-21latency_hist04:12:597
301005799250,23cyclictest1011-21gdbus06:07:597
3010056992523,2cyclictest3363974-21kworker/u16:1+flush-8:007:50:206
3010056992523,2cyclictest3301448-21kworker/u16:2+flush-8:006:28:096
3010056992523,2cyclictest3241327-21kworker/u16:4+flush-8:005:57:016
3010056992523,2cyclictest2998018-21kworker/u16:3+events_unbound04:53:376
3010056992522,2cyclictest3241327-21kworker/u16:4+events_unbound06:03:136
3010050992523,2cyclictest3528212-21kworker/u16:3+events_unbound07:48:124
3010050992523,2cyclictest2932650-21kworker/u16:2+events_unbound04:27:324
3010056992422,2cyclictest3446266-21kworker/u16:0+events_unbound06:59:446
3010056992422,2cyclictest3363974-21kworker/u16:1+events_unbound06:49:096
3010056992422,2cyclictest3271267-21kworker/u16:0+events_unbound05:35:446
3010050992422,2cyclictest3559163-21kworker/u16:0+events_unbound08:30:404
3010050992422,2cyclictest3528212-21kworker/u16:3+flush-8:007:43:144
3010056992321,2cyclictest3559163-21kworker/u16:0+events_unbound08:42:416
3010056992321,2cyclictest3019283-21kworker/u16:1+events_unbound04:22:176
3010052992319,3cyclictest3198624-21kworker/u16:0+flush-8:005:18:155
3010056992220,2cyclictest3653611-21kworker/u16:3+events_unbound09:02:576
3010056992220,2cyclictest3241327-21kworker/u16:4+events_unbound06:37:086
3010050992220,2cyclictest3198624-21kworker/u16:0+events_unbound05:17:164
3010050992220,2cyclictest3019283-21kworker/u16:1+flush-8:004:28:334
3010050992220,2cyclictest2998018-21kworker/u16:3+events_unbound04:54:364
3010050992220,2cyclictest2998018-21kworker/u16:3+events_unbound03:53:574
3010050992220,2cyclictest2932650-21kworker/u16:2+flush-8:003:58:154
3010056992119,2cyclictest3570379-21kworker/u16:2+events_unbound08:06:266
3010056992119,2cyclictest3528212-21kworker/u16:3+events_unbound07:41:126
3010056992119,2cyclictest3497926-21kworker/u16:3+events_unbound07:30:016
3010056992119,2cyclictest2998018-21kworker/u16:3+events_unbound04:41:186
3010050992119,2cyclictest3301448-21kworker/u16:2+events_unbound06:00:454
3010050992118,2cyclictest3652739-21kworker/u16:1+events_unbound08:43:034
3010050992118,2cyclictest3363974-21kworker/u16:1+events_unbound06:38:004
3010056992018,2cyclictest3559163-21kworker/u16:0+events_unbound08:21:126
3010056992018,2cyclictest3301448-21kworker/u16:2+events_unbound06:16:056
3010056992018,2cyclictest3019283-21kworker/u16:1+flush-8:004:04:136
3010056992018,2cyclictest3019283-21kworker/u16:1+events_unbound04:17:496
3010056992018,2cyclictest2998018-21kworker/u16:3+events_unbound04:35:216
3010056992017,3cyclictest3559163-21kworker/u16:0+flush-8:008:12:596
3010052992017,2cyclictest3019283-21kworker/u16:1+flush-8:004:03:165
3010050992018,2cyclictest3528213-21kworker/u16:5+events_unbound08:08:004
3010050992018,2cyclictest3497926-21kworker/u16:3+events_unbound07:26:524
3010050992018,2cyclictest3332349-21kworker/u16:3+events_unbound06:04:334
3010050992018,2cyclictest3301448-21kworker/u16:2+events_unbound07:11:164
3010050992018,2cyclictest3301448-21kworker/u16:2+events_unbound06:10:004
3010050992018,2cyclictest2998018-21kworker/u16:3+events_unbound04:41:444
301005799190,18cyclictest1564-21dockerd04:28:017
3010056991917,2cyclictest3653611-21kworker/u16:3+events_unbound08:51:216
3010056991917,2cyclictest3652739-21kworker/u16:1+events_unbound08:45:206
3010056991917,2cyclictest3363974-21kworker/u16:1+events_unbound07:55:086
3010056991917,2cyclictest2943139-21kworker/u16:4+events_unbound03:33:226
3010050991917,2cyclictest3559163-21kworker/u16:0+events_unbound08:22:384
3010050991917,2cyclictest3395036-21kworker/u16:3+events_unbound06:43:014
3010050991917,2cyclictest3301448-21kworker/u16:2+events_unbound06:15:324
3010050991917,2cyclictest3271267-21kworker/u16:0+events_unbound06:51:354
3010050991917,2cyclictest3168150-21kworker/u16:1+events_unbound05:00:024
3010050991917,2cyclictest3168150-21kworker/u16:1+events_unbound05:00:024
3010050991917,2cyclictest3096189-21kworker/u16:0+events_unbound04:13:144
3010050991917,2cyclictest3019283-21kworker/u16:1+events_unbound04:10:084
3010050991917,2cyclictest3019283-21kworker/u16:1+events_unbound04:10:084
3010050991917,2cyclictest2932650-21kworker/u16:2+events_unbound03:51:574
3010050991912,7cyclictest0-21swapper/406:33:174
301005799180,17cyclictest643-21systemd-journal08:17:537
301005799180,16cyclictest3281793-21latency_hist05:42:597
3010056991816,2cyclictest3497926-21kworker/u16:3+events_unbound07:25:186
3010056991816,2cyclictest3096189-21kworker/u16:0+flush-8:004:28:156
3010056991816,2cyclictest2998018-21kworker/u16:3+events_unbound03:43:496
3010050991816,2cyclictest3528213-21kworker/u16:5+events_unbound08:15:164
3010050991816,2cyclictest3301448-21kworker/u16:2+events_unbound07:31:274
3010050991816,2cyclictest3301448-21kworker/u16:2+events_unbound06:24:044
3010050991816,2cyclictest3280787-21kworker/u16:1+flush-8:005:48:324
3010050991816,2cyclictest3241327-21kworker/u16:4+events_unbound06:29:234
3010050991816,2cyclictest3168150-21kworker/u16:1+flush-8:005:18:174
3010050991816,2cyclictest3136617-21kworker/u16:5+flush-8:004:36:084
3010050991816,2cyclictest3019283-21kworker/u16:1+events_unbound04:05:274
3010050991816,2cyclictest2998018-21kworker/u16:3+events_unbound04:47:164
301005799170,15cyclictest3127219-21latency_hist04:27:597
3010056991715,2cyclictest3395036-21kworker/u16:3+events_unbound06:39:136
3010056991715,2cyclictest3301448-21kworker/u16:2+events_unbound05:59:386
3010056991715,2cyclictest3064415-21kworker/u16:4+events_unbound04:09:506
3010056991715,2cyclictest3064415-21kworker/u16:4+events_unbound04:09:506
3010050991715,2cyclictest3559163-21kworker/u16:0+events_unbound09:01:234
3010050991715,2cyclictest3455669-21kworker/u16:4+events_unbound07:13:364
3010050991715,2cyclictest3271267-21kworker/u16:0+events_unbound06:18:324
3010050991715,2cyclictest3260194-21kworker/u16:6+events_unbound05:47:514
3010050991715,2cyclictest3260194-21kworker/u16:6+events_unbound05:47:514
3010050991715,2cyclictest3168150-21kworker/u16:1+events_unbound05:04:494
3010050991715,2cyclictest2932650-21kworker/u16:2+flush-8:003:38:174
3010050991715,2cyclictest2732652-21kworker/u16:0+events_unbound03:45:364
3010057991613,2cyclictest3611722-21awk08:22:597
301005799160,14cyclictest1313-21snmpd07:05:197
301005799160,14cyclictest1127-21ntpd06:34:237
3010056991614,2cyclictest2932650-21kworker/u16:2+events_unbound03:59:006
3010050991614,2cyclictest3528212-21kworker/u16:3+events_unbound07:41:274
3010050991614,2cyclictest3497894-21kworker/u16:0+events_unbound07:34:354
3010050991614,2cyclictest3395036-21kworker/u16:3+flush-8:006:53:484
3010050991614,2cyclictest3301448-21kworker/u16:2+flush-8:007:05:484
3010050991614,2cyclictest3271267-21kworker/u16:0+events_unbound05:55:544
3010050991614,2cyclictest3260194-21kworker/u16:6+events_unbound05:41:264
3010050991614,2cyclictest3241327-21kworker/u16:4+events_unbound05:29:274
3010050991614,2cyclictest3019283-21kworker/u16:1+events_unbound04:19:484
3010050991614,2cyclictest2998018-21kworker/u16:3+events_unbound05:34:414
3010050991613,2cyclictest3559163-21kworker/u16:0+flush-8:007:53:564
3010057991513,1cyclictest3477591-21idleruntime-cro07:17:587
301005799150,14cyclictest3199390-21turbostat05:02:597
301005799150,14cyclictest3199390-21turbostat05:02:597
301005799150,13cyclictest0-21swapper/707:38:237
3010052991512,2cyclictest3134948-21kthreadcore04:28:215
3010050991513,2cyclictest3621028-21kworker/u16:4+events_unbound08:35:474
3010050991513,2cyclictest3570379-21kworker/u16:2+flush-8:008:40:514
3010050991513,2cyclictest3446266-21kworker/u16:0+flush-8:007:00:304
3010050991513,2cyclictest3363974-21kworker/u16:1+events_unbound08:02:044
3010050991513,2cyclictest2998018-21kworker/u16:3+events_unbound05:10:234
3010050991512,2cyclictest3653611-21kworker/u16:3+flush-8:008:48:094
301005799148,5cyclictest3065406-21turbostat03:57:597
301005799140,13cyclictest1313-21snmpd04:15:097
301005799140,12cyclictest3343647-21munin-run06:12:597
301005799140,12cyclictest1313-21snmpd06:20:007
301005799140,12cyclictest1313-21snmpd06:13:187
3010056991412,2cyclictest2998018-21kworker/u16:3+events_unbound04:50:166
3010050991412,2cyclictest3559163-21kworker/u16:0+events_unbound08:04:354
301005799135,7cyclictest27204-21gmain08:28:127
3010057991310,2cyclictest3250887-21munin-run05:27:597
301005799130,4cyclictest3227694-21kthreadcore05:13:247
301005799130,12cyclictest1313-21snmpd07:51:187
301005799130,11cyclictest3426678-21sendmail06:53:007
3010052991310,2cyclictest3508734-21kworker/u16:4+flush-8:007:33:145
301005299130,13cyclictest0-21swapper/508:10:195
301005299130,13cyclictest0-21swapper/504:48:265
301005799126,5cyclictest27763-21gvfs-afc-volume08:33:197
3010057991212,0cyclictest0-21swapper/708:38:257
3010057991212,0cyclictest0-21swapper/703:45:307
3010057991210,1cyclictest3405330-21awk06:42:597
301005799120,3cyclictest3536382-21kthreadcore07:43:217
301005799120,3cyclictest3107275-21/usr/sbin/munin04:18:287
301005799120,11cyclictest1313-21snmpd07:33:177
301005799120,11cyclictest1313-21snmpd06:24:307
301005799120,11cyclictest1313-21snmpd05:20:237
301005799120,10cyclictest1313-21snmpd05:43:267
301005799120,10cyclictest1313-21snmpd05:43:267
3010052991211,1cyclictest0-21swapper/508:22:095
3010052991211,1cyclictest0-21swapper/507:19:245
3010052991211,1cyclictest0-21swapper/507:19:245
3010052991211,1cyclictest0-21swapper/507:11:035
3010052991211,1cyclictest0-21swapper/506:19:205
3010052991211,1cyclictest0-21swapper/503:47:035
3010052991210,2cyclictest3198624-21kworker/u16:0+flush-8:005:28:155
3010052991210,2cyclictest3198624-21kworker/u16:0+flush-8:005:03:165
301005299120,1cyclictest0-21swapper/508:14:165
301005299120,1cyclictest0-21swapper/506:37:285
301005299120,1cyclictest0-21swapper/506:07:215
301005299120,1cyclictest0-21swapper/504:37:495
301005299120,1cyclictest0-21swapper/503:56:505
301005299120,12cyclictest0-21swapper/507:59:175
301005299120,12cyclictest0-21swapper/507:14:125
301005299120,12cyclictest0-21swapper/506:38:345
301005299120,12cyclictest0-21swapper/505:35:405
3010050991210,2cyclictest3301448-21kworker/u16:2+events_unbound07:21:104
3010050991210,2cyclictest3301448-21kworker/u16:2+events_unbound07:21:104
301005799117,4cyclictest27763-21gvfs-afc-volume04:33:187
301005799117,3cyclictest3055114-21awk03:53:007
301005799115,5cyclictest27763-21gvfs-afc-volume05:08:187
301005799111,9cyclictest3010289-21cyclictest06:58:247
3010057991110,0cyclictest0-21swapper/708:58:167
3010057991110,0cyclictest0-21swapper/707:29:317
301005799110,9cyclictest3653599-21sendmail08:42:597
301005799110,9cyclictest0-21swapper/707:08:167
301005799110,2cyclictest3488624-21/usr/sbin/munin07:23:297
301005799110,10cyclictest3554829-21kthreadcore07:53:197
301005799110,10cyclictest1313-21snmpd08:07:007
301005799110,10cyclictest1313-21snmpd06:46:167
301005799110,10cyclictest1313-21snmpd05:58:367
301005799110,10cyclictest1313-21snmpd05:29:207
301005799110,10cyclictest1313-21snmpd05:04:047
301005799110,10cyclictest1313-21snmpd04:05:277
3010052991110,1cyclictest0-21swapper/508:38:515
3010052991110,1cyclictest0-21swapper/507:38:575
3010052991110,1cyclictest0-21swapper/506:56:235
3010052991110,1cyclictest0-21swapper/506:30:035
3010052991110,1cyclictest0-21swapper/506:10:405
301005299110,1cyclictest0-21swapper/505:14:235
301005299110,11cyclictest0-21swapper/508:34:555
301005299110,11cyclictest0-21swapper/507:27:365
301005299110,11cyclictest0-21swapper/507:01:355
301005299110,11cyclictest0-21swapper/505:51:075
301005299110,11cyclictest0-21swapper/503:41:425
301005099119,2cyclictest3673586-21kworker/u16:2+events_unbound08:53:174
301005099119,2cyclictest3673586-21kworker/u16:2+events_unbound08:53:174
301005799105,5cyclictest27924-21gmain07:58:287
301005799104,5cyclictest27394-21gmain08:53:157
301005799104,5cyclictest27394-21gmain08:53:157
3010057991010,0cyclictest0-21swapper/706:50:357
301005799100,9cyclictest1313-21snmpd08:08:247
301005799100,9cyclictest1313-21snmpd06:29:397
301005799100,9cyclictest1313-21snmpd03:36:267
301005799100,2cyclictest3179484-21/usr/sbin/munin04:53:287
301005799100,2cyclictest3157938-21taskset04:39:457
301005799100,2cyclictest0-21swapper/705:48:307
301005799100,1cyclictest0-21swapper/705:33:267
301005799100,1cyclictest0-21swapper/704:48:187
301005299109,1cyclictest0-21swapper/508:04:205
301005299109,1cyclictest0-21swapper/506:51:115
301005299109,1cyclictest0-21swapper/505:38:495
301005299109,1cyclictest0-21swapper/503:34:365
301005299100,10cyclictest0-21swapper/508:23:235
30100529998,1cyclictest0-21swapper/508:58:345
30100529998,1cyclictest0-21swapper/508:51:375
30100529998,1cyclictest0-21swapper/507:49:125
30100529998,1cyclictest0-21swapper/507:07:545
30100529998,1cyclictest0-21swapper/505:24:295
30100529998,1cyclictest0-21swapper/504:16:045
30100529990,9cyclictest0-21swapper/505:53:175
30100529990,1cyclictest0-21swapper/508:53:135
30100529990,1cyclictest0-21swapper/508:53:135
30100529987,1cyclictest0-21swapper/507:43:035
30100529987,1cyclictest0-21swapper/506:13:595
30100529987,1cyclictest0-21swapper/504:42:425
30100529985,2cyclictest3662212-21ntpq08:43:235
30100529984,3cyclictest3623532-21cat08:28:155
30100529983,4cyclictest3550810-21cstates07:53:115
30100529983,4cyclictest3118478-21sed04:23:195
30100529983,4cyclictest3095077-21cat04:08:245
30100529983,4cyclictest3095077-21cat04:08:245
30100529980,7cyclictest0-21swapper/507:28:265
30100529974,2cyclictest3406174-21cpuspeed_turbos06:43:125
30100529974,2cyclictest3067378-21kthreadcore03:58:175
30100529974,2cyclictest3046781-21kthreadcore03:48:215
30100529973,3cyclictest3283773-21kthreadcore05:43:195
30100529973,3cyclictest3283773-21kthreadcore05:43:195
30100529973,3cyclictest3190117-21cat04:58:135
30100529973,3cyclictest3190117-21cat04:58:135
30100529973,3cyclictest3160133-21uname04:43:185
30100529973,3cyclictest3107780-21df04:18:125
30100529970,1cyclictest0-21swapper/505:58:195
30100529962,3cyclictest3210909-21expr05:08:195
30100529960,1cyclictest0-21swapper/504:53:065
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional