You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2025-05-18 - 06:19
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Sun May 18, 2025 00:43:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
9085625421,29sleep70-21swapper/703:33:377
9099425242,7sleep50-21swapper/503:35:355
8164225241,7sleep60-21swapper/603:31:116
9098424635,8sleep40-21swapper/403:35:264
91349994239,2cyclictest91093-21kworker/u16:2+flush-8:004:16:186
91340993735,2cyclictest197381-21kworker/u16:0+events_unbound05:16:374
91349993634,2cyclictest577182-21kworker/u16:4+flush-8:007:41:076
91349993633,2cyclictest310564-21kworker/u16:2+events_unbound05:31:106
9135399350,33cyclictest679711-21latency_hist08:21:077
9135399320,30cyclictest413078-21latency_hist06:11:087
91344993129,2cyclictest238773-21kworker/u16:3+events_unbound06:48:495
91344993129,2cyclictest238773-21kworker/u16:3+events_unbound06:48:495
91353993029,0cyclictest0-21swapper/705:41:227
91349993028,2cyclictest651544-21kworker/u16:5+flush-8:008:51:076
91344993028,2cyclictest577182-21kworker/u16:4+events_unbound07:39:185
91340993028,2cyclictest167645-21kworker/u16:4+events_unbound04:26:374
91340993028,2cyclictest167645-21kworker/u16:4+events_unbound04:26:374
9135399290,27cyclictest689949-21latency_hist08:26:077
91344992927,2cyclictest433663-21kworker/u16:2+events_unbound07:42:185
91344992927,2cyclictest238773-21kworker/u16:3+events_unbound07:00:585
91344992826,2cyclictest91093-21kworker/u16:2+events_unbound03:48:175
91344992826,2cyclictest710606-21kworker/u16:2+events_unbound09:06:015
91344992826,2cyclictest238773-21kworker/u16:3+flush-8:006:36:495
91344992826,2cyclictest167645-21kworker/u16:4+events_unbound04:28:055
91344992826,2cyclictest167645-21kworker/u16:4+events_unbound04:28:055
91344992725,2cyclictest433663-21kworker/u16:2+flush-8:007:04:575
91340992725,2cyclictest495235-21kworker/u16:5+events_unbound07:18:494
91340992725,2cyclictest310564-21kworker/u16:2+events_unbound05:26:094
91340992725,2cyclictest167639-21kworker/u16:1+flush-8:005:56:074
91340992724,2cyclictest557356-21kworker/u16:1+events_unbound07:31:084
91353992623,2cyclictest156699-21munin-run04:06:087
91349992624,2cyclictest423482-21kworker/u16:0+flush-8:006:31:236
91344992624,2cyclictest651544-21kworker/u16:5+flush-8:008:52:175
91344992624,2cyclictest648897-21kworker/u16:3+flush-8:008:30:575
91344992624,2cyclictest423482-21kworker/u16:0+events_unbound06:45:185
91344992624,2cyclictest167639-21kworker/u16:1+events_unbound06:19:175
91340992624,2cyclictest710606-21kworker/u16:2+events_unbound08:56:364
91340992624,2cyclictest238773-21kworker/u16:3+events_unbound06:21:244
9135399250,24cyclictest648841-21latency_hist08:06:087
91349992523,2cyclictest197381-21kworker/u16:0+flush-8:005:16:026
91349992523,2cyclictest167645-21kworker/u16:4+flush-8:004:31:036
91349992523,2cyclictest167645-21kworker/u16:4+flush-8:004:31:036
91344992523,2cyclictest91093-21kworker/u16:2+events_unbound04:41:185
91344992523,2cyclictest678769-21kworker/u16:1+events_unbound08:50:175
91344992523,2cyclictest651544-21kworker/u16:5+flush-8:008:31:295
91344992523,2cyclictest433663-21kworker/u16:2+events_unbound07:54:495
91344992523,2cyclictest433663-21kworker/u16:2+events_unbound07:28:185
91344992523,2cyclictest4133-21kworker/u16:3+events_unbound03:41:265
91344992523,2cyclictest238773-21kworker/u16:3+events_unbound06:34:495
91344992523,2cyclictest238773-21kworker/u16:3+events_unbound06:24:185
91344992523,2cyclictest238773-21kworker/u16:3+events_unbound06:07:195
91344992523,2cyclictest197381-21kworker/u16:0+events_unbound05:09:175
91344992523,2cyclictest167639-21kworker/u16:1+events_unbound05:42:465
91344992523,2cyclictest167639-21kworker/u16:1+events_unbound04:17:185
91344992523,2cyclictest145503-21kworker/u16:3+events_unbound04:10:185
91340992523,2cyclictest557356-21kworker/u16:1+flush-8:007:47:574
91340992523,2cyclictest238773-21kworker/u16:3+flush-8:004:50:184
9135399240,22cyclictest643-21systemd-journal05:21:077
91349992422,2cyclictest598356-21kworker/u16:0+flush-8:008:09:376
91349992422,2cyclictest433663-21kworker/u16:2+events_unbound06:21:306
91344992422,2cyclictest760853-21kworker/u16:0+flush-8:008:59:175
91344992422,2cyclictest648897-21kworker/u16:3+events_unbound08:36:345
91344992422,2cyclictest598356-21kworker/u16:0+events_unbound08:17:585
91344992422,2cyclictest598356-21kworker/u16:0+events_unbound08:14:175
91344992422,2cyclictest442869-21kworker/u16:4+events_unbound06:27:585
91344992422,2cyclictest197381-21kworker/u16:0+events_unbound05:05:175
91353992321,1cyclictest0-21swapper/707:56:397
91344992321,2cyclictest495235-21kworker/u16:5+flush-8:007:11:495
91344992321,2cyclictest433663-21kworker/u16:2+events_unbound07:08:185
91344992321,2cyclictest433663-21kworker/u16:2+events_unbound06:55:265
91344992321,2cyclictest238773-21kworker/u16:3+events_unbound04:51:285
91344992321,2cyclictest197381-21kworker/u16:0+events_unbound05:17:185
91344992320,2cyclictest91093-21kworker/u16:2+events_unbound03:51:185
91344992320,2cyclictest167645-21kworker/u16:4+flush-8:004:21:175
91340992321,2cyclictest91093-21kworker/u16:2+flush-8:004:36:274
91340992321,2cyclictest423482-21kworker/u16:0+events_unbound07:01:314
91340992321,2cyclictest238773-21kworker/u16:3+flush-8:005:11:374
9135399220,20cyclictest166907-21latency_hist04:11:087
91349992221,1cyclictest433663-21kworker/u16:2+events_unbound07:20:586
91349992220,2cyclictest4133-21kworker/u16:3+flush-8:003:41:216
91349992220,2cyclictest310564-21kworker/u16:2+flush-8:005:36:316
91349992219,2cyclictest91093-21kworker/u16:2+events_unbound04:36:216
91344992220,2cyclictest678769-21kworker/u16:1+events_unbound08:24:585
91344992220,2cyclictest238773-21kworker/u16:3+events_unbound07:35:185
91344992220,2cyclictest238773-21kworker/u16:3+events_unbound05:58:575
91340992220,2cyclictest648897-21kworker/u16:3+events_unbound08:39:184
91340992220,2cyclictest598356-21kworker/u16:0+flush-8:008:27:094
91340992220,2cyclictest238773-21kworker/u16:3+events_unbound05:36:354
91349992119,2cyclictest238773-21kworker/u16:3+events_unbound06:41:336
91349992119,2cyclictest197381-21kworker/u16:0+events_unbound04:31:306
91349992119,2cyclictest167639-21kworker/u16:1+flush-8:004:21:296
91349992119,2cyclictest167639-21kworker/u16:1+events_unbound05:26:376
91344992119,2cyclictest4133-21kworker/u16:3+flush-8:003:36:185
91344992119,2cyclictest167639-21kworker/u16:1+events_unbound05:32:305
91340992118,2cyclictest598356-21kworker/u16:0+flush-8:008:11:254
91353992019,0cyclictest0-21swapper/706:30:257
91353992019,0cyclictest0-21swapper/703:52:387
91353992016,2cyclictest331043-21munin-run05:31:087
91349992018,2cyclictest360910-21kworker/u16:5+events_unbound06:18:496
91344992018,2cyclictest280004-21kworker/u16:4+events_unbound05:23:175
91344992018,2cyclictest238773-21kworker/u16:3+events_unbound06:03:375
91344992018,2cyclictest167639-21kworker/u16:1+events_unbound05:53:095
91344992018,2cyclictest167639-21kworker/u16:1+events_unbound04:58:055
91344992017,2cyclictest360910-21kworker/u16:5+events_unbound06:11:105
91340992018,2cyclictest710606-21kworker/u16:2+events_unbound08:54:174
91340992018,2cyclictest557356-21kworker/u16:1+events_unbound07:23:214
91340992018,2cyclictest423482-21kworker/u16:0+events_unbound07:28:494
91340992018,2cyclictest423482-21kworker/u16:0+events_unbound06:56:384
91340992018,2cyclictest423482-21kworker/u16:0+events_unbound06:40:504
91340992018,2cyclictest310564-21kworker/u16:2+events_unbound05:46:384
91340992018,2cyclictest145503-21kworker/u16:3+events_unbound03:56:414
91353991918,0cyclictest0-21swapper/707:03:057
91349991917,2cyclictest238773-21kworker/u16:3+flush-8:004:56:386
91349991917,2cyclictest135909-21kworker/u16:0+flush-8:004:06:246
91349991916,2cyclictest648897-21kworker/u16:3+flush-8:008:21:296
91344991917,2cyclictest651544-21kworker/u16:5+events_unbound08:07:095
91344991917,2cyclictest557356-21kworker/u16:1+flush-8:008:01:575
91344991917,2cyclictest433663-21kworker/u16:2+events_unbound07:50:505
91344991917,2cyclictest167639-21kworker/u16:1+events_unbound05:30:495
91340991917,2cyclictest577182-21kworker/u16:4+events_unbound07:40:504
91353991816,1cyclictest710577-21latency_hist08:36:077
91353991816,1cyclictest0-21swapper/706:16:337
91353991815,2cyclictest443810-21awk06:26:087
9135399180,16cyclictest515604-21latency_hist07:01:077
91349991816,2cyclictest91924-21kworker/u16:4+events_unbound03:52:096
91349991816,2cyclictest720825-21kworker/u16:4+flush-8:009:01:246
91349991816,2cyclictest710606-21kworker/u16:2+events_unbound08:40:506
91349991816,2cyclictest577182-21kworker/u16:4+flush-8:007:46:026
91349991816,2cyclictest360910-21kworker/u16:5+flush-8:006:15:176
91349991816,2cyclictest167639-21kworker/u16:1+flush-8:006:06:076
91349991815,2cyclictest648897-21kworker/u16:3+events_unbound08:41:086
91344991816,2cyclictest91093-21kworker/u16:2+events_unbound04:15:585
91344991816,2cyclictest197381-21kworker/u16:0+events_unbound05:15:585
91344991816,2cyclictest197381-21kworker/u16:0+events_unbound04:40:085
91340991816,2cyclictest648897-21kworker/u16:3+events_unbound08:19:184
91340991816,2cyclictest557356-21kworker/u16:1+flush-8:008:01:464
91340991816,2cyclictest474358-21kworker/u16:1+flush-8:006:46:314
91340991816,2cyclictest474358-21kworker/u16:1+flush-8:006:46:314
91340991816,2cyclictest474358-21kworker/u16:1+events_unbound06:44:574
91340991816,2cyclictest360910-21kworker/u16:5+events_unbound05:56:354
91340991816,2cyclictest351230-21kworker/u16:0+events_unbound05:41:304
91340991816,2cyclictest238773-21kworker/u16:3+events_unbound06:14:184
91340991816,2cyclictest167639-21kworker/u16:1+flush-8:004:16:244
91340991815,2cyclictest238773-21kworker/u16:3+events_unbound07:41:084
91349991715,2cyclictest423482-21kworker/u16:0+events_unbound06:51:356
91349991714,2cyclictest91093-21kworker/u16:2+events_unbound04:11:186
91349991714,2cyclictest648897-21kworker/u16:3+events_unbound08:51:086
91349991714,2cyclictest135909-21kworker/u16:0+events_unbound04:01:186
9134999170,16cyclictest635460-21kthreadcore07:56:346
91344991715,2cyclictest598356-21kworker/u16:0+flush-8:008:01:055
91344991715,2cyclictest197381-21kworker/u16:0+events_unbound04:33:025
91344991715,2cyclictest167639-21kworker/u16:1+events_unbound04:46:585
91340991715,2cyclictest495235-21kworker/u16:5+events_unbound06:51:294
91340991715,2cyclictest360910-21kworker/u16:5+events_unbound06:10:494
91353991615,0cyclictest0-21swapper/708:41:517
9135399160,15cyclictest659138-21latency_hist08:11:077
9135399160,15cyclictest1313-21snmpd08:12:587
9135399160,14cyclictest402864-21latency_hist06:06:077
9135399160,14cyclictest1313-21snmpd07:12:147
91344991614,2cyclictest167639-21kworker/u16:1+events_unbound05:46:095
91344991614,2cyclictest145503-21kworker/u16:3+events_unbound04:04:595
91340991614,2cyclictest433663-21kworker/u16:2+events_unbound07:10:504
91340991614,2cyclictest4133-21kworker/u16:3+flush-8:003:36:304
91340991614,2cyclictest135909-21kworker/u16:0+flush-8:004:06:264
9134099160,16cyclictest0-21swapper/406:01:234
91353991514,0cyclictest0-21swapper/706:49:117
91353991514,0cyclictest0-21swapper/706:49:117
91353991514,0cyclictest0-21swapper/705:59:447
91353991514,0cyclictest0-21swapper/704:47:337
91353991514,0cyclictest0-21swapper/703:50:207
9135399150,14cyclictest1313-21snmpd06:32:197
91349991513,2cyclictest598356-21kworker/u16:0+flush-8:008:26:386
91349991513,2cyclictest310564-21kworker/u16:2+flush-8:006:06:336
91349991513,2cyclictest197381-21kworker/u16:0+events_unbound05:06:296
91349991512,2cyclictest577182-21kworker/u16:4+flush-8:007:46:176
91344991513,2cyclictest238773-21kworker/u16:3+events_unbound05:40:415
91344991513,2cyclictest135909-21kworker/u16:0+events_unbound04:00:105
91344991512,2cyclictest678769-21kworker/u16:1+events_unbound08:41:315
91340991513,2cyclictest433663-21kworker/u16:2+events_unbound07:56:404
91340991513,2cyclictest360910-21kworker/u16:5+events_unbound06:18:184
91340991513,2cyclictest238773-21kworker/u16:3+flush-8:005:31:234
91340991513,2cyclictest167645-21kworker/u16:4+flush-8:004:21:284
91340991513,2cyclictest167639-21kworker/u16:1+events_unbound05:01:224
9135399143,9cyclictest237948-21smart_sda04:41:367
9135399142,10cyclictest299337-21ntpq05:11:347
91353991414,0cyclictest0-21swapper/708:29:557
91353991413,0cyclictest0-21swapper/708:39:167
91353991413,0cyclictest0-21swapper/706:52:467
91353991413,0cyclictest0-21swapper/705:34:047
91353991413,0cyclictest0-21swapper/704:36:117
91353991413,0cyclictest0-21swapper/703:36:347
9135399141,11cyclictest0-21swapper/705:21:307
9135399140,13cyclictest96-21ksoftirqd/707:06:367
9135399140,13cyclictest1313-21snmpd06:11:367
91349991412,2cyclictest91093-21kworker/u16:2+flush-8:004:46:086
91349991412,2cyclictest433663-21kworker/u16:2+events_unbound07:09:186
91349991412,2cyclictest423482-21kworker/u16:0+events_unbound06:28:186
91349991412,2cyclictest38622-21kworker/u16:1+events_unbound03:58:176
91349991412,2cyclictest310564-21kworker/u16:2+events_unbound05:49:176
91344991412,2cyclictest423482-21kworker/u16:0+events_unbound07:26:055
91340991412,2cyclictest91093-21kworker/u16:2+events_unbound03:55:094
91340991412,2cyclictest678769-21kworker/u16:1+flush-8:008:26:024
91340991412,2cyclictest310564-21kworker/u16:2+events_unbound05:22:174
91340991412,2cyclictest145503-21kworker/u16:3+flush-8:004:01:294
9135399132,2cyclictest350788-21smart_sda05:36:367
9135399132,1cyclictest0-21swapper/707:16:227
91353991312,0cyclictest0-21swapper/707:21:167
91353991312,0cyclictest0-21swapper/704:12:097
9135399130,4cyclictest253395-21grep04:51:277
9135399130,12cyclictest1313-21snmpd07:51:397
9135399130,11cyclictest1313-21snmpd05:51:187
91349991312,1cyclictest0-21swapper/608:56:296
91349991311,2cyclictest557356-21kworker/u16:1+flush-8:007:26:386
91349991311,2cyclictest238773-21kworker/u16:3+flush-8:004:56:036
91349991311,2cyclictest238773-21kworker/u16:3+events_unbound05:59:176
91349991311,1cyclictest673884-21kthreadcore08:16:276
91340991311,2cyclictest91924-21kworker/u16:4+events_unbound03:41:114
91340991311,2cyclictest433663-21kworker/u16:2+flush-8:006:31:494
91340991311,2cyclictest167645-21kworker/u16:4+events_unbound04:11:224
9135399123,7cyclictest0-21swapper/704:21:237
9135399121,2cyclictest114752-21ntpq03:41:337
91353991211,0cyclictest0-21swapper/708:46:527
91353991211,0cyclictest0-21swapper/706:39:327
91353991211,0cyclictest0-21swapper/705:46:507
91353991211,0cyclictest0-21swapper/705:01:127
91353991211,0cyclictest0-21swapper/704:56:137
91353991211,0cyclictest0-21swapper/704:18:157
91353991211,0cyclictest0-21swapper/703:56:137
9135399120,3cyclictest204063-21kthreadcore04:26:297
9135399120,3cyclictest204063-21kthreadcore04:26:297
9135399120,11cyclictest96-21ksoftirqd/707:26:397
9135399120,11cyclictest289257-21smart_sda05:06:377
9135399120,10cyclictest756603-21kthreadcore08:56:327
9135399120,10cyclictest1313-21snmpd07:36:097
91349991210,2cyclictest91924-21kworker/u16:4+events_unbound03:38:056
91349991210,2cyclictest433663-21kworker/u16:2+events_unbound06:36:366
9134999120,1cyclictest0-21swapper/607:14:576
91344991210,2cyclictest238773-21kworker/u16:3+events_unbound07:18:085
91340991211,1cyclictest0-21swapper/408:50:584
91340991210,2cyclictest91093-21kworker/u16:2+flush-8:003:47:084
91340991210,2cyclictest557356-21kworker/u16:1+events_unbound07:53:174
91340991210,2cyclictest167645-21kworker/u16:4+events_unbound04:41:264
9134099120,1cyclictest0-21swapper/404:34:484
9135399119,1cyclictest772119-21turbostat09:06:067
91353991111,0cyclictest0-21swapper/706:45:507
91353991110,0cyclictest0-21swapper/704:31:207
9135399110,9cyclictest0-21swapper/707:46:227
9135399110,10cyclictest587365-21turbostat07:36:077
9135399110,10cyclictest1313-21snmpd08:55:597
9134999119,2cyclictest598356-21kworker/u16:0+flush-8:008:31:346
9134999119,2cyclictest577182-21kworker/u16:4+events_unbound08:03:586
9134999119,2cyclictest495235-21kworker/u16:5+flush-8:007:21:256
9134999119,2cyclictest238773-21kworker/u16:3+flush-8:007:31:386
9134999116,4cyclictest669050-21grep08:11:376
91349991110,1cyclictest0-21swapper/607:02:496
91349991110,1cyclictest0-21swapper/606:59:406
91349991110,1cyclictest0-21swapper/605:54:336
9134099119,2cyclictest720825-21kworker/u16:4+events_unbound09:01:254
9134099119,2cyclictest238773-21kworker/u16:3+events_unbound06:27:174
9134099119,2cyclictest197381-21kworker/u16:0+flush-8:005:06:304
91340991110,1cyclictest0-21swapper/408:44:404
9135399100,8cyclictest0-21swapper/707:41:267
9134999109,1cyclictest0-21swapper/605:16:436
9134999108,2cyclictest38622-21kworker/u16:1+flush-8:003:46:576
9134999108,2cyclictest238773-21kworker/u16:3+events_unbound04:46:416
9134099109,1cyclictest0-21swapper/404:52:274
9134099108,2cyclictest598356-21kworker/u16:0+events_unbound08:31:584
913499997,2cyclictest494144-21kworker/u16:4+events_unbound06:51:026
913499997,2cyclictest494144-21kworker/u16:4+events_unbound06:51:026
913409997,2cyclictest167639-21kworker/u16:1+flush-8:004:57:384
913409990,9cyclictest0-21swapper/408:08:474
913409990,1cyclictest0-21swapper/407:11:234
913499986,1cyclictest0-21swapper/605:21:286
913499984,3cyclictest353387-21irqcore05:41:246
913499983,4cyclictest624831-21cut07:51:336
913499975,1cyclictest0-21swapper/605:01:246
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional