You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2024-03-01 - 18:45
Click here to display the system's profile data or here to proceed to next system.
Click on a legend element to toggle display of that core, ctrl-click inverts display, shift-click enables all.

Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack0slot2.osadl.org (updated Fri Mar 01, 2024 12:43:30)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
388194825225,8sleep50-21swapper/507:02:235
38827022427,30sleep70-21swapper/707:05:067
3883080993127,3cyclictest4021350-21latency_hist08:16:395
3883080993127,3cyclictest4021350-21latency_hist08:16:395
388264323121,7sleep40-21swapper/407:04:154
3883087992827,0cyclictest0-21swapper/712:32:267
388262622820,5sleep60-21swapper/607:04:016
3883077992724,3cyclictest3864858-21kworker/u16:4+flush-8:007:47:294
3883077992521,3cyclictest144128-21kworker/u16:1+flush-8:012:27:054
3883080992420,3cyclictest3925398-21kworker/u16:3+flush-8:008:07:085
3883080992321,2cyclictest4098075-21kworker/u16:2+flush-8:009:22:075
3883080992119,2cyclictest67663-21kworker/u16:1+flush-8:010:37:165
3883077992119,2cyclictest4098075-21kworker/u16:2+flush-8:009:27:024
2029582210,1sleep70-21swapper/711:32:077
3883087992019,0cyclictest0-21swapper/712:27:037
3883080992018,2cyclictest177772-21kworker/u16:0+flush-8:011:37:195
3883080992018,2cyclictest144128-21kworker/u16:1+flush-8:011:42:015
3883077992018,2cyclictest4136600-21kworker/u16:0+flush-8:009:32:194
3883077992016,3cyclictest115522-21kworker/u16:4+events_unbound11:07:014
3883087991918,0cyclictest0-21swapper/707:17:087
3883087991915,3cyclictest143958-21turbostat11:06:337
3883087991915,3cyclictest143958-21turbostat11:06:337
3883085991915,3cyclictest3925180-21latency_hist07:26:396
3883080991917,2cyclictest4164917-21kworker/u16:3+events_unbound10:10:255
3883080991917,2cyclictest3954437-21kworker/u16:0+flush-8:008:32:055
3883080991917,2cyclictest3925397-21kworker/u16:1+events_unbound07:37:125
3883080991917,2cyclictest144128-21kworker/u16:1+flush-8:012:17:025
3883080991917,2cyclictest144128-21kworker/u16:1+events_unbound11:48:435
3883077991917,2cyclictest4098075-21kworker/u16:2+flush-8:009:17:104
3883077991917,2cyclictest3925398-21kworker/u16:3+flush-8:008:01:394
3883077991917,2cyclictest177772-21kworker/u16:0+events_unbound11:47:134
388308799180,17cyclictest0-21swapper/710:37:117
3883080991816,2cyclictest67663-21kworker/u16:1+flush-8:010:32:035
3883080991816,2cyclictest4192281-21kworker/u16:2+events_unbound09:48:295
3883080991816,2cyclictest4136600-21kworker/u16:0+flush-8:009:32:075
3883080991816,2cyclictest3864858-21kworker/u16:4+events_unbound07:42:245
3883080991816,2cyclictest177772-21kworker/u16:0+flush-8:011:57:025
3883080991816,2cyclictest114887-21kworker/u16:0+events_unbound10:47:075
388307799185,5cyclictest0-21swapper/411:12:034
3883077991816,2cyclictest3954437-21kworker/u16:0+flush-8:008:32:304
3883077991816,2cyclictest3925398-21kworker/u16:3+flush-8:008:12:054
3883077991816,2cyclictest3925398-21kworker/u16:3+flush-8:008:12:054
3883077991816,2cyclictest177772-21kworker/u16:0+flush-8:011:37:154
3883077991816,2cyclictest177772-21kworker/u16:0+events_unbound12:16:394
388308799174,5cyclictest0-21swapper/709:48:427
3883080991715,2cyclictest4164917-21kworker/u16:3+flush-8:010:02:405
3883080991715,2cyclictest4098075-21kworker/u16:2+flush-8:009:18:495
3883080991715,2cyclictest3925397-21kworker/u16:1+events_unbound07:32:245
3883080991715,2cyclictest153552-21kworker/u16:2+flush-8:012:32:055
3883080991715,2cyclictest144128-21kworker/u16:1+events_unbound12:11:355
3883080991715,2cyclictest114887-21kworker/u16:0+events_unbound11:01:395
3883080991715,2cyclictest114887-21kworker/u16:0+events_unbound11:01:395
3883077991715,2cyclictest4192281-21kworker/u16:2+events_unbound10:42:104
3883077991715,2cyclictest4192281-21kworker/u16:2+events_unbound09:47:074
3883077991715,2cyclictest3954437-21kworker/u16:0+flush-8:008:07:104
3883077991714,2cyclictest4164917-21kworker/u16:3+flush-8:010:52:014
388308799160,6cyclictest0-21swapper/710:30:297
3883085991613,2cyclictest1155-21snmpd11:41:556
3883085991612,4cyclictest0-21swapper/609:22:486
388308599160,15cyclictest4032069-21grep08:22:036
3883080991614,2cyclictest4192281-21kworker/u16:2+flush-8:009:57:355
3883080991614,2cyclictest4192281-21kworker/u16:2+events_unbound09:56:205
3883080991614,2cyclictest4164917-21kworker/u16:3+events_unbound10:42:285
3883080991614,2cyclictest4164917-21kworker/u16:3+events_unbound10:27:035
3883080991614,2cyclictest3925398-21kworker/u16:3+flush-8:007:26:595
3883080991614,2cyclictest3864858-21kworker/u16:4+flush-8:007:57:165
3883080991614,2cyclictest22062-21kworker/u16:0+events_unbound07:17:275
3883080991614,2cyclictest22062-21kworker/u16:0+events_unbound07:15:525
3883080991614,2cyclictest177772-21kworker/u16:0+flush-8:011:32:135
3883080991614,2cyclictest153552-21kworker/u16:2+events_unbound12:06:325
3883080991614,2cyclictest115522-21kworker/u16:4+flush-8:011:12:175
3883080991614,2cyclictest115522-21kworker/u16:4+events_unbound11:11:135
3883077991614,2cyclictest3954437-21kworker/u16:0+flush-8:008:52:134
3883077991613,2cyclictest4030703-21kworker/u16:1+events_unbound08:41:574
388307799160,15cyclictest0-21swapper/410:27:054
3883087991514,0cyclictest0-21swapper/707:12:007
3883087991513,1cyclictest0-21swapper/712:02:187
3883087991513,1cyclictest0-21swapper/707:26:597
388308799150,6cyclictest0-21swapper/709:33:487
3883080991513,2cyclictest4164917-21kworker/u16:3+flush-8:010:52:115
3883080991513,2cyclictest4164917-21kworker/u16:3+flush-8:009:42:445
3883080991513,2cyclictest4098076-21kworker/u16:4+flush-8:008:57:005
3883080991513,2cyclictest4098075-21kworker/u16:2+events_unbound09:12:015
3883080991513,2cyclictest4078937-21kworker/u16:3+flush-8:009:02:295
3883080991513,2cyclictest4030703-21kworker/u16:1+flush-8:008:42:135
3883080991513,2cyclictest3967962-21kworker/u16:1+flush-8:008:02:155
3883080991513,2cyclictest3864858-21kworker/u16:4+flush-8:007:22:175
3883080991513,2cyclictest144128-21kworker/u16:1+events_unbound11:56:005
3883080991512,2cyclictest0-21swapper/508:16:585
3883080991512,2cyclictest0-21swapper/508:16:585
3883077991513,2cyclictest3864858-21kworker/u16:4+flush-8:007:26:534
3883077991513,2cyclictest153552-21kworker/u16:2+flush-8:011:17:154
388307799150,8cyclictest0-21swapper/409:03:294
388307799150,15cyclictest0-21swapper/410:17:184
388307799150,14cyclictest75960-21ntpq10:22:124
388308799146,7cyclictest0-21swapper/710:52:507
3883087991412,1cyclictest0-21swapper/708:36:397
388308799140,6cyclictest0-21swapper/710:16:017
388308799140,6cyclictest0-21swapper/710:05:077
388308799140,6cyclictest0-21swapper/709:04:357
388308599145,9cyclictest0-21swapper/610:29:146
388308599145,8cyclictest0-21swapper/611:47:436
388308599140,14cyclictest0-21swapper/612:29:536
3883080991412,2cyclictest4192281-21kworker/u16:2+events_unbound10:17:035
3883080991412,2cyclictest4164917-21kworker/u16:3+flush-8:011:17:035
3883080991412,2cyclictest4164917-21kworker/u16:3+events_unbound09:28:105
3883080991412,2cyclictest4078937-21kworker/u16:3+events_unbound08:52:075
3883080991412,2cyclictest3954437-21kworker/u16:0+events_unbound08:47:155
3883080991412,2cyclictest3864858-21kworker/u16:4+events_unbound07:10:355
3883080991412,2cyclictest153552-21kworker/u16:2+events_unbound11:30:035
3883080991412,2cyclictest114887-21kworker/u16:0+events_unbound10:56:465
3883080991412,2cyclictest114887-21kworker/u16:0+events_unbound10:56:465
3883080991411,3cyclictest144128-21kworker/u16:1+flush-8:012:27:125
3883077991413,1cyclictest0-21swapper/408:57:134
3883077991412,2cyclictest4011425-21kworker/u16:2+events_unbound08:18:434
3883077991412,2cyclictest4011425-21kworker/u16:2+events_unbound08:18:434
3883077991412,2cyclictest3925397-21kworker/u16:1+flush-8:007:32:294
3883077991412,2cyclictest3864858-21kworker/u16:4+events_unbound07:11:384
3883077991411,2cyclictest67663-21kworker/u16:1+flush-8:010:36:384
3883077991411,2cyclictest4192281-21kworker/u16:2+events_unbound10:02:044
3883087991312,0cyclictest0-21swapper/712:22:437
388308799130,6cyclictest0-21swapper/708:00:157
388308799130,5cyclictest0-21swapper/710:11:167
388308799130,5cyclictest0-21swapper/709:19:507
388308799130,5cyclictest0-21swapper/707:42:507
388308799130,1cyclictest0-21swapper/710:17:137
388308799130,12cyclictest0-21swapper/709:56:197
388308799130,11cyclictest0-21swapper/708:27:257
388308599139,3cyclictest4051236-21cat08:32:036
3883085991312,1cyclictest0-21swapper/611:33:546
3883085991310,2cyclictest2439531-21JS17:10:446
388308599130,13cyclictest0-21swapper/612:14:166
388308599130,13cyclictest0-21swapper/609:17:046
388308599130,13cyclictest0-21swapper/608:52:106
3883080991311,2cyclictest67663-21kworker/u16:1+events_unbound10:22:495
3883080991311,2cyclictest4174926-21kworker/u16:1+events_unbound09:38:045
3883080991311,2cyclictest3925398-21kworker/u16:3+events_unbound07:47:165
3883080991311,2cyclictest177772-21kworker/u16:0+flush-8:012:11:595
3883080991311,2cyclictest177772-21kworker/u16:0+events_unbound11:22:065
3883077991311,2cyclictest4192281-21kworker/u16:2+events_unbound09:59:304
3883077991311,2cyclictest4164917-21kworker/u16:3+events_unbound10:12:124
3883077991311,2cyclictest3954437-21kworker/u16:0+flush-8:008:22:484
3883077991311,2cyclictest3925397-21kworker/u16:1+events_unbound07:37:144
3883077991311,2cyclictest3864858-21kworker/u16:4+flush-8:007:22:154
3883077991311,2cyclictest144128-21kworker/u16:1+flush-8:012:31:534
3883077991311,2cyclictest144128-21kworker/u16:1+events_unbound12:18:294
3883077991311,2cyclictest144128-21kworker/u16:1+events_unbound11:52:184
3883077991311,2cyclictest114887-21kworker/u16:0+events_unbound11:03:404
3883077991311,2cyclictest114887-21kworker/u16:0+events_unbound11:03:404
3883077991310,2cyclictest4126735-21kworker/u16:1+events_unbound09:11:574
388308799129,2cyclictest0-21swapper/710:56:597
388308799129,2cyclictest0-21swapper/710:56:597
388308799129,2cyclictest0-21swapper/708:40:557
388308799122,9cyclictest0-21swapper/708:06:547
3883087991211,0cyclictest0-21swapper/709:42:097
3883087991211,0cyclictest0-21swapper/708:51:597
388308799120,6cyclictest0-21swapper/707:32:147
388308799120,1cyclictest0-21swapper/708:22:157
388308599129,2cyclictest1155-21snmpd07:34:236
388308599128,3cyclictest4118381-21cat09:07:016
388308599127,4cyclictest4099121-21cat08:57:016
3883085991211,1cyclictest0-21swapper/608:49:576
388308599120,1cyclictest0-21swapper/609:41:496
388308599120,12cyclictest0-21swapper/612:33:396
388308599120,12cyclictest0-21swapper/611:38:476
388308599120,12cyclictest0-21swapper/611:21:336
388308599120,12cyclictest0-21swapper/608:29:186
388308599120,12cyclictest0-21swapper/608:08:496
388308599120,11cyclictest128649-21grep10:52:056
3883080991210,2cyclictest4136600-21kworker/u16:0+events_unbound10:11:475
3883080991210,2cyclictest4098075-21kworker/u16:2+events_unbound09:06:525
3883080991210,2cyclictest4030703-21kworker/u16:1+events_unbound08:38:435
3883080991210,2cyclictest3967962-21kworker/u16:1+events_unbound07:53:355
388307799129,2cyclictest1155-21snmpd09:56:374
3883077991210,2cyclictest4192281-21kworker/u16:2+flush-8:010:07:124
3883077991210,2cyclictest4164917-21kworker/u16:3+events_unbound10:37:034
3883077991210,2cyclictest4030703-21kworker/u16:1+events_unbound08:37:174
3883077991210,2cyclictest3967962-21kworker/u16:1+events_unbound08:02:164
3883077991210,2cyclictest3954437-21kworker/u16:0+flush-8:009:07:004
3883077991210,2cyclictest3954437-21kworker/u16:0+events_unbound08:47:494
3883077991210,2cyclictest3864858-21kworker/u16:4+events_unbound07:43:464
3883077991210,2cyclictest177772-21kworker/u16:0+events_unbound11:42:144
3883077991210,2cyclictest177772-21kworker/u16:0+events_unbound11:32:164
388307799120,1cyclictest0-21swapper/411:00:024
388307799120,1cyclictest0-21swapper/411:00:024
388308799119,1cyclictest0-21swapper/712:21:397
388308799119,1cyclictest0-21swapper/711:07:027
3883087991110,0cyclictest0-21swapper/711:44:537
3883087991110,0cyclictest0-21swapper/711:38:357
388308799110,5cyclictest0-21swapper/710:41:477
388308599119,2cyclictest86-21ksoftirqd/611:52:056
388308599119,2cyclictest86-21ksoftirqd/607:07:006
388308599118,2cyclictest1155-21snmpd10:58:186
388308599118,2cyclictest1155-21snmpd10:58:186
388308599117,3cyclictest3905576-21sed07:12:206
388308599116,4cyclictest634-21systemd-journal09:56:336
388308599113,1cyclictest0-21swapper/608:02:126
3883085991110,1cyclictest0-21swapper/611:01:456
3883085991110,1cyclictest0-21swapper/611:01:456
3883085991110,1cyclictest0-21swapper/610:25:016
3883085991110,1cyclictest0-21swapper/610:07:506
3883085991110,1cyclictest0-21swapper/610:06:346
3883085991110,1cyclictest0-21swapper/607:56:506
3883085991110,1cyclictest0-21swapper/607:42:296
3883085991110,1cyclictest0-21swapper/607:41:136
3883085991110,1cyclictest0-21swapper/607:29:056
388308599110,4cyclictest3971396-21grep07:47:066
388308599110,4cyclictest113033-21cut10:42:076
388308599110,3cyclictest4162574-21cut09:27:076
388308599110,1cyclictest0-21swapper/611:07:446
388308599110,11cyclictest0-21swapper/610:36:036
388308599110,11cyclictest0-21swapper/609:03:316
388308099119,2cyclictest177772-21kworker/u16:0+events_unbound12:22:475
388307799119,2cyclictest177772-21kworker/u16:0+events_unbound11:57:054
388307799119,2cyclictest153552-21kworker/u16:2+flush-8:012:21:554
388307799119,2cyclictest153552-21kworker/u16:2+events_unbound12:05:434
3883077991110,1cyclictest0-21swapper/409:25:264
3883077991110,1cyclictest0-21swapper/408:28:044
3883077991110,1cyclictest0-21swapper/407:11:554
388308799109,0cyclictest0-21swapper/708:47:237
388308799109,0cyclictest0-21swapper/708:41:437
388308799107,2cyclictest0-21swapper/709:06:477
388308799107,2cyclictest0-21swapper/707:52:337
388308799102,7cyclictest0-21swapper/711:16:427
388308799100,9cyclictest0-21swapper/711:57:257
388308799100,9cyclictest0-21swapper/709:21:487
388308799100,9cyclictest0-21swapper/709:16:087
388308799100,4cyclictest0-21swapper/708:56:417
388308599109,1cyclictest164813-21kthreadcore11:12:086
388308599108,2cyclictest177772-21kworker/u16:0+flush-8:011:27:106
388308599107,2cyclictest0-21swapper/610:11:596
388308599105,4cyclictest4127404-21grep09:11:566
388308599101,8cyclictest0-21swapper/612:02:196
388308599101,2cyclictest272448-21kthreadcore12:07:056
388308599100,4cyclictest26896-21kthreadcore09:57:096
388308599100,3cyclictest4165617-21/usr/sbin/munin09:32:156
388308599100,3cyclictest2977-21cut09:47:066
388308099108,2cyclictest3954437-21kworker/u16:0+events_unbound08:29:285
388307799109,1cyclictest0-21swapper/412:08:404
388307799109,1cyclictest0-21swapper/409:45:184
388307799108,2cyclictest22062-21kworker/u16:0+events_unbound07:19:324
388307799108,2cyclictest177772-21kworker/u16:0+flush-8:011:21:594
388307799108,2cyclictest177772-21kworker/u16:0+events_unbound11:27:114
38830879996,2cyclictest0-21swapper/707:39:267
38830879990,9cyclictest0-21swapper/708:04:517
38830879990,8cyclictest0-21swapper/709:30:097
38830879990,7cyclictest0-21swapper/709:57:127
38830859997,2cyclictest86-21ksoftirqd/610:17:186
38830859997,1cyclictest306047-21ntpq12:22:086
38830859997,1cyclictest0-21swapper/608:17:056
38830859997,1cyclictest0-21swapper/608:17:056
38830859996,2cyclictest3910598-21cat07:17:176
38830859996,2cyclictest0-21swapper/609:37:256
38830859990,8cyclictest4076587-21cut08:42:056
38830859990,8cyclictest0-21swapper/612:17:046
38830859990,2cyclictest3973646-21/usr/sbin/munin07:52:176
38830859990,2cyclictest188508-21kthreadcore11:22:056
38830809997,2cyclictest3954437-21kworker/u16:0+events_unbound08:22:115
38830779996,2cyclictest1155-21snmpd07:52:244
38830879987,0cyclictest0-21swapper/712:10:077
38830879987,0cyclictest0-21swapper/711:52:027
38830879987,0cyclictest0-21swapper/710:23:427
38830879987,0cyclictest0-21swapper/708:12:057
38830879987,0cyclictest0-21swapper/708:12:057
38830879980,7cyclictest0-21swapper/707:51:307
38830879980,7cyclictest0-21swapper/707:09:137
38830879980,1cyclictest0-21swapper/712:15:497
38830859987,1cyclictest0-21swapper/611:59:366
38830859986,1cyclictest0-21swapper/608:12:086
38830859986,1cyclictest0-21swapper/608:12:086
38830859984,3cyclictest4064476-21kthreadcore08:37:066
38830859983,4cyclictest96429-21cpuspeed10:36:586
38830779983,4cyclictest4175970-21sh09:37:064
38830779983,4cyclictest121268-21cut10:47:054
38830879977,0cyclictest0-21swapper/711:29:397
38830879976,0cyclictest0-21swapper/711:26:387
38830879976,0cyclictest0-21swapper/708:17:137
38830879976,0cyclictest0-21swapper/708:17:137
38830879974,2cyclictest4174822-21turbostat09:41:337
38830879970,6cyclictest0-21swapper/711:49:227
38830879970,6cyclictest0-21swapper/710:48:597
38830879970,6cyclictest0-21swapper/710:34:267
38830879970,6cyclictest0-21swapper/707:22:157
38830879960,5cyclictest0-21swapper/711:14:037
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional