You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-22 - 23:56

x86 Intel Xeon E3-1220L V2 @2300 MHz, Linux 4.18.7-rt5 (Profile)

Latency plot of system in rack #0, slot #1
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack0slot1.osadl.org (updated Wed Mar 22, 2023 12:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
647321060,1sleep73799-21runrttasks12:17:567
137432750,1sleep59402-21ntop07:46:165
139762740,1sleep6691ktimersoftd/610:07:036
285302710,1sleep450-21rcuc/411:19:204
72792690,1sleep612340-21G123:59:416
37582660,0sleep70-21swapper/707:40:577
296472650,0sleep00-21swapper/009:33:480
269462640,0sleep668-21rcuc/609:32:266
115612640,3sleep4864699cyclictest10:59:094
162252630,0sleep60-21swapper/612:21:406
197472600,0sleep70-21swapper/710:36:137
216492590,0sleep70-21swapper/712:10:117
216492590,0sleep70-21swapper/712:10:117
28872530,0sleep60-21swapper/608:15:586
8643993731,3cyclictest7664-21perl08:47:071
149972360,0sleep70-21swapper/708:22:567
8643993316,14cyclictest2110-21perl10:08:271
8643993220,8cyclictest1951-21perl12:34:301
8643993220,8cyclictest1951-21perl12:34:301
8643993220,7cyclictest9243-21perl12:09:031
8643993220,7cyclictest9243-21perl12:09:031
8645993113,17cyclictest0-21swapper/310:13:573
246272310,22sleep0190150irq/25-eth011:04:420
8645993011,18cyclictest0-21swapper/310:19:473
8643993026,3cyclictest14254-21Collectd-Thread11:07:591
864599298,15cyclictest0-21swapper/310:10:393
8643992921,4cyclictest25983-21perl10:47:591
8643992917,8cyclictest23394-21perl10:13:411
226302290,1sleep422598-21sshd12:10:564
226302290,1sleep422598-21sshd12:10:564
8643992820,4cyclictest2501-21perl10:38:051
8643992814,7cyclictest2453-21openvpn08:36:431
864399280,2cyclictest9402-21ntop09:23:271
864599275,4cyclictest0-21swapper/311:12:053
8643992722,4cyclictest28539-21ssh10:39:271
864399270,22cyclictest0-21swapper/111:44:191
864399262,20cyclictest9237-21perl09:58:271
864399262,20cyclictest9237-21perl09:58:271
8643992620,2cyclictest2501-21perl09:39:431
8643992619,3cyclictest1951-21perl12:13:431
8643992610,0cyclictest9402-21ntop12:07:031
8648992523,1cyclictest70-21ksoftirqd/609:36:576
8645992517,4cyclictest1951-21perl11:00:373
864399253,2cyclictest13874-21Poller-Thread-111:42:211
8643992521,3cyclictest10216-21perl07:37:231
8643992520,4cyclictest0-21swapper/111:59:231
8643992519,3cyclictest27888-21perl09:17:271
8643992517,4cyclictest2492-21perl10:00:431
8643992517,4cyclictest2492-21perl09:02:071
8643992516,5cyclictest27888-21perl08:07:551
8643992515,4cyclictest9402-21ntop09:32:471
864399251,20cyclictest2501-21perl07:17:311
864399250,21cyclictest9402-21ntop11:51:471
864399250,21cyclictest9402-21ntop11:51:471
864399250,20cyclictest0-21swapper/107:39:151
864399250,15cyclictest9402-21ntop10:23:111
38012250,3sleep3205350irq/26-eth1-rx-07:58:483
271552250,1sleep025983-21perl09:18:580
170202250,0sleep50-21swapper/507:29:585
864599243,17cyclictest2169-21perl12:03:483
8645992414,7cyclictest23486-21postmaster11:42:213
8645992413,7cyclictest23394-21perl07:43:593
8645992412,8cyclictest2453-21openvpn10:37:573
864399243,20cyclictest0-21swapper/108:19:191
8643992421,2cyclictest2051-21perl10:55:091
8643992420,3cyclictest25983-21perl12:25:581
8643992420,3cyclictest25983-21perl12:25:581
8643992420,3cyclictest1765-21perl08:03:271
8643992417,4cyclictest29249-21perl09:25:011
8643992416,4cyclictest0-21swapper/108:31:411
8643992416,3cyclictest9402-21ntop10:09:591
864399241,20cyclictest23394-21perl08:09:391
864399241,20cyclictest23394-21perl08:09:391
864399240,3cyclictest0-21swapper/111:33:391
864399240,23cyclictest1951-21perl08:18:051
864399240,23cyclictest0-21swapper/108:48:391
864399240,21cyclictest8565-21ssh12:32:021
864399240,20cyclictest0-21swapper/111:14:111
864399240,20cyclictest0-21swapper/107:24:151
864399240,16cyclictest9237-21perl07:30:431
5202240,1sleep013-21rcuc/007:21:030
5202240,1sleep013-21rcuc/007:21:030
8648992320,2cyclictest70-21ksoftirqd/609:51:596
864599237,6cyclictest0-21swapper/309:46:273
8645992313,6cyclictest2441-21perl08:32:373
8645992312,2cyclictest9827-21perl10:05:163
8645992310,9cyclictest205350irq/26-eth1-rx-07:55:573
8645992310,8cyclictest0-21swapper/312:20:063
864599230,17cyclictest0-21swapper/312:09:413
864599230,17cyclictest0-21swapper/312:09:413
864399230,20cyclictest2453-21openvpn09:04:051
864399230,19cyclictest27888-21perl08:25:331
864399230,19cyclictest23394-21perl11:18:511
864399230,19cyclictest0-21swapper/107:49:431
864399230,16cyclictest0-21swapper/108:42:511
864399230,15cyclictest0-21swapper/107:11:391
864399230,0cyclictest9402-21ntop10:32:031
107222230,1sleep710719-21sshd12:33:037
864999221,19cyclictest10022-21perl10:18:507
864999220,20cyclictest5203-21if_err_tun112:03:487
8648992219,2cyclictest70-21ksoftirqd/609:45:586
8647992218,2cyclictest26867-21perl09:18:575
8645992215,3cyclictest23006-21ssh09:57:513
8645992215,3cyclictest23006-21ssh09:57:513
8645992215,3cyclictest12518-21JNI-ICMP-30467-08:12:133
8645992215,3cyclictest12518-21JNI-ICMP-30467-08:12:133
8645992214,4cyclictest2501-21perl09:28:153
8645992213,5cyclictest0-21swapper/309:06:353
8645992213,4cyclictest0-21swapper/307:11:593
864399224,14cyclictest9402-21ntop11:57:011
8643992219,1cyclictest0-21swapper/107:46:391
8643992214,5cyclictest9965-21perl10:25:511
864399220,21cyclictest2051-21perl10:49:351
864399220,21cyclictest2051-21perl10:49:351
864399220,21cyclictest0-21swapper/109:51:271
864399220,21cyclictest0-21swapper/108:54:071
864399220,21cyclictest0-21swapper/107:23:231
864399220,21cyclictest0-21swapper/107:23:231
864399220,1cyclictest0-21swapper/110:59:491
864399220,17cyclictest9402-21ntop09:34:211
107612220,0sleep20-21swapper/209:12:272
864999212,2cyclictest24856-21postmaster11:04:597
8649992119,1cyclictest79-21ksoftirqd/709:51:597
864999211,2cyclictest9695-21perl10:05:167
864999210,19cyclictest0-21swapper/709:05:407
8648992119,1cyclictest70-21ksoftirqd/608:36:476
8648992118,2cyclictest70-21ksoftirqd/609:57:576
8648992118,2cyclictest70-21ksoftirqd/609:57:576
864599218,8cyclictest0-21swapper/309:42:113
864599218,8cyclictest0-21swapper/308:18:513
864599218,3cyclictest0-21swapper/311:29:573
864599217,8cyclictest0-21swapper/309:34:233
864599217,8cyclictest0-21swapper/307:35:233
864599213,17cyclictest0-21swapper/308:47:073
8645992114,3cyclictest9402-21ntop11:16:453
8645992114,3cyclictest31489-21sshd12:14:033
8645992114,3cyclictest17997-21pool-14-thread-11:56:353
8645992112,2cyclictest24645-21perl11:04:443
8645992111,5cyclictest23394-21perl11:49:273
8645992111,5cyclictest23394-21perl11:49:273
864599210,2cyclictest15293-21perl08:04:293
864599210,17cyclictest0-21swapper/311:35:033
8643992113,4cyclictest2051-21perl09:12:431
864399210,20cyclictest0-21swapper/111:28:431
864399210,1cyclictest0-21swapper/111:09:231
864399210,16cyclictest9402-21ntop12:18:431
864399210,0cyclictest9402-21ntop11:24:591
171642210,0sleep50-21swapper/509:14:385
8649992017,2cyclictest15137-21perl08:04:167
8649992016,2cyclictest30623-21perl08:48:587
8649992016,2cyclictest12038-21perl12:19:297
8648992018,1cyclictest70-21ksoftirqd/611:46:386
8648992017,2cyclictest70-21ksoftirqd/610:38:576
8647992016,2cyclictest17767-21snmp_nas.osadl.10:48:485
8647992016,2cyclictest17767-21snmp_nas.osadl.10:48:485
864599207,9cyclictest9402-21ntop11:45:493
864599205,14cyclictest0-21swapper/309:32:473
8645992015,1cyclictest0-21swapper/307:51:313
8645992013,3cyclictest6532-21sshd10:43:573
8645992013,3cyclictest3799-21runrttasks09:59:233
8645992013,3cyclictest18606-21cut10:48:553
8645992013,3cyclictest18606-21cut10:48:553
8645992012,7cyclictest0-21swapper/309:16:393
8645992012,4cyclictest2364-21perl08:16:433
8645992012,4cyclictest205350irq/26-eth1-rx-08:38:533
8645992012,4cyclictest0-21swapper/312:25:313
8645992012,4cyclictest0-21swapper/312:25:313
8645992012,3cyclictest28185-21perl11:19:093
8645992012,1cyclictest0-21swapper/309:50:373
8645992011,6cyclictest0-21swapper/312:28:443
8645992011,4cyclictest2408-21snmpget10:55:593
8645992011,4cyclictest0-21swapper/311:58:553
864399200,14cyclictest0-21swapper/107:54:191
8648991917,1cyclictest70-21ksoftirqd/610:12:036
8648991917,1cyclictest70-21ksoftirqd/609:21:576
8648991917,1cyclictest70-21ksoftirqd/609:15:586
8647991916,2cyclictest27163-21perl09:04:405
8647991915,2cyclictest2125-21perl12:03:475
864799190,17cyclictest11102-21perl10:06:005
864599198,6cyclictest2501-21perl09:20:393
864599198,4cyclictest0-21swapper/310:39:173
864599196,8cyclictest0-21swapper/307:13:513
8645991914,0cyclictest0-21swapper/308:48:433
8645991912,3cyclictest9503-21snmpget09:11:573
8645991911,4cyclictest23394-21perl09:01:273
8645991911,4cyclictest11241-21snmpget11:26:133
8645991911,4cyclictest0-21swapper/308:58:113
8645991910,5cyclictest0-21swapper/312:33:343
8645991910,5cyclictest0-21swapper/312:33:343
864599190,15cyclictest2492-21perl07:18:593
864599190,15cyclictest2492-21perl07:18:593
864599190,15cyclictest16006-21sshd07:29:033
864399190,15cyclictest0-21swapper/109:43:411
8649991814,2cyclictest29951-21sshd07:19:057
8649991814,2cyclictest29951-21sshd07:19:057
8649991814,2cyclictest28409-21perl11:19:157
8648991816,1cyclictest70-21ksoftirqd/611:32:236
8648991816,1cyclictest70-21ksoftirqd/609:41:576
8647991815,2cyclictest28480-21perl11:19:195
864599189,5cyclictest2441-21perl10:33:253
864599189,5cyclictest0-21swapper/308:28:293
864599188,5cyclictest0-21swapper/308:33:493
8645991814,3cyclictest0-21swapper/307:25:473
8642991816,1cyclictest190150irq/25-eth010:58:590
8649991714,2cyclictest0-21swapper/711:48:557
8649991714,2cyclictest0-21swapper/711:48:557
864999170,2cyclictest7017-21perl07:48:587
8648991715,1cyclictest70-21ksoftirqd/612:32:266
8648991715,1cyclictest70-21ksoftirqd/612:17:316
8648991715,1cyclictest70-21ksoftirqd/612:13:266
8648991715,1cyclictest70-21ksoftirqd/612:13:266
8648991715,1cyclictest70-21ksoftirqd/611:58:316
8648991715,1cyclictest70-21ksoftirqd/611:42:196
8648991715,1cyclictest70-21ksoftirqd/611:37:006
8648991715,1cyclictest70-21ksoftirqd/611:03:276
8648991715,1cyclictest70-21ksoftirqd/610:31:036
8648991715,1cyclictest70-21ksoftirqd/609:26:176
8648991715,1cyclictest70-21ksoftirqd/608:52:556
8648991715,1cyclictest70-21ksoftirqd/608:01:036
8648991715,1cyclictest70-21ksoftirqd/607:16:516
8648991713,3cyclictest9402-21ntop08:46:596
864899170,1cyclictest21064-21kworker/6:011:18:196
864899170,1cyclictest0-21swapper/607:27:336
8647991715,1cyclictest23144-21kworker/5:010:25:155
8647991713,2cyclictest1903-21hddtemp_smartct09:48:525
864799170,1cyclictest9402-21ntop11:04:445
8645991713,3cyclictest0-21swapper/307:40:473
8645991711,5cyclictest0-21swapper/310:23:493
8649991612,2cyclictest79-21ksoftirqd/709:46:577
864999160,3cyclictest0-21swapper/710:54:447
8648991614,1cyclictest70-21ksoftirqd/612:37:366
8648991614,1cyclictest70-21ksoftirqd/612:37:366
8648991614,1cyclictest70-21ksoftirqd/611:57:266
8648991614,1cyclictest70-21ksoftirqd/611:12:096
8648991614,1cyclictest70-21ksoftirqd/610:13:576
8648991614,1cyclictest70-21ksoftirqd/610:02:026
8648991614,1cyclictest70-21ksoftirqd/607:46:056
8648991614,1cyclictest0-21swapper/607:53:556
864899160,1cyclictest14428-21kworker/6:211:07:156
864899160,15cyclictest7100-21kworker/6:010:52:076
864899160,15cyclictest7100-21kworker/6:010:52:076
864899160,15cyclictest6214-21kworker/6:212:08:016
864899160,15cyclictest21064-21kworker/6:011:21:516
8647991613,2cyclictest31075-21ntp_offset08:48:595
8647991613,2cyclictest2890-21snmpd12:34:545
8647991613,2cyclictest2890-21snmpd12:34:545
8646991614,1cyclictest52-21ksoftirqd/409:51:594
864699160,15cyclictest0-21swapper/407:12:214
864699160,13cyclictest2453-21openvpn10:53:534
8644991614,1cyclictest14150irq/24-ahci[00012:13:012
8644991614,1cyclictest14150irq/24-ahci[00012:13:012
864499160,14cyclictest6004-21sshd09:37:082
864299160,15cyclictest0-21swapper/010:35:550
864999153,7cyclictest12530-21JNI-ICMP-3825-S09:59:147
8649991513,1cyclictest1920-21mii-tool11:08:487
8649991513,1cyclictest10482-21kworker/7:108:43:497
8649991512,2cyclictest79-21ksoftirqd/711:37:007
8649991512,2cyclictest2890-21snmpd09:27:087
8648991513,1cyclictest70-21ksoftirqd/612:26:346
8648991513,1cyclictest70-21ksoftirqd/612:26:346
8648991513,1cyclictest70-21ksoftirqd/610:58:296
8648991513,1cyclictest70-21ksoftirqd/610:27:036
8648991513,1cyclictest70-21ksoftirqd/609:01:136
8648991513,1cyclictest70-21ksoftirqd/608:31:086
8648991513,1cyclictest70-21ksoftirqd/608:26:086
8648991513,1cyclictest70-21ksoftirqd/608:11:086
8648991513,1cyclictest70-21ksoftirqd/608:11:086
8648991513,1cyclictest70-21ksoftirqd/608:07:586
8648991513,1cyclictest70-21ksoftirqd/607:51:046
864899150,14cyclictest9402-21ntop07:31:336
864799154,6cyclictest12530-21JNI-ICMP-3825-S07:35:575
8647991513,1cyclictest61-21ksoftirqd/510:31:035
8647991513,1cyclictest3038-21kworker/5:107:27:475
8647991513,1cyclictest25969-21kworker/5:207:21:115
8647991513,1cyclictest25969-21kworker/5:207:21:115
8647991513,1cyclictest11878-21perl08:03:375
8647991512,2cyclictest2890-21snmpd08:55:535
864699151,1cyclictest0-21swapper/408:03:544
8644991512,2cyclictest2890-21snmpd11:51:532
8644991512,2cyclictest2890-21snmpd11:51:532
8644991512,2cyclictest14150irq/24-ahci[00008:36:212
8642991513,1cyclictest190150irq/25-eth007:12:210
864999143,5cyclictest13415-21JNI-ICMP-20259-11:38:427
864999143,5cyclictest12515-21JNI-ICMP-9429-S09:22:357
8649991412,1cyclictest0-21swapper/709:41:147
8649991411,2cyclictest79-21ksoftirqd/707:36:047
8649991411,1cyclictest6018-21kworker/7:111:56:157
8649991411,1cyclictest2453-21openvpn08:33:477
8648991412,1cyclictest70-21ksoftirqd/611:48:396
8648991412,1cyclictest70-21ksoftirqd/611:48:396
8648991412,1cyclictest70-21ksoftirqd/611:26:356
8648991412,1cyclictest70-21ksoftirqd/610:21:146
8648991412,1cyclictest70-21ksoftirqd/609:12:596
8648991412,1cyclictest70-21ksoftirqd/609:05:576
8648991412,1cyclictest70-21ksoftirqd/608:42:566
8648991412,1cyclictest70-21ksoftirqd/607:18:576
8648991412,1cyclictest70-21ksoftirqd/607:18:576
864899140,1cyclictest9402-21ntop07:36:116
864799143,6cyclictest14533-21JNI-ICMP-15390-10:40:035
8647991412,1cyclictest2605-21kworker/5:209:57:015
8647991412,1cyclictest2605-21kworker/5:209:57:015
8647991411,2cyclictest61-21ksoftirqd/508:21:585
8647991411,2cyclictest2890-21snmpd10:11:385
8646991412,1cyclictest5265-21mii-tool12:03:484
8646991412,1cyclictest2890-21snmpd07:30:534
8646991412,1cyclictest21891-21mii-tool11:43:524
8646991412,1cyclictest19500-21postmaster09:27:094
8646991412,1cyclictest14202-21mii-tool09:53:534
8646991412,1cyclictest14202-21mii-tool09:53:534
8646991411,2cyclictest2890-21snmpd09:46:534
864699140,1cyclictest0-21swapper/410:28:494
8644991412,1cyclictest14150irq/24-ahci[00011:54:372
864499140,12cyclictest2492-21perl12:13:492
864299148,1cyclictest0-21swapper/011:33:300
8642991412,1cyclictest27410-21mii-tool07:53:570
8642991412,1cyclictest190150irq/25-eth010:53:530
864999133,6cyclictest14533-21JNI-ICMP-15390-08:32:377
864999133,6cyclictest13409-21JNI-ICMP-19444-08:23:407
864999133,6cyclictest12518-21JNI-ICMP-30467-09:32:597
864999133,5cyclictest13415-21JNI-ICMP-20259-07:48:037
864999133,4cyclictest12515-21JNI-ICMP-9429-S07:08:387
8649991311,1cyclictest79-21ksoftirqd/708:41:097
8649991311,1cyclictest22774-21mii-tool08:08:477
8649991311,1cyclictest22774-21mii-tool08:08:477
8649991311,1cyclictest22461-21mii-tool10:23:537
8649991310,2cyclictest19999-21openvpn10:52:427
8649991310,2cyclictest19999-21openvpn10:52:427
864999130,12cyclictest732-21diskmemload11:46:187
864999130,12cyclictest0-21swapper/712:01:427
864999130,12cyclictest0-21swapper/708:16:187
8648991311,1cyclictest70-21ksoftirqd/610:45:576
8648991311,1cyclictest70-21ksoftirqd/610:37:006
8648991311,1cyclictest70-21ksoftirqd/607:41:566
864799133,6cyclictest13409-21JNI-ICMP-19444-12:32:325
864799133,6cyclictest12515-21JNI-ICMP-9429-S10:13:565
864799133,5cyclictest13415-21JNI-ICMP-20259-09:12:395
864799131,6cyclictest12515-21JNI-ICMP-9429-S10:37:245
8647991311,1cyclictest4982-21snmp_nas.osadl.11:23:505
8647991311,1cyclictest26450-21kworker/5:112:02:275
8647991311,1cyclictest13168-21kworker/5:208:47:475
864799130,12cyclictest9402-21ntop12:23:325
864799130,12cyclictest9402-21ntop12:23:325
8646991311,1cyclictest52-21ksoftirqd/410:02:044
8646991311,1cyclictest52-21ksoftirqd/409:41:574
8646991311,1cyclictest52-21ksoftirqd/409:36:574
8646991311,1cyclictest32436-21mii-tool07:38:534
8646991310,2cyclictest2890-21snmpd08:24:384
864699130,1cyclictest9402-21ntop10:33:564
864699130,1cyclictest0-21swapper/410:08:474
864699130,12cyclictest9402-21ntop10:38:554
864699130,11cyclictest0-21swapper/412:37:244
864699130,11cyclictest0-21swapper/412:37:244
8644991312,0cyclictest14150irq/24-ahci[00010:17:572
8644991311,1cyclictest34-21ksoftirqd/212:22:242
8644991311,1cyclictest34-21ksoftirqd/210:01:052
8644991311,1cyclictest14150irq/24-ahci[00008:15:552
8644991311,1cyclictest14150irq/24-ahci[00008:02:392
8644991310,2cyclictest2890-21snmpd11:44:232
8642991311,1cyclictest1944-21ethtool10:28:490
8642991311,1cyclictest190150irq/25-eth008:03:540
8642991311,1cyclictest18624-21mii-tool10:08:470
864999124,5cyclictest12515-21JNI-ICMP-9429-S08:58:527
864999123,5cyclictest13409-21JNI-ICMP-19444-11:01:007
864999123,5cyclictest12515-21JNI-ICMP-9429-S08:00:527
864999123,4cyclictest13415-21JNI-ICMP-20259-10:38:427
864999123,4cyclictest13409-21JNI-ICMP-19444-08:57:037
8649991210,1cyclictest79-21ksoftirqd/707:32:587
864999120,1cyclictest0-21swapper/707:58:237
864999120,0cyclictest9402-21ntop11:13:507
864799123,4cyclictest14533-21JNI-ICMP-15390-11:08:365
864799123,4cyclictest12515-21JNI-ICMP-9429-S11:32:055
864799122,5cyclictest13409-21JNI-ICMP-19444-12:21:275
864799122,5cyclictest13406-21JNI-ICMP-6049-S11:41:085
864799122,5cyclictest12515-21JNI-ICMP-9429-S08:10:305
864799122,5cyclictest12515-21JNI-ICMP-9429-S08:10:305
864799122,4cyclictest13406-21JNI-ICMP-6049-S10:43:425
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional