You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-07-01 - 19:07

x86 Intel Atom E3845 @1915 MHz, Linux 4.19.37-rt20 (Profile)

Latency plot of system in rack #b, slot #6
Data to construct the above plot have been generated using the RT test utility cyclictest.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 792 highest latencies:
System rackbslot6.osadl.org (updated Fri Jul 01, 2022 12:43:33)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
423921640,4sleep11991799cyclictest13:43:591
911911250,9ptp4l17014-21kworker/3:209:36:183
77422710,4sleep37747-21gltestperf10:18:593
134872690,2sleep10-21swapper/112:44:361
278202670,2sleep30-21swapper/312:59:323
91191650,1ptp4l401ktimersoftd/315:08:373
91191600,1ptp4l401ktimersoftd/312:48:233
91191590,1ptp4l401ktimersoftd/313:57:553
150442590,1sleep20-21swapper/214:29:012
150442590,1sleep20-21swapper/214:29:012
91191560,1ptp4l401ktimersoftd/309:39:003
312802560,5sleep31993199cyclictest12:29:333
195692560,2sleep20-21swapper/212:18:552
91191530,1ptp4l401ktimersoftd/309:58:443
19931995036,5cyclictest41-21ksoftirqd/313:13:593
196632504,33sleep20-21swapper/209:37:222
91191490,2ptp4l23219-21kworker/3:111:55:023
19917994840,4cyclictest25-21ksoftirqd/114:09:011
19917994825,4cyclictest25-21ksoftirqd/112:04:241
91191470,0ptp4l401ktimersoftd/311:50:333
91191470,0ptp4l401ktimersoftd/311:50:333
19917994738,4cyclictest25-21ksoftirqd/114:54:001
91191460,2ptp4l21053-21kworker/3:213:48:043
91191460,0ptp4l401ktimersoftd/310:38:443
197192464,30sleep10-21swapper/109:38:061
91191450,2ptp4l21053-21kworker/3:214:42:013
97191440,3getstats0-21swapper/314:18:533
19917994235,3cyclictest25-21ksoftirqd/111:13:551
91191410,1ptp4l401ktimersoftd/314:29:023
91191410,1ptp4l401ktimersoftd/314:29:023
19931994131,5cyclictest41-21ksoftirqd/314:43:593
91191400,1ptp4l401ktimersoftd/314:57:253
91191400,1ptp4l401ktimersoftd/314:08:283
91191400,1ptp4l401ktimersoftd/312:17:183
91191400,1ptp4l401ktimersoftd/311:44:353
91191400,1ptp4l401ktimersoftd/311:39:063
91191400,1ptp4l401ktimersoftd/311:04:013
19917994018,4cyclictest25-21ksoftirqd/113:03:581
19917994017,4cyclictest25-21ksoftirqd/111:39:061
19917994012,8cyclictest25-21ksoftirqd/110:58:581
91191390,1ptp4l401ktimersoftd/315:03:103
91191390,1ptp4l401ktimersoftd/314:35:103
91191390,1ptp4l401ktimersoftd/314:10:533
91191390,1ptp4l401ktimersoftd/314:01:223
91191390,1ptp4l401ktimersoftd/313:19:453
91191390,1ptp4l401ktimersoftd/312:00:263
19917993927,3cyclictest25-21ksoftirqd/113:18:591
19917993916,4cyclictest25-21ksoftirqd/110:29:001
91191380,1ptp4l401ktimersoftd/314:52:393
91191380,1ptp4l401ktimersoftd/313:42:493
91191380,1ptp4l401ktimersoftd/313:30:023
91191380,1ptp4l401ktimersoftd/313:09:113
91191380,1ptp4l401ktimersoftd/313:09:113
91191380,1ptp4l401ktimersoftd/313:08:413
91191380,1ptp4l401ktimersoftd/312:40:473
91191380,1ptp4l401ktimersoftd/312:33:583
91191380,1ptp4l401ktimersoftd/312:03:523
91191380,0ptp4l401ktimersoftd/312:19:143
1993199384,5cyclictest41-21ksoftirqd/313:33:583
1993199383,3cyclictest121rcu_preempt10:09:123
19931993824,3cyclictest41-21ksoftirqd/310:53:573
1991799386,3cyclictest25-21ksoftirqd/113:59:241
19917993817,4cyclictest25-21ksoftirqd/111:54:051
91191370,1ptp4l401ktimersoftd/314:18:003
91191370,1ptp4l401ktimersoftd/312:52:273
91191370,1ptp4l401ktimersoftd/312:13:413
91191370,14ptp4l401ktimersoftd/311:13:203
19931993722,3cyclictest41-21ksoftirqd/312:54:163
19917993724,3cyclictest25-21ksoftirqd/114:34:131
19917993715,7cyclictest25-21ksoftirqd/112:29:081
19917993715,4cyclictest25-21ksoftirqd/114:59:031
19917993715,4cyclictest25-21ksoftirqd/113:28:121
19917993714,4cyclictest25-21ksoftirqd/112:34:221
1991799363,4cyclictest121rcu_preempt14:24:111
1991799363,4cyclictest121rcu_preempt14:24:111
19917993623,3cyclictest25-21ksoftirqd/114:43:491
19917993621,7cyclictest25-21ksoftirqd/113:53:391
19917993621,12cyclictest25-21ksoftirqd/112:39:021
19917993614,4cyclictest25-21ksoftirqd/112:24:061
91191350,1ptp4l401ktimersoftd/314:24:213
91191350,1ptp4l401ktimersoftd/314:24:213
91191350,1ptp4l401ktimersoftd/312:25:403
19917993527,4cyclictest25-21ksoftirqd/111:08:571
1991799352,4cyclictest121rcu_preempt12:49:041
19917993513,4cyclictest25-21ksoftirqd/113:58:201
19917993513,4cyclictest25-21ksoftirqd/111:34:011
1991799350,3cyclictest121rcu_preempt11:48:581
1991799350,3cyclictest121rcu_preempt11:48:581
91191340,1ptp4l401ktimersoftd/310:04:043
19931993425,4cyclictest41-21ksoftirqd/310:58:563
1991799348,5cyclictest25-21ksoftirqd/111:44:021
1991799347,3cyclictest25-21ksoftirqd/113:30:541
1991799345,3cyclictest25-21ksoftirqd/113:34:001
1991799343,8cyclictest25-21ksoftirqd/111:59:021
19917993421,3cyclictest25-21ksoftirqd/111:19:111
19917993419,4cyclictest25-21ksoftirqd/114:16:061
91191330,1ptp4l401ktimersoftd/310:49:003
91191330,1ptp4l401ktimersoftd/309:44:053
19931993320,3cyclictest41-21ksoftirqd/310:34:003
1991799337,3cyclictest25-21ksoftirqd/112:54:051
19917993327,3cyclictest25-21ksoftirqd/112:15:451
19917993321,4cyclictest25-21ksoftirqd/114:44:171
19917993321,3cyclictest25-21ksoftirqd/114:49:111
19917993321,3cyclictest25-21ksoftirqd/110:23:581
19917993320,3cyclictest25-21ksoftirqd/110:03:581
19917993318,12cyclictest25-21ksoftirqd/110:44:041
19917993315,3cyclictest25-21ksoftirqd/113:09:121
19917993315,3cyclictest25-21ksoftirqd/113:09:121
19917993312,4cyclictest25-21ksoftirqd/109:39:071
19917993311,4cyclictest25-21ksoftirqd/110:33:561
19917993311,4cyclictest25-21ksoftirqd/109:44:031
91191320,2ptp4l17014-21kworker/3:211:34:083
91191320,1ptp4l401ktimersoftd/311:20:493
19931993219,3cyclictest41-21ksoftirqd/310:28:563
1991799325,11cyclictest25-21ksoftirqd/114:32:011
1991799325,11cyclictest25-21ksoftirqd/114:32:011
1991799323,6cyclictest25-21ksoftirqd/111:23:561
1991799323,3cyclictest121rcu_preempt13:03:191
19917993213,7cyclictest25-21ksoftirqd/110:13:561
19917993213,7cyclictest25-21ksoftirqd/110:13:561
1991799321,2cyclictest121rcu_preempt10:49:031
19917993211,7cyclictest25-21ksoftirqd/113:14:051
19917993211,4cyclictest25-21ksoftirqd/114:21:371
19917993211,4cyclictest25-21ksoftirqd/112:18:571
19917993211,3cyclictest25-21ksoftirqd/112:11:461
19917993210,4cyclictest25-21ksoftirqd/109:48:531
1993199319,5cyclictest41-21ksoftirqd/311:14:013
1993199315,3cyclictest121rcu_preempt09:49:003
19931993119,3cyclictest41-21ksoftirqd/311:24:023
1992699316,14cyclictest12611-21ssh13:18:582
1991799319,4cyclictest25-21ksoftirqd/110:39:001
1991799313,3cyclictest121rcu_preempt10:18:581
19917993115,4cyclictest25-21ksoftirqd/114:04:161
19917993113,8cyclictest25-21ksoftirqd/115:04:091
1991799309,4cyclictest25-21ksoftirqd/111:04:001
1991799307,4cyclictest25-21ksoftirqd/110:09:031
1991799305,8cyclictest25-21ksoftirqd/110:03:421
1991799303,4cyclictest121rcu_preempt09:54:011
19917993022,4cyclictest25-21ksoftirqd/110:53:551
1991799301,25cyclictest10349-21cron11:33:441
1991799300,3cyclictest121rcu_preempt13:39:061
177192305,11sleep00-21swapper/009:33:530
19931992923,3cyclictest41-21ksoftirqd/313:49:073
1993199290,5cyclictest41-21ksoftirqd/311:29:003
1993199290,2cyclictest121rcu_preempt13:28:203
1993199285,4cyclictest41-21ksoftirqd/310:23:553
37382270,5sleep0101ktimersoftd/010:09:040
36452270,2sleep00-21swapper/014:16:280
1993199277,7cyclictest41-21ksoftirqd/310:44:053
1993199274,4cyclictest41-21ksoftirqd/309:54:013
19926992712,4cyclictest3462-21/usr/sbin/munin11:18:542
247972260,2sleep00-21swapper/009:44:120
19913992618,5cyclictest3138-21grep13:08:590
19913992618,5cyclictest3138-21grep13:08:590
19931992512,3cyclictest41-21ksoftirqd/310:18:423
19931992512,3cyclictest41-21ksoftirqd/310:18:423
19926992518,4cyclictest28226-21grep11:54:042
19913992512,3cyclictest9-21ksoftirqd/015:08:500
1991399250,3cyclictest121rcu_preempt11:48:580
1991399250,3cyclictest121rcu_preempt11:48:580
1991399245,3cyclictest121rcu_preempt11:58:210
19913992418,3cyclictest9-21ksoftirqd/013:24:050
19913992418,3cyclictest9-21ksoftirqd/012:58:590
19913992418,3cyclictest9-21ksoftirqd/010:39:010
19913992418,3cyclictest9-21ksoftirqd/010:34:110
19913992416,6cyclictest12872-1kworker/u9:211:39:380
140732241,3sleep214083-21ssh12:45:392
1992699233,11cyclictest8992-21ssh13:48:582
1992699232,13cyclictest4936-21ssh12:04:042
19926992317,3cyclictest33-21ksoftirqd/209:55:242
19926992316,4cyclictest1247-21sh12:33:462
19926992315,5cyclictest29543-21grep14:44:012
1991399232,13cyclictest12728-21diskmemload12:53:580
19913992318,3cyclictest9-21ksoftirqd/013:34:060
19913992317,3cyclictest9-21ksoftirqd/014:54:060
19913992317,3cyclictest9-21ksoftirqd/014:33:480
19913992317,3cyclictest9-21ksoftirqd/014:33:480
19913992317,3cyclictest9-21ksoftirqd/014:09:060
19913992317,3cyclictest9-21ksoftirqd/010:02:360
1991399230,5cyclictest3669-21tr11:18:570
185432230,2sleep00-21swapper/012:17:500
3492220,4sleep01991399cyclictest12:32:150
1992699223,11cyclictest12728-21diskmemload14:24:012
1992699223,11cyclictest12728-21diskmemload14:24:012
1992699222,17cyclictest7613-21strings12:39:072
19926992216,3cyclictest33-21ksoftirqd/211:17:392
19926992216,3cyclictest33-21ksoftirqd/210:43:432
19926992216,3cyclictest16063-21grep12:14:132
19926992215,4cyclictest46650irq/102-ens3-Tx12:59:122
19926992215,4cyclictest33-21ksoftirqd/209:39:212
19926992214,5cyclictest17234-21sh11:43:092
1992699221,18cyclictest21273-21cron10:48:432
1991399223,13cyclictest63-21kswapd014:03:580
1991399222,17cyclictest25016-21grep14:39:040
19913992217,3cyclictest9-21ksoftirqd/013:59:120
19913992217,3cyclictest9-21ksoftirqd/013:39:110
19913992217,3cyclictest9-21ksoftirqd/011:46:220
19913992217,3cyclictest9-21ksoftirqd/009:43:430
1991399221,6cyclictest0-21swapper/012:48:450
19913992216,3cyclictest9-21ksoftirqd/014:48:490
19913992216,3cyclictest9-21ksoftirqd/013:29:040
19913992216,3cyclictest9-21ksoftirqd/012:04:120
19913992216,3cyclictest9-21ksoftirqd/011:24:550
19913992216,3cyclictest9-21ksoftirqd/011:03:440
19913992216,3cyclictest9-21ksoftirqd/009:52:360
19913992215,4cyclictest9-21ksoftirqd/011:11:480
1991399221,3cyclictest4443-21ssh12:03:440
1991399220,18cyclictest30862-21expr13:03:590
292612211,16sleep029264-21snmp_o5powerctl09:54:080
19926992116,3cyclictest33-21ksoftirqd/210:57:052
19926992115,3cyclictest33-21ksoftirqd/212:54:052
19926992115,3cyclictest33-21ksoftirqd/211:29:552
19926992115,3cyclictest33-21ksoftirqd/210:01:532
19926992115,3cyclictest13103-21sh12:12:022
19926992114,4cyclictest571-21basename11:59:032
19926992114,4cyclictest33-21ksoftirqd/210:09:422
19926992114,4cyclictest30526-21/usr/sbin/munin13:04:142
19926992114,4cyclictest28447-21sh14:42:492
19926992114,4cyclictest27865-21sed10:59:122
19926992114,4cyclictest14643-21sed13:54:072
1991399218,10cyclictest20338-21sed12:19:030
19913992116,3cyclictest9-21ksoftirqd/014:59:190
19913992116,3cyclictest9-21ksoftirqd/012:09:090
19913992115,3cyclictest9-21ksoftirqd/013:48:590
19913992115,3cyclictest9-21ksoftirqd/013:44:000
19913992115,3cyclictest9-21ksoftirqd/011:37:540
19913992115,3cyclictest9-21ksoftirqd/011:28:560
19913992115,3cyclictest9-21ksoftirqd/010:07:220
19913992115,3cyclictest7065-21grep14:19:160
19913992114,4cyclictest12023-21/usr/sbin/munin10:28:530
19913992112,6cyclictest46450irq/100-ens3-Tx14:49:000
1991399211,17cyclictest20257-21grep14:34:050
173112210,16sleep20-21swapper/213:23:572
1992699208,10cyclictest22318-21cpuspeed_turbos13:28:592
19926992015,3cyclictest33-21ksoftirqd/214:48:502
19926992014,4cyclictest2955-21sh14:15:122
19926992014,3cyclictest8617-21sh14:21:492
19926992014,3cyclictest5854-21sh13:11:032
19926992014,3cyclictest5854-21sh13:11:032
19926992014,3cyclictest4026-21sh13:43:512
19926992014,3cyclictest33-21ksoftirqd/211:26:172
19926992014,3cyclictest31871-21interrupts11:09:032
19926992014,3cyclictest30485-21sh14:10:022
19926992014,3cyclictest27336-21sh12:26:062
19926992014,3cyclictest23086-21/usr/sbin/munin09:44:092
19926992014,3cyclictest22952-21sh14:36:372
19926992014,3cyclictest22880-21apt-get11:48:582
19926992014,3cyclictest22880-21apt-get11:48:582
19926992014,3cyclictest20357-21sh11:45:082
19926992014,3cyclictest1620-21sh13:39:592
19926992014,3cyclictest16134-21sh12:48:522
19926992014,3cyclictest13987-21sh15:00:202
19926992013,4cyclictest9806-21sh13:14:332
19926992013,4cyclictest9630-21/usr/sbin/munin10:23:592
19926992013,4cyclictest7034-21idleruntime-cro10:18:422
19926992013,4cyclictest7034-21idleruntime-cro10:18:422
19926992013,4cyclictest33-21ksoftirqd/209:50:432
19926992013,4cyclictest27568-21basename13:34:042
19926992013,4cyclictest26403-21sh14:06:212
19926992013,4cyclictest20768-21sh13:59:492
19926992013,4cyclictest1822-21cstates12:33:592
1991399208,4cyclictest9-21ksoftirqd/013:19:000
1991399204,3cyclictest12728-21diskmemload13:17:560
19913992015,3cyclictest9-21ksoftirqd/012:24:050
19913992015,3cyclictest9-21ksoftirqd/011:04:030
19913992015,3cyclictest9-21ksoftirqd/010:48:440
19913992015,3cyclictest9-21ksoftirqd/010:14:010
19913992015,3cyclictest9-21ksoftirqd/010:14:010
19913992014,3cyclictest9-21ksoftirqd/010:58:440
19913992013,4cyclictest9684-21bc10:23:530
19913992012,5cyclictest10452-21sh14:24:030
19913992012,5cyclictest10452-21sh14:24:030
1991399201,16cyclictest9105-21ssh12:40:400
19926991914,3cyclictest18349-21sh15:04:352
19926991913,3cyclictest33-21ksoftirqd/210:21:232
19926991913,3cyclictest13038-21latency10:29:022
19926991912,4cyclictest6914-21cpuspeed_turbos14:54:012
19926991912,4cyclictest29068-21expr11:03:562
19926991912,4cyclictest14895-21gltestperf10:33:582
19926991912,4cyclictest12726-21turbostat.cron11:38:442
19913991914,3cyclictest9-21ksoftirqd/013:54:070
19913991914,3cyclictest9-21ksoftirqd/010:48:560
19913991914,3cyclictest9-21ksoftirqd/010:18:560
19913991913,3cyclictest9-21ksoftirqd/012:49:050
19913991913,3cyclictest1241-21cat11:13:550
19926991812,3cyclictest358-21expr10:03:552
19926991812,3cyclictest21766-21seq10:48:562
1991399181,14cyclictest3759-21runrttasks12:34:440
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional