You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2023-03-27 - 03:10

x86 Intel Core i3-2310E @2100 MHz, Linux 5.10.47-rt46 (Profile)

Latency plot of system in rack #8, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by a total of 2 SMIs that occured during the measurement.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot2.osadl.org (updated Mon Mar 27, 2023 00:43:38)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1230699127125,1cyclictest0-21swapper/319:12:013
1230499121112,8cyclictest0-21swapper/119:11:521
245042910,0sleep20-21swapper/222:09:042
1191429148,32sleep00-21swapper/019:10:200
1184029149,31sleep30-21swapper/319:09:233
1187029048,31sleep10-21swapper/119:09:451
1172229047,32sleep20-21swapper/219:07:492
151852840,0sleep20-21swapper/222:01:392
103262840,0sleep30-21swapper/322:53:223
12305997270,1cyclictest0-21swapper/219:11:522
12306996966,1cyclictest0-21swapper/319:43:243
12303994846,1cyclictest0-21swapper/019:11:520
65272370,2sleep20-21swapper/220:27:082
1230699337,25cyclictest17553-21kworker/u8:000:02:013
12304993325,7cyclictest0-21swapper/119:43:241
129002320,1chrt0-21swapper/023:54:110
297232310,1sleep30-21swapper/322:13:303
1230499300,29cyclictest0-21swapper/100:02:001
264342280,0sleep10-21swapper/123:07:391
1230599237,15cyclictest0-21swapper/221:12:002
211662220,0sleep00-21swapper/023:32:110
1230499220,21cyclictest0-21swapper/121:52:011
1230599193,10cyclictest0-21swapper/222:55:192
1230399190,18cyclictest0-21swapper/021:12:000
1230699186,9cyclictest0-21swapper/323:36:083
1230699164,9cyclictest0-21swapper/323:47:023
1230699162,8cyclictest11926-21/usr/sbin/munin20:42:033
1230699162,11cyclictest3261-21ssh21:50:533
1230699161,10cyclictest6984-21ssh22:22:063
1230699160,1cyclictest0-21swapper/300:21:493
1230599165,6cyclictest0-21swapper/220:21:532
1230499160,1cyclictest0-21swapper/100:21:491
1230399162,8cyclictest7387-21perf20:31:470
1230699150,14cyclictest0-21swapper/300:21:483
1230499155,5cyclictest0-21swapper/123:43:021
1230499154,8cyclictest25329-21ssh21:42:021
12304991512,2cyclictest22-21ksoftirqd/123:01:481
12304991512,2cyclictest22-21ksoftirqd/123:01:481
12304991510,3cyclictest16627-21ntp_states23:57:091
1230699142,7cyclictest0-21swapper/322:57:053
1230699142,7cyclictest0-21swapper/322:57:053
1230599148,4cyclictest27192-21ssh21:42:322
1230599143,6cyclictest0-21swapper/223:05:312
1230599140,9cyclictest21085-21sh22:34:362
1230499149,3cyclictest22-21ksoftirqd/100:41:481
1230499144,5cyclictest0-21swapper/119:28:041
1230499140,9cyclictest0-21swapper/120:42:041
98422130,0chrt0-21swapper/122:24:471
35082130,3sleep03514-21mailstats22:47:100
1230699135,3cyclictest0-21swapper/319:23:343
1230699134,4cyclictest2537-21ssh00:14:423
1230599138,3cyclictest9211-21idleruntime-cro21:26:482
1230599138,3cyclictest0-21swapper/219:49:462
1230599135,3cyclictest0-21swapper/222:36:512
1230599134,4cyclictest0-21swapper/223:11:252
1230599134,4cyclictest0-21swapper/220:19:472
1230599134,4cyclictest0-21swapper/200:23:062
1230599134,4cyclictest0-21swapper/200:17:212
1230599134,3cyclictest0-21swapper/221:46:532
1230599131,2cyclictest0-21swapper/222:48:002
1230499139,3cyclictest22-21ksoftirqd/123:36:481
1230499137,4cyclictest23259-21ssh21:39:481
1230499137,2cyclictest5261-1kworker/u9:122:52:311
1230499136,2cyclictest22-21ksoftirqd/100:21:381
1230499133,5cyclictest29116-21ssh00:08:511
1230499133,5cyclictest0-21swapper/121:32:051
1230499133,5cyclictest0-21swapper/119:32:491
1230499131,10cyclictest0-21swapper/123:47:091
1230399134,8cyclictest0-21swapper/020:49:180
1230399133,5cyclictest0-21swapper/023:00:340
1230399133,5cyclictest0-21swapper/023:00:340
1230399133,5cyclictest0-21swapper/000:19:270
6942120,0sleep10-21swapper/121:18:091
6942120,0sleep10-21swapper/121:18:091
204202120,1sleep30-21swapper/323:02:163
1230699127,3cyclictest8286-21lockfile-create23:21:483
1230699125,6cyclictest0-21swapper/322:07:383
1230699125,6cyclictest0-21swapper/321:34:103
1230699125,5cyclictest519-21dbus-daemon22:03:303
1230699124,3cyclictest0-21swapper/322:19:153
1230699124,3cyclictest0-21swapper/320:46:483
1230699124,3cyclictest0-21swapper/300:31:223
1230699122,9cyclictest37050irq/33-enp3s0-r23:28:053
1230699121,6cyclictest24879-21ssh23:07:033
1230599128,3cyclictest28-21ksoftirqd/221:01:482
1230599128,2cyclictest20855-21grep21:02:012
1230599128,2cyclictest17670-21cron19:26:482
1230599126,4cyclictest519-21dbus-daemon22:21:482
1230599126,4cyclictest31587-21awk23:42:032
1230599126,4cyclictest16277-21cut22:02:042
1230599125,6cyclictest0-21swapper/219:29:042
1230599125,5cyclictest26131-21fschecks_time23:37:042
1230599125,5cyclictest25930-21cat00:06:482
1230599124,4cyclictest0-21swapper/223:31:562
1230599124,3cyclictest0-21swapper/200:33:342
1230599123,4cyclictest0-21swapper/223:53:152
1230599123,4cyclictest0-21swapper/219:19:062
1230599123,4cyclictest0-21swapper/200:06:572
1230599122,5cyclictest0-21swapper/220:15:172
1230599121,6cyclictest0-21swapper/221:19:192
1230599121,6cyclictest0-21swapper/221:19:192
1230599120,11cyclictest0-21swapper/223:57:002
1230599120,10cyclictest21064-21apt-get19:41:502
1230499126,4cyclictest18987-21tr23:02:031
1230499126,1cyclictest22-21ksoftirqd/123:37:041
1230499125,1cyclictest22-21ksoftirqd/123:31:351
1230499124,5cyclictest0-21swapper/120:05:201
1230499123,1cyclictest141rcu_preempt23:52:061
1230499122,5cyclictest22-21ksoftirqd/120:13:261
1230499122,2cyclictest141rcu_preempt23:12:121
1230499121,6cyclictest22-21ksoftirqd/121:57:001
1230499121,10cyclictest0-21swapper/121:11:591
1230399127,4cyclictest0-21swapper/023:47:320
1230399126,4cyclictest18163-21cron20:56:480
1230399124,5cyclictest0-21swapper/022:40:070
1230399123,4cyclictest0-21swapper/000:02:390
1230399121,10cyclictest24672-21diskmemload23:23:270
274162110,1sleep20-21swapper/200:38:302
1230699119,1cyclictest0-21swapper/323:16:483
1230699118,2cyclictest0-21swapper/322:37:383
1230699118,2cyclictest0-21swapper/321:45:583
1230699117,2cyclictest564-21runrttasks23:46:073
1230699117,2cyclictest24045-21latency_hist19:51:483
1230699117,2cyclictest10254-21grep21:57:033
1230699116,4cyclictest31890-21ntp_states21:17:103
1230699116,4cyclictest31890-21ntp_states21:17:103
1230699116,3cyclictest21120-21sed00:32:093
1230699116,3cyclictest14155-21sh22:28:053
1230699115,4cyclictest6818-21sed20:27:123
1230699115,4cyclictest24875-21basename21:41:483
1230699115,4cyclictest21169-21head21:02:043
1230699115,4cyclictest16797-21uniq20:52:043
1230699114,2cyclictest20240-21ssh00:01:223
1230699113,3cyclictest0-21swapper/321:27:523
1230699113,3cyclictest0-21swapper/319:58:203
1230699113,3cyclictest0-21swapper/319:54:463
1230699112,7cyclictest25784-21grep23:37:023
1230699112,4cyclictest0-21swapper/321:08:263
1230699111,6cyclictest17927-21taskset19:27:133
1230699111,6cyclictest0-21swapper/320:35:213
1230699110,4cyclictest25604-21cut00:37:043
1230599116,3cyclictest5413-21sh23:18:062
1230599116,3cyclictest408-21sendmail_mailqu00:12:102
1230599115,4cyclictest15260-21grep20:47:102
1230599114,6cyclictest0-21swapper/221:30:432
1230599114,5cyclictest231-21systemd-journal21:41:482
1230599114,5cyclictest19958-21grep19:32:162
1230599112,4cyclictest18470-21basename23:01:482
1230599112,4cyclictest18470-21basename23:01:482
1230599112,4cyclictest0-21swapper/222:42:312
1230599111,7cyclictest27041-21runrttasks22:11:592
1230599111,3cyclictest2699-21ls23:16:482
1230499118,2cyclictest22-21ksoftirqd/121:23:321
1230499118,2cyclictest22-21ksoftirqd/120:21:471
1230499116,3cyclictest3163-21tr23:17:011
1230499115,5cyclictest0-21swapper/122:30:571
1230499115,5cyclictest0-21swapper/121:28:221
1230499115,4cyclictest4139-21irqstats20:22:061
1230499115,4cyclictest29202-21tr22:42:021
1230499115,1cyclictest22-21ksoftirqd/121:47:031
1230499114,5cyclictest12216-21ssh23:24:411
1230499114,5cyclictest0-21swapper/122:48:101
1230499114,5cyclictest0-21swapper/122:02:061
1230499114,5cyclictest0-21swapper/120:53:511
1230499114,5cyclictest0-21swapper/120:10:531
1230499114,5cyclictest0-21swapper/119:58:111
1230499113,6cyclictest141rcu_preempt20:47:051
1230499113,6cyclictest141rcu_preempt20:27:121
1230499113,2cyclictest141rcu_preempt22:17:031
1230499110,8cyclictest0-21swapper/119:18:091
1230499110,6cyclictest27805-21ssh22:40:551
1230499110,6cyclictest0-21swapper/122:35:431
1230399115,4cyclictest26211-21grep19:56:480
1230399115,4cyclictest231-21systemd-journal19:56:480
1230399115,4cyclictest10303-21grep21:57:040
1230399114,5cyclictest31475-21ssh22:15:450
1230399114,5cyclictest0-21swapper/021:42:500
1230399114,5cyclictest0-21swapper/021:32:490
1230399114,5cyclictest0-21swapper/019:21:220
1230399112,4cyclictest0-21swapper/020:18:110
1230399111,7cyclictest31112-21ssh23:41:590
1230399111,5cyclictest0-21swapper/023:27:020
1230399111,5cyclictest0-21swapper/021:47:470
1230399111,5cyclictest0-21swapper/021:07:060
1230399111,5cyclictest0-21swapper/020:38:180
1230399110,9cyclictest3951-21cron00:16:480
1230399110,9cyclictest26202-21grep23:37:050
1230699106,2cyclictest564-21runrttasks00:10:023
1230699106,2cyclictest17451-21sh22:31:573
1230699105,3cyclictest25247-21cpuspeed_turbos21:12:013
1230699104,4cyclictest3930-21dpkg21:22:023
1230699104,4cyclictest3260-21cut22:47:073
1230699104,4cyclictest31430-21cut20:12:013
1230699104,4cyclictest30116-21/usr/sbin/munin22:42:103
1230699104,4cyclictest0-21swapper/321:53:383
1230699103,6cyclictest0-21swapper/323:22:053
1230699103,5cyclictest586-21snmpd20:22:203
1230699103,5cyclictest586-21snmpd20:08:503
1230699103,5cyclictest586-21snmpd19:38:073
1230699102,6cyclictest0-21swapper/320:01:593
1230699101,7cyclictest626-21gdbus23:52:163
1230699101,7cyclictest18856-21diskstats20:57:033
1230699101,2cyclictest14792-21fschecks_time19:16:533
1230599106,3cyclictest15961-21sh23:27:172
1230599105,4cyclictest23816-21tr21:07:092
1230599105,3cyclictest5718-21timerandwakeup23:47:122
1230599105,3cyclictest24291-21acpi19:52:002
1230599105,3cyclictest16462-21needreboot21:32:072
1230599105,3cyclictest0-21swapper/220:56:092
1230599105,3cyclictest0-21swapper/220:46:482
1230599104,4cyclictest9332-21cut23:22:072
1230599104,4cyclictest8818-21processes20:32:122
1230599104,4cyclictest28522-21awk20:02:122
1230599103,5cyclictest586-21snmpd20:00:172
1230599103,5cyclictest586-21snmpd19:37:132
1230599103,5cyclictest29236-21cat20:07:062
1230599101,8cyclictest0-21swapper/222:22:022
1230599101,7cyclictest4362-21/usr/sbin/munin21:52:102
1230599101,3cyclictest12920-21memory22:27:102
1230599101,2cyclictest11678-21idleruntime-cro20:41:472
1230599100,8cyclictest14791-21ssh00:27:022
1230499107,2cyclictest21384-21ssh22:07:011
1230499106,3cyclictest22-21ksoftirqd/119:41:481
1230499105,4cyclictest0-21swapper/119:49:041
1230499104,5cyclictest0-21swapper/121:09:191
1230499104,5cyclictest0-21swapper/120:35:001
1230499104,5cyclictest0-21swapper/100:12:241
1230499104,4cyclictest24917-21head19:52:051
1230499104,4cyclictest20138-21cut00:32:011
1230499104,4cyclictest14828-21cut00:27:031
1230499104,4cyclictest0-21swapper/120:57:211
1230499104,2cyclictest22-21ksoftirqd/120:37:121
1230499100,5cyclictest28828-21ssh22:12:211
1230499100,1cyclictest15775-21apt-get19:21:591
1230399108,1cyclictest0-21swapper/022:19:470
1230399107,2cyclictest8326-21fschecks_count22:52:070
1230399106,3cyclictest17691-21sh22:02:350
1230399105,4cyclictest0-21swapper/023:16:560
1230399105,4cyclictest0-21swapper/023:10:250
1230399105,3cyclictest8249-21grep20:32:060
1230399105,3cyclictest24292-21sh22:08:480
1230399105,3cyclictest18693-21sh23:01:490
1230399104,5cyclictest0-21swapper/023:12:070
1230399104,5cyclictest0-21swapper/022:32:140
1230399104,5cyclictest0-21swapper/019:39:280
1230399103,5cyclictest30054-21strings22:42:090
1230399103,5cyclictest24873-21ssh00:36:550
1230399103,5cyclictest0-21swapper/022:27:030
1230399102,7cyclictest0-21swapper/022:26:210
1230399102,3cyclictest0-21swapper/019:25:010
1230399101,8cyclictest0-21swapper/020:07:070
1230399101,4cyclictest0-21swapper/019:36:340
1230399101,4cyclictest0-21swapper/000:32:280
1230399101,4cyclictest0-21swapper/000:30:160
1230399101,2cyclictest15431-21apt-get23:57:000
1230399100,9cyclictest0-21swapper/021:38:370
1230399100,9cyclictest0-21swapper/021:23:570
1230399100,1cyclictest0-21swapper/000:22:280
123069994,4cyclictest1154-21cut20:17:003
123069993,4cyclictest586-21snmpd20:36:543
123069993,4cyclictest18780-21grep19:32:033
123049994,4cyclictest0-21swapper/121:02:521
123039996,2cyclictest30724-21ssh21:17:010
123039996,2cyclictest30724-21ssh21:17:010
123039996,2cyclictest27802-21if_enp4s020:02:050
123039996,2cyclictest20675-21aten2_r8power_p21:02:010
123039995,2cyclictest9592-21basename21:27:000
123039995,2cyclictest4619-21grep21:52:020
123039995,2cyclictest31644-21tr20:12:020
123039994,4cyclictest3304-21apt-get20:21:590
123039994,4cyclictest22279-21if_err_enp4s019:42:060
123039994,4cyclictest0-21swapper/019:46:550
123039994,4cyclictest0-21swapper/019:31:250
123039994,3cyclictest12966-21needreboot20:42:070
123039993,4cyclictest26532-21cat00:07:020
123039983,4cyclictest18393-21seq20:56:520
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional