You are here: Home / Projects / OSADL QA Farm Real-time / Latency plots / 
2022-12-04 - 03:10

x86 Intel Core i3-2310E @2100 MHz, Linux 5.10.47-rt46 (Profile)

Latency plot of system in rack #8, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by a total of 2 SMIs that occured during the measurement.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the 376 highest latencies:
System rack8slot2.osadl.org (updated Sun Dec 04, 2022 00:43:37)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
21608991430,1cyclictest0-21swapper/019:50:520
2161099142140,1cyclictest0-21swapper/219:10:362
2161099140138,1cyclictest0-21swapper/219:50:522
86662920,1sleep10-21swapper/122:07:221
10232890,1sleep10-21swapper/122:59:361
153452880,1sleep10-21swapper/100:10:591
15802860,0sleep00-21swapper/020:50:500
181862820,0sleep30-21swapper/321:17:423
21611994036,1cyclictest0-21swapper/319:10:513
302812290,0sleep20-21swapper/222:26:232
39022280,1sleep00-21swapper/000:00:520
21609992825,2cyclictest75-21kswapd022:45:351
21609992719,7cyclictest0-21swapper/119:10:511
151072260,1sleep20-21swapper/221:15:412
51412250,0sleep30-21swapper/319:45:443
49742250,1sleep10-21swapper/123:02:331
321592220,1sleep10-21swapper/123:26:541
289462200,1sleep10-21swapper/121:27:101
2160999200,1cyclictest0-21swapper/119:50:441
2161199184,8cyclictest27521-21grep21:25:533
2160899185,7cyclictest16721-21munin-run22:15:330
215682184,10sleep00-21swapper/019:10:350
2161199173,9cyclictest12581-21ssh23:39:133
310392160,5sleep012-21ksoftirqd/022:56:230
2161199157,3cyclictest37050irq/33-enp3s0-r22:30:593
2161199154,9cyclictest0-21swapper/320:50:493
2161199153,4cyclictest0-21swapper/320:06:253
2160999156,5cyclictest26759-21ssh22:52:591
2160899152,7cyclictest27908-21munin-run22:25:320
265322140,0sleep30-21swapper/322:52:413
2161199146,2cyclictest34-21ksoftirqd/322:00:523
2161199144,6cyclictest0-21swapper/322:24:523
2161199140,8cyclictest20330-21ssh22:46:453
2161099146,3cyclictest29560-21irqstats00:25:532
2161199135,5cyclictest0-21swapper/321:12:533
2161199134,8cyclictest0-21swapper/321:47:273
2161199134,8cyclictest0-21swapper/321:34:593
2161199134,8cyclictest0-21swapper/320:48:213
2161199134,8cyclictest0-21swapper/320:14:433
2161199134,4cyclictest0-21swapper/323:42:313
2161199134,4cyclictest0-21swapper/323:29:523
2161199134,4cyclictest0-21swapper/319:42:153
2161199131,1cyclictest141rcu_preempt22:41:293
2161199130,8cyclictest0-21swapper/322:15:413
2161099138,4cyclictest8151-21ntpq22:35:582
2161099135,6cyclictest37150irq/34-enp3s0-t22:15:532
241412120,0sleep00-21swapper/023:49:530
241032120,1sleep00-21swapper/022:50:510
2161199127,3cyclictest564-21runrttasks00:09:123
2161199126,4cyclictest31549-21uniq19:30:483
2161199126,4cyclictest24675-21basename23:50:333
2161199126,4cyclictest13489-21cpuspeed_turbos23:10:503
2161199125,5cyclictest34-21ksoftirqd/323:07:193
2161199124,6cyclictest0-21swapper/322:12:253
2161199124,5cyclictest0-21swapper/323:58:313
2161199124,5cyclictest0-21swapper/322:08:453
2161199124,5cyclictest0-21swapper/321:42:153
2161199124,5cyclictest0-21swapper/321:42:153
2161199124,5cyclictest0-21swapper/320:01:253
2161199124,5cyclictest0-21swapper/319:52:113
2161199124,5cyclictest0-21swapper/319:38:273
2161199124,5cyclictest0-21swapper/300:38:353
2161199124,3cyclictest0-21swapper/300:00:493
2161199123,8cyclictest0-21swapper/320:30:353
2161199123,8cyclictest0-21swapper/319:28:193
2161199123,4cyclictest0-21swapper/321:52:453
2161199123,4cyclictest0-21swapper/320:59:383
2161199122,8cyclictest0-21swapper/322:25:553
2161199122,5cyclictest31990-21ssh22:57:393
2161199121,6cyclictest4186-21ssh23:01:293
2161199120,2cyclictest0-21swapper/321:55:543
2161099126,4cyclictest4782-21wc00:00:582
2161099126,4cyclictest23686-21systemctl20:25:572
2161099120,3cyclictest25486-21df23:50:482
2160999126,4cyclictest895-21tr20:50:461
2160999126,4cyclictest6294-21grep21:00:571
2160999126,4cyclictest4034-21grep00:00:521
2160999126,4cyclictest15602-21grep23:40:581
2160999125,5cyclictest1955-21cat22:30:541
2160999124,4cyclictest0-21swapper/120:24:221
2160899129,2cyclictest21506-21grep21:20:540
2161199115,5cyclictest0-21swapper/300:21:113
2161199115,4cyclictest22518-21basename20:25:473
2161199115,3cyclictest0-21swapper/319:21:143
2161199114,6cyclictest0-21swapper/321:21:123
2161199114,5cyclictest519-21dbus-daemon20:40:323
2161199114,5cyclictest1-21systemd19:15:363
2161199114,5cyclictest0-21swapper/323:18:373
2161199114,5cyclictest0-21swapper/320:20:493
2161199114,5cyclictest0-21swapper/319:56:473
2161199114,5cyclictest0-21swapper/319:56:473
2161199113,5cyclictest0-21swapper/323:20:453
2161199113,5cyclictest0-21swapper/321:35:433
2161199113,5cyclictest0-21swapper/300:29:313
2161199113,3cyclictest2965-21/usr/sbin/munin23:30:553
2161199112,7cyclictest34-21ksoftirqd/300:30:553
2161199111,5cyclictest0-21swapper/300:14:513
2161199110,9cyclictest25797-21awk23:50:503
2161199110,9cyclictest19091-21cpuspeed_turbos00:15:493
2161199110,4cyclictest7463-21fschecks_count22:35:523
2161099117,3cyclictest28923-21apt20:40:442
2161099116,4cyclictest519-21dbus-daemon21:31:012
2161099116,4cyclictest1789-21cstates22:30:532
2161099116,4cyclictest17019-21ntpq20:10:542
2161099116,4cyclictest14376-21cut00:10:502
2161099116,3cyclictest10031-21head19:55:482
2161099116,3cyclictest10031-21head19:55:482
2161099115,4cyclictest9738-21pmu-power23:35:572
2161099114,6cyclictest0-21swapper/221:13:122
2161099114,6cyclictest0-21swapper/219:44:112
2161099114,5cyclictest6646-21expr22:05:512
2161099114,2cyclictest37150irq/34-enp3s0-t21:45:562
2161099114,2cyclictest0-21swapper/221:56:122
2161099113,7cyclictest25493-21awk23:20:582
2161099113,7cyclictest13109-21unixbench_multi22:10:592
2161099113,2cyclictest0-21swapper/223:07:032
2161099113,2cyclictest0-21swapper/221:53:082
2161099111,4cyclictest18046-21sshd23:15:112
2161099110,5cyclictest7263-21latency_hist23:05:332
2161099110,5cyclictest12702-21ntp_states20:00:532
2161099110,4cyclictest9206-21diskmemload21:30:122
2161099110,4cyclictest9206-21diskmemload00:15:362
2161099110,4cyclictest29347-21df_inode19:25:472
2161099110,4cyclictest27901-21dump-pmu-power22:25:322
2161099110,4cyclictest23559-21apt-get22:50:462
2161099110,2cyclictest0-21swapper/222:55:492
2160999117,2cyclictest9206-21diskmemload23:35:551
2160999115,5cyclictest0-21swapper/123:16:211
2160999115,5cyclictest0-21swapper/122:16:211
2160999115,4cyclictest586-21snmpd19:41:161
2160999115,4cyclictest4033-21ntpq20:55:521
2160999115,4cyclictest23970-21open_inodes22:20:571
2160999115,4cyclictest16388-21awk21:15:551
2160999115,4cyclictest1072-21uniq22:00:541
2160999115,4cyclictest10440-21if_err_enp3s021:10:501
2160999113,2cyclictest564-21runrttasks23:09:181
2160999111,5cyclictest0-21swapper/119:18:591
2160999111,3cyclictest12895-21kworker/1:020:20:331
2160999110,9cyclictest26708-21ssh23:21:571
2160999110,2cyclictest9767-21ssh00:05:561
2160899118,2cyclictest0-21swapper/019:30:520
2160899117,2cyclictest9206-21diskmemload22:00:500
2160899117,2cyclictest26951-21perf21:55:330
2160899116,4cyclictest626-21gdbus22:40:590
2160899116,4cyclictest17157-21cut20:10:550
2160899116,4cyclictest12-21ksoftirqd/000:35:330
2160899116,3cyclictest564-21runrttasks20:35:470
2160899116,3cyclictest179762sleep022:15:560
2160899115,4cyclictest3807-21kernelversion20:55:500
2160899115,4cyclictest231-21systemd-journal20:35:330
2160899115,4cyclictest14326-21grep23:40:480
2160899114,5cyclictest24456-21wget23:20:510
2160899114,5cyclictest231-21systemd-journal22:36:030
2160899114,5cyclictest20545-21grep20:20:480
2160899114,5cyclictest1-21systemd00:06:110
2160899113,6cyclictest141rcu_preempt22:48:410
2160899110,9cyclictest4829-21perf21:35:330
2161199104,5cyclictest0-21swapper/321:07:013
2161199104,5cyclictest0-21swapper/321:00:353
2161199104,5cyclictest0-21swapper/320:15:533
2161199101,4cyclictest0-21swapper/320:44:093
2161099108,1cyclictest0-21swapper/220:15:482
2161099107,2cyclictest564-21runrttasks23:47:062
2161099107,2cyclictest29907-21aten2_r8power_p23:25:512
2161099107,2cyclictest19453-21grep23:15:542
2161099107,2cyclictest1684-21ntp_states22:00:592
2161099105,4cyclictest20534-21cut20:20:482
2161099105,4cyclictest18786-21hddtemp_smartct22:45:522
2161099105,4cyclictest1345-21perl19:35:482
2161099105,3cyclictest8688-21sh00:05:392
2161099104,4cyclictest7481-21grep21:05:452
2161099104,4cyclictest5401-21cpuspeed_turbos19:45:472
2161099104,4cyclictest31984-21latency19:30:512
2161099104,1cyclictest0-21swapper/200:21:232
2161099103,6cyclictest37150irq/34-enp3s0-t22:40:452
2161099103,5cyclictest7105-21cat21:05:322
2161099102,4cyclictest586-21snmpd00:36:252
2161099101,5cyclictest8120-21rm21:37:372
2161099101,3cyclictest32207-21users23:55:582
2161099100,4cyclictest9206-21diskmemload23:40:452
2161099100,4cyclictest9206-21diskmemload21:42:402
2161099100,4cyclictest9206-21diskmemload21:42:402
2161099100,4cyclictest5503-21ssh23:32:102
2161099100,4cyclictest26728-21/usr/sbin/munin20:35:472
2161099100,4cyclictest20606-21ssh21:20:332
2161099100,4cyclictest1280-21aten2_r8power_p00:30:522
2160999108,1cyclictest28777-21grep21:55:571
2160999107,2cyclictest28136-21users19:20:571
2160999106,3cyclictest22-21ksoftirqd/122:10:471
2160999106,3cyclictest16852-21head21:45:481
2160999106,2cyclictest25129-21sh21:24:281
2160999105,4cyclictest1125-21rdmsr21:31:011
2160999105,4cyclictest0-21swapper/100:17:231
2160999104,5cyclictest0-21swapper/123:47:311
2160999104,5cyclictest0-21swapper/123:31:491
2160999104,5cyclictest0-21swapper/123:14:331
2160999104,5cyclictest0-21swapper/122:45:141
2160999104,5cyclictest0-21swapper/122:36:321
2160999104,5cyclictest0-21swapper/121:41:391
2160999104,5cyclictest0-21swapper/121:41:391
2160999104,5cyclictest0-21swapper/119:49:451
2160999104,4cyclictest16952-21grep20:10:521
2160999103,5cyclictest0-21swapper/119:37:061
2160999100,9cyclictest13787-21apt-get20:05:461
2160999100,9cyclictest0-21swapper/100:25:571
2160999100,1cyclictest0-21swapper/123:50:371
2160899108,1cyclictest0-21swapper/023:35:420
2160899108,1cyclictest0-21swapper/023:00:510
2160899106,3cyclictest23184-21kworker/u8:121:10:540
2160899106,3cyclictest1617-21cpu22:30:530
2160899106,2cyclictest9206-21diskmemload23:33:190
2160899106,2cyclictest9206-21diskmemload21:29:250
2160899105,4cyclictest9214-21turbostat.cron21:10:320
2160899105,4cyclictest879-21ntpd19:25:530
2160899105,3cyclictest8752-21missed_timers23:05:560
2160899105,3cyclictest30121-21sh22:26:100
2160899105,3cyclictest26393-21mailstats23:50:560
2160899105,3cyclictest17488-21sed21:45:540
2160899104,5cyclictest30753-21cat23:55:470
2160899104,4cyclictest9547-21cut19:55:460
2160899104,4cyclictest9547-21cut19:55:460
2160899104,4cyclictest586-21snmpd23:16:100
2160899104,4cyclictest586-21snmpd21:18:540
2160899104,4cyclictest5661-21fschecks_time19:45:490
2160899104,4cyclictest32094-21cut20:45:520
2160899104,4cyclictest26352-21cron19:20:320
2160899104,4cyclictest25233-21wc00:20:580
2160899104,4cyclictest231-21systemd-journal00:40:330
2160899104,4cyclictest14203-21cstates00:10:490
2160899104,4cyclictest13852-21tail23:10:520
2160899104,4cyclictest11648-21cut20:00:450
2160899103,5cyclictest586-21snmpd21:57:100
2160899103,5cyclictest586-21snmpd21:40:450
2160899103,5cyclictest586-21snmpd21:40:450
2160899103,5cyclictest586-21snmpd20:09:010
2160899103,5cyclictest20442-21awk00:16:000
2160899102,6cyclictest28672-21apt-get00:25:470
2160899102,3cyclictest0-21swapper/020:15:500
2160899101,5cyclictest7647-21ssh21:37:000
2160899101,5cyclictest0-21swapper/019:40:330
2160899101,4cyclictest30154-21cstates23:25:520
216109996,2cyclictest4203-21sendmail_mailst20:55:542
216109994,4cyclictest31236-21cpu20:45:462
216109994,4cyclictest27127-21tune2fs19:20:472
216109994,4cyclictest26010-21unixbench-2d20:30:572
216109994,4cyclictest14214-21df_inode20:05:482
216109992,6cyclictest0-21swapper/219:17:112
216109991,6cyclictest1230-21df_inode20:50:472
216099997,1cyclictest0-21swapper/120:26:271
216099997,1cyclictest0-21swapper/119:30:071
216099996,2cyclictest9206-21diskmemload00:31:361
216099996,2cyclictest736-21sh23:57:141
216099995,3cyclictest29971-21uptime22:26:001
216099995,3cyclictest10476-21needreboot19:55:521
216099995,3cyclictest10476-21needreboot19:55:521
216099995,2cyclictest30829-21latency_hist20:45:321
216099994,4cyclictest586-21snmpd21:52:511
216099994,4cyclictest24307-21tune2fs00:20:501
216099994,4cyclictest231-21systemd-journal21:05:461
216099994,4cyclictest0-21swapper/120:32:551
216099994,4cyclictest0-21swapper/119:31:091
216099994,3cyclictest6759-21grep21:35:591
216099994,3cyclictest32086-21/usr/sbin/munin20:45:521
216099993,5cyclictest6209-21ssh00:35:481
216099992,6cyclictest12895-21kworker/1:020:00:561
216089995,2cyclictest1810-21needreboot19:35:520
216089994,3cyclictest26789-21cpuspeed_turbos19:20:450
216089993,4cyclictest6244-21grep22:05:480
216089993,4cyclictest5166-21cut21:00:470
216089993,4cyclictest29037-21aten2_r8power_v20:40:460
216089992,3cyclictest0-21swapper/020:25:520
216099983,4cyclictest27263-21tune2fs20:35:481
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional