You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2022-01-22 - 16:44

Intel(R) Core(TM) i3-2310E CPU @ 2.10GHz, Linux 5.10.47-rt46 (Profile)

Latency plot of system in rack #8, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by a total of 2 SMIs that occured during the measurement.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of all highest latencies:
System rack8slot2.osadl.org (updated Sat Jan 22, 2022 12:43:35)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
490499145143,1cyclictest0-21swapper/207:20:162
490499139136,2cyclictest0-21swapper/207:12:482
721221130,4sleep1221ksoftirqd/110:32:241
2344721060,1sleep3331rcuc/309:08:043
1551621060,3sleep012-21ksoftirqd/009:34:390
1551621060,3sleep012-21ksoftirqd/009:34:390
1191321040,0sleep30-21swapper/311:09:103
4862210140,45sleep10-21swapper/107:12:471
3092221000,0sleep10-21swapper/110:23:061
443229248,32sleep00-21swapper/007:10:130
451629149,31sleep20-21swapper/207:11:192
393829046,32sleep30-21swapper/307:09:153
270929048,31sleep10-21swapper/107:07:531
4905998987,1cyclictest0-21swapper/307:20:163
4905998280,1cyclictest0-21swapper/307:12:483
4902997472,1cyclictest0-21swapper/007:20:160
4902996664,1cyclictest0-21swapper/007:12:480
4903993628,7cyclictest0-21swapper/107:20:161
173872350,1chrt0-21swapper/007:38:100
490299310,29cyclictest25017-21diskmemload11:08:020
47182310,0sleep30-21swapper/309:57:393
157722310,0sleep20-21swapper/211:47:182
97292290,0sleep00-21swapper/011:40:420
236932290,1sleep20-21swapper/207:53:072
196322260,0sleep00-21swapper/009:38:230
90922240,1sleep00-21swapper/011:07:270
490299221,8cyclictest23734-21ntp_states10:48:110
22382220,0sleep30-21swapper/312:38:143
490299190,9cyclictest27810-21ssh09:14:070
490299190,9cyclictest14412-21ssh11:45:090
490599183,9cyclictest3618-21ssh11:33:423
490399183,5cyclictest141rcu_preempt11:14:531
490299184,9cyclictest0-21swapper/009:03:520
490299180,10cyclictest7346-21sh10:32:370
216852180,1sleep20-21swapper/212:25:462
490399175,2cyclictest141rcu_preempt10:41:331
490399173,2cyclictest141rcu_preempt09:43:171
490299170,10cyclictest6158-21idleruntime-cro11:37:450
490299170,10cyclictest18390-21ssh10:43:060
490399167,8cyclictest0-21swapper/109:06:111
490399166,5cyclictest4454-21tail12:08:071
490399164,2cyclictest141rcu_preempt12:39:011
490399163,4cyclictest141rcu_preempt11:20:591
490399161,1cyclictest141rcu_preempt12:25:331
490299166,8cyclictest19622-21sh12:23:090
490299164,10cyclictest5064-21awk08:23:150
490299164,10cyclictest25721-21fschecks_count10:18:040
4902991614,1cyclictest0-21swapper/007:53:030
490299160,9cyclictest9627-21ssh10:33:260
490599154,6cyclictest4470-21users10:28:163
490599150,13cyclictest12972-21ssh11:43:143
490399154,5cyclictest141rcu_preempt12:36:591
490399154,5cyclictest141rcu_preempt12:36:591
490399154,2cyclictest141rcu_preempt11:08:471
490399154,2cyclictest141rcu_preempt11:06:171
490399154,2cyclictest141rcu_preempt09:52:491
490399154,2cyclictest141rcu_preempt09:13:591
490399153,4cyclictest141rcu_preempt11:53:511
490399151,5cyclictest141rcu_preempt10:07:531
490299158,6cyclictest11863-21unixbench_multi10:03:180
490299156,7cyclictest4790-21sh12:08:110
490299156,7cyclictest25017-21diskmemload10:42:290
490299155,8cyclictest2760-21swap08:18:180
490299155,8cyclictest0-21swapper/008:56:460
490299153,10cyclictest21585-21sendmail_mailqu10:13:140
490299151,8cyclictest31460-21munin-run08:12:450
490299151,12cyclictest16160-21latency_hist07:37:440
490299151,12cyclictest15058-21anacron10:07:520
490299150,8cyclictest9862-21munin-run08:37:450
490299150,13cyclictest31716-21ntp_kernel_pll_09:18:110
490299150,13cyclictest25634-21ssh12:29:070
490299150,13cyclictest24821-21/usr/sbin/munin11:23:060
490299150,13cyclictest23117-21/usr/sbin/munin09:08:010
490599144,5cyclictest0-21swapper/309:00:373
490599141,12cyclictest0-21swapper/309:58:023
490499145,4cyclictest13930-21ntpq10:38:132
490499142,7cyclictest0-21swapper/209:03:042
490399145,4cyclictest0-21swapper/110:48:051
490399144,5cyclictest0-21swapper/111:32:541
490399144,5cyclictest0-21swapper/110:58:501
490399144,1cyclictest141rcu_preempt09:37:571
490399143,2cyclictest141rcu_preempt10:53:251
490399140,3cyclictest141rcu_preempt10:04:271
490399140,2cyclictest141rcu_preempt09:58:071
490299145,7cyclictest7709-21perf08:32:450
490299144,8cyclictest29013-21ssh11:27:490
490299144,8cyclictest25714-21ssh09:45:060
490299144,8cyclictest13124-21interrupts08:43:090
490299143,9cyclictest590-21snmpd07:32:190
490299141,11cyclictest30822-21ssh12:34:290
490299141,11cyclictest30822-21ssh12:34:290
490299141,11cyclictest28162-21sh11:58:340
490299141,11cyclictest22792-21munin-plugin-st07:52:440
490299140,12cyclictest9818-21ntpq12:13:110
490299140,12cyclictest6445-21/usr/sbin/munin09:58:130
490299140,12cyclictest3474-21ssh12:39:500
490299140,12cyclictest3416-21latency_hist12:07:450
490299140,12cyclictest30166-21ssh10:22:590
490299140,12cyclictest27876-21aten2_r8power_e09:48:020
490299140,12cyclictest2575-21ssh10:59:580
490299140,12cyclictest22305-21sed11:53:070
490299140,12cyclictest20911-21ssh11:52:430
490299140,12cyclictest15124-21ssh11:13:090
490299140,12cyclictest13896-21cstates12:18:040
490299140,12cyclictest1316-21/usr/sbin/munin09:53:100
490299140,12cyclictest11844-21ssh09:31:410
490299140,12cyclictest10612-21fschecks_time08:38:050
212052140,0chrt0-21swapper/011:18:250
490599136,5cyclictest25342-21sort11:23:103
490599136,5cyclictest18979-21seq11:17:413
490599135,6cyclictest0-21swapper/307:36:043
490599134,6cyclictest0-21swapper/309:25:263
490599131,8cyclictest0-21swapper/312:33:313
490599131,8cyclictest0-21swapper/312:33:313
490599131,7cyclictest25017-21diskmemload12:03:313
490599130,1cyclictest0-21swapper/307:43:163
490499136,5cyclictest590-21snmpd10:08:472
490499136,5cyclictest16021-21sort11:13:182
490499134,3cyclictest19833-21expr11:18:032
490499131,11cyclictest0-21swapper/210:28:082
490399137,5cyclictest0-21swapper/110:20:531
490399137,5cyclictest0-21swapper/109:37:271
490399137,5cyclictest0-21swapper/109:37:271
490399137,5cyclictest0-21swapper/107:26:311
490399137,4cyclictest24782-21cron10:17:441
490399135,5cyclictest0-21swapper/109:23:271
490399132,2cyclictest141rcu_preempt12:29:031
490399132,2cyclictest141rcu_preempt12:22:071
490399132,2cyclictest141rcu_preempt11:46:111
490399132,2cyclictest141rcu_preempt09:09:471
490399132,2cyclictest141rcu_preempt08:28:031
490399132,2cyclictest141rcu_preempt07:51:491
490399132,2cyclictest141rcu_preempt07:33:031
490399131,2cyclictest141rcu_preempt09:29:431
490399131,1cyclictest141rcu_preempt11:30:271
490399131,1cyclictest141rcu_preempt11:26:431
490299130,11cyclictest32396-21if_eno108:13:090
490299130,11cyclictest3189-21grep09:23:020
490299130,11cyclictest27424-21apt-get10:53:000
490299130,11cyclictest27402-21/usr/sbin/munin08:03:020
490299130,11cyclictest25256-21aten2_r8power_e07:58:020
490299130,11cyclictest19412-21/usr/sbin/munin08:58:080
490299130,11cyclictest19275-21perl07:43:070
490299130,11cyclictest14616-21/usr/sbin/munin08:48:030
490299130,11cyclictest10415-21cut07:23:060
490599125,6cyclictest0-21swapper/310:36:433
490599125,5cyclictest29826-21tail11:28:053
490599124,4cyclictest0-21swapper/308:27:273
490599123,3cyclictest0-21swapper/310:12:363
490599120,5cyclictest23489-21cut09:43:073
490599120,5cyclictest19680-21grep11:18:033
490599120,10cyclictest30819-21ssh09:50:203
490499128,2cyclictest21445-21grep07:48:062
490499127,3cyclictest30105-21gltestperf08:08:082
490499127,3cyclictest10306-21grep12:13:162
490499126,4cyclictest538-21dbus-daemon11:08:142
490499126,4cyclictest19748-21tail07:43:122
490499125,5cyclictest11847-21basename07:27:442
490499124,6cyclictest28-21ksoftirqd/209:13:112
490499124,6cyclictest0-21swapper/210:37:082
490499124,6cyclictest0-21swapper/209:30:012
490499123,4cyclictest24658-21ssh09:43:292
490499123,4cyclictest21330-21ssh10:46:292
490499121,9cyclictest28677-21cpuspeed_turbos12:33:022
490499121,9cyclictest28677-21cpuspeed_turbos12:33:022
490499121,6cyclictest0-21swapper/208:33:152
490499120,5cyclictest5181-21/usr/sbin/munin09:58:152
490399127,4cyclictest0-21swapper/109:48:491
490399127,4cyclictest0-21swapper/108:22:511
490399127,4cyclictest0-21swapper/107:55:491
490399127,3cyclictest27115-21sh08:02:441
490399126,4cyclictest17091-21head11:48:041
490399124,7cyclictest0-21swapper/108:14:191
490399124,5cyclictest0-21swapper/110:36:191
490399122,5cyclictest0-21swapper/107:27:511
490399121,6cyclictest0-21swapper/108:22:301
490399121,4cyclictest141rcu_preempt08:06:131
490399121,2cyclictest141rcu_preempt12:06:051
490599116,3cyclictest9838-21sh12:13:113
490599116,3cyclictest43750irq/33-enp3s0-r10:03:093
490599116,3cyclictest31776-21grep10:23:153
490599116,3cyclictest25462-21sh09:12:523
490599115,4cyclictest26340-21sh10:51:403
490599115,4cyclictest24313-21if_err_enp5s012:28:083
490599115,4cyclictest24229-21wc07:53:143
490599115,4cyclictest21811-21unixbench_multi10:13:163
490599115,4cyclictest18498-21cron08:57:443
490599115,4cyclictest17730-21wc07:38:153
490599115,4cyclictest12307-21tr07:28:033
490599114,5cyclictest21171-21grep07:48:043
490599114,5cyclictest0-21swapper/312:01:343
490599114,3cyclictest13944-21ntp_states10:38:133
490599114,3cyclictest0-21swapper/310:53:063
490599113,6cyclictest0-21swapper/308:30:573
490599111,8cyclictest22167-21perf10:47:443
490599111,8cyclictest18489-21grep09:38:053
490599111,8cyclictest13912-21latency09:33:103
490599111,8cyclictest13912-21latency09:33:103
490599111,5cyclictest0-21swapper/308:48:073
490499116,3cyclictest603-21runrttasks09:54:572
490499115,4cyclictest27126-21systemctl08:02:442
490499115,4cyclictest23127-21cut10:48:052
490499115,4cyclictest14670-21grep08:48:032
490499114,6cyclictest0-21swapper/212:21:572
490499114,6cyclictest0-21swapper/210:26:152
490499114,4cyclictest19675-21apt-get10:12:452
490499113,6cyclictest0-21swapper/212:06:122
490499113,6cyclictest0-21swapper/211:57:572
490499113,6cyclictest0-21swapper/211:53:382
490499113,6cyclictest0-21swapper/211:03:122
490499113,3cyclictest0-21swapper/207:31:522
490499113,2cyclictest0-21swapper/211:33:282
490499113,2cyclictest0-21swapper/209:50:192
490499112,3cyclictest25724-21cat11:23:152
490499111,8cyclictest0-21swapper/212:38:062
490499111,5cyclictest22903-21apt-get12:27:472
490499111,5cyclictest22481-21grep09:42:452
490499111,3cyclictest17482-21ssh09:37:452
490499111,3cyclictest17482-21ssh09:37:452
490499111,2cyclictest0-21swapper/212:08:282
490499110,9cyclictest14206-21perf08:47:442
490499110,6cyclictest0-21swapper/211:47:452
490499110,4cyclictest29449-21cpuspeed_turbos11:28:032
490399116,3cyclictest30499-21sh09:17:581
490399114,5cyclictest18058-21df10:43:031
490399114,5cyclictest0-21swapper/112:01:311
490399114,5cyclictest0-21swapper/107:40:581
490399114,4cyclictest0-21swapper/107:43:431
490399112,4cyclictest0-21swapper/111:40:311
490399111,5cyclictest0-21swapper/109:01:571
490399110,2cyclictest141rcu_preempt08:54:051
490399110,2cyclictest141rcu_preempt08:44:391
325062110,0chrt0-21swapper/309:18:423
490599106,3cyclictest43750irq/33-enp3s0-r12:23:023
490599106,2cyclictest25017-21diskmemload12:07:513
490599106,2cyclictest16648-21cpuspeed_turbos11:48:023
490599105,3cyclictest7510-21latency11:38:093
490599105,3cyclictest30577-21cut08:08:143
490599105,3cyclictest25761-21tr07:58:063
490599105,3cyclictest0-21swapper/312:18:013
490599104,5cyclictest590-21snmpd08:22:333
490599104,4cyclictest9568-21awk09:28:153
490599104,4cyclictest7993-21cut08:33:043
490599104,4cyclictest10221-21cut07:23:043
490599103,5cyclictest590-21snmpd08:37:483
490599102,4cyclictest0-21swapper/311:53:253
490599101,7cyclictest0-21swapper/308:13:063
490599101,4cyclictest27601-21tr08:03:033
490599100,8cyclictest5418-21expr11:03:053
490599100,8cyclictest32735-21ssh10:58:043
490599100,1cyclictest0-21swapper/310:18:123
490499106,2cyclictest603-21runrttasks10:02:542
490499105,4cyclictest8348-21ssh11:38:282
490499105,4cyclictest0-21swapper/208:55:432
490499105,3cyclictest18020-21grep07:38:172
490499104,4cyclictest590-21snmpd08:17:472
490499104,4cyclictest15588-21wc07:33:162
490499103,5cyclictest590-21snmpd10:54:442
490499103,5cyclictest590-21snmpd08:02:562
490499102,6cyclictest0-21swapper/208:13:052
490499101,7cyclictest0-21swapper/210:18:022
490499101,7cyclictest0-21swapper/209:23:062
490499101,7cyclictest0-21swapper/208:58:022
490499101,7cyclictest0-21swapper/208:23:052
490499101,2cyclictest0-21swapper/209:08:162
490499100,5cyclictest12000-21cron08:42:442
490499100,5cyclictest0-21swapper/210:58:072
490499100,4cyclictest720-21ssh09:19:442
490399105,3cyclictest8504-21gltestperf08:33:071
490399104,5cyclictest0-21swapper/108:08:541
490399103,5cyclictest0-21swapper/112:13:011
490399101,1cyclictest0-21swapper/108:38:421
490399100,6cyclictest15355-21sort08:48:091
49059993,4cyclictest590-21snmpd08:42:513
49059991,3cyclictest21197-21grep09:03:013
49049991,2cyclictest0-21swapper/208:28:042
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional