You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-07-29 - 17:13

Intel(R) Core(TM) i3-2310E CPU @ 2.10GHz, Linux 5.10.47-rt46 (Profile)

Latency plot of system in rack #8, slot #2
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by a total of 2 SMIs that occured during the measurement.
Command line: cyclictest -l100000000 -m -Sp99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Compare latency of primary with shadow system
Characteristics of the highest latencies:
System rack8slot2.osadl.org (updated Thu Jul 29, 2021 12:43:31)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
26993991470,146cyclictest0-21swapper/007:15:230
26568210695,6sleep20-21swapper/207:08:122
26414210444,20sleep10-21swapper/107:06:151
86882980,0sleep00-21swapper/010:03:180
179632980,0sleep10-21swapper/110:11:171
165002970,0sleep10-21swapper/112:38:171
320392960,0sleep20-21swapper/207:20:102
320392960,0sleep20-21swapper/207:20:102
2479928453,20sleep30-21swapper/307:05:173
26994998273,8cyclictest29826-21apt-get07:15:231
2643827846,20sleep00-21swapper/007:06:340
26995997270,1cyclictest0-21swapper/207:10:122
87552400,1sleep20-21swapper/208:55:092
26996994038,1cyclictest0-21swapper/307:10:123
153362380,1sleep10-21swapper/111:37:561
26993993735,1cyclictest0-21swapper/007:10:120
61882340,1sleep00-21swapper/010:30:320
178442330,1sleep00-21swapper/008:00:280
2699599291,27cyclictest0-21swapper/211:40:222
133402290,1sleep00-21swapper/011:06:000
26432280,0sleep00-21swapper/011:25:520
2699599270,26cyclictest0-21swapper/208:35:242
61242260,1sleep26128-21cpuspeed_turbos11:00:252
246442260,1sleep10-21swapper/110:18:031
26995992510,12cyclictest15292-21diskmemload09:36:012
2699699231,21cyclictest0-21swapper/308:05:233
2699599221,1cyclictest0-21swapper/210:35:332
2699499210,1cyclictest0-21swapper/110:45:291
245132210,5sleep228-21ksoftirqd/211:45:352
2699699204,10cyclictest4487-21ssh12:26:523
26996992016,3cyclictest28561-21kworker/u8:009:22:313
2699599191,16cyclictest0-21swapper/212:20:242
2699399194,8cyclictest6542-21awk12:00:090
26996991815,2cyclictest43750irq/33-enp3s0-r10:45:293
2699499180,5cyclictest1088-21basename10:55:251
2699699173,10cyclictest238-21systemd-journal08:50:293
2699499174,4cyclictest0-21swapper/109:49:261
2699499174,3cyclictest31413-21cron09:25:101
2699499171,5cyclictest28951-21idleruntime-cro07:15:091
2699599165,6cyclictest603-21runrttasks08:40:382
2699499167,1cyclictest0-21swapper/111:06:161
2699499166,2cyclictest603-21runrttasks09:39:481
2699499165,2cyclictest13131-21munin-plugin-st09:05:091
2699499164,4cyclictest0-21swapper/110:07:471
2699499163,5cyclictest0-21swapper/107:24:151
2699499163,5cyclictest0-21swapper/107:24:151
2699499163,4cyclictest12958-21df_inode12:05:321
2699499162,5cyclictest31555-21cat08:30:311
2699499162,5cyclictest0-21swapper/109:33:371
2699499161,5cyclictest7606-21grep12:30:261
2699499161,5cyclictest7196-21sort11:30:261
2699499161,5cyclictest28475-21ls12:20:101
2699499161,4cyclictest21268-21turbostat.cron10:45:101
2699499160,6cyclictest27541-21ssh09:50:361
26993991614,1cyclictest0-21swapper/010:20:210
2699699152,8cyclictest0-21swapper/307:40:343
2699699151,6cyclictest27555-21ssh12:18:573
2699499156,2cyclictest3172-21sh12:25:411
2699499154,5cyclictest0-21swapper/110:40:031
2699499154,2cyclictest8430-21rm10:32:391
2699499154,2cyclictest0-21swapper/111:01:141
2699499153,5cyclictest590-21snmpd08:20:021
2699499153,4cyclictest590-21snmpd12:20:341
2699499153,4cyclictest21848-21cron11:15:091
2699499153,4cyclictest15292-21diskmemload09:25:331
2699499153,4cyclictest0-21swapper/109:15:191
2699499152,4cyclictest713-21rm11:25:191
2699499152,4cyclictest2791-21mailstats11:55:341
2699499151,6cyclictest23504-21latency_hist08:15:091
2699499151,6cyclictest15292-21diskmemload11:20:411
2699499151,5cyclictest27511-21grep08:20:351
2699499151,5cyclictest18378-21fschecks_count12:10:261
2699499151,5cyclictest15292-21diskmemload10:00:291
2699499151,5cyclictest12874-21grep07:50:221
2699499151,4cyclictest22844-21grep11:15:251
2699499151,4cyclictest10567-21unixbench_singl08:55:551
2699499151,4cyclictest10567-21unixbench_singl08:55:551
2699499150,8cyclictest24167-21proc_pri11:45:321
2699499150,5cyclictest999-21ssh09:55:361
2699499150,5cyclictest8687-21latency_hist08:55:091
2699499150,5cyclictest6081-21grep08:45:331
2699499150,5cyclictest32505-21find10:55:101
2699499150,5cyclictest19210-21awk08:05:091
2699499150,5cyclictest15746-21aten2_r8power_e09:40:241
2699499150,5cyclictest12564-21idleruntime-cro07:50:101
2699499150,4cyclictest21452-21latency_hist08:10:091
2699399155,5cyclictest0-21swapper/011:45:280
2699699148,4cyclictest18745-21ssh10:12:213
2699699147,5cyclictest0-21swapper/310:01:353
2699699142,7cyclictest15292-21diskmemload09:15:493
2699599144,5cyclictest15292-21diskmemload11:54:212
2699499144,2cyclictest419-21grep08:35:261
2699499144,2cyclictest15640-21sh09:10:211
2699499143,4cyclictest590-21snmpd10:25:151
2699499143,4cyclictest13750-21expr09:05:281
2699499143,4cyclictest0-21swapper/107:41:191
2699499141,5cyclictest3348-21unixbench_singl07:25:371
2699499141,5cyclictest28043-21taskset10:20:311
2699499141,4cyclictest16627-21unixbench_singl07:55:401
2699499140,5cyclictest7164-21grep12:00:261
2699499140,5cyclictest7096-21cat07:35:321
2699499140,5cyclictest4316-21tr07:30:261
2699499140,5cyclictest28457-21/usr/sbin/munin11:50:281
2699499140,4cyclictest30160-21basename08:30:091
2699399144,5cyclictest0-21swapper/009:14:130
2699399144,5cyclictest0-21swapper/009:05:330
2699399144,5cyclictest0-21swapper/008:56:150
2699399144,5cyclictest0-21swapper/008:56:150
145532140,0chrt0-21swapper/311:36:543
61532130,1chrt6152-21sed07:35:103
2699699139,3cyclictest23129-21kworker/u8:009:10:243
2699699138,3cyclictest12336-21sh10:35:353
2699699136,6cyclictest26460-21kworker/3:109:00:283
2699699136,6cyclictest26460-21kworker/3:108:10:323
2699699136,5cyclictest9691-21ssh12:31:353
2699699135,4cyclictest17544-21ssh09:40:533
2699699135,3cyclictest0-21swapper/310:30:423
2699699134,4cyclictest0-21swapper/311:30:553
2699699132,6cyclictest15292-21diskmemload09:53:233
2699699131,9cyclictest75-21kswapd012:10:223
2699699131,5cyclictest32307-21cron10:55:103
2699599139,2cyclictest27085-21sed09:20:262
2699599136,5cyclictest14514-21ssh12:35:402
2699599135,6cyclictest0-21swapper/210:40:212
2699599135,3cyclictest0-21swapper/210:02:052
2699599135,3cyclictest0-21swapper/207:38:132
2699599134,4cyclictest0-21swapper/210:23:252
2699599134,4cyclictest0-21swapper/209:32:512
2699599132,6cyclictest19648-21ssh09:13:522
2699599132,6cyclictest0-21swapper/207:25:322
2699599131,9cyclictest11497-21basename11:05:252
2699599131,7cyclictest8089-21ssh10:32:122
2699499130,5cyclictest18748-21fgrep11:40:331
2699399133,5cyclictest0-21swapper/012:22:520
2699399130,4cyclictest23797-21sh10:46:030
2699699125,6cyclictest26460-21kworker/3:107:15:123
2699699124,6cyclictest26460-21kworker/3:107:55:383
2699699124,6cyclictest13046-21kworker/3:210:07:243
2699699124,6cyclictest0-21swapper/311:55:123
2699699124,3cyclictest0-21swapper/312:39:433
2699699124,3cyclictest0-21swapper/312:24:343
2699699123,4cyclictest31022-21ssh11:22:383
2699699122,5cyclictest16750-21/usr/sbin/munin11:10:313
2699699122,5cyclictest0-21swapper/310:20:263
2699699121,9cyclictest4433-21cron09:30:103
2699699121,9cyclictest0-21swapper/308:32:313
2699699121,6cyclictest238-21systemd-journal11:48:343
2699599124,7cyclictest11449-1kworker/u9:212:05:362
2699599124,6cyclictest0-21swapper/211:15:132
2699599124,3cyclictest0-21swapper/210:45:272
2699599124,3cyclictest0-21swapper/209:47:382
2699599124,3cyclictest0-21swapper/209:08:002
2699599124,3cyclictest0-21swapper/208:28:232
2699599123,8cyclictest0-21swapper/210:16:102
2699599123,4cyclictest4196-21ssh11:57:062
2699599123,4cyclictest0-21swapper/212:30:502
2699599123,4cyclictest0-21swapper/210:06:132
2699599123,4cyclictest0-21swapper/208:45:262
2699599122,5cyclictest238-21systemd-journal12:02:222
2699599121,6cyclictest29772-21ssh11:20:562
2699599121,5cyclictest17955-21ssh09:41:262
2699599121,2cyclictest0-21swapper/211:26:552
2699599120,5cyclictest22635-21vmstat09:15:342
2699399126,4cyclictest911-21uniq10:55:240
2699399126,4cyclictest6523-21cat11:30:210
2699399126,4cyclictest590-21snmpd07:38:110
2699399126,4cyclictest5898-21sensors08:45:320
2699399126,4cyclictest2866-21diskstats08:40:260
2699399126,4cyclictest21724-21uniq10:15:250
2699399126,4cyclictest11483-21ntpq09:35:370
2699399122,5cyclictest0-21swapper/007:27:450
2699399120,11cyclictest0-21swapper/011:40:220
246182120,2sleep334-21ksoftirqd/310:18:013
2699699117,3cyclictest12818-21aten2_r8power_c07:50:223
2699699117,2cyclictest10557-21grep09:35:283
2699699116,3cyclictest25069-21sh11:17:093
2699699116,3cyclictest1846-21grep11:25:293
2699699115,5cyclictest26460-21kworker/3:108:00:303
2699699115,5cyclictest13046-21kworker/3:211:01:483
2699699115,5cyclictest13046-21kworker/3:210:28:243
2699699115,5cyclictest13046-21kworker/3:209:31:423
2699699115,4cyclictest14419-21cut09:05:363
2699699115,4cyclictest12604-21cut12:05:303
2699699114,6cyclictest26460-21kworker/3:108:15:103
2699699114,6cyclictest0-21swapper/312:01:123
2699699114,6cyclictest0-21swapper/307:38:013
2699699114,5cyclictest17741-21threads10:40:443
2699699114,5cyclictest0-21swapper/308:43:243
2699699111,8cyclictest12223-21if_err_enp3s011:05:293
2699699111,5cyclictest0-21swapper/307:45:293
2699699110,6cyclictest521-21grep09:55:323
2699699110,6cyclictest30624-21ssh11:51:243
2699599117,2cyclictest24137-21tr08:15:242
2699599115,5cyclictest0-21swapper/207:30:152
2699599115,4cyclictest31612-21cron09:55:092
2699599115,4cyclictest28351-21missed_timers10:50:372
2699599113,3cyclictest0-21swapper/209:56:412
2699599113,3cyclictest0-21swapper/208:34:532
2699599113,3cyclictest0-21swapper/208:21:292
2699599113,3cyclictest0-21swapper/207:53:222
2699599112,8cyclictest0-21swapper/212:15:562
2699599112,7cyclictest1303-21run-parts10:25:492
2699599112,4cyclictest0-21swapper/212:12:032
2699599112,4cyclictest0-21swapper/211:35:592
2699599111,8cyclictest1042-21sed10:55:242
2699599111,6cyclictest1151-21users09:25:462
2699599111,5cyclictest21869-21tr08:10:282
2699599110,6cyclictest16227-21ssh10:10:282
2699399118,2cyclictest16726-21diskstats10:40:330
2699399116,4cyclictest770-21ntpq07:20:310
2699399116,4cyclictest770-21ntpq07:20:310
2699399116,4cyclictest13278-21if_enp4s012:05:340
2699399116,4cyclictest0-21swapper/008:24:440
2699399116,3cyclictest6859-21basename08:50:200
2699399116,3cyclictest32684-21basename08:35:240
2699399115,4cyclictest8381-21aten2_r8power_c07:40:290
2699399115,4cyclictest590-21snmpd08:10:420
2699399115,4cyclictest4460-21df07:30:280
2699399115,4cyclictest27386-21cat09:20:280
2699399115,4cyclictest24095-21cpuspeed_turbos08:15:240
2699399115,4cyclictest18932-21missed_timers12:10:300
2699399115,4cyclictest1763-21sh12:25:160
2699399115,4cyclictest15148-21sh11:37:420
2699399115,4cyclictest10828-21cpu07:45:250
2699399115,4cyclictest0-21swapper/010:08:560
2699399114,5cyclictest14322-21unixbench-2d12:35:360
2699399114,5cyclictest0-21swapper/011:00:490
2699399114,5cyclictest0-21swapper/009:34:160
2699399114,1cyclictest12-21ksoftirqd/009:45:260
2699399113,2cyclictest141rcu_preempt11:52:450
2699399112,2cyclictest141rcu_preempt12:34:530
2699399111,8cyclictest75-21kswapd010:35:220
2699399111,8cyclictest29862-21sh10:51:530
2699399111,5cyclictest0-21swapper/008:26:500
2699399110,9cyclictest15828-21ssh10:10:250
2699399110,6cyclictest13993-21/usr/sbin/munin07:50:310
55292100,0sleep30-21swapper/308:45:283
2699699108,1cyclictest0-21swapper/308:56:113
2699699108,1cyclictest0-21swapper/308:56:113
2699699105,4cyclictest26460-21kworker/3:108:37:103
2699699105,4cyclictest13046-21kworker/3:209:45:343
2699699105,3cyclictest0-21swapper/308:20:203
2699699105,3cyclictest0-21swapper/307:25:283
2699699104,5cyclictest0-21swapper/308:26:193
2699699103,5cyclictest17245-21ssh11:40:113
2699699100,4cyclictest404-21aten2_r8power_e10:55:213
2699599105,3cyclictest19530-21aten2_r8power_p08:05:242
2699599105,3cyclictest18298-21needreboot08:00:322
2699599105,3cyclictest11658-21/usr/sbin/munin09:00:232
2699599105,3cyclictest10501-21unixbench_multi08:55:552
2699599105,3cyclictest10501-21unixbench_multi08:55:552
2699599105,3cyclictest0-21swapper/207:40:372
2699599104,4cyclictest6781-21grep11:30:232
2699599104,4cyclictest16617-21grep07:55:402
2699599104,4cyclictest0-21swapper/212:25:302
2699599103,6cyclictest0-21swapper/207:15:272
2699599101,7cyclictest16775-21apt-get11:10:202
2699499104,4cyclictest2870-21entropy08:40:271
2699399106,3cyclictest32486-21aten2_r8power_v10:25:400
2699399106,3cyclictest28089-21sensors_temp09:50:420
2699399106,3cyclictest21481-21uniq09:15:240
2699399106,3cyclictest16331-21processes07:55:380
2699399106,3cyclictest12-21ksoftirqd/011:15:280
2699399105,4cyclictest23768-21expr12:15:240
2699399105,3cyclictest28650-21idleruntime11:20:350
2699399105,3cyclictest2385-21sh09:57:080
2699399105,3cyclictest18844-21sh09:42:400
2699399105,3cyclictest13274-21latency_hist09:05:100
2699399104,4cyclictest4612-21cut09:30:100
2699399104,4cyclictest16767-21cat11:10:200
2699399103,5cyclictest0-21swapper/012:01:510
269969993,4cyclictest590-21snmpd07:20:183
269969993,4cyclictest590-21snmpd07:20:183
269959991,7cyclictest0-21swapper/207:45:262
269939994,4cyclictest31061-21uniq08:30:270
269939994,4cyclictest19520-21cut08:05:240
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional