You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-09-17 - 21:59

Intel(R) Atom(TM) CPU E3845 @ 1.91GHz, Linux 4.19.37-rt19 (Profile)

Latency plot of system in rack #d, slot #1
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Up99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the alll highest latencies:
System rackdslot1.osadl.org (updated Fri Sep 17, 2021 12:46:08)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
2048922570,8sleep00-21swapper/007:05:160
1146911960,29ptp4l21823-21which07:05:323
1146911910,14ptp4l0-21swapper/207:08:272
2575121550,4sleep02294299cyclictest09:10:250
2575121550,4sleep02294299cyclictest09:10:250
1146911510,16ptp4l0-21swapper/107:05:041
229449910394,5cyclictest33-21ksoftirqd/210:50:132
1146911010,20ptp4l29708-21missed_timers08:35:290
22944999181,6cyclictest33-21ksoftirqd/209:55:162
22944999079,7cyclictest33-21ksoftirqd/208:50:162
22943998881,4cyclictest25-21ksoftirqd/108:50:161
306992870,5sleep022941-21cyclictest07:19:590
22944998772,9cyclictest33-21ksoftirqd/212:25:152
22944998673,7cyclictest33-21ksoftirqd/208:00:142
22944998472,7cyclictest33-21ksoftirqd/210:35:152
22944998269,6cyclictest33-21ksoftirqd/212:30:152
22943997971,5cyclictest25-21ksoftirqd/110:20:151
22944997866,6cyclictest33-21ksoftirqd/211:50:152
22943997871,3cyclictest25-21ksoftirqd/109:55:151
22944997761,8cyclictest33-21ksoftirqd/211:40:152
22944997664,6cyclictest33-21ksoftirqd/212:15:132
22944997663,6cyclictest33-21ksoftirqd/208:10:172
22945997557,10cyclictest41-21ksoftirqd/311:55:153
22943997466,4cyclictest25-21ksoftirqd/112:00:131
22944997361,7cyclictest33-21ksoftirqd/209:10:162
22944997361,7cyclictest33-21ksoftirqd/209:10:152
22944997360,7cyclictest33-21ksoftirqd/207:35:152
22944997360,6cyclictest33-21ksoftirqd/207:15:142
22944997260,5cyclictest33-21ksoftirqd/211:45:152
22943997163,4cyclictest25-21ksoftirqd/110:55:151
114691710,2ptp4l0-21swapper/111:40:011
22943997062,5cyclictest25-21ksoftirqd/109:15:141
22943996960,5cyclictest25-21ksoftirqd/107:50:161
169712690,9sleep22294499cyclictest07:40:262
22944996857,6cyclictest33-21ksoftirqd/211:20:152
22943996860,5cyclictest25-21ksoftirqd/109:00:141
22943996859,5cyclictest25-21ksoftirqd/111:25:151
22943996855,8cyclictest25-21ksoftirqd/107:10:151
313172670,2sleep00-21swapper/010:40:020
22944996756,6cyclictest33-21ksoftirqd/209:20:142
22944996756,4cyclictest33-21ksoftirqd/209:35:162
22944996755,6cyclictest33-21ksoftirqd/211:30:152
22944996755,6cyclictest33-21ksoftirqd/211:30:142
22944996652,7cyclictest33-21ksoftirqd/211:10:192
114691660,4ptp4l25233-21df_inode10:30:180
234692650,9sleep22294499cyclictest10:30:052
114691650,4ptp4l27945-21hddtemp_smartct11:55:060
252112640,1sleep00-21swapper/007:10:200
114691640,3ptp4l30396-21/usr/sbin/munin12:35:330
22944996352,7cyclictest33-21ksoftirqd/208:45:172
22944996349,8cyclictest33-21ksoftirqd/209:05:192
22944996349,8cyclictest33-21ksoftirqd/209:05:182
22944996349,8cyclictest33-21ksoftirqd/208:35:172
114691620,2ptp4l0-21swapper/108:05:321
114691620,2ptp4l0-21swapper/012:12:000
114691620,2ptp4l0-21swapper/007:39:080
22944996151,5cyclictest33-21ksoftirqd/209:25:162
22944996147,7cyclictest33-21ksoftirqd/209:00:142
173782610,2sleep00-21swapper/011:00:340
25422600,2sleep10-21swapper/111:24:591
22943996048,6cyclictest25-21ksoftirqd/108:25:161
22943996047,7cyclictest25-21ksoftirqd/112:10:141
22944995947,7cyclictest33-21ksoftirqd/212:10:242
114691590,2ptp4l0-21swapper/108:15:181
114691590,2ptp4l0-21swapper/010:02:220
22943995848,5cyclictest25-21ksoftirqd/111:50:231
114691580,2ptp4l0-21swapper/210:55:162
114691580,2ptp4l0-21swapper/112:30:261
114691580,2ptp4l0-21swapper/110:50:291
114691580,2ptp4l0-21swapper/008:00:440
114691580,14ptp4l0-21swapper/111:35:081
114691580,14ptp4l0-21swapper/111:35:081
114691580,10ptp4l0-21swapper/108:55:181
22944995750,4cyclictest33-21ksoftirqd/207:50:162
114791570,12getstats0-21swapper/111:55:241
114691570,8ptp4l0-21swapper/212:37:242
114691570,2ptp4l0-21swapper/108:20:061
114691570,2ptp4l0-21swapper/011:33:530
114691570,2ptp4l0-21swapper/011:33:530
22944995647,5cyclictest33-21ksoftirqd/207:25:152
22944995647,4cyclictest33-21ksoftirqd/207:20:152
114691560,5ptp4l121rcu_preempt11:28:523
114691560,2ptp4l0-21swapper/109:06:051
114691560,2ptp4l0-21swapper/109:06:041
114691560,2ptp4l0-21swapper/012:06:020
114691560,2ptp4l0-21swapper/010:53:170
114691560,2ptp4l0-21swapper/007:51:060
114691560,1ptp4l0-21swapper/012:33:430
114691550,2ptp4l0-21swapper/307:58:223
114691550,2ptp4l0-21swapper/208:40:162
114691550,2ptp4l0-21swapper/108:10:351
114691550,2ptp4l0-21swapper/010:45:250
114691550,2ptp4l0-21swapper/009:38:150
114691550,1ptp4l0-21swapper/011:36:570
114691550,1ptp4l0-21swapper/011:36:560
114691550,1ptp4l0-21swapper/011:20:030
114691550,1ptp4l0-21swapper/011:07:050
22944995444,5cyclictest33-21ksoftirqd/209:40:202
22944995443,6cyclictest33-21ksoftirqd/210:45:192
14752540,2sleep00-21swapper/009:20:330
114691540,2ptp4l0-21swapper/212:23:332
114691540,2ptp4l0-21swapper/010:10:310
114691540,1ptp4l0-21swapper/009:50:350
114691540,1ptp4l0-21swapper/009:45:300
114691540,13ptp4l0-21swapper/010:55:200
22944995328,17cyclictest33-21ksoftirqd/207:10:302
22943995339,8cyclictest25-21ksoftirqd/109:35:151
114791530,12getstats0-21swapper/309:25:043
114791530,10getstats0-21swapper/112:15:271
114691530,2ptp4l0-21swapper/310:48:083
114691530,2ptp4l0-21swapper/209:34:402
114691530,2ptp4l0-21swapper/107:26:301
114691530,2ptp4l0-21swapper/008:17:210
114691530,1ptp4l0-21swapper/109:48:451
22944995244,5cyclictest33-21ksoftirqd/209:15:152
114791520,11getstats0-21swapper/108:35:071
114691520,2ptp4l0-21swapper/212:01:112
114691520,2ptp4l0-21swapper/211:56:562
114691520,2ptp4l0-21swapper/109:10:291
114691520,2ptp4l0-21swapper/109:10:281
114691520,2ptp4l0-21swapper/012:02:230
114691520,2ptp4l0-21swapper/007:24:110
114691520,1ptp4l0-21swapper/111:12:151
114691520,1ptp4l0-21swapper/011:49:410
114691520,1ptp4l0-21swapper/009:32:130
114691520,12ptp4l0-21swapper/310:25:263
114691520,12ptp4l0-21swapper/010:37:270
2294499510,4cyclictest121rcu_preempt11:25:142
114791510,12getstats28394-21meminfo12:35:111
114691510,2ptp4l0-21swapper/310:51:403
114691510,2ptp4l0-21swapper/212:07:192
114691510,2ptp4l0-21swapper/107:58:261
114691510,2ptp4l0-21swapper/107:45:121
114691510,2ptp4l0-21swapper/008:27:170
114691510,2ptp4l0-21swapper/008:06:560
114691510,2ptp4l0-21swapper/007:29:520
22944995021,7cyclictest33-21ksoftirqd/211:00:132
22944995020,4cyclictest33-21ksoftirqd/210:00:222
114791500,10getstats0-21swapper/109:25:321
114691500,2ptp4l0-21swapper/209:52:452
114691500,2ptp4l0-21swapper/112:06:281
114691500,2ptp4l0-21swapper/111:15:171
114691500,2ptp4l0-21swapper/110:29:081
114691500,11ptp4l0-21swapper/207:31:552
114691500,10ptp4l0-21swapper/012:20:580
22944994942,4cyclictest33-21ksoftirqd/211:15:152
22944994920,7cyclictest33-21ksoftirqd/208:55:022
114691490,2ptp4l0-21swapper/307:50:513
114691490,2ptp4l0-21swapper/208:21:012
114691490,2ptp4l0-21swapper/207:59:362
114691490,2ptp4l0-21swapper/111:34:051
114691490,2ptp4l0-21swapper/111:34:051
114691490,2ptp4l0-21swapper/110:40:591
114691490,2ptp4l0-21swapper/110:18:241
114691490,2ptp4l0-21swapper/110:05:171
114691490,2ptp4l0-21swapper/109:40:041
114691490,2ptp4l0-21swapper/108:41:271
114691490,2ptp4l0-21swapper/108:30:451
114691490,2ptp4l0-21swapper/009:04:110
114691490,2ptp4l0-21swapper/008:47:040
114691490,1ptp4l0-21swapper/112:25:171
114691490,1ptp4l0-21swapper/110:02:161
114691490,1ptp4l0-21swapper/109:51:311
114691490,1ptp4l0-21swapper/010:15:430
114691490,13ptp4l14754-21df_inode07:40:010
114691490,13ptp4l0-21swapper/009:18:400
22944994816,5cyclictest33-21ksoftirqd/209:45:402
22943994841,4cyclictest25-21ksoftirqd/108:00:141
114791480,9getstats0-21swapper/310:20:153
114791480,9getstats0-21swapper/108:48:291
114791480,10getstats0-21swapper/107:20:281
114691480,2ptp4l0-21swapper/310:19:293
114691480,2ptp4l0-21swapper/307:38:323
114691480,2ptp4l0-21swapper/210:11:492
114691480,1ptp4l0-21swapper/311:16:553
114691480,1ptp4l0-21swapper/307:27:283
114691480,1ptp4l0-21swapper/110:10:241
114691480,12ptp4l0-21swapper/307:18:123
22944994739,4cyclictest33-21ksoftirqd/211:35:162
22944994739,4cyclictest33-21ksoftirqd/211:35:152
22944994720,7cyclictest33-21ksoftirqd/208:15:212
22944994716,4cyclictest33-21ksoftirqd/210:05:232
22944994714,5cyclictest33-21ksoftirqd/208:10:002
22943994732,9cyclictest25-21ksoftirqd/107:35:161
114791470,9getstats0-21swapper/107:34:161
114691470,8ptp4l0-21swapper/211:09:082
114691470,8ptp4l0-21swapper/210:17:452
114691470,8ptp4l0-21swapper/011:28:130
114691470,7ptp4l16407-21/usr/sbin/munin10:20:130
114691470,2ptp4l0-21swapper/307:11:553
114691470,2ptp4l0-21swapper/207:46:492
114691470,2ptp4l0-21swapper/110:38:421
114691470,2ptp4l0-21swapper/012:27:210
114691470,2ptp4l0-21swapper/008:54:360
114691470,1ptp4l0-21swapper/012:17:060
114791460,8getstats0-21swapper/109:30:391
114691460,2ptp4l0-21swapper/309:57:383
114691460,2ptp4l0-21swapper/307:44:033
114691460,2ptp4l0-21swapper/307:34:473
114691460,2ptp4l0-21swapper/008:20:360
114691460,1ptp4l0-21swapper/307:22:343
114691460,1ptp4l0-21swapper/008:55:450
114691450,2ptp4l0-21swapper/308:04:333
114691450,2ptp4l0-21swapper/210:26:592
114691450,2ptp4l0-21swapper/009:06:300
114691450,2ptp4l0-21swapper/009:06:290
114691450,2ptp4l0-21swapper/008:42:060
114691450,2ptp4l0-21swapper/007:59:170
22944994436,4cyclictest33-21ksoftirqd/208:30:002
22943994437,4cyclictest25-21ksoftirqd/107:15:031
114691440,5ptp4l0-21swapper/111:45:141
114691440,2ptp4l0-21swapper/208:31:592
114691440,1ptp4l0-21swapper/111:08:161
114691440,1ptp4l0-21swapper/011:53:270
2294499435,11cyclictest33-21ksoftirqd/210:40:372
114691430,9ptp4l0-21swapper/008:14:320
114691430,6ptp4l19729-21ntp_states11:45:073
114691430,4ptp4l7009-21diskstats07:30:040
114691430,2ptp4l0-21swapper/111:00:261
114691430,2ptp4l0-21swapper/009:58:310
114691430,1ptp4l0-21swapper/309:00:203
114691430,1ptp4l0-21swapper/011:43:040
114691430,1ptp4l0-21swapper/009:26:570
22944994228,11cyclictest33-21ksoftirqd/210:20:032
114691420,9ptp4l0-21swapper/308:41:213
114691420,8ptp4l0-21swapper/311:01:023
114691420,8ptp4l0-21swapper/107:41:541
114691420,8ptp4l0-21swapper/010:28:590
114691420,5ptp4l41-21ksoftirqd/309:21:433
114691420,3ptp4l16878-21apt-get12:20:141
114691420,2ptp4l0-21swapper/011:15:210
114691420,1ptp4l0-21swapper/110:34:301
114691410,6ptp4l22058-21timerwakeupswit07:45:403
114691410,6ptp4l2128-21unixbench_singl09:20:401
114691410,5ptp4l13645-21irqstats12:15:233
114691410,5ptp4l1218-21irqstats11:20:283
114691410,4ptp4l4404-21apt-get10:45:161
114691410,4ptp4l24764-21apt-get10:30:143
114691410,3ptp4l12645-21apt-get08:55:163
114691410,2ptp4l0-21swapper/312:05:313
114691410,2ptp4l0-21swapper/312:01:563
114691410,2ptp4l0-21swapper/311:10:223
114691410,2ptp4l0-21swapper/011:11:480
114691410,2ptp4l0-21swapper/007:48:290
22945994025,9cyclictest41-21ksoftirqd/308:05:143
114691400,6ptp4l15512-21ntp_states08:20:073
114691400,5ptp4l955-21diskstats10:00:203
114691400,5ptp4l28133-21iostat_ios08:35:103
114691400,4ptp4l3267-21diskstats10:05:043
114691400,4ptp4l24448-21iostat_ios11:50:113
114691400,4ptp4l23697-21hddtemp_smartct09:10:073
114691400,4ptp4l23697-21hddtemp_smartct09:10:063
114691400,2ptp4l0-21swapper/009:44:570
114691400,2ptp4l0-21swapper/008:32:170
114691390,5ptp4l25533-21snmpd11:43:243
114691390,5ptp4l10822-21/usr/sbin/munin10:55:033
114691390,4ptp4l3725-21tr10:05:080
114691390,4ptp4l25683-21missed_timers08:30:283
114691390,4ptp4l23859-21ntp_states12:30:083
114691390,4ptp4l21111-21diskstats09:45:233
114691390,4ptp4l16533-21apt-get12:20:123
114691390,1ptp4l0-21swapper/311:32:353
114691390,1ptp4l0-21swapper/311:32:353
114691380,5ptp4l28601-21python10:35:133
114691380,5ptp4l27223-21cpu09:15:023
114691380,4ptp4l22941-21cyclictest10:10:053
114691380,2ptp4l0-21swapper/312:10:073
114691380,2ptp4l0-21swapper/311:35:053
114691380,2ptp4l0-21swapper/311:35:043
114691380,2ptp4l0-21swapper/311:05:113
114691370,3ptp4l0-21swapper/309:30:123
114691370,2ptp4l0-21swapper/312:35:273
114691360,2ptp4l0-21swapper/312:28:183
114691360,2ptp4l0-21swapper/310:40:093
114691360,2ptp4l0-21swapper/308:45:013
114691350,2ptp4l0-21swapper/309:38:493
114691350,2ptp4l0-21swapper/308:51:163
114691350,2ptp4l0-21swapper/308:25:233
114691350,2ptp4l0-21swapper/308:12:013
114691350,1ptp4l0-21swapper/309:05:333
114691350,1ptp4l0-21swapper/309:05:333
114691330,1ptp4l0-21swapper/309:43:083
114691330,1ptp4l0-21swapper/308:16:413
114691320,2ptp4l0-21swapper/309:52:593
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional