You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-01-20 - 11:10

Intel(R) Atom(TM) CPU E3845 @ 1.91GHz, Linux 4.19.37-rt19 (Profile)

Latency plot of system in rack #d, slot #1
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Up99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of all highest latencies:
System rackdslot1.osadl.org (updated Fri Jan 15, 2021 00:46:07)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1146911880,15ptp4l0-21swapper/219:08:102
1146911880,15ptp4l0-21swapper/219:08:102
3056721790,9sleep22614899cyclictest22:50:182
1146911750,13ptp4l0-21swapper/319:06:203
1146911750,13ptp4l0-21swapper/319:06:203
2281421660,8sleep12614799cyclictest19:45:161
1146911660,8ptp4l0-21swapper/019:05:340
1146911660,8ptp4l0-21swapper/019:05:330
1139621660,8sleep32614999cyclictest19:30:173
118791163142,9phc2sys0-21swapper/119:09:541
118791163142,9phc2sys0-21swapper/119:09:531
26146997862,11cyclictest9-21ksoftirqd/021:15:070
26147997233,8cyclictest25-21ksoftirqd/123:25:101
26147997233,8cyclictest25-21ksoftirqd/122:05:071
26147997134,9cyclictest25-21ksoftirqd/119:30:021
26147997132,7cyclictest25-21ksoftirqd/119:55:231
114691680,4ptp4l17378-21iostat_ios20:25:100
26147996731,8cyclictest25-21ksoftirqd/123:55:181
114691670,2ptp4l0-21swapper/021:25:050
26147996636,4cyclictest25-21ksoftirqd/123:10:231
26147996632,9cyclictest25-21ksoftirqd/100:15:051
26147996630,9cyclictest25-21ksoftirqd/119:35:581
26147996630,8cyclictest25-21ksoftirqd/121:20:151
26147996629,8cyclictest25-21ksoftirqd/123:15:061
114691660,4ptp4l17073-21memory00:00:150
26147996531,9cyclictest25-21ksoftirqd/122:55:141
26147996529,7cyclictest25-21ksoftirqd/121:50:091
114691650,6ptp4l30771-21df_abs00:20:071
26147996433,4cyclictest25-21ksoftirqd/120:01:111
26147996433,4cyclictest25-21ksoftirqd/119:10:171
26147996433,4cyclictest25-21ksoftirqd/119:10:171
26147996433,4cyclictest25-21ksoftirqd/100:25:211
26147996432,8cyclictest25-21ksoftirqd/119:25:071
26147996431,9cyclictest25-21ksoftirqd/122:50:031
26147996431,8cyclictest25-21ksoftirqd/120:35:081
26147996431,5cyclictest25-21ksoftirqd/121:12:291
26147996429,8cyclictest25-21ksoftirqd/122:40:101
26147996428,9cyclictest25-21ksoftirqd/120:17:581
26147996427,8cyclictest25-21ksoftirqd/123:30:111
114691640,6ptp4l20039-21apt22:40:001
91002630,5sleep09103-21timerandwakeup23:50:140
2614799634,7cyclictest25-21ksoftirqd/119:15:051
26147996331,9cyclictest25-21ksoftirqd/119:40:131
26147996330,9cyclictest25-21ksoftirqd/100:00:061
26147996327,8cyclictest25-21ksoftirqd/121:30:021
26147996327,8cyclictest25-21ksoftirqd/121:01:341
114691630,5ptp4l31223-21iostat_ios22:10:110
114691630,5ptp4l31223-21iostat_ios22:10:100
114691630,2ptp4l0-21swapper/023:05:050
2614999629,7cyclictest41-21ksoftirqd/321:10:533
26147996230,8cyclictest25-21ksoftirqd/100:10:121
2614799619,4cyclictest25-21ksoftirqd/123:35:181
2614799619,4cyclictest25-21ksoftirqd/123:35:171
2614799616,7cyclictest25-21ksoftirqd/123:45:101
26147996130,4cyclictest25-21ksoftirqd/121:40:161
26147996129,4cyclictest25-21ksoftirqd/100:08:561
26147996128,8cyclictest25-21ksoftirqd/121:35:091
26147996127,7cyclictest25-21ksoftirqd/120:20:131
26147996127,5cyclictest25-21ksoftirqd/121:55:051
26147996126,9cyclictest25-21ksoftirqd/120:25:061
26147996125,7cyclictest25-21ksoftirqd/122:25:121
114691610,14ptp4l0-21swapper/023:00:120
2614899601,16cyclictest114691ptp4l19:40:182
26147996048,4cyclictest25-21ksoftirqd/120:50:071
114691600,2ptp4l0-21swapper/222:41:002
114691600,1ptp4l0-21swapper/019:54:340
35062590,3sleep0111rcuc/022:15:140
2614799597,6cyclictest7618-21/usr/sbin/munin00:30:141
2614799596,8cyclictest25-21ksoftirqd/119:20:061
2614799594,7cyclictest25-21ksoftirqd/120:45:161
26147995928,4cyclictest25-21ksoftirqd/122:48:181
114691590,2ptp4l0-21swapper/222:26:382
114691590,2ptp4l0-21swapper/022:55:070
114691590,2ptp4l0-21swapper/021:40:340
114691590,2ptp4l0-21swapper/021:20:150
114691590,2ptp4l0-21swapper/020:45:250
114691590,1ptp4l0-21swapper/022:20:370
26147995826,8cyclictest25-21ksoftirqd/122:30:051
114691580,6ptp4l24188-21df_inode22:45:030
114691580,2ptp4l0-21swapper/022:35:000
114691580,2ptp4l0-21swapper/020:43:030
2614999578,7cyclictest41-21ksoftirqd/321:05:053
26147995737,8cyclictest25-21ksoftirqd/121:05:061
26147995723,8cyclictest25-21ksoftirqd/121:45:181
2614799571,7cyclictest5784-21apt-get23:05:071
26147995711,3cyclictest25-21ksoftirqd/120:30:081
26147995710,4cyclictest25-21ksoftirqd/120:05:141
114691570,2ptp4l0-21swapper/023:55:150
67212560,2sleep20-21swapper/200:30:102
2614799565,4cyclictest26958-21apt-get20:40:041
26147995641,4cyclictest25-21ksoftirqd/122:25:001
26147995639,7cyclictest25-21ksoftirqd/120:55:231
114691560,2ptp4l0-21swapper/223:15:242
114691560,2ptp4l0-21swapper/100:35:181
114691560,2ptp4l0-21swapper/000:35:220
2614799556,9cyclictest25-21ksoftirqd/123:20:151
2614799554,7cyclictest25-21ksoftirqd/122:00:231
26147995535,8cyclictest25-21ksoftirqd/121:25:101
114691550,4ptp4l20325-21/usr/sbin/munin22:40:020
114691550,2ptp4l0-21swapper/319:10:023
114691550,2ptp4l0-21swapper/319:10:013
114691550,2ptp4l0-21swapper/020:30:180
114691550,1ptp4l0-21swapper/023:25:130
114691550,1ptp4l0-21swapper/022:04:230
26149995435,7cyclictest41-21ksoftirqd/319:40:183
2614799544,7cyclictest21172-21apt-get21:15:081
26147995442,4cyclictest25-21ksoftirqd/122:10:071
26147995442,4cyclictest25-21ksoftirqd/122:10:061
2614799541,8cyclictest7815-21apt-get23:50:071
2614799541,7cyclictest7558-21diskstats20:10:081
114691540,2ptp4l0-21swapper/122:15:051
114691540,2ptp4l0-21swapper/000:28:230
114691540,2ptp4l0-21swapper/000:10:460
114691540,11ptp4l0-21swapper/021:10:390
2614799535,9cyclictest25-21ksoftirqd/119:50:021
2614799532,8cyclictest25-21ksoftirqd/123:00:001
2614799531,4cyclictest32094-21apt-get23:40:041
114691530,5ptp4l1076-21runrttasks23:30:250
114691530,2ptp4l0-21swapper/023:24:370
114691530,2ptp4l0-21swapper/019:49:180
114691530,2ptp4l0-21swapper/019:25:250
114691530,2ptp4l0-21swapper/019:19:550
2614899510,31cyclictest114691ptp4l00:20:052
114691510,2ptp4l0-21swapper/023:19:260
114691510,2ptp4l0-21swapper/020:24:230
114691510,1ptp4l0-21swapper/021:45:220
114691510,15ptp4l1525-21meminfo22:15:032
2614999509,3cyclictest41-21ksoftirqd/323:50:063
2614999508,5cyclictest41-21ksoftirqd/323:25:093
2614999504,2cyclictest114691ptp4l00:05:033
114791500,11getstats0-21swapper/321:35:333
114691500,2ptp4l0-21swapper/023:39:520
114691500,2ptp4l0-21swapper/023:39:520
114691500,1ptp4l0-21swapper/019:24:490
114691500,10ptp4l0-21swapper/223:30:062
26149994913,6cyclictest41-21ksoftirqd/321:15:113
26146994934,8cyclictest9-21ksoftirqd/021:05:040
114691490,1ptp4l0-21swapper/019:30:440
114691490,14ptp4l32097-21meminfo23:40:042
114691480,6ptp4l23810-21apt-get22:45:052
114691480,2ptp4l0-21swapper/219:38:302
114691480,2ptp4l0-21swapper/021:31:020
114691480,2ptp4l0-21swapper/019:14:560
114691480,2ptp4l0-21swapper/019:14:550
114691480,1ptp4l0-21swapper/321:59:343
114691480,1ptp4l0-21swapper/000:08:310
26149994711,6cyclictest41-21ksoftirqd/322:45:163
114691470,8ptp4l0-21swapper/221:15:392
114691470,8ptp4l0-21swapper/219:59:452
114691470,8ptp4l0-21swapper/219:11:332
114691470,8ptp4l0-21swapper/219:11:322
114691470,2ptp4l0-21swapper/321:01:003
114691470,2ptp4l0-21swapper/319:29:423
114691470,2ptp4l0-21swapper/023:43:350
114691470,2ptp4l0-21swapper/022:54:560
114691470,2ptp4l0-21swapper/022:06:370
114691470,2ptp4l0-21swapper/021:55:450
114691470,2ptp4l0-21swapper/000:24:560
2614999468,3cyclictest114691ptp4l20:05:043
26149994613,8cyclictest41-21ksoftirqd/320:20:173
114691460,8ptp4l0-21swapper/222:39:322
114691460,8ptp4l0-21swapper/221:59:592
114691460,8ptp4l0-21swapper/221:09:462
114691460,8ptp4l0-21swapper/220:30:452
114691460,8ptp4l0-21swapper/220:22:272
114691460,8ptp4l0-21swapper/219:29:102
114691460,8ptp4l0-21swapper/200:39:102
114691460,8ptp4l0-21swapper/200:13:032
114691460,2ptp4l0-21swapper/322:08:313
114691460,2ptp4l0-21swapper/320:12:493
114691460,2ptp4l0-21swapper/319:19:253
114691460,2ptp4l0-21swapper/220:39:322
114691460,2ptp4l0-21swapper/022:28:040
114691460,2ptp4l0-21swapper/020:03:160
114691460,1ptp4l0-21swapper/220:12:352
114691460,1ptp4l0-21swapper/021:04:410
114691460,1ptp4l0-21swapper/020:57:160
114691460,1ptp4l0-21swapper/000:15:560
2614999459,5cyclictest41-21ksoftirqd/320:45:193
2614999455,3cyclictest114691ptp4l20:15:053
26149994514,6cyclictest41-21ksoftirqd/300:10:233
2614699456,3cyclictest121rcu_preempt20:35:020
114691450,8ptp4l0-21swapper/221:38:162
114691450,8ptp4l0-21swapper/221:11:112
114691450,8ptp4l0-21swapper/220:43:212
114691450,8ptp4l0-21swapper/219:30:502
114691450,7ptp4l0-21swapper/221:41:372
114691450,7ptp4l0-21swapper/200:02:372
114691450,3ptp4l3498-21apt-get23:00:072
114691450,2ptp4l0-21swapper/320:01:023
114691450,2ptp4l0-21swapper/219:20:002
114691450,2ptp4l0-21swapper/200:15:142
114691450,11ptp4l0-21swapper/221:50:022
114691450,10ptp4l0-21swapper/222:00:062
2614899445,18cyclictest114791getstats23:25:222
2614899440,2cyclictest0-21swapper/223:58:262
114691440,8ptp4l0-21swapper/221:48:552
114691440,7ptp4l0-21swapper/223:39:522
114691440,7ptp4l0-21swapper/223:39:512
114691440,2ptp4l0-21swapper/323:32:103
114691440,2ptp4l0-21swapper/322:35:073
114691440,2ptp4l0-21swapper/320:37:553
114691440,2ptp4l0-21swapper/320:31:353
114691440,2ptp4l0-21swapper/023:45:300
114691440,2ptp4l0-21swapper/020:12:160
114691440,2ptp4l0-21swapper/000:30:250
2614899432,34cyclictest1576-21apt-get00:25:072
114691430,8ptp4l0-21swapper/221:32:092
114691430,7ptp4l28437-21/usr/sbin/munin21:25:073
114691430,5ptp4l33-21ksoftirqd/222:55:092
114691430,3ptp4l29176-21apt-get21:25:102
114691430,2ptp4l0-21swapper/320:25:423
114691430,2ptp4l0-21swapper/219:21:042
114691430,2ptp4l0-21swapper/022:36:020
114691430,2ptp4l0-21swapper/021:53:020
114691430,2ptp4l0-21swapper/021:39:300
114691430,1ptp4l0-21swapper/220:29:392
114691430,10ptp4l0-21swapper/323:05:493
307302420,2sleep30-21swapper/323:35:203
307302420,2sleep30-21swapper/323:35:193
2614899424,32cyclictest0-21swapper/223:10:042
114691420,8ptp4l0-21swapper/223:21:522
114691420,5ptp4l24450-21apt-get19:50:062
114691420,5ptp4l18164-21apt-get00:05:052
114691420,4ptp4l9684-21apt-get21:00:042
114691420,4ptp4l26436-21apt-get22:05:042
114691420,2ptp4l0-21swapper/220:50:512
114691420,2ptp4l0-21swapper/220:17:542
114691420,2ptp4l0-21swapper/220:01:482
114691420,2ptp4l0-21swapper/219:45:032
114691420,2ptp4l0-21swapper/019:41:020
26149994137,2cyclictest391rcuc/321:45:123
26149994112,9cyclictest41-21ksoftirqd/323:55:223
26149994112,6cyclictest41-21ksoftirqd/323:22:233
26149994112,6cyclictest41-21ksoftirqd/300:31:253
26149994111,7cyclictest41-21ksoftirqd/322:28:583
26146994134,4cyclictest9-21ksoftirqd/023:10:070
114691410,7ptp4l0-21swapper/019:38:520
114691410,5ptp4l6643-21grep22:20:113
114691410,5ptp4l41-21ksoftirqd/319:55:053
114691410,5ptp4l1993-21/usr/sbin/munin20:50:190
114691410,4ptp4l24829-21ls21:20:043
114691410,4ptp4l12934-21apt-get22:30:052
114691410,2ptp4l0-21swapper/319:35:043
114691410,2ptp4l0-21swapper/223:54:462
114691410,2ptp4l0-21swapper/220:57:452
114691410,2ptp4l0-21swapper/220:47:222
114691410,2ptp4l0-21swapper/020:19:080
114691410,2ptp4l0-21swapper/020:08:570
114691410,2ptp4l0-21swapper/019:57:040
2614999409,7cyclictest41-21ksoftirqd/322:51:343
26149994012,6cyclictest41-21ksoftirqd/300:15:213
26149994011,7cyclictest41-21ksoftirqd/322:11:293
26149994011,7cyclictest41-21ksoftirqd/322:11:283
26149994011,7cyclictest41-21ksoftirqd/320:57:573
26149994011,7cyclictest41-21ksoftirqd/300:01:063
26149994010,7cyclictest41-21ksoftirqd/323:40:413
2614899402,29cyclictest5841-21apt-get22:20:062
114691400,5ptp4l24120-21ntp_states19:50:033
114691400,5ptp4l2101-21meminfo20:50:033
114691400,4ptp4l5158-21apt-get23:05:032
114691400,4ptp4l23355-21diskstats22:00:083
114691400,2ptp4l0-21swapper/319:20:133
114691400,2ptp4l0-21swapper/223:46:292
114691400,2ptp4l0-21swapper/222:13:172
114691400,2ptp4l0-21swapper/222:13:162
114691400,2ptp4l0-21swapper/221:22:312
114691400,2ptp4l0-21swapper/220:06:192
26149993913,6cyclictest41-21ksoftirqd/322:43:123
26149993911,7cyclictest41-21ksoftirqd/321:52:053
26149993910,8cyclictest41-21ksoftirqd/322:15:463
26149993910,7cyclictest41-21ksoftirqd/321:34:023
26149993910,7cyclictest41-21ksoftirqd/300:36:423
114691390,4ptp4l21426-21sed19:45:103
2614999389,7cyclictest41-21ksoftirqd/300:20:073
26149993811,7cyclictest41-21ksoftirqd/322:30:313
26149993810,7cyclictest41-21ksoftirqd/323:46:433
26149993810,7cyclictest41-21ksoftirqd/300:27:553
114691380,2ptp4l0-21swapper/323:15:233
26149993710,6cyclictest41-21ksoftirqd/322:59:093
114691370,2ptp4l0-21swapper/323:10:123
114691370,2ptp4l0-21swapper/323:00:073
114691370,2ptp4l0-21swapper/321:40:033
114691370,2ptp4l0-21swapper/320:40:043
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional