You are here: Home / Projects / QA Farm Realtime / Latency plots / 
2021-10-22 - 03:26

Intel(R) Atom(TM) CPU E3845 @ 1.91GHz, Linux 4.19.37-rt19 (Profile)

Latency plot of system in rack #d, slot #1
Data to construct the above plot have been generated using the RT test utility cyclictest.
Unexpectedly long latencies may be caused by SMIs
Command line: cyclictest -l100000000 -m -Up99 -i200 -h400 --smi -q
Total number of samples: 100 million
Resolution of latency scale: normal
Duration: 5 hours, 33 minutes, lowest P state: performance
Characteristics of the highest latencies:
System rackdslot1.osadl.org (updated Fri Oct 15, 2021 12:46:10)
Delayed (victim)Switcher (culprit)TimestampCPU
PIDPrioTotal
latency
(µs)
T*(,W**)
latency
(µs)
CmdPIDPrioCmd
1146911970,12ptp4l9-21ksoftirqd/007:05:390
2126621780,8sleep12184699cyclictest12:25:281
1146911730,48ptp4l0-21swapper/307:05:073
1146911730,14ptp4l0-21swapper/107:06:241
722421690,6sleep02184599cyclictest11:30:110
36721690,6sleep12184699cyclictest10:40:211
1146911410,14ptp4l0-21swapper/207:06:532
21848999885,6cyclictest41-21ksoftirqd/311:50:123
21848999883,7cyclictest41-21ksoftirqd/308:10:193
21848999180,6cyclictest41-21ksoftirqd/312:35:153
21848999180,6cyclictest41-21ksoftirqd/312:35:153
21848998571,8cyclictest41-21ksoftirqd/308:45:183
21848998471,7cyclictest41-21ksoftirqd/310:55:173
21848998269,7cyclictest41-21ksoftirqd/311:40:143
21848998066,7cyclictest41-21ksoftirqd/312:05:153
21848997966,8cyclictest41-21ksoftirqd/307:15:153
69592780,5sleep321844-21cyclictest10:10:083
21848997765,6cyclictest41-21ksoftirqd/310:00:153
21848997765,6cyclictest41-21ksoftirqd/310:00:143
21848997765,5cyclictest41-21ksoftirqd/309:35:163
21848997360,7cyclictest41-21ksoftirqd/308:00:153
21848997265,4cyclictest41-21ksoftirqd/312:15:143
42642710,2sleep30-21swapper/307:25:283
42642710,2sleep30-21swapper/307:25:273
21848997157,7cyclictest41-21ksoftirqd/307:10:183
21848997057,6cyclictest41-21ksoftirqd/310:25:153
21845997055,7cyclictest9-21ksoftirqd/008:30:160
21848996856,6cyclictest41-21ksoftirqd/312:10:153
114691680,4ptp4l480-21cut09:20:210
114691680,22ptp4l29113-21sed09:15:270
21848996760,4cyclictest41-21ksoftirqd/309:30:143
21848996741,10cyclictest41-21ksoftirqd/308:25:093
189412670,2sleep0111rcuc/007:45:080
114691670,2ptp4l0-21swapper/310:30:083
114691670,2ptp4l0-21swapper/210:20:112
60962660,5sleep26074-21df_inode11:30:012
114691660,3ptp4l23900-21apt-get07:50:152
114691660,2ptp4l0-21swapper/209:10:322
114691660,2ptp4l0-21swapper/010:35:060
114691650,10ptp4l0-21swapper/011:05:190
21848996426,9cyclictest41-21ksoftirqd/308:35:023
114691640,17ptp4l12208-21ls10:15:142
21848996350,7cyclictest41-21ksoftirqd/312:20:163
21847996349,8cyclictest33-21ksoftirqd/212:30:132
21847996349,8cyclictest33-21ksoftirqd/212:30:132
114691630,2ptp4l0-21swapper/311:09:063
114691630,2ptp4l0-21swapper/208:25:362
21848996252,5cyclictest41-21ksoftirqd/310:05:193
114691620,2ptp4l0-21swapper/309:47:423
114691620,17ptp4l0-21swapper/010:25:150
305992610,2sleep00-21swapper/011:20:020
2184699617,8cyclictest25-21ksoftirqd/107:25:381
2184699617,8cyclictest25-21ksoftirqd/107:25:371
21848996024,4cyclictest41-21ksoftirqd/307:55:223
114691600,2ptp4l0-21swapper/109:10:311
2184899594,4cyclictest20765-21kworker/3:109:20:133
21846995928,7cyclictest25-21ksoftirqd/112:05:271
114691590,5ptp4l28169-21diskstats07:15:220
114691590,4ptp4l16487-21hddtemp_smartct11:00:200
114691590,3ptp4l17593-21ls09:00:440
114691590,2ptp4l0-21swapper/208:50:162
21848995847,5cyclictest41-21ksoftirqd/307:50:143
21848995827,4cyclictest41-21ksoftirqd/312:25:203
2184699587,11cyclictest25-21ksoftirqd/108:20:231
2184699587,10cyclictest25-21ksoftirqd/110:55:091
2184699586,10cyclictest0-21swapper/108:00:281
114691580,5ptp4l27501-21df_abs09:55:110
114691580,2ptp4l0-21swapper/311:00:093
114691580,2ptp4l0-21swapper/007:10:290
2184699577,8cyclictest25-21ksoftirqd/109:15:331
2184699577,10cyclictest25-21ksoftirqd/107:40:241
114691570,2ptp4l0-21swapper/012:33:120
114691570,2ptp4l0-21swapper/012:33:110
114691570,2ptp4l0-21swapper/011:55:290
114691570,2ptp4l0-21swapper/010:40:080
21848995622,5cyclictest41-21ksoftirqd/310:15:153
2184699568,4cyclictest5885-21strings11:30:011
2184699567,8cyclictest25-21ksoftirqd/110:30:231
2184699566,9cyclictest25-21ksoftirqd/109:40:271
2184699566,8cyclictest25-21ksoftirqd/111:20:001
2184699565,9cyclictest25-21ksoftirqd/109:35:331
2184699565,9cyclictest25-21ksoftirqd/107:15:161
21846995626,7cyclictest25-21ksoftirqd/108:40:271
21846995626,7cyclictest25-21ksoftirqd/108:40:261
114691560,2ptp4l0-21swapper/208:35:042
114691560,2ptp4l0-21swapper/207:20:162
114691560,2ptp4l0-21swapper/207:20:152
21848995524,5cyclictest41-21ksoftirqd/307:40:453
2184699557,9cyclictest25-21ksoftirqd/111:00:061
2184699557,8cyclictest25-21ksoftirqd/111:20:021
2184699557,7cyclictest25-21ksoftirqd/110:20:131
2184699554,8cyclictest25-21ksoftirqd/110:00:081
2184699554,8cyclictest25-21ksoftirqd/110:00:081
2184699554,5cyclictest25-21ksoftirqd/107:35:141
2184699552,11cyclictest25-21ksoftirqd/107:10:051
114691550,4ptp4l5447-21chrt07:26:160
114691550,4ptp4l5447-21chrt07:26:160
114691550,2ptp4l0-21swapper/108:15:351
114691550,2ptp4l0-21swapper/010:48:330
114691550,2ptp4l0-21swapper/010:05:170
21848995424,4cyclictest41-21ksoftirqd/312:30:293
21848995424,4cyclictest41-21ksoftirqd/312:30:283
2184699547,4cyclictest12063-21apt-get12:15:141
2184699546,9cyclictest25-21ksoftirqd/111:50:151
2184699546,8cyclictest25-21ksoftirqd/111:35:091
2184699546,8cyclictest25-21ksoftirqd/110:45:141
2184699546,8cyclictest25-21ksoftirqd/108:40:011
2184699546,8cyclictest25-21ksoftirqd/108:05:101
2184699546,7cyclictest25-21ksoftirqd/110:35:381
2184699546,7cyclictest25-21ksoftirqd/110:05:051
2184699546,7cyclictest25-21ksoftirqd/109:30:051
2184699546,10cyclictest25-21ksoftirqd/108:25:021
2184699546,10cyclictest25-21ksoftirqd/107:20:101
2184699546,10cyclictest25-21ksoftirqd/107:20:091
2184699545,9cyclictest25-21ksoftirqd/108:30:031
2184699545,7cyclictest25-21ksoftirqd/109:10:011
2184699544,8cyclictest25-21ksoftirqd/111:05:021
2184699543,8cyclictest25-21ksoftirqd/112:35:011
2184699543,8cyclictest25-21ksoftirqd/112:35:011
114691540,4ptp4l17048-21ntp_states09:40:373
114691540,2ptp4l0-21swapper/308:30:143
114691540,2ptp4l0-21swapper/112:10:171
114691540,2ptp4l0-21swapper/012:03:120
114691540,2ptp4l0-21swapper/010:20:120
114691540,2ptp4l0-21swapper/009:48:510
114691540,1ptp4l0-21swapper/011:41:220
67272530,4sleep16729-21irqrtprio08:50:041
302232530,2sleep20-21swapper/209:20:012
21848995347,4cyclictest41-21ksoftirqd/311:45:143
21848995312,10cyclictest41-21ksoftirqd/307:35:383
2184699537,8cyclictest25-21ksoftirqd/107:45:021
2184699537,7cyclictest25-21ksoftirqd/109:20:131
2184699537,5cyclictest25-21ksoftirqd/109:50:121
2184699536,8cyclictest25-21ksoftirqd/112:20:101
2184699536,8cyclictest25-21ksoftirqd/111:45:091
2184699536,8cyclictest25-21ksoftirqd/111:30:131
2184699536,8cyclictest25-21ksoftirqd/109:25:091
2184699536,7cyclictest18095-21strings11:45:001
2184699535,8cyclictest25-21ksoftirqd/110:10:311
2184699535,8cyclictest25-21ksoftirqd/108:55:081
2184699535,7cyclictest25-21ksoftirqd/107:55:071
2184699534,9cyclictest25-21ksoftirqd/111:10:131
114691530,2ptp4l0-21swapper/009:28:500
114691530,2ptp4l0-21swapper/007:53:130
2184699527,9cyclictest25-21ksoftirqd/108:45:101
2184699527,7cyclictest25-21ksoftirqd/112:00:041
2184699526,8cyclictest25-21ksoftirqd/112:30:081
2184699526,8cyclictest25-21ksoftirqd/112:30:071
2184699526,8cyclictest25-21ksoftirqd/110:50:041
2184699526,8cyclictest25-21ksoftirqd/110:25:081
2184699526,8cyclictest25-21ksoftirqd/109:45:251
2184699526,8cyclictest25-21ksoftirqd/107:50:031
2184699525,8cyclictest25-21ksoftirqd/111:55:041
2184699525,7cyclictest25-21ksoftirqd/110:15:171
2184699521,7cyclictest241ktimersoftd/108:10:191
114691520,2ptp4l0-21swapper/211:35:132
114691520,10ptp4l3865-21apt-get12:05:242
21848995122,4cyclictest41-21ksoftirqd/309:25:173
21848995122,4cyclictest41-21ksoftirqd/308:20:313
21848995115,9cyclictest41-21ksoftirqd/310:50:433
2184699517,9cyclictest25-21ksoftirqd/107:30:271
2184699515,8cyclictest25-21ksoftirqd/109:55:291
114691510,2ptp4l0-21swapper/212:26:422
21848995034,8cyclictest41-21ksoftirqd/309:05:313
21848995020,4cyclictest41-21ksoftirqd/311:25:383
21848995020,4cyclictest41-21ksoftirqd/307:30:373
21848995019,5cyclictest41-21ksoftirqd/310:40:263
21848995018,4cyclictest41-21ksoftirqd/309:55:033
21848995012,11cyclictest41-21ksoftirqd/311:30:093
21848995011,11cyclictest41-21ksoftirqd/308:55:333
2184899500,3cyclictest121rcu_preempt08:15:093
2184699504,8cyclictest25-21ksoftirqd/109:00:021
114691500,4ptp4l7805-21apt-get09:30:140
114691500,2ptp4l0-21swapper/008:06:300
114691500,1ptp4l0-21swapper/012:05:290
21848994943,4cyclictest41-21ksoftirqd/309:15:163
21848994942,4cyclictest41-21ksoftirqd/308:05:163
21848994934,11cyclictest41-21ksoftirqd/309:50:313
21848994912,8cyclictest41-21ksoftirqd/310:25:023
21848994911,9cyclictest41-21ksoftirqd/308:50:093
21845994936,7cyclictest9-21ksoftirqd/011:10:140
114691490,1ptp4l0-21swapper/212:20:272
114691490,1ptp4l0-21swapper/209:05:222
114691490,12ptp4l0-21swapper/211:20:152
2184899489,10cyclictest41-21ksoftirqd/310:40:013
21848994810,10cyclictest41-21ksoftirqd/311:55:143
114691480,2ptp4l0-21swapper/207:40:512
114691480,2ptp4l0-21swapper/012:35:570
114691480,2ptp4l0-21swapper/012:35:560
114691480,1ptp4l0-21swapper/010:12:450
114691480,1ptp4l0-21swapper/008:00:510
2184899479,8cyclictest41-21ksoftirqd/311:15:023
2184899479,8cyclictest41-21ksoftirqd/309:05:003
2184899475,11cyclictest41-21ksoftirqd/310:45:023
21848994711,12cyclictest41-21ksoftirqd/309:10:373
21848994710,11cyclictest41-21ksoftirqd/307:45:133
114691470,2ptp4l0-21swapper/212:12:522
114691470,2ptp4l0-21swapper/211:08:532
114691470,2ptp4l0-21swapper/009:54:120
114691470,2ptp4l0-21swapper/009:10:420
21848994620,7cyclictest41-21ksoftirqd/311:10:013
21848994617,4cyclictest41-21ksoftirqd/311:20:263
21848994611,5cyclictest41-21ksoftirqd/312:00:113
21845994634,5cyclictest9-21ksoftirqd/007:35:160
114691460,9ptp4l0-21swapper/207:30:052
114691460,8ptp4l0-21swapper/212:38:162
114691460,8ptp4l0-21swapper/212:38:162
114691460,8ptp4l0-21swapper/211:45:042
114691460,7ptp4l0-21swapper/210:53:192
114691460,4ptp4l121rcu_preempt07:20:030
114691460,4ptp4l121rcu_preempt07:20:020
114691460,2ptp4l0-21swapper/012:21:570
114691460,2ptp4l0-21swapper/010:17:210
114691460,1ptp4l0-21swapper/210:01:422
114691460,1ptp4l0-21swapper/210:01:422
114691460,1ptp4l0-21swapper/011:19:110
114691460,1ptp4l0-21swapper/008:18:490
114691460,13ptp4l0-21swapper/209:27:362
114691460,10ptp4l0-21swapper/211:30:082
21848994540,3cyclictest41-21ksoftirqd/311:35:363
21848994519,8cyclictest41-21ksoftirqd/307:20:113
21848994519,8cyclictest41-21ksoftirqd/307:20:113
114691450,8ptp4l0-21swapper/211:59:442
114691450,8ptp4l0-21swapper/010:02:350
114691450,8ptp4l0-21swapper/010:02:350
114691450,8ptp4l0-21swapper/007:42:130
114691450,7ptp4l0-21swapper/008:41:140
114691450,7ptp4l0-21swapper/008:41:130
114691450,4ptp4l2040-21df11:25:020
114691450,4ptp4l12956-21irqstats12:15:222
114691450,2ptp4l0-21swapper/209:57:502
114691450,2ptp4l0-21swapper/207:59:582
114691450,2ptp4l0-21swapper/008:35:160
114691450,2ptp4l0-21swapper/008:21:510
114691450,10ptp4l0-21swapper/210:25:312
114691450,10ptp4l0-21swapper/208:10:212
114691440,8ptp4l0-21swapper/211:14:222
114691440,7ptp4l0-21swapper/207:38:432
114691440,7ptp4l0-21swapper/207:13:032
114691440,2ptp4l0-21swapper/210:11:022
114691440,2ptp4l0-21swapper/008:26:440
114691440,2ptp4l0-21swapper/007:34:040
114691440,1ptp4l0-21swapper/012:28:270
114691440,10ptp4l0-21swapper/207:47:532
2184899438,9cyclictest41-21ksoftirqd/308:40:193
2184899438,9cyclictest41-21ksoftirqd/308:40:183
114691430,7ptp4l0-21swapper/010:34:250
114691430,6ptp4l30737-21grep08:00:052
114691430,5ptp4l13903-21taskset10:59:352
114691430,3ptp4l15091-21irqstats11:00:082
114691430,2ptp4l0-21swapper/210:45:042
114691430,2ptp4l0-21swapper/209:40:202
114691430,2ptp4l0-21swapper/209:39:342
114691430,2ptp4l0-21swapper/208:49:052
114691430,2ptp4l0-21swapper/011:49:400
114691430,2ptp4l0-21swapper/010:50:250
114691430,1ptp4l0-21swapper/011:39:300
21845994235,4cyclictest9-21ksoftirqd/012:15:050
114691420,4ptp4l3510-21apt-get07:25:152
114691420,4ptp4l3510-21apt-get07:25:142
114691420,4ptp4l33-21ksoftirqd/208:59:402
114691420,2ptp4l0-21swapper/209:21:262
114691420,2ptp4l0-21swapper/007:59:540
114691420,1ptp4l0-21swapper/012:11:300
114691420,1ptp4l0-21swapper/008:55:550
114691410,2ptp4l0-21swapper/212:02:542
114691410,2ptp4l0-21swapper/210:41:452
114691410,2ptp4l0-21swapper/210:34:402
114691410,2ptp4l0-21swapper/210:07:382
114691410,2ptp4l0-21swapper/011:54:500
114691410,2ptp4l0-21swapper/010:55:320
114691410,2ptp4l0-21swapper/009:09:210
114691410,1ptp4l0-21swapper/211:43:152
114691400,5ptp4l8367-21diskstats08:50:210
114691400,5ptp4l6989-21irqstats08:10:130
114691400,5ptp4l31099-21memory08:40:062
114691400,5ptp4l31099-21memory08:40:062
114691400,5ptp4l24288-21df_inode09:50:162
114691400,5ptp4l17423-21timerwakeupswit08:20:332
114691400,5ptp4l13131-21/usr/sbin/munin09:35:330
114691400,4ptp4l24218-21df08:30:172
114691400,4ptp4l22540-21diskstats11:50:052
114691400,4ptp4l12553-21iostat_ios08:15:262
114691400,3ptp4l7453-21/usr/sbin/munin09:30:132
114691400,2ptp4l0-21swapper/211:16:442
114691400,2ptp4l0-21swapper/210:36:262
114691400,2ptp4l0-21swapper/208:07:402
114691400,2ptp4l0-21swapper/207:16:352
114691400,2ptp4l0-21swapper/009:43:180
114691390,2ptp4l0-21swapper/209:00:232
114691390,1ptp4l0-21swapper/209:48:372
114691380,5ptp4l1076-21runrttasks08:45:360
*Timer  **Wakeup  (latency=timer+wakeup+contextswitch)

 

Valid XHTML 1.0 Transitional