blib/lib/Paws/DMS/S3Settings.pm untested

filter: Statementstmt 0.0% 0 / 31 Branchbran n/a Conditioncond n/a MC/DCmcdc n/a Subroutinesub 0.0% 0 / 1 Podpod n/a totaltotal 0.0% 0 / 32 CC 1 SCAR 6.9
CC
1
Cov
0%
BEGIN
2.0
CRAP 2.0
1 # Generated by default/object.tt
2 package Paws::DMS::S3Settings;
3 use Moose;
4 has BucketFolder => (is => 'ro', isa => 'Str');
5 has BucketName => (is => 'ro', isa => 'Str');
6 has CdcInsertsAndUpdates => (is => 'ro', isa => 'Bool');
7 has CdcInsertsOnly => (is => 'ro', isa => 'Bool');
8 has CdcPath => (is => 'ro', isa => 'Str');
9 has CompressionType => (is => 'ro', isa => 'Str');
10 has CsvDelimiter => (is => 'ro', isa => 'Str');
11 has CsvNoSupValue => (is => 'ro', isa => 'Str');
12 has CsvRowDelimiter => (is => 'ro', isa => 'Str');
13 has DataFormat => (is => 'ro', isa => 'Str');
14 has DataPageSize => (is => 'ro', isa => 'Int');
15 has DatePartitionDelimiter => (is => 'ro', isa => 'Str');
16 has DatePartitionEnabled => (is => 'ro', isa => 'Bool');
17 has DatePartitionSequence => (is => 'ro', isa => 'Str');
18 has DictPageSizeLimit => (is => 'ro', isa => 'Int');
19 has EnableStatistics => (is => 'ro', isa => 'Bool');
20 has EncodingType => (is => 'ro', isa => 'Str');
21 has EncryptionMode => (is => 'ro', isa => 'Str');
22 has ExternalTableDefinition => (is => 'ro', isa => 'Str');
23 has IncludeOpForFullLoad => (is => 'ro', isa => 'Bool');
24 has ParquetTimestampInMillisecond => (is => 'ro', isa => 'Bool');
25 has ParquetVersion => (is => 'ro', isa => 'Str');
26 has PreserveTransactions => (is => 'ro', isa => 'Bool');
27 has RowGroupLength => (is => 'ro', isa => 'Int');
28 has ServerSideEncryptionKmsKeyId => (is => 'ro', isa => 'Str');
29 has ServiceAccessRoleArn => (is => 'ro', isa => 'Str');
30 has TimestampColumnName => (is => 'ro', isa => 'Str');
31 has UseCsvNoSupValue => (is => 'ro', isa => 'Bool');
32  
33 1;
34  
35 ### main pod documentation begin ###
36  
37 =head1 NAME
38  
39 Paws::DMS::S3Settings
40  
41 =head1 USAGE
42  
43 This class represents one of two things:
44  
45 =head3 Arguments in a call to a service
46  
47 Use the attributes of this class as arguments to methods. You shouldn't make instances of this class.
48 Each attribute should be used as a named argument in the calls that expect this type of object.
49  
50 As an example, if Att1 is expected to be a Paws::DMS::S3Settings object:
51  
52 $service_obj->Method(Att1 => { BucketFolder => $value, ..., UseCsvNoSupValue => $value });
53  
54 =head3 Results returned from an API call
55  
56 Use accessors for each attribute. If Att1 is expected to be an Paws::DMS::S3Settings object:
57  
58 $result = $service_obj->Method(...);
59 $result->Att1->BucketFolder
60  
61 =head1 DESCRIPTION
62  
63 Settings for exporting data to Amazon S3.
64  
65 =head1 ATTRIBUTES
66  
67  
68 =head2 BucketFolder => Str
69  
70 An optional parameter to set a folder name in the S3 bucket. If
71 provided, tables are created in the path C<
72 I<bucketFolder>/I<schema_name>/I<table_name>/>. If this parameter isn't
73 specified, then the path used is C< I<schema_name>/I<table_name>/>.
74  
75  
76 =head2 BucketName => Str
77  
78 The name of the S3 bucket.
79  
80  
81 =head2 CdcInsertsAndUpdates => Bool
82  
83 A value that enables a change data capture (CDC) load to write INSERT
84 and UPDATE operations to .csv or .parquet (columnar storage) output
85 files. The default setting is C<false>, but when
86 C<CdcInsertsAndUpdates> is set to C<true> or C<y>, only INSERTs and
87 UPDATEs from the source database are migrated to the .csv or .parquet
88 file.
89  
90 For .csv file format only, how these INSERTs and UPDATEs are recorded
91 depends on the value of the C<IncludeOpForFullLoad> parameter. If
92 C<IncludeOpForFullLoad> is set to C<true>, the first field of every CDC
93 record is set to either C<I> or C<U> to indicate INSERT and UPDATE
94 operations at the source. But if C<IncludeOpForFullLoad> is set to
95 C<false>, CDC records are written without an indication of INSERT or
96 UPDATE operations at the source. For more information about how these
97 settings work together, see Indicating Source DB Operations in Migrated
98 S3 Data
99 (https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.S3.html#CHAP_Target.S3.Configuring.InsertOps)
100 in the I<AWS Database Migration Service User Guide.>.
101  
102 AWS DMS supports the use of the C<CdcInsertsAndUpdates> parameter in
103 versions 3.3.1 and later.
104  
105 C<CdcInsertsOnly> and C<CdcInsertsAndUpdates> can't both be set to
106 C<true> for the same endpoint. Set either C<CdcInsertsOnly> or
107 C<CdcInsertsAndUpdates> to C<true> for the same endpoint, but not both.
108  
109  
110 =head2 CdcInsertsOnly => Bool
111  
112 A value that enables a change data capture (CDC) load to write only
113 INSERT operations to .csv or columnar storage (.parquet) output files.
114 By default (the C<false> setting), the first field in a .csv or
115 .parquet record contains the letter I (INSERT), U (UPDATE), or D
116 (DELETE). These values indicate whether the row was inserted, updated,
117 or deleted at the source database for a CDC load to the target.
118  
119 If C<CdcInsertsOnly> is set to C<true> or C<y>, only INSERTs from the
120 source database are migrated to the .csv or .parquet file. For .csv
121 format only, how these INSERTs are recorded depends on the value of
122 C<IncludeOpForFullLoad>. If C<IncludeOpForFullLoad> is set to C<true>,
123 the first field of every CDC record is set to I to indicate the INSERT
124 operation at the source. If C<IncludeOpForFullLoad> is set to C<false>,
125 every CDC record is written without a first field to indicate the
126 INSERT operation at the source. For more information about how these
127 settings work together, see Indicating Source DB Operations in Migrated
128 S3 Data
129 (https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.S3.html#CHAP_Target.S3.Configuring.InsertOps)
130 in the I<AWS Database Migration Service User Guide.>.
131  
132 AWS DMS supports the interaction described preceding between the
133 C<CdcInsertsOnly> and C<IncludeOpForFullLoad> parameters in versions
134 3.1.4 and later.
135  
136 C<CdcInsertsOnly> and C<CdcInsertsAndUpdates> can't both be set to
137 C<true> for the same endpoint. Set either C<CdcInsertsOnly> or
138 C<CdcInsertsAndUpdates> to C<true> for the same endpoint, but not both.
139  
140  
141 =head2 CdcPath => Str
142  
143 Specifies the folder path of CDC files. For an S3 source, this setting
144 is required if a task captures change data; otherwise, it's optional.
145 If C<CdcPath> is set, AWS DMS reads CDC files from this path and
146 replicates the data changes to the target endpoint. For an S3 target if
147 you set C<PreserveTransactions>
148 (https://docs.aws.amazon.com/dms/latest/APIReference/API_S3Settings.html#DMS-Type-S3Settings-PreserveTransactions)
149 to C<true>, AWS DMS verifies that you have set this parameter to a
150 folder path on your S3 target where AWS DMS can save the transaction
151 order for the CDC load. AWS DMS creates this CDC folder path in either
152 your S3 target working directory or the S3 target location specified by
153 C<BucketFolder>
154 (https://docs.aws.amazon.com/dms/latest/APIReference/API_S3Settings.html#DMS-Type-S3Settings-BucketFolder)
155 and C<BucketName>
156 (https://docs.aws.amazon.com/dms/latest/APIReference/API_S3Settings.html#DMS-Type-S3Settings-BucketName).
157  
158 For example, if you specify C<CdcPath> as C<MyChangedData>, and you
159 specify C<BucketName> as C<MyTargetBucket> but do not specify
160 C<BucketFolder>, AWS DMS creates the CDC folder path following:
161 C<MyTargetBucket/MyChangedData>.
162  
163 If you specify the same C<CdcPath>, and you specify C<BucketName> as
164 C<MyTargetBucket> and C<BucketFolder> as C<MyTargetData>, AWS DMS
165 creates the CDC folder path following:
166 C<MyTargetBucket/MyTargetData/MyChangedData>.
167  
168 For more information on CDC including transaction order on an S3
169 target, see Capturing data changes (CDC) including transaction order on
170 the S3 target
171 (https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.S3.html#CHAP_Target.S3.EndpointSettings.CdcPath).
172  
173 This setting is supported in AWS DMS versions 3.4.2 and later.
174  
175  
176 =head2 CompressionType => Str
177  
178 An optional parameter to use GZIP to compress the target files. Set to
179 GZIP to compress the target files. Either set this parameter to NONE
180 (the default) or don't use it to leave the files uncompressed. This
181 parameter applies to both .csv and .parquet file formats.
182  
183  
184 =head2 CsvDelimiter => Str
185  
186 The delimiter used to separate columns in the .csv file for both source
187 and target. The default is a comma.
188  
189  
190 =head2 CsvNoSupValue => Str
191  
192 This setting only applies if your Amazon S3 output files during a
193 change data capture (CDC) load are written in .csv format. If
194 C<UseCsvNoSupValue>
195 (https://docs.aws.amazon.com/dms/latest/APIReference/API_S3Settings.html#DMS-Type-S3Settings-UseCsvNoSupValue)
196 is set to true, specify a string value that you want AWS DMS to use for
197 all columns not included in the supplemental log. If you do not specify
198 a string value, AWS DMS uses the null value for these columns
199 regardless of the C<UseCsvNoSupValue> setting.
200  
201 This setting is supported in AWS DMS versions 3.4.1 and later.
202  
203  
204 =head2 CsvRowDelimiter => Str
205  
206 The delimiter used to separate rows in the .csv file for both source
207 and target. The default is a carriage return (C<\n>).
208  
209  
210 =head2 DataFormat => Str
211  
212 The format of the data that you want to use for output. You can choose
213 one of the following:
214  
215 =over
216  
217 =item *
218  
219 C<csv> : This is a row-based file format with comma-separated values
220 (.csv).
221  
222 =item *
223  
224 C<parquet> : Apache Parquet (.parquet) is a columnar storage file
225 format that features efficient compression and provides faster query
226 response.
227  
228 =back
229  
230  
231  
232 =head2 DataPageSize => Int
233  
234 The size of one data page in bytes. This parameter defaults to 1024 *
235 1024 bytes (1 MiB). This number is used for .parquet file format only.
236  
237  
238 =head2 DatePartitionDelimiter => Str
239  
240 Specifies a date separating delimiter to use during folder
241 partitioning. The default value is C<SLASH>. Use this parameter when
242 C<DatePartitionedEnabled> is set to C<true>.
243  
244  
245 =head2 DatePartitionEnabled => Bool
246  
247 When set to C<true>, this parameter partitions S3 bucket folders based
248 on transaction commit dates. The default value is C<false>. For more
249 information about date-based folder partitoning, see Using date-based
250 folder partitioning
251 (https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.S3.html#CHAP_Target.S3.DatePartitioning).
252  
253  
254 =head2 DatePartitionSequence => Str
255  
256 Identifies the sequence of the date format to use during folder
257 partitioning. The default value is C<YYYYMMDD>. Use this parameter when
258 C<DatePartitionedEnabled> is set to C<true>.
259  
260  
261 =head2 DictPageSizeLimit => Int
262  
263 The maximum size of an encoded dictionary page of a column. If the
264 dictionary page exceeds this, this column is stored using an encoding
265 type of C<PLAIN>. This parameter defaults to 1024 * 1024 bytes (1 MiB),
266 the maximum size of a dictionary page before it reverts to C<PLAIN>
267 encoding. This size is used for .parquet file format only.
268  
269  
270 =head2 EnableStatistics => Bool
271  
272 A value that enables statistics for Parquet pages and row groups.
273 Choose C<true> to enable statistics, C<false> to disable. Statistics
274 include C<NULL>, C<DISTINCT>, C<MAX>, and C<MIN> values. This parameter
275 defaults to C<true>. This value is used for .parquet file format only.
276  
277  
278 =head2 EncodingType => Str
279  
280 The type of encoding you are using:
281  
282 =over
283  
284 =item *
285  
286 C<RLE_DICTIONARY> uses a combination of bit-packing and run-length
287 encoding to store repeated values more efficiently. This is the
288 default.
289  
290 =item *
291  
292 C<PLAIN> doesn't use encoding at all. Values are stored as they are.
293  
294 =item *
295  
296 C<PLAIN_DICTIONARY> builds a dictionary of the values encountered in a
297 given column. The dictionary is stored in a dictionary page for each
298 column chunk.
299  
300 =back
301  
302  
303  
304 =head2 EncryptionMode => Str
305  
306 The type of server-side encryption that you want to use for your data.
307 This encryption type is part of the endpoint settings or the extra
308 connections attributes for Amazon S3. You can choose either C<SSE_S3>
309 (the default) or C<SSE_KMS>.
310  
311 For the C<ModifyEndpoint> operation, you can change the existing value
312 of the C<EncryptionMode> parameter from C<SSE_KMS> to C<SSE_S3>. But
313 you canE<rsquo>t change the existing value from C<SSE_S3> to
314 C<SSE_KMS>.
315  
316 To use C<SSE_S3>, you need an AWS Identity and Access Management (IAM)
317 role with permission to allow C<"arn:aws:s3:::dms-*"> to use the
318 following actions:
319  
320 =over
321  
322 =item *
323  
324 C<s3:CreateBucket>
325  
326 =item *
327  
328 C<s3:ListBucket>
329  
330 =item *
331  
332 C<s3:DeleteBucket>
333  
334 =item *
335  
336 C<s3:GetBucketLocation>
337  
338 =item *
339  
340 C<s3:GetObject>
341  
342 =item *
343  
344 C<s3:PutObject>
345  
346 =item *
347  
348 C<s3:DeleteObject>
349  
350 =item *
351  
352 C<s3:GetObjectVersion>
353  
354 =item *
355  
356 C<s3:GetBucketPolicy>
357  
358 =item *
359  
360 C<s3:PutBucketPolicy>
361  
362 =item *
363  
364 C<s3:DeleteBucketPolicy>
365  
366 =back
367  
368  
369  
370 =head2 ExternalTableDefinition => Str
371  
372 Specifies how tables are defined in the S3 source files only.
373  
374  
375 =head2 IncludeOpForFullLoad => Bool
376  
377 A value that enables a full load to write INSERT operations to the
378 comma-separated value (.csv) output files only to indicate how the rows
379 were added to the source database.
380  
381 AWS DMS supports the C<IncludeOpForFullLoad> parameter in versions
382 3.1.4 and later.
383  
384 For full load, records can only be inserted. By default (the C<false>
385 setting), no information is recorded in these output files for a full
386 load to indicate that the rows were inserted at the source database. If
387 C<IncludeOpForFullLoad> is set to C<true> or C<y>, the INSERT is
388 recorded as an I annotation in the first field of the .csv file. This
389 allows the format of your target records from a full load to be
390 consistent with the target records from a CDC load.
391  
392 This setting works together with the C<CdcInsertsOnly> and the
393 C<CdcInsertsAndUpdates> parameters for output to .csv files only. For
394 more information about how these settings work together, see Indicating
395 Source DB Operations in Migrated S3 Data
396 (https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.S3.html#CHAP_Target.S3.Configuring.InsertOps)
397 in the I<AWS Database Migration Service User Guide.>.
398  
399  
400 =head2 ParquetTimestampInMillisecond => Bool
401  
402 A value that specifies the precision of any C<TIMESTAMP> column values
403 that are written to an Amazon S3 object file in .parquet format.
404  
405 AWS DMS supports the C<ParquetTimestampInMillisecond> parameter in
406 versions 3.1.4 and later.
407  
408 When C<ParquetTimestampInMillisecond> is set to C<true> or C<y>, AWS
409 DMS writes all C<TIMESTAMP> columns in a .parquet formatted file with
410 millisecond precision. Otherwise, DMS writes them with microsecond
411 precision.
412  
413 Currently, Amazon Athena and AWS Glue can handle only millisecond
414 precision for C<TIMESTAMP> values. Set this parameter to C<true> for S3
415 endpoint object files that are .parquet formatted only if you plan to
416 query or process the data with Athena or AWS Glue.
417  
418 AWS DMS writes any C<TIMESTAMP> column values written to an S3 file in
419 .csv format with microsecond precision.
420  
421 Setting C<ParquetTimestampInMillisecond> has no effect on the string
422 format of the timestamp column value that is inserted by setting the
423 C<TimestampColumnName> parameter.
424  
425  
426 =head2 ParquetVersion => Str
427  
428 The version of the Apache Parquet format that you want to use:
429 C<parquet_1_0> (the default) or C<parquet_2_0>.
430  
431  
432 =head2 PreserveTransactions => Bool
433  
434 If set to C<true>, AWS DMS saves the transaction order for a change
435 data capture (CDC) load on the Amazon S3 target specified by C<CdcPath>
436 (https://docs.aws.amazon.com/dms/latest/APIReference/API_S3Settings.html#DMS-Type-S3Settings-CdcPath).
437 For more information, see Capturing data changes (CDC) including
438 transaction order on the S3 target
439 (https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.S3.html#CHAP_Target.S3.EndpointSettings.CdcPath).
440  
441 This setting is supported in AWS DMS versions 3.4.2 and later.
442  
443  
444 =head2 RowGroupLength => Int
445  
446 The number of rows in a row group. A smaller row group size provides
447 faster reads. But as the number of row groups grows, the slower writes
448 become. This parameter defaults to 10,000 rows. This number is used for
449 .parquet file format only.
450  
451 If you choose a value larger than the maximum, C<RowGroupLength> is set
452 to the max row group length in bytes (64 * 1024 * 1024).
453  
454  
455 =head2 ServerSideEncryptionKmsKeyId => Str
456  
457 If you are using C<SSE_KMS> for the C<EncryptionMode>, provide the AWS
458 KMS key ID. The key that you use needs an attached policy that enables
459 AWS Identity and Access Management (IAM) user permissions and allows
460 use of the key.
461  
462 Here is a CLI example: C<aws dms create-endpoint --endpoint-identifier
463 I<value> --endpoint-type target --engine-name s3 --s3-settings
464 ServiceAccessRoleArn=I<value>,BucketFolder=I<value>,BucketName=I<value>,EncryptionMode=SSE_KMS,ServerSideEncryptionKmsKeyId=I<value>>
465  
466  
467 =head2 ServiceAccessRoleArn => Str
468  
469 The Amazon Resource Name (ARN) used by the service access IAM role. It
470 is a required parameter that enables DMS to write and read objects from
471 an S3 bucket.
472  
473  
474 =head2 TimestampColumnName => Str
475  
476 A value that when nonblank causes AWS DMS to add a column with
477 timestamp information to the endpoint data for an Amazon S3 target.
478  
479 AWS DMS supports the C<TimestampColumnName> parameter in versions 3.1.4
480 and later.
481  
482 DMS includes an additional C<STRING> column in the .csv or .parquet
483 object files of your migrated data when you set C<TimestampColumnName>
484 to a nonblank value.
485  
486 For a full load, each row of this timestamp column contains a timestamp
487 for when the data was transferred from the source to the target by DMS.
488  
489 For a change data capture (CDC) load, each row of the timestamp column
490 contains the timestamp for the commit of that row in the source
491 database.
492  
493 The string format for this timestamp column value is C<yyyy-MM-dd
494 HH:mm:ss.SSSSSS>. By default, the precision of this value is in
495 microseconds. For a CDC load, the rounding of the precision depends on
496 the commit timestamp supported by DMS for the source database.
497  
498 When the C<AddColumnName> parameter is set to C<true>, DMS also
499 includes a name for the timestamp column that you set with
500 C<TimestampColumnName>.
501  
502  
503 =head2 UseCsvNoSupValue => Bool
504  
505 This setting applies if the S3 output files during a change data
506 capture (CDC) load are written in .csv format. If set to C<true> for
507 columns not included in the supplemental log, AWS DMS uses the value
508 specified by C<CsvNoSupValue>
509 (https://docs.aws.amazon.com/dms/latest/APIReference/API_S3Settings.html#DMS-Type-S3Settings-CsvNoSupValue).
510 If not set or set to C<false>, AWS DMS uses the null value for these
511 columns.
512  
513 This setting is supported in AWS DMS versions 3.4.1 and later.
514  
515  
516  
517 =head1 SEE ALSO
518  
519 This class forms part of L<Paws>, describing an object used in L<Paws::DMS>
520  
521 =head1 BUGS and CONTRIBUTIONS
522  
523 The source code is located here: L<https://github.com/pplu/aws-sdk-perl>
524  
525 Please report bugs to: L<https://github.com/pplu/aws-sdk-perl/issues>
526  
527 =cut
528