@aws-cdk/aws-glue-alpha 2.89.0-alpha.0 → 2.91.0-alpha.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/.jsii CHANGED
@@ -8,7 +8,7 @@
8
8
  "url": "https://aws.amazon.com"
9
9
  },
10
10
  "dependencies": {
11
- "aws-cdk-lib": "2.89.0",
11
+ "aws-cdk-lib": "2.91.0",
12
12
  "constructs": "^10.0.0"
13
13
  },
14
14
  "dependencyClosure": {
@@ -3492,7 +3492,7 @@
3492
3492
  "stability": "experimental"
3493
3493
  },
3494
3494
  "homepage": "https://github.com/aws/aws-cdk",
3495
- "jsiiVersion": "5.1.9 (build fa65a10)",
3495
+ "jsiiVersion": "5.1.10 (build 041401a)",
3496
3496
  "keywords": [
3497
3497
  "aws",
3498
3498
  "cdk",
@@ -3513,7 +3513,7 @@
3513
3513
  },
3514
3514
  "name": "@aws-cdk/aws-glue-alpha",
3515
3515
  "readme": {
3516
- "markdown": "# AWS Glue Construct Library\n<!--BEGIN STABILITY BANNER-->\n\n---\n\n![cdk-constructs: Experimental](https://img.shields.io/badge/cdk--constructs-experimental-important.svg?style=for-the-badge)\n\n> The APIs of higher level constructs in this module are experimental and under active development.\n> They are subject to non-backward compatible changes or removal in any future version. These are\n> not subject to the [Semantic Versioning](https://semver.org/) model and breaking changes will be\n> announced in the release notes. This means that while you may use them, you may need to update\n> your source code when upgrading to a newer version of this package.\n\n---\n\n<!--END STABILITY BANNER-->\n\nThis module is part of the [AWS Cloud Development Kit](https://github.com/aws/aws-cdk) project.\n\n## Job\n\nA `Job` encapsulates a script that connects to data sources, processes them, and then writes output to a data target.\n\nThere are 3 types of jobs supported by AWS Glue: Spark ETL, Spark Streaming, and Python Shell jobs.\n\nThe `glue.JobExecutable` allows you to specify the type of job, the language to use and the code assets required by the job.\n\n`glue.Code` allows you to refer to the different code assets required by the job, either from an existing S3 location or from a local file path.\n\n`glue.ExecutionClass` allows you to specify `FLEX` or `STANDARD`. `FLEX` is appropriate for non-urgent jobs such as pre-production jobs, testing, and one-time data loads.\n\n### Spark Jobs\n\nThese jobs run in an Apache Spark environment managed by AWS Glue.\n\n#### ETL Jobs\n\nAn ETL job processes data in batches using Apache Spark.\n\n```ts\ndeclare const bucket: s3.Bucket;\nnew glue.Job(this, 'ScalaSparkEtlJob', {\n executable: glue.JobExecutable.scalaEtl({\n glueVersion: glue.GlueVersion.V4_0,\n script: glue.Code.fromBucket(bucket, 'src/com/example/HelloWorld.scala'),\n className: 'com.example.HelloWorld',\n extraJars: [glue.Code.fromBucket(bucket, 'jars/HelloWorld.jar')],\n }),\n workerType: glue.WorkerType.G_8X,\n description: 'an example Scala ETL job',\n});\n```\n\n#### Streaming Jobs\n\nA Streaming job is similar to an ETL job, except that it performs ETL on data streams. It uses the Apache Spark Structured Streaming framework. Some Spark job features are not available to streaming ETL jobs.\n\n```ts\nnew glue.Job(this, 'PythonSparkStreamingJob', {\n executable: glue.JobExecutable.pythonStreaming({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n description: 'an example Python Streaming job',\n});\n```\n\n### Python Shell Jobs\n\nA Python shell job runs Python scripts as a shell and supports a Python version that depends on the AWS Glue version you are using.\nThis can be used to schedule and run tasks that don't require an Apache Spark environment. Currently, three flavors are supported:\n\n* PythonVersion.TWO (2.7; EOL)\n* PythonVersion.THREE (3.6)\n* PythonVersion.THREE_NINE (3.9)\n\n```ts\ndeclare const bucket: s3.Bucket;\nnew glue.Job(this, 'PythonShellJob', {\n executable: glue.JobExecutable.pythonShell({\n glueVersion: glue.GlueVersion.V1_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromBucket(bucket, 'script.py'),\n }),\n description: 'an example Python Shell job',\n});\n```\n\n### Ray Jobs\n\nThese jobs run in a Ray environment managed by AWS Glue.\n\n```ts\nnew glue.Job(this, 'RayJob', {\n executable: glue.JobExecutable.pythonRay({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE_NINE,\n runtime: glue.Runtime.RAY_TWO_FOUR,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n workerType: glue.WorkerType.Z_2X,\n workerCount: 2,\n description: 'an example Ray job'\n});\n```\n\nSee [documentation](https://docs.aws.amazon.com/glue/latest/dg/add-job.html) for more information on adding jobs in Glue.\n\n## Connection\n\nA `Connection` allows Glue jobs, crawlers and development endpoints to access certain types of data stores. For example, to create a network connection to connect to a data source within a VPC:\n\n```ts\ndeclare const securityGroup: ec2.SecurityGroup;\ndeclare const subnet: ec2.Subnet;\nnew glue.Connection(this, 'MyConnection', {\n type: glue.ConnectionType.NETWORK,\n // The security groups granting AWS Glue inbound access to the data source within the VPC\n securityGroups: [securityGroup],\n // The VPC subnet which contains the data source\n subnet,\n});\n```\n\nFor RDS `Connection` by JDBC, it is recommended to manage credentials using AWS Secrets Manager. To use Secret, specify `SECRET_ID` in `properties` like the following code. Note that in this case, the subnet must have a route to the AWS Secrets Manager VPC endpoint or to the AWS Secrets Manager endpoint through a NAT gateway.\n\n```ts\ndeclare const securityGroup: ec2.SecurityGroup;\ndeclare const subnet: ec2.Subnet;\ndeclare const db: rds.DatabaseCluster;\nnew glue.Connection(this, \"RdsConnection\", {\n type: glue.ConnectionType.JDBC,\n securityGroups: [securityGroup],\n subnet,\n properties: {\n JDBC_CONNECTION_URL: `jdbc:mysql://${db.clusterEndpoint.socketAddress}/databasename`,\n JDBC_ENFORCE_SSL: \"false\",\n SECRET_ID: db.secret!.secretName,\n },\n});\n```\n\nIf you need to use a connection type that doesn't exist as a static member on `ConnectionType`, you can instantiate a `ConnectionType` object, e.g: `new glue.ConnectionType('NEW_TYPE')`.\n\nSee [Adding a Connection to Your Data Store](https://docs.aws.amazon.com/glue/latest/dg/populate-add-connection.html) and [Connection Structure](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-catalog-connections.html#aws-glue-api-catalog-connections-Connection) documentation for more information on the supported data stores and their configurations.\n\n## SecurityConfiguration\n\nA `SecurityConfiguration` is a set of security properties that can be used by AWS Glue to encrypt data at rest.\n\n```ts\nnew glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n },\n jobBookmarksEncryption: {\n mode: glue.JobBookmarksEncryptionMode.CLIENT_SIDE_KMS,\n },\n s3Encryption: {\n mode: glue.S3EncryptionMode.KMS,\n },\n});\n```\n\nBy default, a shared KMS key is created for use with the encryption configurations that require one. You can also supply your own key for each encryption config, for example, for CloudWatch encryption:\n\n```ts\ndeclare const key: kms.Key;\nnew glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n kmsKey: key,\n },\n});\n```\n\nSee [documentation](https://docs.aws.amazon.com/glue/latest/dg/encryption-security-configuration.html) for more info for Glue encrypting data written by Crawlers, Jobs, and Development Endpoints.\n\n## Database\n\nA `Database` is a logical grouping of `Tables` in the Glue Catalog.\n\n```ts\nnew glue.Database(this, 'MyDatabase');\n```\n\n## Table\n\nA Glue table describes a table of data in S3: its structure (column names and types), location of data (S3 objects with a common prefix in a S3 bucket), and format for the files (Json, Avro, Parquet, etc.):\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }, {\n name: 'col2',\n type: glue.Schema.array(glue.Schema.STRING),\n comment: 'col2 is an array of strings' // comment is optional\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nBy default, a S3 bucket will be created to store the table's data but you can manually pass the `bucket` and `s3Prefix`:\n\n```ts\ndeclare const myBucket: s3.Bucket;\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n bucket: myBucket,\n s3Prefix: 'my-table/',\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nBy default, an S3 bucket will be created to store the table's data and stored in the bucket root. You can also manually pass the `bucket` and `s3Prefix`:\n\n### Partition Keys\n\nTo improve query performance, a table can specify `partitionKeys` on which data is stored and queried separately. For example, you might partition a table by `year` and `month` to optimize queries based on a time window:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n### Partition Indexes\n\nAnother way to improve query performance is to specify partition indexes. If no partition indexes are\npresent on the table, AWS Glue loads all partitions of the table and filters the loaded partitions using\nthe query expression. The query takes more time to run as the number of partitions increase. With an\nindex, the query will try to fetch a subset of the partitions instead of loading all partitions of the\ntable.\n\nThe keys of a partition index must be a subset of the partition keys of the table. You can have a\nmaximum of 3 partition indexes per table. To specify a partition index, you can use the `partitionIndexes`\nproperty:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n partitionIndexes: [{\n indexName: 'my-index', // optional\n keyNames: ['year'],\n }], // supply up to 3 indexes\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nAlternatively, you can call the `addPartitionIndex()` function on a table:\n\n```ts\ndeclare const myTable: glue.Table;\nmyTable.addPartitionIndex({\n indexName: 'my-index',\n keyNames: ['year'],\n});\n```\n\n### Partition Filtering\n\nIf you have a table with a large number of partitions that grows over time, consider using AWS Glue partition indexing and filtering.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n dataFormat: glue.DataFormat.JSON,\n enablePartitionFiltering: true,\n});\n```\n\n## [Encryption](https://docs.aws.amazon.com/athena/latest/ug/encryption.html)\n\nYou can enable encryption on a Table's data:\n\n* [S3Managed](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html) - (default) Server side encryption (`SSE-S3`) with an Amazon S3-managed key.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.S3_MANAGED,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [Kms](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html) - Server-side encryption (`SSE-KMS`) with an AWS KMS Key managed by the account owner.\n\n```ts\ndeclare const myDatabase: glue.Database;\n// KMS key is created automatically\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n\n// with an explicit KMS key\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS,\n encryptionKey: new kms.Key(this, 'MyKey'),\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [KmsManaged](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html) - Server-side encryption (`SSE-KMS`), like `Kms`, except with an AWS KMS Key managed by the AWS Key Management Service.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS_MANAGED,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [ClientSideKms](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html#client-side-encryption-kms-managed-master-key-intro) - Client-side encryption (`CSE-KMS`) with an AWS KMS Key managed by the account owner.\n\n```ts\ndeclare const myDatabase: glue.Database;\n// KMS key is created automatically\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.CLIENT_SIDE_KMS,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n\n// with an explicit KMS key\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.CLIENT_SIDE_KMS,\n encryptionKey: new kms.Key(this, 'MyKey'),\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n*Note: you cannot provide a `Bucket` when creating the `Table` if you wish to use server-side encryption (`KMS`, `KMS_MANAGED` or `S3_MANAGED`)*.\n\n## Types\n\nA table's schema is a collection of columns, each of which have a `name` and a `type`. Types are recursive structures, consisting of primitive and complex types:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n columns: [{\n name: 'primitive_column',\n type: glue.Schema.STRING,\n }, {\n name: 'array_column',\n type: glue.Schema.array(glue.Schema.INTEGER),\n comment: 'array<integer>',\n }, {\n name: 'map_column',\n type: glue.Schema.map(\n glue.Schema.STRING,\n glue.Schema.TIMESTAMP),\n comment: 'map<string,string>',\n }, {\n name: 'struct_column',\n type: glue.Schema.struct([{\n name: 'nested_column',\n type: glue.Schema.DATE,\n comment: 'nested comment',\n }]),\n comment: \"struct<nested_column:date COMMENT 'nested comment'>\",\n }],\n // ...\n database: myDatabase,\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n### Primitives\n\n#### Numeric\n\n| Name \t| Type \t| Comments |\n|-----------\t|----------\t|------------------------------------------------------------------------------------------------------------------\t|\n| FLOAT \t| Constant \t| A 32-bit single-precision floating point number |\n| INTEGER \t| Constant \t| A 32-bit signed value in two's complement format, with a minimum value of -2^31 and a maximum value of 2^31-1 \t|\n| DOUBLE \t| Constant \t| A 64-bit double-precision floating point number |\n| BIG_INT \t| Constant \t| A 64-bit signed INTEGER in two’s complement format, with a minimum value of -2^63 and a maximum value of 2^63 -1 |\n| SMALL_INT \t| Constant \t| A 16-bit signed INTEGER in two’s complement format, with a minimum value of -2^15 and a maximum value of 2^15-1 |\n| TINY_INT \t| Constant \t| A 8-bit signed INTEGER in two’s complement format, with a minimum value of -2^7 and a maximum value of 2^7-1 |\n\n#### Date and time\n\n| Name \t| Type \t| Comments \t|\n|-----------\t|----------\t|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------\t|\n| DATE \t| Constant \t| A date in UNIX format, such as YYYY-MM-DD. \t|\n| TIMESTAMP \t| Constant \t| Date and time instant in the UNiX format, such as yyyy-mm-dd hh:mm:ss[.f...]. For example, TIMESTAMP '2008-09-15 03:04:05.324'. This format uses the session time zone. \t|\n\n#### String\n\n| Name \t| Type \t| Comments \t|\n|--------------------------------------------\t|----------\t|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\t|\n| STRING \t| Constant \t| A string literal enclosed in single or double quotes \t|\n| decimal(precision: number, scale?: number) \t| Function \t| `precision` is the total number of digits. `scale` (optional) is the number of digits in fractional part with a default of 0. For example, use these type definitions: decimal(11,5), decimal(15) \t|\n| char(length: number) \t| Function \t| Fixed length character data, with a specified length between 1 and 255, such as char(10) \t|\n| varchar(length: number) \t| Function \t| Variable length character data, with a specified length between 1 and 65535, such as varchar(10) \t|\n\n#### Miscellaneous\n\n| Name \t| Type \t| Comments \t|\n|---------\t|----------\t|-------------------------------\t|\n| BOOLEAN \t| Constant \t| Values are `true` and `false` \t|\n| BINARY \t| Constant \t| Value is in binary \t|\n\n### Complex\n\n| Name \t| Type \t| Comments \t|\n|-------------------------------------\t|----------\t|-------------------------------------------------------------------\t|\n| array(itemType: Type) \t| Function \t| An array of some other type \t|\n| map(keyType: Type, valueType: Type) \t| Function \t| A map of some primitive key type to any value type \t|\n| struct(collumns: Column[]) \t| Function \t| Nested structure containing individually named and typed collumns \t|\n\n## Data Quality Ruleset\n\nA `DataQualityRuleset` specifies a data quality ruleset with DQDL rules applied to a specified AWS Glue table. For example, to create a data quality ruleset for a given table:\n\n```ts\nnew glue.DataQualityRuleset(this, 'MyDataQualityRuleset', {\n clientToken: 'client_token',\n description: 'description',\n rulesetName: 'ruleset_name',\n rulesetDqdl: 'ruleset_dqdl',\n tags: {\n key1: 'value1',\n key2: 'value2',\n },\n targetTable: new glue.DataQualityTargetTable('database_name', 'table_name'),\n});\n```\n\nFor more information, see [AWS Glue Data Quality](https://docs.aws.amazon.com/glue/latest/dg/glue-data-quality.html).\n"
3516
+ "markdown": "# AWS Glue Construct Library\n<!--BEGIN STABILITY BANNER-->\n\n---\n\n![cdk-constructs: Experimental](https://img.shields.io/badge/cdk--constructs-experimental-important.svg?style=for-the-badge)\n\n> The APIs of higher level constructs in this module are experimental and under active development.\n> They are subject to non-backward compatible changes or removal in any future version. These are\n> not subject to the [Semantic Versioning](https://semver.org/) model and breaking changes will be\n> announced in the release notes. This means that while you may use them, you may need to update\n> your source code when upgrading to a newer version of this package.\n\n---\n\n<!--END STABILITY BANNER-->\n\nThis module is part of the [AWS Cloud Development Kit](https://github.com/aws/aws-cdk) project.\n\n## Job\n\nA `Job` encapsulates a script that connects to data sources, processes them, and then writes output to a data target.\n\nThere are 3 types of jobs supported by AWS Glue: Spark ETL, Spark Streaming, and Python Shell jobs.\n\nThe `glue.JobExecutable` allows you to specify the type of job, the language to use and the code assets required by the job.\n\n`glue.Code` allows you to refer to the different code assets required by the job, either from an existing S3 location or from a local file path.\n\n`glue.ExecutionClass` allows you to specify `FLEX` or `STANDARD`. `FLEX` is appropriate for non-urgent jobs such as pre-production jobs, testing, and one-time data loads.\n\n### Spark Jobs\n\nThese jobs run in an Apache Spark environment managed by AWS Glue.\n\n#### ETL Jobs\n\nAn ETL job processes data in batches using Apache Spark.\n\n```ts\ndeclare const bucket: s3.Bucket;\nnew glue.Job(this, 'ScalaSparkEtlJob', {\n executable: glue.JobExecutable.scalaEtl({\n glueVersion: glue.GlueVersion.V4_0,\n script: glue.Code.fromBucket(bucket, 'src/com/example/HelloWorld.scala'),\n className: 'com.example.HelloWorld',\n extraJars: [glue.Code.fromBucket(bucket, 'jars/HelloWorld.jar')],\n }),\n workerType: glue.WorkerType.G_8X,\n description: 'an example Scala ETL job',\n});\n```\n\n#### Streaming Jobs\n\nA Streaming job is similar to an ETL job, except that it performs ETL on data streams. It uses the Apache Spark Structured Streaming framework. Some Spark job features are not available to streaming ETL jobs.\n\n```ts\nnew glue.Job(this, 'PythonSparkStreamingJob', {\n executable: glue.JobExecutable.pythonStreaming({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n description: 'an example Python Streaming job',\n});\n```\n\n### Python Shell Jobs\n\nA Python shell job runs Python scripts as a shell and supports a Python version that depends on the AWS Glue version you are using.\nThis can be used to schedule and run tasks that don't require an Apache Spark environment. Currently, three flavors are supported:\n\n* PythonVersion.TWO (2.7; EOL)\n* PythonVersion.THREE (3.6)\n* PythonVersion.THREE_NINE (3.9)\n\n```ts\ndeclare const bucket: s3.Bucket;\nnew glue.Job(this, 'PythonShellJob', {\n executable: glue.JobExecutable.pythonShell({\n glueVersion: glue.GlueVersion.V1_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromBucket(bucket, 'script.py'),\n }),\n description: 'an example Python Shell job',\n});\n```\n\n### Ray Jobs\n\nThese jobs run in a Ray environment managed by AWS Glue.\n\n```ts\nnew glue.Job(this, 'RayJob', {\n executable: glue.JobExecutable.pythonRay({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE_NINE,\n runtime: glue.Runtime.RAY_TWO_FOUR,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n workerType: glue.WorkerType.Z_2X,\n workerCount: 2,\n description: 'an example Ray job'\n});\n```\n\n### Enable Spark UI\n\nEnable Spark UI setting the `sparkUI` property.\n\n```ts\nnew glue.Job(this, 'EnableSparkUI', {\n jobName: 'EtlJobWithSparkUIPrefix',\n sparkUI: {\n enabled: true,\n },\n executable: glue.JobExecutable.pythonEtl({\n glueVersion: glue.GlueVersion.V3_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n});\n```\n\nThe `sparkUI` property also allows the specification of an s3 bucket and a bucket prefix.\n\nSee [documentation](https://docs.aws.amazon.com/glue/latest/dg/add-job.html) for more information on adding jobs in Glue.\n\n## Connection\n\nA `Connection` allows Glue jobs, crawlers and development endpoints to access certain types of data stores. For example, to create a network connection to connect to a data source within a VPC:\n\n```ts\ndeclare const securityGroup: ec2.SecurityGroup;\ndeclare const subnet: ec2.Subnet;\nnew glue.Connection(this, 'MyConnection', {\n type: glue.ConnectionType.NETWORK,\n // The security groups granting AWS Glue inbound access to the data source within the VPC\n securityGroups: [securityGroup],\n // The VPC subnet which contains the data source\n subnet,\n});\n```\n\nFor RDS `Connection` by JDBC, it is recommended to manage credentials using AWS Secrets Manager. To use Secret, specify `SECRET_ID` in `properties` like the following code. Note that in this case, the subnet must have a route to the AWS Secrets Manager VPC endpoint or to the AWS Secrets Manager endpoint through a NAT gateway.\n\n```ts\ndeclare const securityGroup: ec2.SecurityGroup;\ndeclare const subnet: ec2.Subnet;\ndeclare const db: rds.DatabaseCluster;\nnew glue.Connection(this, \"RdsConnection\", {\n type: glue.ConnectionType.JDBC,\n securityGroups: [securityGroup],\n subnet,\n properties: {\n JDBC_CONNECTION_URL: `jdbc:mysql://${db.clusterEndpoint.socketAddress}/databasename`,\n JDBC_ENFORCE_SSL: \"false\",\n SECRET_ID: db.secret!.secretName,\n },\n});\n```\n\nIf you need to use a connection type that doesn't exist as a static member on `ConnectionType`, you can instantiate a `ConnectionType` object, e.g: `new glue.ConnectionType('NEW_TYPE')`.\n\nSee [Adding a Connection to Your Data Store](https://docs.aws.amazon.com/glue/latest/dg/populate-add-connection.html) and [Connection Structure](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-catalog-connections.html#aws-glue-api-catalog-connections-Connection) documentation for more information on the supported data stores and their configurations.\n\n## SecurityConfiguration\n\nA `SecurityConfiguration` is a set of security properties that can be used by AWS Glue to encrypt data at rest.\n\n```ts\nnew glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n },\n jobBookmarksEncryption: {\n mode: glue.JobBookmarksEncryptionMode.CLIENT_SIDE_KMS,\n },\n s3Encryption: {\n mode: glue.S3EncryptionMode.KMS,\n },\n});\n```\n\nBy default, a shared KMS key is created for use with the encryption configurations that require one. You can also supply your own key for each encryption config, for example, for CloudWatch encryption:\n\n```ts\ndeclare const key: kms.Key;\nnew glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n kmsKey: key,\n },\n});\n```\n\nSee [documentation](https://docs.aws.amazon.com/glue/latest/dg/encryption-security-configuration.html) for more info for Glue encrypting data written by Crawlers, Jobs, and Development Endpoints.\n\n## Database\n\nA `Database` is a logical grouping of `Tables` in the Glue Catalog.\n\n```ts\nnew glue.Database(this, 'MyDatabase');\n```\n\n## Table\n\nA Glue table describes a table of data in S3: its structure (column names and types), location of data (S3 objects with a common prefix in a S3 bucket), and format for the files (Json, Avro, Parquet, etc.):\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }, {\n name: 'col2',\n type: glue.Schema.array(glue.Schema.STRING),\n comment: 'col2 is an array of strings' // comment is optional\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nBy default, a S3 bucket will be created to store the table's data but you can manually pass the `bucket` and `s3Prefix`:\n\n```ts\ndeclare const myBucket: s3.Bucket;\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n bucket: myBucket,\n s3Prefix: 'my-table/',\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nGlue tables can be configured to contain user-defined properties, to describe the physical storage of table data, through the `storageParameters` property:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n storageParameters: [\n glue.StorageParameter.skipHeaderLineCount(1),\n glue.StorageParameter.compressionType(glue.CompressionType.GZIP),\n glue.StorageParameter.custom('separatorChar', ',')\n ],\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n### Partition Keys\n\nTo improve query performance, a table can specify `partitionKeys` on which data is stored and queried separately. For example, you might partition a table by `year` and `month` to optimize queries based on a time window:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n### Partition Indexes\n\nAnother way to improve query performance is to specify partition indexes. If no partition indexes are\npresent on the table, AWS Glue loads all partitions of the table and filters the loaded partitions using\nthe query expression. The query takes more time to run as the number of partitions increase. With an\nindex, the query will try to fetch a subset of the partitions instead of loading all partitions of the\ntable.\n\nThe keys of a partition index must be a subset of the partition keys of the table. You can have a\nmaximum of 3 partition indexes per table. To specify a partition index, you can use the `partitionIndexes`\nproperty:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n partitionIndexes: [{\n indexName: 'my-index', // optional\n keyNames: ['year'],\n }], // supply up to 3 indexes\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nAlternatively, you can call the `addPartitionIndex()` function on a table:\n\n```ts\ndeclare const myTable: glue.Table;\nmyTable.addPartitionIndex({\n indexName: 'my-index',\n keyNames: ['year'],\n});\n```\n\n### Partition Filtering\n\nIf you have a table with a large number of partitions that grows over time, consider using AWS Glue partition indexing and filtering.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n dataFormat: glue.DataFormat.JSON,\n enablePartitionFiltering: true,\n});\n```\n\n## [Encryption](https://docs.aws.amazon.com/athena/latest/ug/encryption.html)\n\nYou can enable encryption on a Table's data:\n\n* [S3Managed](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html) - (default) Server side encryption (`SSE-S3`) with an Amazon S3-managed key.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.S3_MANAGED,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [Kms](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html) - Server-side encryption (`SSE-KMS`) with an AWS KMS Key managed by the account owner.\n\n```ts\ndeclare const myDatabase: glue.Database;\n// KMS key is created automatically\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n\n// with an explicit KMS key\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS,\n encryptionKey: new kms.Key(this, 'MyKey'),\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [KmsManaged](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html) - Server-side encryption (`SSE-KMS`), like `Kms`, except with an AWS KMS Key managed by the AWS Key Management Service.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS_MANAGED,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [ClientSideKms](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html#client-side-encryption-kms-managed-master-key-intro) - Client-side encryption (`CSE-KMS`) with an AWS KMS Key managed by the account owner.\n\n```ts\ndeclare const myDatabase: glue.Database;\n// KMS key is created automatically\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.CLIENT_SIDE_KMS,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n\n// with an explicit KMS key\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.CLIENT_SIDE_KMS,\n encryptionKey: new kms.Key(this, 'MyKey'),\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n*Note: you cannot provide a `Bucket` when creating the `Table` if you wish to use server-side encryption (`KMS`, `KMS_MANAGED` or `S3_MANAGED`)*.\n\n## Types\n\nA table's schema is a collection of columns, each of which have a `name` and a `type`. Types are recursive structures, consisting of primitive and complex types:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n columns: [{\n name: 'primitive_column',\n type: glue.Schema.STRING,\n }, {\n name: 'array_column',\n type: glue.Schema.array(glue.Schema.INTEGER),\n comment: 'array<integer>',\n }, {\n name: 'map_column',\n type: glue.Schema.map(\n glue.Schema.STRING,\n glue.Schema.TIMESTAMP),\n comment: 'map<string,string>',\n }, {\n name: 'struct_column',\n type: glue.Schema.struct([{\n name: 'nested_column',\n type: glue.Schema.DATE,\n comment: 'nested comment',\n }]),\n comment: \"struct<nested_column:date COMMENT 'nested comment'>\",\n }],\n // ...\n database: myDatabase,\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n### Primitives\n\n#### Numeric\n\n| Name \t| Type \t| Comments |\n|-----------\t|----------\t|------------------------------------------------------------------------------------------------------------------\t|\n| FLOAT \t| Constant \t| A 32-bit single-precision floating point number |\n| INTEGER \t| Constant \t| A 32-bit signed value in two's complement format, with a minimum value of -2^31 and a maximum value of 2^31-1 \t|\n| DOUBLE \t| Constant \t| A 64-bit double-precision floating point number |\n| BIG_INT \t| Constant \t| A 64-bit signed INTEGER in two’s complement format, with a minimum value of -2^63 and a maximum value of 2^63 -1 |\n| SMALL_INT \t| Constant \t| A 16-bit signed INTEGER in two’s complement format, with a minimum value of -2^15 and a maximum value of 2^15-1 |\n| TINY_INT \t| Constant \t| A 8-bit signed INTEGER in two’s complement format, with a minimum value of -2^7 and a maximum value of 2^7-1 |\n\n#### Date and time\n\n| Name \t| Type \t| Comments \t|\n|-----------\t|----------\t|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------\t|\n| DATE \t| Constant \t| A date in UNIX format, such as YYYY-MM-DD. \t|\n| TIMESTAMP \t| Constant \t| Date and time instant in the UNiX format, such as yyyy-mm-dd hh:mm:ss[.f...]. For example, TIMESTAMP '2008-09-15 03:04:05.324'. This format uses the session time zone. \t|\n\n#### String\n\n| Name \t| Type \t| Comments \t|\n|--------------------------------------------\t|----------\t|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\t|\n| STRING \t| Constant \t| A string literal enclosed in single or double quotes \t|\n| decimal(precision: number, scale?: number) \t| Function \t| `precision` is the total number of digits. `scale` (optional) is the number of digits in fractional part with a default of 0. For example, use these type definitions: decimal(11,5), decimal(15) \t|\n| char(length: number) \t| Function \t| Fixed length character data, with a specified length between 1 and 255, such as char(10) \t|\n| varchar(length: number) \t| Function \t| Variable length character data, with a specified length between 1 and 65535, such as varchar(10) \t|\n\n#### Miscellaneous\n\n| Name \t| Type \t| Comments \t|\n|---------\t|----------\t|-------------------------------\t|\n| BOOLEAN \t| Constant \t| Values are `true` and `false` \t|\n| BINARY \t| Constant \t| Value is in binary \t|\n\n### Complex\n\n| Name \t| Type \t| Comments \t|\n|-------------------------------------\t|----------\t|-------------------------------------------------------------------\t|\n| array(itemType: Type) \t| Function \t| An array of some other type \t|\n| map(keyType: Type, valueType: Type) \t| Function \t| A map of some primitive key type to any value type \t|\n| struct(collumns: Column[]) \t| Function \t| Nested structure containing individually named and typed collumns \t|\n\n## Data Quality Ruleset\n\nA `DataQualityRuleset` specifies a data quality ruleset with DQDL rules applied to a specified AWS Glue table. For example, to create a data quality ruleset for a given table:\n\n```ts\nnew glue.DataQualityRuleset(this, 'MyDataQualityRuleset', {\n clientToken: 'client_token',\n description: 'description',\n rulesetName: 'ruleset_name',\n rulesetDqdl: 'ruleset_dqdl',\n tags: {\n key1: 'value1',\n key2: 'value2',\n },\n targetTable: new glue.DataQualityTargetTable('database_name', 'table_name'),\n});\n```\n\nFor more information, see [AWS Glue Data Quality](https://docs.aws.amazon.com/glue/latest/dg/glue-data-quality.html).\n"
3517
3517
  },
3518
3518
  "repository": {
3519
3519
  "directory": "packages/@aws-cdk/aws-glue-alpha",
@@ -3878,7 +3878,7 @@
3878
3878
  "docs": {
3879
3879
  "stability": "experimental",
3880
3880
  "summary": "Represents a Glue Job's Code assets (an asset can be a scripts, a jar, a python file or any other file).",
3881
- "example": "declare const bucket: s3.Bucket;\nnew glue.Job(this, 'ScalaSparkEtlJob', {\n executable: glue.JobExecutable.scalaEtl({\n glueVersion: glue.GlueVersion.V4_0,\n script: glue.Code.fromBucket(bucket, 'src/com/example/HelloWorld.scala'),\n className: 'com.example.HelloWorld',\n extraJars: [glue.Code.fromBucket(bucket, 'jars/HelloWorld.jar')],\n }),\n workerType: glue.WorkerType.G_8X,\n description: 'an example Scala ETL job',\n});",
3881
+ "example": "new glue.Job(this, 'EnableSparkUI', {\n jobName: 'EtlJobWithSparkUIPrefix',\n sparkUI: {\n enabled: true,\n },\n executable: glue.JobExecutable.pythonEtl({\n glueVersion: glue.GlueVersion.V3_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n});",
3882
3882
  "custom": {
3883
3883
  "exampleMetadata": "infused"
3884
3884
  }
@@ -4112,6 +4112,103 @@
4112
4112
  ],
4113
4113
  "symbolId": "lib/schema:Column"
4114
4114
  },
4115
+ "@aws-cdk/aws-glue-alpha.ColumnCountMismatchHandlingAction": {
4116
+ "assembly": "@aws-cdk/aws-glue-alpha",
4117
+ "docs": {
4118
+ "remarks": "This property is only available for an uncompressed text file format.",
4119
+ "see": "https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_EXTERNAL_TABLE.html#r_CREATE_EXTERNAL_TABLE-parameters - under _\"TABLE PROPERTIES\"_ > _\"column_count_mismatch_handling\"_",
4120
+ "stability": "experimental",
4121
+ "summary": "Identifies if the file contains less or more values for a row than the number of columns specified in the external table definition."
4122
+ },
4123
+ "fqn": "@aws-cdk/aws-glue-alpha.ColumnCountMismatchHandlingAction",
4124
+ "kind": "enum",
4125
+ "locationInModule": {
4126
+ "filename": "lib/storage-parameter.ts",
4127
+ "line": 156
4128
+ },
4129
+ "members": [
4130
+ {
4131
+ "docs": {
4132
+ "stability": "experimental",
4133
+ "summary": "Column count mismatch handling is turned off."
4134
+ },
4135
+ "name": "DISABLED"
4136
+ },
4137
+ {
4138
+ "docs": {
4139
+ "stability": "experimental",
4140
+ "summary": "Fail the query if the column count mismatch is detected."
4141
+ },
4142
+ "name": "FAIL"
4143
+ },
4144
+ {
4145
+ "docs": {
4146
+ "stability": "experimental",
4147
+ "summary": "Fill missing values with NULL and ignore the additional values in each row."
4148
+ },
4149
+ "name": "SET_TO_NULL"
4150
+ },
4151
+ {
4152
+ "docs": {
4153
+ "stability": "experimental",
4154
+ "summary": "Drop all rows that contain column count mismatch error from the scan."
4155
+ },
4156
+ "name": "DROP_ROW"
4157
+ }
4158
+ ],
4159
+ "name": "ColumnCountMismatchHandlingAction",
4160
+ "symbolId": "lib/storage-parameter:ColumnCountMismatchHandlingAction"
4161
+ },
4162
+ "@aws-cdk/aws-glue-alpha.CompressionType": {
4163
+ "assembly": "@aws-cdk/aws-glue-alpha",
4164
+ "docs": {
4165
+ "see": "https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_EXTERNAL_TABLE.html#r_CREATE_EXTERNAL_TABLE-parameters - under _\"TABLE PROPERTIES\"_ > _\"compression_type\"_",
4166
+ "stability": "experimental",
4167
+ "summary": "The compression type.",
4168
+ "example": "declare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n storageParameters: [\n glue.StorageParameter.skipHeaderLineCount(1),\n glue.StorageParameter.compressionType(glue.CompressionType.GZIP),\n glue.StorageParameter.custom('separatorChar', ',')\n ],\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});",
4169
+ "custom": {
4170
+ "exampleMetadata": "infused"
4171
+ }
4172
+ },
4173
+ "fqn": "@aws-cdk/aws-glue-alpha.CompressionType",
4174
+ "kind": "enum",
4175
+ "locationInModule": {
4176
+ "filename": "lib/storage-parameter.ts",
4177
+ "line": 6
4178
+ },
4179
+ "members": [
4180
+ {
4181
+ "docs": {
4182
+ "stability": "experimental",
4183
+ "summary": "No compression."
4184
+ },
4185
+ "name": "NONE"
4186
+ },
4187
+ {
4188
+ "docs": {
4189
+ "stability": "experimental",
4190
+ "summary": "Burrows-Wheeler compression."
4191
+ },
4192
+ "name": "BZIP2"
4193
+ },
4194
+ {
4195
+ "docs": {
4196
+ "stability": "experimental",
4197
+ "summary": "Deflate compression."
4198
+ },
4199
+ "name": "GZIP"
4200
+ },
4201
+ {
4202
+ "docs": {
4203
+ "stability": "experimental",
4204
+ "summary": "Compression algorithm focused on high compression and decompression speeds, rather than the maximum possible compression."
4205
+ },
4206
+ "name": "SNAPPY"
4207
+ }
4208
+ ],
4209
+ "name": "CompressionType",
4210
+ "symbolId": "lib/storage-parameter:CompressionType"
4211
+ },
4115
4212
  "@aws-cdk/aws-glue-alpha.Connection": {
4116
4213
  "assembly": "@aws-cdk/aws-glue-alpha",
4117
4214
  "base": "aws-cdk-lib.Resource",
@@ -4668,7 +4765,7 @@
4668
4765
  "kind": "interface",
4669
4766
  "locationInModule": {
4670
4767
  "filename": "lib/job.ts",
4671
- "line": 424
4768
+ "line": 426
4672
4769
  },
4673
4770
  "name": "ContinuousLoggingProps",
4674
4771
  "properties": [
@@ -4681,7 +4778,7 @@
4681
4778
  "immutable": true,
4682
4779
  "locationInModule": {
4683
4780
  "filename": "lib/job.ts",
4684
- "line": 428
4781
+ "line": 430
4685
4782
  },
4686
4783
  "name": "enabled",
4687
4784
  "type": {
@@ -4699,7 +4796,7 @@
4699
4796
  "immutable": true,
4700
4797
  "locationInModule": {
4701
4798
  "filename": "lib/job.ts",
4702
- "line": 458
4799
+ "line": 460
4703
4800
  },
4704
4801
  "name": "conversionPattern",
4705
4802
  "optional": true,
@@ -4717,7 +4814,7 @@
4717
4814
  "immutable": true,
4718
4815
  "locationInModule": {
4719
4816
  "filename": "lib/job.ts",
4720
- "line": 435
4817
+ "line": 437
4721
4818
  },
4722
4819
  "name": "logGroup",
4723
4820
  "optional": true,
@@ -4735,7 +4832,7 @@
4735
4832
  "immutable": true,
4736
4833
  "locationInModule": {
4737
4834
  "filename": "lib/job.ts",
4738
- "line": 442
4835
+ "line": 444
4739
4836
  },
4740
4837
  "name": "logStreamPrefix",
4741
4838
  "optional": true,
@@ -4753,7 +4850,7 @@
4753
4850
  "immutable": true,
4754
4851
  "locationInModule": {
4755
4852
  "filename": "lib/job.ts",
4756
- "line": 449
4853
+ "line": 451
4757
4854
  },
4758
4855
  "name": "quiet",
4759
4856
  "optional": true,
@@ -5732,7 +5829,7 @@
5732
5829
  "kind": "enum",
5733
5830
  "locationInModule": {
5734
5831
  "filename": "lib/job.ts",
5735
- "line": 138
5832
+ "line": 139
5736
5833
  },
5737
5834
  "members": [
5738
5835
  {
@@ -5759,7 +5856,7 @@
5759
5856
  "see": "https://docs.aws.amazon.com/glue/latest/dg/add-job.html.\n\nIf you need to use a GlueVersion that doesn't exist as a static member, you\ncan instantiate a `GlueVersion` object, e.g: `GlueVersion.of('1.5')`.",
5760
5857
  "stability": "experimental",
5761
5858
  "summary": "AWS Glue version determines the versions of Apache Spark and Python that are available to the job.",
5762
- "example": "declare const bucket: s3.Bucket;\nnew glue.Job(this, 'ScalaSparkEtlJob', {\n executable: glue.JobExecutable.scalaEtl({\n glueVersion: glue.GlueVersion.V4_0,\n script: glue.Code.fromBucket(bucket, 'src/com/example/HelloWorld.scala'),\n className: 'com.example.HelloWorld',\n extraJars: [glue.Code.fromBucket(bucket, 'jars/HelloWorld.jar')],\n }),\n workerType: glue.WorkerType.G_8X,\n description: 'an example Scala ETL job',\n});",
5859
+ "example": "new glue.Job(this, 'EnableSparkUI', {\n jobName: 'EtlJobWithSparkUIPrefix',\n sparkUI: {\n enabled: true,\n },\n executable: glue.JobExecutable.pythonEtl({\n glueVersion: glue.GlueVersion.V3_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n});",
5763
5860
  "custom": {
5764
5861
  "exampleMetadata": "infused"
5765
5862
  }
@@ -6123,7 +6220,7 @@
6123
6220
  "kind": "interface",
6124
6221
  "locationInModule": {
6125
6222
  "filename": "lib/job.ts",
6126
- "line": 155
6223
+ "line": 156
6127
6224
  },
6128
6225
  "methods": [
6129
6226
  {
@@ -6135,7 +6232,7 @@
6135
6232
  },
6136
6233
  "locationInModule": {
6137
6234
  "filename": "lib/job.ts",
6138
- "line": 212
6235
+ "line": 213
6139
6236
  },
6140
6237
  "name": "metric",
6141
6238
  "parameters": [
@@ -6182,7 +6279,7 @@
6182
6279
  },
6183
6280
  "locationInModule": {
6184
6281
  "filename": "lib/job.ts",
6185
- "line": 222
6282
+ "line": 223
6186
6283
  },
6187
6284
  "name": "metricFailure",
6188
6285
  "parameters": [
@@ -6208,7 +6305,7 @@
6208
6305
  },
6209
6306
  "locationInModule": {
6210
6307
  "filename": "lib/job.ts",
6211
- "line": 217
6308
+ "line": 218
6212
6309
  },
6213
6310
  "name": "metricSuccess",
6214
6311
  "parameters": [
@@ -6234,7 +6331,7 @@
6234
6331
  },
6235
6332
  "locationInModule": {
6236
6333
  "filename": "lib/job.ts",
6237
- "line": 227
6334
+ "line": 228
6238
6335
  },
6239
6336
  "name": "metricTimeout",
6240
6337
  "parameters": [
@@ -6261,7 +6358,7 @@
6261
6358
  },
6262
6359
  "locationInModule": {
6263
6360
  "filename": "lib/job.ts",
6264
- "line": 173
6361
+ "line": 174
6265
6362
  },
6266
6363
  "name": "onEvent",
6267
6364
  "parameters": [
@@ -6294,7 +6391,7 @@
6294
6391
  },
6295
6392
  "locationInModule": {
6296
6393
  "filename": "lib/job.ts",
6297
- "line": 194
6394
+ "line": 195
6298
6395
  },
6299
6396
  "name": "onFailure",
6300
6397
  "parameters": [
@@ -6327,7 +6424,7 @@
6327
6424
  },
6328
6425
  "locationInModule": {
6329
6426
  "filename": "lib/job.ts",
6330
- "line": 180
6427
+ "line": 181
6331
6428
  },
6332
6429
  "name": "onStateChange",
6333
6430
  "parameters": [
@@ -6366,7 +6463,7 @@
6366
6463
  },
6367
6464
  "locationInModule": {
6368
6465
  "filename": "lib/job.ts",
6369
- "line": 187
6466
+ "line": 188
6370
6467
  },
6371
6468
  "name": "onSuccess",
6372
6469
  "parameters": [
@@ -6399,7 +6496,7 @@
6399
6496
  },
6400
6497
  "locationInModule": {
6401
6498
  "filename": "lib/job.ts",
6402
- "line": 201
6499
+ "line": 202
6403
6500
  },
6404
6501
  "name": "onTimeout",
6405
6502
  "parameters": [
@@ -6438,7 +6535,7 @@
6438
6535
  "immutable": true,
6439
6536
  "locationInModule": {
6440
6537
  "filename": "lib/job.ts",
6441
- "line": 166
6538
+ "line": 167
6442
6539
  },
6443
6540
  "name": "jobArn",
6444
6541
  "type": {
@@ -6457,7 +6554,7 @@
6457
6554
  "immutable": true,
6458
6555
  "locationInModule": {
6459
6556
  "filename": "lib/job.ts",
6460
- "line": 160
6557
+ "line": 161
6461
6558
  },
6462
6559
  "name": "jobName",
6463
6560
  "type": {
@@ -6518,7 +6615,7 @@
6518
6615
  "kind": "interface",
6519
6616
  "locationInModule": {
6520
6617
  "filename": "lib/table.ts",
6521
- "line": 31
6618
+ "line": 32
6522
6619
  },
6523
6620
  "name": "ITable",
6524
6621
  "properties": [
@@ -6533,7 +6630,7 @@
6533
6630
  "immutable": true,
6534
6631
  "locationInModule": {
6535
6632
  "filename": "lib/table.ts",
6536
- "line": 35
6633
+ "line": 36
6537
6634
  },
6538
6635
  "name": "tableArn",
6539
6636
  "type": {
@@ -6551,7 +6648,7 @@
6551
6648
  "immutable": true,
6552
6649
  "locationInModule": {
6553
6650
  "filename": "lib/table.ts",
6554
- "line": 40
6651
+ "line": 41
6555
6652
  },
6556
6653
  "name": "tableName",
6557
6654
  "type": {
@@ -6704,13 +6801,66 @@
6704
6801
  ],
6705
6802
  "symbolId": "lib/data-format:InputFormat"
6706
6803
  },
6804
+ "@aws-cdk/aws-glue-alpha.InvalidCharHandlingAction": {
6805
+ "assembly": "@aws-cdk/aws-glue-alpha",
6806
+ "docs": {
6807
+ "see": "https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_EXTERNAL_TABLE.html#r_CREATE_EXTERNAL_TABLE-parameters - under _\"TABLE PROPERTIES\"_ > _\"invalid_char_handling\"_",
6808
+ "stability": "experimental",
6809
+ "summary": "Specifies the action to perform when query results contain invalid UTF-8 character values."
6810
+ },
6811
+ "fqn": "@aws-cdk/aws-glue-alpha.InvalidCharHandlingAction",
6812
+ "kind": "enum",
6813
+ "locationInModule": {
6814
+ "filename": "lib/storage-parameter.ts",
6815
+ "line": 33
6816
+ },
6817
+ "members": [
6818
+ {
6819
+ "docs": {
6820
+ "stability": "experimental",
6821
+ "summary": "Doesn't perform invalid character handling."
6822
+ },
6823
+ "name": "DISABLED"
6824
+ },
6825
+ {
6826
+ "docs": {
6827
+ "stability": "experimental",
6828
+ "summary": "Cancels queries that return data containing invalid UTF-8 values."
6829
+ },
6830
+ "name": "FAIL"
6831
+ },
6832
+ {
6833
+ "docs": {
6834
+ "stability": "experimental",
6835
+ "summary": "Replaces invalid UTF-8 values with null."
6836
+ },
6837
+ "name": "SET_TO_NULL"
6838
+ },
6839
+ {
6840
+ "docs": {
6841
+ "stability": "experimental",
6842
+ "summary": "Replaces each value in the row with null."
6843
+ },
6844
+ "name": "DROP_ROW"
6845
+ },
6846
+ {
6847
+ "docs": {
6848
+ "stability": "experimental",
6849
+ "summary": "Replaces the invalid character with the replacement character you specify using `REPLACEMENT_CHAR`."
6850
+ },
6851
+ "name": "REPLACE"
6852
+ }
6853
+ ],
6854
+ "name": "InvalidCharHandlingAction",
6855
+ "symbolId": "lib/storage-parameter:InvalidCharHandlingAction"
6856
+ },
6707
6857
  "@aws-cdk/aws-glue-alpha.Job": {
6708
6858
  "assembly": "@aws-cdk/aws-glue-alpha",
6709
6859
  "base": "aws-cdk-lib.Resource",
6710
6860
  "docs": {
6711
6861
  "stability": "experimental",
6712
6862
  "summary": "A Glue Job.",
6713
- "example": "declare const bucket: s3.Bucket;\nnew glue.Job(this, 'ScalaSparkEtlJob', {\n executable: glue.JobExecutable.scalaEtl({\n glueVersion: glue.GlueVersion.V4_0,\n script: glue.Code.fromBucket(bucket, 'src/com/example/HelloWorld.scala'),\n className: 'com.example.HelloWorld',\n extraJars: [glue.Code.fromBucket(bucket, 'jars/HelloWorld.jar')],\n }),\n workerType: glue.WorkerType.G_8X,\n description: 'an example Scala ETL job',\n});",
6863
+ "example": "new glue.Job(this, 'EnableSparkUI', {\n jobName: 'EtlJobWithSparkUIPrefix',\n sparkUI: {\n enabled: true,\n },\n executable: glue.JobExecutable.pythonEtl({\n glueVersion: glue.GlueVersion.V3_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n});",
6714
6864
  "custom": {
6715
6865
  "exampleMetadata": "infused"
6716
6866
  }
@@ -6722,7 +6872,7 @@
6722
6872
  },
6723
6873
  "locationInModule": {
6724
6874
  "filename": "lib/job.ts",
6725
- "line": 684
6875
+ "line": 686
6726
6876
  },
6727
6877
  "parameters": [
6728
6878
  {
@@ -6751,7 +6901,7 @@
6751
6901
  "kind": "class",
6752
6902
  "locationInModule": {
6753
6903
  "filename": "lib/job.ts",
6754
- "line": 638
6904
+ "line": 640
6755
6905
  },
6756
6906
  "methods": [
6757
6907
  {
@@ -6761,7 +6911,7 @@
6761
6911
  },
6762
6912
  "locationInModule": {
6763
6913
  "filename": "lib/job.ts",
6764
- "line": 646
6914
+ "line": 648
6765
6915
  },
6766
6916
  "name": "fromJobAttributes",
6767
6917
  "parameters": [
@@ -6808,7 +6958,7 @@
6808
6958
  },
6809
6959
  "locationInModule": {
6810
6960
  "filename": "lib/job.ts",
6811
- "line": 319
6961
+ "line": 320
6812
6962
  },
6813
6963
  "name": "metric",
6814
6964
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -6856,7 +7006,7 @@
6856
7006
  },
6857
7007
  "locationInModule": {
6858
7008
  "filename": "lib/job.ts",
6859
- "line": 346
7009
+ "line": 347
6860
7010
  },
6861
7011
  "name": "metricFailure",
6862
7012
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -6883,7 +7033,7 @@
6883
7033
  },
6884
7034
  "locationInModule": {
6885
7035
  "filename": "lib/job.ts",
6886
- "line": 337
7036
+ "line": 338
6887
7037
  },
6888
7038
  "name": "metricSuccess",
6889
7039
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -6910,7 +7060,7 @@
6910
7060
  },
6911
7061
  "locationInModule": {
6912
7062
  "filename": "lib/job.ts",
6913
- "line": 355
7063
+ "line": 356
6914
7064
  },
6915
7065
  "name": "metricTimeout",
6916
7066
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -6937,7 +7087,7 @@
6937
7087
  },
6938
7088
  "locationInModule": {
6939
7089
  "filename": "lib/job.ts",
6940
- "line": 247
7090
+ "line": 248
6941
7091
  },
6942
7092
  "name": "onEvent",
6943
7093
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -6976,7 +7126,7 @@
6976
7126
  },
6977
7127
  "locationInModule": {
6978
7128
  "filename": "lib/job.ts",
6979
- "line": 296
7129
+ "line": 297
6980
7130
  },
6981
7131
  "name": "onFailure",
6982
7132
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -7015,7 +7165,7 @@
7015
7165
  },
7016
7166
  "locationInModule": {
7017
7167
  "filename": "lib/job.ts",
7018
- "line": 267
7168
+ "line": 268
7019
7169
  },
7020
7170
  "name": "onStateChange",
7021
7171
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -7062,7 +7212,7 @@
7062
7212
  },
7063
7213
  "locationInModule": {
7064
7214
  "filename": "lib/job.ts",
7065
- "line": 286
7215
+ "line": 287
7066
7216
  },
7067
7217
  "name": "onSuccess",
7068
7218
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -7101,7 +7251,7 @@
7101
7251
  },
7102
7252
  "locationInModule": {
7103
7253
  "filename": "lib/job.ts",
7104
- "line": 306
7254
+ "line": 307
7105
7255
  },
7106
7256
  "name": "onTimeout",
7107
7257
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -7144,7 +7294,7 @@
7144
7294
  "immutable": true,
7145
7295
  "locationInModule": {
7146
7296
  "filename": "lib/job.ts",
7147
- "line": 674
7297
+ "line": 676
7148
7298
  },
7149
7299
  "name": "grantPrincipal",
7150
7300
  "overrides": "aws-cdk-lib.aws_iam.IGrantable",
@@ -7160,7 +7310,7 @@
7160
7310
  "immutable": true,
7161
7311
  "locationInModule": {
7162
7312
  "filename": "lib/job.ts",
7163
- "line": 659
7313
+ "line": 661
7164
7314
  },
7165
7315
  "name": "jobArn",
7166
7316
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -7176,7 +7326,7 @@
7176
7326
  "immutable": true,
7177
7327
  "locationInModule": {
7178
7328
  "filename": "lib/job.ts",
7179
- "line": 664
7329
+ "line": 666
7180
7330
  },
7181
7331
  "name": "jobName",
7182
7332
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -7192,7 +7342,7 @@
7192
7342
  "immutable": true,
7193
7343
  "locationInModule": {
7194
7344
  "filename": "lib/job.ts",
7195
- "line": 669
7345
+ "line": 671
7196
7346
  },
7197
7347
  "name": "role",
7198
7348
  "type": {
@@ -7208,7 +7358,7 @@
7208
7358
  "immutable": true,
7209
7359
  "locationInModule": {
7210
7360
  "filename": "lib/job.ts",
7211
- "line": 682
7361
+ "line": 684
7212
7362
  },
7213
7363
  "name": "sparkUILoggingLocation",
7214
7364
  "optional": true,
@@ -7234,7 +7384,7 @@
7234
7384
  "kind": "interface",
7235
7385
  "locationInModule": {
7236
7386
  "filename": "lib/job.ts",
7237
- "line": 464
7387
+ "line": 466
7238
7388
  },
7239
7389
  "name": "JobAttributes",
7240
7390
  "properties": [
@@ -7247,7 +7397,7 @@
7247
7397
  "immutable": true,
7248
7398
  "locationInModule": {
7249
7399
  "filename": "lib/job.ts",
7250
- "line": 468
7400
+ "line": 470
7251
7401
  },
7252
7402
  "name": "jobName",
7253
7403
  "type": {
@@ -7264,7 +7414,7 @@
7264
7414
  "immutable": true,
7265
7415
  "locationInModule": {
7266
7416
  "filename": "lib/job.ts",
7267
- "line": 475
7417
+ "line": 477
7268
7418
  },
7269
7419
  "name": "role",
7270
7420
  "optional": true,
@@ -7366,7 +7516,7 @@
7366
7516
  "docs": {
7367
7517
  "stability": "experimental",
7368
7518
  "summary": "The executable properties related to the Glue job's GlueVersion, JobType and code.",
7369
- "example": "declare const bucket: s3.Bucket;\nnew glue.Job(this, 'ScalaSparkEtlJob', {\n executable: glue.JobExecutable.scalaEtl({\n glueVersion: glue.GlueVersion.V4_0,\n script: glue.Code.fromBucket(bucket, 'src/com/example/HelloWorld.scala'),\n className: 'com.example.HelloWorld',\n extraJars: [glue.Code.fromBucket(bucket, 'jars/HelloWorld.jar')],\n }),\n workerType: glue.WorkerType.G_8X,\n description: 'an example Scala ETL job',\n});",
7519
+ "example": "new glue.Job(this, 'EnableSparkUI', {\n jobName: 'EtlJobWithSparkUIPrefix',\n sparkUI: {\n enabled: true,\n },\n executable: glue.JobExecutable.pythonEtl({\n glueVersion: glue.GlueVersion.V3_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n});",
7370
7520
  "custom": {
7371
7521
  "exampleMetadata": "infused"
7372
7522
  }
@@ -7581,7 +7731,7 @@
7581
7731
  },
7582
7732
  "locationInModule": {
7583
7733
  "filename": "lib/job-executable.ts",
7584
- "line": 393
7734
+ "line": 395
7585
7735
  },
7586
7736
  "name": "bind",
7587
7737
  "returns": {
@@ -7609,7 +7759,7 @@
7609
7759
  "kind": "interface",
7610
7760
  "locationInModule": {
7611
7761
  "filename": "lib/job-executable.ts",
7612
- "line": 401
7762
+ "line": 403
7613
7763
  },
7614
7764
  "name": "JobExecutableConfig",
7615
7765
  "properties": [
@@ -7623,7 +7773,7 @@
7623
7773
  "immutable": true,
7624
7774
  "locationInModule": {
7625
7775
  "filename": "lib/job-executable.ts",
7626
- "line": 407
7776
+ "line": 409
7627
7777
  },
7628
7778
  "name": "glueVersion",
7629
7779
  "type": {
@@ -7640,7 +7790,7 @@
7640
7790
  "immutable": true,
7641
7791
  "locationInModule": {
7642
7792
  "filename": "lib/job-executable.ts",
7643
- "line": 414
7793
+ "line": 416
7644
7794
  },
7645
7795
  "name": "language",
7646
7796
  "type": {
@@ -7656,7 +7806,7 @@
7656
7806
  "immutable": true,
7657
7807
  "locationInModule": {
7658
7808
  "filename": "lib/job-executable.ts",
7659
- "line": 438
7809
+ "line": 440
7660
7810
  },
7661
7811
  "name": "script",
7662
7812
  "type": {
@@ -7672,7 +7822,7 @@
7672
7822
  "immutable": true,
7673
7823
  "locationInModule": {
7674
7824
  "filename": "lib/job-executable.ts",
7675
- "line": 419
7825
+ "line": 421
7676
7826
  },
7677
7827
  "name": "type",
7678
7828
  "type": {
@@ -7691,7 +7841,7 @@
7691
7841
  "immutable": true,
7692
7842
  "locationInModule": {
7693
7843
  "filename": "lib/job-executable.ts",
7694
- "line": 447
7844
+ "line": 449
7695
7845
  },
7696
7846
  "name": "className",
7697
7847
  "optional": true,
@@ -7710,7 +7860,7 @@
7710
7860
  "immutable": true,
7711
7861
  "locationInModule": {
7712
7862
  "filename": "lib/job-executable.ts",
7713
- "line": 474
7863
+ "line": 476
7714
7864
  },
7715
7865
  "name": "extraFiles",
7716
7866
  "optional": true,
@@ -7734,7 +7884,7 @@
7734
7884
  "immutable": true,
7735
7885
  "locationInModule": {
7736
7886
  "filename": "lib/job-executable.ts",
7737
- "line": 456
7887
+ "line": 458
7738
7888
  },
7739
7889
  "name": "extraJars",
7740
7890
  "optional": true,
@@ -7758,7 +7908,7 @@
7758
7908
  "immutable": true,
7759
7909
  "locationInModule": {
7760
7910
  "filename": "lib/job-executable.ts",
7761
- "line": 483
7911
+ "line": 485
7762
7912
  },
7763
7913
  "name": "extraJarsFirst",
7764
7914
  "optional": true,
@@ -7777,7 +7927,7 @@
7777
7927
  "immutable": true,
7778
7928
  "locationInModule": {
7779
7929
  "filename": "lib/job-executable.ts",
7780
- "line": 465
7930
+ "line": 467
7781
7931
  },
7782
7932
  "name": "extraPythonFiles",
7783
7933
  "optional": true,
@@ -7800,7 +7950,7 @@
7800
7950
  "immutable": true,
7801
7951
  "locationInModule": {
7802
7952
  "filename": "lib/job-executable.ts",
7803
- "line": 426
7953
+ "line": 428
7804
7954
  },
7805
7955
  "name": "pythonVersion",
7806
7956
  "optional": true,
@@ -7818,7 +7968,7 @@
7818
7968
  "immutable": true,
7819
7969
  "locationInModule": {
7820
7970
  "filename": "lib/job-executable.ts",
7821
- "line": 433
7971
+ "line": 435
7822
7972
  },
7823
7973
  "name": "runtime",
7824
7974
  "optional": true,
@@ -7866,7 +8016,7 @@
7866
8016
  "docs": {
7867
8017
  "stability": "experimental",
7868
8018
  "summary": "Construction properties for `Job`.",
7869
- "example": "declare const bucket: s3.Bucket;\nnew glue.Job(this, 'ScalaSparkEtlJob', {\n executable: glue.JobExecutable.scalaEtl({\n glueVersion: glue.GlueVersion.V4_0,\n script: glue.Code.fromBucket(bucket, 'src/com/example/HelloWorld.scala'),\n className: 'com.example.HelloWorld',\n extraJars: [glue.Code.fromBucket(bucket, 'jars/HelloWorld.jar')],\n }),\n workerType: glue.WorkerType.G_8X,\n description: 'an example Scala ETL job',\n});",
8019
+ "example": "new glue.Job(this, 'EnableSparkUI', {\n jobName: 'EtlJobWithSparkUIPrefix',\n sparkUI: {\n enabled: true,\n },\n executable: glue.JobExecutable.pythonEtl({\n glueVersion: glue.GlueVersion.V3_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n});",
7870
8020
  "custom": {
7871
8021
  "exampleMetadata": "infused"
7872
8022
  }
@@ -7875,7 +8025,7 @@
7875
8025
  "kind": "interface",
7876
8026
  "locationInModule": {
7877
8027
  "filename": "lib/job.ts",
7878
- "line": 481
8028
+ "line": 483
7879
8029
  },
7880
8030
  "name": "JobProps",
7881
8031
  "properties": [
@@ -7888,7 +8038,7 @@
7888
8038
  "immutable": true,
7889
8039
  "locationInModule": {
7890
8040
  "filename": "lib/job.ts",
7891
- "line": 485
8041
+ "line": 487
7892
8042
  },
7893
8043
  "name": "executable",
7894
8044
  "type": {
@@ -7906,7 +8056,7 @@
7906
8056
  "immutable": true,
7907
8057
  "locationInModule": {
7908
8058
  "filename": "lib/job.ts",
7909
- "line": 560
8059
+ "line": 562
7910
8060
  },
7911
8061
  "name": "connections",
7912
8062
  "optional": true,
@@ -7930,7 +8080,7 @@
7930
8080
  "immutable": true,
7931
8081
  "locationInModule": {
7932
8082
  "filename": "lib/job.ts",
7933
- "line": 622
8083
+ "line": 624
7934
8084
  },
7935
8085
  "name": "continuousLogging",
7936
8086
  "optional": true,
@@ -7949,7 +8099,7 @@
7949
8099
  "immutable": true,
7950
8100
  "locationInModule": {
7951
8101
  "filename": "lib/job.ts",
7952
- "line": 575
8102
+ "line": 577
7953
8103
  },
7954
8104
  "name": "defaultArguments",
7955
8105
  "optional": true,
@@ -7972,7 +8122,7 @@
7972
8122
  "immutable": true,
7973
8123
  "locationInModule": {
7974
8124
  "filename": "lib/job.ts",
7975
- "line": 499
8125
+ "line": 501
7976
8126
  },
7977
8127
  "name": "description",
7978
8128
  "optional": true,
@@ -7991,7 +8141,7 @@
7991
8141
  "immutable": true,
7992
8142
  "locationInModule": {
7993
8143
  "filename": "lib/job.ts",
7994
- "line": 602
8144
+ "line": 604
7995
8145
  },
7996
8146
  "name": "enableProfilingMetrics",
7997
8147
  "optional": true,
@@ -8010,7 +8160,7 @@
8010
8160
  "immutable": true,
8011
8161
  "locationInModule": {
8012
8162
  "filename": "lib/job.ts",
8013
- "line": 632
8163
+ "line": 634
8014
8164
  },
8015
8165
  "name": "executionClass",
8016
8166
  "optional": true,
@@ -8028,7 +8178,7 @@
8028
8178
  "immutable": true,
8029
8179
  "locationInModule": {
8030
8180
  "filename": "lib/job.ts",
8031
- "line": 492
8181
+ "line": 494
8032
8182
  },
8033
8183
  "name": "jobName",
8034
8184
  "optional": true,
@@ -8047,7 +8197,7 @@
8047
8197
  "immutable": true,
8048
8198
  "locationInModule": {
8049
8199
  "filename": "lib/job.ts",
8050
- "line": 507
8200
+ "line": 509
8051
8201
  },
8052
8202
  "name": "maxCapacity",
8053
8203
  "optional": true,
@@ -8066,7 +8216,7 @@
8066
8216
  "immutable": true,
8067
8217
  "locationInModule": {
8068
8218
  "filename": "lib/job.ts",
8069
- "line": 523
8219
+ "line": 525
8070
8220
  },
8071
8221
  "name": "maxConcurrentRuns",
8072
8222
  "optional": true,
@@ -8084,7 +8234,7 @@
8084
8234
  "immutable": true,
8085
8235
  "locationInModule": {
8086
8236
  "filename": "lib/job.ts",
8087
- "line": 514
8237
+ "line": 516
8088
8238
  },
8089
8239
  "name": "maxRetries",
8090
8240
  "optional": true,
@@ -8102,7 +8252,7 @@
8102
8252
  "immutable": true,
8103
8253
  "locationInModule": {
8104
8254
  "filename": "lib/job.ts",
8105
- "line": 530
8255
+ "line": 532
8106
8256
  },
8107
8257
  "name": "notifyDelayAfter",
8108
8258
  "optional": true,
@@ -8122,7 +8272,7 @@
8122
8272
  "immutable": true,
8123
8273
  "locationInModule": {
8124
8274
  "filename": "lib/job.ts",
8125
- "line": 593
8275
+ "line": 595
8126
8276
  },
8127
8277
  "name": "role",
8128
8278
  "optional": true,
@@ -8140,7 +8290,7 @@
8140
8290
  "immutable": true,
8141
8291
  "locationInModule": {
8142
8292
  "filename": "lib/job.ts",
8143
- "line": 567
8293
+ "line": 569
8144
8294
  },
8145
8295
  "name": "securityConfiguration",
8146
8296
  "optional": true,
@@ -8159,7 +8309,7 @@
8159
8309
  "immutable": true,
8160
8310
  "locationInModule": {
8161
8311
  "filename": "lib/job.ts",
8162
- "line": 612
8312
+ "line": 614
8163
8313
  },
8164
8314
  "name": "sparkUI",
8165
8315
  "optional": true,
@@ -8177,7 +8327,7 @@
8177
8327
  "immutable": true,
8178
8328
  "locationInModule": {
8179
8329
  "filename": "lib/job.ts",
8180
- "line": 582
8330
+ "line": 584
8181
8331
  },
8182
8332
  "name": "tags",
8183
8333
  "optional": true,
@@ -8200,7 +8350,7 @@
8200
8350
  "immutable": true,
8201
8351
  "locationInModule": {
8202
8352
  "filename": "lib/job.ts",
8203
- "line": 537
8353
+ "line": 539
8204
8354
  },
8205
8355
  "name": "timeout",
8206
8356
  "optional": true,
@@ -8218,7 +8368,7 @@
8218
8368
  "immutable": true,
8219
8369
  "locationInModule": {
8220
8370
  "filename": "lib/job.ts",
8221
- "line": 551
8371
+ "line": 553
8222
8372
  },
8223
8373
  "name": "workerCount",
8224
8374
  "optional": true,
@@ -8236,7 +8386,7 @@
8236
8386
  "immutable": true,
8237
8387
  "locationInModule": {
8238
8388
  "filename": "lib/job.ts",
8239
- "line": 544
8389
+ "line": 546
8240
8390
  },
8241
8391
  "name": "workerType",
8242
8392
  "optional": true,
@@ -8258,7 +8408,7 @@
8258
8408
  "kind": "enum",
8259
8409
  "locationInModule": {
8260
8410
  "filename": "lib/job.ts",
8261
- "line": 78
8411
+ "line": 79
8262
8412
  },
8263
8413
  "members": [
8264
8414
  {
@@ -8460,7 +8610,7 @@
8460
8610
  "kind": "enum",
8461
8611
  "locationInModule": {
8462
8612
  "filename": "lib/job.ts",
8463
- "line": 120
8613
+ "line": 121
8464
8614
  },
8465
8615
  "members": [
8466
8616
  {
@@ -8481,6 +8631,84 @@
8481
8631
  "name": "MetricType",
8482
8632
  "symbolId": "lib/job:MetricType"
8483
8633
  },
8634
+ "@aws-cdk/aws-glue-alpha.NumericOverflowHandlingAction": {
8635
+ "assembly": "@aws-cdk/aws-glue-alpha",
8636
+ "docs": {
8637
+ "see": "https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_EXTERNAL_TABLE.html#r_CREATE_EXTERNAL_TABLE-parameters - under _\"TABLE PROPERTIES\"_ > _\"numeric_overflow_handling\"_",
8638
+ "stability": "experimental",
8639
+ "summary": "Specifies the action to perform when ORC data contains an integer (for example, BIGINT or int64) that is larger than the column definition (for example, SMALLINT or int16)."
8640
+ },
8641
+ "fqn": "@aws-cdk/aws-glue-alpha.NumericOverflowHandlingAction",
8642
+ "kind": "enum",
8643
+ "locationInModule": {
8644
+ "filename": "lib/storage-parameter.ts",
8645
+ "line": 65
8646
+ },
8647
+ "members": [
8648
+ {
8649
+ "docs": {
8650
+ "stability": "experimental",
8651
+ "summary": "Invalid character handling is turned off."
8652
+ },
8653
+ "name": "DISABLED"
8654
+ },
8655
+ {
8656
+ "docs": {
8657
+ "stability": "experimental",
8658
+ "summary": "Cancel the query when the data includes invalid characters."
8659
+ },
8660
+ "name": "FAIL"
8661
+ },
8662
+ {
8663
+ "docs": {
8664
+ "stability": "experimental",
8665
+ "summary": "Set invalid characters to null."
8666
+ },
8667
+ "name": "SET_TO_NULL"
8668
+ },
8669
+ {
8670
+ "docs": {
8671
+ "stability": "experimental",
8672
+ "summary": "Set each value in the row to null."
8673
+ },
8674
+ "name": "DROP_ROW"
8675
+ }
8676
+ ],
8677
+ "name": "NumericOverflowHandlingAction",
8678
+ "symbolId": "lib/storage-parameter:NumericOverflowHandlingAction"
8679
+ },
8680
+ "@aws-cdk/aws-glue-alpha.OrcColumnMappingType": {
8681
+ "assembly": "@aws-cdk/aws-glue-alpha",
8682
+ "docs": {
8683
+ "see": "https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_EXTERNAL_TABLE.html#r_CREATE_EXTERNAL_TABLE-parameters - under _\"TABLE PROPERTIES\"_ > _\"orc.schema.resolution\"_",
8684
+ "stability": "experimental",
8685
+ "summary": "Specifies how to map columns when the table uses ORC data format."
8686
+ },
8687
+ "fqn": "@aws-cdk/aws-glue-alpha.OrcColumnMappingType",
8688
+ "kind": "enum",
8689
+ "locationInModule": {
8690
+ "filename": "lib/storage-parameter.ts",
8691
+ "line": 200
8692
+ },
8693
+ "members": [
8694
+ {
8695
+ "docs": {
8696
+ "stability": "experimental",
8697
+ "summary": "Map columns by name."
8698
+ },
8699
+ "name": "NAME"
8700
+ },
8701
+ {
8702
+ "docs": {
8703
+ "stability": "experimental",
8704
+ "summary": "Map columns by position."
8705
+ },
8706
+ "name": "POSITION"
8707
+ }
8708
+ ],
8709
+ "name": "OrcColumnMappingType",
8710
+ "symbolId": "lib/storage-parameter:OrcColumnMappingType"
8711
+ },
8484
8712
  "@aws-cdk/aws-glue-alpha.OutputFormat": {
8485
8713
  "assembly": "@aws-cdk/aws-glue-alpha",
8486
8714
  "docs": {
@@ -8620,7 +8848,7 @@
8620
8848
  "kind": "interface",
8621
8849
  "locationInModule": {
8622
8850
  "filename": "lib/table.ts",
8623
- "line": 16
8851
+ "line": 17
8624
8852
  },
8625
8853
  "name": "PartitionIndex",
8626
8854
  "properties": [
@@ -8634,7 +8862,7 @@
8634
8862
  "immutable": true,
8635
8863
  "locationInModule": {
8636
8864
  "filename": "lib/table.ts",
8637
- "line": 29
8865
+ "line": 30
8638
8866
  },
8639
8867
  "name": "keyNames",
8640
8868
  "type": {
@@ -8656,7 +8884,7 @@
8656
8884
  "immutable": true,
8657
8885
  "locationInModule": {
8658
8886
  "filename": "lib/table.ts",
8659
- "line": 22
8887
+ "line": 23
8660
8888
  },
8661
8889
  "name": "indexName",
8662
8890
  "optional": true,
@@ -9132,7 +9360,7 @@
9132
9360
  "docs": {
9133
9361
  "stability": "experimental",
9134
9362
  "summary": "Python version.",
9135
- "example": "new glue.Job(this, 'PythonSparkStreamingJob', {\n executable: glue.JobExecutable.pythonStreaming({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n description: 'an example Python Streaming job',\n});",
9363
+ "example": "new glue.Job(this, 'EnableSparkUI', {\n jobName: 'EtlJobWithSparkUIPrefix',\n sparkUI: {\n enabled: true,\n },\n executable: glue.JobExecutable.pythonEtl({\n glueVersion: glue.GlueVersion.V3_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n});",
9136
9364
  "custom": {
9137
9365
  "exampleMetadata": "infused"
9138
9366
  }
@@ -10488,7 +10716,7 @@
10488
10716
  "kind": "interface",
10489
10717
  "locationInModule": {
10490
10718
  "filename": "lib/job.ts",
10491
- "line": 404
10719
+ "line": 406
10492
10720
  },
10493
10721
  "name": "SparkUILoggingLocation",
10494
10722
  "properties": [
@@ -10501,7 +10729,7 @@
10501
10729
  "immutable": true,
10502
10730
  "locationInModule": {
10503
10731
  "filename": "lib/job.ts",
10504
- "line": 408
10732
+ "line": 410
10505
10733
  },
10506
10734
  "name": "bucket",
10507
10735
  "type": {
@@ -10518,7 +10746,7 @@
10518
10746
  "immutable": true,
10519
10747
  "locationInModule": {
10520
10748
  "filename": "lib/job.ts",
10521
- "line": 415
10749
+ "line": 417
10522
10750
  },
10523
10751
  "name": "prefix",
10524
10752
  "optional": true,
@@ -10536,16 +10764,16 @@
10536
10764
  "see": "https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html",
10537
10765
  "stability": "experimental",
10538
10766
  "summary": "Properties for enabling Spark UI monitoring feature for Spark-based Glue jobs.",
10539
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport { aws_s3 as s3 } from 'aws-cdk-lib';\n\ndeclare const bucket: s3.Bucket;\nconst sparkUIProps: glue_alpha.SparkUIProps = {\n enabled: false,\n\n // the properties below are optional\n bucket: bucket,\n prefix: 'prefix',\n};",
10767
+ "example": "new glue.Job(this, 'EnableSparkUI', {\n jobName: 'EtlJobWithSparkUIPrefix',\n sparkUI: {\n enabled: true,\n },\n executable: glue.JobExecutable.pythonEtl({\n glueVersion: glue.GlueVersion.V3_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n});",
10540
10768
  "custom": {
10541
- "exampleMetadata": "fixture=_generated"
10769
+ "exampleMetadata": "infused"
10542
10770
  }
10543
10771
  },
10544
10772
  "fqn": "@aws-cdk/aws-glue-alpha.SparkUIProps",
10545
10773
  "kind": "interface",
10546
10774
  "locationInModule": {
10547
10775
  "filename": "lib/job.ts",
10548
- "line": 377
10776
+ "line": 378
10549
10777
  },
10550
10778
  "name": "SparkUIProps",
10551
10779
  "properties": [
@@ -10558,7 +10786,7 @@
10558
10786
  "immutable": true,
10559
10787
  "locationInModule": {
10560
10788
  "filename": "lib/job.ts",
10561
- "line": 381
10789
+ "line": 382
10562
10790
  },
10563
10791
  "name": "enabled",
10564
10792
  "type": {
@@ -10575,7 +10803,7 @@
10575
10803
  "immutable": true,
10576
10804
  "locationInModule": {
10577
10805
  "filename": "lib/job.ts",
10578
- "line": 388
10806
+ "line": 389
10579
10807
  },
10580
10808
  "name": "bucket",
10581
10809
  "optional": true,
@@ -10586,14 +10814,15 @@
10586
10814
  {
10587
10815
  "abstract": true,
10588
10816
  "docs": {
10589
- "default": "'/' - the logs will be written at the root of the bucket",
10817
+ "default": "- the logs will be written at the root of the bucket",
10818
+ "remarks": "Use format `'/foo/bar'`",
10590
10819
  "stability": "experimental",
10591
10820
  "summary": "The path inside the bucket (objects prefix) where the Glue job stores the logs."
10592
10821
  },
10593
10822
  "immutable": true,
10594
10823
  "locationInModule": {
10595
10824
  "filename": "lib/job.ts",
10596
- "line": 395
10825
+ "line": 397
10597
10826
  },
10598
10827
  "name": "prefix",
10599
10828
  "optional": true,
@@ -10604,88 +10833,97 @@
10604
10833
  ],
10605
10834
  "symbolId": "lib/job:SparkUIProps"
10606
10835
  },
10607
- "@aws-cdk/aws-glue-alpha.Table": {
10836
+ "@aws-cdk/aws-glue-alpha.StorageParameter": {
10608
10837
  "assembly": "@aws-cdk/aws-glue-alpha",
10609
- "base": "aws-cdk-lib.Resource",
10610
10838
  "docs": {
10839
+ "remarks": "If you would like to specify a storage parameter that is not available as a static member of this class, use the `StorageParameter.custom` method.\n\nThe list of storage parameters currently known within the CDK is listed.",
10840
+ "see": "https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_EXTERNAL_TABLE.html#r_CREATE_EXTERNAL_TABLE-parameters - under _\"TABLE PROPERTIES\"_",
10611
10841
  "stability": "experimental",
10612
- "summary": "A Glue table.",
10613
- "example": "declare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n dataFormat: glue.DataFormat.JSON,\n});",
10842
+ "summary": "A storage parameter. The list of storage parameters available is not exhaustive and other keys may be used.",
10843
+ "example": "declare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n storageParameters: [\n glue.StorageParameter.skipHeaderLineCount(1),\n glue.StorageParameter.compressionType(glue.CompressionType.GZIP),\n glue.StorageParameter.custom('separatorChar', ',')\n ],\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});",
10614
10844
  "custom": {
10615
10845
  "exampleMetadata": "infused"
10616
10846
  }
10617
10847
  },
10618
- "fqn": "@aws-cdk/aws-glue-alpha.Table",
10848
+ "fqn": "@aws-cdk/aws-glue-alpha.StorageParameter",
10619
10849
  "initializer": {
10620
10850
  "docs": {
10621
10851
  "stability": "experimental"
10622
10852
  },
10623
10853
  "locationInModule": {
10624
- "filename": "lib/table.ts",
10625
- "line": 283
10854
+ "filename": "lib/storage-parameter.ts",
10855
+ "line": 422
10626
10856
  },
10627
10857
  "parameters": [
10628
10858
  {
10629
- "name": "scope",
10630
- "type": {
10631
- "fqn": "constructs.Construct"
10632
- }
10633
- },
10634
- {
10635
- "name": "id",
10859
+ "name": "key",
10636
10860
  "type": {
10637
10861
  "primitive": "string"
10638
10862
  }
10639
10863
  },
10640
10864
  {
10641
- "name": "props",
10865
+ "name": "value",
10642
10866
  "type": {
10643
- "fqn": "@aws-cdk/aws-glue-alpha.TableProps"
10867
+ "primitive": "string"
10644
10868
  }
10645
10869
  }
10646
- ]
10870
+ ],
10871
+ "protected": true
10647
10872
  },
10648
- "interfaces": [
10649
- "@aws-cdk/aws-glue-alpha.ITable"
10650
- ],
10651
10873
  "kind": "class",
10652
10874
  "locationInModule": {
10653
- "filename": "lib/table.ts",
10654
- "line": 189
10875
+ "filename": "lib/storage-parameter.ts",
10876
+ "line": 303
10655
10877
  },
10656
10878
  "methods": [
10657
10879
  {
10658
10880
  "docs": {
10659
- "stability": "experimental"
10881
+ "remarks": "This property is only available for an uncompressed text file format.",
10882
+ "stability": "experimental",
10883
+ "summary": "Identifies if the file contains less or more values for a row than the number of columns specified in the external table definition."
10660
10884
  },
10661
10885
  "locationInModule": {
10662
- "filename": "lib/table.ts",
10663
- "line": 191
10886
+ "filename": "lib/storage-parameter.ts",
10887
+ "line": 363
10664
10888
  },
10665
- "name": "fromTableArn",
10889
+ "name": "columnCountMismatchHandling",
10666
10890
  "parameters": [
10667
10891
  {
10668
- "name": "scope",
10669
- "type": {
10670
- "fqn": "constructs.Construct"
10671
- }
10672
- },
10673
- {
10674
- "name": "id",
10892
+ "name": "value",
10675
10893
  "type": {
10676
- "primitive": "string"
10894
+ "fqn": "@aws-cdk/aws-glue-alpha.ColumnCountMismatchHandlingAction"
10677
10895
  }
10678
- },
10896
+ }
10897
+ ],
10898
+ "returns": {
10899
+ "type": {
10900
+ "fqn": "@aws-cdk/aws-glue-alpha.StorageParameter"
10901
+ }
10902
+ },
10903
+ "static": true
10904
+ },
10905
+ {
10906
+ "docs": {
10907
+ "remarks": "This value overrides the compression type specified through the extension.",
10908
+ "stability": "experimental",
10909
+ "summary": "The type of compression used on the table, when the file name does not contain an extension."
10910
+ },
10911
+ "locationInModule": {
10912
+ "filename": "lib/storage-parameter.ts",
10913
+ "line": 321
10914
+ },
10915
+ "name": "compressionType",
10916
+ "parameters": [
10679
10917
  {
10680
- "name": "tableArn",
10918
+ "name": "value",
10681
10919
  "type": {
10682
- "primitive": "string"
10920
+ "fqn": "@aws-cdk/aws-glue-alpha.CompressionType"
10683
10921
  }
10684
10922
  }
10685
10923
  ],
10686
10924
  "returns": {
10687
10925
  "type": {
10688
- "fqn": "@aws-cdk/aws-glue-alpha.ITable"
10926
+ "fqn": "@aws-cdk/aws-glue-alpha.StorageParameter"
10689
10927
  }
10690
10928
  },
10691
10929
  "static": true
@@ -10693,19 +10931,751 @@
10693
10931
  {
10694
10932
  "docs": {
10695
10933
  "stability": "experimental",
10696
- "summary": "Creates a Table construct that represents an external table."
10934
+ "summary": "A custom storage parameter."
10697
10935
  },
10698
10936
  "locationInModule": {
10699
- "filename": "lib/table.ts",
10700
- "line": 207
10937
+ "filename": "lib/storage-parameter.ts",
10938
+ "line": 418
10701
10939
  },
10702
- "name": "fromTableAttributes",
10940
+ "name": "custom",
10703
10941
  "parameters": [
10704
10942
  {
10705
10943
  "docs": {
10706
- "summary": "The scope creating construct (usually `this`)."
10944
+ "summary": "- The key of the storage parameter."
10707
10945
  },
10708
- "name": "scope",
10946
+ "name": "key",
10947
+ "type": {
10948
+ "primitive": "string"
10949
+ }
10950
+ },
10951
+ {
10952
+ "docs": {
10953
+ "summary": "- The value of the storage parameter."
10954
+ },
10955
+ "name": "value",
10956
+ "type": {
10957
+ "primitive": "any"
10958
+ }
10959
+ }
10960
+ ],
10961
+ "returns": {
10962
+ "type": {
10963
+ "fqn": "@aws-cdk/aws-glue-alpha.StorageParameter"
10964
+ }
10965
+ },
10966
+ "static": true
10967
+ },
10968
+ {
10969
+ "docs": {
10970
+ "stability": "experimental",
10971
+ "summary": "Determines whether data handling is on for the table."
10972
+ },
10973
+ "locationInModule": {
10974
+ "filename": "lib/storage-parameter.ts",
10975
+ "line": 314
10976
+ },
10977
+ "name": "dataCleansingEnabled",
10978
+ "parameters": [
10979
+ {
10980
+ "name": "value",
10981
+ "type": {
10982
+ "primitive": "boolean"
10983
+ }
10984
+ }
10985
+ ],
10986
+ "returns": {
10987
+ "type": {
10988
+ "fqn": "@aws-cdk/aws-glue-alpha.StorageParameter"
10989
+ }
10990
+ },
10991
+ "static": true
10992
+ },
10993
+ {
10994
+ "docs": {
10995
+ "stability": "experimental",
10996
+ "summary": "Specifies the action to perform when query results contain invalid UTF-8 character values."
10997
+ },
10998
+ "locationInModule": {
10999
+ "filename": "lib/storage-parameter.ts",
11000
+ "line": 328
11001
+ },
11002
+ "name": "invalidCharHandling",
11003
+ "parameters": [
11004
+ {
11005
+ "name": "value",
11006
+ "type": {
11007
+ "fqn": "@aws-cdk/aws-glue-alpha.InvalidCharHandlingAction"
11008
+ }
11009
+ }
11010
+ ],
11011
+ "returns": {
11012
+ "type": {
11013
+ "fqn": "@aws-cdk/aws-glue-alpha.StorageParameter"
11014
+ }
11015
+ },
11016
+ "static": true
11017
+ },
11018
+ {
11019
+ "docs": {
11020
+ "stability": "experimental",
11021
+ "summary": "Specifies the action to perform when ORC data contains an integer (for example, BIGINT or int64) that is larger than the column definition (for example, SMALLINT or int16)."
11022
+ },
11023
+ "locationInModule": {
11024
+ "filename": "lib/storage-parameter.ts",
11025
+ "line": 342
11026
+ },
11027
+ "name": "numericOverflowHandling",
11028
+ "parameters": [
11029
+ {
11030
+ "name": "value",
11031
+ "type": {
11032
+ "fqn": "@aws-cdk/aws-glue-alpha.NumericOverflowHandlingAction"
11033
+ }
11034
+ }
11035
+ ],
11036
+ "returns": {
11037
+ "type": {
11038
+ "fqn": "@aws-cdk/aws-glue-alpha.StorageParameter"
11039
+ }
11040
+ },
11041
+ "static": true
11042
+ },
11043
+ {
11044
+ "docs": {
11045
+ "remarks": "To explicitly update an external table's statistics, set the numRows property to indicate the size of the table. Amazon Redshift doesn't analyze external tables to generate the table statistics that the query optimizer uses to generate a query plan. If table statistics aren't set for an external table, Amazon Redshift generates a query execution plan based on an assumption that external tables are the larger tables and local tables are the smaller tables.",
11046
+ "stability": "experimental",
11047
+ "summary": "A property that sets the numRows value for the table definition."
11048
+ },
11049
+ "locationInModule": {
11050
+ "filename": "lib/storage-parameter.ts",
11051
+ "line": 370
11052
+ },
11053
+ "name": "numRows",
11054
+ "parameters": [
11055
+ {
11056
+ "name": "value",
11057
+ "type": {
11058
+ "primitive": "number"
11059
+ }
11060
+ }
11061
+ ],
11062
+ "returns": {
11063
+ "type": {
11064
+ "fqn": "@aws-cdk/aws-glue-alpha.StorageParameter"
11065
+ }
11066
+ },
11067
+ "static": true
11068
+ },
11069
+ {
11070
+ "docs": {
11071
+ "default": "OrcColumnMappingType.NAME",
11072
+ "remarks": "This property is ignored for other data formats. If this property is omitted, columns are mapped by `OrcColumnMappingType.NAME` by default.",
11073
+ "stability": "experimental",
11074
+ "summary": "A property that sets the column mapping type for tables that use ORC data format."
11075
+ },
11076
+ "locationInModule": {
11077
+ "filename": "lib/storage-parameter.ts",
11078
+ "line": 386
11079
+ },
11080
+ "name": "orcSchemaResolution",
11081
+ "parameters": [
11082
+ {
11083
+ "name": "value",
11084
+ "type": {
11085
+ "fqn": "@aws-cdk/aws-glue-alpha.OrcColumnMappingType"
11086
+ }
11087
+ }
11088
+ ],
11089
+ "returns": {
11090
+ "type": {
11091
+ "fqn": "@aws-cdk/aws-glue-alpha.StorageParameter"
11092
+ }
11093
+ },
11094
+ "static": true
11095
+ },
11096
+ {
11097
+ "docs": {
11098
+ "stability": "experimental",
11099
+ "summary": "Specifies the replacement character to use when you set `INVALID_CHAR_HANDLING` to `REPLACE`."
11100
+ },
11101
+ "locationInModule": {
11102
+ "filename": "lib/storage-parameter.ts",
11103
+ "line": 335
11104
+ },
11105
+ "name": "replacementChar",
11106
+ "parameters": [
11107
+ {
11108
+ "name": "value",
11109
+ "type": {
11110
+ "primitive": "string"
11111
+ }
11112
+ }
11113
+ ],
11114
+ "returns": {
11115
+ "type": {
11116
+ "fqn": "@aws-cdk/aws-glue-alpha.StorageParameter"
11117
+ }
11118
+ },
11119
+ "static": true
11120
+ },
11121
+ {
11122
+ "docs": {
11123
+ "stability": "experimental",
11124
+ "summary": "A property that sets number of rows to skip at the beginning of each source file."
11125
+ },
11126
+ "locationInModule": {
11127
+ "filename": "lib/storage-parameter.ts",
11128
+ "line": 377
11129
+ },
11130
+ "name": "serializationNullFormat",
11131
+ "parameters": [
11132
+ {
11133
+ "name": "value",
11134
+ "type": {
11135
+ "primitive": "string"
11136
+ }
11137
+ }
11138
+ ],
11139
+ "returns": {
11140
+ "type": {
11141
+ "fqn": "@aws-cdk/aws-glue-alpha.StorageParameter"
11142
+ }
11143
+ },
11144
+ "static": true
11145
+ },
11146
+ {
11147
+ "docs": {
11148
+ "stability": "experimental",
11149
+ "summary": "The number of rows to skip at the top of a CSV file when the table is being created."
11150
+ },
11151
+ "locationInModule": {
11152
+ "filename": "lib/storage-parameter.ts",
11153
+ "line": 307
11154
+ },
11155
+ "name": "skipHeaderLineCount",
11156
+ "parameters": [
11157
+ {
11158
+ "name": "value",
11159
+ "type": {
11160
+ "primitive": "number"
11161
+ }
11162
+ }
11163
+ ],
11164
+ "returns": {
11165
+ "type": {
11166
+ "fqn": "@aws-cdk/aws-glue-alpha.StorageParameter"
11167
+ }
11168
+ },
11169
+ "static": true
11170
+ },
11171
+ {
11172
+ "docs": {
11173
+ "remarks": "By default, Redshift Spectrum sets the value to null for data that exceeds the width of the column.",
11174
+ "stability": "experimental",
11175
+ "summary": "Specifies how to handle data being loaded that exceeds the length of the data type defined for columns containing VARBYTE data."
11176
+ },
11177
+ "locationInModule": {
11178
+ "filename": "lib/storage-parameter.ts",
11179
+ "line": 349
11180
+ },
11181
+ "name": "surplusBytesHandling",
11182
+ "parameters": [
11183
+ {
11184
+ "name": "value",
11185
+ "type": {
11186
+ "fqn": "@aws-cdk/aws-glue-alpha.SurplusBytesHandlingAction"
11187
+ }
11188
+ }
11189
+ ],
11190
+ "returns": {
11191
+ "type": {
11192
+ "fqn": "@aws-cdk/aws-glue-alpha.StorageParameter"
11193
+ }
11194
+ },
11195
+ "static": true
11196
+ },
11197
+ {
11198
+ "docs": {
11199
+ "remarks": "By default, Redshift Spectrum sets the value to null for data that exceeds the width of the column.",
11200
+ "stability": "experimental",
11201
+ "summary": "Specifies how to handle data being loaded that exceeds the length of the data type defined for columns containing VARCHAR, CHAR, or string data."
11202
+ },
11203
+ "locationInModule": {
11204
+ "filename": "lib/storage-parameter.ts",
11205
+ "line": 356
11206
+ },
11207
+ "name": "surplusCharHandling",
11208
+ "parameters": [
11209
+ {
11210
+ "name": "value",
11211
+ "type": {
11212
+ "fqn": "@aws-cdk/aws-glue-alpha.SurplusCharHandlingAction"
11213
+ }
11214
+ }
11215
+ ],
11216
+ "returns": {
11217
+ "type": {
11218
+ "fqn": "@aws-cdk/aws-glue-alpha.StorageParameter"
11219
+ }
11220
+ },
11221
+ "static": true
11222
+ },
11223
+ {
11224
+ "docs": {
11225
+ "stability": "experimental",
11226
+ "summary": "You can specify an AWS Key Management Service key to enable Server–Side Encryption (SSE) for Amazon S3 objects."
11227
+ },
11228
+ "locationInModule": {
11229
+ "filename": "lib/storage-parameter.ts",
11230
+ "line": 409
11231
+ },
11232
+ "name": "writeKmsKeyId",
11233
+ "parameters": [
11234
+ {
11235
+ "name": "value",
11236
+ "type": {
11237
+ "primitive": "string"
11238
+ }
11239
+ }
11240
+ ],
11241
+ "returns": {
11242
+ "type": {
11243
+ "fqn": "@aws-cdk/aws-glue-alpha.StorageParameter"
11244
+ }
11245
+ },
11246
+ "static": true
11247
+ },
11248
+ {
11249
+ "docs": {
11250
+ "remarks": "The size must be a valid integer between 5 and 6200. The default maximum file size is 6,200 MB. This table property also applies to any subsequent INSERT statement into the same external table.",
11251
+ "stability": "experimental",
11252
+ "summary": "A property that sets the maximum size (in MB) of each file written to Amazon S3 by CREATE EXTERNAL TABLE AS."
11253
+ },
11254
+ "locationInModule": {
11255
+ "filename": "lib/storage-parameter.ts",
11256
+ "line": 402
11257
+ },
11258
+ "name": "writeMaxFileSizeMb",
11259
+ "parameters": [
11260
+ {
11261
+ "name": "value",
11262
+ "type": {
11263
+ "primitive": "number"
11264
+ }
11265
+ }
11266
+ ],
11267
+ "returns": {
11268
+ "type": {
11269
+ "fqn": "@aws-cdk/aws-glue-alpha.StorageParameter"
11270
+ }
11271
+ },
11272
+ "static": true
11273
+ },
11274
+ {
11275
+ "docs": {
11276
+ "default": "WriteParallel.ON",
11277
+ "remarks": "When 'write.parallel' is set to off, CREATE EXTERNAL TABLE AS writes to one or more data files serially onto Amazon S3. This table property also applies to any subsequent INSERT statement into the same external table.",
11278
+ "stability": "experimental",
11279
+ "summary": "A property that sets whether CREATE EXTERNAL TABLE AS should write data in parallel."
11280
+ },
11281
+ "locationInModule": {
11282
+ "filename": "lib/storage-parameter.ts",
11283
+ "line": 395
11284
+ },
11285
+ "name": "writeParallel",
11286
+ "parameters": [
11287
+ {
11288
+ "name": "value",
11289
+ "type": {
11290
+ "fqn": "@aws-cdk/aws-glue-alpha.WriteParallel"
11291
+ }
11292
+ }
11293
+ ],
11294
+ "returns": {
11295
+ "type": {
11296
+ "fqn": "@aws-cdk/aws-glue-alpha.StorageParameter"
11297
+ }
11298
+ },
11299
+ "static": true
11300
+ }
11301
+ ],
11302
+ "name": "StorageParameter",
11303
+ "properties": [
11304
+ {
11305
+ "docs": {
11306
+ "stability": "experimental"
11307
+ },
11308
+ "immutable": true,
11309
+ "locationInModule": {
11310
+ "filename": "lib/storage-parameter.ts",
11311
+ "line": 422
11312
+ },
11313
+ "name": "key",
11314
+ "type": {
11315
+ "primitive": "string"
11316
+ }
11317
+ },
11318
+ {
11319
+ "docs": {
11320
+ "stability": "experimental"
11321
+ },
11322
+ "immutable": true,
11323
+ "locationInModule": {
11324
+ "filename": "lib/storage-parameter.ts",
11325
+ "line": 422
11326
+ },
11327
+ "name": "value",
11328
+ "type": {
11329
+ "primitive": "string"
11330
+ }
11331
+ }
11332
+ ],
11333
+ "symbolId": "lib/storage-parameter:StorageParameter"
11334
+ },
11335
+ "@aws-cdk/aws-glue-alpha.StorageParameters": {
11336
+ "assembly": "@aws-cdk/aws-glue-alpha",
11337
+ "docs": {
11338
+ "stability": "experimental",
11339
+ "summary": "The storage parameter keys that are currently known, this list is not exhaustive and other keys may be used.",
11340
+ "example": " declare const glueDatabase: glue.IDatabase;\n const table = new glue.Table(this, 'Table', {\n storageParameters: [\n glue.StorageParameter.skipHeaderLineCount(1),\n glue.StorageParameter.compressionType(glue.CompressionType.GZIP),\n glue.StorageParameter.custom('foo', 'bar'), // Will have no effect\n glue.StorageParameter.custom('separatorChar', ','), // Will describe the separator char used in the data\n glue.StorageParameter.custom(glue.StorageParameters.WRITE_PARALLEL, 'off'),\n ],\n // ...\n database: glueDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.CSV,\n });",
11341
+ "custom": {
11342
+ "exampleMetadata": "infused"
11343
+ }
11344
+ },
11345
+ "fqn": "@aws-cdk/aws-glue-alpha.StorageParameters",
11346
+ "kind": "enum",
11347
+ "locationInModule": {
11348
+ "filename": "lib/storage-parameter.ts",
11349
+ "line": 215
11350
+ },
11351
+ "members": [
11352
+ {
11353
+ "docs": {
11354
+ "stability": "experimental",
11355
+ "summary": "The number of rows to skip at the top of a CSV file when the table is being created."
11356
+ },
11357
+ "name": "SKIP_HEADER_LINE_COUNT"
11358
+ },
11359
+ {
11360
+ "docs": {
11361
+ "stability": "experimental",
11362
+ "summary": "Determines whether data handling is on for the table."
11363
+ },
11364
+ "name": "DATA_CLEANSING_ENABLED"
11365
+ },
11366
+ {
11367
+ "docs": {
11368
+ "remarks": "This value overrides the compression type specified through the extension.",
11369
+ "stability": "experimental",
11370
+ "summary": "The type of compression used on the table, when the file name does not contain an extension."
11371
+ },
11372
+ "name": "COMPRESSION_TYPE"
11373
+ },
11374
+ {
11375
+ "docs": {
11376
+ "stability": "experimental",
11377
+ "summary": "Specifies the action to perform when query results contain invalid UTF-8 character values."
11378
+ },
11379
+ "name": "INVALID_CHAR_HANDLING"
11380
+ },
11381
+ {
11382
+ "docs": {
11383
+ "stability": "experimental",
11384
+ "summary": "Specifies the replacement character to use when you set `INVALID_CHAR_HANDLING` to `REPLACE`."
11385
+ },
11386
+ "name": "REPLACEMENT_CHAR"
11387
+ },
11388
+ {
11389
+ "docs": {
11390
+ "stability": "experimental",
11391
+ "summary": "Specifies the action to perform when ORC data contains an integer (for example, BIGINT or int64) that is larger than the column definition (for example, SMALLINT or int16)."
11392
+ },
11393
+ "name": "NUMERIC_OVERFLOW_HANDLING"
11394
+ },
11395
+ {
11396
+ "docs": {
11397
+ "remarks": "By default, Redshift Spectrum sets the value to null for data that exceeds the width of the column.",
11398
+ "stability": "experimental",
11399
+ "summary": "Specifies how to handle data being loaded that exceeds the length of the data type defined for columns containing VARBYTE data."
11400
+ },
11401
+ "name": "SURPLUS_BYTES_HANDLING"
11402
+ },
11403
+ {
11404
+ "docs": {
11405
+ "remarks": "By default, Redshift Spectrum sets the value to null for data that exceeds the width of the column.",
11406
+ "stability": "experimental",
11407
+ "summary": "Specifies how to handle data being loaded that exceeds the length of the data type defined for columns containing VARCHAR, CHAR, or string data."
11408
+ },
11409
+ "name": "SURPLUS_CHAR_HANDLING"
11410
+ },
11411
+ {
11412
+ "docs": {
11413
+ "remarks": "This property is only available for an uncompressed text file format.",
11414
+ "stability": "experimental",
11415
+ "summary": "Identifies if the file contains less or more values for a row than the number of columns specified in the external table definition."
11416
+ },
11417
+ "name": "COLUMN_COUNT_MISMATCH_HANDLING"
11418
+ },
11419
+ {
11420
+ "docs": {
11421
+ "remarks": "To explicitly update an external table's statistics, set the numRows property to indicate the size of the table. Amazon Redshift doesn't analyze external tables to generate the table statistics that the query optimizer uses to generate a query plan. If table statistics aren't set for an external table, Amazon Redshift generates a query execution plan based on an assumption that external tables are the larger tables and local tables are the smaller tables.",
11422
+ "stability": "experimental",
11423
+ "summary": "A property that sets the numRows value for the table definition."
11424
+ },
11425
+ "name": "NUM_ROWS"
11426
+ },
11427
+ {
11428
+ "docs": {
11429
+ "stability": "experimental",
11430
+ "summary": "A property that sets number of rows to skip at the beginning of each source file."
11431
+ },
11432
+ "name": "SERIALIZATION_NULL_FORMAT"
11433
+ },
11434
+ {
11435
+ "docs": {
11436
+ "remarks": "This property is ignored for other data formats.",
11437
+ "stability": "experimental",
11438
+ "summary": "A property that sets the column mapping type for tables that use ORC data format."
11439
+ },
11440
+ "name": "ORC_SCHEMA_RESOLUTION"
11441
+ },
11442
+ {
11443
+ "docs": {
11444
+ "remarks": "When 'write.parallel' is set to off, CREATE EXTERNAL TABLE AS writes to one or more data files serially onto Amazon S3. This table property also applies to any subsequent INSERT statement into the same external table.",
11445
+ "stability": "experimental",
11446
+ "summary": "A property that sets whether CREATE EXTERNAL TABLE AS should write data in parallel."
11447
+ },
11448
+ "name": "WRITE_PARALLEL"
11449
+ },
11450
+ {
11451
+ "docs": {
11452
+ "remarks": "The size must be a valid integer between 5 and 6200. The default maximum file size is 6,200 MB. This table property also applies to any subsequent INSERT statement into the same external table.",
11453
+ "stability": "experimental",
11454
+ "summary": "A property that sets the maximum size (in MB) of each file written to Amazon S3 by CREATE EXTERNAL TABLE AS."
11455
+ },
11456
+ "name": "WRITE_MAX_FILESIZE_MB"
11457
+ },
11458
+ {
11459
+ "docs": {
11460
+ "stability": "experimental",
11461
+ "summary": "You can specify an AWS Key Management Service key to enable Server–Side Encryption (SSE) for Amazon S3 objects."
11462
+ },
11463
+ "name": "WRITE_KMS_KEY_ID"
11464
+ }
11465
+ ],
11466
+ "name": "StorageParameters",
11467
+ "symbolId": "lib/storage-parameter:StorageParameters"
11468
+ },
11469
+ "@aws-cdk/aws-glue-alpha.SurplusBytesHandlingAction": {
11470
+ "assembly": "@aws-cdk/aws-glue-alpha",
11471
+ "docs": {
11472
+ "remarks": "By default, Redshift Spectrum sets the value to null for data that exceeds the width of the column.",
11473
+ "see": "https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_EXTERNAL_TABLE.html#r_CREATE_EXTERNAL_TABLE-parameters - under _\"TABLE PROPERTIES\"_ > _\"surplus_bytes_handling\"_",
11474
+ "stability": "experimental",
11475
+ "summary": "Specifies how to handle data being loaded that exceeds the length of the data type defined for columns containing VARBYTE data."
11476
+ },
11477
+ "fqn": "@aws-cdk/aws-glue-alpha.SurplusBytesHandlingAction",
11478
+ "kind": "enum",
11479
+ "locationInModule": {
11480
+ "filename": "lib/storage-parameter.ts",
11481
+ "line": 92
11482
+ },
11483
+ "members": [
11484
+ {
11485
+ "docs": {
11486
+ "stability": "experimental",
11487
+ "summary": "Replaces data that exceeds the column width with null."
11488
+ },
11489
+ "name": "SET_TO_NULL"
11490
+ },
11491
+ {
11492
+ "docs": {
11493
+ "stability": "experimental",
11494
+ "summary": "Doesn't perform surplus byte handling."
11495
+ },
11496
+ "name": "DISABLED"
11497
+ },
11498
+ {
11499
+ "docs": {
11500
+ "stability": "experimental",
11501
+ "summary": "Cancels queries that return data exceeding the column width."
11502
+ },
11503
+ "name": "FAIL"
11504
+ },
11505
+ {
11506
+ "docs": {
11507
+ "stability": "experimental",
11508
+ "summary": "Drop all rows that contain data exceeding column width."
11509
+ },
11510
+ "name": "DROP_ROW"
11511
+ },
11512
+ {
11513
+ "docs": {
11514
+ "stability": "experimental",
11515
+ "summary": "Removes the characters that exceed the maximum number of characters defined for the column."
11516
+ },
11517
+ "name": "TRUNCATE"
11518
+ }
11519
+ ],
11520
+ "name": "SurplusBytesHandlingAction",
11521
+ "symbolId": "lib/storage-parameter:SurplusBytesHandlingAction"
11522
+ },
11523
+ "@aws-cdk/aws-glue-alpha.SurplusCharHandlingAction": {
11524
+ "assembly": "@aws-cdk/aws-glue-alpha",
11525
+ "docs": {
11526
+ "remarks": "By default, Redshift Spectrum sets the value to null for data that exceeds the width of the column.",
11527
+ "see": "https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_EXTERNAL_TABLE.html#r_CREATE_EXTERNAL_TABLE-parameters - under _\"TABLE PROPERTIES\"_ > _\"surplus_char_handling\"_",
11528
+ "stability": "experimental",
11529
+ "summary": "Specifies how to handle data being loaded that exceeds the length of the data type defined for columns containing VARCHAR, CHAR, or string data."
11530
+ },
11531
+ "fqn": "@aws-cdk/aws-glue-alpha.SurplusCharHandlingAction",
11532
+ "kind": "enum",
11533
+ "locationInModule": {
11534
+ "filename": "lib/storage-parameter.ts",
11535
+ "line": 124
11536
+ },
11537
+ "members": [
11538
+ {
11539
+ "docs": {
11540
+ "stability": "experimental",
11541
+ "summary": "Replaces data that exceeds the column width with null."
11542
+ },
11543
+ "name": "SET_TO_NULL"
11544
+ },
11545
+ {
11546
+ "docs": {
11547
+ "stability": "experimental",
11548
+ "summary": "Doesn't perform surplus character handling."
11549
+ },
11550
+ "name": "DISABLED"
11551
+ },
11552
+ {
11553
+ "docs": {
11554
+ "stability": "experimental",
11555
+ "summary": "Cancels queries that return data exceeding the column width."
11556
+ },
11557
+ "name": "FAIL"
11558
+ },
11559
+ {
11560
+ "docs": {
11561
+ "stability": "experimental",
11562
+ "summary": "Replaces each value in the row with null."
11563
+ },
11564
+ "name": "DROP_ROW"
11565
+ },
11566
+ {
11567
+ "docs": {
11568
+ "stability": "experimental",
11569
+ "summary": "Removes the characters that exceed the maximum number of characters defined for the column."
11570
+ },
11571
+ "name": "TRUNCATE"
11572
+ }
11573
+ ],
11574
+ "name": "SurplusCharHandlingAction",
11575
+ "symbolId": "lib/storage-parameter:SurplusCharHandlingAction"
11576
+ },
11577
+ "@aws-cdk/aws-glue-alpha.Table": {
11578
+ "assembly": "@aws-cdk/aws-glue-alpha",
11579
+ "base": "aws-cdk-lib.Resource",
11580
+ "docs": {
11581
+ "stability": "experimental",
11582
+ "summary": "A Glue table.",
11583
+ "example": "declare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n dataFormat: glue.DataFormat.JSON,\n});",
11584
+ "custom": {
11585
+ "exampleMetadata": "infused"
11586
+ }
11587
+ },
11588
+ "fqn": "@aws-cdk/aws-glue-alpha.Table",
11589
+ "initializer": {
11590
+ "docs": {
11591
+ "stability": "experimental"
11592
+ },
11593
+ "locationInModule": {
11594
+ "filename": "lib/table.ts",
11595
+ "line": 324
11596
+ },
11597
+ "parameters": [
11598
+ {
11599
+ "name": "scope",
11600
+ "type": {
11601
+ "fqn": "constructs.Construct"
11602
+ }
11603
+ },
11604
+ {
11605
+ "name": "id",
11606
+ "type": {
11607
+ "primitive": "string"
11608
+ }
11609
+ },
11610
+ {
11611
+ "name": "props",
11612
+ "type": {
11613
+ "fqn": "@aws-cdk/aws-glue-alpha.TableProps"
11614
+ }
11615
+ }
11616
+ ]
11617
+ },
11618
+ "interfaces": [
11619
+ "@aws-cdk/aws-glue-alpha.ITable"
11620
+ ],
11621
+ "kind": "class",
11622
+ "locationInModule": {
11623
+ "filename": "lib/table.ts",
11624
+ "line": 225
11625
+ },
11626
+ "methods": [
11627
+ {
11628
+ "docs": {
11629
+ "stability": "experimental"
11630
+ },
11631
+ "locationInModule": {
11632
+ "filename": "lib/table.ts",
11633
+ "line": 227
11634
+ },
11635
+ "name": "fromTableArn",
11636
+ "parameters": [
11637
+ {
11638
+ "name": "scope",
11639
+ "type": {
11640
+ "fqn": "constructs.Construct"
11641
+ }
11642
+ },
11643
+ {
11644
+ "name": "id",
11645
+ "type": {
11646
+ "primitive": "string"
11647
+ }
11648
+ },
11649
+ {
11650
+ "name": "tableArn",
11651
+ "type": {
11652
+ "primitive": "string"
11653
+ }
11654
+ }
11655
+ ],
11656
+ "returns": {
11657
+ "type": {
11658
+ "fqn": "@aws-cdk/aws-glue-alpha.ITable"
11659
+ }
11660
+ },
11661
+ "static": true
11662
+ },
11663
+ {
11664
+ "docs": {
11665
+ "stability": "experimental",
11666
+ "summary": "Creates a Table construct that represents an external table."
11667
+ },
11668
+ "locationInModule": {
11669
+ "filename": "lib/table.ts",
11670
+ "line": 243
11671
+ },
11672
+ "name": "fromTableAttributes",
11673
+ "parameters": [
11674
+ {
11675
+ "docs": {
11676
+ "summary": "The scope creating construct (usually `this`)."
11677
+ },
11678
+ "name": "scope",
10709
11679
  "type": {
10710
11680
  "fqn": "constructs.Construct"
10711
11681
  }
@@ -10745,7 +11715,7 @@
10745
11715
  },
10746
11716
  "locationInModule": {
10747
11717
  "filename": "lib/table.ts",
10748
- "line": 359
11718
+ "line": 409
10749
11719
  },
10750
11720
  "name": "addPartitionIndex",
10751
11721
  "parameters": [
@@ -10764,7 +11734,7 @@
10764
11734
  },
10765
11735
  "locationInModule": {
10766
11736
  "filename": "lib/table.ts",
10767
- "line": 458
11737
+ "line": 508
10768
11738
  },
10769
11739
  "name": "grant",
10770
11740
  "parameters": [
@@ -10799,7 +11769,7 @@
10799
11769
  },
10800
11770
  "locationInModule": {
10801
11771
  "filename": "lib/table.ts",
10802
- "line": 424
11772
+ "line": 474
10803
11773
  },
10804
11774
  "name": "grantRead",
10805
11775
  "parameters": [
@@ -10826,7 +11796,7 @@
10826
11796
  },
10827
11797
  "locationInModule": {
10828
11798
  "filename": "lib/table.ts",
10829
- "line": 448
11799
+ "line": 498
10830
11800
  },
10831
11801
  "name": "grantReadWrite",
10832
11802
  "parameters": [
@@ -10854,7 +11824,7 @@
10854
11824
  },
10855
11825
  "locationInModule": {
10856
11826
  "filename": "lib/table.ts",
10857
- "line": 470
11827
+ "line": 520
10858
11828
  },
10859
11829
  "name": "grantToUnderlyingResources",
10860
11830
  "parameters": [
@@ -10889,7 +11859,7 @@
10889
11859
  },
10890
11860
  "locationInModule": {
10891
11861
  "filename": "lib/table.ts",
10892
- "line": 436
11862
+ "line": 486
10893
11863
  },
10894
11864
  "name": "grantWrite",
10895
11865
  "parameters": [
@@ -10920,7 +11890,7 @@
10920
11890
  "immutable": true,
10921
11891
  "locationInModule": {
10922
11892
  "filename": "lib/table.ts",
10923
- "line": 239
11893
+ "line": 275
10924
11894
  },
10925
11895
  "name": "bucket",
10926
11896
  "type": {
@@ -10935,7 +11905,7 @@
10935
11905
  "immutable": true,
10936
11906
  "locationInModule": {
10937
11907
  "filename": "lib/table.ts",
10938
- "line": 264
11908
+ "line": 300
10939
11909
  },
10940
11910
  "name": "columns",
10941
11911
  "type": {
@@ -10955,7 +11925,7 @@
10955
11925
  "immutable": true,
10956
11926
  "locationInModule": {
10957
11927
  "filename": "lib/table.ts",
10958
- "line": 224
11928
+ "line": 260
10959
11929
  },
10960
11930
  "name": "compressed",
10961
11931
  "type": {
@@ -10970,7 +11940,7 @@
10970
11940
  "immutable": true,
10971
11941
  "locationInModule": {
10972
11942
  "filename": "lib/table.ts",
10973
- "line": 219
11943
+ "line": 255
10974
11944
  },
10975
11945
  "name": "database",
10976
11946
  "type": {
@@ -10985,7 +11955,7 @@
10985
11955
  "immutable": true,
10986
11956
  "locationInModule": {
10987
11957
  "filename": "lib/table.ts",
10988
- "line": 259
11958
+ "line": 295
10989
11959
  },
10990
11960
  "name": "dataFormat",
10991
11961
  "type": {
@@ -11000,7 +11970,7 @@
11000
11970
  "immutable": true,
11001
11971
  "locationInModule": {
11002
11972
  "filename": "lib/table.ts",
11003
- "line": 229
11973
+ "line": 265
11004
11974
  },
11005
11975
  "name": "encryption",
11006
11976
  "type": {
@@ -11015,7 +11985,7 @@
11015
11985
  "immutable": true,
11016
11986
  "locationInModule": {
11017
11987
  "filename": "lib/table.ts",
11018
- "line": 244
11988
+ "line": 280
11019
11989
  },
11020
11990
  "name": "s3Prefix",
11021
11991
  "type": {
@@ -11030,7 +12000,7 @@
11030
12000
  "immutable": true,
11031
12001
  "locationInModule": {
11032
12002
  "filename": "lib/table.ts",
11033
- "line": 254
12003
+ "line": 290
11034
12004
  },
11035
12005
  "name": "tableArn",
11036
12006
  "overrides": "@aws-cdk/aws-glue-alpha.ITable",
@@ -11046,7 +12016,7 @@
11046
12016
  "immutable": true,
11047
12017
  "locationInModule": {
11048
12018
  "filename": "lib/table.ts",
11049
- "line": 249
12019
+ "line": 285
11050
12020
  },
11051
12021
  "name": "tableName",
11052
12022
  "overrides": "@aws-cdk/aws-glue-alpha.ITable",
@@ -11063,7 +12033,7 @@
11063
12033
  "immutable": true,
11064
12034
  "locationInModule": {
11065
12035
  "filename": "lib/table.ts",
11066
- "line": 234
12036
+ "line": 270
11067
12037
  },
11068
12038
  "name": "encryptionKey",
11069
12039
  "optional": true,
@@ -11079,7 +12049,7 @@
11079
12049
  "immutable": true,
11080
12050
  "locationInModule": {
11081
12051
  "filename": "lib/table.ts",
11082
- "line": 274
12052
+ "line": 310
11083
12053
  },
11084
12054
  "name": "partitionIndexes",
11085
12055
  "optional": true,
@@ -11100,7 +12070,7 @@
11100
12070
  "immutable": true,
11101
12071
  "locationInModule": {
11102
12072
  "filename": "lib/table.ts",
11103
- "line": 269
12073
+ "line": 305
11104
12074
  },
11105
12075
  "name": "partitionKeys",
11106
12076
  "optional": true,
@@ -11112,6 +12082,27 @@
11112
12082
  "kind": "array"
11113
12083
  }
11114
12084
  }
12085
+ },
12086
+ {
12087
+ "docs": {
12088
+ "stability": "experimental",
12089
+ "summary": "The tables' storage descriptor properties."
12090
+ },
12091
+ "immutable": true,
12092
+ "locationInModule": {
12093
+ "filename": "lib/table.ts",
12094
+ "line": 315
12095
+ },
12096
+ "name": "storageParameters",
12097
+ "optional": true,
12098
+ "type": {
12099
+ "collection": {
12100
+ "elementtype": {
12101
+ "fqn": "@aws-cdk/aws-glue-alpha.StorageParameter"
12102
+ },
12103
+ "kind": "array"
12104
+ }
12105
+ }
11115
12106
  }
11116
12107
  ],
11117
12108
  "symbolId": "lib/table:Table"
@@ -11130,7 +12121,7 @@
11130
12121
  "kind": "interface",
11131
12122
  "locationInModule": {
11132
12123
  "filename": "lib/table.ts",
11133
- "line": 76
12124
+ "line": 77
11134
12125
  },
11135
12126
  "name": "TableAttributes",
11136
12127
  "properties": [
@@ -11142,7 +12133,7 @@
11142
12133
  "immutable": true,
11143
12134
  "locationInModule": {
11144
12135
  "filename": "lib/table.ts",
11145
- "line": 77
12136
+ "line": 78
11146
12137
  },
11147
12138
  "name": "tableArn",
11148
12139
  "type": {
@@ -11157,7 +12148,7 @@
11157
12148
  "immutable": true,
11158
12149
  "locationInModule": {
11159
12150
  "filename": "lib/table.ts",
11160
- "line": 78
12151
+ "line": 79
11161
12152
  },
11162
12153
  "name": "tableName",
11163
12154
  "type": {
@@ -11182,7 +12173,7 @@
11182
12173
  "kind": "enum",
11183
12174
  "locationInModule": {
11184
12175
  "filename": "lib/table.ts",
11185
- "line": 48
12176
+ "line": 49
11186
12177
  },
11187
12178
  "members": [
11188
12179
  {
@@ -11234,7 +12225,7 @@
11234
12225
  "kind": "interface",
11235
12226
  "locationInModule": {
11236
12227
  "filename": "lib/table.ts",
11237
- "line": 81
12228
+ "line": 82
11238
12229
  },
11239
12230
  "name": "TableProps",
11240
12231
  "properties": [
@@ -11247,7 +12238,7 @@
11247
12238
  "immutable": true,
11248
12239
  "locationInModule": {
11249
12240
  "filename": "lib/table.ts",
11250
- "line": 118
12241
+ "line": 119
11251
12242
  },
11252
12243
  "name": "columns",
11253
12244
  "type": {
@@ -11268,7 +12259,7 @@
11268
12259
  "immutable": true,
11269
12260
  "locationInModule": {
11270
12261
  "filename": "lib/table.ts",
11271
- "line": 99
12262
+ "line": 100
11272
12263
  },
11273
12264
  "name": "database",
11274
12265
  "type": {
@@ -11284,7 +12275,7 @@
11284
12275
  "immutable": true,
11285
12276
  "locationInModule": {
11286
12277
  "filename": "lib/table.ts",
11287
- "line": 139
12278
+ "line": 140
11288
12279
  },
11289
12280
  "name": "dataFormat",
11290
12281
  "type": {
@@ -11301,7 +12292,7 @@
11301
12292
  "immutable": true,
11302
12293
  "locationInModule": {
11303
12294
  "filename": "lib/table.ts",
11304
- "line": 106
12295
+ "line": 107
11305
12296
  },
11306
12297
  "name": "bucket",
11307
12298
  "optional": true,
@@ -11319,7 +12310,7 @@
11319
12310
  "immutable": true,
11320
12311
  "locationInModule": {
11321
12312
  "filename": "lib/table.ts",
11322
- "line": 146
12313
+ "line": 147
11323
12314
  },
11324
12315
  "name": "compressed",
11325
12316
  "optional": true,
@@ -11337,7 +12328,7 @@
11337
12328
  "immutable": true,
11338
12329
  "locationInModule": {
11339
12330
  "filename": "lib/table.ts",
11340
- "line": 94
12331
+ "line": 95
11341
12332
  },
11342
12333
  "name": "description",
11343
12334
  "optional": true,
@@ -11356,7 +12347,7 @@
11356
12347
  "immutable": true,
11357
12348
  "locationInModule": {
11358
12349
  "filename": "lib/table.ts",
11359
- "line": 183
12350
+ "line": 184
11360
12351
  },
11361
12352
  "name": "enablePartitionFiltering",
11362
12353
  "optional": true,
@@ -11375,7 +12366,7 @@
11375
12366
  "immutable": true,
11376
12367
  "locationInModule": {
11377
12368
  "filename": "lib/table.ts",
11378
- "line": 158
12369
+ "line": 159
11379
12370
  },
11380
12371
  "name": "encryption",
11381
12372
  "optional": true,
@@ -11394,7 +12385,7 @@
11394
12385
  "immutable": true,
11395
12386
  "locationInModule": {
11396
12387
  "filename": "lib/table.ts",
11397
- "line": 167
12388
+ "line": 168
11398
12389
  },
11399
12390
  "name": "encryptionKey",
11400
12391
  "optional": true,
@@ -11413,7 +12404,7 @@
11413
12404
  "immutable": true,
11414
12405
  "locationInModule": {
11415
12406
  "filename": "lib/table.ts",
11416
- "line": 134
12407
+ "line": 135
11417
12408
  },
11418
12409
  "name": "partitionIndexes",
11419
12410
  "optional": true,
@@ -11436,7 +12427,7 @@
11436
12427
  "immutable": true,
11437
12428
  "locationInModule": {
11438
12429
  "filename": "lib/table.ts",
11439
- "line": 125
12430
+ "line": 126
11440
12431
  },
11441
12432
  "name": "partitionKeys",
11442
12433
  "optional": true,
@@ -11459,7 +12450,7 @@
11459
12450
  "immutable": true,
11460
12451
  "locationInModule": {
11461
12452
  "filename": "lib/table.ts",
11462
- "line": 113
12453
+ "line": 114
11463
12454
  },
11464
12455
  "name": "s3Prefix",
11465
12456
  "optional": true,
@@ -11467,6 +12458,32 @@
11467
12458
  "primitive": "string"
11468
12459
  }
11469
12460
  },
12461
+ {
12462
+ "abstract": true,
12463
+ "docs": {
12464
+ "default": "- The parameter is not defined",
12465
+ "example": " declare const glueDatabase: glue.IDatabase;\n const table = new glue.Table(this, 'Table', {\n storageParameters: [\n glue.StorageParameter.skipHeaderLineCount(1),\n glue.StorageParameter.compressionType(glue.CompressionType.GZIP),\n glue.StorageParameter.custom('foo', 'bar'), // Will have no effect\n glue.StorageParameter.custom('separatorChar', ','), // Will describe the separator char used in the data\n glue.StorageParameter.custom(glue.StorageParameters.WRITE_PARALLEL, 'off'),\n ],\n // ...\n database: glueDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.CSV,\n });",
12466
+ "remarks": "These properties help describe the format of the data that is stored within the crawled data sources.\n\nThe key/value pairs that are allowed to be submitted are not limited, however their functionality is not guaranteed.\n\nSome keys will be auto-populated by glue crawlers, however, you can override them by specifying the key and value in this property.",
12467
+ "see": "https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_EXTERNAL_TABLE.html#r_CREATE_EXTERNAL_TABLE-parameters - under _\"TABLE PROPERTIES\"_",
12468
+ "stability": "experimental",
12469
+ "summary": "The user-supplied properties for the description of the physical storage of this table."
12470
+ },
12471
+ "immutable": true,
12472
+ "locationInModule": {
12473
+ "filename": "lib/table.ts",
12474
+ "line": 219
12475
+ },
12476
+ "name": "storageParameters",
12477
+ "optional": true,
12478
+ "type": {
12479
+ "collection": {
12480
+ "elementtype": {
12481
+ "fqn": "@aws-cdk/aws-glue-alpha.StorageParameter"
12482
+ },
12483
+ "kind": "array"
12484
+ }
12485
+ }
12486
+ },
11470
12487
  {
11471
12488
  "abstract": true,
11472
12489
  "docs": {
@@ -11477,7 +12494,7 @@
11477
12494
  "immutable": true,
11478
12495
  "locationInModule": {
11479
12496
  "filename": "lib/table.ts",
11480
- "line": 174
12497
+ "line": 175
11481
12498
  },
11482
12499
  "name": "storedAsSubDirectories",
11483
12500
  "optional": true,
@@ -11495,7 +12512,7 @@
11495
12512
  "immutable": true,
11496
12513
  "locationInModule": {
11497
12514
  "filename": "lib/table.ts",
11498
- "line": 87
12515
+ "line": 88
11499
12516
  },
11500
12517
  "name": "tableName",
11501
12518
  "optional": true,
@@ -11575,7 +12592,7 @@
11575
12592
  "kind": "class",
11576
12593
  "locationInModule": {
11577
12594
  "filename": "lib/job.ts",
11578
- "line": 19
12595
+ "line": 20
11579
12596
  },
11580
12597
  "methods": [
11581
12598
  {
@@ -11585,7 +12602,7 @@
11585
12602
  },
11586
12603
  "locationInModule": {
11587
12604
  "filename": "lib/job.ts",
11588
- "line": 59
12605
+ "line": 60
11589
12606
  },
11590
12607
  "name": "of",
11591
12608
  "parameters": [
@@ -11618,7 +12635,7 @@
11618
12635
  "immutable": true,
11619
12636
  "locationInModule": {
11620
12637
  "filename": "lib/job.ts",
11621
- "line": 48
12638
+ "line": 49
11622
12639
  },
11623
12640
  "name": "G_025X",
11624
12641
  "static": true,
@@ -11636,7 +12653,7 @@
11636
12653
  "immutable": true,
11637
12654
  "locationInModule": {
11638
12655
  "filename": "lib/job.ts",
11639
- "line": 28
12656
+ "line": 29
11640
12657
  },
11641
12658
  "name": "G_1X",
11642
12659
  "static": true,
@@ -11654,7 +12671,7 @@
11654
12671
  "immutable": true,
11655
12672
  "locationInModule": {
11656
12673
  "filename": "lib/job.ts",
11657
- "line": 33
12674
+ "line": 34
11658
12675
  },
11659
12676
  "name": "G_2X",
11660
12677
  "static": true,
@@ -11672,7 +12689,7 @@
11672
12689
  "immutable": true,
11673
12690
  "locationInModule": {
11674
12691
  "filename": "lib/job.ts",
11675
- "line": 38
12692
+ "line": 39
11676
12693
  },
11677
12694
  "name": "G_4X",
11678
12695
  "static": true,
@@ -11690,7 +12707,7 @@
11690
12707
  "immutable": true,
11691
12708
  "locationInModule": {
11692
12709
  "filename": "lib/job.ts",
11693
- "line": 43
12710
+ "line": 44
11694
12711
  },
11695
12712
  "name": "G_8X",
11696
12713
  "static": true,
@@ -11707,7 +12724,7 @@
11707
12724
  "immutable": true,
11708
12725
  "locationInModule": {
11709
12726
  "filename": "lib/job.ts",
11710
- "line": 23
12727
+ "line": 24
11711
12728
  },
11712
12729
  "name": "STANDARD",
11713
12730
  "static": true,
@@ -11725,7 +12742,7 @@
11725
12742
  "immutable": true,
11726
12743
  "locationInModule": {
11727
12744
  "filename": "lib/job.ts",
11728
- "line": 53
12745
+ "line": 54
11729
12746
  },
11730
12747
  "name": "Z_2X",
11731
12748
  "static": true,
@@ -11741,7 +12758,7 @@
11741
12758
  "immutable": true,
11742
12759
  "locationInModule": {
11743
12760
  "filename": "lib/job.ts",
11744
- "line": 66
12761
+ "line": 67
11745
12762
  },
11746
12763
  "name": "name",
11747
12764
  "type": {
@@ -11750,8 +12767,41 @@
11750
12767
  }
11751
12768
  ],
11752
12769
  "symbolId": "lib/job:WorkerType"
12770
+ },
12771
+ "@aws-cdk/aws-glue-alpha.WriteParallel": {
12772
+ "assembly": "@aws-cdk/aws-glue-alpha",
12773
+ "docs": {
12774
+ "remarks": "By default, Redshift Spectrum sets the value to null for data that exceeds the width of the column.",
12775
+ "see": "https://docs.aws.amazon.com/redshift/latest/dg/r_CREATE_EXTERNAL_TABLE.html#r_CREATE_EXTERNAL_TABLE-parameters - under _\"TABLE PROPERTIES\"_ > _\"surplus_char_handling\"_",
12776
+ "stability": "experimental",
12777
+ "summary": "Specifies how to handle data being loaded that exceeds the length of the data type defined for columns containing VARCHAR, CHAR, or string data."
12778
+ },
12779
+ "fqn": "@aws-cdk/aws-glue-alpha.WriteParallel",
12780
+ "kind": "enum",
12781
+ "locationInModule": {
12782
+ "filename": "lib/storage-parameter.ts",
12783
+ "line": 183
12784
+ },
12785
+ "members": [
12786
+ {
12787
+ "docs": {
12788
+ "stability": "experimental",
12789
+ "summary": "Write data in parallel."
12790
+ },
12791
+ "name": "ON"
12792
+ },
12793
+ {
12794
+ "docs": {
12795
+ "stability": "experimental",
12796
+ "summary": "Write data serially."
12797
+ },
12798
+ "name": "OFF"
12799
+ }
12800
+ ],
12801
+ "name": "WriteParallel",
12802
+ "symbolId": "lib/storage-parameter:WriteParallel"
11753
12803
  }
11754
12804
  },
11755
- "version": "2.89.0-alpha.0",
12805
+ "version": "2.91.0-alpha.0",
11756
12806
  "fingerprint": "**********"
11757
12807
  }