@aws-cdk/aws-glue-alpha 2.87.0-alpha.0 → 2.88.0-alpha.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/.jsii CHANGED
@@ -8,7 +8,7 @@
8
8
  "url": "https://aws.amazon.com"
9
9
  },
10
10
  "dependencies": {
11
- "aws-cdk-lib": "2.87.0",
11
+ "aws-cdk-lib": "2.88.0",
12
12
  "constructs": "^10.0.0"
13
13
  },
14
14
  "dependencyClosure": {
@@ -3492,7 +3492,7 @@
3492
3492
  "stability": "experimental"
3493
3493
  },
3494
3494
  "homepage": "https://github.com/aws/aws-cdk",
3495
- "jsiiVersion": "5.0.11 (build 5e2d6be)",
3495
+ "jsiiVersion": "5.1.8 (build acd3d7c)",
3496
3496
  "keywords": [
3497
3497
  "aws",
3498
3498
  "cdk",
@@ -3513,7 +3513,7 @@
3513
3513
  },
3514
3514
  "name": "@aws-cdk/aws-glue-alpha",
3515
3515
  "readme": {
3516
- "markdown": "# AWS Glue Construct Library\n<!--BEGIN STABILITY BANNER-->\n\n---\n\n![cdk-constructs: Experimental](https://img.shields.io/badge/cdk--constructs-experimental-important.svg?style=for-the-badge)\n\n> The APIs of higher level constructs in this module are experimental and under active development.\n> They are subject to non-backward compatible changes or removal in any future version. These are\n> not subject to the [Semantic Versioning](https://semver.org/) model and breaking changes will be\n> announced in the release notes. This means that while you may use them, you may need to update\n> your source code when upgrading to a newer version of this package.\n\n---\n\n<!--END STABILITY BANNER-->\n\nThis module is part of the [AWS Cloud Development Kit](https://github.com/aws/aws-cdk) project.\n\n## Job\n\nA `Job` encapsulates a script that connects to data sources, processes them, and then writes output to a data target.\n\nThere are 3 types of jobs supported by AWS Glue: Spark ETL, Spark Streaming, and Python Shell jobs.\n\nThe `glue.JobExecutable` allows you to specify the type of job, the language to use and the code assets required by the job.\n\n`glue.Code` allows you to refer to the different code assets required by the job, either from an existing S3 location or from a local file path.\n\n`glue.ExecutionClass` allows you to specify `FLEX` or `STANDARD`. `FLEX` is appropriate for non-urgent jobs such as pre-production jobs, testing, and one-time data loads.\n\n### Spark Jobs\n\nThese jobs run in an Apache Spark environment managed by AWS Glue.\n\n#### ETL Jobs\n\nAn ETL job processes data in batches using Apache Spark.\n\n```ts\ndeclare const bucket: s3.Bucket;\nnew glue.Job(this, 'ScalaSparkEtlJob', {\n executable: glue.JobExecutable.scalaEtl({\n glueVersion: glue.GlueVersion.V4_0,\n script: glue.Code.fromBucket(bucket, 'src/com/example/HelloWorld.scala'),\n className: 'com.example.HelloWorld',\n extraJars: [glue.Code.fromBucket(bucket, 'jars/HelloWorld.jar')],\n }),\n workerType: glue.WorkerType.G_8X,\n description: 'an example Scala ETL job',\n});\n```\n\n#### Streaming Jobs\n\nA Streaming job is similar to an ETL job, except that it performs ETL on data streams. It uses the Apache Spark Structured Streaming framework. Some Spark job features are not available to streaming ETL jobs.\n\n```ts\nnew glue.Job(this, 'PythonSparkStreamingJob', {\n executable: glue.JobExecutable.pythonStreaming({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n description: 'an example Python Streaming job',\n});\n```\n\n### Python Shell Jobs\n\nA Python shell job runs Python scripts as a shell and supports a Python version that depends on the AWS Glue version you are using.\nThis can be used to schedule and run tasks that don't require an Apache Spark environment. Currently, three flavors are supported:\n\n* PythonVersion.TWO (2.7; EOL)\n* PythonVersion.THREE (3.6)\n* PythonVersion.THREE_NINE (3.9)\n\n```ts\ndeclare const bucket: s3.Bucket;\nnew glue.Job(this, 'PythonShellJob', {\n executable: glue.JobExecutable.pythonShell({\n glueVersion: glue.GlueVersion.V1_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromBucket(bucket, 'script.py'),\n }),\n description: 'an example Python Shell job',\n});\n```\n\n### Ray Jobs\n\nThese jobs run in a Ray environment managed by AWS Glue.\n\n```ts\nnew glue.Job(this, 'RayJob', {\n executable: glue.JobExecutable.pythonRay({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE_NINE,\n runtime: glue.Runtime.RAY_TWO_FOUR,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n workerType: glue.WorkerType.Z_2X,\n workerCount: 2,\n description: 'an example Ray job'\n});\n```\n\nSee [documentation](https://docs.aws.amazon.com/glue/latest/dg/add-job.html) for more information on adding jobs in Glue.\n\n## Connection\n\nA `Connection` allows Glue jobs, crawlers and development endpoints to access certain types of data stores. For example, to create a network connection to connect to a data source within a VPC:\n\n```ts\ndeclare const securityGroup: ec2.SecurityGroup;\ndeclare const subnet: ec2.Subnet;\nnew glue.Connection(this, 'MyConnection', {\n type: glue.ConnectionType.NETWORK,\n // The security groups granting AWS Glue inbound access to the data source within the VPC\n securityGroups: [securityGroup],\n // The VPC subnet which contains the data source\n subnet,\n});\n```\n\nFor RDS `Connection` by JDBC, it is recommended to manage credentials using AWS Secrets Manager. To use Secret, specify `SECRET_ID` in `properties` like the following code. Note that in this case, the subnet must have a route to the AWS Secrets Manager VPC endpoint or to the AWS Secrets Manager endpoint through a NAT gateway.\n\n```ts\ndeclare const securityGroup: ec2.SecurityGroup;\ndeclare const subnet: ec2.Subnet;\ndeclare const db: rds.DatabaseCluster;\nnew glue.Connection(this, \"RdsConnection\", {\n type: glue.ConnectionType.JDBC,\n securityGroups: [securityGroup],\n subnet,\n properties: {\n JDBC_CONNECTION_URL: `jdbc:mysql://${db.clusterEndpoint.socketAddress}/databasename`,\n JDBC_ENFORCE_SSL: \"false\",\n SECRET_ID: db.secret!.secretName,\n },\n});\n```\n\nIf you need to use a connection type that doesn't exist as a static member on `ConnectionType`, you can instantiate a `ConnectionType` object, e.g: `new glue.ConnectionType('NEW_TYPE')`.\n\nSee [Adding a Connection to Your Data Store](https://docs.aws.amazon.com/glue/latest/dg/populate-add-connection.html) and [Connection Structure](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-catalog-connections.html#aws-glue-api-catalog-connections-Connection) documentation for more information on the supported data stores and their configurations.\n\n## SecurityConfiguration\n\nA `SecurityConfiguration` is a set of security properties that can be used by AWS Glue to encrypt data at rest.\n\n```ts\nnew glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n },\n jobBookmarksEncryption: {\n mode: glue.JobBookmarksEncryptionMode.CLIENT_SIDE_KMS,\n },\n s3Encryption: {\n mode: glue.S3EncryptionMode.KMS,\n },\n});\n```\n\nBy default, a shared KMS key is created for use with the encryption configurations that require one. You can also supply your own key for each encryption config, for example, for CloudWatch encryption:\n\n```ts\ndeclare const key: kms.Key;\nnew glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n kmsKey: key,\n },\n});\n```\n\nSee [documentation](https://docs.aws.amazon.com/glue/latest/dg/encryption-security-configuration.html) for more info for Glue encrypting data written by Crawlers, Jobs, and Development Endpoints.\n\n## Database\n\nA `Database` is a logical grouping of `Tables` in the Glue Catalog.\n\n```ts\nnew glue.Database(this, 'MyDatabase');\n```\n\n## Table\n\nA Glue table describes a table of data in S3: its structure (column names and types), location of data (S3 objects with a common prefix in a S3 bucket), and format for the files (Json, Avro, Parquet, etc.):\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }, {\n name: 'col2',\n type: glue.Schema.array(glue.Schema.STRING),\n comment: 'col2 is an array of strings' // comment is optional\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nBy default, a S3 bucket will be created to store the table's data but you can manually pass the `bucket` and `s3Prefix`:\n\n```ts\ndeclare const myBucket: s3.Bucket;\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n bucket: myBucket,\n s3Prefix: 'my-table/',\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nBy default, an S3 bucket will be created to store the table's data and stored in the bucket root. You can also manually pass the `bucket` and `s3Prefix`:\n\n### Partition Keys\n\nTo improve query performance, a table can specify `partitionKeys` on which data is stored and queried separately. For example, you might partition a table by `year` and `month` to optimize queries based on a time window:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n### Partition Indexes\n\nAnother way to improve query performance is to specify partition indexes. If no partition indexes are\npresent on the table, AWS Glue loads all partitions of the table and filters the loaded partitions using\nthe query expression. The query takes more time to run as the number of partitions increase. With an\nindex, the query will try to fetch a subset of the partitions instead of loading all partitions of the\ntable.\n\nThe keys of a partition index must be a subset of the partition keys of the table. You can have a\nmaximum of 3 partition indexes per table. To specify a partition index, you can use the `partitionIndexes`\nproperty:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n partitionIndexes: [{\n indexName: 'my-index', // optional\n keyNames: ['year'],\n }], // supply up to 3 indexes\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nAlternatively, you can call the `addPartitionIndex()` function on a table:\n\n```ts\ndeclare const myTable: glue.Table;\nmyTable.addPartitionIndex({\n indexName: 'my-index',\n keyNames: ['year'],\n});\n```\n\n### Partition Filtering\n\nIf you have a table with a large number of partitions that grows over time, consider using AWS Glue partition indexing and filtering.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n dataFormat: glue.DataFormat.JSON,\n enablePartitionFiltering: true,\n});\n```\n\n## [Encryption](https://docs.aws.amazon.com/athena/latest/ug/encryption.html)\n\nYou can enable encryption on a Table's data:\n\n* [S3Managed](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html) - (default) Server side encryption (`SSE-S3`) with an Amazon S3-managed key.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.S3_MANAGED,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [Kms](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html) - Server-side encryption (`SSE-KMS`) with an AWS KMS Key managed by the account owner.\n\n```ts\ndeclare const myDatabase: glue.Database;\n// KMS key is created automatically\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n\n// with an explicit KMS key\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS,\n encryptionKey: new kms.Key(this, 'MyKey'),\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [KmsManaged](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html) - Server-side encryption (`SSE-KMS`), like `Kms`, except with an AWS KMS Key managed by the AWS Key Management Service.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS_MANAGED,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [ClientSideKms](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html#client-side-encryption-kms-managed-master-key-intro) - Client-side encryption (`CSE-KMS`) with an AWS KMS Key managed by the account owner.\n\n```ts\ndeclare const myDatabase: glue.Database;\n// KMS key is created automatically\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.CLIENT_SIDE_KMS,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n\n// with an explicit KMS key\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.CLIENT_SIDE_KMS,\n encryptionKey: new kms.Key(this, 'MyKey'),\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n*Note: you cannot provide a `Bucket` when creating the `Table` if you wish to use server-side encryption (`KMS`, `KMS_MANAGED` or `S3_MANAGED`)*.\n\n## Types\n\nA table's schema is a collection of columns, each of which have a `name` and a `type`. Types are recursive structures, consisting of primitive and complex types:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n columns: [{\n name: 'primitive_column',\n type: glue.Schema.STRING,\n }, {\n name: 'array_column',\n type: glue.Schema.array(glue.Schema.INTEGER),\n comment: 'array<integer>',\n }, {\n name: 'map_column',\n type: glue.Schema.map(\n glue.Schema.STRING,\n glue.Schema.TIMESTAMP),\n comment: 'map<string,string>',\n }, {\n name: 'struct_column',\n type: glue.Schema.struct([{\n name: 'nested_column',\n type: glue.Schema.DATE,\n comment: 'nested comment',\n }]),\n comment: \"struct<nested_column:date COMMENT 'nested comment'>\",\n }],\n // ...\n database: myDatabase,\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n### Primitives\n\n#### Numeric\n\n| Name \t| Type \t| Comments |\n|-----------\t|----------\t|------------------------------------------------------------------------------------------------------------------\t|\n| FLOAT \t| Constant \t| A 32-bit single-precision floating point number |\n| INTEGER \t| Constant \t| A 32-bit signed value in two's complement format, with a minimum value of -2^31 and a maximum value of 2^31-1 \t|\n| DOUBLE \t| Constant \t| A 64-bit double-precision floating point number |\n| BIG_INT \t| Constant \t| A 64-bit signed INTEGER in two’s complement format, with a minimum value of -2^63 and a maximum value of 2^63 -1 |\n| SMALL_INT \t| Constant \t| A 16-bit signed INTEGER in two’s complement format, with a minimum value of -2^15 and a maximum value of 2^15-1 |\n| TINY_INT \t| Constant \t| A 8-bit signed INTEGER in two’s complement format, with a minimum value of -2^7 and a maximum value of 2^7-1 |\n\n#### Date and time\n\n| Name \t| Type \t| Comments \t|\n|-----------\t|----------\t|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------\t|\n| DATE \t| Constant \t| A date in UNIX format, such as YYYY-MM-DD. \t|\n| TIMESTAMP \t| Constant \t| Date and time instant in the UNiX format, such as yyyy-mm-dd hh:mm:ss[.f...]. For example, TIMESTAMP '2008-09-15 03:04:05.324'. This format uses the session time zone. \t|\n\n#### String\n\n| Name \t| Type \t| Comments \t|\n|--------------------------------------------\t|----------\t|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\t|\n| STRING \t| Constant \t| A string literal enclosed in single or double quotes \t|\n| decimal(precision: number, scale?: number) \t| Function \t| `precision` is the total number of digits. `scale` (optional) is the number of digits in fractional part with a default of 0. For example, use these type definitions: decimal(11,5), decimal(15) \t|\n| char(length: number) \t| Function \t| Fixed length character data, with a specified length between 1 and 255, such as char(10) \t|\n| varchar(length: number) \t| Function \t| Variable length character data, with a specified length between 1 and 65535, such as varchar(10) \t|\n\n#### Miscellaneous\n\n| Name \t| Type \t| Comments \t|\n|---------\t|----------\t|-------------------------------\t|\n| BOOLEAN \t| Constant \t| Values are `true` and `false` \t|\n| BINARY \t| Constant \t| Value is in binary \t|\n\n### Complex\n\n| Name \t| Type \t| Comments \t|\n|-------------------------------------\t|----------\t|-------------------------------------------------------------------\t|\n| array(itemType: Type) \t| Function \t| An array of some other type \t|\n| map(keyType: Type, valueType: Type) \t| Function \t| A map of some primitive key type to any value type \t|\n| struct(collumns: Column[]) \t| Function \t| Nested structure containing individually named and typed collumns \t|\n"
3516
+ "markdown": "# AWS Glue Construct Library\n<!--BEGIN STABILITY BANNER-->\n\n---\n\n![cdk-constructs: Experimental](https://img.shields.io/badge/cdk--constructs-experimental-important.svg?style=for-the-badge)\n\n> The APIs of higher level constructs in this module are experimental and under active development.\n> They are subject to non-backward compatible changes or removal in any future version. These are\n> not subject to the [Semantic Versioning](https://semver.org/) model and breaking changes will be\n> announced in the release notes. This means that while you may use them, you may need to update\n> your source code when upgrading to a newer version of this package.\n\n---\n\n<!--END STABILITY BANNER-->\n\nThis module is part of the [AWS Cloud Development Kit](https://github.com/aws/aws-cdk) project.\n\n## Job\n\nA `Job` encapsulates a script that connects to data sources, processes them, and then writes output to a data target.\n\nThere are 3 types of jobs supported by AWS Glue: Spark ETL, Spark Streaming, and Python Shell jobs.\n\nThe `glue.JobExecutable` allows you to specify the type of job, the language to use and the code assets required by the job.\n\n`glue.Code` allows you to refer to the different code assets required by the job, either from an existing S3 location or from a local file path.\n\n`glue.ExecutionClass` allows you to specify `FLEX` or `STANDARD`. `FLEX` is appropriate for non-urgent jobs such as pre-production jobs, testing, and one-time data loads.\n\n### Spark Jobs\n\nThese jobs run in an Apache Spark environment managed by AWS Glue.\n\n#### ETL Jobs\n\nAn ETL job processes data in batches using Apache Spark.\n\n```ts\ndeclare const bucket: s3.Bucket;\nnew glue.Job(this, 'ScalaSparkEtlJob', {\n executable: glue.JobExecutable.scalaEtl({\n glueVersion: glue.GlueVersion.V4_0,\n script: glue.Code.fromBucket(bucket, 'src/com/example/HelloWorld.scala'),\n className: 'com.example.HelloWorld',\n extraJars: [glue.Code.fromBucket(bucket, 'jars/HelloWorld.jar')],\n }),\n workerType: glue.WorkerType.G_8X,\n description: 'an example Scala ETL job',\n});\n```\n\n#### Streaming Jobs\n\nA Streaming job is similar to an ETL job, except that it performs ETL on data streams. It uses the Apache Spark Structured Streaming framework. Some Spark job features are not available to streaming ETL jobs.\n\n```ts\nnew glue.Job(this, 'PythonSparkStreamingJob', {\n executable: glue.JobExecutable.pythonStreaming({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n description: 'an example Python Streaming job',\n});\n```\n\n### Python Shell Jobs\n\nA Python shell job runs Python scripts as a shell and supports a Python version that depends on the AWS Glue version you are using.\nThis can be used to schedule and run tasks that don't require an Apache Spark environment. Currently, three flavors are supported:\n\n* PythonVersion.TWO (2.7; EOL)\n* PythonVersion.THREE (3.6)\n* PythonVersion.THREE_NINE (3.9)\n\n```ts\ndeclare const bucket: s3.Bucket;\nnew glue.Job(this, 'PythonShellJob', {\n executable: glue.JobExecutable.pythonShell({\n glueVersion: glue.GlueVersion.V1_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromBucket(bucket, 'script.py'),\n }),\n description: 'an example Python Shell job',\n});\n```\n\n### Ray Jobs\n\nThese jobs run in a Ray environment managed by AWS Glue.\n\n```ts\nnew glue.Job(this, 'RayJob', {\n executable: glue.JobExecutable.pythonRay({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE_NINE,\n runtime: glue.Runtime.RAY_TWO_FOUR,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n workerType: glue.WorkerType.Z_2X,\n workerCount: 2,\n description: 'an example Ray job'\n});\n```\n\nSee [documentation](https://docs.aws.amazon.com/glue/latest/dg/add-job.html) for more information on adding jobs in Glue.\n\n## Connection\n\nA `Connection` allows Glue jobs, crawlers and development endpoints to access certain types of data stores. For example, to create a network connection to connect to a data source within a VPC:\n\n```ts\ndeclare const securityGroup: ec2.SecurityGroup;\ndeclare const subnet: ec2.Subnet;\nnew glue.Connection(this, 'MyConnection', {\n type: glue.ConnectionType.NETWORK,\n // The security groups granting AWS Glue inbound access to the data source within the VPC\n securityGroups: [securityGroup],\n // The VPC subnet which contains the data source\n subnet,\n});\n```\n\nFor RDS `Connection` by JDBC, it is recommended to manage credentials using AWS Secrets Manager. To use Secret, specify `SECRET_ID` in `properties` like the following code. Note that in this case, the subnet must have a route to the AWS Secrets Manager VPC endpoint or to the AWS Secrets Manager endpoint through a NAT gateway.\n\n```ts\ndeclare const securityGroup: ec2.SecurityGroup;\ndeclare const subnet: ec2.Subnet;\ndeclare const db: rds.DatabaseCluster;\nnew glue.Connection(this, \"RdsConnection\", {\n type: glue.ConnectionType.JDBC,\n securityGroups: [securityGroup],\n subnet,\n properties: {\n JDBC_CONNECTION_URL: `jdbc:mysql://${db.clusterEndpoint.socketAddress}/databasename`,\n JDBC_ENFORCE_SSL: \"false\",\n SECRET_ID: db.secret!.secretName,\n },\n});\n```\n\nIf you need to use a connection type that doesn't exist as a static member on `ConnectionType`, you can instantiate a `ConnectionType` object, e.g: `new glue.ConnectionType('NEW_TYPE')`.\n\nSee [Adding a Connection to Your Data Store](https://docs.aws.amazon.com/glue/latest/dg/populate-add-connection.html) and [Connection Structure](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-catalog-connections.html#aws-glue-api-catalog-connections-Connection) documentation for more information on the supported data stores and their configurations.\n\n## SecurityConfiguration\n\nA `SecurityConfiguration` is a set of security properties that can be used by AWS Glue to encrypt data at rest.\n\n```ts\nnew glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n },\n jobBookmarksEncryption: {\n mode: glue.JobBookmarksEncryptionMode.CLIENT_SIDE_KMS,\n },\n s3Encryption: {\n mode: glue.S3EncryptionMode.KMS,\n },\n});\n```\n\nBy default, a shared KMS key is created for use with the encryption configurations that require one. You can also supply your own key for each encryption config, for example, for CloudWatch encryption:\n\n```ts\ndeclare const key: kms.Key;\nnew glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n kmsKey: key,\n },\n});\n```\n\nSee [documentation](https://docs.aws.amazon.com/glue/latest/dg/encryption-security-configuration.html) for more info for Glue encrypting data written by Crawlers, Jobs, and Development Endpoints.\n\n## Database\n\nA `Database` is a logical grouping of `Tables` in the Glue Catalog.\n\n```ts\nnew glue.Database(this, 'MyDatabase');\n```\n\n## Table\n\nA Glue table describes a table of data in S3: its structure (column names and types), location of data (S3 objects with a common prefix in a S3 bucket), and format for the files (Json, Avro, Parquet, etc.):\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }, {\n name: 'col2',\n type: glue.Schema.array(glue.Schema.STRING),\n comment: 'col2 is an array of strings' // comment is optional\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nBy default, a S3 bucket will be created to store the table's data but you can manually pass the `bucket` and `s3Prefix`:\n\n```ts\ndeclare const myBucket: s3.Bucket;\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n bucket: myBucket,\n s3Prefix: 'my-table/',\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nBy default, an S3 bucket will be created to store the table's data and stored in the bucket root. You can also manually pass the `bucket` and `s3Prefix`:\n\n### Partition Keys\n\nTo improve query performance, a table can specify `partitionKeys` on which data is stored and queried separately. For example, you might partition a table by `year` and `month` to optimize queries based on a time window:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n### Partition Indexes\n\nAnother way to improve query performance is to specify partition indexes. If no partition indexes are\npresent on the table, AWS Glue loads all partitions of the table and filters the loaded partitions using\nthe query expression. The query takes more time to run as the number of partitions increase. With an\nindex, the query will try to fetch a subset of the partitions instead of loading all partitions of the\ntable.\n\nThe keys of a partition index must be a subset of the partition keys of the table. You can have a\nmaximum of 3 partition indexes per table. To specify a partition index, you can use the `partitionIndexes`\nproperty:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n partitionIndexes: [{\n indexName: 'my-index', // optional\n keyNames: ['year'],\n }], // supply up to 3 indexes\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nAlternatively, you can call the `addPartitionIndex()` function on a table:\n\n```ts\ndeclare const myTable: glue.Table;\nmyTable.addPartitionIndex({\n indexName: 'my-index',\n keyNames: ['year'],\n});\n```\n\n### Partition Filtering\n\nIf you have a table with a large number of partitions that grows over time, consider using AWS Glue partition indexing and filtering.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n dataFormat: glue.DataFormat.JSON,\n enablePartitionFiltering: true,\n});\n```\n\n## [Encryption](https://docs.aws.amazon.com/athena/latest/ug/encryption.html)\n\nYou can enable encryption on a Table's data:\n\n* [S3Managed](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html) - (default) Server side encryption (`SSE-S3`) with an Amazon S3-managed key.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.S3_MANAGED,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [Kms](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html) - Server-side encryption (`SSE-KMS`) with an AWS KMS Key managed by the account owner.\n\n```ts\ndeclare const myDatabase: glue.Database;\n// KMS key is created automatically\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n\n// with an explicit KMS key\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS,\n encryptionKey: new kms.Key(this, 'MyKey'),\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [KmsManaged](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html) - Server-side encryption (`SSE-KMS`), like `Kms`, except with an AWS KMS Key managed by the AWS Key Management Service.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS_MANAGED,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [ClientSideKms](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html#client-side-encryption-kms-managed-master-key-intro) - Client-side encryption (`CSE-KMS`) with an AWS KMS Key managed by the account owner.\n\n```ts\ndeclare const myDatabase: glue.Database;\n// KMS key is created automatically\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.CLIENT_SIDE_KMS,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n\n// with an explicit KMS key\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.CLIENT_SIDE_KMS,\n encryptionKey: new kms.Key(this, 'MyKey'),\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n*Note: you cannot provide a `Bucket` when creating the `Table` if you wish to use server-side encryption (`KMS`, `KMS_MANAGED` or `S3_MANAGED`)*.\n\n## Types\n\nA table's schema is a collection of columns, each of which have a `name` and a `type`. Types are recursive structures, consisting of primitive and complex types:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n columns: [{\n name: 'primitive_column',\n type: glue.Schema.STRING,\n }, {\n name: 'array_column',\n type: glue.Schema.array(glue.Schema.INTEGER),\n comment: 'array<integer>',\n }, {\n name: 'map_column',\n type: glue.Schema.map(\n glue.Schema.STRING,\n glue.Schema.TIMESTAMP),\n comment: 'map<string,string>',\n }, {\n name: 'struct_column',\n type: glue.Schema.struct([{\n name: 'nested_column',\n type: glue.Schema.DATE,\n comment: 'nested comment',\n }]),\n comment: \"struct<nested_column:date COMMENT 'nested comment'>\",\n }],\n // ...\n database: myDatabase,\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n### Primitives\n\n#### Numeric\n\n| Name \t| Type \t| Comments |\n|-----------\t|----------\t|------------------------------------------------------------------------------------------------------------------\t|\n| FLOAT \t| Constant \t| A 32-bit single-precision floating point number |\n| INTEGER \t| Constant \t| A 32-bit signed value in two's complement format, with a minimum value of -2^31 and a maximum value of 2^31-1 \t|\n| DOUBLE \t| Constant \t| A 64-bit double-precision floating point number |\n| BIG_INT \t| Constant \t| A 64-bit signed INTEGER in two’s complement format, with a minimum value of -2^63 and a maximum value of 2^63 -1 |\n| SMALL_INT \t| Constant \t| A 16-bit signed INTEGER in two’s complement format, with a minimum value of -2^15 and a maximum value of 2^15-1 |\n| TINY_INT \t| Constant \t| A 8-bit signed INTEGER in two’s complement format, with a minimum value of -2^7 and a maximum value of 2^7-1 |\n\n#### Date and time\n\n| Name \t| Type \t| Comments \t|\n|-----------\t|----------\t|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------\t|\n| DATE \t| Constant \t| A date in UNIX format, such as YYYY-MM-DD. \t|\n| TIMESTAMP \t| Constant \t| Date and time instant in the UNiX format, such as yyyy-mm-dd hh:mm:ss[.f...]. For example, TIMESTAMP '2008-09-15 03:04:05.324'. This format uses the session time zone. \t|\n\n#### String\n\n| Name \t| Type \t| Comments \t|\n|--------------------------------------------\t|----------\t|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\t|\n| STRING \t| Constant \t| A string literal enclosed in single or double quotes \t|\n| decimal(precision: number, scale?: number) \t| Function \t| `precision` is the total number of digits. `scale` (optional) is the number of digits in fractional part with a default of 0. For example, use these type definitions: decimal(11,5), decimal(15) \t|\n| char(length: number) \t| Function \t| Fixed length character data, with a specified length between 1 and 255, such as char(10) \t|\n| varchar(length: number) \t| Function \t| Variable length character data, with a specified length between 1 and 65535, such as varchar(10) \t|\n\n#### Miscellaneous\n\n| Name \t| Type \t| Comments \t|\n|---------\t|----------\t|-------------------------------\t|\n| BOOLEAN \t| Constant \t| Values are `true` and `false` \t|\n| BINARY \t| Constant \t| Value is in binary \t|\n\n### Complex\n\n| Name \t| Type \t| Comments \t|\n|-------------------------------------\t|----------\t|-------------------------------------------------------------------\t|\n| array(itemType: Type) \t| Function \t| An array of some other type \t|\n| map(keyType: Type, valueType: Type) \t| Function \t| A map of some primitive key type to any value type \t|\n| struct(collumns: Column[]) \t| Function \t| Nested structure containing individually named and typed collumns \t|\n\n## Data Quality Ruleset\n\nA `DataQualityRuleset` specifies a data quality ruleset with DQDL rules applied to a specified AWS Glue table. For example, to create a data quality ruleset for a given table:\n\n```ts\nnew glue.DataQualityRuleset(this, 'MyDataQualityRuleset', {\n clientToken: 'client_token',\n description: 'description',\n rulesetName: 'ruleset_name',\n rulesetDqdl: 'ruleset_dqdl',\n tags: {\n key1: 'value1',\n key2: 'value2',\n },\n targetTable: new glue.DataQualityTargetTable('database_name', 'table_name'),\n});\n```\n\nFor more information, see [AWS Glue Data Quality](https://docs.aws.amazon.com/glue/latest/dg/glue-data-quality.html).\n"
3517
3517
  },
3518
3518
  "repository": {
3519
3519
  "directory": "packages/@aws-cdk/aws-glue-alpha",
@@ -3557,7 +3557,7 @@
3557
3557
  "docs": {
3558
3558
  "stability": "experimental",
3559
3559
  "summary": "Job Code from a local file.",
3560
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\n\ndeclare const dockerImage: cdk.DockerImage;\ndeclare const grantable: iam.IGrantable;\ndeclare const localBundling: cdk.ILocalBundling;\nconst assetCode = new glue_alpha.AssetCode('path', /* all optional props */ {\n assetHash: 'assetHash',\n assetHashType: cdk.AssetHashType.SOURCE,\n bundling: {\n image: dockerImage,\n\n // the properties below are optional\n bundlingFileAccess: cdk.BundlingFileAccess.VOLUME_COPY,\n command: ['command'],\n entrypoint: ['entrypoint'],\n environment: {\n environmentKey: 'environment',\n },\n local: localBundling,\n network: 'network',\n outputType: cdk.BundlingOutput.ARCHIVED,\n securityOpt: 'securityOpt',\n user: 'user',\n volumes: [{\n containerPath: 'containerPath',\n hostPath: 'hostPath',\n\n // the properties below are optional\n consistency: cdk.DockerVolumeConsistency.CONSISTENT,\n }],\n volumesFrom: ['volumesFrom'],\n workingDirectory: 'workingDirectory',\n },\n deployTime: false,\n exclude: ['exclude'],\n followSymlinks: cdk.SymlinkFollowMode.NEVER,\n ignoreMode: cdk.IgnoreMode.GLOB,\n readers: [grantable],\n});",
3560
+ "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\n\ndeclare const dockerImage: cdk.DockerImage;\ndeclare const grantable: iam.IGrantable;\ndeclare const localBundling: cdk.ILocalBundling;\nconst assetCode = new glue_alpha.AssetCode('path', /* all optional props */ {\n assetHash: 'assetHash',\n assetHashType: cdk.AssetHashType.SOURCE,\n bundling: {\n image: dockerImage,\n\n // the properties below are optional\n bundlingFileAccess: cdk.BundlingFileAccess.VOLUME_COPY,\n command: ['command'],\n entrypoint: ['entrypoint'],\n environment: {\n environmentKey: 'environment',\n },\n local: localBundling,\n network: 'network',\n outputType: cdk.BundlingOutput.ARCHIVED,\n platform: 'platform',\n securityOpt: 'securityOpt',\n user: 'user',\n volumes: [{\n containerPath: 'containerPath',\n hostPath: 'hostPath',\n\n // the properties below are optional\n consistency: cdk.DockerVolumeConsistency.CONSISTENT,\n }],\n volumesFrom: ['volumesFrom'],\n workingDirectory: 'workingDirectory',\n },\n deployTime: false,\n exclude: ['exclude'],\n followSymlinks: cdk.SymlinkFollowMode.NEVER,\n ignoreMode: cdk.IgnoreMode.GLOB,\n readers: [grantable],\n});",
3561
3561
  "custom": {
3562
3562
  "exampleMetadata": "fixture=_generated"
3563
3563
  }
@@ -5115,6 +5115,381 @@
5115
5115
  ],
5116
5116
  "symbolId": "lib/data-format:DataFormatProps"
5117
5117
  },
5118
+ "@aws-cdk/aws-glue-alpha.DataQualityRuleset": {
5119
+ "assembly": "@aws-cdk/aws-glue-alpha",
5120
+ "base": "aws-cdk-lib.Resource",
5121
+ "docs": {
5122
+ "stability": "experimental",
5123
+ "summary": "A Glue Data Quality ruleset.",
5124
+ "example": "new glue.DataQualityRuleset(this, 'MyDataQualityRuleset', {\n clientToken: 'client_token',\n description: 'description',\n rulesetName: 'ruleset_name',\n rulesetDqdl: 'ruleset_dqdl',\n tags: {\n key1: 'value1',\n key2: 'value2',\n },\n targetTable: new glue.DataQualityTargetTable('database_name', 'table_name'),\n});",
5125
+ "custom": {
5126
+ "exampleMetadata": "infused"
5127
+ }
5128
+ },
5129
+ "fqn": "@aws-cdk/aws-glue-alpha.DataQualityRuleset",
5130
+ "initializer": {
5131
+ "docs": {
5132
+ "stability": "experimental"
5133
+ },
5134
+ "locationInModule": {
5135
+ "filename": "lib/data-quality-ruleset.ts",
5136
+ "line": 121
5137
+ },
5138
+ "parameters": [
5139
+ {
5140
+ "name": "scope",
5141
+ "type": {
5142
+ "fqn": "constructs.Construct"
5143
+ }
5144
+ },
5145
+ {
5146
+ "name": "id",
5147
+ "type": {
5148
+ "primitive": "string"
5149
+ }
5150
+ },
5151
+ {
5152
+ "name": "props",
5153
+ "type": {
5154
+ "fqn": "@aws-cdk/aws-glue-alpha.DataQualityRulesetProps"
5155
+ }
5156
+ }
5157
+ ]
5158
+ },
5159
+ "interfaces": [
5160
+ "@aws-cdk/aws-glue-alpha.IDataQualityRuleset"
5161
+ ],
5162
+ "kind": "class",
5163
+ "locationInModule": {
5164
+ "filename": "lib/data-quality-ruleset.ts",
5165
+ "line": 84
5166
+ },
5167
+ "methods": [
5168
+ {
5169
+ "docs": {
5170
+ "stability": "experimental"
5171
+ },
5172
+ "locationInModule": {
5173
+ "filename": "lib/data-quality-ruleset.ts",
5174
+ "line": 85
5175
+ },
5176
+ "name": "fromRulesetArn",
5177
+ "parameters": [
5178
+ {
5179
+ "name": "scope",
5180
+ "type": {
5181
+ "fqn": "constructs.Construct"
5182
+ }
5183
+ },
5184
+ {
5185
+ "name": "id",
5186
+ "type": {
5187
+ "primitive": "string"
5188
+ }
5189
+ },
5190
+ {
5191
+ "name": "rulesetArn",
5192
+ "type": {
5193
+ "primitive": "string"
5194
+ }
5195
+ }
5196
+ ],
5197
+ "returns": {
5198
+ "type": {
5199
+ "fqn": "@aws-cdk/aws-glue-alpha.IDataQualityRuleset"
5200
+ }
5201
+ },
5202
+ "static": true
5203
+ },
5204
+ {
5205
+ "docs": {
5206
+ "stability": "experimental"
5207
+ },
5208
+ "locationInModule": {
5209
+ "filename": "lib/data-quality-ruleset.ts",
5210
+ "line": 94
5211
+ },
5212
+ "name": "fromRulesetName",
5213
+ "parameters": [
5214
+ {
5215
+ "name": "scope",
5216
+ "type": {
5217
+ "fqn": "constructs.Construct"
5218
+ }
5219
+ },
5220
+ {
5221
+ "name": "id",
5222
+ "type": {
5223
+ "primitive": "string"
5224
+ }
5225
+ },
5226
+ {
5227
+ "name": "rulesetName",
5228
+ "type": {
5229
+ "primitive": "string"
5230
+ }
5231
+ }
5232
+ ],
5233
+ "returns": {
5234
+ "type": {
5235
+ "fqn": "@aws-cdk/aws-glue-alpha.IDataQualityRuleset"
5236
+ }
5237
+ },
5238
+ "static": true
5239
+ }
5240
+ ],
5241
+ "name": "DataQualityRuleset",
5242
+ "properties": [
5243
+ {
5244
+ "docs": {
5245
+ "stability": "experimental",
5246
+ "summary": "ARN of this ruleset."
5247
+ },
5248
+ "immutable": true,
5249
+ "locationInModule": {
5250
+ "filename": "lib/data-quality-ruleset.ts",
5251
+ "line": 119
5252
+ },
5253
+ "name": "rulesetArn",
5254
+ "overrides": "@aws-cdk/aws-glue-alpha.IDataQualityRuleset",
5255
+ "type": {
5256
+ "primitive": "string"
5257
+ }
5258
+ },
5259
+ {
5260
+ "docs": {
5261
+ "stability": "experimental",
5262
+ "summary": "Name of this ruleset."
5263
+ },
5264
+ "immutable": true,
5265
+ "locationInModule": {
5266
+ "filename": "lib/data-quality-ruleset.ts",
5267
+ "line": 114
5268
+ },
5269
+ "name": "rulesetName",
5270
+ "overrides": "@aws-cdk/aws-glue-alpha.IDataQualityRuleset",
5271
+ "type": {
5272
+ "primitive": "string"
5273
+ }
5274
+ }
5275
+ ],
5276
+ "symbolId": "lib/data-quality-ruleset:DataQualityRuleset"
5277
+ },
5278
+ "@aws-cdk/aws-glue-alpha.DataQualityRulesetProps": {
5279
+ "assembly": "@aws-cdk/aws-glue-alpha",
5280
+ "datatype": true,
5281
+ "docs": {
5282
+ "stability": "experimental",
5283
+ "summary": "Construction properties for `DataQualityRuleset`.",
5284
+ "example": "new glue.DataQualityRuleset(this, 'MyDataQualityRuleset', {\n clientToken: 'client_token',\n description: 'description',\n rulesetName: 'ruleset_name',\n rulesetDqdl: 'ruleset_dqdl',\n tags: {\n key1: 'value1',\n key2: 'value2',\n },\n targetTable: new glue.DataQualityTargetTable('database_name', 'table_name'),\n});",
5285
+ "custom": {
5286
+ "exampleMetadata": "infused"
5287
+ }
5288
+ },
5289
+ "fqn": "@aws-cdk/aws-glue-alpha.DataQualityRulesetProps",
5290
+ "kind": "interface",
5291
+ "locationInModule": {
5292
+ "filename": "lib/data-quality-ruleset.ts",
5293
+ "line": 43
5294
+ },
5295
+ "name": "DataQualityRulesetProps",
5296
+ "properties": [
5297
+ {
5298
+ "abstract": true,
5299
+ "docs": {
5300
+ "custom": {
5301
+ "attribute": "true"
5302
+ },
5303
+ "stability": "experimental",
5304
+ "summary": "The dqdl of the ruleset."
5305
+ },
5306
+ "immutable": true,
5307
+ "locationInModule": {
5308
+ "filename": "lib/data-quality-ruleset.ts",
5309
+ "line": 66
5310
+ },
5311
+ "name": "rulesetDqdl",
5312
+ "type": {
5313
+ "primitive": "string"
5314
+ }
5315
+ },
5316
+ {
5317
+ "abstract": true,
5318
+ "docs": {
5319
+ "custom": {
5320
+ "attribute": "true"
5321
+ },
5322
+ "stability": "experimental",
5323
+ "summary": "The target table of the ruleset."
5324
+ },
5325
+ "immutable": true,
5326
+ "locationInModule": {
5327
+ "filename": "lib/data-quality-ruleset.ts",
5328
+ "line": 78
5329
+ },
5330
+ "name": "targetTable",
5331
+ "type": {
5332
+ "fqn": "@aws-cdk/aws-glue-alpha.DataQualityTargetTable"
5333
+ }
5334
+ },
5335
+ {
5336
+ "abstract": true,
5337
+ "docs": {
5338
+ "custom": {
5339
+ "attribute": "true"
5340
+ },
5341
+ "stability": "experimental",
5342
+ "summary": "The client token of the ruleset."
5343
+ },
5344
+ "immutable": true,
5345
+ "locationInModule": {
5346
+ "filename": "lib/data-quality-ruleset.ts",
5347
+ "line": 54
5348
+ },
5349
+ "name": "clientToken",
5350
+ "optional": true,
5351
+ "type": {
5352
+ "primitive": "string"
5353
+ }
5354
+ },
5355
+ {
5356
+ "abstract": true,
5357
+ "docs": {
5358
+ "custom": {
5359
+ "attribute": "true"
5360
+ },
5361
+ "stability": "experimental",
5362
+ "summary": "The description of the ruleset."
5363
+ },
5364
+ "immutable": true,
5365
+ "locationInModule": {
5366
+ "filename": "lib/data-quality-ruleset.ts",
5367
+ "line": 60
5368
+ },
5369
+ "name": "description",
5370
+ "optional": true,
5371
+ "type": {
5372
+ "primitive": "string"
5373
+ }
5374
+ },
5375
+ {
5376
+ "abstract": true,
5377
+ "docs": {
5378
+ "default": "cloudformation generated name",
5379
+ "stability": "experimental",
5380
+ "summary": "The name of the ruleset."
5381
+ },
5382
+ "immutable": true,
5383
+ "locationInModule": {
5384
+ "filename": "lib/data-quality-ruleset.ts",
5385
+ "line": 48
5386
+ },
5387
+ "name": "rulesetName",
5388
+ "optional": true,
5389
+ "type": {
5390
+ "primitive": "string"
5391
+ }
5392
+ },
5393
+ {
5394
+ "abstract": true,
5395
+ "docs": {
5396
+ "default": "empty tags",
5397
+ "stability": "experimental",
5398
+ "summary": "Key-Value pairs that define tags for the ruleset."
5399
+ },
5400
+ "immutable": true,
5401
+ "locationInModule": {
5402
+ "filename": "lib/data-quality-ruleset.ts",
5403
+ "line": 72
5404
+ },
5405
+ "name": "tags",
5406
+ "optional": true,
5407
+ "type": {
5408
+ "collection": {
5409
+ "elementtype": {
5410
+ "primitive": "string"
5411
+ },
5412
+ "kind": "map"
5413
+ }
5414
+ }
5415
+ }
5416
+ ],
5417
+ "symbolId": "lib/data-quality-ruleset:DataQualityRulesetProps"
5418
+ },
5419
+ "@aws-cdk/aws-glue-alpha.DataQualityTargetTable": {
5420
+ "assembly": "@aws-cdk/aws-glue-alpha",
5421
+ "docs": {
5422
+ "stability": "experimental",
5423
+ "summary": "Properties of a DataQualityTargetTable.",
5424
+ "example": "new glue.DataQualityRuleset(this, 'MyDataQualityRuleset', {\n clientToken: 'client_token',\n description: 'description',\n rulesetName: 'ruleset_name',\n rulesetDqdl: 'ruleset_dqdl',\n tags: {\n key1: 'value1',\n key2: 'value2',\n },\n targetTable: new glue.DataQualityTargetTable('database_name', 'table_name'),\n});",
5425
+ "custom": {
5426
+ "exampleMetadata": "infused"
5427
+ }
5428
+ },
5429
+ "fqn": "@aws-cdk/aws-glue-alpha.DataQualityTargetTable",
5430
+ "initializer": {
5431
+ "docs": {
5432
+ "stability": "experimental"
5433
+ },
5434
+ "locationInModule": {
5435
+ "filename": "lib/data-quality-ruleset.ts",
5436
+ "line": 20
5437
+ },
5438
+ "parameters": [
5439
+ {
5440
+ "name": "databaseName",
5441
+ "type": {
5442
+ "primitive": "string"
5443
+ }
5444
+ },
5445
+ {
5446
+ "name": "tableName",
5447
+ "type": {
5448
+ "primitive": "string"
5449
+ }
5450
+ }
5451
+ ]
5452
+ },
5453
+ "kind": "class",
5454
+ "locationInModule": {
5455
+ "filename": "lib/data-quality-ruleset.ts",
5456
+ "line": 9
5457
+ },
5458
+ "name": "DataQualityTargetTable",
5459
+ "properties": [
5460
+ {
5461
+ "docs": {
5462
+ "stability": "experimental",
5463
+ "summary": "The database name of the target table."
5464
+ },
5465
+ "immutable": true,
5466
+ "locationInModule": {
5467
+ "filename": "lib/data-quality-ruleset.ts",
5468
+ "line": 13
5469
+ },
5470
+ "name": "databaseName",
5471
+ "type": {
5472
+ "primitive": "string"
5473
+ }
5474
+ },
5475
+ {
5476
+ "docs": {
5477
+ "stability": "experimental",
5478
+ "summary": "The table name of the target table."
5479
+ },
5480
+ "immutable": true,
5481
+ "locationInModule": {
5482
+ "filename": "lib/data-quality-ruleset.ts",
5483
+ "line": 18
5484
+ },
5485
+ "name": "tableName",
5486
+ "type": {
5487
+ "primitive": "string"
5488
+ }
5489
+ }
5490
+ ],
5491
+ "symbolId": "lib/data-quality-ruleset:DataQualityTargetTable"
5492
+ },
5118
5493
  "@aws-cdk/aws-glue-alpha.Database": {
5119
5494
  "assembly": "@aws-cdk/aws-glue-alpha",
5120
5495
  "base": "aws-cdk-lib.Resource",
@@ -5588,6 +5963,63 @@
5588
5963
  ],
5589
5964
  "symbolId": "lib/connection:IConnection"
5590
5965
  },
5966
+ "@aws-cdk/aws-glue-alpha.IDataQualityRuleset": {
5967
+ "assembly": "@aws-cdk/aws-glue-alpha",
5968
+ "docs": {
5969
+ "stability": "experimental"
5970
+ },
5971
+ "fqn": "@aws-cdk/aws-glue-alpha.IDataQualityRuleset",
5972
+ "interfaces": [
5973
+ "aws-cdk-lib.IResource"
5974
+ ],
5975
+ "kind": "interface",
5976
+ "locationInModule": {
5977
+ "filename": "lib/data-quality-ruleset.ts",
5978
+ "line": 26
5979
+ },
5980
+ "name": "IDataQualityRuleset",
5981
+ "properties": [
5982
+ {
5983
+ "abstract": true,
5984
+ "docs": {
5985
+ "custom": {
5986
+ "attribute": "true"
5987
+ },
5988
+ "stability": "experimental",
5989
+ "summary": "The ARN of the ruleset."
5990
+ },
5991
+ "immutable": true,
5992
+ "locationInModule": {
5993
+ "filename": "lib/data-quality-ruleset.ts",
5994
+ "line": 31
5995
+ },
5996
+ "name": "rulesetArn",
5997
+ "type": {
5998
+ "primitive": "string"
5999
+ }
6000
+ },
6001
+ {
6002
+ "abstract": true,
6003
+ "docs": {
6004
+ "custom": {
6005
+ "attribute": "true"
6006
+ },
6007
+ "stability": "experimental",
6008
+ "summary": "The name of the ruleset."
6009
+ },
6010
+ "immutable": true,
6011
+ "locationInModule": {
6012
+ "filename": "lib/data-quality-ruleset.ts",
6013
+ "line": 37
6014
+ },
6015
+ "name": "rulesetName",
6016
+ "type": {
6017
+ "primitive": "string"
6018
+ }
6019
+ }
6020
+ ],
6021
+ "symbolId": "lib/data-quality-ruleset:IDataQualityRuleset"
6022
+ },
5591
6023
  "@aws-cdk/aws-glue-alpha.IDatabase": {
5592
6024
  "assembly": "@aws-cdk/aws-glue-alpha",
5593
6025
  "docs": {
@@ -11320,6 +11752,6 @@
11320
11752
  "symbolId": "lib/job:WorkerType"
11321
11753
  }
11322
11754
  },
11323
- "version": "2.87.0-alpha.0",
11755
+ "version": "2.88.0-alpha.0",
11324
11756
  "fingerprint": "**********"
11325
11757
  }
Binary file
package/.warnings.jsii.js CHANGED
@@ -62,6 +62,24 @@ function _aws_cdk_aws_glue_alpha_DataFormatProps(p) {
62
62
  }
63
63
  function _aws_cdk_aws_glue_alpha_DataFormat(p) {
64
64
  }
65
+ function _aws_cdk_aws_glue_alpha_DataQualityTargetTable(p) {
66
+ }
67
+ function _aws_cdk_aws_glue_alpha_IDataQualityRuleset(p) {
68
+ }
69
+ function _aws_cdk_aws_glue_alpha_DataQualityRulesetProps(p) {
70
+ if (p == null)
71
+ return;
72
+ visitedObjects.add(p);
73
+ try {
74
+ if (!visitedObjects.has(p.targetTable))
75
+ _aws_cdk_aws_glue_alpha_DataQualityTargetTable(p.targetTable);
76
+ }
77
+ finally {
78
+ visitedObjects.delete(p);
79
+ }
80
+ }
81
+ function _aws_cdk_aws_glue_alpha_DataQualityRuleset(p) {
82
+ }
65
83
  function _aws_cdk_aws_glue_alpha_IDatabase(p) {
66
84
  }
67
85
  function _aws_cdk_aws_glue_alpha_DatabaseProps(p) {
@@ -426,4 +444,4 @@ class DeprecationError extends Error {
426
444
  });
427
445
  }
428
446
  }
429
- module.exports = { print, getPropertyDescriptor, DeprecationError, _aws_cdk_aws_glue_alpha_ConnectionType, _aws_cdk_aws_glue_alpha_IConnection, _aws_cdk_aws_glue_alpha_ConnectionOptions, _aws_cdk_aws_glue_alpha_ConnectionProps, _aws_cdk_aws_glue_alpha_Connection, _aws_cdk_aws_glue_alpha_InputFormat, _aws_cdk_aws_glue_alpha_OutputFormat, _aws_cdk_aws_glue_alpha_SerializationLibrary, _aws_cdk_aws_glue_alpha_ClassificationString, _aws_cdk_aws_glue_alpha_DataFormatProps, _aws_cdk_aws_glue_alpha_DataFormat, _aws_cdk_aws_glue_alpha_IDatabase, _aws_cdk_aws_glue_alpha_DatabaseProps, _aws_cdk_aws_glue_alpha_Database, _aws_cdk_aws_glue_alpha_WorkerType, _aws_cdk_aws_glue_alpha_JobState, _aws_cdk_aws_glue_alpha_MetricType, _aws_cdk_aws_glue_alpha_ExecutionClass, _aws_cdk_aws_glue_alpha_IJob, _aws_cdk_aws_glue_alpha_SparkUIProps, _aws_cdk_aws_glue_alpha_SparkUILoggingLocation, _aws_cdk_aws_glue_alpha_ContinuousLoggingProps, _aws_cdk_aws_glue_alpha_JobAttributes, _aws_cdk_aws_glue_alpha_JobProps, _aws_cdk_aws_glue_alpha_Job, _aws_cdk_aws_glue_alpha_GlueVersion, _aws_cdk_aws_glue_alpha_JobLanguage, _aws_cdk_aws_glue_alpha_PythonVersion, _aws_cdk_aws_glue_alpha_Runtime, _aws_cdk_aws_glue_alpha_JobType, _aws_cdk_aws_glue_alpha_ScalaJobExecutableProps, _aws_cdk_aws_glue_alpha_PythonSparkJobExecutableProps, _aws_cdk_aws_glue_alpha_PythonShellExecutableProps, _aws_cdk_aws_glue_alpha_PythonRayExecutableProps, _aws_cdk_aws_glue_alpha_JobExecutable, _aws_cdk_aws_glue_alpha_JobExecutableConfig, _aws_cdk_aws_glue_alpha_Code, _aws_cdk_aws_glue_alpha_S3Code, _aws_cdk_aws_glue_alpha_AssetCode, _aws_cdk_aws_glue_alpha_CodeConfig, _aws_cdk_aws_glue_alpha_Column, _aws_cdk_aws_glue_alpha_Type, _aws_cdk_aws_glue_alpha_Schema, _aws_cdk_aws_glue_alpha_ISecurityConfiguration, _aws_cdk_aws_glue_alpha_S3EncryptionMode, _aws_cdk_aws_glue_alpha_CloudWatchEncryptionMode, _aws_cdk_aws_glue_alpha_JobBookmarksEncryptionMode, _aws_cdk_aws_glue_alpha_S3Encryption, _aws_cdk_aws_glue_alpha_CloudWatchEncryption, _aws_cdk_aws_glue_alpha_JobBookmarksEncryption, _aws_cdk_aws_glue_alpha_SecurityConfigurationProps, _aws_cdk_aws_glue_alpha_SecurityConfiguration, _aws_cdk_aws_glue_alpha_PartitionIndex, _aws_cdk_aws_glue_alpha_ITable, _aws_cdk_aws_glue_alpha_TableEncryption, _aws_cdk_aws_glue_alpha_TableAttributes, _aws_cdk_aws_glue_alpha_TableProps, _aws_cdk_aws_glue_alpha_Table };
447
+ module.exports = { print, getPropertyDescriptor, DeprecationError, _aws_cdk_aws_glue_alpha_ConnectionType, _aws_cdk_aws_glue_alpha_IConnection, _aws_cdk_aws_glue_alpha_ConnectionOptions, _aws_cdk_aws_glue_alpha_ConnectionProps, _aws_cdk_aws_glue_alpha_Connection, _aws_cdk_aws_glue_alpha_InputFormat, _aws_cdk_aws_glue_alpha_OutputFormat, _aws_cdk_aws_glue_alpha_SerializationLibrary, _aws_cdk_aws_glue_alpha_ClassificationString, _aws_cdk_aws_glue_alpha_DataFormatProps, _aws_cdk_aws_glue_alpha_DataFormat, _aws_cdk_aws_glue_alpha_DataQualityTargetTable, _aws_cdk_aws_glue_alpha_IDataQualityRuleset, _aws_cdk_aws_glue_alpha_DataQualityRulesetProps, _aws_cdk_aws_glue_alpha_DataQualityRuleset, _aws_cdk_aws_glue_alpha_IDatabase, _aws_cdk_aws_glue_alpha_DatabaseProps, _aws_cdk_aws_glue_alpha_Database, _aws_cdk_aws_glue_alpha_WorkerType, _aws_cdk_aws_glue_alpha_JobState, _aws_cdk_aws_glue_alpha_MetricType, _aws_cdk_aws_glue_alpha_ExecutionClass, _aws_cdk_aws_glue_alpha_IJob, _aws_cdk_aws_glue_alpha_SparkUIProps, _aws_cdk_aws_glue_alpha_SparkUILoggingLocation, _aws_cdk_aws_glue_alpha_ContinuousLoggingProps, _aws_cdk_aws_glue_alpha_JobAttributes, _aws_cdk_aws_glue_alpha_JobProps, _aws_cdk_aws_glue_alpha_Job, _aws_cdk_aws_glue_alpha_GlueVersion, _aws_cdk_aws_glue_alpha_JobLanguage, _aws_cdk_aws_glue_alpha_PythonVersion, _aws_cdk_aws_glue_alpha_Runtime, _aws_cdk_aws_glue_alpha_JobType, _aws_cdk_aws_glue_alpha_ScalaJobExecutableProps, _aws_cdk_aws_glue_alpha_PythonSparkJobExecutableProps, _aws_cdk_aws_glue_alpha_PythonShellExecutableProps, _aws_cdk_aws_glue_alpha_PythonRayExecutableProps, _aws_cdk_aws_glue_alpha_JobExecutable, _aws_cdk_aws_glue_alpha_JobExecutableConfig, _aws_cdk_aws_glue_alpha_Code, _aws_cdk_aws_glue_alpha_S3Code, _aws_cdk_aws_glue_alpha_AssetCode, _aws_cdk_aws_glue_alpha_CodeConfig, _aws_cdk_aws_glue_alpha_Column, _aws_cdk_aws_glue_alpha_Type, _aws_cdk_aws_glue_alpha_Schema, _aws_cdk_aws_glue_alpha_ISecurityConfiguration, _aws_cdk_aws_glue_alpha_S3EncryptionMode, _aws_cdk_aws_glue_alpha_CloudWatchEncryptionMode, _aws_cdk_aws_glue_alpha_JobBookmarksEncryptionMode, _aws_cdk_aws_glue_alpha_S3Encryption, _aws_cdk_aws_glue_alpha_CloudWatchEncryption, _aws_cdk_aws_glue_alpha_JobBookmarksEncryption, _aws_cdk_aws_glue_alpha_SecurityConfigurationProps, _aws_cdk_aws_glue_alpha_SecurityConfiguration, _aws_cdk_aws_glue_alpha_PartitionIndex, _aws_cdk_aws_glue_alpha_ITable, _aws_cdk_aws_glue_alpha_TableEncryption, _aws_cdk_aws_glue_alpha_TableAttributes, _aws_cdk_aws_glue_alpha_TableProps, _aws_cdk_aws_glue_alpha_Table };
package/README.md CHANGED
@@ -493,3 +493,23 @@ new glue.Table(this, 'MyTable', {
493
493
  | array(itemType: Type) | Function | An array of some other type |
494
494
  | map(keyType: Type, valueType: Type) | Function | A map of some primitive key type to any value type |
495
495
  | struct(collumns: Column[]) | Function | Nested structure containing individually named and typed collumns |
496
+
497
+ ## Data Quality Ruleset
498
+
499
+ A `DataQualityRuleset` specifies a data quality ruleset with DQDL rules applied to a specified AWS Glue table. For example, to create a data quality ruleset for a given table:
500
+
501
+ ```ts
502
+ new glue.DataQualityRuleset(this, 'MyDataQualityRuleset', {
503
+ clientToken: 'client_token',
504
+ description: 'description',
505
+ rulesetName: 'ruleset_name',
506
+ rulesetDqdl: 'ruleset_dqdl',
507
+ tags: {
508
+ key1: 'value1',
509
+ key2: 'value2',
510
+ },
511
+ targetTable: new glue.DataQualityTargetTable('database_name', 'table_name'),
512
+ });
513
+ ```
514
+
515
+ For more information, see [AWS Glue Data Quality](https://docs.aws.amazon.com/glue/latest/dg/glue-data-quality.html).