@aws-cdk/aws-glue-alpha 2.85.0-alpha.0 → 2.87.0-alpha.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/.jsii CHANGED
@@ -8,7 +8,7 @@
8
8
  "url": "https://aws.amazon.com"
9
9
  },
10
10
  "dependencies": {
11
- "aws-cdk-lib": "2.85.0",
11
+ "aws-cdk-lib": "2.87.0",
12
12
  "constructs": "^10.0.0"
13
13
  },
14
14
  "dependencyClosure": {
@@ -3187,6 +3187,19 @@
3187
3187
  }
3188
3188
  }
3189
3189
  },
3190
+ "aws-cdk-lib.aws_verifiedpermissions": {
3191
+ "targets": {
3192
+ "dotnet": {
3193
+ "package": "Amazon.CDK.AWS.VerifiedPermissions"
3194
+ },
3195
+ "java": {
3196
+ "package": "services.verifiedpermissions"
3197
+ },
3198
+ "python": {
3199
+ "module": "aws_cdk.aws_verifiedpermissions"
3200
+ }
3201
+ }
3202
+ },
3190
3203
  "aws-cdk-lib.aws_voiceid": {
3191
3204
  "targets": {
3192
3205
  "dotnet": {
@@ -3500,7 +3513,7 @@
3500
3513
  },
3501
3514
  "name": "@aws-cdk/aws-glue-alpha",
3502
3515
  "readme": {
3503
- "markdown": "# AWS Glue Construct Library\n<!--BEGIN STABILITY BANNER-->\n\n---\n\n![cdk-constructs: Experimental](https://img.shields.io/badge/cdk--constructs-experimental-important.svg?style=for-the-badge)\n\n> The APIs of higher level constructs in this module are experimental and under active development.\n> They are subject to non-backward compatible changes or removal in any future version. These are\n> not subject to the [Semantic Versioning](https://semver.org/) model and breaking changes will be\n> announced in the release notes. This means that while you may use them, you may need to update\n> your source code when upgrading to a newer version of this package.\n\n---\n\n<!--END STABILITY BANNER-->\n\nThis module is part of the [AWS Cloud Development Kit](https://github.com/aws/aws-cdk) project.\n\n## Job\n\nA `Job` encapsulates a script that connects to data sources, processes them, and then writes output to a data target.\n\nThere are 3 types of jobs supported by AWS Glue: Spark ETL, Spark Streaming, and Python Shell jobs.\n\nThe `glue.JobExecutable` allows you to specify the type of job, the language to use and the code assets required by the job.\n\n`glue.Code` allows you to refer to the different code assets required by the job, either from an existing S3 location or from a local file path.\n\n### Spark Jobs\n\nThese jobs run in an Apache Spark environment managed by AWS Glue.\n\n#### ETL Jobs\n\nAn ETL job processes data in batches using Apache Spark.\n\n```ts\ndeclare const bucket: s3.Bucket;\nnew glue.Job(this, 'ScalaSparkEtlJob', {\n executable: glue.JobExecutable.scalaEtl({\n glueVersion: glue.GlueVersion.V4_0,\n script: glue.Code.fromBucket(bucket, 'src/com/example/HelloWorld.scala'),\n className: 'com.example.HelloWorld',\n extraJars: [glue.Code.fromBucket(bucket, 'jars/HelloWorld.jar')],\n }),\n workerType: glue.WorkerType.G_8X,\n description: 'an example Scala ETL job',\n});\n```\n\n#### Streaming Jobs\n\nA Streaming job is similar to an ETL job, except that it performs ETL on data streams. It uses the Apache Spark Structured Streaming framework. Some Spark job features are not available to streaming ETL jobs.\n\n```ts\nnew glue.Job(this, 'PythonSparkStreamingJob', {\n executable: glue.JobExecutable.pythonStreaming({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n description: 'an example Python Streaming job',\n});\n```\n\n### Python Shell Jobs\n\nA Python shell job runs Python scripts as a shell and supports a Python version that depends on the AWS Glue version you are using.\nThis can be used to schedule and run tasks that don't require an Apache Spark environment. Currently, three flavors are supported:\n\n* PythonVersion.TWO (2.7; EOL)\n* PythonVersion.THREE (3.6)\n* PythonVersion.THREE_NINE (3.9)\n\n```ts\ndeclare const bucket: s3.Bucket;\nnew glue.Job(this, 'PythonShellJob', {\n executable: glue.JobExecutable.pythonShell({\n glueVersion: glue.GlueVersion.V1_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromBucket(bucket, 'script.py'),\n }),\n description: 'an example Python Shell job',\n});\n```\n\n### Ray Jobs\n\nThese jobs run in a Ray environment managed by AWS Glue.\n\n```ts\nnew glue.Job(this, 'RayJob', {\n executable: glue.JobExecutable.pythonRay({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE_NINE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n workerType: glue.WorkerType.Z_2X,\n workerCount: 2,\n description: 'an example Ray job'\n});\n```\n\nSee [documentation](https://docs.aws.amazon.com/glue/latest/dg/add-job.html) for more information on adding jobs in Glue.\n\n## Connection\n\nA `Connection` allows Glue jobs, crawlers and development endpoints to access certain types of data stores. For example, to create a network connection to connect to a data source within a VPC:\n\n```ts\ndeclare const securityGroup: ec2.SecurityGroup;\ndeclare const subnet: ec2.Subnet;\nnew glue.Connection(this, 'MyConnection', {\n type: glue.ConnectionType.NETWORK,\n // The security groups granting AWS Glue inbound access to the data source within the VPC\n securityGroups: [securityGroup],\n // The VPC subnet which contains the data source\n subnet,\n});\n```\n\nFor RDS `Connection` by JDBC, it is recommended to manage credentials using AWS Secrets Manager. To use Secret, specify `SECRET_ID` in `properties` like the following code. Note that in this case, the subnet must have a route to the AWS Secrets Manager VPC endpoint or to the AWS Secrets Manager endpoint through a NAT gateway.\n\n```ts\ndeclare const securityGroup: ec2.SecurityGroup;\ndeclare const subnet: ec2.Subnet;\ndeclare const db: rds.DatabaseCluster;\nnew glue.Connection(this, \"RdsConnection\", {\n type: glue.ConnectionType.JDBC,\n securityGroups: [securityGroup],\n subnet,\n properties: {\n JDBC_CONNECTION_URL: `jdbc:mysql://${db.clusterEndpoint.socketAddress}/databasename`,\n JDBC_ENFORCE_SSL: \"false\",\n SECRET_ID: db.secret!.secretName,\n },\n});\n```\n\nIf you need to use a connection type that doesn't exist as a static member on `ConnectionType`, you can instantiate a `ConnectionType` object, e.g: `new glue.ConnectionType('NEW_TYPE')`.\n\nSee [Adding a Connection to Your Data Store](https://docs.aws.amazon.com/glue/latest/dg/populate-add-connection.html) and [Connection Structure](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-catalog-connections.html#aws-glue-api-catalog-connections-Connection) documentation for more information on the supported data stores and their configurations.\n\n## SecurityConfiguration\n\nA `SecurityConfiguration` is a set of security properties that can be used by AWS Glue to encrypt data at rest.\n\n```ts\nnew glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n },\n jobBookmarksEncryption: {\n mode: glue.JobBookmarksEncryptionMode.CLIENT_SIDE_KMS,\n },\n s3Encryption: {\n mode: glue.S3EncryptionMode.KMS,\n },\n});\n```\n\nBy default, a shared KMS key is created for use with the encryption configurations that require one. You can also supply your own key for each encryption config, for example, for CloudWatch encryption:\n\n```ts\ndeclare const key: kms.Key;\nnew glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n kmsKey: key,\n },\n});\n```\n\nSee [documentation](https://docs.aws.amazon.com/glue/latest/dg/encryption-security-configuration.html) for more info for Glue encrypting data written by Crawlers, Jobs, and Development Endpoints.\n\n## Database\n\nA `Database` is a logical grouping of `Tables` in the Glue Catalog.\n\n```ts\nnew glue.Database(this, 'MyDatabase');\n```\n\n## Table\n\nA Glue table describes a table of data in S3: its structure (column names and types), location of data (S3 objects with a common prefix in a S3 bucket), and format for the files (Json, Avro, Parquet, etc.):\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }, {\n name: 'col2',\n type: glue.Schema.array(glue.Schema.STRING),\n comment: 'col2 is an array of strings' // comment is optional\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nBy default, a S3 bucket will be created to store the table's data but you can manually pass the `bucket` and `s3Prefix`:\n\n```ts\ndeclare const myBucket: s3.Bucket;\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n bucket: myBucket,\n s3Prefix: 'my-table/',\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nBy default, an S3 bucket will be created to store the table's data and stored in the bucket root. You can also manually pass the `bucket` and `s3Prefix`:\n\n### Partition Keys\n\nTo improve query performance, a table can specify `partitionKeys` on which data is stored and queried separately. For example, you might partition a table by `year` and `month` to optimize queries based on a time window:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n### Partition Indexes\n\nAnother way to improve query performance is to specify partition indexes. If no partition indexes are\npresent on the table, AWS Glue loads all partitions of the table and filters the loaded partitions using\nthe query expression. The query takes more time to run as the number of partitions increase. With an\nindex, the query will try to fetch a subset of the partitions instead of loading all partitions of the\ntable.\n\nThe keys of a partition index must be a subset of the partition keys of the table. You can have a\nmaximum of 3 partition indexes per table. To specify a partition index, you can use the `partitionIndexes`\nproperty:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n partitionIndexes: [{\n indexName: 'my-index', // optional\n keyNames: ['year'],\n }], // supply up to 3 indexes\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nAlternatively, you can call the `addPartitionIndex()` function on a table:\n\n```ts\ndeclare const myTable: glue.Table;\nmyTable.addPartitionIndex({\n indexName: 'my-index',\n keyNames: ['year'],\n});\n```\n\n### Partition Filtering\n\nIf you have a table with a large number of partitions that grows over time, consider using AWS Glue partition indexing and filtering.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n dataFormat: glue.DataFormat.JSON,\n enablePartitionFiltering: true,\n});\n```\n\n## [Encryption](https://docs.aws.amazon.com/athena/latest/ug/encryption.html)\n\nYou can enable encryption on a Table's data:\n\n* [S3Managed](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html) - (default) Server side encryption (`SSE-S3`) with an Amazon S3-managed key.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.S3_MANAGED,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [Kms](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html) - Server-side encryption (`SSE-KMS`) with an AWS KMS Key managed by the account owner.\n\n```ts\ndeclare const myDatabase: glue.Database;\n// KMS key is created automatically\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n\n// with an explicit KMS key\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS,\n encryptionKey: new kms.Key(this, 'MyKey'),\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [KmsManaged](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html) - Server-side encryption (`SSE-KMS`), like `Kms`, except with an AWS KMS Key managed by the AWS Key Management Service.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS_MANAGED,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [ClientSideKms](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html#client-side-encryption-kms-managed-master-key-intro) - Client-side encryption (`CSE-KMS`) with an AWS KMS Key managed by the account owner.\n\n```ts\ndeclare const myDatabase: glue.Database;\n// KMS key is created automatically\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.CLIENT_SIDE_KMS,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n\n// with an explicit KMS key\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.CLIENT_SIDE_KMS,\n encryptionKey: new kms.Key(this, 'MyKey'),\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n*Note: you cannot provide a `Bucket` when creating the `Table` if you wish to use server-side encryption (`KMS`, `KMS_MANAGED` or `S3_MANAGED`)*.\n\n## Types\n\nA table's schema is a collection of columns, each of which have a `name` and a `type`. Types are recursive structures, consisting of primitive and complex types:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n columns: [{\n name: 'primitive_column',\n type: glue.Schema.STRING,\n }, {\n name: 'array_column',\n type: glue.Schema.array(glue.Schema.INTEGER),\n comment: 'array<integer>',\n }, {\n name: 'map_column',\n type: glue.Schema.map(\n glue.Schema.STRING,\n glue.Schema.TIMESTAMP),\n comment: 'map<string,string>',\n }, {\n name: 'struct_column',\n type: glue.Schema.struct([{\n name: 'nested_column',\n type: glue.Schema.DATE,\n comment: 'nested comment',\n }]),\n comment: \"struct<nested_column:date COMMENT 'nested comment'>\",\n }],\n // ...\n database: myDatabase,\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n### Primitives\n\n#### Numeric\n\n| Name \t| Type \t| Comments |\n|-----------\t|----------\t|------------------------------------------------------------------------------------------------------------------\t|\n| FLOAT \t| Constant \t| A 32-bit single-precision floating point number |\n| INTEGER \t| Constant \t| A 32-bit signed value in two's complement format, with a minimum value of -2^31 and a maximum value of 2^31-1 \t|\n| DOUBLE \t| Constant \t| A 64-bit double-precision floating point number |\n| BIG_INT \t| Constant \t| A 64-bit signed INTEGER in two’s complement format, with a minimum value of -2^63 and a maximum value of 2^63 -1 |\n| SMALL_INT \t| Constant \t| A 16-bit signed INTEGER in two’s complement format, with a minimum value of -2^15 and a maximum value of 2^15-1 |\n| TINY_INT \t| Constant \t| A 8-bit signed INTEGER in two’s complement format, with a minimum value of -2^7 and a maximum value of 2^7-1 |\n\n#### Date and time\n\n| Name \t| Type \t| Comments \t|\n|-----------\t|----------\t|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------\t|\n| DATE \t| Constant \t| A date in UNIX format, such as YYYY-MM-DD. \t|\n| TIMESTAMP \t| Constant \t| Date and time instant in the UNiX format, such as yyyy-mm-dd hh:mm:ss[.f...]. For example, TIMESTAMP '2008-09-15 03:04:05.324'. This format uses the session time zone. \t|\n\n#### String\n\n| Name \t| Type \t| Comments \t|\n|--------------------------------------------\t|----------\t|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\t|\n| STRING \t| Constant \t| A string literal enclosed in single or double quotes \t|\n| decimal(precision: number, scale?: number) \t| Function \t| `precision` is the total number of digits. `scale` (optional) is the number of digits in fractional part with a default of 0. For example, use these type definitions: decimal(11,5), decimal(15) \t|\n| char(length: number) \t| Function \t| Fixed length character data, with a specified length between 1 and 255, such as char(10) \t|\n| varchar(length: number) \t| Function \t| Variable length character data, with a specified length between 1 and 65535, such as varchar(10) \t|\n\n#### Miscellaneous\n\n| Name \t| Type \t| Comments \t|\n|---------\t|----------\t|-------------------------------\t|\n| BOOLEAN \t| Constant \t| Values are `true` and `false` \t|\n| BINARY \t| Constant \t| Value is in binary \t|\n\n### Complex\n\n| Name \t| Type \t| Comments \t|\n|-------------------------------------\t|----------\t|-------------------------------------------------------------------\t|\n| array(itemType: Type) \t| Function \t| An array of some other type \t|\n| map(keyType: Type, valueType: Type) \t| Function \t| A map of some primitive key type to any value type \t|\n| struct(collumns: Column[]) \t| Function \t| Nested structure containing individually named and typed collumns \t|\n"
3516
+ "markdown": "# AWS Glue Construct Library\n<!--BEGIN STABILITY BANNER-->\n\n---\n\n![cdk-constructs: Experimental](https://img.shields.io/badge/cdk--constructs-experimental-important.svg?style=for-the-badge)\n\n> The APIs of higher level constructs in this module are experimental and under active development.\n> They are subject to non-backward compatible changes or removal in any future version. These are\n> not subject to the [Semantic Versioning](https://semver.org/) model and breaking changes will be\n> announced in the release notes. This means that while you may use them, you may need to update\n> your source code when upgrading to a newer version of this package.\n\n---\n\n<!--END STABILITY BANNER-->\n\nThis module is part of the [AWS Cloud Development Kit](https://github.com/aws/aws-cdk) project.\n\n## Job\n\nA `Job` encapsulates a script that connects to data sources, processes them, and then writes output to a data target.\n\nThere are 3 types of jobs supported by AWS Glue: Spark ETL, Spark Streaming, and Python Shell jobs.\n\nThe `glue.JobExecutable` allows you to specify the type of job, the language to use and the code assets required by the job.\n\n`glue.Code` allows you to refer to the different code assets required by the job, either from an existing S3 location or from a local file path.\n\n`glue.ExecutionClass` allows you to specify `FLEX` or `STANDARD`. `FLEX` is appropriate for non-urgent jobs such as pre-production jobs, testing, and one-time data loads.\n\n### Spark Jobs\n\nThese jobs run in an Apache Spark environment managed by AWS Glue.\n\n#### ETL Jobs\n\nAn ETL job processes data in batches using Apache Spark.\n\n```ts\ndeclare const bucket: s3.Bucket;\nnew glue.Job(this, 'ScalaSparkEtlJob', {\n executable: glue.JobExecutable.scalaEtl({\n glueVersion: glue.GlueVersion.V4_0,\n script: glue.Code.fromBucket(bucket, 'src/com/example/HelloWorld.scala'),\n className: 'com.example.HelloWorld',\n extraJars: [glue.Code.fromBucket(bucket, 'jars/HelloWorld.jar')],\n }),\n workerType: glue.WorkerType.G_8X,\n description: 'an example Scala ETL job',\n});\n```\n\n#### Streaming Jobs\n\nA Streaming job is similar to an ETL job, except that it performs ETL on data streams. It uses the Apache Spark Structured Streaming framework. Some Spark job features are not available to streaming ETL jobs.\n\n```ts\nnew glue.Job(this, 'PythonSparkStreamingJob', {\n executable: glue.JobExecutable.pythonStreaming({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n description: 'an example Python Streaming job',\n});\n```\n\n### Python Shell Jobs\n\nA Python shell job runs Python scripts as a shell and supports a Python version that depends on the AWS Glue version you are using.\nThis can be used to schedule and run tasks that don't require an Apache Spark environment. Currently, three flavors are supported:\n\n* PythonVersion.TWO (2.7; EOL)\n* PythonVersion.THREE (3.6)\n* PythonVersion.THREE_NINE (3.9)\n\n```ts\ndeclare const bucket: s3.Bucket;\nnew glue.Job(this, 'PythonShellJob', {\n executable: glue.JobExecutable.pythonShell({\n glueVersion: glue.GlueVersion.V1_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromBucket(bucket, 'script.py'),\n }),\n description: 'an example Python Shell job',\n});\n```\n\n### Ray Jobs\n\nThese jobs run in a Ray environment managed by AWS Glue.\n\n```ts\nnew glue.Job(this, 'RayJob', {\n executable: glue.JobExecutable.pythonRay({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE_NINE,\n runtime: glue.Runtime.RAY_TWO_FOUR,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n workerType: glue.WorkerType.Z_2X,\n workerCount: 2,\n description: 'an example Ray job'\n});\n```\n\nSee [documentation](https://docs.aws.amazon.com/glue/latest/dg/add-job.html) for more information on adding jobs in Glue.\n\n## Connection\n\nA `Connection` allows Glue jobs, crawlers and development endpoints to access certain types of data stores. For example, to create a network connection to connect to a data source within a VPC:\n\n```ts\ndeclare const securityGroup: ec2.SecurityGroup;\ndeclare const subnet: ec2.Subnet;\nnew glue.Connection(this, 'MyConnection', {\n type: glue.ConnectionType.NETWORK,\n // The security groups granting AWS Glue inbound access to the data source within the VPC\n securityGroups: [securityGroup],\n // The VPC subnet which contains the data source\n subnet,\n});\n```\n\nFor RDS `Connection` by JDBC, it is recommended to manage credentials using AWS Secrets Manager. To use Secret, specify `SECRET_ID` in `properties` like the following code. Note that in this case, the subnet must have a route to the AWS Secrets Manager VPC endpoint or to the AWS Secrets Manager endpoint through a NAT gateway.\n\n```ts\ndeclare const securityGroup: ec2.SecurityGroup;\ndeclare const subnet: ec2.Subnet;\ndeclare const db: rds.DatabaseCluster;\nnew glue.Connection(this, \"RdsConnection\", {\n type: glue.ConnectionType.JDBC,\n securityGroups: [securityGroup],\n subnet,\n properties: {\n JDBC_CONNECTION_URL: `jdbc:mysql://${db.clusterEndpoint.socketAddress}/databasename`,\n JDBC_ENFORCE_SSL: \"false\",\n SECRET_ID: db.secret!.secretName,\n },\n});\n```\n\nIf you need to use a connection type that doesn't exist as a static member on `ConnectionType`, you can instantiate a `ConnectionType` object, e.g: `new glue.ConnectionType('NEW_TYPE')`.\n\nSee [Adding a Connection to Your Data Store](https://docs.aws.amazon.com/glue/latest/dg/populate-add-connection.html) and [Connection Structure](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-catalog-connections.html#aws-glue-api-catalog-connections-Connection) documentation for more information on the supported data stores and their configurations.\n\n## SecurityConfiguration\n\nA `SecurityConfiguration` is a set of security properties that can be used by AWS Glue to encrypt data at rest.\n\n```ts\nnew glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n },\n jobBookmarksEncryption: {\n mode: glue.JobBookmarksEncryptionMode.CLIENT_SIDE_KMS,\n },\n s3Encryption: {\n mode: glue.S3EncryptionMode.KMS,\n },\n});\n```\n\nBy default, a shared KMS key is created for use with the encryption configurations that require one. You can also supply your own key for each encryption config, for example, for CloudWatch encryption:\n\n```ts\ndeclare const key: kms.Key;\nnew glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n kmsKey: key,\n },\n});\n```\n\nSee [documentation](https://docs.aws.amazon.com/glue/latest/dg/encryption-security-configuration.html) for more info for Glue encrypting data written by Crawlers, Jobs, and Development Endpoints.\n\n## Database\n\nA `Database` is a logical grouping of `Tables` in the Glue Catalog.\n\n```ts\nnew glue.Database(this, 'MyDatabase');\n```\n\n## Table\n\nA Glue table describes a table of data in S3: its structure (column names and types), location of data (S3 objects with a common prefix in a S3 bucket), and format for the files (Json, Avro, Parquet, etc.):\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }, {\n name: 'col2',\n type: glue.Schema.array(glue.Schema.STRING),\n comment: 'col2 is an array of strings' // comment is optional\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nBy default, a S3 bucket will be created to store the table's data but you can manually pass the `bucket` and `s3Prefix`:\n\n```ts\ndeclare const myBucket: s3.Bucket;\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n bucket: myBucket,\n s3Prefix: 'my-table/',\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nBy default, an S3 bucket will be created to store the table's data and stored in the bucket root. You can also manually pass the `bucket` and `s3Prefix`:\n\n### Partition Keys\n\nTo improve query performance, a table can specify `partitionKeys` on which data is stored and queried separately. For example, you might partition a table by `year` and `month` to optimize queries based on a time window:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n### Partition Indexes\n\nAnother way to improve query performance is to specify partition indexes. If no partition indexes are\npresent on the table, AWS Glue loads all partitions of the table and filters the loaded partitions using\nthe query expression. The query takes more time to run as the number of partitions increase. With an\nindex, the query will try to fetch a subset of the partitions instead of loading all partitions of the\ntable.\n\nThe keys of a partition index must be a subset of the partition keys of the table. You can have a\nmaximum of 3 partition indexes per table. To specify a partition index, you can use the `partitionIndexes`\nproperty:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n partitionIndexes: [{\n indexName: 'my-index', // optional\n keyNames: ['year'],\n }], // supply up to 3 indexes\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nAlternatively, you can call the `addPartitionIndex()` function on a table:\n\n```ts\ndeclare const myTable: glue.Table;\nmyTable.addPartitionIndex({\n indexName: 'my-index',\n keyNames: ['year'],\n});\n```\n\n### Partition Filtering\n\nIf you have a table with a large number of partitions that grows over time, consider using AWS Glue partition indexing and filtering.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n dataFormat: glue.DataFormat.JSON,\n enablePartitionFiltering: true,\n});\n```\n\n## [Encryption](https://docs.aws.amazon.com/athena/latest/ug/encryption.html)\n\nYou can enable encryption on a Table's data:\n\n* [S3Managed](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html) - (default) Server side encryption (`SSE-S3`) with an Amazon S3-managed key.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.S3_MANAGED,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [Kms](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html) - Server-side encryption (`SSE-KMS`) with an AWS KMS Key managed by the account owner.\n\n```ts\ndeclare const myDatabase: glue.Database;\n// KMS key is created automatically\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n\n// with an explicit KMS key\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS,\n encryptionKey: new kms.Key(this, 'MyKey'),\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [KmsManaged](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html) - Server-side encryption (`SSE-KMS`), like `Kms`, except with an AWS KMS Key managed by the AWS Key Management Service.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS_MANAGED,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [ClientSideKms](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html#client-side-encryption-kms-managed-master-key-intro) - Client-side encryption (`CSE-KMS`) with an AWS KMS Key managed by the account owner.\n\n```ts\ndeclare const myDatabase: glue.Database;\n// KMS key is created automatically\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.CLIENT_SIDE_KMS,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n\n// with an explicit KMS key\nnew glue.Table(this, 'MyTable', {\n encryption: glue.TableEncryption.CLIENT_SIDE_KMS,\n encryptionKey: new kms.Key(this, 'MyKey'),\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n*Note: you cannot provide a `Bucket` when creating the `Table` if you wish to use server-side encryption (`KMS`, `KMS_MANAGED` or `S3_MANAGED`)*.\n\n## Types\n\nA table's schema is a collection of columns, each of which have a `name` and a `type`. Types are recursive structures, consisting of primitive and complex types:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.Table(this, 'MyTable', {\n columns: [{\n name: 'primitive_column',\n type: glue.Schema.STRING,\n }, {\n name: 'array_column',\n type: glue.Schema.array(glue.Schema.INTEGER),\n comment: 'array<integer>',\n }, {\n name: 'map_column',\n type: glue.Schema.map(\n glue.Schema.STRING,\n glue.Schema.TIMESTAMP),\n comment: 'map<string,string>',\n }, {\n name: 'struct_column',\n type: glue.Schema.struct([{\n name: 'nested_column',\n type: glue.Schema.DATE,\n comment: 'nested comment',\n }]),\n comment: \"struct<nested_column:date COMMENT 'nested comment'>\",\n }],\n // ...\n database: myDatabase,\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n### Primitives\n\n#### Numeric\n\n| Name \t| Type \t| Comments |\n|-----------\t|----------\t|------------------------------------------------------------------------------------------------------------------\t|\n| FLOAT \t| Constant \t| A 32-bit single-precision floating point number |\n| INTEGER \t| Constant \t| A 32-bit signed value in two's complement format, with a minimum value of -2^31 and a maximum value of 2^31-1 \t|\n| DOUBLE \t| Constant \t| A 64-bit double-precision floating point number |\n| BIG_INT \t| Constant \t| A 64-bit signed INTEGER in two’s complement format, with a minimum value of -2^63 and a maximum value of 2^63 -1 |\n| SMALL_INT \t| Constant \t| A 16-bit signed INTEGER in two’s complement format, with a minimum value of -2^15 and a maximum value of 2^15-1 |\n| TINY_INT \t| Constant \t| A 8-bit signed INTEGER in two’s complement format, with a minimum value of -2^7 and a maximum value of 2^7-1 |\n\n#### Date and time\n\n| Name \t| Type \t| Comments \t|\n|-----------\t|----------\t|-------------------------------------------------------------------------------------------------------------------------------------------------------------------------\t|\n| DATE \t| Constant \t| A date in UNIX format, such as YYYY-MM-DD. \t|\n| TIMESTAMP \t| Constant \t| Date and time instant in the UNiX format, such as yyyy-mm-dd hh:mm:ss[.f...]. For example, TIMESTAMP '2008-09-15 03:04:05.324'. This format uses the session time zone. \t|\n\n#### String\n\n| Name \t| Type \t| Comments \t|\n|--------------------------------------------\t|----------\t|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------\t|\n| STRING \t| Constant \t| A string literal enclosed in single or double quotes \t|\n| decimal(precision: number, scale?: number) \t| Function \t| `precision` is the total number of digits. `scale` (optional) is the number of digits in fractional part with a default of 0. For example, use these type definitions: decimal(11,5), decimal(15) \t|\n| char(length: number) \t| Function \t| Fixed length character data, with a specified length between 1 and 255, such as char(10) \t|\n| varchar(length: number) \t| Function \t| Variable length character data, with a specified length between 1 and 65535, such as varchar(10) \t|\n\n#### Miscellaneous\n\n| Name \t| Type \t| Comments \t|\n|---------\t|----------\t|-------------------------------\t|\n| BOOLEAN \t| Constant \t| Values are `true` and `false` \t|\n| BINARY \t| Constant \t| Value is in binary \t|\n\n### Complex\n\n| Name \t| Type \t| Comments \t|\n|-------------------------------------\t|----------\t|-------------------------------------------------------------------\t|\n| array(itemType: Type) \t| Function \t| An array of some other type \t|\n| map(keyType: Type, valueType: Type) \t| Function \t| A map of some primitive key type to any value type \t|\n| struct(collumns: Column[]) \t| Function \t| Nested structure containing individually named and typed collumns \t|\n"
3504
3517
  },
3505
3518
  "repository": {
3506
3519
  "directory": "packages/@aws-cdk/aws-glue-alpha",
@@ -3865,7 +3878,7 @@
3865
3878
  "docs": {
3866
3879
  "stability": "experimental",
3867
3880
  "summary": "Represents a Glue Job's Code assets (an asset can be a scripts, a jar, a python file or any other file).",
3868
- "example": "new glue.Job(this, 'PythonSparkStreamingJob', {\n executable: glue.JobExecutable.pythonStreaming({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n description: 'an example Python Streaming job',\n});",
3881
+ "example": "declare const bucket: s3.Bucket;\nnew glue.Job(this, 'ScalaSparkEtlJob', {\n executable: glue.JobExecutable.scalaEtl({\n glueVersion: glue.GlueVersion.V4_0,\n script: glue.Code.fromBucket(bucket, 'src/com/example/HelloWorld.scala'),\n className: 'com.example.HelloWorld',\n extraJars: [glue.Code.fromBucket(bucket, 'jars/HelloWorld.jar')],\n }),\n workerType: glue.WorkerType.G_8X,\n description: 'an example Scala ETL job',\n});",
3869
3882
  "custom": {
3870
3883
  "exampleMetadata": "infused"
3871
3884
  }
@@ -4655,7 +4668,7 @@
4655
4668
  "kind": "interface",
4656
4669
  "locationInModule": {
4657
4670
  "filename": "lib/job.ts",
4658
- "line": 404
4671
+ "line": 424
4659
4672
  },
4660
4673
  "name": "ContinuousLoggingProps",
4661
4674
  "properties": [
@@ -4668,7 +4681,7 @@
4668
4681
  "immutable": true,
4669
4682
  "locationInModule": {
4670
4683
  "filename": "lib/job.ts",
4671
- "line": 408
4684
+ "line": 428
4672
4685
  },
4673
4686
  "name": "enabled",
4674
4687
  "type": {
@@ -4686,7 +4699,7 @@
4686
4699
  "immutable": true,
4687
4700
  "locationInModule": {
4688
4701
  "filename": "lib/job.ts",
4689
- "line": 438
4702
+ "line": 458
4690
4703
  },
4691
4704
  "name": "conversionPattern",
4692
4705
  "optional": true,
@@ -4704,7 +4717,7 @@
4704
4717
  "immutable": true,
4705
4718
  "locationInModule": {
4706
4719
  "filename": "lib/job.ts",
4707
- "line": 415
4720
+ "line": 435
4708
4721
  },
4709
4722
  "name": "logGroup",
4710
4723
  "optional": true,
@@ -4722,7 +4735,7 @@
4722
4735
  "immutable": true,
4723
4736
  "locationInModule": {
4724
4737
  "filename": "lib/job.ts",
4725
- "line": 422
4738
+ "line": 442
4726
4739
  },
4727
4740
  "name": "logStreamPrefix",
4728
4741
  "optional": true,
@@ -4740,7 +4753,7 @@
4740
4753
  "immutable": true,
4741
4754
  "locationInModule": {
4742
4755
  "filename": "lib/job.ts",
4743
- "line": 429
4756
+ "line": 449
4744
4757
  },
4745
4758
  "name": "quiet",
4746
4759
  "optional": true,
@@ -5333,13 +5346,45 @@
5333
5346
  ],
5334
5347
  "symbolId": "lib/database:DatabaseProps"
5335
5348
  },
5349
+ "@aws-cdk/aws-glue-alpha.ExecutionClass": {
5350
+ "assembly": "@aws-cdk/aws-glue-alpha",
5351
+ "docs": {
5352
+ "see": "https://docs.aws.amazon.com/glue/latest/dg/add-job.html",
5353
+ "stability": "experimental",
5354
+ "summary": "The ExecutionClass whether the job is run with a standard or flexible execution class."
5355
+ },
5356
+ "fqn": "@aws-cdk/aws-glue-alpha.ExecutionClass",
5357
+ "kind": "enum",
5358
+ "locationInModule": {
5359
+ "filename": "lib/job.ts",
5360
+ "line": 138
5361
+ },
5362
+ "members": [
5363
+ {
5364
+ "docs": {
5365
+ "stability": "experimental",
5366
+ "summary": "The flexible execution class is appropriate for time-insensitive jobs whose start and completion times may vary."
5367
+ },
5368
+ "name": "FLEX"
5369
+ },
5370
+ {
5371
+ "docs": {
5372
+ "stability": "experimental",
5373
+ "summary": "The standard execution class is ideal for time-sensitive workloads that require fast job startup and dedicated resources."
5374
+ },
5375
+ "name": "STANDARD"
5376
+ }
5377
+ ],
5378
+ "name": "ExecutionClass",
5379
+ "symbolId": "lib/job:ExecutionClass"
5380
+ },
5336
5381
  "@aws-cdk/aws-glue-alpha.GlueVersion": {
5337
5382
  "assembly": "@aws-cdk/aws-glue-alpha",
5338
5383
  "docs": {
5339
5384
  "see": "https://docs.aws.amazon.com/glue/latest/dg/add-job.html.\n\nIf you need to use a GlueVersion that doesn't exist as a static member, you\ncan instantiate a `GlueVersion` object, e.g: `GlueVersion.of('1.5')`.",
5340
5385
  "stability": "experimental",
5341
5386
  "summary": "AWS Glue version determines the versions of Apache Spark and Python that are available to the job.",
5342
- "example": "new glue.Job(this, 'PythonSparkStreamingJob', {\n executable: glue.JobExecutable.pythonStreaming({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n description: 'an example Python Streaming job',\n});",
5387
+ "example": "declare const bucket: s3.Bucket;\nnew glue.Job(this, 'ScalaSparkEtlJob', {\n executable: glue.JobExecutable.scalaEtl({\n glueVersion: glue.GlueVersion.V4_0,\n script: glue.Code.fromBucket(bucket, 'src/com/example/HelloWorld.scala'),\n className: 'com.example.HelloWorld',\n extraJars: [glue.Code.fromBucket(bucket, 'jars/HelloWorld.jar')],\n }),\n workerType: glue.WorkerType.G_8X,\n description: 'an example Scala ETL job',\n});",
5343
5388
  "custom": {
5344
5389
  "exampleMetadata": "infused"
5345
5390
  }
@@ -5646,7 +5691,7 @@
5646
5691
  "kind": "interface",
5647
5692
  "locationInModule": {
5648
5693
  "filename": "lib/job.ts",
5649
- "line": 135
5694
+ "line": 155
5650
5695
  },
5651
5696
  "methods": [
5652
5697
  {
@@ -5658,7 +5703,7 @@
5658
5703
  },
5659
5704
  "locationInModule": {
5660
5705
  "filename": "lib/job.ts",
5661
- "line": 192
5706
+ "line": 212
5662
5707
  },
5663
5708
  "name": "metric",
5664
5709
  "parameters": [
@@ -5705,7 +5750,7 @@
5705
5750
  },
5706
5751
  "locationInModule": {
5707
5752
  "filename": "lib/job.ts",
5708
- "line": 202
5753
+ "line": 222
5709
5754
  },
5710
5755
  "name": "metricFailure",
5711
5756
  "parameters": [
@@ -5731,7 +5776,7 @@
5731
5776
  },
5732
5777
  "locationInModule": {
5733
5778
  "filename": "lib/job.ts",
5734
- "line": 197
5779
+ "line": 217
5735
5780
  },
5736
5781
  "name": "metricSuccess",
5737
5782
  "parameters": [
@@ -5757,7 +5802,7 @@
5757
5802
  },
5758
5803
  "locationInModule": {
5759
5804
  "filename": "lib/job.ts",
5760
- "line": 207
5805
+ "line": 227
5761
5806
  },
5762
5807
  "name": "metricTimeout",
5763
5808
  "parameters": [
@@ -5784,7 +5829,7 @@
5784
5829
  },
5785
5830
  "locationInModule": {
5786
5831
  "filename": "lib/job.ts",
5787
- "line": 153
5832
+ "line": 173
5788
5833
  },
5789
5834
  "name": "onEvent",
5790
5835
  "parameters": [
@@ -5817,7 +5862,7 @@
5817
5862
  },
5818
5863
  "locationInModule": {
5819
5864
  "filename": "lib/job.ts",
5820
- "line": 174
5865
+ "line": 194
5821
5866
  },
5822
5867
  "name": "onFailure",
5823
5868
  "parameters": [
@@ -5850,7 +5895,7 @@
5850
5895
  },
5851
5896
  "locationInModule": {
5852
5897
  "filename": "lib/job.ts",
5853
- "line": 160
5898
+ "line": 180
5854
5899
  },
5855
5900
  "name": "onStateChange",
5856
5901
  "parameters": [
@@ -5889,7 +5934,7 @@
5889
5934
  },
5890
5935
  "locationInModule": {
5891
5936
  "filename": "lib/job.ts",
5892
- "line": 167
5937
+ "line": 187
5893
5938
  },
5894
5939
  "name": "onSuccess",
5895
5940
  "parameters": [
@@ -5922,7 +5967,7 @@
5922
5967
  },
5923
5968
  "locationInModule": {
5924
5969
  "filename": "lib/job.ts",
5925
- "line": 181
5970
+ "line": 201
5926
5971
  },
5927
5972
  "name": "onTimeout",
5928
5973
  "parameters": [
@@ -5961,7 +6006,7 @@
5961
6006
  "immutable": true,
5962
6007
  "locationInModule": {
5963
6008
  "filename": "lib/job.ts",
5964
- "line": 146
6009
+ "line": 166
5965
6010
  },
5966
6011
  "name": "jobArn",
5967
6012
  "type": {
@@ -5980,7 +6025,7 @@
5980
6025
  "immutable": true,
5981
6026
  "locationInModule": {
5982
6027
  "filename": "lib/job.ts",
5983
- "line": 140
6028
+ "line": 160
5984
6029
  },
5985
6030
  "name": "jobName",
5986
6031
  "type": {
@@ -6233,7 +6278,7 @@
6233
6278
  "docs": {
6234
6279
  "stability": "experimental",
6235
6280
  "summary": "A Glue Job.",
6236
- "example": "new glue.Job(this, 'PythonSparkStreamingJob', {\n executable: glue.JobExecutable.pythonStreaming({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n description: 'an example Python Streaming job',\n});",
6281
+ "example": "declare const bucket: s3.Bucket;\nnew glue.Job(this, 'ScalaSparkEtlJob', {\n executable: glue.JobExecutable.scalaEtl({\n glueVersion: glue.GlueVersion.V4_0,\n script: glue.Code.fromBucket(bucket, 'src/com/example/HelloWorld.scala'),\n className: 'com.example.HelloWorld',\n extraJars: [glue.Code.fromBucket(bucket, 'jars/HelloWorld.jar')],\n }),\n workerType: glue.WorkerType.G_8X,\n description: 'an example Scala ETL job',\n});",
6237
6282
  "custom": {
6238
6283
  "exampleMetadata": "infused"
6239
6284
  }
@@ -6245,7 +6290,7 @@
6245
6290
  },
6246
6291
  "locationInModule": {
6247
6292
  "filename": "lib/job.ts",
6248
- "line": 654
6293
+ "line": 684
6249
6294
  },
6250
6295
  "parameters": [
6251
6296
  {
@@ -6274,7 +6319,7 @@
6274
6319
  "kind": "class",
6275
6320
  "locationInModule": {
6276
6321
  "filename": "lib/job.ts",
6277
- "line": 608
6322
+ "line": 638
6278
6323
  },
6279
6324
  "methods": [
6280
6325
  {
@@ -6284,7 +6329,7 @@
6284
6329
  },
6285
6330
  "locationInModule": {
6286
6331
  "filename": "lib/job.ts",
6287
- "line": 616
6332
+ "line": 646
6288
6333
  },
6289
6334
  "name": "fromJobAttributes",
6290
6335
  "parameters": [
@@ -6331,7 +6376,7 @@
6331
6376
  },
6332
6377
  "locationInModule": {
6333
6378
  "filename": "lib/job.ts",
6334
- "line": 299
6379
+ "line": 319
6335
6380
  },
6336
6381
  "name": "metric",
6337
6382
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -6379,7 +6424,7 @@
6379
6424
  },
6380
6425
  "locationInModule": {
6381
6426
  "filename": "lib/job.ts",
6382
- "line": 326
6427
+ "line": 346
6383
6428
  },
6384
6429
  "name": "metricFailure",
6385
6430
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -6406,7 +6451,7 @@
6406
6451
  },
6407
6452
  "locationInModule": {
6408
6453
  "filename": "lib/job.ts",
6409
- "line": 317
6454
+ "line": 337
6410
6455
  },
6411
6456
  "name": "metricSuccess",
6412
6457
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -6433,7 +6478,7 @@
6433
6478
  },
6434
6479
  "locationInModule": {
6435
6480
  "filename": "lib/job.ts",
6436
- "line": 335
6481
+ "line": 355
6437
6482
  },
6438
6483
  "name": "metricTimeout",
6439
6484
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -6460,7 +6505,7 @@
6460
6505
  },
6461
6506
  "locationInModule": {
6462
6507
  "filename": "lib/job.ts",
6463
- "line": 227
6508
+ "line": 247
6464
6509
  },
6465
6510
  "name": "onEvent",
6466
6511
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -6499,7 +6544,7 @@
6499
6544
  },
6500
6545
  "locationInModule": {
6501
6546
  "filename": "lib/job.ts",
6502
- "line": 276
6547
+ "line": 296
6503
6548
  },
6504
6549
  "name": "onFailure",
6505
6550
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -6538,7 +6583,7 @@
6538
6583
  },
6539
6584
  "locationInModule": {
6540
6585
  "filename": "lib/job.ts",
6541
- "line": 247
6586
+ "line": 267
6542
6587
  },
6543
6588
  "name": "onStateChange",
6544
6589
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -6585,7 +6630,7 @@
6585
6630
  },
6586
6631
  "locationInModule": {
6587
6632
  "filename": "lib/job.ts",
6588
- "line": 266
6633
+ "line": 286
6589
6634
  },
6590
6635
  "name": "onSuccess",
6591
6636
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -6624,7 +6669,7 @@
6624
6669
  },
6625
6670
  "locationInModule": {
6626
6671
  "filename": "lib/job.ts",
6627
- "line": 286
6672
+ "line": 306
6628
6673
  },
6629
6674
  "name": "onTimeout",
6630
6675
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -6667,7 +6712,7 @@
6667
6712
  "immutable": true,
6668
6713
  "locationInModule": {
6669
6714
  "filename": "lib/job.ts",
6670
- "line": 644
6715
+ "line": 674
6671
6716
  },
6672
6717
  "name": "grantPrincipal",
6673
6718
  "overrides": "aws-cdk-lib.aws_iam.IGrantable",
@@ -6683,7 +6728,7 @@
6683
6728
  "immutable": true,
6684
6729
  "locationInModule": {
6685
6730
  "filename": "lib/job.ts",
6686
- "line": 629
6731
+ "line": 659
6687
6732
  },
6688
6733
  "name": "jobArn",
6689
6734
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -6699,7 +6744,7 @@
6699
6744
  "immutable": true,
6700
6745
  "locationInModule": {
6701
6746
  "filename": "lib/job.ts",
6702
- "line": 634
6747
+ "line": 664
6703
6748
  },
6704
6749
  "name": "jobName",
6705
6750
  "overrides": "@aws-cdk/aws-glue-alpha.IJob",
@@ -6715,7 +6760,7 @@
6715
6760
  "immutable": true,
6716
6761
  "locationInModule": {
6717
6762
  "filename": "lib/job.ts",
6718
- "line": 639
6763
+ "line": 669
6719
6764
  },
6720
6765
  "name": "role",
6721
6766
  "type": {
@@ -6731,7 +6776,7 @@
6731
6776
  "immutable": true,
6732
6777
  "locationInModule": {
6733
6778
  "filename": "lib/job.ts",
6734
- "line": 652
6779
+ "line": 682
6735
6780
  },
6736
6781
  "name": "sparkUILoggingLocation",
6737
6782
  "optional": true,
@@ -6757,7 +6802,7 @@
6757
6802
  "kind": "interface",
6758
6803
  "locationInModule": {
6759
6804
  "filename": "lib/job.ts",
6760
- "line": 444
6805
+ "line": 464
6761
6806
  },
6762
6807
  "name": "JobAttributes",
6763
6808
  "properties": [
@@ -6770,7 +6815,7 @@
6770
6815
  "immutable": true,
6771
6816
  "locationInModule": {
6772
6817
  "filename": "lib/job.ts",
6773
- "line": 448
6818
+ "line": 468
6774
6819
  },
6775
6820
  "name": "jobName",
6776
6821
  "type": {
@@ -6787,7 +6832,7 @@
6787
6832
  "immutable": true,
6788
6833
  "locationInModule": {
6789
6834
  "filename": "lib/job.ts",
6790
- "line": 455
6835
+ "line": 475
6791
6836
  },
6792
6837
  "name": "role",
6793
6838
  "optional": true,
@@ -6889,7 +6934,7 @@
6889
6934
  "docs": {
6890
6935
  "stability": "experimental",
6891
6936
  "summary": "The executable properties related to the Glue job's GlueVersion, JobType and code.",
6892
- "example": "new glue.Job(this, 'PythonSparkStreamingJob', {\n executable: glue.JobExecutable.pythonStreaming({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n description: 'an example Python Streaming job',\n});",
6937
+ "example": "declare const bucket: s3.Bucket;\nnew glue.Job(this, 'ScalaSparkEtlJob', {\n executable: glue.JobExecutable.scalaEtl({\n glueVersion: glue.GlueVersion.V4_0,\n script: glue.Code.fromBucket(bucket, 'src/com/example/HelloWorld.scala'),\n className: 'com.example.HelloWorld',\n extraJars: [glue.Code.fromBucket(bucket, 'jars/HelloWorld.jar')],\n }),\n workerType: glue.WorkerType.G_8X,\n description: 'an example Scala ETL job',\n});",
6893
6938
  "custom": {
6894
6939
  "exampleMetadata": "infused"
6895
6940
  }
@@ -6898,7 +6943,7 @@
6898
6943
  "kind": "class",
6899
6944
  "locationInModule": {
6900
6945
  "filename": "lib/job-executable.ts",
6901
- "line": 227
6946
+ "line": 261
6902
6947
  },
6903
6948
  "methods": [
6904
6949
  {
@@ -6908,7 +6953,7 @@
6908
6953
  },
6909
6954
  "locationInModule": {
6910
6955
  "filename": "lib/job-executable.ts",
6911
- "line": 312
6956
+ "line": 346
6912
6957
  },
6913
6958
  "name": "of",
6914
6959
  "parameters": [
@@ -6936,7 +6981,7 @@
6936
6981
  },
6937
6982
  "locationInModule": {
6938
6983
  "filename": "lib/job-executable.ts",
6939
- "line": 260
6984
+ "line": 294
6940
6985
  },
6941
6986
  "name": "pythonEtl",
6942
6987
  "parameters": [
@@ -6964,7 +7009,7 @@
6964
7009
  },
6965
7010
  "locationInModule": {
6966
7011
  "filename": "lib/job-executable.ts",
6967
- "line": 299
7012
+ "line": 333
6968
7013
  },
6969
7014
  "name": "pythonRay",
6970
7015
  "parameters": [
@@ -6992,7 +7037,7 @@
6992
7037
  },
6993
7038
  "locationInModule": {
6994
7039
  "filename": "lib/job-executable.ts",
6995
- "line": 286
7040
+ "line": 320
6996
7041
  },
6997
7042
  "name": "pythonShell",
6998
7043
  "parameters": [
@@ -7020,7 +7065,7 @@
7020
7065
  },
7021
7066
  "locationInModule": {
7022
7067
  "filename": "lib/job-executable.ts",
7023
- "line": 273
7068
+ "line": 307
7024
7069
  },
7025
7070
  "name": "pythonStreaming",
7026
7071
  "parameters": [
@@ -7048,7 +7093,7 @@
7048
7093
  },
7049
7094
  "locationInModule": {
7050
7095
  "filename": "lib/job-executable.ts",
7051
- "line": 234
7096
+ "line": 268
7052
7097
  },
7053
7098
  "name": "scalaEtl",
7054
7099
  "parameters": [
@@ -7076,7 +7121,7 @@
7076
7121
  },
7077
7122
  "locationInModule": {
7078
7123
  "filename": "lib/job-executable.ts",
7079
- "line": 247
7124
+ "line": 281
7080
7125
  },
7081
7126
  "name": "scalaStreaming",
7082
7127
  "parameters": [
@@ -7104,7 +7149,7 @@
7104
7149
  },
7105
7150
  "locationInModule": {
7106
7151
  "filename": "lib/job-executable.ts",
7107
- "line": 356
7152
+ "line": 393
7108
7153
  },
7109
7154
  "name": "bind",
7110
7155
  "returns": {
@@ -7123,7 +7168,7 @@
7123
7168
  "docs": {
7124
7169
  "stability": "experimental",
7125
7170
  "summary": "Result of binding a `JobExecutable` into a `Job`.",
7126
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\n\ndeclare const code: glue_alpha.Code;\ndeclare const glueVersion: glue_alpha.GlueVersion;\ndeclare const jobType: glue_alpha.JobType;\nconst jobExecutableConfig: glue_alpha.JobExecutableConfig = {\n glueVersion: glueVersion,\n language: glue_alpha.JobLanguage.SCALA,\n script: code,\n type: jobType,\n\n // the properties below are optional\n className: 'className',\n extraFiles: [code],\n extraJars: [code],\n extraJarsFirst: false,\n extraPythonFiles: [code],\n pythonVersion: glue_alpha.PythonVersion.TWO,\n};",
7171
+ "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\n\ndeclare const code: glue_alpha.Code;\ndeclare const glueVersion: glue_alpha.GlueVersion;\ndeclare const jobType: glue_alpha.JobType;\ndeclare const runtime: glue_alpha.Runtime;\nconst jobExecutableConfig: glue_alpha.JobExecutableConfig = {\n glueVersion: glueVersion,\n language: glue_alpha.JobLanguage.SCALA,\n script: code,\n type: jobType,\n\n // the properties below are optional\n className: 'className',\n extraFiles: [code],\n extraJars: [code],\n extraJarsFirst: false,\n extraPythonFiles: [code],\n pythonVersion: glue_alpha.PythonVersion.TWO,\n runtime: runtime,\n};",
7127
7172
  "custom": {
7128
7173
  "exampleMetadata": "fixture=_generated"
7129
7174
  }
@@ -7132,7 +7177,7 @@
7132
7177
  "kind": "interface",
7133
7178
  "locationInModule": {
7134
7179
  "filename": "lib/job-executable.ts",
7135
- "line": 364
7180
+ "line": 401
7136
7181
  },
7137
7182
  "name": "JobExecutableConfig",
7138
7183
  "properties": [
@@ -7146,7 +7191,7 @@
7146
7191
  "immutable": true,
7147
7192
  "locationInModule": {
7148
7193
  "filename": "lib/job-executable.ts",
7149
- "line": 370
7194
+ "line": 407
7150
7195
  },
7151
7196
  "name": "glueVersion",
7152
7197
  "type": {
@@ -7163,7 +7208,7 @@
7163
7208
  "immutable": true,
7164
7209
  "locationInModule": {
7165
7210
  "filename": "lib/job-executable.ts",
7166
- "line": 377
7211
+ "line": 414
7167
7212
  },
7168
7213
  "name": "language",
7169
7214
  "type": {
@@ -7179,7 +7224,7 @@
7179
7224
  "immutable": true,
7180
7225
  "locationInModule": {
7181
7226
  "filename": "lib/job-executable.ts",
7182
- "line": 394
7227
+ "line": 438
7183
7228
  },
7184
7229
  "name": "script",
7185
7230
  "type": {
@@ -7195,7 +7240,7 @@
7195
7240
  "immutable": true,
7196
7241
  "locationInModule": {
7197
7242
  "filename": "lib/job-executable.ts",
7198
- "line": 382
7243
+ "line": 419
7199
7244
  },
7200
7245
  "name": "type",
7201
7246
  "type": {
@@ -7214,7 +7259,7 @@
7214
7259
  "immutable": true,
7215
7260
  "locationInModule": {
7216
7261
  "filename": "lib/job-executable.ts",
7217
- "line": 403
7262
+ "line": 447
7218
7263
  },
7219
7264
  "name": "className",
7220
7265
  "optional": true,
@@ -7233,7 +7278,7 @@
7233
7278
  "immutable": true,
7234
7279
  "locationInModule": {
7235
7280
  "filename": "lib/job-executable.ts",
7236
- "line": 430
7281
+ "line": 474
7237
7282
  },
7238
7283
  "name": "extraFiles",
7239
7284
  "optional": true,
@@ -7257,7 +7302,7 @@
7257
7302
  "immutable": true,
7258
7303
  "locationInModule": {
7259
7304
  "filename": "lib/job-executable.ts",
7260
- "line": 412
7305
+ "line": 456
7261
7306
  },
7262
7307
  "name": "extraJars",
7263
7308
  "optional": true,
@@ -7281,7 +7326,7 @@
7281
7326
  "immutable": true,
7282
7327
  "locationInModule": {
7283
7328
  "filename": "lib/job-executable.ts",
7284
- "line": 439
7329
+ "line": 483
7285
7330
  },
7286
7331
  "name": "extraJarsFirst",
7287
7332
  "optional": true,
@@ -7300,7 +7345,7 @@
7300
7345
  "immutable": true,
7301
7346
  "locationInModule": {
7302
7347
  "filename": "lib/job-executable.ts",
7303
- "line": 421
7348
+ "line": 465
7304
7349
  },
7305
7350
  "name": "extraPythonFiles",
7306
7351
  "optional": true,
@@ -7323,13 +7368,31 @@
7323
7368
  "immutable": true,
7324
7369
  "locationInModule": {
7325
7370
  "filename": "lib/job-executable.ts",
7326
- "line": 389
7371
+ "line": 426
7327
7372
  },
7328
7373
  "name": "pythonVersion",
7329
7374
  "optional": true,
7330
7375
  "type": {
7331
7376
  "fqn": "@aws-cdk/aws-glue-alpha.PythonVersion"
7332
7377
  }
7378
+ },
7379
+ {
7380
+ "abstract": true,
7381
+ "docs": {
7382
+ "default": "- no runtime specified",
7383
+ "stability": "experimental",
7384
+ "summary": "The Runtime to use."
7385
+ },
7386
+ "immutable": true,
7387
+ "locationInModule": {
7388
+ "filename": "lib/job-executable.ts",
7389
+ "line": 433
7390
+ },
7391
+ "name": "runtime",
7392
+ "optional": true,
7393
+ "type": {
7394
+ "fqn": "@aws-cdk/aws-glue-alpha.Runtime"
7395
+ }
7333
7396
  }
7334
7397
  ],
7335
7398
  "symbolId": "lib/job-executable:JobExecutableConfig"
@@ -7371,7 +7434,7 @@
7371
7434
  "docs": {
7372
7435
  "stability": "experimental",
7373
7436
  "summary": "Construction properties for `Job`.",
7374
- "example": "new glue.Job(this, 'PythonSparkStreamingJob', {\n executable: glue.JobExecutable.pythonStreaming({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n description: 'an example Python Streaming job',\n});",
7437
+ "example": "declare const bucket: s3.Bucket;\nnew glue.Job(this, 'ScalaSparkEtlJob', {\n executable: glue.JobExecutable.scalaEtl({\n glueVersion: glue.GlueVersion.V4_0,\n script: glue.Code.fromBucket(bucket, 'src/com/example/HelloWorld.scala'),\n className: 'com.example.HelloWorld',\n extraJars: [glue.Code.fromBucket(bucket, 'jars/HelloWorld.jar')],\n }),\n workerType: glue.WorkerType.G_8X,\n description: 'an example Scala ETL job',\n});",
7375
7438
  "custom": {
7376
7439
  "exampleMetadata": "infused"
7377
7440
  }
@@ -7380,7 +7443,7 @@
7380
7443
  "kind": "interface",
7381
7444
  "locationInModule": {
7382
7445
  "filename": "lib/job.ts",
7383
- "line": 461
7446
+ "line": 481
7384
7447
  },
7385
7448
  "name": "JobProps",
7386
7449
  "properties": [
@@ -7393,7 +7456,7 @@
7393
7456
  "immutable": true,
7394
7457
  "locationInModule": {
7395
7458
  "filename": "lib/job.ts",
7396
- "line": 465
7459
+ "line": 485
7397
7460
  },
7398
7461
  "name": "executable",
7399
7462
  "type": {
@@ -7411,7 +7474,7 @@
7411
7474
  "immutable": true,
7412
7475
  "locationInModule": {
7413
7476
  "filename": "lib/job.ts",
7414
- "line": 540
7477
+ "line": 560
7415
7478
  },
7416
7479
  "name": "connections",
7417
7480
  "optional": true,
@@ -7435,7 +7498,7 @@
7435
7498
  "immutable": true,
7436
7499
  "locationInModule": {
7437
7500
  "filename": "lib/job.ts",
7438
- "line": 602
7501
+ "line": 622
7439
7502
  },
7440
7503
  "name": "continuousLogging",
7441
7504
  "optional": true,
@@ -7454,7 +7517,7 @@
7454
7517
  "immutable": true,
7455
7518
  "locationInModule": {
7456
7519
  "filename": "lib/job.ts",
7457
- "line": 555
7520
+ "line": 575
7458
7521
  },
7459
7522
  "name": "defaultArguments",
7460
7523
  "optional": true,
@@ -7477,7 +7540,7 @@
7477
7540
  "immutable": true,
7478
7541
  "locationInModule": {
7479
7542
  "filename": "lib/job.ts",
7480
- "line": 479
7543
+ "line": 499
7481
7544
  },
7482
7545
  "name": "description",
7483
7546
  "optional": true,
@@ -7496,7 +7559,7 @@
7496
7559
  "immutable": true,
7497
7560
  "locationInModule": {
7498
7561
  "filename": "lib/job.ts",
7499
- "line": 582
7562
+ "line": 602
7500
7563
  },
7501
7564
  "name": "enableProfilingMetrics",
7502
7565
  "optional": true,
@@ -7504,6 +7567,25 @@
7504
7567
  "primitive": "boolean"
7505
7568
  }
7506
7569
  },
7570
+ {
7571
+ "abstract": true,
7572
+ "docs": {
7573
+ "default": "- STANDARD",
7574
+ "see": "https://docs.aws.amazon.com/glue/latest/dg/add-job.html",
7575
+ "stability": "experimental",
7576
+ "summary": "The ExecutionClass whether the job is run with a standard or flexible execution class."
7577
+ },
7578
+ "immutable": true,
7579
+ "locationInModule": {
7580
+ "filename": "lib/job.ts",
7581
+ "line": 632
7582
+ },
7583
+ "name": "executionClass",
7584
+ "optional": true,
7585
+ "type": {
7586
+ "fqn": "@aws-cdk/aws-glue-alpha.ExecutionClass"
7587
+ }
7588
+ },
7507
7589
  {
7508
7590
  "abstract": true,
7509
7591
  "docs": {
@@ -7514,7 +7596,7 @@
7514
7596
  "immutable": true,
7515
7597
  "locationInModule": {
7516
7598
  "filename": "lib/job.ts",
7517
- "line": 472
7599
+ "line": 492
7518
7600
  },
7519
7601
  "name": "jobName",
7520
7602
  "optional": true,
@@ -7533,7 +7615,7 @@
7533
7615
  "immutable": true,
7534
7616
  "locationInModule": {
7535
7617
  "filename": "lib/job.ts",
7536
- "line": 487
7618
+ "line": 507
7537
7619
  },
7538
7620
  "name": "maxCapacity",
7539
7621
  "optional": true,
@@ -7552,7 +7634,7 @@
7552
7634
  "immutable": true,
7553
7635
  "locationInModule": {
7554
7636
  "filename": "lib/job.ts",
7555
- "line": 503
7637
+ "line": 523
7556
7638
  },
7557
7639
  "name": "maxConcurrentRuns",
7558
7640
  "optional": true,
@@ -7570,7 +7652,7 @@
7570
7652
  "immutable": true,
7571
7653
  "locationInModule": {
7572
7654
  "filename": "lib/job.ts",
7573
- "line": 494
7655
+ "line": 514
7574
7656
  },
7575
7657
  "name": "maxRetries",
7576
7658
  "optional": true,
@@ -7588,7 +7670,7 @@
7588
7670
  "immutable": true,
7589
7671
  "locationInModule": {
7590
7672
  "filename": "lib/job.ts",
7591
- "line": 510
7673
+ "line": 530
7592
7674
  },
7593
7675
  "name": "notifyDelayAfter",
7594
7676
  "optional": true,
@@ -7608,7 +7690,7 @@
7608
7690
  "immutable": true,
7609
7691
  "locationInModule": {
7610
7692
  "filename": "lib/job.ts",
7611
- "line": 573
7693
+ "line": 593
7612
7694
  },
7613
7695
  "name": "role",
7614
7696
  "optional": true,
@@ -7626,7 +7708,7 @@
7626
7708
  "immutable": true,
7627
7709
  "locationInModule": {
7628
7710
  "filename": "lib/job.ts",
7629
- "line": 547
7711
+ "line": 567
7630
7712
  },
7631
7713
  "name": "securityConfiguration",
7632
7714
  "optional": true,
@@ -7645,7 +7727,7 @@
7645
7727
  "immutable": true,
7646
7728
  "locationInModule": {
7647
7729
  "filename": "lib/job.ts",
7648
- "line": 592
7730
+ "line": 612
7649
7731
  },
7650
7732
  "name": "sparkUI",
7651
7733
  "optional": true,
@@ -7663,7 +7745,7 @@
7663
7745
  "immutable": true,
7664
7746
  "locationInModule": {
7665
7747
  "filename": "lib/job.ts",
7666
- "line": 562
7748
+ "line": 582
7667
7749
  },
7668
7750
  "name": "tags",
7669
7751
  "optional": true,
@@ -7686,7 +7768,7 @@
7686
7768
  "immutable": true,
7687
7769
  "locationInModule": {
7688
7770
  "filename": "lib/job.ts",
7689
- "line": 517
7771
+ "line": 537
7690
7772
  },
7691
7773
  "name": "timeout",
7692
7774
  "optional": true,
@@ -7704,7 +7786,7 @@
7704
7786
  "immutable": true,
7705
7787
  "locationInModule": {
7706
7788
  "filename": "lib/job.ts",
7707
- "line": 531
7789
+ "line": 551
7708
7790
  },
7709
7791
  "name": "workerCount",
7710
7792
  "optional": true,
@@ -7722,7 +7804,7 @@
7722
7804
  "immutable": true,
7723
7805
  "locationInModule": {
7724
7806
  "filename": "lib/job.ts",
7725
- "line": 524
7807
+ "line": 544
7726
7808
  },
7727
7809
  "name": "workerType",
7728
7810
  "optional": true,
@@ -7815,7 +7897,7 @@
7815
7897
  "kind": "class",
7816
7898
  "locationInModule": {
7817
7899
  "filename": "lib/job-executable.ts",
7818
- "line": 96
7900
+ "line": 124
7819
7901
  },
7820
7902
  "methods": [
7821
7903
  {
@@ -7825,7 +7907,7 @@
7825
7907
  },
7826
7908
  "locationInModule": {
7827
7909
  "filename": "lib/job-executable.ts",
7828
- "line": 121
7910
+ "line": 149
7829
7911
  },
7830
7912
  "name": "of",
7831
7913
  "parameters": [
@@ -7858,7 +7940,7 @@
7858
7940
  "immutable": true,
7859
7941
  "locationInModule": {
7860
7942
  "filename": "lib/job-executable.ts",
7861
- "line": 100
7943
+ "line": 128
7862
7944
  },
7863
7945
  "name": "ETL",
7864
7946
  "static": true,
@@ -7875,7 +7957,7 @@
7875
7957
  "immutable": true,
7876
7958
  "locationInModule": {
7877
7959
  "filename": "lib/job-executable.ts",
7878
- "line": 110
7960
+ "line": 138
7879
7961
  },
7880
7962
  "name": "PYTHON_SHELL",
7881
7963
  "static": true,
@@ -7892,7 +7974,7 @@
7892
7974
  "immutable": true,
7893
7975
  "locationInModule": {
7894
7976
  "filename": "lib/job-executable.ts",
7895
- "line": 115
7977
+ "line": 143
7896
7978
  },
7897
7979
  "name": "RAY",
7898
7980
  "static": true,
@@ -7909,7 +7991,7 @@
7909
7991
  "immutable": true,
7910
7992
  "locationInModule": {
7911
7993
  "filename": "lib/job-executable.ts",
7912
- "line": 105
7994
+ "line": 133
7913
7995
  },
7914
7996
  "name": "STREAMING",
7915
7997
  "static": true,
@@ -7925,7 +8007,7 @@
7925
8007
  "immutable": true,
7926
8008
  "locationInModule": {
7927
8009
  "filename": "lib/job-executable.ts",
7928
- "line": 128
8010
+ "line": 156
7929
8011
  },
7930
8012
  "name": "name",
7931
8013
  "type": {
@@ -8159,7 +8241,7 @@
8159
8241
  "docs": {
8160
8242
  "stability": "experimental",
8161
8243
  "summary": "Props for creating a Python Ray job executable.",
8162
- "example": "new glue.Job(this, 'RayJob', {\n executable: glue.JobExecutable.pythonRay({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE_NINE,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n workerType: glue.WorkerType.Z_2X,\n workerCount: 2,\n description: 'an example Ray job'\n});",
8244
+ "example": "new glue.Job(this, 'RayJob', {\n executable: glue.JobExecutable.pythonRay({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE_NINE,\n runtime: glue.Runtime.RAY_TWO_FOUR,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n workerType: glue.WorkerType.Z_2X,\n workerCount: 2,\n description: 'an example Ray job'\n});",
8163
8245
  "custom": {
8164
8246
  "exampleMetadata": "infused"
8165
8247
  }
@@ -8168,7 +8250,7 @@
8168
8250
  "kind": "interface",
8169
8251
  "locationInModule": {
8170
8252
  "filename": "lib/job-executable.ts",
8171
- "line": 222
8253
+ "line": 256
8172
8254
  },
8173
8255
  "name": "PythonRayExecutableProps",
8174
8256
  "properties": [
@@ -8182,7 +8264,7 @@
8182
8264
  "immutable": true,
8183
8265
  "locationInModule": {
8184
8266
  "filename": "lib/job-executable.ts",
8185
- "line": 158
8267
+ "line": 192
8186
8268
  },
8187
8269
  "name": "glueVersion",
8188
8270
  "type": {
@@ -8198,7 +8280,7 @@
8198
8280
  "immutable": true,
8199
8281
  "locationInModule": {
8200
8282
  "filename": "lib/job-executable.ts",
8201
- "line": 139
8283
+ "line": 167
8202
8284
  },
8203
8285
  "name": "pythonVersion",
8204
8286
  "type": {
@@ -8214,7 +8296,7 @@
8214
8296
  "immutable": true,
8215
8297
  "locationInModule": {
8216
8298
  "filename": "lib/job-executable.ts",
8217
- "line": 163
8299
+ "line": 197
8218
8300
  },
8219
8301
  "name": "script",
8220
8302
  "type": {
@@ -8233,7 +8315,7 @@
8233
8315
  "immutable": true,
8234
8316
  "locationInModule": {
8235
8317
  "filename": "lib/job-executable.ts",
8236
- "line": 173
8318
+ "line": 207
8237
8319
  },
8238
8320
  "name": "extraFiles",
8239
8321
  "optional": true,
@@ -8258,7 +8340,7 @@
8258
8340
  "immutable": true,
8259
8341
  "locationInModule": {
8260
8342
  "filename": "lib/job-executable.ts",
8261
- "line": 149
8343
+ "line": 177
8262
8344
  },
8263
8345
  "name": "extraPythonFiles",
8264
8346
  "optional": true,
@@ -8270,6 +8352,24 @@
8270
8352
  "kind": "array"
8271
8353
  }
8272
8354
  }
8355
+ },
8356
+ {
8357
+ "abstract": true,
8358
+ "docs": {
8359
+ "remarks": "It is required for Ray jobs.",
8360
+ "stability": "experimental",
8361
+ "summary": "Runtime."
8362
+ },
8363
+ "immutable": true,
8364
+ "locationInModule": {
8365
+ "filename": "lib/job-executable.ts",
8366
+ "line": 185
8367
+ },
8368
+ "name": "runtime",
8369
+ "optional": true,
8370
+ "type": {
8371
+ "fqn": "@aws-cdk/aws-glue-alpha.Runtime"
8372
+ }
8273
8373
  }
8274
8374
  ],
8275
8375
  "symbolId": "lib/job-executable:PythonRayExecutableProps"
@@ -8289,7 +8389,7 @@
8289
8389
  "kind": "interface",
8290
8390
  "locationInModule": {
8291
8391
  "filename": "lib/job-executable.ts",
8292
- "line": 217
8392
+ "line": 251
8293
8393
  },
8294
8394
  "name": "PythonShellExecutableProps",
8295
8395
  "properties": [
@@ -8303,7 +8403,7 @@
8303
8403
  "immutable": true,
8304
8404
  "locationInModule": {
8305
8405
  "filename": "lib/job-executable.ts",
8306
- "line": 158
8406
+ "line": 192
8307
8407
  },
8308
8408
  "name": "glueVersion",
8309
8409
  "type": {
@@ -8319,7 +8419,7 @@
8319
8419
  "immutable": true,
8320
8420
  "locationInModule": {
8321
8421
  "filename": "lib/job-executable.ts",
8322
- "line": 139
8422
+ "line": 167
8323
8423
  },
8324
8424
  "name": "pythonVersion",
8325
8425
  "type": {
@@ -8335,7 +8435,7 @@
8335
8435
  "immutable": true,
8336
8436
  "locationInModule": {
8337
8437
  "filename": "lib/job-executable.ts",
8338
- "line": 163
8438
+ "line": 197
8339
8439
  },
8340
8440
  "name": "script",
8341
8441
  "type": {
@@ -8354,7 +8454,7 @@
8354
8454
  "immutable": true,
8355
8455
  "locationInModule": {
8356
8456
  "filename": "lib/job-executable.ts",
8357
- "line": 173
8457
+ "line": 207
8358
8458
  },
8359
8459
  "name": "extraFiles",
8360
8460
  "optional": true,
@@ -8379,7 +8479,7 @@
8379
8479
  "immutable": true,
8380
8480
  "locationInModule": {
8381
8481
  "filename": "lib/job-executable.ts",
8382
- "line": 149
8482
+ "line": 177
8383
8483
  },
8384
8484
  "name": "extraPythonFiles",
8385
8485
  "optional": true,
@@ -8391,6 +8491,24 @@
8391
8491
  "kind": "array"
8392
8492
  }
8393
8493
  }
8494
+ },
8495
+ {
8496
+ "abstract": true,
8497
+ "docs": {
8498
+ "remarks": "It is required for Ray jobs.",
8499
+ "stability": "experimental",
8500
+ "summary": "Runtime."
8501
+ },
8502
+ "immutable": true,
8503
+ "locationInModule": {
8504
+ "filename": "lib/job-executable.ts",
8505
+ "line": 185
8506
+ },
8507
+ "name": "runtime",
8508
+ "optional": true,
8509
+ "type": {
8510
+ "fqn": "@aws-cdk/aws-glue-alpha.Runtime"
8511
+ }
8394
8512
  }
8395
8513
  ],
8396
8514
  "symbolId": "lib/job-executable:PythonShellExecutableProps"
@@ -8410,7 +8528,7 @@
8410
8528
  "kind": "interface",
8411
8529
  "locationInModule": {
8412
8530
  "filename": "lib/job-executable.ts",
8413
- "line": 212
8531
+ "line": 246
8414
8532
  },
8415
8533
  "name": "PythonSparkJobExecutableProps",
8416
8534
  "properties": [
@@ -8424,7 +8542,7 @@
8424
8542
  "immutable": true,
8425
8543
  "locationInModule": {
8426
8544
  "filename": "lib/job-executable.ts",
8427
- "line": 158
8545
+ "line": 192
8428
8546
  },
8429
8547
  "name": "glueVersion",
8430
8548
  "type": {
@@ -8440,7 +8558,7 @@
8440
8558
  "immutable": true,
8441
8559
  "locationInModule": {
8442
8560
  "filename": "lib/job-executable.ts",
8443
- "line": 139
8561
+ "line": 167
8444
8562
  },
8445
8563
  "name": "pythonVersion",
8446
8564
  "type": {
@@ -8456,7 +8574,7 @@
8456
8574
  "immutable": true,
8457
8575
  "locationInModule": {
8458
8576
  "filename": "lib/job-executable.ts",
8459
- "line": 163
8577
+ "line": 197
8460
8578
  },
8461
8579
  "name": "script",
8462
8580
  "type": {
@@ -8475,7 +8593,7 @@
8475
8593
  "immutable": true,
8476
8594
  "locationInModule": {
8477
8595
  "filename": "lib/job-executable.ts",
8478
- "line": 173
8596
+ "line": 207
8479
8597
  },
8480
8598
  "name": "extraFiles",
8481
8599
  "optional": true,
@@ -8499,7 +8617,7 @@
8499
8617
  "immutable": true,
8500
8618
  "locationInModule": {
8501
8619
  "filename": "lib/job-executable.ts",
8502
- "line": 185
8620
+ "line": 219
8503
8621
  },
8504
8622
  "name": "extraJars",
8505
8623
  "optional": true,
@@ -8523,7 +8641,7 @@
8523
8641
  "immutable": true,
8524
8642
  "locationInModule": {
8525
8643
  "filename": "lib/job-executable.ts",
8526
- "line": 194
8644
+ "line": 228
8527
8645
  },
8528
8646
  "name": "extraJarsFirst",
8529
8647
  "optional": true,
@@ -8543,7 +8661,7 @@
8543
8661
  "immutable": true,
8544
8662
  "locationInModule": {
8545
8663
  "filename": "lib/job-executable.ts",
8546
- "line": 149
8664
+ "line": 177
8547
8665
  },
8548
8666
  "name": "extraPythonFiles",
8549
8667
  "optional": true,
@@ -8555,6 +8673,24 @@
8555
8673
  "kind": "array"
8556
8674
  }
8557
8675
  }
8676
+ },
8677
+ {
8678
+ "abstract": true,
8679
+ "docs": {
8680
+ "remarks": "It is required for Ray jobs.",
8681
+ "stability": "experimental",
8682
+ "summary": "Runtime."
8683
+ },
8684
+ "immutable": true,
8685
+ "locationInModule": {
8686
+ "filename": "lib/job-executable.ts",
8687
+ "line": 185
8688
+ },
8689
+ "name": "runtime",
8690
+ "optional": true,
8691
+ "type": {
8692
+ "fqn": "@aws-cdk/aws-glue-alpha.Runtime"
8693
+ }
8558
8694
  }
8559
8695
  ],
8560
8696
  "symbolId": "lib/job-executable:PythonSparkJobExecutableProps"
@@ -8601,6 +8737,89 @@
8601
8737
  "name": "PythonVersion",
8602
8738
  "symbolId": "lib/job-executable:PythonVersion"
8603
8739
  },
8740
+ "@aws-cdk/aws-glue-alpha.Runtime": {
8741
+ "assembly": "@aws-cdk/aws-glue-alpha",
8742
+ "docs": {
8743
+ "stability": "experimental",
8744
+ "summary": "AWS Glue runtime determines the runtime engine of the job.",
8745
+ "example": "new glue.Job(this, 'RayJob', {\n executable: glue.JobExecutable.pythonRay({\n glueVersion: glue.GlueVersion.V4_0,\n pythonVersion: glue.PythonVersion.THREE_NINE,\n runtime: glue.Runtime.RAY_TWO_FOUR,\n script: glue.Code.fromAsset(path.join(__dirname, 'job-script/hello_world.py')),\n }),\n workerType: glue.WorkerType.Z_2X,\n workerCount: 2,\n description: 'an example Ray job'\n});",
8746
+ "custom": {
8747
+ "exampleMetadata": "infused"
8748
+ }
8749
+ },
8750
+ "fqn": "@aws-cdk/aws-glue-alpha.Runtime",
8751
+ "kind": "class",
8752
+ "locationInModule": {
8753
+ "filename": "lib/job-executable.ts",
8754
+ "line": 94
8755
+ },
8756
+ "methods": [
8757
+ {
8758
+ "docs": {
8759
+ "stability": "experimental",
8760
+ "summary": "Custom runtime."
8761
+ },
8762
+ "locationInModule": {
8763
+ "filename": "lib/job-executable.ts",
8764
+ "line": 104
8765
+ },
8766
+ "name": "of",
8767
+ "parameters": [
8768
+ {
8769
+ "docs": {
8770
+ "summary": "custom runtime."
8771
+ },
8772
+ "name": "runtime",
8773
+ "type": {
8774
+ "primitive": "string"
8775
+ }
8776
+ }
8777
+ ],
8778
+ "returns": {
8779
+ "type": {
8780
+ "fqn": "@aws-cdk/aws-glue-alpha.Runtime"
8781
+ }
8782
+ },
8783
+ "static": true
8784
+ }
8785
+ ],
8786
+ "name": "Runtime",
8787
+ "properties": [
8788
+ {
8789
+ "const": true,
8790
+ "docs": {
8791
+ "stability": "experimental",
8792
+ "summary": "Runtime for a Glue for Ray 2.4."
8793
+ },
8794
+ "immutable": true,
8795
+ "locationInModule": {
8796
+ "filename": "lib/job-executable.ts",
8797
+ "line": 98
8798
+ },
8799
+ "name": "RAY_TWO_FOUR",
8800
+ "static": true,
8801
+ "type": {
8802
+ "fqn": "@aws-cdk/aws-glue-alpha.Runtime"
8803
+ }
8804
+ },
8805
+ {
8806
+ "docs": {
8807
+ "stability": "experimental",
8808
+ "summary": "The name of this Runtime."
8809
+ },
8810
+ "immutable": true,
8811
+ "locationInModule": {
8812
+ "filename": "lib/job-executable.ts",
8813
+ "line": 111
8814
+ },
8815
+ "name": "name",
8816
+ "type": {
8817
+ "primitive": "string"
8818
+ }
8819
+ }
8820
+ ],
8821
+ "symbolId": "lib/job-executable:Runtime"
8822
+ },
8604
8823
  "@aws-cdk/aws-glue-alpha.S3Code": {
8605
8824
  "assembly": "@aws-cdk/aws-glue-alpha",
8606
8825
  "base": "@aws-cdk/aws-glue-alpha.Code",
@@ -8786,7 +9005,7 @@
8786
9005
  "kind": "interface",
8787
9006
  "locationInModule": {
8788
9007
  "filename": "lib/job-executable.ts",
8789
- "line": 200
9008
+ "line": 234
8790
9009
  },
8791
9010
  "name": "ScalaJobExecutableProps",
8792
9011
  "properties": [
@@ -8800,7 +9019,7 @@
8800
9019
  "immutable": true,
8801
9020
  "locationInModule": {
8802
9021
  "filename": "lib/job-executable.ts",
8803
- "line": 206
9022
+ "line": 240
8804
9023
  },
8805
9024
  "name": "className",
8806
9025
  "type": {
@@ -8817,7 +9036,7 @@
8817
9036
  "immutable": true,
8818
9037
  "locationInModule": {
8819
9038
  "filename": "lib/job-executable.ts",
8820
- "line": 158
9039
+ "line": 192
8821
9040
  },
8822
9041
  "name": "glueVersion",
8823
9042
  "type": {
@@ -8833,7 +9052,7 @@
8833
9052
  "immutable": true,
8834
9053
  "locationInModule": {
8835
9054
  "filename": "lib/job-executable.ts",
8836
- "line": 163
9055
+ "line": 197
8837
9056
  },
8838
9057
  "name": "script",
8839
9058
  "type": {
@@ -8852,7 +9071,7 @@
8852
9071
  "immutable": true,
8853
9072
  "locationInModule": {
8854
9073
  "filename": "lib/job-executable.ts",
8855
- "line": 173
9074
+ "line": 207
8856
9075
  },
8857
9076
  "name": "extraFiles",
8858
9077
  "optional": true,
@@ -8876,7 +9095,7 @@
8876
9095
  "immutable": true,
8877
9096
  "locationInModule": {
8878
9097
  "filename": "lib/job-executable.ts",
8879
- "line": 185
9098
+ "line": 219
8880
9099
  },
8881
9100
  "name": "extraJars",
8882
9101
  "optional": true,
@@ -8900,13 +9119,31 @@
8900
9119
  "immutable": true,
8901
9120
  "locationInModule": {
8902
9121
  "filename": "lib/job-executable.ts",
8903
- "line": 194
9122
+ "line": 228
8904
9123
  },
8905
9124
  "name": "extraJarsFirst",
8906
9125
  "optional": true,
8907
9126
  "type": {
8908
9127
  "primitive": "boolean"
8909
9128
  }
9129
+ },
9130
+ {
9131
+ "abstract": true,
9132
+ "docs": {
9133
+ "remarks": "It is required for Ray jobs.",
9134
+ "stability": "experimental",
9135
+ "summary": "Runtime."
9136
+ },
9137
+ "immutable": true,
9138
+ "locationInModule": {
9139
+ "filename": "lib/job-executable.ts",
9140
+ "line": 185
9141
+ },
9142
+ "name": "runtime",
9143
+ "optional": true,
9144
+ "type": {
9145
+ "fqn": "@aws-cdk/aws-glue-alpha.Runtime"
9146
+ }
8910
9147
  }
8911
9148
  ],
8912
9149
  "symbolId": "lib/job-executable:ScalaJobExecutableProps"
@@ -9819,7 +10056,7 @@
9819
10056
  "kind": "interface",
9820
10057
  "locationInModule": {
9821
10058
  "filename": "lib/job.ts",
9822
- "line": 384
10059
+ "line": 404
9823
10060
  },
9824
10061
  "name": "SparkUILoggingLocation",
9825
10062
  "properties": [
@@ -9832,7 +10069,7 @@
9832
10069
  "immutable": true,
9833
10070
  "locationInModule": {
9834
10071
  "filename": "lib/job.ts",
9835
- "line": 388
10072
+ "line": 408
9836
10073
  },
9837
10074
  "name": "bucket",
9838
10075
  "type": {
@@ -9849,7 +10086,7 @@
9849
10086
  "immutable": true,
9850
10087
  "locationInModule": {
9851
10088
  "filename": "lib/job.ts",
9852
- "line": 395
10089
+ "line": 415
9853
10090
  },
9854
10091
  "name": "prefix",
9855
10092
  "optional": true,
@@ -9876,7 +10113,7 @@
9876
10113
  "kind": "interface",
9877
10114
  "locationInModule": {
9878
10115
  "filename": "lib/job.ts",
9879
- "line": 357
10116
+ "line": 377
9880
10117
  },
9881
10118
  "name": "SparkUIProps",
9882
10119
  "properties": [
@@ -9889,7 +10126,7 @@
9889
10126
  "immutable": true,
9890
10127
  "locationInModule": {
9891
10128
  "filename": "lib/job.ts",
9892
- "line": 361
10129
+ "line": 381
9893
10130
  },
9894
10131
  "name": "enabled",
9895
10132
  "type": {
@@ -9906,7 +10143,7 @@
9906
10143
  "immutable": true,
9907
10144
  "locationInModule": {
9908
10145
  "filename": "lib/job.ts",
9909
- "line": 368
10146
+ "line": 388
9910
10147
  },
9911
10148
  "name": "bucket",
9912
10149
  "optional": true,
@@ -9924,7 +10161,7 @@
9924
10161
  "immutable": true,
9925
10162
  "locationInModule": {
9926
10163
  "filename": "lib/job.ts",
9927
- "line": 375
10164
+ "line": 395
9928
10165
  },
9929
10166
  "name": "prefix",
9930
10167
  "optional": true,
@@ -11083,6 +11320,6 @@
11083
11320
  "symbolId": "lib/job:WorkerType"
11084
11321
  }
11085
11322
  },
11086
- "version": "2.85.0-alpha.0",
11323
+ "version": "2.87.0-alpha.0",
11087
11324
  "fingerprint": "**********"
11088
11325
  }