@aws-cdk/aws-glue-alpha 2.265.0-alpha.0 → 2.266.0-alpha.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. package/.jsii +504 -436
  2. package/.jsii.tabl.json.gz +0 -0
  3. package/.warnings.jsii.js +25 -33
  4. package/README.md +111 -43
  5. package/awslint.json +2 -1
  6. package/lib/catalog.js +3 -3
  7. package/lib/code.js +3 -3
  8. package/lib/connection.js +2 -2
  9. package/lib/data-format.d.ts +2 -2
  10. package/lib/data-format.js +8 -8
  11. package/lib/data-quality-ruleset.d.ts +28 -3
  12. package/lib/data-quality-ruleset.js +37 -5
  13. package/lib/database.d.ts +1 -1
  14. package/lib/database.js +2 -2
  15. package/lib/external-table.js +6 -4
  16. package/lib/jobs/job.d.ts +1 -24
  17. package/lib/jobs/job.js +3 -3
  18. package/lib/jobs/pyspark-etl-job.d.ts +2 -1
  19. package/lib/jobs/pyspark-etl-job.js +10 -5
  20. package/lib/jobs/pyspark-flex-etl-job.d.ts +3 -2
  21. package/lib/jobs/pyspark-flex-etl-job.js +7 -6
  22. package/lib/jobs/pyspark-streaming-job.d.ts +2 -1
  23. package/lib/jobs/pyspark-streaming-job.js +6 -5
  24. package/lib/jobs/python-shell-job.js +1 -1
  25. package/lib/jobs/ray-job.d.ts +8 -0
  26. package/lib/jobs/ray-job.js +3 -7
  27. package/lib/jobs/scala-spark-etl-job.js +4 -8
  28. package/lib/jobs/scala-spark-flex-etl-job.js +4 -4
  29. package/lib/jobs/scala-spark-streaming-job.js +4 -8
  30. package/lib/jobs/spark-job.d.ts +26 -0
  31. package/lib/jobs/spark-job.js +2 -2
  32. package/lib/partition-projection.js +1 -1
  33. package/lib/s3-table.d.ts +2 -1
  34. package/lib/s3-table.js +5 -3
  35. package/lib/schema.js +3 -8
  36. package/lib/security-configuration.d.ts +41 -47
  37. package/lib/security-configuration.js +83 -25
  38. package/lib/storage-parameter.js +1 -1
  39. package/lib/table-base.d.ts +33 -0
  40. package/lib/table-base.js +33 -2
  41. package/lib/table-deprecated.js +1 -1
  42. package/lib/triggers/trigger-options.js +1 -1
  43. package/lib/triggers/workflow.js +2 -2
  44. package/package.json +7 -7
package/.jsii CHANGED
@@ -8,7 +8,7 @@
8
8
  "url": "https://aws.amazon.com"
9
9
  },
10
10
  "dependencies": {
11
- "aws-cdk-lib": "^2.265.0",
11
+ "aws-cdk-lib": "^2.266.0",
12
12
  "constructs": "^10.5.0"
13
13
  },
14
14
  "dependencyClosure": {
@@ -9409,7 +9409,7 @@
9409
9409
  },
9410
9410
  "name": "@aws-cdk/aws-glue-alpha",
9411
9411
  "readme": {
9412
- "markdown": "# AWS Glue Construct Library\n<!--BEGIN STABILITY BANNER-->\n\n---\n\n![cdk-constructs: Experimental](https://img.shields.io/badge/cdk--constructs-experimental-important.svg?style=for-the-badge)\n\n> The APIs of higher level constructs in this module are experimental and under active development.\n> They are subject to non-backward compatible changes or removal in any future version. These are\n> not subject to the [Semantic Versioning](https://semver.org/) model and breaking changes will be\n> announced in the release notes. This means that while you may use them, you may need to update\n> your source code when upgrading to a newer version of this package.\n\n---\n\n<!--END STABILITY BANNER-->\n\nThis module is part of the [AWS Cloud Development Kit](https://github.com/aws/aws-cdk) project.\n\n## README\n\n[AWS Glue](https://aws.amazon.com/glue/) is a serverless data integration\nservice that makes it easier to discover, prepare, move, and integrate data\nfrom multiple sources for analytics, machine learning (ML), and application\ndevelopment.\n\nThe Glue L2 construct has convenience methods working backwards from common\nuse cases and sets required parameters to defaults that align with recommended\nbest practices for each job type. It also provides customers with a balance\nbetween flexibility via optional parameter overrides, and opinionated\ninterfaces that discouraging anti-patterns, resulting in reduced time to develop\nand deploy new resources.\n\n### References\n\n* [Glue Launch Announcement](https://aws.amazon.com/blogs/aws/launch-aws-glue-now-generally-available/)\n* [Glue Documentation](https://docs.aws.amazon.com/glue/index.html)\n* [Glue L1 (CloudFormation) Constructs](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/AWS_Glue.html)\n* Prior version of the [@aws-cdk/aws-glue-alpha module](https://github.com/aws/aws-cdk/blob/v2.51.1/packages/%40aws-cdk/aws-glue/README.md)\n\n## Create a Glue Job\n\nA Job encapsulates a script that connects to data sources, processes\nthem, and then writes output to a data target. There are four types of Glue\nJobs: Spark (ETL and Streaming), Python Shell, and Flex Jobs. Most\nof the required parameters for these jobs are common across all types,\nbut there are a few differences depending on the languages supported\nand features provided by each type. For all job types, the L2 defaults\nto AWS best practice recommendations, such as:\n\n* Use of Secrets Manager for Connection JDBC strings\n* Glue job autoscaling\n* Default parameter values for Glue job creation\n\nThis iteration of the L2 construct introduces breaking changes to\nthe existing glue-alpha-module, but these changes streamline the developer\nexperience, introduce new constants for defaults, and replacing synth-time\nvalidations with interface contracts for enforcement of the parameter combinations\nthat Glue supports. As an opinionated construct, the Glue L2 construct does\nnot allow developers to create resources that use non-current versions\nof Glue or deprecated language dependencies (e.g. deprecated versions of Python).\nAs always, L1s allow you to specify a wider range of parameters if you need\nor want to use alternative configurations.\n\nOptional and required parameters for each job are enforced via interface\nrather than validation; see [Glue's public documentation](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api.html)\nfor more granular details.\n\n### Spark Jobs\n\n#### ETL Jobs\n\nETL jobs support pySpark and Scala languages, for which there are separate but\nsimilar constructors. ETL jobs default to the G2 worker type, but you can\noverride this default with other supported worker type values (G1, G2, G4\nand G8). ETL jobs defaults to Glue version 4.0, which you can override to 3.0.\nThe following ETL features are enabled by default:\n`—enable-metrics, —enable-spark-ui, —enable-continuous-cloudwatch-log.`\nYou can find more details about version, worker type and other features in\n[Glue's public documentation](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-jobs-job.html).\n\nReference the pyspark-etl-jobs.test.ts and scalaspark-etl-jobs.test.ts unit tests\nfor examples of required-only and optional job parameters when creating these\ntypes of jobs.\n\nFor the sake of brevity, examples are shown using the pySpark job variety.\n\nExample with only required parameters:\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkEtlJob(stack, 'PySparkETLJob', {\n role,\n script,\n jobName: 'PySparkETLJob',\n});\n```\n\nExample with optional override parameters:\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkEtlJob(stack, 'PySparkETLJob', {\n jobName: 'PySparkETLJobCustomName',\n description: 'This is a description',\n role,\n script,\n glueVersion: glue.GlueVersion.V5_1,\n continuousLogging: { enabled: false },\n workerType: glue.WorkerType.G_2X,\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n numberOfWorkers: 2,\n maxRetries: 2,\n});\n```\n\n#### Streaming Jobs\n\nStreaming jobs are similar to ETL jobs, except that they perform ETL on data\nstreams using the Apache Spark Structured Streaming framework. Some Spark\njob features are not available to Streaming ETL jobs. They support Scala\nand pySpark languages. PySpark streaming jobs default Python 3.9,\nwhich you can override with any non-deprecated version of Python. It\ndefaults to the G2 worker type and Glue 4.0, both of which you can override.\nThe following best practice features are enabled by default:\n`—enable-metrics, —enable-spark-ui, —enable-continuous-cloudwatch-log`.\n\nReference the pyspark-streaming-jobs.test.ts and scalaspark-streaming-jobs.test.ts\nunit tests for examples of required-only and optional job parameters when creating\nthese types of jobs.\n\nExample with only required parameters:\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkStreamingJob(stack, 'ImportedJob', { role, script });\n```\n\nExample with optional override parameters:\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkStreamingJob(stack, 'PySparkStreamingJob', {\n jobName: 'PySparkStreamingJobCustomName',\n description: 'This is a description',\n role,\n script,\n glueVersion: glue.GlueVersion.V5_1,\n continuousLogging: { enabled: false },\n workerType: glue.WorkerType.G_2X,\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n numberOfWorkers: 2,\n maxRetries: 2,\n});\n```\n\n#### Flex Jobs\n\nThe flexible execution class is appropriate for non-urgent jobs such as\npre-production jobs, testing, and one-time data loads. Flexible jobs default\nto Glue version 5.0 and worker type `G_2X`. The following best practice\nfeatures are enabled by default:\n`—enable-metrics, —enable-spark-ui, —enable-continuous-cloudwatch-log`\n\nReference the pyspark-flex-etl-jobs.test.ts and scalaspark-flex-etl-jobs.test.ts\nunit tests for examples of required-only and optional job parameters when creating\nthese types of jobs.\n\nExample with only required parameters:\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkFlexEtlJob(stack, 'ImportedJob', { role, script });\n```\n\nExample with optional override parameters:\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkEtlJob(stack, 'pySparkEtlJob', {\n jobName: 'pySparkEtlJob',\n description: 'This is a description',\n role,\n script,\n glueVersion: glue.GlueVersion.V5_1,\n continuousLogging: { enabled: false },\n workerType: glue.WorkerType.G_2X,\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n numberOfWorkers: 2,\n maxRetries: 2,\n});\n```\n\n### Python Shell Jobs\n\nPython shell jobs support a Python version that depends on the AWS Glue\nversion you use. These can be used to schedule and run tasks that don't\nrequire an Apache Spark environment. Python shell jobs default to\nPython 3.9 and a MaxCapacity of `0.0625`. Python 3.9 supports pre-loaded\nanalytics libraries using the `library-set=analytics` flag, which is\nenabled by default.\n\nReference the pyspark-shell-job.test.ts unit tests for examples of\nrequired-only and optional job parameters when creating these types of jobs.\n\nExample with only required parameters:\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PythonShellJob(stack, 'ImportedJob', { role, script });\n```\n\nExample with optional override parameters:\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\ndeclare const extraPythonFile: glue.Code;\nnew glue.PythonShellJob(stack, 'PythonShellJob', {\n jobName: 'PythonShellJobCustomName',\n description: 'This is a description',\n pythonVersion: glue.PythonVersion.TWO,\n maxCapacity: glue.MaxCapacity.DPU_1,\n role,\n script,\n extraPythonFiles: [extraPythonFile],\n glueVersion: glue.GlueVersion.V2_0,\n continuousLogging: { enabled: false },\n workerType: glue.WorkerType.G_2X,\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n numberOfWorkers: 2,\n maxRetries: 2,\n});\n```\n\n### Ray Jobs\n\n> **⚠️ DEPRECATED:** AWS Glue for Ray is closed to new customers as of April 30, 2026 and is in maintenance mode.\n> Migrate to [Amazon EKS with KubeRay Operator](https://docs.aws.amazon.com/glue/latest/dg/awsglue-ray-jobs-availability-change.html).\n\nThe `RayJob` construct, `Runtime.RAY_TWO_FOUR`, and `JobType.RAY` are deprecated and will be removed in a future release.\n\n### Metrics Control\n\nBy default, Glue jobs enable CloudWatch metrics (`--enable-metrics`) and observability metrics (`--enable-observability-metrics`) for monitoring and debugging. You can disable these metrics to reduce CloudWatch costs:\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\n\n// Disable both metrics for cost optimization\nnew glue.PySparkEtlJob(stack, 'CostOptimizedJob', {\n role,\n script,\n enableMetrics: false,\n enableObservabilityMetrics: false,\n});\n\n// Selective control - keep observability, disable profiling\nnew glue.PySparkEtlJob(stack, 'SelectiveJob', {\n role,\n script,\n enableMetrics: false,\n // enableObservabilityMetrics defaults to true\n});\n```\n\nThis feature is available for all Spark job types (ETL, Streaming, Flex).\n\n### Enable Job Run Queuing\n\nAWS Glue job queuing monitors your account level quotas and limits. If quotas or limits are insufficient to start a Glue job run, AWS Glue will automatically queue the job and wait for limits to free up. Once limits become available, AWS Glue will retry the job run. Glue jobs will queue for limits like max concurrent job runs per account, max concurrent Data Processing Units (DPU), and resource unavailable due to IP address exhaustion in Amazon Virtual Private Cloud (Amazon VPC).\n\nEnable job run queuing by setting the `jobRunQueuingEnabled` property to `true`.\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkEtlJob(stack, 'PySparkETLJob', {\n role,\n script,\n jobName: 'PySparkETLJob',\n jobRunQueuingEnabled: true\n});\n```\n\n### Uploading scripts from the CDK app repository to S3\n\nSimilar to other L2 constructs, the Glue L2 automates uploading / updating\nscripts to S3 via an optional fromAsset parameter pointing to a script\nin the local file structure. You provide the existing S3 bucket and\npath to which you'd like the script to be uploaded.\n\nReference the unit tests for examples of repo and S3 code target examples.\n\n### Workflow Triggers\n\nYou can use Glue workflows to create and visualize complex\nextract, transform, and load (ETL) activities involving multiple crawlers,\njobs, and triggers. Standalone triggers are an anti-pattern, so you must\ncreate triggers from within a workflow using the L2 construct.\n\nWithin a workflow object, there are functions to create different\ntypes of triggers with actions and predicates. You then add those triggers\nto jobs.\n\nStartOnCreation defaults to true for all trigger types, but you can\noverride it if you prefer for your trigger not to start on creation.\n\nReference the workflow-triggers.test.ts unit tests for examples of creating\nworkflows and triggers.\n\n#### **1. On-Demand Triggers**\n\nOn-demand triggers can start glue jobs or crawlers. This construct provides\nconvenience functions to create on-demand crawler or job triggers. The constructor\ntakes an optional description parameter, but abstracts the requirement of an\nactions list using the job or crawler objects using conditional types.\n\n#### **2. Scheduled Triggers**\n\nYou can create scheduled triggers using cron expressions. This construct\nprovides daily, weekly, and monthly convenience functions,\nas well as a custom function that allows you to create your own\ncustom timing using the [existing event Schedule class](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_events.Schedule.html)\nwithout having to build your own cron expressions. The L2 extracts\nthe expression that Glue requires from the Schedule object. The constructor\ntakes an optional description and a list of jobs or crawlers as actions.\n\n#### **3. Notify Event Triggers**\n\nThere are two types of notify event triggers: batching and non-batching.\nFor batching triggers, you must specify `BatchSize`. For non-batching\ntriggers, `BatchSize` defaults to 1. For both triggers, `BatchWindow`\ndefaults to 900 seconds, but you can override the window to align with\nyour workload's requirements.\n\n#### **4. Conditional Triggers**\n\nConditional triggers have a predicate and actions associated with them.\nThe trigger actions are executed when the predicateCondition is true.\n\n### Connection Properties\n\nA `Connection` allows Glue jobs, crawlers and development endpoints to access\ncertain types of data stores.\n\n* **Secrets Management**\n You must specify JDBC connection credentials in Secrets Manager and\n provide the Secrets Manager Key name as a property to the job connection.\n\n* **Networking - the CDK determines the best fit subnet for Glue connection\nconfiguration**\n The prior version of the glue-alpha-module requires the developer to\n specify the subnet of the Connection when it’s defined. Now, you can still\n specify the specific subnet you want to use, but are no longer required\n to. You are only required to provide a VPC and either a public or private\n subnet selection. Without a specific subnet provided, the L2 leverages the\n existing [EC2 Subnet Selection](https://docs.aws.amazon.com/cdk/api/v2/python/aws_cdk.aws_ec2/SubnetSelection.html)\n library to make the best choice selection for the subnet.\n\n```ts\ndeclare const securityGroup: ec2.SecurityGroup;\ndeclare const subnet: ec2.Subnet;\nnew glue.Connection(this, 'MyConnection', {\n type: glue.ConnectionType.NETWORK,\n // The security groups granting AWS Glue inbound access to the data source within the VPC\n securityGroups: [securityGroup],\n // The VPC subnet which contains the data source\n subnet,\n});\n```\n\nFor RDS `Connection` by JDBC, it is recommended to manage credentials using AWS Secrets Manager. To use Secret, specify `SECRET_ID` in `properties` like the following code. Note that in this case, the subnet must have a route to the AWS Secrets Manager VPC endpoint or to the AWS Secrets Manager endpoint through a NAT gateway.\n\n```ts\ndeclare const securityGroup: ec2.SecurityGroup;\ndeclare const subnet: ec2.Subnet;\ndeclare const db: rds.DatabaseCluster;\nnew glue.Connection(this, \"RdsConnection\", {\n type: glue.ConnectionType.JDBC,\n securityGroups: [securityGroup],\n subnet,\n properties: {\n JDBC_CONNECTION_URL: `jdbc:mysql://${db.clusterEndpoint.socketAddress}/databasename`,\n JDBC_ENFORCE_SSL: \"false\",\n SECRET_ID: db.secret!.secretName,\n },\n});\n```\n\nConnection `properties` are emitted verbatim into the CloudFormation template, so\nany credential placed there in plaintext is stored in plaintext in the template,\n`cdk.out`, and source control. Reference a Secrets Manager secret through\n`SECRET_ID` (as above) instead. If a property key looks like a credential (for\nexample `PASSWORD`, `SECRET`, or `TOKEN`) and holds a plaintext literal, the\nconstruct emits a synthesis-time warning.\n\nIf you need to use a connection type that doesn't exist as a static member on `ConnectionType`, you can instantiate a `ConnectionType` object, e.g: `new glue.ConnectionType('NEW_TYPE')`.\n\nSee [Adding a Connection to Your Data Store](https://docs.aws.amazon.com/glue/latest/dg/populate-add-connection.html) and [Connection Structure](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-catalog-connections.html#aws-glue-api-catalog-connections-Connection) documentation for more information on the supported data stores and their configurations.\n\n## SecurityConfiguration\n\nA `SecurityConfiguration` is a set of security properties that can be used by AWS Glue to encrypt data at rest.\n\n```ts\nnew glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n },\n jobBookmarksEncryption: {\n mode: glue.JobBookmarksEncryptionMode.CLIENT_SIDE_KMS,\n },\n s3Encryption: {\n mode: glue.S3EncryptionMode.KMS,\n },\n});\n```\n\nBy default, a shared KMS key is created for use with the encryption configurations that require one. You can also supply your own key for each encryption config, for example, for CloudWatch encryption:\n\n```ts\ndeclare const key: kms.Key;\nnew glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n kmsKey: key,\n },\n});\n```\n\nSee [documentation](https://docs.aws.amazon.com/glue/latest/dg/encryption-security-configuration.html) for more info for Glue encrypting data written by Crawlers, Jobs, and Development Endpoints.\n\n## Catalog\n\nThe Glue Data Catalog is a persistent metadata store for your data assets. Every\naccount has an implicit, account-wide catalog that always exists, and you can also\ncreate additional catalogs as `AWS::Glue::Catalog` resources (for example, to\nfederate to another metastore).\n\nA catalog's encryption is fixed when the catalog is created: a catalog either\ncarries encryption settings or it does not. This keeps its configuration easy to\nreason about — there are no mutation methods that change encryption after the fact.\n\n### The account-wide catalog\n\nUse `Catalog.forAccount(scope)` to obtain the implicit account catalog. It is not\na CloudFormation resource — it always exists. Repeated calls within the same stack\nreturn the same instance:\n\n```ts\nconst catalog = glue.Catalog.forAccount(this);\n```\n\nTo configure Data Catalog encryption for the account, use\n`Catalog.encryptAccount(scope, options)`:\n\n```ts\ndeclare const key: kms.Key;\nglue.Catalog.encryptAccount(this, {\n encryptionAtRest: glue.DataCatalogEncryptionAtRest.kms(key),\n});\n```\n\nBecause encryption is fixed at construction, `encryptAccount` must be called\n*before* the account catalog is first used in the stack — before any\n`Catalog.forAccount(this)` call, and before any `Database` that uses the account\ncatalog. Calling it after the account catalog has been materialized throws.\n\nThe account catalog's encryption is an account- and region-wide setting, managed\nthrough the singleton `PutDataCatalogEncryptionSettings` API. Configure it in\nexactly one stack. Configuring it from multiple stacks in the same account and\nregion makes those stacks overwrite one another at deploy time, and the result is\norder-dependent. Unlike duplicate settings within a single stack (which\nCloudFormation rejects), this cross-stack conflict is not caught at synthesis\ntime, because each stack synthesizes to its own template.\n\n### Creating a catalog\n\nTo create a new catalog resource, use the `Catalog` constructor. Encryption is\nconfigured through the `encryptionAtRest` and `connectionPasswordEncryption` props:\n\n```ts\nnew glue.Catalog(this, 'MyCatalog', {\n catalogName: 'my-catalog',\n description: 'my catalog description',\n});\n```\n\n### Encryption at rest\n\nConfigure Data Catalog encryption at rest through the `encryptionAtRest` option\n(on `Catalog.encryptAccount`, the `Catalog` constructor, or the import factories).\nIt accepts a `DataCatalogEncryptionAtRest` describing the mode:\n\n```ts\ndeclare const key: kms.Key;\n\n// SSE-KMS with a customer-managed key\nglue.Catalog.encryptAccount(this, {\n encryptionAtRest: glue.DataCatalogEncryptionAtRest.kms(key),\n});\n\n// SSE-KMS with an AWS-managed key (omit the key)\nnew glue.Catalog(this, 'ManagedKeyCatalog', {\n catalogName: 'managed-key-catalog',\n encryptionAtRest: glue.DataCatalogEncryptionAtRest.kms(),\n});\n\n// Disable encryption at rest\nnew glue.Catalog(this, 'PlaintextCatalog', {\n catalogName: 'plaintext-catalog',\n encryptionAtRest: glue.DataCatalogEncryptionAtRest.disabled(),\n});\n```\n\nWhen you use `SSE-KMS-WITH-SERVICE-ROLE`, AWS Glue accesses the KMS key through a\nservice role you provide. If you pass a customer-managed key, the role is\nautomatically granted the permissions it needs to encrypt and decrypt catalog data:\n\n```ts\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const key: kms.Key;\ndeclare const role: iam.IRole;\nglue.Catalog.encryptAccount(this, {\n encryptionAtRest: glue.DataCatalogEncryptionAtRest.kmsWithServiceRole(role, key),\n});\n```\n\nThe customer-managed key, when configured, is exposed on the catalog as\n`encryptionKey` (and the connection-password key as `connectionPasswordKey`), so\nyou can reference it to grant additional access. It is undefined when encryption is\ndisabled or an AWS-managed key is used.\n\n### Connection password encryption\n\nIndependently from encryption at rest, the Data Catalog can encrypt the passwords\nstored in connection properties. Configure it through the\n`connectionPasswordEncryption` option:\n\n```ts\ndeclare const key: kms.Key;\nglue.Catalog.encryptAccount(this, {\n connectionPasswordEncryption: {\n kmsKey: key,\n // Whether GetConnection/GetConnections return the password encrypted (default: true)\n returnConnectionPasswordEncrypted: true,\n },\n});\n```\n\nThe two encryption blocks are independent: enabling one does not require the other,\nand each may use a different KMS key. The customer-managed key for connection\npasswords is exposed as `connectionPasswordKey`.\n\n### Importing a catalog\n\nYou can import an existing catalog by ARN or by id. An imported catalog is a pure\nidentity handle — it emits no resources and does not manage the catalog's\nencryption:\n\n```ts\nconst byId = glue.Catalog.fromCatalogId(this, 'ById', 'my-catalog-id');\nconst byArn = glue.Catalog.fromCatalogArn(this, 'ByArn', 'arn:aws:glue:us-east-1:123456789012:catalog/my-catalog-id');\n```\n\nTo manage the Data Catalog encryption of a catalog you did not create in this\nstack, add a `CfnDataCatalogEncryptionSettings` resource targeting its id\ndirectly. Do this from exactly one stack: like the account catalog, a catalog has\na single encryption configuration, so two settings resources targeting the same id\nrace to overwrite one another at deploy time. Within a single stack this is caught\nby CloudFormation template validation (E3019, duplicate primary identifiers);\nacross stacks it is not, since each stack synthesizes to its own template.\n\n```ts\nimport { CfnDataCatalogEncryptionSettings } from 'aws-cdk-lib/aws-glue';\n\nnew CfnDataCatalogEncryptionSettings(this, 'Encryption', {\n catalogId: 'my-catalog-id',\n dataCatalogEncryptionSettings: {\n encryptionAtRest: { catalogEncryptionMode: 'SSE-KMS' },\n },\n});\n```\n\n## Database\n\nA `Database` is a logical grouping of `Tables` in the Glue Catalog.\n\n```ts\nnew glue.Database(this, 'MyDatabase', {\n databaseName: 'my_database',\n description: 'my_database_description',\n});\n```\n\nBecause a database is a container for tables and their metadata, it is retained\nby default when removed from the stack, to avoid accidental data loss. Set\n`removalPolicy` to `RemovalPolicy.DESTROY` to have it deleted instead:\n\n```ts\nimport { RemovalPolicy } from 'aws-cdk-lib';\n\nnew glue.Database(this, 'MyDatabase', {\n databaseName: 'my_database',\n removalPolicy: RemovalPolicy.DESTROY,\n});\n```\n\n## Table\n\nA Glue table describes a table of data in S3: its structure (column names and types), location of data (S3 objects with a common prefix in a S3 bucket), and format for the files (Json, Avro, Parquet, etc.):\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }, {\n name: 'col2',\n type: glue.Schema.array(glue.Schema.STRING),\n comment: 'col2 is an array of strings' // comment is optional\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nBy default, a S3 bucket will be created to store the table's data but you can manually pass the `bucket` and `s3Prefix`:\n\n```ts\ndeclare const myBucket: s3.Bucket;\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n bucket: myBucket,\n s3Prefix: 'my-table/',\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nGlue tables can be configured to contain user-defined properties, to describe the physical storage of table data, through the `storageParameters` property:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n storageParameters: [\n glue.StorageParameter.skipHeaderLineCount(1),\n glue.StorageParameter.compressionType(glue.CompressionType.GZIP),\n glue.StorageParameter.custom('separatorChar', ',')\n ],\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nGlue tables can also be configured to contain user-defined table properties through the [`parameters`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-glue-table-tableinput.html#cfn-glue-table-tableinput-parameters) property:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n parameters: {\n key1: 'val1',\n key2: 'val2',\n },\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n### Partition Keys\n\nTo improve query performance, a table can specify `partitionKeys` on which data is stored and queried separately. For example, you might partition a table by `year` and `month` to optimize queries based on a time window:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n### Partition Indexes\n\nAnother way to improve query performance is to specify partition indexes. If no partition indexes are\npresent on the table, AWS Glue loads all partitions of the table and filters the loaded partitions using\nthe query expression. The query takes more time to run as the number of partitions increase. With an\nindex, the query will try to fetch a subset of the partitions instead of loading all partitions of the\ntable.\n\nThe keys of a partition index must be a subset of the partition keys of the table. You can have a\nmaximum of 3 partition indexes per table. To specify a partition index, you can use the `partitionIndexes`\nproperty:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n partitionIndexes: [{\n indexName: 'my-index', // optional\n keyNames: ['year'],\n }], // supply up to 3 indexes\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nAlternatively, you can call the `addPartitionIndex()` function on a table:\n\n```ts\ndeclare const myTable: glue.Table;\nmyTable.addPartitionIndex({\n indexName: 'my-index',\n keyNames: ['year'],\n});\n```\n\n### Partition Filtering\n\nIf you have a table with a large number of partitions that grows over time, consider using AWS Glue partition indexing and filtering.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n dataFormat: glue.DataFormat.JSON,\n enablePartitionFiltering: true,\n});\n```\n\n### Partition Projection\n\nPartition projection allows Athena to automatically add new partitions as new data arrives, without requiring `ALTER TABLE ADD PARTITION` statements. This improves query performance and reduces management overhead by eliminating the need to manually manage partition metadata.\n\nFor more information, see the [AWS documentation on partition projection](https://docs.aws.amazon.com/athena/latest/ug/partition-projection.html).\n\n#### INTEGER Projection\n\nFor partition keys with sequential numeric values:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'data',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.INTEGER,\n }],\n dataFormat: glue.DataFormat.JSON,\n partitionProjection: {\n year: glue.PartitionProjectionConfiguration.integer({\n min: 2020,\n max: 2023,\n interval: 1, // optional, defaults to 1\n digits: 4, // optional, pads with leading zeros\n }),\n },\n});\n```\n\n#### DATE Projection\n\nFor partition keys with date or timestamp values. Supports both fixed dates and relative dates using `NOW`:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'data',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'date',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n partitionProjection: {\n date: glue.PartitionProjectionConfiguration.date({\n min: '2020-01-01',\n max: '2023-12-31',\n format: 'yyyy-MM-dd',\n interval: 1, // optional, defaults to 1\n intervalUnit: glue.DateIntervalUnit.DAYS, // optional: YEARS, MONTHS, WEEKS, DAYS, HOURS, MINUTES, SECONDS\n }),\n },\n});\n```\n\nYou can also use relative dates with `NOW`:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'data',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'date',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n partitionProjection: {\n date: glue.PartitionProjectionConfiguration.date({\n min: 'NOW-3YEARS',\n max: 'NOW',\n format: 'yyyy-MM-dd',\n }),\n },\n});\n```\n\n#### ENUM Projection\n\nFor partition keys with a known set of values:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'data',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'region',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n partitionProjection: {\n region: glue.PartitionProjectionConfiguration.enum({\n values: ['us-east-1', 'us-west-2', 'eu-west-1'],\n }),\n },\n});\n```\n\n#### INJECTED Projection\n\nFor custom partition values injected at query time:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'data',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'custom',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n partitionProjection: {\n custom: glue.PartitionProjectionConfiguration.injected(),\n },\n});\n```\n\n#### Multiple Partition Projections\n\nYou can configure partition projection for multiple partition keys:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'data',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [\n {\n name: 'year',\n type: glue.Schema.INTEGER,\n },\n {\n name: 'month',\n type: glue.Schema.INTEGER,\n },\n {\n name: 'region',\n type: glue.Schema.STRING,\n },\n ],\n dataFormat: glue.DataFormat.JSON,\n partitionProjection: {\n year: glue.PartitionProjectionConfiguration.integer({\n min: 2020,\n max: 2023,\n }),\n month: glue.PartitionProjectionConfiguration.integer({\n min: 1,\n max: 12,\n digits: 2,\n }),\n region: glue.PartitionProjectionConfiguration.enum({\n values: ['us-east-1', 'us-west-2'],\n }),\n },\n});\n```\n\n### Glue Connections\n\nGlue connections allow external data connections to third party databases and data warehouses. However, these connections can also be assigned to Glue Tables, allowing you to query external data sources using the Glue Data Catalog.\n\nWhereas `S3Table` will point to (and if needed, create) a bucket to store the tables' data, `ExternalTable` will point to an existing table in a data source. For example, to create a table in Glue that points to a table in Redshift:\n\n```ts\ndeclare const myConnection: glue.Connection;\ndeclare const myDatabase: glue.Database;\nnew glue.ExternalTable(this, 'MyTable', {\n connection: myConnection,\n externalDataLocation: 'default_db_public_example', // A table in Redshift\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n## [Encryption](https://docs.aws.amazon.com/athena/latest/ug/encryption.html)\n\nWhen the table creates its own S3 bucket (i.e. you do not pass an explicit `bucket`), that bucket enforces SSL: a bucket policy denies any request made over plain HTTP. If you provide your own bucket, enabling `enforceSSL` on it is your responsibility.\n\nYou can enable encryption on a Table's data:\n\n* [S3Managed](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html) - (default) Server side encryption (`SSE-S3`) with an Amazon S3-managed key.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n encryption: glue.TableEncryption.S3_MANAGED,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [Kms](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html) - Server-side encryption (`SSE-KMS`) with an AWS KMS Key managed by the account owner.\n\n```ts\ndeclare const myDatabase: glue.Database;\n// KMS key is created automatically\nnew glue.S3Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n\n// with an explicit KMS key\nnew glue.S3Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS,\n encryptionKey: new kms.Key(this, 'MyKey'),\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [KmsManaged](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html) - Server-side encryption (`SSE-KMS`), like `Kms`, except with an AWS KMS Key managed by the AWS Key Management Service.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS_MANAGED,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [ClientSideKms](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html#client-side-encryption-kms-managed-master-key-intro) - Client-side encryption (`CSE-KMS`) with an AWS KMS Key managed by the account owner.\n\n```ts\ndeclare const myDatabase: glue.Database;\n// KMS key is created automatically\nnew glue.S3Table(this, 'MyTable', {\n encryption: glue.TableEncryption.CLIENT_SIDE_KMS,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n\n// with an explicit KMS key\nnew glue.S3Table(this, 'MyTable', {\n encryption: glue.TableEncryption.CLIENT_SIDE_KMS,\n encryptionKey: new kms.Key(this, 'MyKey'),\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n*Note: you cannot provide a `Bucket` when creating the `S3Table` if you wish to use server-side encryption (`KMS`, `KMS_MANAGED` or `S3_MANAGED`)*.\n\n## Types\n\nA table's schema is a collection of columns, each of which have a `name` and a `type`. Types are recursive structures, consisting of primitive and complex types:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n columns: [{\n name: 'primitive_column',\n type: glue.Schema.STRING,\n }, {\n name: 'array_column',\n type: glue.Schema.array(glue.Schema.INTEGER),\n comment: 'array<integer>',\n }, {\n name: 'map_column',\n type: glue.Schema.map(\n glue.Schema.STRING,\n glue.Schema.TIMESTAMP),\n comment: 'map<string,string>',\n }, {\n name: 'struct_column',\n type: glue.Schema.struct([{\n name: 'nested_column',\n type: glue.Schema.DATE,\n comment: 'nested comment',\n }]),\n comment: \"struct<nested_column:date COMMENT 'nested comment'>\",\n }],\n // ...\n database: myDatabase,\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n## Public FAQ\n\n### What are we launching today?\n\nWe’re launching new features to an AWS CDK Glue L2 Construct to provide\nbest-practice defaults and convenience methods to create Glue Jobs, Connections,\nTriggers, Workflows, and the underlying permissions and configuration.\n\n### Why should I use this Construct?\n\nDevelopers should use this Construct to reduce the amount of boilerplate\ncode and complexity each individual has to navigate, and make it easier to\ncreate best-practice Glue resources.\n\n### What’s not in scope?\n\nGlue Crawlers and other resources that are now managed by the AWS LakeFormation\nteam are not in scope for this effort. Developers should use existing methods\nto create these resources, and the new Glue L2 construct assumes they already\nexist as inputs. While best practice is for application and infrastructure code\nto be as close as possible for teams using fully-implemented DevOps mechanisms,\nin practice these ETL scripts are likely managed by a data science team who\nknow Python or Scala and don’t necessarily own or manage their own\ninfrastructure deployments. We want to meet developers where they are, and not\nassume that all of the code resides in the same repository, Developers can\nautomate this themselves via the CDK, however, if they do own both.\n\nValidating Glue version and feature use per AWS region at synth time is also\nnot in scope. AWS’ intention is for all features to eventually be propagated to\nall Global regions, so the complexity involved in creating and updating region-\nspecific configuration to match shifting feature sets does not out-weigh the\nlikelihood that a developer will use this construct to deploy resources to a\nregion without a particular new feature to a region that doesn’t yet support\nit without researching or manually attempting to use that feature before\ndeveloping it via IaC. The developer will, of course, still get feedback from\nthe underlying Glue APIs as CloudFormation deploys the resources similar to the\ncurrent CDK L1 Glue experience.\n"
9412
+ "markdown": "# AWS Glue Construct Library\n<!--BEGIN STABILITY BANNER-->\n\n---\n\n![cdk-constructs: Experimental](https://img.shields.io/badge/cdk--constructs-experimental-important.svg?style=for-the-badge)\n\n> The APIs of higher level constructs in this module are experimental and under active development.\n> They are subject to non-backward compatible changes or removal in any future version. These are\n> not subject to the [Semantic Versioning](https://semver.org/) model and breaking changes will be\n> announced in the release notes. This means that while you may use them, you may need to update\n> your source code when upgrading to a newer version of this package.\n\n---\n\n<!--END STABILITY BANNER-->\n\nThis module is part of the [AWS Cloud Development Kit](https://github.com/aws/aws-cdk) project.\n\n## README\n\n[AWS Glue](https://aws.amazon.com/glue/) is a serverless data integration\nservice that makes it easier to discover, prepare, move, and integrate data\nfrom multiple sources for analytics, machine learning (ML), and application\ndevelopment.\n\nThe Glue L2 construct has convenience methods working backwards from common\nuse cases and sets required parameters to defaults that align with recommended\nbest practices for each job type. It also provides customers with a balance\nbetween flexibility via optional parameter overrides, and opinionated\ninterfaces that discouraging anti-patterns, resulting in reduced time to develop\nand deploy new resources.\n\n### References\n\n* [Glue Launch Announcement](https://aws.amazon.com/blogs/aws/launch-aws-glue-now-generally-available/)\n* [Glue Documentation](https://docs.aws.amazon.com/glue/index.html)\n* [Glue L1 (CloudFormation) Constructs](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/AWS_Glue.html)\n* Prior version of the [@aws-cdk/aws-glue-alpha module](https://github.com/aws/aws-cdk/blob/v2.51.1/packages/%40aws-cdk/aws-glue/README.md)\n\n## Create a Glue Job\n\nA Job encapsulates a script that connects to data sources, processes\nthem, and then writes output to a data target. There are four types of Glue\nJobs: Spark (ETL and Streaming), Python Shell, and Flex Jobs. Most\nof the required parameters for these jobs are common across all types,\nbut there are a few differences depending on the languages supported\nand features provided by each type. For all job types, the L2 defaults\nto AWS best practice recommendations, such as:\n\n* Use of Secrets Manager for Connection JDBC strings\n* Glue job autoscaling\n* Default parameter values for Glue job creation\n\nThis iteration of the L2 construct introduces breaking changes to\nthe existing glue-alpha-module, but these changes streamline the developer\nexperience, introduce new constants for defaults, and replacing synth-time\nvalidations with interface contracts for enforcement of the parameter combinations\nthat Glue supports. As an opinionated construct, the Glue L2 construct does\nnot allow developers to create resources that use non-current versions\nof Glue or deprecated language dependencies (e.g. deprecated versions of Python).\nAs always, L1s allow you to specify a wider range of parameters if you need\nor want to use alternative configurations.\n\nOptional and required parameters for each job are enforced via interface\nrather than validation; see [Glue's public documentation](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api.html)\nfor more granular details.\n\n### Spark Jobs\n\n#### ETL Jobs\n\nETL jobs support pySpark and Scala languages, for which there are separate but\nsimilar constructors. ETL jobs default to the G2 worker type, but you can\noverride this default with other supported worker type values (G1, G2, G4\nand G8). ETL jobs defaults to Glue version 4.0, which you can override to 3.0.\nThe following ETL features are enabled by default:\n`—enable-metrics, —enable-continuous-cloudwatch-log.`\nThe Spark UI (`—enable-spark-ui`) is off by default; enable it by setting the\n`sparkUI` prop.\nYou can find more details about version, worker type and other features in\n[Glue's public documentation](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-jobs-job.html).\n\nReference the pyspark-etl-jobs.test.ts and scalaspark-etl-jobs.test.ts unit tests\nfor examples of required-only and optional job parameters when creating these\ntypes of jobs.\n\nFor the sake of brevity, examples are shown using the pySpark job variety.\n\nExample with only required parameters:\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkEtlJob(stack, 'PySparkETLJob', {\n role,\n script,\n jobName: 'PySparkETLJob',\n});\n```\n\nExample with optional override parameters:\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkEtlJob(stack, 'PySparkETLJob', {\n jobName: 'PySparkETLJobCustomName',\n description: 'This is a description',\n role,\n script,\n glueVersion: glue.GlueVersion.V5_1,\n continuousLogging: { enabled: false },\n workerConfiguration: {\n workerType: glue.WorkerType.G_2X,\n numberOfWorkers: 2,\n },\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n maxRetries: 2,\n});\n```\n\n#### Streaming Jobs\n\nStreaming jobs are similar to ETL jobs, except that they perform ETL on data\nstreams using the Apache Spark Structured Streaming framework. Some Spark\njob features are not available to Streaming ETL jobs. They support Scala\nand pySpark languages. PySpark streaming jobs run on Python 3. It\ndefaults to the G2 worker type and Glue 4.0, both of which you can override.\nThe following best practice features are enabled by default:\n`—enable-metrics, —enable-continuous-cloudwatch-log`.\nThe Spark UI (`—enable-spark-ui`) is off by default; enable it by setting the\n`sparkUI` prop.\n\nReference the pyspark-streaming-jobs.test.ts and scalaspark-streaming-jobs.test.ts\nunit tests for examples of required-only and optional job parameters when creating\nthese types of jobs.\n\nExample with only required parameters:\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkStreamingJob(stack, 'ImportedJob', { role, script });\n```\n\nExample with optional override parameters:\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkStreamingJob(stack, 'PySparkStreamingJob', {\n jobName: 'PySparkStreamingJobCustomName',\n description: 'This is a description',\n role,\n script,\n glueVersion: glue.GlueVersion.V5_1,\n continuousLogging: { enabled: false },\n workerConfiguration: {\n workerType: glue.WorkerType.G_2X,\n numberOfWorkers: 2,\n },\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n maxRetries: 2,\n});\n```\n\n#### Flex Jobs\n\nThe flexible execution class is appropriate for non-urgent jobs such as\npre-production jobs, testing, and one-time data loads. Flexible jobs default\nto Glue version 5.0 and worker type `G_2X`. The following best practice\nfeatures are enabled by default:\n`—enable-metrics, —enable-continuous-cloudwatch-log`\nThe Spark UI (`—enable-spark-ui`) is off by default; enable it by setting the\n`sparkUI` prop.\n\nReference the pyspark-flex-etl-jobs.test.ts and scalaspark-flex-etl-jobs.test.ts\nunit tests for examples of required-only and optional job parameters when creating\nthese types of jobs.\n\nExample with only required parameters:\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkFlexEtlJob(stack, 'ImportedJob', { role, script });\n```\n\nExample with optional override parameters:\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkFlexEtlJob(stack, 'pySparkFlexEtlJob', {\n jobName: 'pySparkFlexEtlJob',\n description: 'This is a description',\n role,\n script,\n glueVersion: glue.GlueVersion.V5_1,\n continuousLogging: { enabled: false },\n workerConfiguration: {\n workerType: glue.WorkerType.G_2X,\n numberOfWorkers: 2,\n },\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n maxRetries: 2,\n});\n```\n\n### Python Shell Jobs\n\nPython shell jobs support a Python version that depends on the AWS Glue\nversion you use. These can be used to schedule and run tasks that don't\nrequire an Apache Spark environment. Python shell jobs default to\nPython 3.9 and a MaxCapacity of `0.0625`. Python 3.9 supports pre-loaded\nanalytics libraries using the `library-set=analytics` flag, which is\nenabled by default.\n\nReference the pyspark-shell-job.test.ts unit tests for examples of\nrequired-only and optional job parameters when creating these types of jobs.\n\nExample with only required parameters:\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PythonShellJob(stack, 'ImportedJob', { role, script });\n```\n\nExample with optional override parameters:\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\ndeclare const extraPythonFile: glue.Code;\nnew glue.PythonShellJob(stack, 'PythonShellJob', {\n jobName: 'PythonShellJobCustomName',\n description: 'This is a description',\n pythonVersion: glue.PythonVersion.THREE_NINE,\n maxCapacity: glue.MaxCapacity.DPU_1,\n role,\n script,\n extraPythonFiles: [extraPythonFile],\n glueVersion: glue.GlueVersion.V3_0,\n continuousLogging: { enabled: false },\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n maxRetries: 2,\n});\n```\n\n### Ray Jobs\n\n> **⚠️ DEPRECATED:** AWS Glue for Ray is closed to new customers as of April 30, 2026 and is in maintenance mode.\n> Migrate to [Amazon EKS with KubeRay Operator](https://docs.aws.amazon.com/glue/latest/dg/awsglue-ray-jobs-availability-change.html).\n\nThe `RayJob` construct, `Runtime.RAY_TWO_FOUR`, and `JobType.RAY` are deprecated and will be removed in a future release.\n\n### Metrics Control\n\nBy default, Glue jobs enable CloudWatch metrics (`--enable-metrics`) and observability metrics (`--enable-observability-metrics`) for monitoring and debugging. You can disable these metrics to reduce CloudWatch costs:\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\n\n// Disable both metrics for cost optimization\nnew glue.PySparkEtlJob(stack, 'CostOptimizedJob', {\n role,\n script,\n enableMetrics: false,\n enableObservabilityMetrics: false,\n});\n\n// Selective control - keep observability, disable profiling\nnew glue.PySparkEtlJob(stack, 'SelectiveJob', {\n role,\n script,\n enableMetrics: false,\n // enableObservabilityMetrics defaults to true\n});\n```\n\nThis feature is available for all Spark job types (ETL, Streaming, Flex).\n\n### Enable Job Run Queuing\n\nAWS Glue job queuing monitors your account level quotas and limits. If quotas or limits are insufficient to start a Glue job run, AWS Glue will automatically queue the job and wait for limits to free up. Once limits become available, AWS Glue will retry the job run. Glue jobs will queue for limits like max concurrent job runs per account, max concurrent Data Processing Units (DPU), and resource unavailable due to IP address exhaustion in Amazon Virtual Private Cloud (Amazon VPC).\n\nEnable job run queuing by setting the `jobRunQueuingEnabled` property to `true`.\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkEtlJob(stack, 'PySparkETLJob', {\n role,\n script,\n jobName: 'PySparkETLJob',\n jobRunQueuingEnabled: true\n});\n```\n\n### Uploading scripts from the CDK app repository to S3\n\nSimilar to other L2 constructs, the Glue L2 automates uploading local\nscripts to S3. Use `glue.Code.fromAsset(path)` to point at a script in your\nlocal file structure; it is uploaded to the CDK-managed asset bucket. To\nreference a script that already exists in S3, use\n`glue.Code.fromBucket(bucket, key)`, which performs no upload. A `script` is\nrequired for every job.\n\nReference the unit tests for examples of repo and S3 code target examples.\n\n### Workflow Triggers\n\nYou can use Glue workflows to create and visualize complex\nextract, transform, and load (ETL) activities involving multiple crawlers,\njobs, and triggers. Standalone triggers are an anti-pattern, so you must\ncreate triggers from within a workflow using the L2 construct.\n\nWithin a workflow object, there are functions to create different\ntypes of triggers with actions and predicates. You add triggers to the\nworkflow, and each trigger references the jobs or crawlers it runs as its\nactions.\n\n`startOnCreation` applies to scheduled triggers (and, via\n`ConditionalTriggerOptions`, conditional triggers) only. It defaults to `false`,\nbut you can override it if you prefer for your trigger to start on creation.\n\nReference the workflow-triggers.test.ts unit tests for examples of creating\nworkflows and triggers.\n\n```ts\nimport * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\n\n// Create a job to run from the workflow\nconst job = new glue.PySparkEtlJob(stack, 'Job', { role, script });\n\n// Create a workflow and add a trigger that runs the job\nconst workflow = new glue.Workflow(stack, 'Workflow');\nworkflow.addOnDemandTrigger('OnDemandTrigger', {\n actions: [{ job }],\n});\n```\n\n#### **1. On-Demand Triggers**\n\nOn-demand triggers can start glue jobs or crawlers. This construct provides\nconvenience functions to create on-demand crawler or job triggers. The constructor\ntakes an optional description parameter, but abstracts the requirement of an\nactions list using the job or crawler objects using conditional types.\n\n#### **2. Scheduled Triggers**\n\nYou can create scheduled triggers using cron expressions. This construct\nprovides daily and weekly convenience functions,\nas well as a custom function that allows you to create your own\ncustom timing using the [existing event Schedule class](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_events.Schedule.html)\nwithout having to build your own cron expressions. The L2 extracts\nthe expression that Glue requires from the Schedule object. The constructor\ntakes an optional description and a list of jobs or crawlers as actions.\n\n#### **3. Notify Event Triggers**\n\nThere are two types of notify event triggers: batching and non-batching.\nFor batching triggers, you must specify `BatchSize`. For non-batching\ntriggers, `BatchSize` defaults to 1. For both triggers, `BatchWindow`\ndefaults to 900 seconds, but you can override the window to align with\nyour workload's requirements.\n\n#### **4. Conditional Triggers**\n\nConditional triggers have a predicate and actions associated with them.\nThe trigger actions are executed when the predicateCondition is true.\n\n### Connection Properties\n\nA `Connection` allows Glue jobs, crawlers and development endpoints to access\ncertain types of data stores.\n\n* **Secrets Management**\n You must specify JDBC connection credentials in Secrets Manager and\n provide the Secrets Manager Key name as a property to the job connection.\n\n* **Networking - the CDK determines the best fit subnet for Glue connection\nconfiguration**\n The prior version of the glue-alpha-module requires the developer to\n specify the subnet of the Connection when it’s defined. Now, you can still\n specify the specific subnet you want to use, but are no longer required\n to. You are only required to provide a VPC and either a public or private\n subnet selection. Without a specific subnet provided, the L2 leverages the\n existing [EC2 Subnet Selection](https://docs.aws.amazon.com/cdk/api/v2/python/aws_cdk.aws_ec2/SubnetSelection.html)\n library to make the best choice selection for the subnet.\n\n```ts\ndeclare const securityGroup: ec2.SecurityGroup;\ndeclare const subnet: ec2.Subnet;\nnew glue.Connection(this, 'MyConnection', {\n type: glue.ConnectionType.NETWORK,\n // The security groups granting AWS Glue inbound access to the data source within the VPC\n securityGroups: [securityGroup],\n // The VPC subnet which contains the data source\n subnet,\n});\n```\n\nFor RDS `Connection` by JDBC, it is recommended to manage credentials using AWS Secrets Manager. To use Secret, specify `SECRET_ID` in `properties` like the following code. Note that in this case, the subnet must have a route to the AWS Secrets Manager VPC endpoint or to the AWS Secrets Manager endpoint through a NAT gateway.\n\n```ts\ndeclare const securityGroup: ec2.SecurityGroup;\ndeclare const subnet: ec2.Subnet;\ndeclare const db: rds.DatabaseCluster;\nnew glue.Connection(this, \"RdsConnection\", {\n type: glue.ConnectionType.JDBC,\n securityGroups: [securityGroup],\n subnet,\n properties: {\n JDBC_CONNECTION_URL: `jdbc:mysql://${db.clusterEndpoint.socketAddress}/databasename`,\n JDBC_ENFORCE_SSL: \"false\",\n SECRET_ID: db.secret!.secretName,\n },\n});\n```\n\nConnection `properties` are emitted verbatim into the CloudFormation template, so\nany credential placed there in plaintext is stored in plaintext in the template,\n`cdk.out`, and source control. Reference a Secrets Manager secret through\n`SECRET_ID` (as above) instead. If a property key looks like a credential (for\nexample `PASSWORD`, `SECRET`, or `TOKEN`) and holds a plaintext literal, the\nconstruct emits a synthesis-time warning.\n\nIf you need to use a connection type that doesn't exist as a static member on `ConnectionType`, you can instantiate a `ConnectionType` object, e.g: `new glue.ConnectionType('NEW_TYPE')`.\n\nSee [Adding a Connection to Your Data Store](https://docs.aws.amazon.com/glue/latest/dg/populate-add-connection.html) and [Connection Structure](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-api-catalog-connections.html#aws-glue-api-catalog-connections-Connection) documentation for more information on the supported data stores and their configurations.\n\n## SecurityConfiguration\n\nA `SecurityConfiguration` is a set of security properties that can be used by AWS Glue to encrypt data at rest.\n\nEach encryption config is built with a factory that pairs the encryption mode\nwith its key, so illegal combinations (such as an S3-managed encryption carrying\na KMS key) cannot be expressed:\n\n```ts\nnew glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: glue.CloudWatchEncryption.kms(),\n jobBookmarksEncryption: glue.JobBookmarksEncryption.clientSideKms(),\n s3Encryption: glue.S3Encryption.kms(),\n});\n```\n\nBy default, a shared KMS key is created for use with the encryption configurations that require one. You can also supply your own key to any factory, for example, for CloudWatch encryption:\n\n```ts\ndeclare const key: kms.Key;\nnew glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: glue.CloudWatchEncryption.kms(key),\n});\n```\n\nUse `glue.S3Encryption.s3Managed()` for S3-managed (SSE-S3) encryption, which takes no key.\n\nSee [documentation](https://docs.aws.amazon.com/glue/latest/dg/encryption-security-configuration.html) for more info for Glue encrypting data written by Crawlers, Jobs, and Development Endpoints.\n\n## Catalog\n\nThe Glue Data Catalog is a persistent metadata store for your data assets. Every\naccount has an implicit, account-wide catalog that always exists, and you can also\ncreate additional catalogs as `AWS::Glue::Catalog` resources (for example, to\nfederate to another metastore).\n\nA catalog's encryption is fixed when the catalog is created: a catalog either\ncarries encryption settings or it does not. This keeps its configuration easy to\nreason about — there are no mutation methods that change encryption after the fact.\n\n### The account-wide catalog\n\nUse `Catalog.forAccount(scope)` to obtain the implicit account catalog. It is not\na CloudFormation resource — it always exists. Repeated calls within the same stack\nreturn the same instance:\n\n```ts\nconst catalog = glue.Catalog.forAccount(this);\n```\n\nTo configure Data Catalog encryption for the account, use\n`Catalog.encryptAccount(scope, options)`:\n\n```ts\ndeclare const key: kms.Key;\nglue.Catalog.encryptAccount(this, {\n encryptionAtRest: glue.DataCatalogEncryptionAtRest.kms(key),\n});\n```\n\nBecause encryption is fixed at construction, `encryptAccount` must be called\n*before* the account catalog is first used in the stack — before any\n`Catalog.forAccount(this)` call, and before any `Database` that uses the account\ncatalog. Calling it after the account catalog has been materialized throws.\n\nThe account catalog's encryption is an account- and region-wide setting, managed\nthrough the singleton `PutDataCatalogEncryptionSettings` API. Configure it in\nexactly one stack. Configuring it from multiple stacks in the same account and\nregion makes those stacks overwrite one another at deploy time, and the result is\norder-dependent. Unlike duplicate settings within a single stack (which\nCloudFormation rejects), this cross-stack conflict is not caught at synthesis\ntime, because each stack synthesizes to its own template.\n\n### Creating a catalog\n\nTo create a new catalog resource, use the `Catalog` constructor. Encryption is\nconfigured through the `encryptionAtRest` and `connectionPasswordEncryption` props:\n\n```ts\nnew glue.Catalog(this, 'MyCatalog', {\n catalogName: 'my-catalog',\n description: 'my catalog description',\n});\n```\n\n### Encryption at rest\n\nConfigure Data Catalog encryption at rest through the `encryptionAtRest` option\n(on `Catalog.encryptAccount` or the `Catalog` constructor).\nIt accepts a `DataCatalogEncryptionAtRest` describing the mode:\n\n```ts\ndeclare const key: kms.Key;\n\n// SSE-KMS with a customer-managed key\nglue.Catalog.encryptAccount(this, {\n encryptionAtRest: glue.DataCatalogEncryptionAtRest.kms(key),\n});\n\n// SSE-KMS with an AWS-managed key (omit the key)\nnew glue.Catalog(this, 'ManagedKeyCatalog', {\n catalogName: 'managed-key-catalog',\n encryptionAtRest: glue.DataCatalogEncryptionAtRest.kms(),\n});\n\n// Disable encryption at rest\nnew glue.Catalog(this, 'PlaintextCatalog', {\n catalogName: 'plaintext-catalog',\n encryptionAtRest: glue.DataCatalogEncryptionAtRest.disabled(),\n});\n```\n\nWhen you use `SSE-KMS-WITH-SERVICE-ROLE`, AWS Glue accesses the KMS key through a\nservice role you provide. If you pass a customer-managed key, the role is\nautomatically granted the permissions it needs to encrypt and decrypt catalog data:\n\n```ts\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const key: kms.Key;\ndeclare const role: iam.IRole;\nglue.Catalog.encryptAccount(this, {\n encryptionAtRest: glue.DataCatalogEncryptionAtRest.kmsWithServiceRole(role, key),\n});\n```\n\nThe customer-managed key, when configured, is exposed on the catalog as\n`encryptionKey` (and the connection-password key as `connectionPasswordKey`), so\nyou can reference it to grant additional access. It is undefined when encryption is\ndisabled or an AWS-managed key is used.\n\n### Connection password encryption\n\nIndependently from encryption at rest, the Data Catalog can encrypt the passwords\nstored in connection properties. Configure it through the\n`connectionPasswordEncryption` option:\n\n```ts\ndeclare const key: kms.Key;\nglue.Catalog.encryptAccount(this, {\n connectionPasswordEncryption: {\n kmsKey: key,\n // Whether GetConnection/GetConnections return the password encrypted (default: true)\n returnConnectionPasswordEncrypted: true,\n },\n});\n```\n\nThe two encryption blocks are independent: enabling one does not require the other,\nand each may use a different KMS key. The customer-managed key for connection\npasswords is exposed as `connectionPasswordKey`.\n\n### Importing a catalog\n\nYou can import an existing catalog by ARN or by id. An imported catalog is a pure\nidentity handle — it emits no resources and does not manage the catalog's\nencryption:\n\n```ts\nconst byId = glue.Catalog.fromCatalogId(this, 'ById', 'my-catalog-id');\nconst byArn = glue.Catalog.fromCatalogArn(this, 'ByArn', 'arn:aws:glue:us-east-1:123456789012:catalog/my-catalog-id');\n```\n\nTo manage the Data Catalog encryption of a catalog you did not create in this\nstack, add a `CfnDataCatalogEncryptionSettings` resource targeting its id\ndirectly. Do this from exactly one stack: like the account catalog, a catalog has\na single encryption configuration, so two settings resources targeting the same id\nrace to overwrite one another at deploy time. Within a single stack this is caught\nby CloudFormation template validation (E3019, duplicate primary identifiers);\nacross stacks it is not, since each stack synthesizes to its own template.\n\n```ts\nimport { CfnDataCatalogEncryptionSettings } from 'aws-cdk-lib/aws-glue';\n\nnew CfnDataCatalogEncryptionSettings(this, 'Encryption', {\n catalogId: 'my-catalog-id',\n dataCatalogEncryptionSettings: {\n encryptionAtRest: { catalogEncryptionMode: 'SSE-KMS' },\n },\n});\n```\n\n## Database\n\nA `Database` is a logical grouping of `Tables` in the Glue Catalog.\n\n```ts\nnew glue.Database(this, 'MyDatabase', {\n databaseName: 'my_database',\n description: 'my_database_description',\n});\n```\n\nBecause a database is a container for tables and their metadata, it is retained\nby default when removed from the stack, to avoid accidental data loss. Set\n`removalPolicy` to `RemovalPolicy.DESTROY` to have it deleted instead:\n\n```ts\nimport { RemovalPolicy } from 'aws-cdk-lib';\n\nnew glue.Database(this, 'MyDatabase', {\n databaseName: 'my_database',\n removalPolicy: RemovalPolicy.DESTROY,\n});\n```\n\n## Table\n\nA Glue table describes a table of data in S3: its structure (column names and types), location of data (S3 objects with a common prefix in a S3 bucket), and format for the files (Json, Avro, Parquet, etc.):\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }, {\n name: 'col2',\n type: glue.Schema.array(glue.Schema.STRING),\n comment: 'col2 is an array of strings' // comment is optional\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nBy default, a S3 bucket will be created to store the table's data but you can manually pass the `bucket` and `s3Prefix`:\n\n```ts\ndeclare const myBucket: s3.Bucket;\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n bucket: myBucket,\n s3Prefix: 'my-table/',\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nGlue tables can be configured to contain user-defined properties, to describe the physical storage of table data, through the `storageParameters` property:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n storageParameters: [\n glue.StorageParameter.skipHeaderLineCount(1),\n glue.StorageParameter.compressionType(glue.CompressionType.GZIP),\n glue.StorageParameter.custom('separatorChar', ',')\n ],\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nGlue tables can also be configured to contain user-defined table properties through the [`parameters`](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-glue-table-tableinput.html#cfn-glue-table-tableinput-parameters) property:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n parameters: {\n key1: 'val1',\n key2: 'val2',\n },\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n### Partition Keys\n\nTo improve query performance, a table can specify `partitionKeys` on which data is stored and queried separately. For example, you might partition a table by `year` and `month` to optimize queries based on a time window:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n### Partition Indexes\n\nAnother way to improve query performance is to specify partition indexes. If no partition indexes are\npresent on the table, AWS Glue loads all partitions of the table and filters the loaded partitions using\nthe query expression. The query takes more time to run as the number of partitions increase. With an\nindex, the query will try to fetch a subset of the partitions instead of loading all partitions of the\ntable.\n\nThe keys of a partition index must be a subset of the partition keys of the table. You can have a\nmaximum of 3 partition indexes per table. To specify a partition index, you can use the `partitionIndexes`\nproperty:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n partitionIndexes: [{\n indexName: 'my-index', // optional\n keyNames: ['year'],\n }], // supply up to 3 indexes\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nAlternatively, you can call the `addPartitionIndex()` function on a table:\n\n```ts\ndeclare const myTable: glue.S3Table;\nmyTable.addPartitionIndex({\n indexName: 'my-index',\n keyNames: ['year'],\n});\n```\n\n### Partition Filtering\n\nIf you have a table with a large number of partitions that grows over time, consider using AWS Glue partition indexing and filtering.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n dataFormat: glue.DataFormat.JSON,\n enablePartitionFiltering: true,\n});\n```\n\n### Partition Projection\n\nPartition projection allows Athena to automatically add new partitions as new data arrives, without requiring `ALTER TABLE ADD PARTITION` statements. This improves query performance and reduces management overhead by eliminating the need to manually manage partition metadata.\n\nFor more information, see the [AWS documentation on partition projection](https://docs.aws.amazon.com/athena/latest/ug/partition-projection.html).\n\n#### INTEGER Projection\n\nFor partition keys with sequential numeric values:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'data',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.INTEGER,\n }],\n dataFormat: glue.DataFormat.JSON,\n partitionProjection: {\n year: glue.PartitionProjectionConfiguration.integer({\n min: 2020,\n max: 2023,\n interval: 1, // optional, defaults to 1\n digits: 4, // optional, pads with leading zeros\n }),\n },\n});\n```\n\n#### DATE Projection\n\nFor partition keys with date or timestamp values. Supports both fixed dates and relative dates using `NOW`:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'data',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'date',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n partitionProjection: {\n date: glue.PartitionProjectionConfiguration.date({\n min: '2020-01-01',\n max: '2023-12-31',\n format: 'yyyy-MM-dd',\n interval: 1, // optional, defaults to 1\n intervalUnit: glue.DateIntervalUnit.DAYS, // optional: YEARS, MONTHS, WEEKS, DAYS, HOURS, MINUTES, SECONDS\n }),\n },\n});\n```\n\nYou can also use relative dates with `NOW`:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'data',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'date',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n partitionProjection: {\n date: glue.PartitionProjectionConfiguration.date({\n min: 'NOW-3YEARS',\n max: 'NOW',\n format: 'yyyy-MM-dd',\n }),\n },\n});\n```\n\n#### ENUM Projection\n\nFor partition keys with a known set of values:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'data',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'region',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n partitionProjection: {\n region: glue.PartitionProjectionConfiguration.enum({\n values: ['us-east-1', 'us-west-2', 'eu-west-1'],\n }),\n },\n});\n```\n\n#### INJECTED Projection\n\nFor custom partition values injected at query time:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'data',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'custom',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n partitionProjection: {\n custom: glue.PartitionProjectionConfiguration.injected(),\n },\n});\n```\n\n#### Multiple Partition Projections\n\nYou can configure partition projection for multiple partition keys:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'data',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [\n {\n name: 'year',\n type: glue.Schema.INTEGER,\n },\n {\n name: 'month',\n type: glue.Schema.INTEGER,\n },\n {\n name: 'region',\n type: glue.Schema.STRING,\n },\n ],\n dataFormat: glue.DataFormat.JSON,\n partitionProjection: {\n year: glue.PartitionProjectionConfiguration.integer({\n min: 2020,\n max: 2023,\n }),\n month: glue.PartitionProjectionConfiguration.integer({\n min: 1,\n max: 12,\n digits: 2,\n }),\n region: glue.PartitionProjectionConfiguration.enum({\n values: ['us-east-1', 'us-west-2'],\n }),\n },\n});\n```\n\n### Glue Connections\n\nGlue connections allow external data connections to third party databases and data warehouses. However, these connections can also be assigned to Glue Tables, allowing you to query external data sources using the Glue Data Catalog.\n\nWhereas `S3Table` will point to (and if needed, create) a bucket to store the tables' data, `ExternalTable` will point to an existing table in a data source. For example, to create a table in Glue that points to a table in Redshift:\n\n```ts\ndeclare const myConnection: glue.Connection;\ndeclare const myDatabase: glue.Database;\nnew glue.ExternalTable(this, 'MyTable', {\n connection: myConnection,\n externalDataLocation: 'default_db_public_example', // A table in Redshift\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n## Data Quality Ruleset\n\nA `DataQualityRuleset` defines a set of data quality rules — authored in Glue's\nData Quality Definition Language (DQDL) — that are evaluated against a table in\nthe Data Catalog.\n\n```ts\nnew glue.DataQualityRuleset(this, 'MyRuleset', {\n rulesetName: 'my_ruleset',\n dqdl: glue.Dqdl.fromString('Rules = [ RowCount > 100, IsComplete \"order_id\" ]'),\n targetTable: new glue.DataQualityTargetTable('my_database', 'my_table'),\n});\n```\n\nBuild the DQDL document with `Dqdl.fromString(...)`. Glue parses and validates the\nDQDL when the ruleset is deployed; see the\n[DQDL reference](https://docs.aws.amazon.com/glue/latest/dg/dqdl.html) for the\nfull rule syntax.\n\n## [Encryption](https://docs.aws.amazon.com/athena/latest/ug/encryption.html)\n\nWhen the table creates its own S3 bucket (i.e. you do not pass an explicit `bucket`), that bucket enforces SSL: a bucket policy denies any request made over plain HTTP. If you provide your own bucket, enabling `enforceSSL` on it is your responsibility.\n\nYou can enable encryption on a Table's data:\n\n* [S3Managed](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html) - (default) Server side encryption (`SSE-S3`) with an Amazon S3-managed key.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n encryption: glue.TableEncryption.S3_MANAGED,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [Kms](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html) - Server-side encryption (`SSE-KMS`) with an AWS KMS Key managed by the account owner.\n\n```ts\ndeclare const myDatabase: glue.Database;\n// KMS key is created automatically\nnew glue.S3Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n\n// with an explicit KMS key\nnew glue.S3Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS,\n encryptionKey: new kms.Key(this, 'MyKey'),\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [KmsManaged](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html) - Server-side encryption (`SSE-KMS`), like `Kms`, except with an AWS KMS Key managed by the AWS Key Management Service.\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n encryption: glue.TableEncryption.KMS_MANAGED,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n* [ClientSideKms](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html#client-side-encryption-kms-managed-master-key-intro) - Client-side encryption (`CSE-KMS`) with an AWS KMS Key managed by the account owner.\n\n```ts\ndeclare const myDatabase: glue.Database;\n// KMS key is created automatically\nnew glue.S3Table(this, 'MyTable', {\n encryption: glue.TableEncryption.CLIENT_SIDE_KMS,\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n\n// with an explicit KMS key\nnew glue.S3Table(this, 'MyTable', {\n encryption: glue.TableEncryption.CLIENT_SIDE_KMS,\n encryptionKey: new kms.Key(this, 'MyKey'),\n // ...\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n*Note: you cannot provide a `Bucket` when creating the `S3Table` if you wish to use server-side encryption (`KMS`, `KMS_MANAGED` or `S3_MANAGED`)*.\n\n### Marking table data as encrypted\n\nBoth `S3Table` and `ExternalTable` set the `has_encrypted_data` table parameter, which\nAthena reads when querying client-side (`CSE-KMS`) encrypted datasets. It defaults to `true`.\nSet `hasEncryptedData` to `false` when the underlying data is not encrypted:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n hasEncryptedData: false,\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\nDo not set `has_encrypted_data` through the free-form `parameters` map as well - a value\nthere that conflicts with `hasEncryptedData` is rejected at synthesis time.\n\n## Types\n\nA table's schema is a collection of columns, each of which have a `name` and a `type`. Types are recursive structures, consisting of primitive and complex types:\n\n```ts\ndeclare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n columns: [{\n name: 'primitive_column',\n type: glue.Schema.STRING,\n }, {\n name: 'array_column',\n type: glue.Schema.array(glue.Schema.INTEGER),\n comment: 'array<integer>',\n }, {\n name: 'map_column',\n type: glue.Schema.map(\n glue.Schema.STRING,\n glue.Schema.TIMESTAMP),\n comment: 'map<string,timestamp>',\n }, {\n name: 'struct_column',\n type: glue.Schema.struct([{\n name: 'nested_column',\n type: glue.Schema.DATE,\n comment: 'nested comment',\n }]),\n comment: \"struct<nested_column:date COMMENT 'nested comment'>\",\n }],\n // ...\n database: myDatabase,\n dataFormat: glue.DataFormat.JSON,\n});\n```\n\n## Public FAQ\n\n### What are we launching today?\n\nWe’re launching new features to an AWS CDK Glue L2 Construct to provide\nbest-practice defaults and convenience methods to create Glue Jobs, Connections,\nTriggers, Workflows, and the underlying permissions and configuration.\n\n### Why should I use this Construct?\n\nDevelopers should use this Construct to reduce the amount of boilerplate\ncode and complexity each individual has to navigate, and make it easier to\ncreate best-practice Glue resources.\n\n### What’s not in scope?\n\nGlue Crawlers and other resources that are now managed by the AWS LakeFormation\nteam are not in scope for this effort. Developers should use existing methods\nto create these resources, and the new Glue L2 construct assumes they already\nexist as inputs. While best practice is for application and infrastructure code\nto be as close as possible for teams using fully-implemented DevOps mechanisms,\nin practice these ETL scripts are likely managed by a data science team who\nknow Python or Scala and don’t necessarily own or manage their own\ninfrastructure deployments. We want to meet developers where they are, and not\nassume that all of the code resides in the same repository, Developers can\nautomate this themselves via the CDK, however, if they do own both.\n\nValidating Glue version and feature use per AWS region at synth time is also\nnot in scope. AWS’ intention is for all features to eventually be propagated to\nall Global regions, so the complexity involved in creating and updating region-\nspecific configuration to match shifting feature sets does not out-weigh the\nlikelihood that a developer will use this construct to deploy resources to a\nregion without a particular new feature to a region that doesn’t yet support\nit without researching or manually attempting to use that feature before\ndeveloping it via IaC. The developer will, of course, still get feedback from\nthe underlying Glue APIs as CloudFormation deploys the resources similar to the\ncurrent CDK L1 Glue experience.\n"
9413
9413
  },
9414
9414
  "repository": {
9415
9415
  "directory": "packages/@aws-cdk/aws-glue-alpha",
@@ -10358,97 +10358,63 @@
10358
10358
  },
10359
10359
  "@aws-cdk/aws-glue-alpha.CloudWatchEncryption": {
10360
10360
  "assembly": "@aws-cdk/aws-glue-alpha",
10361
- "datatype": true,
10362
10361
  "docs": {
10362
+ "remarks": "CloudWatch Logs support only server-side encryption with a KMS key.",
10363
10363
  "stability": "experimental",
10364
- "summary": "CloudWatch Logs encryption configuration.",
10365
- "example": "new glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n },\n jobBookmarksEncryption: {\n mode: glue.JobBookmarksEncryptionMode.CLIENT_SIDE_KMS,\n },\n s3Encryption: {\n mode: glue.S3EncryptionMode.KMS,\n },\n});",
10364
+ "summary": "CloudWatch Logs encryption configuration for a `SecurityConfiguration`.",
10365
+ "example": "new glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: glue.CloudWatchEncryption.kms(),\n jobBookmarksEncryption: glue.JobBookmarksEncryption.clientSideKms(),\n s3Encryption: glue.S3Encryption.kms(),\n});",
10366
10366
  "custom": {
10367
10367
  "exampleMetadata": "infused"
10368
10368
  }
10369
10369
  },
10370
10370
  "fqn": "@aws-cdk/aws-glue-alpha.CloudWatchEncryption",
10371
- "kind": "interface",
10371
+ "kind": "class",
10372
10372
  "locationInModule": {
10373
10373
  "filename": "lib/security-configuration.ts",
10374
- "line": 86
10374
+ "line": 89
10375
10375
  },
10376
- "name": "CloudWatchEncryption",
10377
- "properties": [
10376
+ "methods": [
10378
10377
  {
10379
- "abstract": true,
10380
10378
  "docs": {
10381
10379
  "stability": "experimental",
10382
- "summary": "Encryption mode."
10380
+ "summary": "Server-side encryption (SSE) with an AWS KMS key managed by the account owner."
10383
10381
  },
10384
- "immutable": true,
10385
10382
  "locationInModule": {
10386
10383
  "filename": "lib/security-configuration.ts",
10387
- "line": 90
10388
- },
10389
- "name": "mode",
10390
- "type": {
10391
- "fqn": "@aws-cdk/aws-glue-alpha.CloudWatchEncryptionMode"
10392
- }
10393
- },
10394
- {
10395
- "abstract": true,
10396
- "docs": {
10397
- "default": "A key will be created if one is not provided.",
10398
- "stability": "experimental",
10399
- "summary": "The KMS key to be used to encrypt the data."
10384
+ "line": 95
10400
10385
  },
10401
- "immutable": true,
10402
- "locationInModule": {
10403
- "filename": "lib/security-configuration.ts",
10404
- "line": 96
10386
+ "name": "kms",
10387
+ "parameters": [
10388
+ {
10389
+ "docs": {
10390
+ "remarks": "A key is created if one is not provided.",
10391
+ "summary": "the KMS key used to encrypt the data."
10392
+ },
10393
+ "name": "kmsKey",
10394
+ "optional": true,
10395
+ "type": {
10396
+ "fqn": "aws-cdk-lib.interfaces.aws_kms.IKeyRef"
10397
+ }
10398
+ }
10399
+ ],
10400
+ "returns": {
10401
+ "type": {
10402
+ "fqn": "@aws-cdk/aws-glue-alpha.CloudWatchEncryption"
10403
+ }
10405
10404
  },
10406
- "name": "kmsKey",
10407
- "optional": true,
10408
- "type": {
10409
- "fqn": "aws-cdk-lib.interfaces.aws_kms.IKeyRef"
10410
- }
10405
+ "static": true
10411
10406
  }
10412
10407
  ],
10408
+ "name": "CloudWatchEncryption",
10413
10409
  "symbolId": "lib/security-configuration:CloudWatchEncryption"
10414
10410
  },
10415
- "@aws-cdk/aws-glue-alpha.CloudWatchEncryptionMode": {
10416
- "assembly": "@aws-cdk/aws-glue-alpha",
10417
- "docs": {
10418
- "see": "https://docs.aws.amazon.com/glue/latest/webapi/API_CloudWatchEncryption.html#Glue-Type-CloudWatchEncryption-CloudWatchEncryptionMode",
10419
- "stability": "experimental",
10420
- "summary": "Encryption mode for CloudWatch Logs.",
10421
- "example": "new glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n },\n jobBookmarksEncryption: {\n mode: glue.JobBookmarksEncryptionMode.CLIENT_SIDE_KMS,\n },\n s3Encryption: {\n mode: glue.S3EncryptionMode.KMS,\n },\n});",
10422
- "custom": {
10423
- "exampleMetadata": "infused"
10424
- }
10425
- },
10426
- "fqn": "@aws-cdk/aws-glue-alpha.CloudWatchEncryptionMode",
10427
- "kind": "enum",
10428
- "locationInModule": {
10429
- "filename": "lib/security-configuration.ts",
10430
- "line": 45
10431
- },
10432
- "members": [
10433
- {
10434
- "docs": {
10435
- "see": "https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html",
10436
- "stability": "experimental",
10437
- "summary": "Server-side encryption (SSE) with an AWS KMS key managed by the account owner."
10438
- },
10439
- "name": "KMS"
10440
- }
10441
- ],
10442
- "name": "CloudWatchEncryptionMode",
10443
- "symbolId": "lib/security-configuration:CloudWatchEncryptionMode"
10444
- },
10445
10411
  "@aws-cdk/aws-glue-alpha.Code": {
10446
10412
  "abstract": true,
10447
10413
  "assembly": "@aws-cdk/aws-glue-alpha",
10448
10414
  "docs": {
10449
10415
  "stability": "experimental",
10450
10416
  "summary": "Represents a Glue Job's Code assets (an asset can be a scripts, a jar, a python file or any other file).",
10451
- "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\n\n// Disable both metrics for cost optimization\nnew glue.PySparkEtlJob(stack, 'CostOptimizedJob', {\n role,\n script,\n enableMetrics: false,\n enableObservabilityMetrics: false,\n});\n\n// Selective control - keep observability, disable profiling\nnew glue.PySparkEtlJob(stack, 'SelectiveJob', {\n role,\n script,\n enableMetrics: false,\n // enableObservabilityMetrics defaults to true\n});",
10417
+ "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\n\n// Create a job to run from the workflow\nconst job = new glue.PySparkEtlJob(stack, 'Job', { role, script });\n\n// Create a workflow and add a trigger that runs the job\nconst workflow = new glue.Workflow(stack, 'Workflow');\nworkflow.addOnDemandTrigger('OnDemandTrigger', {\n actions: [{ job }],\n});",
10452
10418
  "custom": {
10453
10419
  "exampleMetadata": "infused"
10454
10420
  }
@@ -10962,7 +10928,7 @@
10962
10928
  "docs": {
10963
10929
  "stability": "experimental",
10964
10930
  "summary": "An AWS Glue connection to a data source.",
10965
- "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkEtlJob(stack, 'PySparkETLJob', {\n jobName: 'PySparkETLJobCustomName',\n description: 'This is a description',\n role,\n script,\n glueVersion: glue.GlueVersion.V5_1,\n continuousLogging: { enabled: false },\n workerType: glue.WorkerType.G_2X,\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n numberOfWorkers: 2,\n maxRetries: 2,\n});",
10931
+ "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkEtlJob(stack, 'PySparkETLJob', {\n jobName: 'PySparkETLJobCustomName',\n description: 'This is a description',\n role,\n script,\n glueVersion: glue.GlueVersion.V5_1,\n continuousLogging: { enabled: false },\n workerConfiguration: {\n workerType: glue.WorkerType.G_2X,\n numberOfWorkers: 2,\n },\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n maxRetries: 2,\n});",
10966
10932
  "custom": {
10967
10933
  "exampleMetadata": "infused"
10968
10934
  }
@@ -12194,7 +12160,7 @@
12194
12160
  "see": "https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html",
12195
12161
  "stability": "experimental",
12196
12162
  "summary": "Properties for enabling Continuous Logging for Glue Jobs.",
12197
- "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkEtlJob(stack, 'PySparkETLJob', {\n jobName: 'PySparkETLJobCustomName',\n description: 'This is a description',\n role,\n script,\n glueVersion: glue.GlueVersion.V5_1,\n continuousLogging: { enabled: false },\n workerType: glue.WorkerType.G_2X,\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n numberOfWorkers: 2,\n maxRetries: 2,\n});",
12163
+ "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkEtlJob(stack, 'PySparkETLJob', {\n jobName: 'PySparkETLJobCustomName',\n description: 'This is a description',\n role,\n script,\n glueVersion: glue.GlueVersion.V5_1,\n continuousLogging: { enabled: false },\n workerConfiguration: {\n workerType: glue.WorkerType.G_2X,\n numberOfWorkers: 2,\n },\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n maxRetries: 2,\n});",
12198
12164
  "custom": {
12199
12165
  "exampleMetadata": "infused"
12200
12166
  }
@@ -12606,7 +12572,7 @@
12606
12572
  "docs": {
12607
12573
  "stability": "experimental",
12608
12574
  "summary": "Defines the input/output formats and ser/de for a single DataFormat.",
12609
- "example": "declare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'data',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'date',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n partitionProjection: {\n date: glue.PartitionProjectionConfiguration.date({\n min: '2020-01-01',\n max: '2023-12-31',\n format: 'yyyy-MM-dd',\n interval: 1, // optional, defaults to 1\n intervalUnit: glue.DateIntervalUnit.DAYS, // optional: YEARS, MONTHS, WEEKS, DAYS, HOURS, MINUTES, SECONDS\n }),\n },\n});",
12575
+ "example": "declare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'data',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'date',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n partitionProjection: {\n date: glue.PartitionProjectionConfiguration.date({\n min: 'NOW-3YEARS',\n max: 'NOW',\n format: 'yyyy-MM-dd',\n }),\n },\n});",
12610
12576
  "custom": {
12611
12577
  "exampleMetadata": "infused"
12612
12578
  }
@@ -12958,9 +12924,9 @@
12958
12924
  "docs": {
12959
12925
  "stability": "experimental",
12960
12926
  "summary": "A Glue Data Quality ruleset.",
12961
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\n\ndeclare const dataQualityTargetTable: glue_alpha.DataQualityTargetTable;\nconst dataQualityRuleset = new glue_alpha.DataQualityRuleset(this, 'MyDataQualityRuleset', {\n rulesetDqdl: 'rulesetDqdl',\n targetTable: dataQualityTargetTable,\n\n // the properties below are optional\n clientToken: 'clientToken',\n description: 'description',\n removalPolicy: cdk.RemovalPolicy.DESTROY,\n rulesetName: 'rulesetName',\n tags: {\n tagsKey: 'tags',\n },\n});",
12927
+ "example": "new glue.DataQualityRuleset(this, 'MyRuleset', {\n rulesetName: 'my_ruleset',\n dqdl: glue.Dqdl.fromString('Rules = [ RowCount > 100, IsComplete \"order_id\" ]'),\n targetTable: new glue.DataQualityTargetTable('my_database', 'my_table'),\n});",
12962
12928
  "custom": {
12963
- "exampleMetadata": "fixture=_generated"
12929
+ "exampleMetadata": "infused"
12964
12930
  }
12965
12931
  },
12966
12932
  "fqn": "@aws-cdk/aws-glue-alpha.DataQualityRuleset",
@@ -12970,7 +12936,7 @@
12970
12936
  },
12971
12937
  "locationInModule": {
12972
12938
  "filename": "lib/data-quality-ruleset.ts",
12973
- "line": 128
12939
+ "line": 159
12974
12940
  },
12975
12941
  "parameters": [
12976
12942
  {
@@ -12999,7 +12965,7 @@
12999
12965
  "kind": "class",
13000
12966
  "locationInModule": {
13001
12967
  "filename": "lib/data-quality-ruleset.ts",
13002
- "line": 95
12968
+ "line": 126
13003
12969
  },
13004
12970
  "methods": [
13005
12971
  {
@@ -13008,7 +12974,7 @@
13008
12974
  },
13009
12975
  "locationInModule": {
13010
12976
  "filename": "lib/data-quality-ruleset.ts",
13011
- "line": 100
12977
+ "line": 131
13012
12978
  },
13013
12979
  "name": "fromRulesetArn",
13014
12980
  "parameters": [
@@ -13044,7 +13010,7 @@
13044
13010
  },
13045
13011
  "locationInModule": {
13046
13012
  "filename": "lib/data-quality-ruleset.ts",
13047
- "line": 109
13013
+ "line": 140
13048
13014
  },
13049
13015
  "name": "fromRulesetName",
13050
13016
  "parameters": [
@@ -13086,7 +13052,7 @@
13086
13052
  "immutable": true,
13087
13053
  "locationInModule": {
13088
13054
  "filename": "lib/data-quality-ruleset.ts",
13089
- "line": 98
13055
+ "line": 129
13090
13056
  },
13091
13057
  "name": "PROPERTY_INJECTION_ID",
13092
13058
  "static": true,
@@ -13102,7 +13068,7 @@
13102
13068
  "immutable": true,
13103
13069
  "locationInModule": {
13104
13070
  "filename": "lib/data-quality-ruleset.ts",
13105
- "line": 158
13071
+ "line": 189
13106
13072
  },
13107
13073
  "name": "rulesetArn",
13108
13074
  "overrides": "@aws-cdk/aws-glue-alpha.IDataQualityRuleset",
@@ -13118,7 +13084,7 @@
13118
13084
  "immutable": true,
13119
13085
  "locationInModule": {
13120
13086
  "filename": "lib/data-quality-ruleset.ts",
13121
- "line": 150
13087
+ "line": 181
13122
13088
  },
13123
13089
  "name": "rulesetName",
13124
13090
  "overrides": "@aws-cdk/aws-glue-alpha.IDataQualityRuleset",
@@ -13135,36 +13101,34 @@
13135
13101
  "docs": {
13136
13102
  "stability": "experimental",
13137
13103
  "summary": "Construction properties for `DataQualityRuleset`.",
13138
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\n\ndeclare const dataQualityTargetTable: glue_alpha.DataQualityTargetTable;\nconst dataQualityRulesetProps: glue_alpha.DataQualityRulesetProps = {\n rulesetDqdl: 'rulesetDqdl',\n targetTable: dataQualityTargetTable,\n\n // the properties below are optional\n clientToken: 'clientToken',\n description: 'description',\n removalPolicy: cdk.RemovalPolicy.DESTROY,\n rulesetName: 'rulesetName',\n tags: {\n tagsKey: 'tags',\n },\n};",
13104
+ "example": "new glue.DataQualityRuleset(this, 'MyRuleset', {\n rulesetName: 'my_ruleset',\n dqdl: glue.Dqdl.fromString('Rules = [ RowCount > 100, IsComplete \"order_id\" ]'),\n targetTable: new glue.DataQualityTargetTable('my_database', 'my_table'),\n});",
13139
13105
  "custom": {
13140
- "exampleMetadata": "fixture=_generated"
13106
+ "exampleMetadata": "infused"
13141
13107
  }
13142
13108
  },
13143
13109
  "fqn": "@aws-cdk/aws-glue-alpha.DataQualityRulesetProps",
13144
13110
  "kind": "interface",
13145
13111
  "locationInModule": {
13146
13112
  "filename": "lib/data-quality-ruleset.ts",
13147
- "line": 47
13113
+ "line": 77
13148
13114
  },
13149
13115
  "name": "DataQualityRulesetProps",
13150
13116
  "properties": [
13151
13117
  {
13152
13118
  "abstract": true,
13153
13119
  "docs": {
13154
- "custom": {
13155
- "attribute": "true"
13156
- },
13120
+ "remarks": "Build it with `Dqdl.fromString(...)`.",
13157
13121
  "stability": "experimental",
13158
- "summary": "The dqdl of the ruleset."
13122
+ "summary": "The DQDL document defining the ruleset's data quality rules."
13159
13123
  },
13160
13124
  "immutable": true,
13161
13125
  "locationInModule": {
13162
13126
  "filename": "lib/data-quality-ruleset.ts",
13163
- "line": 70
13127
+ "line": 101
13164
13128
  },
13165
- "name": "rulesetDqdl",
13129
+ "name": "dqdl",
13166
13130
  "type": {
13167
- "primitive": "string"
13131
+ "fqn": "@aws-cdk/aws-glue-alpha.Dqdl"
13168
13132
  }
13169
13133
  },
13170
13134
  {
@@ -13179,7 +13143,7 @@
13179
13143
  "immutable": true,
13180
13144
  "locationInModule": {
13181
13145
  "filename": "lib/data-quality-ruleset.ts",
13182
- "line": 82
13146
+ "line": 113
13183
13147
  },
13184
13148
  "name": "targetTable",
13185
13149
  "type": {
@@ -13198,7 +13162,7 @@
13198
13162
  "immutable": true,
13199
13163
  "locationInModule": {
13200
13164
  "filename": "lib/data-quality-ruleset.ts",
13201
- "line": 58
13165
+ "line": 88
13202
13166
  },
13203
13167
  "name": "clientToken",
13204
13168
  "optional": true,
@@ -13218,7 +13182,7 @@
13218
13182
  "immutable": true,
13219
13183
  "locationInModule": {
13220
13184
  "filename": "lib/data-quality-ruleset.ts",
13221
- "line": 64
13185
+ "line": 94
13222
13186
  },
13223
13187
  "name": "description",
13224
13188
  "optional": true,
@@ -13236,7 +13200,7 @@
13236
13200
  "immutable": true,
13237
13201
  "locationInModule": {
13238
13202
  "filename": "lib/data-quality-ruleset.ts",
13239
- "line": 89
13203
+ "line": 120
13240
13204
  },
13241
13205
  "name": "removalPolicy",
13242
13206
  "optional": true,
@@ -13254,7 +13218,7 @@
13254
13218
  "immutable": true,
13255
13219
  "locationInModule": {
13256
13220
  "filename": "lib/data-quality-ruleset.ts",
13257
- "line": 52
13221
+ "line": 82
13258
13222
  },
13259
13223
  "name": "rulesetName",
13260
13224
  "optional": true,
@@ -13272,7 +13236,7 @@
13272
13236
  "immutable": true,
13273
13237
  "locationInModule": {
13274
13238
  "filename": "lib/data-quality-ruleset.ts",
13275
- "line": 76
13239
+ "line": 107
13276
13240
  },
13277
13241
  "name": "tags",
13278
13242
  "optional": true,
@@ -13293,9 +13257,9 @@
13293
13257
  "docs": {
13294
13258
  "stability": "experimental",
13295
13259
  "summary": "Properties of a DataQualityTargetTable.",
13296
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nconst dataQualityTargetTable = new glue_alpha.DataQualityTargetTable('databaseName', 'tableName');",
13260
+ "example": "new glue.DataQualityRuleset(this, 'MyRuleset', {\n rulesetName: 'my_ruleset',\n dqdl: glue.Dqdl.fromString('Rules = [ RowCount > 100, IsComplete \"order_id\" ]'),\n targetTable: new glue.DataQualityTargetTable('my_database', 'my_table'),\n});",
13297
13261
  "custom": {
13298
- "exampleMetadata": "fixture=_generated"
13262
+ "exampleMetadata": "infused"
13299
13263
  }
13300
13264
  },
13301
13265
  "fqn": "@aws-cdk/aws-glue-alpha.DataQualityTargetTable",
@@ -13523,6 +13487,7 @@
13523
13487
  "stability": "experimental",
13524
13488
  "summary": "Location URI of this database."
13525
13489
  },
13490
+ "immutable": true,
13526
13491
  "locationInModule": {
13527
13492
  "filename": "lib/database.ts",
13528
13493
  "line": 103
@@ -13833,6 +13798,57 @@
13833
13798
  ],
13834
13799
  "symbolId": "lib/partition-projection:DatePartitionProjectionConfigurationProps"
13835
13800
  },
13801
+ "@aws-cdk/aws-glue-alpha.Dqdl": {
13802
+ "assembly": "@aws-cdk/aws-glue-alpha",
13803
+ "docs": {
13804
+ "remarks": "DQDL is an authored string that Glue parses and validates at deploy time. Build\none from a raw DQDL string with {@link Dqdl.fromString}.",
13805
+ "see": "https://docs.aws.amazon.com/glue/latest/dg/dqdl.html",
13806
+ "stability": "experimental",
13807
+ "summary": "The Data Quality Definition Language (DQDL) document for a `DataQualityRuleset`.",
13808
+ "example": "new glue.DataQualityRuleset(this, 'MyRuleset', {\n rulesetName: 'my_ruleset',\n dqdl: glue.Dqdl.fromString('Rules = [ RowCount > 100, IsComplete \"order_id\" ]'),\n targetTable: new glue.DataQualityTargetTable('my_database', 'my_table'),\n});",
13809
+ "custom": {
13810
+ "exampleMetadata": "infused"
13811
+ }
13812
+ },
13813
+ "fqn": "@aws-cdk/aws-glue-alpha.Dqdl",
13814
+ "kind": "class",
13815
+ "locationInModule": {
13816
+ "filename": "lib/data-quality-ruleset.ts",
13817
+ "line": 38
13818
+ },
13819
+ "methods": [
13820
+ {
13821
+ "docs": {
13822
+ "stability": "experimental",
13823
+ "summary": "Create a `Dqdl` from a raw DQDL string."
13824
+ },
13825
+ "locationInModule": {
13826
+ "filename": "lib/data-quality-ruleset.ts",
13827
+ "line": 44
13828
+ },
13829
+ "name": "fromString",
13830
+ "parameters": [
13831
+ {
13832
+ "docs": {
13833
+ "summary": "the DQDL document, e.g. `Rules = [ RowCount > 100 ]`."
13834
+ },
13835
+ "name": "dqdl",
13836
+ "type": {
13837
+ "primitive": "string"
13838
+ }
13839
+ }
13840
+ ],
13841
+ "returns": {
13842
+ "type": {
13843
+ "fqn": "@aws-cdk/aws-glue-alpha.Dqdl"
13844
+ }
13845
+ },
13846
+ "static": true
13847
+ }
13848
+ ],
13849
+ "name": "Dqdl",
13850
+ "symbolId": "lib/data-quality-ruleset:Dqdl"
13851
+ },
13836
13852
  "@aws-cdk/aws-glue-alpha.EnumPartitionProjectionConfigurationProps": {
13837
13853
  "assembly": "@aws-cdk/aws-glue-alpha",
13838
13854
  "datatype": true,
@@ -14020,7 +14036,7 @@
14020
14036
  },
14021
14037
  "locationInModule": {
14022
14038
  "filename": "lib/external-table.ts",
14023
- "line": 137
14039
+ "line": 139
14024
14040
  },
14025
14041
  "name": "grantRead",
14026
14042
  "overrides": "@aws-cdk/aws-glue-alpha.TableBase",
@@ -14048,7 +14064,7 @@
14048
14064
  },
14049
14065
  "locationInModule": {
14050
14066
  "filename": "lib/external-table.ts",
14051
- "line": 161
14067
+ "line": 163
14052
14068
  },
14053
14069
  "name": "grantReadWrite",
14054
14070
  "overrides": "@aws-cdk/aws-glue-alpha.TableBase",
@@ -14076,7 +14092,7 @@
14076
14092
  },
14077
14093
  "locationInModule": {
14078
14094
  "filename": "lib/external-table.ts",
14079
- "line": 149
14095
+ "line": 151
14080
14096
  },
14081
14097
  "name": "grantWrite",
14082
14098
  "overrides": "@aws-cdk/aws-glue-alpha.TableBase",
@@ -14140,7 +14156,7 @@
14140
14156
  "immutable": true,
14141
14157
  "locationInModule": {
14142
14158
  "filename": "lib/external-table.ts",
14143
- "line": 122
14159
+ "line": 124
14144
14160
  },
14145
14161
  "name": "tableArn",
14146
14162
  "overrides": "@aws-cdk/aws-glue-alpha.TableBase",
@@ -14156,7 +14172,7 @@
14156
14172
  "immutable": true,
14157
14173
  "locationInModule": {
14158
14174
  "filename": "lib/external-table.ts",
14159
- "line": 114
14175
+ "line": 116
14160
14176
  },
14161
14177
  "name": "tableName",
14162
14178
  "overrides": "@aws-cdk/aws-glue-alpha.TableBase",
@@ -14270,7 +14286,7 @@
14270
14286
  "see": "https://docs.aws.amazon.com/glue/latest/dg/add-job.html.",
14271
14287
  "stability": "experimental",
14272
14288
  "summary": "AWS Glue version determines the versions of Apache Spark and Python that are available to the job.",
14273
- "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkEtlJob(stack, 'PySparkETLJob', {\n jobName: 'PySparkETLJobCustomName',\n description: 'This is a description',\n role,\n script,\n glueVersion: glue.GlueVersion.V5_1,\n continuousLogging: { enabled: false },\n workerType: glue.WorkerType.G_2X,\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n numberOfWorkers: 2,\n maxRetries: 2,\n});",
14289
+ "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkEtlJob(stack, 'PySparkETLJob', {\n jobName: 'PySparkETLJobCustomName',\n description: 'This is a description',\n role,\n script,\n glueVersion: glue.GlueVersion.V5_1,\n continuousLogging: { enabled: false },\n workerConfiguration: {\n workerType: glue.WorkerType.G_2X,\n numberOfWorkers: 2,\n },\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n maxRetries: 2,\n});",
14274
14290
  "custom": {
14275
14291
  "exampleMetadata": "infused"
14276
14292
  }
@@ -14500,7 +14516,7 @@
14500
14516
  "kind": "interface",
14501
14517
  "locationInModule": {
14502
14518
  "filename": "lib/data-quality-ruleset.ts",
14503
- "line": 30
14519
+ "line": 60
14504
14520
  },
14505
14521
  "name": "IDataQualityRuleset",
14506
14522
  "properties": [
@@ -14516,7 +14532,7 @@
14516
14532
  "immutable": true,
14517
14533
  "locationInModule": {
14518
14534
  "filename": "lib/data-quality-ruleset.ts",
14519
- "line": 35
14535
+ "line": 65
14520
14536
  },
14521
14537
  "name": "rulesetArn",
14522
14538
  "type": {
@@ -14535,7 +14551,7 @@
14535
14551
  "immutable": true,
14536
14552
  "locationInModule": {
14537
14553
  "filename": "lib/data-quality-ruleset.ts",
14538
- "line": 41
14554
+ "line": 71
14539
14555
  },
14540
14556
  "name": "rulesetName",
14541
14557
  "type": {
@@ -15550,7 +15566,7 @@
15550
15566
  "kind": "class",
15551
15567
  "locationInModule": {
15552
15568
  "filename": "lib/jobs/job.ts",
15553
- "line": 465
15569
+ "line": 439
15554
15570
  },
15555
15571
  "methods": [
15556
15572
  {
@@ -15560,7 +15576,7 @@
15560
15576
  },
15561
15577
  "locationInModule": {
15562
15578
  "filename": "lib/jobs/job.ts",
15563
- "line": 474
15579
+ "line": 448
15564
15580
  },
15565
15581
  "name": "fromJobAttributes",
15566
15582
  "parameters": [
@@ -15607,7 +15623,7 @@
15607
15623
  },
15608
15624
  "locationInModule": {
15609
15625
  "filename": "lib/jobs/job.ts",
15610
- "line": 494
15626
+ "line": 468
15611
15627
  },
15612
15628
  "name": "checkNoReservedArgs",
15613
15629
  "parameters": [
@@ -15643,7 +15659,7 @@
15643
15659
  },
15644
15660
  "locationInModule": {
15645
15661
  "filename": "lib/jobs/job.ts",
15646
- "line": 550
15662
+ "line": 524
15647
15663
  },
15648
15664
  "name": "codeS3ObjectUrl",
15649
15665
  "parameters": [
@@ -15669,7 +15685,7 @@
15669
15685
  },
15670
15686
  "locationInModule": {
15671
15687
  "filename": "lib/jobs/job.ts",
15672
- "line": 518
15688
+ "line": 492
15673
15689
  },
15674
15690
  "name": "setupContinuousLogging",
15675
15691
  "parameters": [
@@ -15712,7 +15728,7 @@
15712
15728
  "immutable": true,
15713
15729
  "locationInModule": {
15714
15730
  "filename": "lib/jobs/job.ts",
15715
- "line": 487
15731
+ "line": 461
15716
15732
  },
15717
15733
  "name": "role",
15718
15734
  "type": {
@@ -16251,90 +16267,57 @@
16251
16267
  },
16252
16268
  "@aws-cdk/aws-glue-alpha.JobBookmarksEncryption": {
16253
16269
  "assembly": "@aws-cdk/aws-glue-alpha",
16254
- "datatype": true,
16255
16270
  "docs": {
16271
+ "remarks": "Job bookmarks support only client-side encryption with a KMS key.",
16256
16272
  "stability": "experimental",
16257
- "summary": "Job bookmarks encryption configuration.",
16258
- "example": "new glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n },\n jobBookmarksEncryption: {\n mode: glue.JobBookmarksEncryptionMode.CLIENT_SIDE_KMS,\n },\n s3Encryption: {\n mode: glue.S3EncryptionMode.KMS,\n },\n});",
16273
+ "summary": "Job bookmarks encryption configuration for a `SecurityConfiguration`.",
16274
+ "example": "new glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: glue.CloudWatchEncryption.kms(),\n jobBookmarksEncryption: glue.JobBookmarksEncryption.clientSideKms(),\n s3Encryption: glue.S3Encryption.kms(),\n});",
16259
16275
  "custom": {
16260
16276
  "exampleMetadata": "infused"
16261
16277
  }
16262
16278
  },
16263
16279
  "fqn": "@aws-cdk/aws-glue-alpha.JobBookmarksEncryption",
16264
- "kind": "interface",
16280
+ "kind": "class",
16265
16281
  "locationInModule": {
16266
16282
  "filename": "lib/security-configuration.ts",
16267
- "line": 102
16283
+ "line": 112
16268
16284
  },
16269
- "name": "JobBookmarksEncryption",
16270
- "properties": [
16285
+ "methods": [
16271
16286
  {
16272
- "abstract": true,
16273
16287
  "docs": {
16288
+ "see": "https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html",
16274
16289
  "stability": "experimental",
16275
- "summary": "Encryption mode."
16290
+ "summary": "Client-side encryption (CSE) with an AWS KMS key managed by the account owner."
16276
16291
  },
16277
- "immutable": true,
16278
16292
  "locationInModule": {
16279
16293
  "filename": "lib/security-configuration.ts",
16280
- "line": 106
16281
- },
16282
- "name": "mode",
16283
- "type": {
16284
- "fqn": "@aws-cdk/aws-glue-alpha.JobBookmarksEncryptionMode"
16285
- }
16286
- },
16287
- {
16288
- "abstract": true,
16289
- "docs": {
16290
- "default": "A key will be created if one is not provided.",
16291
- "stability": "experimental",
16292
- "summary": "The KMS key to be used to encrypt the data."
16294
+ "line": 119
16293
16295
  },
16294
- "immutable": true,
16295
- "locationInModule": {
16296
- "filename": "lib/security-configuration.ts",
16297
- "line": 112
16296
+ "name": "clientSideKms",
16297
+ "parameters": [
16298
+ {
16299
+ "docs": {
16300
+ "remarks": "A key is created if one is not provided.",
16301
+ "summary": "the KMS key used to encrypt the data."
16302
+ },
16303
+ "name": "kmsKey",
16304
+ "optional": true,
16305
+ "type": {
16306
+ "fqn": "aws-cdk-lib.interfaces.aws_kms.IKeyRef"
16307
+ }
16308
+ }
16309
+ ],
16310
+ "returns": {
16311
+ "type": {
16312
+ "fqn": "@aws-cdk/aws-glue-alpha.JobBookmarksEncryption"
16313
+ }
16298
16314
  },
16299
- "name": "kmsKey",
16300
- "optional": true,
16301
- "type": {
16302
- "fqn": "aws-cdk-lib.interfaces.aws_kms.IKeyRef"
16303
- }
16315
+ "static": true
16304
16316
  }
16305
16317
  ],
16318
+ "name": "JobBookmarksEncryption",
16306
16319
  "symbolId": "lib/security-configuration:JobBookmarksEncryption"
16307
16320
  },
16308
- "@aws-cdk/aws-glue-alpha.JobBookmarksEncryptionMode": {
16309
- "assembly": "@aws-cdk/aws-glue-alpha",
16310
- "docs": {
16311
- "see": "https://docs.aws.amazon.com/glue/latest/webapi/API_JobBookmarksEncryption.html#Glue-Type-JobBookmarksEncryption-JobBookmarksEncryptionMode",
16312
- "stability": "experimental",
16313
- "summary": "Encryption mode for Job Bookmarks.",
16314
- "example": "new glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n },\n jobBookmarksEncryption: {\n mode: glue.JobBookmarksEncryptionMode.CLIENT_SIDE_KMS,\n },\n s3Encryption: {\n mode: glue.S3EncryptionMode.KMS,\n },\n});",
16315
- "custom": {
16316
- "exampleMetadata": "infused"
16317
- }
16318
- },
16319
- "fqn": "@aws-cdk/aws-glue-alpha.JobBookmarksEncryptionMode",
16320
- "kind": "enum",
16321
- "locationInModule": {
16322
- "filename": "lib/security-configuration.ts",
16323
- "line": 58
16324
- },
16325
- "members": [
16326
- {
16327
- "docs": {
16328
- "see": "https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingClientSideEncryption.html",
16329
- "stability": "experimental",
16330
- "summary": "Client-side encryption (CSE) with an AWS KMS key managed by the account owner."
16331
- },
16332
- "name": "CLIENT_SIDE_KMS"
16333
- }
16334
- ],
16335
- "name": "JobBookmarksEncryptionMode",
16336
- "symbolId": "lib/security-configuration:JobBookmarksEncryptionMode"
16337
- },
16338
16321
  "@aws-cdk/aws-glue-alpha.JobLanguage": {
16339
16322
  "assembly": "@aws-cdk/aws-glue-alpha",
16340
16323
  "docs": {
@@ -16372,7 +16355,7 @@
16372
16355
  "docs": {
16373
16356
  "stability": "experimental",
16374
16357
  "summary": "JobProps will be used to create new Glue Jobs using this L2 Construct.",
16375
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\nimport { aws_logs as logs } from 'aws-cdk-lib';\n\ndeclare const code: glue_alpha.Code;\ndeclare const connection: glue_alpha.Connection;\ndeclare const logGroup: logs.LogGroup;\ndeclare const role: iam.Role;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst jobProps: glue_alpha.JobProps = {\n role: role,\n script: code,\n\n // the properties below are optional\n connections: [connection],\n continuousLogging: {\n enabled: false,\n\n // the properties below are optional\n conversionPattern: 'conversionPattern',\n logGroup: logGroup,\n logStreamPrefix: 'logStreamPrefix',\n quiet: false,\n },\n defaultArguments: {\n defaultArgumentsKey: 'defaultArguments',\n },\n description: 'description',\n enableProfilingMetrics: false,\n glueVersion: glue_alpha.GlueVersion.V0_9,\n jobName: 'jobName',\n maxConcurrentRuns: 123,\n maxRetries: 123,\n numberOfWorkers: 123,\n securityConfiguration: securityConfiguration,\n tags: {\n tagsKey: 'tags',\n },\n timeout: cdk.Duration.minutes(30),\n workerType: glue_alpha.WorkerType.STANDARD,\n};",
16358
+ "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\nimport { aws_logs as logs } from 'aws-cdk-lib';\n\ndeclare const code: glue_alpha.Code;\ndeclare const connection: glue_alpha.Connection;\ndeclare const logGroup: logs.LogGroup;\ndeclare const role: iam.Role;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst jobProps: glue_alpha.JobProps = {\n role: role,\n script: code,\n\n // the properties below are optional\n connections: [connection],\n continuousLogging: {\n enabled: false,\n\n // the properties below are optional\n conversionPattern: 'conversionPattern',\n logGroup: logGroup,\n logStreamPrefix: 'logStreamPrefix',\n quiet: false,\n },\n defaultArguments: {\n defaultArgumentsKey: 'defaultArguments',\n },\n description: 'description',\n glueVersion: glue_alpha.GlueVersion.V0_9,\n jobName: 'jobName',\n maxConcurrentRuns: 123,\n maxRetries: 123,\n securityConfiguration: securityConfiguration,\n tags: {\n tagsKey: 'tags',\n },\n timeout: cdk.Duration.minutes(30),\n};",
16376
16359
  "custom": {
16377
16360
  "exampleMetadata": "fixture=_generated"
16378
16361
  }
@@ -16429,7 +16412,7 @@
16429
16412
  "immutable": true,
16430
16413
  "locationInModule": {
16431
16414
  "filename": "lib/jobs/job.ts",
16432
- "line": 397
16415
+ "line": 380
16433
16416
  },
16434
16417
  "name": "connections",
16435
16418
  "optional": true,
@@ -16453,7 +16436,7 @@
16453
16436
  "immutable": true,
16454
16437
  "locationInModule": {
16455
16438
  "filename": "lib/jobs/job.ts",
16456
- "line": 458
16439
+ "line": 432
16457
16440
  },
16458
16441
  "name": "continuousLogging",
16459
16442
  "optional": true,
@@ -16473,7 +16456,7 @@
16473
16456
  "immutable": true,
16474
16457
  "locationInModule": {
16475
16458
  "filename": "lib/jobs/job.ts",
16476
- "line": 388
16459
+ "line": 371
16477
16460
  },
16478
16461
  "name": "defaultArguments",
16479
16462
  "optional": true,
@@ -16504,25 +16487,6 @@
16504
16487
  "primitive": "string"
16505
16488
  }
16506
16489
  },
16507
- {
16508
- "abstract": true,
16509
- "docs": {
16510
- "default": "- no profiling metrics emitted.",
16511
- "see": "https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html",
16512
- "stability": "experimental",
16513
- "summary": "Enables the collection of metrics for job profiling."
16514
- },
16515
- "immutable": true,
16516
- "locationInModule": {
16517
- "filename": "lib/jobs/job.ts",
16518
- "line": 448
16519
- },
16520
- "name": "enableProfilingMetrics",
16521
- "optional": true,
16522
- "type": {
16523
- "primitive": "boolean"
16524
- }
16525
- },
16526
16490
  {
16527
16491
  "abstract": true,
16528
16492
  "docs": {
@@ -16533,7 +16497,7 @@
16533
16497
  "immutable": true,
16534
16498
  "locationInModule": {
16535
16499
  "filename": "lib/jobs/job.ts",
16536
- "line": 439
16500
+ "line": 422
16537
16501
  },
16538
16502
  "name": "glueVersion",
16539
16503
  "optional": true,
@@ -16570,7 +16534,7 @@
16570
16534
  "immutable": true,
16571
16535
  "locationInModule": {
16572
16536
  "filename": "lib/jobs/job.ts",
16573
- "line": 372
16537
+ "line": 355
16574
16538
  },
16575
16539
  "name": "maxConcurrentRuns",
16576
16540
  "optional": true,
@@ -16588,7 +16552,7 @@
16588
16552
  "immutable": true,
16589
16553
  "locationInModule": {
16590
16554
  "filename": "lib/jobs/job.ts",
16591
- "line": 405
16555
+ "line": 388
16592
16556
  },
16593
16557
  "name": "maxRetries",
16594
16558
  "optional": true,
@@ -16596,24 +16560,6 @@
16596
16560
  "primitive": "number"
16597
16561
  }
16598
16562
  },
16599
- {
16600
- "abstract": true,
16601
- "docs": {
16602
- "default": "10",
16603
- "stability": "experimental",
16604
- "summary": "Number of Workers (optional) Number of workers for Glue to use during job execution."
16605
- },
16606
- "immutable": true,
16607
- "locationInModule": {
16608
- "filename": "lib/jobs/job.ts",
16609
- "line": 352
16610
- },
16611
- "name": "numberOfWorkers",
16612
- "optional": true,
16613
- "type": {
16614
- "primitive": "number"
16615
- }
16616
- },
16617
16563
  {
16618
16564
  "abstract": true,
16619
16565
  "docs": {
@@ -16624,7 +16570,7 @@
16624
16570
  "immutable": true,
16625
16571
  "locationInModule": {
16626
16572
  "filename": "lib/jobs/job.ts",
16627
- "line": 423
16573
+ "line": 406
16628
16574
  },
16629
16575
  "name": "securityConfiguration",
16630
16576
  "optional": true,
@@ -16642,7 +16588,7 @@
16642
16588
  "immutable": true,
16643
16589
  "locationInModule": {
16644
16590
  "filename": "lib/jobs/job.ts",
16645
- "line": 431
16591
+ "line": 414
16646
16592
  },
16647
16593
  "name": "tags",
16648
16594
  "optional": true,
@@ -16666,32 +16612,13 @@
16666
16612
  "immutable": true,
16667
16613
  "locationInModule": {
16668
16614
  "filename": "lib/jobs/job.ts",
16669
- "line": 415
16615
+ "line": 398
16670
16616
  },
16671
16617
  "name": "timeout",
16672
16618
  "optional": true,
16673
16619
  "type": {
16674
16620
  "fqn": "aws-cdk-lib.Duration"
16675
16621
  }
16676
- },
16677
- {
16678
- "abstract": true,
16679
- "docs": {
16680
- "default": "WorkerType.G_1X",
16681
- "remarks": "G_4X, G_8X, Z_2X",
16682
- "stability": "experimental",
16683
- "summary": "Worker Type (optional) Type of Worker for Glue to use during job execution Enum options: Standard, G_1X, G_2X, G_025X."
16684
- },
16685
- "immutable": true,
16686
- "locationInModule": {
16687
- "filename": "lib/jobs/job.ts",
16688
- "line": 361
16689
- },
16690
- "name": "workerType",
16691
- "optional": true,
16692
- "type": {
16693
- "fqn": "@aws-cdk/aws-glue-alpha.WorkerType"
16694
- }
16695
16622
  }
16696
16623
  ],
16697
16624
  "symbolId": "lib/jobs/job:JobProps"
@@ -16815,7 +16742,7 @@
16815
16742
  "remarks": "A DPU is a relative measure of processing power that consists of 4 vCPUs of compute capacity and 16 GB of memory.",
16816
16743
  "stability": "experimental",
16817
16744
  "summary": "The number of AWS Glue data processing units (DPUs) that can be allocated when this job runs.",
16818
- "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\ndeclare const extraPythonFile: glue.Code;\nnew glue.PythonShellJob(stack, 'PythonShellJob', {\n jobName: 'PythonShellJobCustomName',\n description: 'This is a description',\n pythonVersion: glue.PythonVersion.TWO,\n maxCapacity: glue.MaxCapacity.DPU_1,\n role,\n script,\n extraPythonFiles: [extraPythonFile],\n glueVersion: glue.GlueVersion.V2_0,\n continuousLogging: { enabled: false },\n workerType: glue.WorkerType.G_2X,\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n numberOfWorkers: 2,\n maxRetries: 2,\n});",
16745
+ "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\ndeclare const extraPythonFile: glue.Code;\nnew glue.PythonShellJob(stack, 'PythonShellJob', {\n jobName: 'PythonShellJobCustomName',\n description: 'This is a description',\n pythonVersion: glue.PythonVersion.THREE_NINE,\n maxCapacity: glue.MaxCapacity.DPU_1,\n role,\n script,\n extraPythonFiles: [extraPythonFile],\n glueVersion: glue.GlueVersion.V3_0,\n continuousLogging: { enabled: false },\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n maxRetries: 2,\n});",
16819
16746
  "custom": {
16820
16747
  "exampleMetadata": "infused"
16821
16748
  }
@@ -16972,9 +16899,9 @@
16972
16899
  "docs": {
16973
16900
  "stability": "experimental",
16974
16901
  "summary": "Properties for configuring an on-demand Glue Trigger.",
16975
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_glue as glue } from 'aws-cdk-lib';\n\ndeclare const cfnCrawler: glue.CfnCrawler;\ndeclare const job: glue_alpha.Job;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst onDemandTriggerOptions: glue_alpha.OnDemandTriggerOptions = {\n actions: [{\n arguments: {\n argumentsKey: 'arguments',\n },\n crawler: cfnCrawler,\n job: job,\n securityConfiguration: securityConfiguration,\n timeout: cdk.Duration.minutes(30),\n }],\n\n // the properties below are optional\n description: 'description',\n name: 'name',\n};",
16902
+ "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\n\n// Create a job to run from the workflow\nconst job = new glue.PySparkEtlJob(stack, 'Job', { role, script });\n\n// Create a workflow and add a trigger that runs the job\nconst workflow = new glue.Workflow(stack, 'Workflow');\nworkflow.addOnDemandTrigger('OnDemandTrigger', {\n actions: [{ job }],\n});",
16976
16903
  "custom": {
16977
- "exampleMetadata": "fixture=_generated"
16904
+ "exampleMetadata": "infused"
16978
16905
  }
16979
16906
  },
16980
16907
  "fqn": "@aws-cdk/aws-glue-alpha.OnDemandTriggerOptions",
@@ -17026,7 +16953,7 @@
17026
16953
  "docs": {
17027
16954
  "stability": "experimental",
17028
16955
  "summary": "Absolute class name of the Hadoop `OutputFormat` to use when writing table files.",
17029
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nconst outputFormat = new glue_alpha.OutputFormat('className');",
16956
+ "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nconst outputFormat = glue_alpha.OutputFormat.AVRO;",
17030
16957
  "custom": {
17031
16958
  "exampleMetadata": "fixture=_generated"
17032
16959
  }
@@ -17071,7 +16998,7 @@
17071
16998
  "name": "AVRO",
17072
16999
  "static": true,
17073
17000
  "type": {
17074
- "fqn": "@aws-cdk/aws-glue-alpha.InputFormat"
17001
+ "fqn": "@aws-cdk/aws-glue-alpha.OutputFormat"
17075
17002
  }
17076
17003
  },
17077
17004
  {
@@ -17107,7 +17034,7 @@
17107
17034
  "name": "ORC",
17108
17035
  "static": true,
17109
17036
  "type": {
17110
- "fqn": "@aws-cdk/aws-glue-alpha.InputFormat"
17037
+ "fqn": "@aws-cdk/aws-glue-alpha.OutputFormat"
17111
17038
  }
17112
17039
  },
17113
17040
  {
@@ -17151,7 +17078,7 @@
17151
17078
  "docs": {
17152
17079
  "stability": "experimental",
17153
17080
  "summary": "Properties of a Partition Index.",
17154
- "example": "declare const myTable: glue.Table;\nmyTable.addPartitionIndex({\n indexName: 'my-index',\n keyNames: ['year'],\n});",
17081
+ "example": "declare const myTable: glue.S3Table;\nmyTable.addPartitionIndex({\n indexName: 'my-index',\n keyNames: ['year'],\n});",
17155
17082
  "custom": {
17156
17083
  "exampleMetadata": "infused"
17157
17084
  }
@@ -17612,10 +17539,10 @@
17612
17539
  "assembly": "@aws-cdk/aws-glue-alpha",
17613
17540
  "base": "@aws-cdk/aws-glue-alpha.SparkJob",
17614
17541
  "docs": {
17615
- "remarks": "ETL jobs support pySpark and Scala languages, for which there are separate\nbut similar constructors. ETL jobs default to the G2 worker type, but you\ncan override this default with other supported worker type values\n(G1, G2, G4 and G8). ETL jobs defaults to Glue version 4.0, which you can\noverride to 3.0. The following ETL features are enabled by default:\nenable-metrics, enable-spark-ui, enable-continuous-cloudwatch-log.\nYou can find more details about version, worker type and other features\nin Glue's public documentation.",
17542
+ "remarks": "ETL jobs support pySpark and Scala languages, for which there are separate\nbut similar constructors. ETL jobs default to the G2 worker type, but you\ncan override this default with other supported worker type values\n(G1, G2, G4 and G8). ETL jobs defaults to Glue version 4.0, which you can\noverride to 3.0. The following ETL features are enabled by default:\n--enable-metrics, --enable-continuous-cloudwatch-log. The Spark UI\n(--enable-spark-ui) is off by default; enable it by setting the `sparkUI` prop.\nYou can find more details about version, worker type and other features\nin Glue's public documentation.",
17616
17543
  "stability": "experimental",
17617
17544
  "summary": "PySpark ETL Jobs class.",
17618
- "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\n\n// Disable both metrics for cost optimization\nnew glue.PySparkEtlJob(stack, 'CostOptimizedJob', {\n role,\n script,\n enableMetrics: false,\n enableObservabilityMetrics: false,\n});\n\n// Selective control - keep observability, disable profiling\nnew glue.PySparkEtlJob(stack, 'SelectiveJob', {\n role,\n script,\n enableMetrics: false,\n // enableObservabilityMetrics defaults to true\n});",
17545
+ "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\n\n// Create a job to run from the workflow\nconst job = new glue.PySparkEtlJob(stack, 'Job', { role, script });\n\n// Create a workflow and add a trigger that runs the job\nconst workflow = new glue.Workflow(stack, 'Workflow');\nworkflow.addOnDemandTrigger('OnDemandTrigger', {\n actions: [{ job }],\n});",
17619
17546
  "custom": {
17620
17547
  "exampleMetadata": "infused"
17621
17548
  }
@@ -17628,7 +17555,7 @@
17628
17555
  },
17629
17556
  "locationInModule": {
17630
17557
  "filename": "lib/jobs/pyspark-etl-job.ts",
17631
- "line": 90
17558
+ "line": 92
17632
17559
  },
17633
17560
  "parameters": [
17634
17561
  {
@@ -17654,7 +17581,7 @@
17654
17581
  "kind": "class",
17655
17582
  "locationInModule": {
17656
17583
  "filename": "lib/jobs/pyspark-etl-job.ts",
17657
- "line": 81
17584
+ "line": 83
17658
17585
  },
17659
17586
  "name": "PySparkEtlJob",
17660
17587
  "properties": [
@@ -17667,7 +17594,7 @@
17667
17594
  "immutable": true,
17668
17595
  "locationInModule": {
17669
17596
  "filename": "lib/jobs/pyspark-etl-job.ts",
17670
- "line": 84
17597
+ "line": 86
17671
17598
  },
17672
17599
  "name": "PROPERTY_INJECTION_ID",
17673
17600
  "static": true,
@@ -17683,7 +17610,7 @@
17683
17610
  "immutable": true,
17684
17611
  "locationInModule": {
17685
17612
  "filename": "lib/jobs/pyspark-etl-job.ts",
17686
- "line": 125
17613
+ "line": 132
17687
17614
  },
17688
17615
  "name": "jobArn",
17689
17616
  "overrides": "@aws-cdk/aws-glue-alpha.JobBase",
@@ -17699,7 +17626,7 @@
17699
17626
  "immutable": true,
17700
17627
  "locationInModule": {
17701
17628
  "filename": "lib/jobs/pyspark-etl-job.ts",
17702
- "line": 130
17629
+ "line": 137
17703
17630
  },
17704
17631
  "name": "jobName",
17705
17632
  "overrides": "@aws-cdk/aws-glue-alpha.JobBase",
@@ -17716,7 +17643,7 @@
17716
17643
  "docs": {
17717
17644
  "stability": "experimental",
17718
17645
  "summary": "Properties for creating a Python Spark ETL job.",
17719
- "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\n\n// Disable both metrics for cost optimization\nnew glue.PySparkEtlJob(stack, 'CostOptimizedJob', {\n role,\n script,\n enableMetrics: false,\n enableObservabilityMetrics: false,\n});\n\n// Selective control - keep observability, disable profiling\nnew glue.PySparkEtlJob(stack, 'SelectiveJob', {\n role,\n script,\n enableMetrics: false,\n // enableObservabilityMetrics defaults to true\n});",
17646
+ "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\n\n// Create a job to run from the workflow\nconst job = new glue.PySparkEtlJob(stack, 'Job', { role, script });\n\n// Create a workflow and add a trigger that runs the job\nconst workflow = new glue.Workflow(stack, 'Workflow');\nworkflow.addOnDemandTrigger('OnDemandTrigger', {\n actions: [{ job }],\n});",
17720
17647
  "custom": {
17721
17648
  "exampleMetadata": "infused"
17722
17649
  }
@@ -17728,7 +17655,7 @@
17728
17655
  "kind": "interface",
17729
17656
  "locationInModule": {
17730
17657
  "filename": "lib/jobs/pyspark-etl-job.ts",
17731
- "line": 15
17658
+ "line": 16
17732
17659
  },
17733
17660
  "name": "PySparkEtlJobProps",
17734
17661
  "properties": [
@@ -17743,7 +17670,7 @@
17743
17670
  "immutable": true,
17744
17671
  "locationInModule": {
17745
17672
  "filename": "lib/jobs/pyspark-etl-job.ts",
17746
- "line": 38
17673
+ "line": 39
17747
17674
  },
17748
17675
  "name": "extraFiles",
17749
17676
  "optional": true,
@@ -17766,7 +17693,7 @@
17766
17693
  "immutable": true,
17767
17694
  "locationInModule": {
17768
17695
  "filename": "lib/jobs/pyspark-etl-job.ts",
17769
- "line": 45
17696
+ "line": 46
17770
17697
  },
17771
17698
  "name": "extraJars",
17772
17699
  "optional": true,
@@ -17790,7 +17717,7 @@
17790
17717
  "immutable": true,
17791
17718
  "locationInModule": {
17792
17719
  "filename": "lib/jobs/pyspark-etl-job.ts",
17793
- "line": 54
17720
+ "line": 55
17794
17721
  },
17795
17722
  "name": "extraJarsFirst",
17796
17723
  "optional": true,
@@ -17808,7 +17735,7 @@
17808
17735
  "immutable": true,
17809
17736
  "locationInModule": {
17810
17737
  "filename": "lib/jobs/pyspark-etl-job.ts",
17811
- "line": 29
17738
+ "line": 30
17812
17739
  },
17813
17740
  "name": "extraPythonFiles",
17814
17741
  "optional": true,
@@ -17832,7 +17759,7 @@
17832
17759
  "immutable": true,
17833
17760
  "locationInModule": {
17834
17761
  "filename": "lib/jobs/pyspark-etl-job.ts",
17835
- "line": 66
17762
+ "line": 67
17836
17763
  },
17837
17764
  "name": "jobRunQueuingEnabled",
17838
17765
  "optional": true,
@@ -17851,7 +17778,7 @@
17851
17778
  "immutable": true,
17852
17779
  "locationInModule": {
17853
17780
  "filename": "lib/jobs/pyspark-etl-job.ts",
17854
- "line": 21
17781
+ "line": 22
17855
17782
  },
17856
17783
  "name": "notifyDelayAfter",
17857
17784
  "optional": true,
@@ -17866,7 +17793,7 @@
17866
17793
  "assembly": "@aws-cdk/aws-glue-alpha",
17867
17794
  "base": "@aws-cdk/aws-glue-alpha.SparkJob",
17868
17795
  "docs": {
17869
- "remarks": "Flex jobs supports Python and Scala language.\nThe flexible execution class is appropriate for non-urgent jobs such as\npre-production jobs, testing, and one-time data loads.\nFlexible job runs are supported for jobs using AWS Glue version 3.0 or later and G.1X or\nG.2X worker types but will default to the latest version of Glue (currently Glue 3.0.)\n\nSimilar to ETL, we’ll enable these features: enable-metrics, —enable-spark-ui,\nenable-continuous-cloudwatch-log",
17796
+ "remarks": "Flex jobs supports Python and Scala language.\nThe flexible execution class is appropriate for non-urgent jobs such as\npre-production jobs, testing, and one-time data loads.\nFlexible job runs are supported for jobs using AWS Glue version 3.0 or later and G.1X or\nG.2X worker types but will default to the latest version of Glue (currently Glue 3.0.)\n\nSimilar to ETL, we’ll enable these features: --enable-metrics,\n--enable-continuous-cloudwatch-log. The Spark UI (--enable-spark-ui) is off by\ndefault; enable it by setting the `sparkUI` prop.",
17870
17797
  "stability": "experimental",
17871
17798
  "summary": "Flex Jobs class.",
17872
17799
  "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkFlexEtlJob(stack, 'ImportedJob', { role, script });",
@@ -17882,7 +17809,7 @@
17882
17809
  },
17883
17810
  "locationInModule": {
17884
17811
  "filename": "lib/jobs/pyspark-flex-etl-job.ts",
17885
- "line": 78
17812
+ "line": 79
17886
17813
  },
17887
17814
  "parameters": [
17888
17815
  {
@@ -17908,7 +17835,7 @@
17908
17835
  "kind": "class",
17909
17836
  "locationInModule": {
17910
17837
  "filename": "lib/jobs/pyspark-flex-etl-job.ts",
17911
- "line": 69
17838
+ "line": 70
17912
17839
  },
17913
17840
  "name": "PySparkFlexEtlJob",
17914
17841
  "properties": [
@@ -17921,7 +17848,7 @@
17921
17848
  "immutable": true,
17922
17849
  "locationInModule": {
17923
17850
  "filename": "lib/jobs/pyspark-flex-etl-job.ts",
17924
- "line": 72
17851
+ "line": 73
17925
17852
  },
17926
17853
  "name": "PROPERTY_INJECTION_ID",
17927
17854
  "static": true,
@@ -17937,7 +17864,7 @@
17937
17864
  "immutable": true,
17938
17865
  "locationInModule": {
17939
17866
  "filename": "lib/jobs/pyspark-flex-etl-job.ts",
17940
- "line": 115
17867
+ "line": 116
17941
17868
  },
17942
17869
  "name": "jobArn",
17943
17870
  "overrides": "@aws-cdk/aws-glue-alpha.JobBase",
@@ -17953,7 +17880,7 @@
17953
17880
  "immutable": true,
17954
17881
  "locationInModule": {
17955
17882
  "filename": "lib/jobs/pyspark-flex-etl-job.ts",
17956
- "line": 120
17883
+ "line": 121
17957
17884
  },
17958
17885
  "name": "jobName",
17959
17886
  "overrides": "@aws-cdk/aws-glue-alpha.JobBase",
@@ -18101,7 +18028,7 @@
18101
18028
  "assembly": "@aws-cdk/aws-glue-alpha",
18102
18029
  "base": "@aws-cdk/aws-glue-alpha.SparkJob",
18103
18030
  "docs": {
18104
- "remarks": "A Streaming job is similar to an ETL job, except that it performs ETL on data streams\nusing the Apache Spark Structured Streaming framework.\nThese jobs will default to use Python 3.9.\n\nSimilar to ETL jobs, streaming job supports Scala and Python languages. Similar to ETL,\nit supports G1 and G2 worker type and 2.0, 3.0 and 4.0 version. We’ll default to G2 worker\nand 4.0 version for streaming jobs which developers can override.\nWe will enable enable-metrics, enable-spark-ui, —enable-continuous-cloudwatch-log.",
18031
+ "remarks": "A Streaming job is similar to an ETL job, except that it performs ETL on data streams\nusing the Apache Spark Structured Streaming framework.\nThese jobs will default to use Python 3.9.\n\nSimilar to ETL jobs, streaming job supports Scala and Python languages. Similar to ETL,\nit supports G1 and G2 worker type and 2.0, 3.0 and 4.0 version. We’ll default to G2 worker\nand 4.0 version for streaming jobs which developers can override.\nWe will enable --enable-metrics, --enable-continuous-cloudwatch-log. The Spark UI\n(--enable-spark-ui) is off by default; enable it by setting the `sparkUI` prop.",
18105
18032
  "stability": "experimental",
18106
18033
  "summary": "Python Spark Streaming Jobs class.",
18107
18034
  "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkStreamingJob(stack, 'ImportedJob', { role, script });",
@@ -18117,7 +18044,7 @@
18117
18044
  },
18118
18045
  "locationInModule": {
18119
18046
  "filename": "lib/jobs/pyspark-streaming-job.ts",
18120
- "line": 82
18047
+ "line": 83
18121
18048
  },
18122
18049
  "parameters": [
18123
18050
  {
@@ -18143,7 +18070,7 @@
18143
18070
  "kind": "class",
18144
18071
  "locationInModule": {
18145
18072
  "filename": "lib/jobs/pyspark-streaming-job.ts",
18146
- "line": 73
18073
+ "line": 74
18147
18074
  },
18148
18075
  "name": "PySparkStreamingJob",
18149
18076
  "properties": [
@@ -18156,7 +18083,7 @@
18156
18083
  "immutable": true,
18157
18084
  "locationInModule": {
18158
18085
  "filename": "lib/jobs/pyspark-streaming-job.ts",
18159
- "line": 76
18086
+ "line": 77
18160
18087
  },
18161
18088
  "name": "PROPERTY_INJECTION_ID",
18162
18089
  "static": true,
@@ -18172,7 +18099,7 @@
18172
18099
  "immutable": true,
18173
18100
  "locationInModule": {
18174
18101
  "filename": "lib/jobs/pyspark-streaming-job.ts",
18175
- "line": 116
18102
+ "line": 117
18176
18103
  },
18177
18104
  "name": "jobArn",
18178
18105
  "overrides": "@aws-cdk/aws-glue-alpha.JobBase",
@@ -18188,7 +18115,7 @@
18188
18115
  "immutable": true,
18189
18116
  "locationInModule": {
18190
18117
  "filename": "lib/jobs/pyspark-streaming-job.ts",
18191
- "line": 121
18118
+ "line": 122
18192
18119
  },
18193
18120
  "name": "jobName",
18194
18121
  "overrides": "@aws-cdk/aws-glue-alpha.JobBase",
@@ -18576,7 +18503,7 @@
18576
18503
  "docs": {
18577
18504
  "stability": "experimental",
18578
18505
  "summary": "Python version.",
18579
- "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\ndeclare const extraPythonFile: glue.Code;\nnew glue.PythonShellJob(stack, 'PythonShellJob', {\n jobName: 'PythonShellJobCustomName',\n description: 'This is a description',\n pythonVersion: glue.PythonVersion.TWO,\n maxCapacity: glue.MaxCapacity.DPU_1,\n role,\n script,\n extraPythonFiles: [extraPythonFile],\n glueVersion: glue.GlueVersion.V2_0,\n continuousLogging: { enabled: false },\n workerType: glue.WorkerType.G_2X,\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n numberOfWorkers: 2,\n maxRetries: 2,\n});",
18506
+ "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\ndeclare const extraPythonFile: glue.Code;\nnew glue.PythonShellJob(stack, 'PythonShellJob', {\n jobName: 'PythonShellJobCustomName',\n description: 'This is a description',\n pythonVersion: glue.PythonVersion.THREE_NINE,\n maxCapacity: glue.MaxCapacity.DPU_1,\n role,\n script,\n extraPythonFiles: [extraPythonFile],\n glueVersion: glue.GlueVersion.V3_0,\n continuousLogging: { enabled: false },\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n maxRetries: 2,\n});",
18580
18507
  "custom": {
18581
18508
  "exampleMetadata": "infused"
18582
18509
  }
@@ -18621,7 +18548,7 @@
18621
18548
  "remarks": "Glue Ray jobs use worker type Z.2X and Glue version 4.0.\nThese are not overrideable since these are the only configuration that\nGlue Ray jobs currently support. The runtime defaults to Ray2.4 and min\nworkers defaults to 3.",
18622
18549
  "stability": "deprecated",
18623
18550
  "summary": "Ray Jobs class.",
18624
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\nimport { aws_logs as logs } from 'aws-cdk-lib';\n\ndeclare const code: glue_alpha.Code;\ndeclare const connection: glue_alpha.Connection;\ndeclare const logGroup: logs.LogGroup;\ndeclare const role: iam.Role;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst rayJob = new glue_alpha.RayJob(this, 'MyRayJob', {\n role: role,\n script: code,\n\n // the properties below are optional\n connections: [connection],\n continuousLogging: {\n enabled: false,\n\n // the properties below are optional\n conversionPattern: 'conversionPattern',\n logGroup: logGroup,\n logStreamPrefix: 'logStreamPrefix',\n quiet: false,\n },\n defaultArguments: {\n defaultArgumentsKey: 'defaultArguments',\n },\n description: 'description',\n enableMetrics: false,\n enableObservabilityMetrics: false,\n enableProfilingMetrics: false,\n glueVersion: glue_alpha.GlueVersion.V0_9,\n jobName: 'jobName',\n jobRunQueuingEnabled: false,\n maxConcurrentRuns: 123,\n maxRetries: 123,\n numberOfWorkers: 123,\n runtime: glue_alpha.Runtime.RAY_TWO_FOUR,\n securityConfiguration: securityConfiguration,\n tags: {\n tagsKey: 'tags',\n },\n timeout: cdk.Duration.minutes(30),\n workerType: glue_alpha.WorkerType.STANDARD,\n});",
18551
+ "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\nimport { aws_logs as logs } from 'aws-cdk-lib';\n\ndeclare const code: glue_alpha.Code;\ndeclare const connection: glue_alpha.Connection;\ndeclare const logGroup: logs.LogGroup;\ndeclare const role: iam.Role;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst rayJob = new glue_alpha.RayJob(this, 'MyRayJob', {\n role: role,\n script: code,\n\n // the properties below are optional\n connections: [connection],\n continuousLogging: {\n enabled: false,\n\n // the properties below are optional\n conversionPattern: 'conversionPattern',\n logGroup: logGroup,\n logStreamPrefix: 'logStreamPrefix',\n quiet: false,\n },\n defaultArguments: {\n defaultArgumentsKey: 'defaultArguments',\n },\n description: 'description',\n enableMetrics: false,\n enableObservabilityMetrics: false,\n glueVersion: glue_alpha.GlueVersion.V0_9,\n jobName: 'jobName',\n jobRunQueuingEnabled: false,\n maxConcurrentRuns: 123,\n maxRetries: 123,\n numberOfWorkers: 123,\n runtime: glue_alpha.Runtime.RAY_TWO_FOUR,\n securityConfiguration: securityConfiguration,\n tags: {\n tagsKey: 'tags',\n },\n timeout: cdk.Duration.minutes(30),\n});",
18625
18552
  "custom": {
18626
18553
  "exampleMetadata": "fixture=_generated"
18627
18554
  }
@@ -18634,7 +18561,7 @@
18634
18561
  },
18635
18562
  "locationInModule": {
18636
18563
  "filename": "lib/jobs/ray-job.ts",
18637
- "line": 81
18564
+ "line": 89
18638
18565
  },
18639
18566
  "parameters": [
18640
18567
  {
@@ -18660,7 +18587,7 @@
18660
18587
  "kind": "class",
18661
18588
  "locationInModule": {
18662
18589
  "filename": "lib/jobs/ray-job.ts",
18663
- "line": 70
18590
+ "line": 78
18664
18591
  },
18665
18592
  "name": "RayJob",
18666
18593
  "properties": [
@@ -18673,7 +18600,7 @@
18673
18600
  "immutable": true,
18674
18601
  "locationInModule": {
18675
18602
  "filename": "lib/jobs/ray-job.ts",
18676
- "line": 73
18603
+ "line": 81
18677
18604
  },
18678
18605
  "name": "PROPERTY_INJECTION_ID",
18679
18606
  "static": true,
@@ -18689,7 +18616,7 @@
18689
18616
  "immutable": true,
18690
18617
  "locationInModule": {
18691
18618
  "filename": "lib/jobs/ray-job.ts",
18692
- "line": 75
18619
+ "line": 83
18693
18620
  },
18694
18621
  "name": "grantPrincipal",
18695
18622
  "overrides": "@aws-cdk/aws-glue-alpha.JobBase",
@@ -18705,7 +18632,7 @@
18705
18632
  "immutable": true,
18706
18633
  "locationInModule": {
18707
18634
  "filename": "lib/jobs/ray-job.ts",
18708
- "line": 134
18635
+ "line": 138
18709
18636
  },
18710
18637
  "name": "jobArn",
18711
18638
  "overrides": "@aws-cdk/aws-glue-alpha.JobBase",
@@ -18721,7 +18648,7 @@
18721
18648
  "immutable": true,
18722
18649
  "locationInModule": {
18723
18650
  "filename": "lib/jobs/ray-job.ts",
18724
- "line": 139
18651
+ "line": 143
18725
18652
  },
18726
18653
  "name": "jobName",
18727
18654
  "overrides": "@aws-cdk/aws-glue-alpha.JobBase",
@@ -18737,7 +18664,7 @@
18737
18664
  "immutable": true,
18738
18665
  "locationInModule": {
18739
18666
  "filename": "lib/jobs/ray-job.ts",
18740
- "line": 74
18667
+ "line": 82
18741
18668
  },
18742
18669
  "name": "role",
18743
18670
  "overrides": "@aws-cdk/aws-glue-alpha.Job",
@@ -18755,7 +18682,7 @@
18755
18682
  "deprecated": "AWS Glue for Ray is closed to new customers as of April 30, 2026.\nMigrate to Amazon EKS with KubeRay Operator. See\nhttps://docs.aws.amazon.com/glue/latest/dg/awsglue-ray-jobs-availability-change.html",
18756
18683
  "stability": "deprecated",
18757
18684
  "summary": "Properties for creating a Ray Glue job.",
18758
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\nimport { aws_logs as logs } from 'aws-cdk-lib';\n\ndeclare const code: glue_alpha.Code;\ndeclare const connection: glue_alpha.Connection;\ndeclare const logGroup: logs.LogGroup;\ndeclare const role: iam.Role;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst rayJobProps: glue_alpha.RayJobProps = {\n role: role,\n script: code,\n\n // the properties below are optional\n connections: [connection],\n continuousLogging: {\n enabled: false,\n\n // the properties below are optional\n conversionPattern: 'conversionPattern',\n logGroup: logGroup,\n logStreamPrefix: 'logStreamPrefix',\n quiet: false,\n },\n defaultArguments: {\n defaultArgumentsKey: 'defaultArguments',\n },\n description: 'description',\n enableMetrics: false,\n enableObservabilityMetrics: false,\n enableProfilingMetrics: false,\n glueVersion: glue_alpha.GlueVersion.V0_9,\n jobName: 'jobName',\n jobRunQueuingEnabled: false,\n maxConcurrentRuns: 123,\n maxRetries: 123,\n numberOfWorkers: 123,\n runtime: glue_alpha.Runtime.RAY_TWO_FOUR,\n securityConfiguration: securityConfiguration,\n tags: {\n tagsKey: 'tags',\n },\n timeout: cdk.Duration.minutes(30),\n workerType: glue_alpha.WorkerType.STANDARD,\n};",
18685
+ "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\nimport { aws_logs as logs } from 'aws-cdk-lib';\n\ndeclare const code: glue_alpha.Code;\ndeclare const connection: glue_alpha.Connection;\ndeclare const logGroup: logs.LogGroup;\ndeclare const role: iam.Role;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst rayJobProps: glue_alpha.RayJobProps = {\n role: role,\n script: code,\n\n // the properties below are optional\n connections: [connection],\n continuousLogging: {\n enabled: false,\n\n // the properties below are optional\n conversionPattern: 'conversionPattern',\n logGroup: logGroup,\n logStreamPrefix: 'logStreamPrefix',\n quiet: false,\n },\n defaultArguments: {\n defaultArgumentsKey: 'defaultArguments',\n },\n description: 'description',\n enableMetrics: false,\n enableObservabilityMetrics: false,\n glueVersion: glue_alpha.GlueVersion.V0_9,\n jobName: 'jobName',\n jobRunQueuingEnabled: false,\n maxConcurrentRuns: 123,\n maxRetries: 123,\n numberOfWorkers: 123,\n runtime: glue_alpha.Runtime.RAY_TWO_FOUR,\n securityConfiguration: securityConfiguration,\n tags: {\n tagsKey: 'tags',\n },\n timeout: cdk.Duration.minutes(30),\n};",
18759
18686
  "custom": {
18760
18687
  "exampleMetadata": "fixture=_generated"
18761
18688
  }
@@ -18767,7 +18694,7 @@
18767
18694
  "kind": "interface",
18768
18695
  "locationInModule": {
18769
18696
  "filename": "lib/jobs/ray-job.ts",
18770
- "line": 19
18697
+ "line": 18
18771
18698
  },
18772
18699
  "name": "RayJobProps",
18773
18700
  "properties": [
@@ -18782,7 +18709,7 @@
18782
18709
  "immutable": true,
18783
18710
  "locationInModule": {
18784
18711
  "filename": "lib/jobs/ray-job.ts",
18785
- "line": 46
18712
+ "line": 54
18786
18713
  },
18787
18714
  "name": "enableMetrics",
18788
18715
  "optional": true,
@@ -18801,7 +18728,7 @@
18801
18728
  "immutable": true,
18802
18729
  "locationInModule": {
18803
18730
  "filename": "lib/jobs/ray-job.ts",
18804
- "line": 55
18731
+ "line": 63
18805
18732
  },
18806
18733
  "name": "enableObservabilityMetrics",
18807
18734
  "optional": true,
@@ -18820,7 +18747,7 @@
18820
18747
  "immutable": true,
18821
18748
  "locationInModule": {
18822
18749
  "filename": "lib/jobs/ray-job.ts",
18823
- "line": 37
18750
+ "line": 45
18824
18751
  },
18825
18752
  "name": "jobRunQueuingEnabled",
18826
18753
  "optional": true,
@@ -18828,6 +18755,25 @@
18828
18755
  "primitive": "boolean"
18829
18756
  }
18830
18757
  },
18758
+ {
18759
+ "abstract": true,
18760
+ "docs": {
18761
+ "default": "3",
18762
+ "remarks": "Ray jobs only support the Z.2X worker type, so the worker type is not configurable.",
18763
+ "stability": "deprecated",
18764
+ "summary": "The number of workers allocated when a job runs."
18765
+ },
18766
+ "immutable": true,
18767
+ "locationInModule": {
18768
+ "filename": "lib/jobs/ray-job.ts",
18769
+ "line": 33
18770
+ },
18771
+ "name": "numberOfWorkers",
18772
+ "optional": true,
18773
+ "type": {
18774
+ "primitive": "number"
18775
+ }
18776
+ },
18831
18777
  {
18832
18778
  "abstract": true,
18833
18779
  "docs": {
@@ -18838,7 +18784,7 @@
18838
18784
  "immutable": true,
18839
18785
  "locationInModule": {
18840
18786
  "filename": "lib/jobs/ray-job.ts",
18841
- "line": 25
18787
+ "line": 24
18842
18788
  },
18843
18789
  "name": "runtime",
18844
18790
  "optional": true,
@@ -18952,58 +18898,73 @@
18952
18898
  },
18953
18899
  "@aws-cdk/aws-glue-alpha.S3Encryption": {
18954
18900
  "assembly": "@aws-cdk/aws-glue-alpha",
18955
- "datatype": true,
18956
18901
  "docs": {
18902
+ "remarks": "Use {@link S3Encryption.s3Managed} for SSE-S3 or {@link S3Encryption.kms} for\nSSE-KMS. Because these are separate factories, a KMS key can never be paired\nwith S3-managed encryption.",
18957
18903
  "stability": "experimental",
18958
- "summary": "S3 encryption configuration.",
18959
- "example": "new glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n },\n jobBookmarksEncryption: {\n mode: glue.JobBookmarksEncryptionMode.CLIENT_SIDE_KMS,\n },\n s3Encryption: {\n mode: glue.S3EncryptionMode.KMS,\n },\n});",
18904
+ "summary": "S3 encryption configuration for a `SecurityConfiguration`.",
18905
+ "example": "new glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: glue.CloudWatchEncryption.kms(),\n jobBookmarksEncryption: glue.JobBookmarksEncryption.clientSideKms(),\n s3Encryption: glue.S3Encryption.kms(),\n});",
18960
18906
  "custom": {
18961
18907
  "exampleMetadata": "infused"
18962
18908
  }
18963
18909
  },
18964
18910
  "fqn": "@aws-cdk/aws-glue-alpha.S3Encryption",
18965
- "kind": "interface",
18911
+ "kind": "class",
18966
18912
  "locationInModule": {
18967
18913
  "filename": "lib/security-configuration.ts",
18968
- "line": 70
18914
+ "line": 52
18969
18915
  },
18970
- "name": "S3Encryption",
18971
- "properties": [
18916
+ "methods": [
18972
18917
  {
18973
- "abstract": true,
18974
18918
  "docs": {
18919
+ "see": "https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingKMSEncryption.html",
18975
18920
  "stability": "experimental",
18976
- "summary": "Encryption mode."
18921
+ "summary": "Server-side encryption (SSE) with an AWS KMS key managed by the account owner."
18977
18922
  },
18978
- "immutable": true,
18979
18923
  "locationInModule": {
18980
18924
  "filename": "lib/security-configuration.ts",
18981
- "line": 74
18925
+ "line": 68
18982
18926
  },
18983
- "name": "mode",
18984
- "type": {
18985
- "fqn": "@aws-cdk/aws-glue-alpha.S3EncryptionMode"
18986
- }
18927
+ "name": "kms",
18928
+ "parameters": [
18929
+ {
18930
+ "docs": {
18931
+ "remarks": "A key is created if one is not provided.",
18932
+ "summary": "the KMS key used to encrypt the data."
18933
+ },
18934
+ "name": "kmsKey",
18935
+ "optional": true,
18936
+ "type": {
18937
+ "fqn": "aws-cdk-lib.interfaces.aws_kms.IKeyRef"
18938
+ }
18939
+ }
18940
+ ],
18941
+ "returns": {
18942
+ "type": {
18943
+ "fqn": "@aws-cdk/aws-glue-alpha.S3Encryption"
18944
+ }
18945
+ },
18946
+ "static": true
18987
18947
  },
18988
18948
  {
18989
- "abstract": true,
18990
18949
  "docs": {
18991
- "default": "no kms key if mode = S3_MANAGED. A key will be created if one is not provided and mode = KMS.",
18950
+ "see": "https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html",
18992
18951
  "stability": "experimental",
18993
- "summary": "The KMS key to be used to encrypt the data."
18952
+ "summary": "Server-side encryption (SSE) with an Amazon S3-managed key."
18994
18953
  },
18995
- "immutable": true,
18996
18954
  "locationInModule": {
18997
18955
  "filename": "lib/security-configuration.ts",
18998
- "line": 80
18956
+ "line": 58
18999
18957
  },
19000
- "name": "kmsKey",
19001
- "optional": true,
19002
- "type": {
19003
- "fqn": "aws-cdk-lib.interfaces.aws_kms.IKeyRef"
19004
- }
18958
+ "name": "s3Managed",
18959
+ "returns": {
18960
+ "type": {
18961
+ "fqn": "@aws-cdk/aws-glue-alpha.S3Encryption"
18962
+ }
18963
+ },
18964
+ "static": true
19005
18965
  }
19006
18966
  ],
18967
+ "name": "S3Encryption",
19007
18968
  "symbolId": "lib/security-configuration:S3Encryption"
19008
18969
  },
19009
18970
  "@aws-cdk/aws-glue-alpha.S3EncryptionMode": {
@@ -19011,11 +18972,7 @@
19011
18972
  "docs": {
19012
18973
  "see": "https://docs.aws.amazon.com/glue/latest/webapi/API_S3Encryption.html#Glue-Type-S3Encryption-S3EncryptionMode",
19013
18974
  "stability": "experimental",
19014
- "summary": "Encryption mode for S3.",
19015
- "example": "new glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n },\n jobBookmarksEncryption: {\n mode: glue.JobBookmarksEncryptionMode.CLIENT_SIDE_KMS,\n },\n s3Encryption: {\n mode: glue.S3EncryptionMode.KMS,\n },\n});",
19016
- "custom": {
19017
- "exampleMetadata": "infused"
19018
- }
18975
+ "summary": "Encryption mode for S3."
19019
18976
  },
19020
18977
  "fqn": "@aws-cdk/aws-glue-alpha.S3EncryptionMode",
19021
18978
  "kind": "enum",
@@ -19054,7 +19011,7 @@
19054
19011
  },
19055
19012
  "stability": "experimental",
19056
19013
  "summary": "A Glue table that targets a S3 dataset.",
19057
- "example": "declare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'data',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'date',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n partitionProjection: {\n date: glue.PartitionProjectionConfiguration.date({\n min: '2020-01-01',\n max: '2023-12-31',\n format: 'yyyy-MM-dd',\n interval: 1, // optional, defaults to 1\n intervalUnit: glue.DateIntervalUnit.DAYS, // optional: YEARS, MONTHS, WEEKS, DAYS, HOURS, MINUTES, SECONDS\n }),\n },\n});"
19014
+ "example": "declare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'data',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'date',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n partitionProjection: {\n date: glue.PartitionProjectionConfiguration.date({\n min: 'NOW-3YEARS',\n max: 'NOW',\n format: 'yyyy-MM-dd',\n }),\n },\n});"
19058
19015
  },
19059
19016
  "fqn": "@aws-cdk/aws-glue-alpha.S3Table",
19060
19017
  "initializer": {
@@ -19063,7 +19020,7 @@
19063
19020
  },
19064
19021
  "locationInModule": {
19065
19022
  "filename": "lib/s3-table.ts",
19066
- "line": 133
19023
+ "line": 134
19067
19024
  },
19068
19025
  "parameters": [
19069
19026
  {
@@ -19089,7 +19046,7 @@
19089
19046
  "kind": "class",
19090
19047
  "locationInModule": {
19091
19048
  "filename": "lib/s3-table.ts",
19092
- "line": 92
19049
+ "line": 93
19093
19050
  },
19094
19051
  "methods": [
19095
19052
  {
@@ -19098,7 +19055,7 @@
19098
19055
  },
19099
19056
  "locationInModule": {
19100
19057
  "filename": "lib/s3-table.ts",
19101
- "line": 257
19058
+ "line": 260
19102
19059
  },
19103
19060
  "name": "generateS3PrefixForGrant",
19104
19061
  "protected": true,
@@ -19116,7 +19073,7 @@
19116
19073
  },
19117
19074
  "locationInModule": {
19118
19075
  "filename": "lib/s3-table.ts",
19119
- "line": 221
19076
+ "line": 224
19120
19077
  },
19121
19078
  "name": "grantRead",
19122
19079
  "overrides": "@aws-cdk/aws-glue-alpha.TableBase",
@@ -19145,7 +19102,7 @@
19145
19102
  },
19146
19103
  "locationInModule": {
19147
19104
  "filename": "lib/s3-table.ts",
19148
- "line": 249
19105
+ "line": 252
19149
19106
  },
19150
19107
  "name": "grantReadWrite",
19151
19108
  "overrides": "@aws-cdk/aws-glue-alpha.TableBase",
@@ -19174,7 +19131,7 @@
19174
19131
  },
19175
19132
  "locationInModule": {
19176
19133
  "filename": "lib/s3-table.ts",
19177
- "line": 235
19134
+ "line": 238
19178
19135
  },
19179
19136
  "name": "grantWrite",
19180
19137
  "overrides": "@aws-cdk/aws-glue-alpha.TableBase",
@@ -19207,7 +19164,7 @@
19207
19164
  "immutable": true,
19208
19165
  "locationInModule": {
19209
19166
  "filename": "lib/s3-table.ts",
19210
- "line": 95
19167
+ "line": 96
19211
19168
  },
19212
19169
  "name": "PROPERTY_INJECTION_ID",
19213
19170
  "static": true,
@@ -19223,7 +19180,7 @@
19223
19180
  "immutable": true,
19224
19181
  "locationInModule": {
19225
19182
  "filename": "lib/s3-table.ts",
19226
- "line": 102
19183
+ "line": 103
19227
19184
  },
19228
19185
  "name": "bucket",
19229
19186
  "type": {
@@ -19238,7 +19195,7 @@
19238
19195
  "immutable": true,
19239
19196
  "locationInModule": {
19240
19197
  "filename": "lib/s3-table.ts",
19241
- "line": 112
19198
+ "line": 113
19242
19199
  },
19243
19200
  "name": "encryption",
19244
19201
  "type": {
@@ -19253,7 +19210,7 @@
19253
19210
  "immutable": true,
19254
19211
  "locationInModule": {
19255
19212
  "filename": "lib/s3-table.ts",
19256
- "line": 107
19213
+ "line": 108
19257
19214
  },
19258
19215
  "name": "s3Prefix",
19259
19216
  "type": {
@@ -19268,7 +19225,7 @@
19268
19225
  "immutable": true,
19269
19226
  "locationInModule": {
19270
19227
  "filename": "lib/s3-table.ts",
19271
- "line": 206
19228
+ "line": 209
19272
19229
  },
19273
19230
  "name": "tableArn",
19274
19231
  "overrides": "@aws-cdk/aws-glue-alpha.TableBase",
@@ -19284,7 +19241,7 @@
19284
19241
  "immutable": true,
19285
19242
  "locationInModule": {
19286
19243
  "filename": "lib/s3-table.ts",
19287
- "line": 198
19244
+ "line": 201
19288
19245
  },
19289
19246
  "name": "tableName",
19290
19247
  "overrides": "@aws-cdk/aws-glue-alpha.TableBase",
@@ -19299,7 +19256,7 @@
19299
19256
  "immutable": true,
19300
19257
  "locationInModule": {
19301
19258
  "filename": "lib/s3-table.ts",
19302
- "line": 124
19259
+ "line": 125
19303
19260
  },
19304
19261
  "name": "tableResource",
19305
19262
  "overrides": "@aws-cdk/aws-glue-alpha.TableBase",
@@ -19317,7 +19274,7 @@
19317
19274
  "immutable": true,
19318
19275
  "locationInModule": {
19319
19276
  "filename": "lib/s3-table.ts",
19320
- "line": 117
19277
+ "line": 118
19321
19278
  },
19322
19279
  "name": "encryptionKey",
19323
19280
  "optional": true,
@@ -19333,7 +19290,7 @@
19333
19290
  "immutable": true,
19334
19291
  "locationInModule": {
19335
19292
  "filename": "lib/s3-table.ts",
19336
- "line": 122
19293
+ "line": 123
19337
19294
  },
19338
19295
  "name": "partitionIndexes",
19339
19296
  "optional": true,
@@ -19393,14 +19350,14 @@
19393
19350
  "abstract": true,
19394
19351
  "docs": {
19395
19352
  "default": "BucketEncryption.S3_MANAGED",
19396
- "remarks": "You can only provide this option if you are not explicitly passing in a bucket.\n\nIf you choose `SSE-KMS`, you *can* provide an un-managed KMS key with `encryptionKey`.\nIf you choose `CSE-KMS`, you *must* provide an un-managed KMS key with `encryptionKey`.",
19353
+ "remarks": "You can only provide this option if you are not explicitly passing in a bucket.\n\nIf you choose `SSE-KMS`, you *can* provide an un-managed KMS key with `encryptionKey`.\nIf you choose `CSE-KMS`, you *may* provide an un-managed KMS key with `encryptionKey`;\none is created automatically if omitted.",
19397
19354
  "stability": "experimental",
19398
19355
  "summary": "The kind of encryption to secure the data with."
19399
19356
  },
19400
19357
  "immutable": true,
19401
19358
  "locationInModule": {
19402
19359
  "filename": "lib/s3-table.ts",
19403
- "line": 76
19360
+ "line": 77
19404
19361
  },
19405
19362
  "name": "encryption",
19406
19363
  "optional": true,
@@ -19419,7 +19376,7 @@
19419
19376
  "immutable": true,
19420
19377
  "locationInModule": {
19421
19378
  "filename": "lib/s3-table.ts",
19422
- "line": 85
19379
+ "line": 86
19423
19380
  },
19424
19381
  "name": "encryptionKey",
19425
19382
  "optional": true,
@@ -19456,7 +19413,7 @@
19456
19413
  "remarks": "ETL jobs support pySpark and Scala languages, for which there are separate\nbut similar constructors. ETL jobs default to the G2 worker type, but you\ncan override this default with other supported worker type values\n(G1, G2, G4 and G8). ETL jobs defaults to Glue version 4.0, which you can\noverride to 3.0. The following ETL features are enabled by default:\n—enable-metrics, —enable-spark-ui, —enable-continuous-cloudwatch-log.\nYou can find more details about version, worker type and other features\nin Glue's public documentation.",
19457
19414
  "stability": "experimental",
19458
19415
  "summary": "Spark ETL Jobs class.",
19459
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\nimport { aws_logs as logs } from 'aws-cdk-lib';\nimport { aws_s3 as s3 } from 'aws-cdk-lib';\n\ndeclare const bucket: s3.Bucket;\ndeclare const code: glue_alpha.Code;\ndeclare const connection: glue_alpha.Connection;\ndeclare const logGroup: logs.LogGroup;\ndeclare const role: iam.Role;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst scalaSparkEtlJob = new glue_alpha.ScalaSparkEtlJob(this, 'MyScalaSparkEtlJob', {\n className: 'className',\n role: role,\n script: code,\n\n // the properties below are optional\n connections: [connection],\n continuousLogging: {\n enabled: false,\n\n // the properties below are optional\n conversionPattern: 'conversionPattern',\n logGroup: logGroup,\n logStreamPrefix: 'logStreamPrefix',\n quiet: false,\n },\n defaultArguments: {\n defaultArgumentsKey: 'defaultArguments',\n },\n description: 'description',\n enableMetrics: false,\n enableObservabilityMetrics: false,\n enableProfilingMetrics: false,\n extraFiles: [code],\n extraJars: [code],\n extraJarsFirst: false,\n glueVersion: glue_alpha.GlueVersion.V0_9,\n jobName: 'jobName',\n jobRunQueuingEnabled: false,\n maxConcurrentRuns: 123,\n maxRetries: 123,\n notifyDelayAfter: cdk.Duration.minutes(30),\n numberOfWorkers: 123,\n securityConfiguration: securityConfiguration,\n sparkUI: {\n bucket: bucket,\n prefix: 'prefix',\n },\n tags: {\n tagsKey: 'tags',\n },\n timeout: cdk.Duration.minutes(30),\n workerType: glue_alpha.WorkerType.STANDARD,\n});",
19416
+ "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\nimport { aws_logs as logs } from 'aws-cdk-lib';\nimport { aws_s3 as s3 } from 'aws-cdk-lib';\n\ndeclare const bucket: s3.Bucket;\ndeclare const code: glue_alpha.Code;\ndeclare const connection: glue_alpha.Connection;\ndeclare const logGroup: logs.LogGroup;\ndeclare const role: iam.Role;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst scalaSparkEtlJob = new glue_alpha.ScalaSparkEtlJob(this, 'MyScalaSparkEtlJob', {\n className: 'className',\n role: role,\n script: code,\n\n // the properties below are optional\n connections: [connection],\n continuousLogging: {\n enabled: false,\n\n // the properties below are optional\n conversionPattern: 'conversionPattern',\n logGroup: logGroup,\n logStreamPrefix: 'logStreamPrefix',\n quiet: false,\n },\n defaultArguments: {\n defaultArgumentsKey: 'defaultArguments',\n },\n description: 'description',\n enableMetrics: false,\n enableObservabilityMetrics: false,\n extraFiles: [code],\n extraJars: [code],\n extraJarsFirst: false,\n glueVersion: glue_alpha.GlueVersion.V0_9,\n jobName: 'jobName',\n jobRunQueuingEnabled: false,\n maxConcurrentRuns: 123,\n maxRetries: 123,\n notifyDelayAfter: cdk.Duration.minutes(30),\n securityConfiguration: securityConfiguration,\n sparkUI: {\n bucket: bucket,\n prefix: 'prefix',\n },\n tags: {\n tagsKey: 'tags',\n },\n timeout: cdk.Duration.minutes(30),\n workerConfiguration: {\n numberOfWorkers: 123,\n workerType: glue_alpha.WorkerType.STANDARD,\n },\n});",
19460
19417
  "custom": {
19461
19418
  "exampleMetadata": "fixture=_generated"
19462
19419
  }
@@ -19469,7 +19426,7 @@
19469
19426
  },
19470
19427
  "locationInModule": {
19471
19428
  "filename": "lib/jobs/scala-spark-etl-job.ts",
19472
- "line": 90
19429
+ "line": 89
19473
19430
  },
19474
19431
  "parameters": [
19475
19432
  {
@@ -19495,7 +19452,7 @@
19495
19452
  "kind": "class",
19496
19453
  "locationInModule": {
19497
19454
  "filename": "lib/jobs/scala-spark-etl-job.ts",
19498
- "line": 81
19455
+ "line": 80
19499
19456
  },
19500
19457
  "name": "ScalaSparkEtlJob",
19501
19458
  "properties": [
@@ -19508,7 +19465,7 @@
19508
19465
  "immutable": true,
19509
19466
  "locationInModule": {
19510
19467
  "filename": "lib/jobs/scala-spark-etl-job.ts",
19511
- "line": 84
19468
+ "line": 83
19512
19469
  },
19513
19470
  "name": "PROPERTY_INJECTION_ID",
19514
19471
  "static": true,
@@ -19524,7 +19481,7 @@
19524
19481
  "immutable": true,
19525
19482
  "locationInModule": {
19526
19483
  "filename": "lib/jobs/scala-spark-etl-job.ts",
19527
- "line": 128
19484
+ "line": 123
19528
19485
  },
19529
19486
  "name": "jobArn",
19530
19487
  "overrides": "@aws-cdk/aws-glue-alpha.JobBase",
@@ -19540,7 +19497,7 @@
19540
19497
  "immutable": true,
19541
19498
  "locationInModule": {
19542
19499
  "filename": "lib/jobs/scala-spark-etl-job.ts",
19543
- "line": 133
19500
+ "line": 128
19544
19501
  },
19545
19502
  "name": "jobName",
19546
19503
  "overrides": "@aws-cdk/aws-glue-alpha.JobBase",
@@ -19557,7 +19514,7 @@
19557
19514
  "docs": {
19558
19515
  "stability": "experimental",
19559
19516
  "summary": "Properties for creating a Scala Spark ETL job.",
19560
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\nimport { aws_logs as logs } from 'aws-cdk-lib';\nimport { aws_s3 as s3 } from 'aws-cdk-lib';\n\ndeclare const bucket: s3.Bucket;\ndeclare const code: glue_alpha.Code;\ndeclare const connection: glue_alpha.Connection;\ndeclare const logGroup: logs.LogGroup;\ndeclare const role: iam.Role;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst scalaSparkEtlJobProps: glue_alpha.ScalaSparkEtlJobProps = {\n className: 'className',\n role: role,\n script: code,\n\n // the properties below are optional\n connections: [connection],\n continuousLogging: {\n enabled: false,\n\n // the properties below are optional\n conversionPattern: 'conversionPattern',\n logGroup: logGroup,\n logStreamPrefix: 'logStreamPrefix',\n quiet: false,\n },\n defaultArguments: {\n defaultArgumentsKey: 'defaultArguments',\n },\n description: 'description',\n enableMetrics: false,\n enableObservabilityMetrics: false,\n enableProfilingMetrics: false,\n extraFiles: [code],\n extraJars: [code],\n extraJarsFirst: false,\n glueVersion: glue_alpha.GlueVersion.V0_9,\n jobName: 'jobName',\n jobRunQueuingEnabled: false,\n maxConcurrentRuns: 123,\n maxRetries: 123,\n notifyDelayAfter: cdk.Duration.minutes(30),\n numberOfWorkers: 123,\n securityConfiguration: securityConfiguration,\n sparkUI: {\n bucket: bucket,\n prefix: 'prefix',\n },\n tags: {\n tagsKey: 'tags',\n },\n timeout: cdk.Duration.minutes(30),\n workerType: glue_alpha.WorkerType.STANDARD,\n};",
19517
+ "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\nimport { aws_logs as logs } from 'aws-cdk-lib';\nimport { aws_s3 as s3 } from 'aws-cdk-lib';\n\ndeclare const bucket: s3.Bucket;\ndeclare const code: glue_alpha.Code;\ndeclare const connection: glue_alpha.Connection;\ndeclare const logGroup: logs.LogGroup;\ndeclare const role: iam.Role;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst scalaSparkEtlJobProps: glue_alpha.ScalaSparkEtlJobProps = {\n className: 'className',\n role: role,\n script: code,\n\n // the properties below are optional\n connections: [connection],\n continuousLogging: {\n enabled: false,\n\n // the properties below are optional\n conversionPattern: 'conversionPattern',\n logGroup: logGroup,\n logStreamPrefix: 'logStreamPrefix',\n quiet: false,\n },\n defaultArguments: {\n defaultArgumentsKey: 'defaultArguments',\n },\n description: 'description',\n enableMetrics: false,\n enableObservabilityMetrics: false,\n extraFiles: [code],\n extraJars: [code],\n extraJarsFirst: false,\n glueVersion: glue_alpha.GlueVersion.V0_9,\n jobName: 'jobName',\n jobRunQueuingEnabled: false,\n maxConcurrentRuns: 123,\n maxRetries: 123,\n notifyDelayAfter: cdk.Duration.minutes(30),\n securityConfiguration: securityConfiguration,\n sparkUI: {\n bucket: bucket,\n prefix: 'prefix',\n },\n tags: {\n tagsKey: 'tags',\n },\n timeout: cdk.Duration.minutes(30),\n workerConfiguration: {\n numberOfWorkers: 123,\n workerType: glue_alpha.WorkerType.STANDARD,\n },\n};",
19561
19518
  "custom": {
19562
19519
  "exampleMetadata": "fixture=_generated"
19563
19520
  }
@@ -19569,7 +19526,7 @@
19569
19526
  "kind": "interface",
19570
19527
  "locationInModule": {
19571
19528
  "filename": "lib/jobs/scala-spark-etl-job.ts",
19572
- "line": 16
19529
+ "line": 15
19573
19530
  },
19574
19531
  "name": "ScalaSparkEtlJobProps",
19575
19532
  "properties": [
@@ -19582,7 +19539,7 @@
19582
19539
  "immutable": true,
19583
19540
  "locationInModule": {
19584
19541
  "filename": "lib/jobs/scala-spark-etl-job.ts",
19585
- "line": 22
19542
+ "line": 21
19586
19543
  },
19587
19544
  "name": "className",
19588
19545
  "type": {
@@ -19600,7 +19557,7 @@
19600
19557
  "immutable": true,
19601
19558
  "locationInModule": {
19602
19559
  "filename": "lib/jobs/scala-spark-etl-job.ts",
19603
- "line": 38
19560
+ "line": 37
19604
19561
  },
19605
19562
  "name": "extraFiles",
19606
19563
  "optional": true,
@@ -19623,7 +19580,7 @@
19623
19580
  "immutable": true,
19624
19581
  "locationInModule": {
19625
19582
  "filename": "lib/jobs/scala-spark-etl-job.ts",
19626
- "line": 45
19583
+ "line": 44
19627
19584
  },
19628
19585
  "name": "extraJars",
19629
19586
  "optional": true,
@@ -19647,7 +19604,7 @@
19647
19604
  "immutable": true,
19648
19605
  "locationInModule": {
19649
19606
  "filename": "lib/jobs/scala-spark-etl-job.ts",
19650
- "line": 54
19607
+ "line": 53
19651
19608
  },
19652
19609
  "name": "extraJarsFirst",
19653
19610
  "optional": true,
@@ -19666,7 +19623,7 @@
19666
19623
  "immutable": true,
19667
19624
  "locationInModule": {
19668
19625
  "filename": "lib/jobs/scala-spark-etl-job.ts",
19669
- "line": 66
19626
+ "line": 65
19670
19627
  },
19671
19628
  "name": "jobRunQueuingEnabled",
19672
19629
  "optional": true,
@@ -19685,7 +19642,7 @@
19685
19642
  "immutable": true,
19686
19643
  "locationInModule": {
19687
19644
  "filename": "lib/jobs/scala-spark-etl-job.ts",
19688
- "line": 29
19645
+ "line": 28
19689
19646
  },
19690
19647
  "name": "notifyDelayAfter",
19691
19648
  "optional": true,
@@ -19703,7 +19660,7 @@
19703
19660
  "remarks": "ETL jobs support pySpark and Scala languages, for which there are separate\nbut similar constructors. ETL jobs default to the G2 worker type, but you\ncan override this default with other supported worker type values\n(G1, G2, G4 and G8). ETL jobs defaults to Glue version 4.0, which you can\noverride to 3.0. The following ETL features are enabled by default:\n—enable-metrics, —enable-spark-ui, —enable-continuous-cloudwatch-log.\nYou can find more details about version, worker type and other features\nin Glue's public documentation.",
19704
19661
  "stability": "experimental",
19705
19662
  "summary": "Spark ETL Jobs class.",
19706
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\nimport { aws_logs as logs } from 'aws-cdk-lib';\nimport { aws_s3 as s3 } from 'aws-cdk-lib';\n\ndeclare const bucket: s3.Bucket;\ndeclare const code: glue_alpha.Code;\ndeclare const connection: glue_alpha.Connection;\ndeclare const logGroup: logs.LogGroup;\ndeclare const role: iam.Role;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst scalaSparkFlexEtlJob = new glue_alpha.ScalaSparkFlexEtlJob(this, 'MyScalaSparkFlexEtlJob', {\n className: 'className',\n role: role,\n script: code,\n\n // the properties below are optional\n connections: [connection],\n continuousLogging: {\n enabled: false,\n\n // the properties below are optional\n conversionPattern: 'conversionPattern',\n logGroup: logGroup,\n logStreamPrefix: 'logStreamPrefix',\n quiet: false,\n },\n defaultArguments: {\n defaultArgumentsKey: 'defaultArguments',\n },\n description: 'description',\n enableMetrics: false,\n enableObservabilityMetrics: false,\n enableProfilingMetrics: false,\n extraFiles: [code],\n extraJars: [code],\n extraJarsFirst: false,\n glueVersion: glue_alpha.GlueVersion.V0_9,\n jobName: 'jobName',\n maxConcurrentRuns: 123,\n maxRetries: 123,\n notifyDelayAfter: cdk.Duration.minutes(30),\n numberOfWorkers: 123,\n securityConfiguration: securityConfiguration,\n sparkUI: {\n bucket: bucket,\n prefix: 'prefix',\n },\n tags: {\n tagsKey: 'tags',\n },\n timeout: cdk.Duration.minutes(30),\n workerType: glue_alpha.WorkerType.STANDARD,\n});",
19663
+ "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\nimport { aws_logs as logs } from 'aws-cdk-lib';\nimport { aws_s3 as s3 } from 'aws-cdk-lib';\n\ndeclare const bucket: s3.Bucket;\ndeclare const code: glue_alpha.Code;\ndeclare const connection: glue_alpha.Connection;\ndeclare const logGroup: logs.LogGroup;\ndeclare const role: iam.Role;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst scalaSparkFlexEtlJob = new glue_alpha.ScalaSparkFlexEtlJob(this, 'MyScalaSparkFlexEtlJob', {\n className: 'className',\n role: role,\n script: code,\n\n // the properties below are optional\n connections: [connection],\n continuousLogging: {\n enabled: false,\n\n // the properties below are optional\n conversionPattern: 'conversionPattern',\n logGroup: logGroup,\n logStreamPrefix: 'logStreamPrefix',\n quiet: false,\n },\n defaultArguments: {\n defaultArgumentsKey: 'defaultArguments',\n },\n description: 'description',\n enableMetrics: false,\n enableObservabilityMetrics: false,\n extraFiles: [code],\n extraJars: [code],\n extraJarsFirst: false,\n glueVersion: glue_alpha.GlueVersion.V0_9,\n jobName: 'jobName',\n maxConcurrentRuns: 123,\n maxRetries: 123,\n notifyDelayAfter: cdk.Duration.minutes(30),\n securityConfiguration: securityConfiguration,\n sparkUI: {\n bucket: bucket,\n prefix: 'prefix',\n },\n tags: {\n tagsKey: 'tags',\n },\n timeout: cdk.Duration.minutes(30),\n workerConfiguration: {\n numberOfWorkers: 123,\n workerType: glue_alpha.WorkerType.STANDARD,\n },\n});",
19707
19664
  "custom": {
19708
19665
  "exampleMetadata": "fixture=_generated"
19709
19666
  }
@@ -19805,7 +19762,7 @@
19805
19762
  "remarks": "Flex jobs supports Python and Scala language.\nThe flexible execution class is appropriate for non-urgent jobs such as\npre-production jobs, testing, and one-time data loads.\nFlexible job runs are supported for jobs using AWS Glue version 3.0 or later and G.1X or\nG.2X worker types but will default to the latest version of Glue (currently Glue 3.0.)\n\nSimilar to ETL, we’ll enable these features: —enable-metrics, —enable-spark-ui,\n—enable-continuous-cloudwatch-log",
19806
19763
  "stability": "experimental",
19807
19764
  "summary": "Flex Jobs class.",
19808
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\nimport { aws_logs as logs } from 'aws-cdk-lib';\nimport { aws_s3 as s3 } from 'aws-cdk-lib';\n\ndeclare const bucket: s3.Bucket;\ndeclare const code: glue_alpha.Code;\ndeclare const connection: glue_alpha.Connection;\ndeclare const logGroup: logs.LogGroup;\ndeclare const role: iam.Role;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst scalaSparkFlexEtlJobProps: glue_alpha.ScalaSparkFlexEtlJobProps = {\n className: 'className',\n role: role,\n script: code,\n\n // the properties below are optional\n connections: [connection],\n continuousLogging: {\n enabled: false,\n\n // the properties below are optional\n conversionPattern: 'conversionPattern',\n logGroup: logGroup,\n logStreamPrefix: 'logStreamPrefix',\n quiet: false,\n },\n defaultArguments: {\n defaultArgumentsKey: 'defaultArguments',\n },\n description: 'description',\n enableMetrics: false,\n enableObservabilityMetrics: false,\n enableProfilingMetrics: false,\n extraFiles: [code],\n extraJars: [code],\n extraJarsFirst: false,\n glueVersion: glue_alpha.GlueVersion.V0_9,\n jobName: 'jobName',\n maxConcurrentRuns: 123,\n maxRetries: 123,\n notifyDelayAfter: cdk.Duration.minutes(30),\n numberOfWorkers: 123,\n securityConfiguration: securityConfiguration,\n sparkUI: {\n bucket: bucket,\n prefix: 'prefix',\n },\n tags: {\n tagsKey: 'tags',\n },\n timeout: cdk.Duration.minutes(30),\n workerType: glue_alpha.WorkerType.STANDARD,\n};",
19765
+ "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\nimport { aws_logs as logs } from 'aws-cdk-lib';\nimport { aws_s3 as s3 } from 'aws-cdk-lib';\n\ndeclare const bucket: s3.Bucket;\ndeclare const code: glue_alpha.Code;\ndeclare const connection: glue_alpha.Connection;\ndeclare const logGroup: logs.LogGroup;\ndeclare const role: iam.Role;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst scalaSparkFlexEtlJobProps: glue_alpha.ScalaSparkFlexEtlJobProps = {\n className: 'className',\n role: role,\n script: code,\n\n // the properties below are optional\n connections: [connection],\n continuousLogging: {\n enabled: false,\n\n // the properties below are optional\n conversionPattern: 'conversionPattern',\n logGroup: logGroup,\n logStreamPrefix: 'logStreamPrefix',\n quiet: false,\n },\n defaultArguments: {\n defaultArgumentsKey: 'defaultArguments',\n },\n description: 'description',\n enableMetrics: false,\n enableObservabilityMetrics: false,\n extraFiles: [code],\n extraJars: [code],\n extraJarsFirst: false,\n glueVersion: glue_alpha.GlueVersion.V0_9,\n jobName: 'jobName',\n maxConcurrentRuns: 123,\n maxRetries: 123,\n notifyDelayAfter: cdk.Duration.minutes(30),\n securityConfiguration: securityConfiguration,\n sparkUI: {\n bucket: bucket,\n prefix: 'prefix',\n },\n tags: {\n tagsKey: 'tags',\n },\n timeout: cdk.Duration.minutes(30),\n workerConfiguration: {\n numberOfWorkers: 123,\n workerType: glue_alpha.WorkerType.STANDARD,\n },\n};",
19809
19766
  "custom": {
19810
19767
  "exampleMetadata": "fixture=_generated"
19811
19768
  }
@@ -19933,7 +19890,7 @@
19933
19890
  "remarks": "A Streaming job is similar to an ETL job, except that it performs ETL on data streams\nusing the Apache Spark Structured Streaming framework.\nThese jobs will default to use Python 3.9.\n\nSimilar to ETL jobs, streaming job supports Scala and Python languages. Similar to ETL,\nit supports G1 and G2 worker type and 2.0, 3.0 and 4.0 version. We’ll default to G2 worker\nand 4.0 version for streaming jobs which developers can override.\nWe will enable —enable-metrics, —enable-spark-ui, —enable-continuous-cloudwatch-log.",
19934
19891
  "stability": "experimental",
19935
19892
  "summary": "Scala Streaming Jobs class.",
19936
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\nimport { aws_logs as logs } from 'aws-cdk-lib';\nimport { aws_s3 as s3 } from 'aws-cdk-lib';\n\ndeclare const bucket: s3.Bucket;\ndeclare const code: glue_alpha.Code;\ndeclare const connection: glue_alpha.Connection;\ndeclare const logGroup: logs.LogGroup;\ndeclare const role: iam.Role;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst scalaSparkStreamingJob = new glue_alpha.ScalaSparkStreamingJob(this, 'MyScalaSparkStreamingJob', {\n className: 'className',\n role: role,\n script: code,\n\n // the properties below are optional\n connections: [connection],\n continuousLogging: {\n enabled: false,\n\n // the properties below are optional\n conversionPattern: 'conversionPattern',\n logGroup: logGroup,\n logStreamPrefix: 'logStreamPrefix',\n quiet: false,\n },\n defaultArguments: {\n defaultArgumentsKey: 'defaultArguments',\n },\n description: 'description',\n enableMetrics: false,\n enableObservabilityMetrics: false,\n enableProfilingMetrics: false,\n extraFiles: [code],\n extraJars: [code],\n extraJarsFirst: false,\n glueVersion: glue_alpha.GlueVersion.V0_9,\n jobName: 'jobName',\n jobRunQueuingEnabled: false,\n maxConcurrentRuns: 123,\n maxRetries: 123,\n numberOfWorkers: 123,\n securityConfiguration: securityConfiguration,\n sparkUI: {\n bucket: bucket,\n prefix: 'prefix',\n },\n tags: {\n tagsKey: 'tags',\n },\n timeout: cdk.Duration.minutes(30),\n workerType: glue_alpha.WorkerType.STANDARD,\n});",
19893
+ "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\nimport { aws_logs as logs } from 'aws-cdk-lib';\nimport { aws_s3 as s3 } from 'aws-cdk-lib';\n\ndeclare const bucket: s3.Bucket;\ndeclare const code: glue_alpha.Code;\ndeclare const connection: glue_alpha.Connection;\ndeclare const logGroup: logs.LogGroup;\ndeclare const role: iam.Role;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst scalaSparkStreamingJob = new glue_alpha.ScalaSparkStreamingJob(this, 'MyScalaSparkStreamingJob', {\n className: 'className',\n role: role,\n script: code,\n\n // the properties below are optional\n connections: [connection],\n continuousLogging: {\n enabled: false,\n\n // the properties below are optional\n conversionPattern: 'conversionPattern',\n logGroup: logGroup,\n logStreamPrefix: 'logStreamPrefix',\n quiet: false,\n },\n defaultArguments: {\n defaultArgumentsKey: 'defaultArguments',\n },\n description: 'description',\n enableMetrics: false,\n enableObservabilityMetrics: false,\n extraFiles: [code],\n extraJars: [code],\n extraJarsFirst: false,\n glueVersion: glue_alpha.GlueVersion.V0_9,\n jobName: 'jobName',\n jobRunQueuingEnabled: false,\n maxConcurrentRuns: 123,\n maxRetries: 123,\n securityConfiguration: securityConfiguration,\n sparkUI: {\n bucket: bucket,\n prefix: 'prefix',\n },\n tags: {\n tagsKey: 'tags',\n },\n timeout: cdk.Duration.minutes(30),\n workerConfiguration: {\n numberOfWorkers: 123,\n workerType: glue_alpha.WorkerType.STANDARD,\n },\n});",
19937
19894
  "custom": {
19938
19895
  "exampleMetadata": "fixture=_generated"
19939
19896
  }
@@ -19946,7 +19903,7 @@
19946
19903
  },
19947
19904
  "locationInModule": {
19948
19905
  "filename": "lib/jobs/scala-spark-streaming-job.ts",
19949
- "line": 82
19906
+ "line": 81
19950
19907
  },
19951
19908
  "parameters": [
19952
19909
  {
@@ -19972,7 +19929,7 @@
19972
19929
  "kind": "class",
19973
19930
  "locationInModule": {
19974
19931
  "filename": "lib/jobs/scala-spark-streaming-job.ts",
19975
- "line": 73
19932
+ "line": 72
19976
19933
  },
19977
19934
  "name": "ScalaSparkStreamingJob",
19978
19935
  "properties": [
@@ -19985,7 +19942,7 @@
19985
19942
  "immutable": true,
19986
19943
  "locationInModule": {
19987
19944
  "filename": "lib/jobs/scala-spark-streaming-job.ts",
19988
- "line": 76
19945
+ "line": 75
19989
19946
  },
19990
19947
  "name": "PROPERTY_INJECTION_ID",
19991
19948
  "static": true,
@@ -20001,7 +19958,7 @@
20001
19958
  "immutable": true,
20002
19959
  "locationInModule": {
20003
19960
  "filename": "lib/jobs/scala-spark-streaming-job.ts",
20004
- "line": 119
19961
+ "line": 114
20005
19962
  },
20006
19963
  "name": "jobArn",
20007
19964
  "overrides": "@aws-cdk/aws-glue-alpha.JobBase",
@@ -20017,7 +19974,7 @@
20017
19974
  "immutable": true,
20018
19975
  "locationInModule": {
20019
19976
  "filename": "lib/jobs/scala-spark-streaming-job.ts",
20020
- "line": 124
19977
+ "line": 119
20021
19978
  },
20022
19979
  "name": "jobName",
20023
19980
  "overrides": "@aws-cdk/aws-glue-alpha.JobBase",
@@ -20034,7 +19991,7 @@
20034
19991
  "docs": {
20035
19992
  "stability": "experimental",
20036
19993
  "summary": "Properties for creating a Scala Spark ETL job.",
20037
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\nimport { aws_logs as logs } from 'aws-cdk-lib';\nimport { aws_s3 as s3 } from 'aws-cdk-lib';\n\ndeclare const bucket: s3.Bucket;\ndeclare const code: glue_alpha.Code;\ndeclare const connection: glue_alpha.Connection;\ndeclare const logGroup: logs.LogGroup;\ndeclare const role: iam.Role;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst scalaSparkStreamingJobProps: glue_alpha.ScalaSparkStreamingJobProps = {\n className: 'className',\n role: role,\n script: code,\n\n // the properties below are optional\n connections: [connection],\n continuousLogging: {\n enabled: false,\n\n // the properties below are optional\n conversionPattern: 'conversionPattern',\n logGroup: logGroup,\n logStreamPrefix: 'logStreamPrefix',\n quiet: false,\n },\n defaultArguments: {\n defaultArgumentsKey: 'defaultArguments',\n },\n description: 'description',\n enableMetrics: false,\n enableObservabilityMetrics: false,\n enableProfilingMetrics: false,\n extraFiles: [code],\n extraJars: [code],\n extraJarsFirst: false,\n glueVersion: glue_alpha.GlueVersion.V0_9,\n jobName: 'jobName',\n jobRunQueuingEnabled: false,\n maxConcurrentRuns: 123,\n maxRetries: 123,\n numberOfWorkers: 123,\n securityConfiguration: securityConfiguration,\n sparkUI: {\n bucket: bucket,\n prefix: 'prefix',\n },\n tags: {\n tagsKey: 'tags',\n },\n timeout: cdk.Duration.minutes(30),\n workerType: glue_alpha.WorkerType.STANDARD,\n};",
19994
+ "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\nimport { aws_logs as logs } from 'aws-cdk-lib';\nimport { aws_s3 as s3 } from 'aws-cdk-lib';\n\ndeclare const bucket: s3.Bucket;\ndeclare const code: glue_alpha.Code;\ndeclare const connection: glue_alpha.Connection;\ndeclare const logGroup: logs.LogGroup;\ndeclare const role: iam.Role;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst scalaSparkStreamingJobProps: glue_alpha.ScalaSparkStreamingJobProps = {\n className: 'className',\n role: role,\n script: code,\n\n // the properties below are optional\n connections: [connection],\n continuousLogging: {\n enabled: false,\n\n // the properties below are optional\n conversionPattern: 'conversionPattern',\n logGroup: logGroup,\n logStreamPrefix: 'logStreamPrefix',\n quiet: false,\n },\n defaultArguments: {\n defaultArgumentsKey: 'defaultArguments',\n },\n description: 'description',\n enableMetrics: false,\n enableObservabilityMetrics: false,\n extraFiles: [code],\n extraJars: [code],\n extraJarsFirst: false,\n glueVersion: glue_alpha.GlueVersion.V0_9,\n jobName: 'jobName',\n jobRunQueuingEnabled: false,\n maxConcurrentRuns: 123,\n maxRetries: 123,\n securityConfiguration: securityConfiguration,\n sparkUI: {\n bucket: bucket,\n prefix: 'prefix',\n },\n tags: {\n tagsKey: 'tags',\n },\n timeout: cdk.Duration.minutes(30),\n workerConfiguration: {\n numberOfWorkers: 123,\n workerType: glue_alpha.WorkerType.STANDARD,\n },\n};",
20038
19995
  "custom": {
20039
19996
  "exampleMetadata": "fixture=_generated"
20040
19997
  }
@@ -20046,7 +20003,7 @@
20046
20003
  "kind": "interface",
20047
20004
  "locationInModule": {
20048
20005
  "filename": "lib/jobs/scala-spark-streaming-job.ts",
20049
- "line": 15
20006
+ "line": 14
20050
20007
  },
20051
20008
  "name": "ScalaSparkStreamingJobProps",
20052
20009
  "properties": [
@@ -20059,7 +20016,7 @@
20059
20016
  "immutable": true,
20060
20017
  "locationInModule": {
20061
20018
  "filename": "lib/jobs/scala-spark-streaming-job.ts",
20062
- "line": 21
20019
+ "line": 20
20063
20020
  },
20064
20021
  "name": "className",
20065
20022
  "type": {
@@ -20077,7 +20034,7 @@
20077
20034
  "immutable": true,
20078
20035
  "locationInModule": {
20079
20036
  "filename": "lib/jobs/scala-spark-streaming-job.ts",
20080
- "line": 30
20037
+ "line": 29
20081
20038
  },
20082
20039
  "name": "extraFiles",
20083
20040
  "optional": true,
@@ -20100,7 +20057,7 @@
20100
20057
  "immutable": true,
20101
20058
  "locationInModule": {
20102
20059
  "filename": "lib/jobs/scala-spark-streaming-job.ts",
20103
- "line": 37
20060
+ "line": 36
20104
20061
  },
20105
20062
  "name": "extraJars",
20106
20063
  "optional": true,
@@ -20124,7 +20081,7 @@
20124
20081
  "immutable": true,
20125
20082
  "locationInModule": {
20126
20083
  "filename": "lib/jobs/scala-spark-streaming-job.ts",
20127
- "line": 46
20084
+ "line": 45
20128
20085
  },
20129
20086
  "name": "extraJarsFirst",
20130
20087
  "optional": true,
@@ -20143,7 +20100,7 @@
20143
20100
  "immutable": true,
20144
20101
  "locationInModule": {
20145
20102
  "filename": "lib/jobs/scala-spark-streaming-job.ts",
20146
- "line": 58
20103
+ "line": 57
20147
20104
  },
20148
20105
  "name": "jobRunQueuingEnabled",
20149
20106
  "optional": true,
@@ -20159,7 +20116,7 @@
20159
20116
  "docs": {
20160
20117
  "see": "https://docs.aws.amazon.com/athena/latest/ug/data-types.html",
20161
20118
  "stability": "experimental",
20162
- "example": "declare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'data',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'date',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n partitionProjection: {\n date: glue.PartitionProjectionConfiguration.date({\n min: 'NOW-3YEARS',\n max: 'NOW',\n format: 'yyyy-MM-dd',\n }),\n },\n});",
20119
+ "example": "declare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'col1',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'year',\n type: glue.Schema.SMALL_INT,\n }, {\n name: 'month',\n type: glue.Schema.SMALL_INT,\n }],\n dataFormat: glue.DataFormat.JSON,\n enablePartitionFiltering: true,\n});",
20163
20120
  "custom": {
20164
20121
  "exampleMetadata": "infused"
20165
20122
  }
@@ -20565,7 +20522,7 @@
20565
20522
  "remarks": "The following scenarios show some of the ways that you can use a security configuration.\n- Attach a security configuration to an AWS Glue crawler to write encrypted Amazon CloudWatch Logs.\n- Attach a security configuration to an extract, transform, and load (ETL) job to write encrypted Amazon Simple Storage Service (Amazon S3) targets and encrypted CloudWatch Logs.\n- Attach a security configuration to an ETL job to write its jobs bookmarks as encrypted Amazon S3 data.\n- Attach a security configuration to a development endpoint to write encrypted Amazon S3 targets.",
20566
20523
  "stability": "experimental",
20567
20524
  "summary": "A security configuration is a set of security properties that can be used by AWS Glue to encrypt data at rest.",
20568
- "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkEtlJob(stack, 'PySparkETLJob', {\n jobName: 'PySparkETLJobCustomName',\n description: 'This is a description',\n role,\n script,\n glueVersion: glue.GlueVersion.V5_1,\n continuousLogging: { enabled: false },\n workerType: glue.WorkerType.G_2X,\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n numberOfWorkers: 2,\n maxRetries: 2,\n});",
20525
+ "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkEtlJob(stack, 'PySparkETLJob', {\n jobName: 'PySparkETLJobCustomName',\n description: 'This is a description',\n role,\n script,\n glueVersion: glue.GlueVersion.V5_1,\n continuousLogging: { enabled: false },\n workerConfiguration: {\n workerType: glue.WorkerType.G_2X,\n numberOfWorkers: 2,\n },\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n maxRetries: 2,\n});",
20569
20526
  "custom": {
20570
20527
  "exampleMetadata": "infused"
20571
20528
  }
@@ -20577,7 +20534,7 @@
20577
20534
  },
20578
20535
  "locationInModule": {
20579
20536
  "filename": "lib/security-configuration.ts",
20580
- "line": 198
20537
+ "line": 214
20581
20538
  },
20582
20539
  "parameters": [
20583
20540
  {
@@ -20607,7 +20564,7 @@
20607
20564
  "kind": "class",
20608
20565
  "locationInModule": {
20609
20566
  "filename": "lib/security-configuration.ts",
20610
- "line": 161
20567
+ "line": 177
20611
20568
  },
20612
20569
  "methods": [
20613
20570
  {
@@ -20617,7 +20574,7 @@
20617
20574
  },
20618
20575
  "locationInModule": {
20619
20576
  "filename": "lib/security-configuration.ts",
20620
- "line": 173
20577
+ "line": 189
20621
20578
  },
20622
20579
  "name": "fromSecurityConfigurationName",
20623
20580
  "parameters": [
@@ -20668,7 +20625,7 @@
20668
20625
  "immutable": true,
20669
20626
  "locationInModule": {
20670
20627
  "filename": "lib/security-configuration.ts",
20671
- "line": 164
20628
+ "line": 180
20672
20629
  },
20673
20630
  "name": "PROPERTY_INJECTION_ID",
20674
20631
  "static": true,
@@ -20684,7 +20641,7 @@
20684
20641
  "immutable": true,
20685
20642
  "locationInModule": {
20686
20643
  "filename": "lib/security-configuration.ts",
20687
- "line": 262
20644
+ "line": 278
20688
20645
  },
20689
20646
  "name": "securityConfigurationName",
20690
20647
  "overrides": "@aws-cdk/aws-glue-alpha.ISecurityConfiguration",
@@ -20700,7 +20657,7 @@
20700
20657
  "immutable": true,
20701
20658
  "locationInModule": {
20702
20659
  "filename": "lib/security-configuration.ts",
20703
- "line": 184
20660
+ "line": 200
20704
20661
  },
20705
20662
  "name": "cloudWatchEncryptionKey",
20706
20663
  "optional": true,
@@ -20716,7 +20673,7 @@
20716
20673
  "immutable": true,
20717
20674
  "locationInModule": {
20718
20675
  "filename": "lib/security-configuration.ts",
20719
- "line": 189
20676
+ "line": 205
20720
20677
  },
20721
20678
  "name": "jobBookmarksEncryptionKey",
20722
20679
  "optional": true,
@@ -20732,7 +20689,7 @@
20732
20689
  "immutable": true,
20733
20690
  "locationInModule": {
20734
20691
  "filename": "lib/security-configuration.ts",
20735
- "line": 194
20692
+ "line": 210
20736
20693
  },
20737
20694
  "name": "s3EncryptionKey",
20738
20695
  "optional": true,
@@ -20749,7 +20706,7 @@
20749
20706
  "docs": {
20750
20707
  "stability": "experimental",
20751
20708
  "summary": "Constructions properties of `SecurityConfiguration`.",
20752
- "example": "new glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: {\n mode: glue.CloudWatchEncryptionMode.KMS,\n },\n jobBookmarksEncryption: {\n mode: glue.JobBookmarksEncryptionMode.CLIENT_SIDE_KMS,\n },\n s3Encryption: {\n mode: glue.S3EncryptionMode.KMS,\n },\n});",
20709
+ "example": "new glue.SecurityConfiguration(this, 'MySecurityConfiguration', {\n cloudWatchEncryption: glue.CloudWatchEncryption.kms(),\n jobBookmarksEncryption: glue.JobBookmarksEncryption.clientSideKms(),\n s3Encryption: glue.S3Encryption.kms(),\n});",
20753
20710
  "custom": {
20754
20711
  "exampleMetadata": "infused"
20755
20712
  }
@@ -20758,7 +20715,7 @@
20758
20715
  "kind": "interface",
20759
20716
  "locationInModule": {
20760
20717
  "filename": "lib/security-configuration.ts",
20761
- "line": 118
20718
+ "line": 134
20762
20719
  },
20763
20720
  "name": "SecurityConfigurationProps",
20764
20721
  "properties": [
@@ -20772,7 +20729,7 @@
20772
20729
  "immutable": true,
20773
20730
  "locationInModule": {
20774
20731
  "filename": "lib/security-configuration.ts",
20775
- "line": 130
20732
+ "line": 146
20776
20733
  },
20777
20734
  "name": "cloudWatchEncryption",
20778
20735
  "optional": true,
@@ -20790,7 +20747,7 @@
20790
20747
  "immutable": true,
20791
20748
  "locationInModule": {
20792
20749
  "filename": "lib/security-configuration.ts",
20793
- "line": 136
20750
+ "line": 152
20794
20751
  },
20795
20752
  "name": "jobBookmarksEncryption",
20796
20753
  "optional": true,
@@ -20808,7 +20765,7 @@
20808
20765
  "immutable": true,
20809
20766
  "locationInModule": {
20810
20767
  "filename": "lib/security-configuration.ts",
20811
- "line": 149
20768
+ "line": 165
20812
20769
  },
20813
20770
  "name": "removalPolicy",
20814
20771
  "optional": true,
@@ -20826,7 +20783,7 @@
20826
20783
  "immutable": true,
20827
20784
  "locationInModule": {
20828
20785
  "filename": "lib/security-configuration.ts",
20829
- "line": 142
20786
+ "line": 158
20830
20787
  },
20831
20788
  "name": "s3Encryption",
20832
20789
  "optional": true,
@@ -20844,7 +20801,7 @@
20844
20801
  "immutable": true,
20845
20802
  "locationInModule": {
20846
20803
  "filename": "lib/security-configuration.ts",
20847
- "line": 124
20804
+ "line": 140
20848
20805
  },
20849
20806
  "name": "securityConfigurationName",
20850
20807
  "optional": true,
@@ -21093,7 +21050,7 @@
21093
21050
  "kind": "interface",
21094
21051
  "locationInModule": {
21095
21052
  "filename": "lib/jobs/spark-job.ts",
21096
- "line": 15
21053
+ "line": 16
21097
21054
  },
21098
21055
  "name": "SparkExtraCodeProps",
21099
21056
  "properties": [
@@ -21108,7 +21065,7 @@
21108
21065
  "immutable": true,
21109
21066
  "locationInModule": {
21110
21067
  "filename": "lib/jobs/spark-job.ts",
21111
- "line": 31
21068
+ "line": 32
21112
21069
  },
21113
21070
  "name": "extraFiles",
21114
21071
  "optional": true,
@@ -21131,7 +21088,7 @@
21131
21088
  "immutable": true,
21132
21089
  "locationInModule": {
21133
21090
  "filename": "lib/jobs/spark-job.ts",
21134
- "line": 38
21091
+ "line": 39
21135
21092
  },
21136
21093
  "name": "extraJars",
21137
21094
  "optional": true,
@@ -21155,7 +21112,7 @@
21155
21112
  "immutable": true,
21156
21113
  "locationInModule": {
21157
21114
  "filename": "lib/jobs/spark-job.ts",
21158
- "line": 47
21115
+ "line": 48
21159
21116
  },
21160
21117
  "name": "extraJarsFirst",
21161
21118
  "optional": true,
@@ -21173,7 +21130,7 @@
21173
21130
  "immutable": true,
21174
21131
  "locationInModule": {
21175
21132
  "filename": "lib/jobs/spark-job.ts",
21176
- "line": 22
21133
+ "line": 23
21177
21134
  },
21178
21135
  "name": "extraPythonFiles",
21179
21136
  "optional": true,
@@ -21208,7 +21165,7 @@
21208
21165
  },
21209
21166
  "locationInModule": {
21210
21167
  "filename": "lib/jobs/spark-job.ts",
21211
- "line": 141
21168
+ "line": 170
21212
21169
  },
21213
21170
  "parameters": [
21214
21171
  {
@@ -21234,7 +21191,7 @@
21234
21191
  "kind": "class",
21235
21192
  "locationInModule": {
21236
21193
  "filename": "lib/jobs/spark-job.ts",
21237
- "line": 129
21194
+ "line": 158
21238
21195
  },
21239
21196
  "methods": [
21240
21197
  {
@@ -21243,7 +21200,7 @@
21243
21200
  },
21244
21201
  "locationInModule": {
21245
21202
  "filename": "lib/jobs/spark-job.ts",
21246
- "line": 154
21203
+ "line": 183
21247
21204
  },
21248
21205
  "name": "nonExecutableCommonArguments",
21249
21206
  "parameters": [
@@ -21273,7 +21230,7 @@
21273
21230
  },
21274
21231
  "locationInModule": {
21275
21232
  "filename": "lib/jobs/spark-job.ts",
21276
- "line": 180
21233
+ "line": 209
21277
21234
  },
21278
21235
  "name": "setupExtraCodeArguments",
21279
21236
  "parameters": [
@@ -21308,7 +21265,7 @@
21308
21265
  "immutable": true,
21309
21266
  "locationInModule": {
21310
21267
  "filename": "lib/jobs/spark-job.ts",
21311
- "line": 131
21268
+ "line": 160
21312
21269
  },
21313
21270
  "name": "grantPrincipal",
21314
21271
  "overrides": "@aws-cdk/aws-glue-alpha.JobBase",
@@ -21324,7 +21281,7 @@
21324
21281
  "immutable": true,
21325
21282
  "locationInModule": {
21326
21283
  "filename": "lib/jobs/spark-job.ts",
21327
- "line": 130
21284
+ "line": 159
21328
21285
  },
21329
21286
  "name": "role",
21330
21287
  "overrides": "@aws-cdk/aws-glue-alpha.Job",
@@ -21341,7 +21298,7 @@
21341
21298
  "immutable": true,
21342
21299
  "locationInModule": {
21343
21300
  "filename": "lib/jobs/spark-job.ts",
21344
- "line": 139
21301
+ "line": 168
21345
21302
  },
21346
21303
  "name": "sparkUILoggingLocation",
21347
21304
  "optional": true,
@@ -21358,7 +21315,7 @@
21358
21315
  "docs": {
21359
21316
  "stability": "experimental",
21360
21317
  "summary": "Common properties for different types of Spark jobs.",
21361
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\nimport { aws_logs as logs } from 'aws-cdk-lib';\nimport { aws_s3 as s3 } from 'aws-cdk-lib';\n\ndeclare const bucket: s3.Bucket;\ndeclare const code: glue_alpha.Code;\ndeclare const connection: glue_alpha.Connection;\ndeclare const logGroup: logs.LogGroup;\ndeclare const role: iam.Role;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst sparkJobProps: glue_alpha.SparkJobProps = {\n role: role,\n script: code,\n\n // the properties below are optional\n connections: [connection],\n continuousLogging: {\n enabled: false,\n\n // the properties below are optional\n conversionPattern: 'conversionPattern',\n logGroup: logGroup,\n logStreamPrefix: 'logStreamPrefix',\n quiet: false,\n },\n defaultArguments: {\n defaultArgumentsKey: 'defaultArguments',\n },\n description: 'description',\n enableMetrics: false,\n enableObservabilityMetrics: false,\n enableProfilingMetrics: false,\n glueVersion: glue_alpha.GlueVersion.V0_9,\n jobName: 'jobName',\n maxConcurrentRuns: 123,\n maxRetries: 123,\n numberOfWorkers: 123,\n securityConfiguration: securityConfiguration,\n sparkUI: {\n bucket: bucket,\n prefix: 'prefix',\n },\n tags: {\n tagsKey: 'tags',\n },\n timeout: cdk.Duration.minutes(30),\n workerType: glue_alpha.WorkerType.STANDARD,\n};",
21318
+ "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport * as cdk from 'aws-cdk-lib';\nimport { aws_iam as iam } from 'aws-cdk-lib';\nimport { aws_logs as logs } from 'aws-cdk-lib';\nimport { aws_s3 as s3 } from 'aws-cdk-lib';\n\ndeclare const bucket: s3.Bucket;\ndeclare const code: glue_alpha.Code;\ndeclare const connection: glue_alpha.Connection;\ndeclare const logGroup: logs.LogGroup;\ndeclare const role: iam.Role;\ndeclare const securityConfiguration: glue_alpha.SecurityConfiguration;\nconst sparkJobProps: glue_alpha.SparkJobProps = {\n role: role,\n script: code,\n\n // the properties below are optional\n connections: [connection],\n continuousLogging: {\n enabled: false,\n\n // the properties below are optional\n conversionPattern: 'conversionPattern',\n logGroup: logGroup,\n logStreamPrefix: 'logStreamPrefix',\n quiet: false,\n },\n defaultArguments: {\n defaultArgumentsKey: 'defaultArguments',\n },\n description: 'description',\n enableMetrics: false,\n enableObservabilityMetrics: false,\n glueVersion: glue_alpha.GlueVersion.V0_9,\n jobName: 'jobName',\n maxConcurrentRuns: 123,\n maxRetries: 123,\n securityConfiguration: securityConfiguration,\n sparkUI: {\n bucket: bucket,\n prefix: 'prefix',\n },\n tags: {\n tagsKey: 'tags',\n },\n timeout: cdk.Duration.minutes(30),\n workerConfiguration: {\n numberOfWorkers: 123,\n workerType: glue_alpha.WorkerType.STANDARD,\n },\n};",
21362
21319
  "custom": {
21363
21320
  "exampleMetadata": "fixture=_generated"
21364
21321
  }
@@ -21370,7 +21327,7 @@
21370
21327
  "kind": "interface",
21371
21328
  "locationInModule": {
21372
21329
  "filename": "lib/jobs/spark-job.ts",
21373
- "line": 96
21330
+ "line": 118
21374
21331
  },
21375
21332
  "name": "SparkJobProps",
21376
21333
  "properties": [
@@ -21385,7 +21342,7 @@
21385
21342
  "immutable": true,
21386
21343
  "locationInModule": {
21387
21344
  "filename": "lib/jobs/spark-job.ts",
21388
- "line": 114
21345
+ "line": 143
21389
21346
  },
21390
21347
  "name": "enableMetrics",
21391
21348
  "optional": true,
@@ -21404,7 +21361,7 @@
21404
21361
  "immutable": true,
21405
21362
  "locationInModule": {
21406
21363
  "filename": "lib/jobs/spark-job.ts",
21407
- "line": 123
21364
+ "line": 152
21408
21365
  },
21409
21366
  "name": "enableObservabilityMetrics",
21410
21367
  "optional": true,
@@ -21423,13 +21380,31 @@
21423
21380
  "immutable": true,
21424
21381
  "locationInModule": {
21425
21382
  "filename": "lib/jobs/spark-job.ts",
21426
- "line": 105
21383
+ "line": 134
21427
21384
  },
21428
21385
  "name": "sparkUI",
21429
21386
  "optional": true,
21430
21387
  "type": {
21431
21388
  "fqn": "@aws-cdk/aws-glue-alpha.SparkUIProps"
21432
21389
  }
21390
+ },
21391
+ {
21392
+ "abstract": true,
21393
+ "docs": {
21394
+ "default": "- the job runs with the G_1X worker type and 10 workers.",
21395
+ "stability": "experimental",
21396
+ "summary": "The worker type and the number of workers allocated when a job runs."
21397
+ },
21398
+ "immutable": true,
21399
+ "locationInModule": {
21400
+ "filename": "lib/jobs/spark-job.ts",
21401
+ "line": 124
21402
+ },
21403
+ "name": "workerConfiguration",
21404
+ "optional": true,
21405
+ "type": {
21406
+ "fqn": "@aws-cdk/aws-glue-alpha.WorkerConfiguration"
21407
+ }
21433
21408
  }
21434
21409
  ],
21435
21410
  "symbolId": "lib/jobs/spark-job:SparkJobProps"
@@ -21450,7 +21425,7 @@
21450
21425
  "kind": "interface",
21451
21426
  "locationInModule": {
21452
21427
  "filename": "lib/jobs/spark-job.ts",
21453
- "line": 79
21428
+ "line": 80
21454
21429
  },
21455
21430
  "name": "SparkUILoggingLocation",
21456
21431
  "properties": [
@@ -21463,7 +21438,7 @@
21463
21438
  "immutable": true,
21464
21439
  "locationInModule": {
21465
21440
  "filename": "lib/jobs/spark-job.ts",
21466
- "line": 83
21441
+ "line": 84
21467
21442
  },
21468
21443
  "name": "bucket",
21469
21444
  "type": {
@@ -21480,7 +21455,7 @@
21480
21455
  "immutable": true,
21481
21456
  "locationInModule": {
21482
21457
  "filename": "lib/jobs/spark-job.ts",
21483
- "line": 90
21458
+ "line": 91
21484
21459
  },
21485
21460
  "name": "prefix",
21486
21461
  "optional": true,
@@ -21507,7 +21482,7 @@
21507
21482
  "kind": "interface",
21508
21483
  "locationInModule": {
21509
21484
  "filename": "lib/jobs/spark-job.ts",
21510
- "line": 56
21485
+ "line": 57
21511
21486
  },
21512
21487
  "name": "SparkUIProps",
21513
21488
  "properties": [
@@ -21521,7 +21496,7 @@
21521
21496
  "immutable": true,
21522
21497
  "locationInModule": {
21523
21498
  "filename": "lib/jobs/spark-job.ts",
21524
- "line": 62
21499
+ "line": 63
21525
21500
  },
21526
21501
  "name": "bucket",
21527
21502
  "optional": true,
@@ -21540,7 +21515,7 @@
21540
21515
  "immutable": true,
21541
21516
  "locationInModule": {
21542
21517
  "filename": "lib/jobs/spark-job.ts",
21543
- "line": 70
21518
+ "line": 71
21544
21519
  },
21545
21520
  "name": "prefix",
21546
21521
  "optional": true,
@@ -22311,7 +22286,7 @@
22311
22286
  },
22312
22287
  "locationInModule": {
22313
22288
  "filename": "lib/s3-table.ts",
22314
- "line": 133
22289
+ "line": 134
22315
22290
  },
22316
22291
  "parameters": [
22317
22292
  {
@@ -22432,7 +22407,7 @@
22432
22407
  },
22433
22408
  "locationInModule": {
22434
22409
  "filename": "lib/table-base.ts",
22435
- "line": 261
22410
+ "line": 284
22436
22411
  },
22437
22412
  "parameters": [
22438
22413
  {
@@ -22461,7 +22436,7 @@
22461
22436
  "kind": "class",
22462
22437
  "locationInModule": {
22463
22438
  "filename": "lib/table-base.ts",
22464
- "line": 182
22439
+ "line": 199
22465
22440
  },
22466
22441
  "methods": [
22467
22442
  {
@@ -22470,7 +22445,7 @@
22470
22445
  },
22471
22446
  "locationInModule": {
22472
22447
  "filename": "lib/table-base.ts",
22473
- "line": 183
22448
+ "line": 200
22474
22449
  },
22475
22450
  "name": "fromTableArn",
22476
22451
  "parameters": [
@@ -22507,7 +22482,7 @@
22507
22482
  },
22508
22483
  "locationInModule": {
22509
22484
  "filename": "lib/table-base.ts",
22510
- "line": 199
22485
+ "line": 216
22511
22486
  },
22512
22487
  "name": "fromTableAttributes",
22513
22488
  "parameters": [
@@ -22555,7 +22530,7 @@
22555
22530
  },
22556
22531
  "locationInModule": {
22557
22532
  "filename": "lib/table-base.ts",
22558
- "line": 295
22533
+ "line": 321
22559
22534
  },
22560
22535
  "name": "addPartitionIndex",
22561
22536
  "parameters": [
@@ -22575,7 +22550,7 @@
22575
22550
  },
22576
22551
  "locationInModule": {
22577
22552
  "filename": "lib/table-base.ts",
22578
- "line": 481
22553
+ "line": 536
22579
22554
  },
22580
22555
  "name": "grant",
22581
22556
  "parameters": [
@@ -22616,7 +22591,7 @@
22616
22591
  },
22617
22592
  "locationInModule": {
22618
22593
  "filename": "lib/table-base.ts",
22619
- "line": 284
22594
+ "line": 310
22620
22595
  },
22621
22596
  "name": "grantRead",
22622
22597
  "parameters": [
@@ -22640,7 +22615,7 @@
22640
22615
  },
22641
22616
  "locationInModule": {
22642
22617
  "filename": "lib/table-base.ts",
22643
- "line": 286
22618
+ "line": 312
22644
22619
  },
22645
22620
  "name": "grantReadWrite",
22646
22621
  "parameters": [
@@ -22665,7 +22640,7 @@
22665
22640
  },
22666
22641
  "locationInModule": {
22667
22642
  "filename": "lib/table-base.ts",
22668
- "line": 505
22643
+ "line": 560
22669
22644
  },
22670
22645
  "name": "grantToUnderlyingResources",
22671
22646
  "parameters": [
@@ -22706,7 +22681,7 @@
22706
22681
  },
22707
22682
  "locationInModule": {
22708
22683
  "filename": "lib/table-base.ts",
22709
- "line": 285
22684
+ "line": 311
22710
22685
  },
22711
22686
  "name": "grantWrite",
22712
22687
  "parameters": [
@@ -22734,7 +22709,7 @@
22734
22709
  "immutable": true,
22735
22710
  "locationInModule": {
22736
22711
  "filename": "lib/table-base.ts",
22737
- "line": 231
22712
+ "line": 248
22738
22713
  },
22739
22714
  "name": "columns",
22740
22715
  "type": {
@@ -22754,7 +22729,7 @@
22754
22729
  "immutable": true,
22755
22730
  "locationInModule": {
22756
22731
  "filename": "lib/table-base.ts",
22757
- "line": 221
22732
+ "line": 238
22758
22733
  },
22759
22734
  "name": "compressed",
22760
22735
  "type": {
@@ -22769,7 +22744,7 @@
22769
22744
  "immutable": true,
22770
22745
  "locationInModule": {
22771
22746
  "filename": "lib/table-base.ts",
22772
- "line": 216
22747
+ "line": 233
22773
22748
  },
22774
22749
  "name": "database",
22775
22750
  "type": {
@@ -22784,13 +22759,30 @@
22784
22759
  "immutable": true,
22785
22760
  "locationInModule": {
22786
22761
  "filename": "lib/table-base.ts",
22787
- "line": 226
22762
+ "line": 243
22788
22763
  },
22789
22764
  "name": "dataFormat",
22790
22765
  "type": {
22791
22766
  "fqn": "@aws-cdk/aws-glue-alpha.DataFormat"
22792
22767
  }
22793
22768
  },
22769
+ {
22770
+ "docs": {
22771
+ "remarks": "Emitted as the\n`has_encrypted_data` table parameter.",
22772
+ "stability": "experimental",
22773
+ "summary": "Whether the data stored in the table is encrypted."
22774
+ },
22775
+ "immutable": true,
22776
+ "locationInModule": {
22777
+ "filename": "lib/table-base.ts",
22778
+ "line": 275
22779
+ },
22780
+ "name": "hasEncryptedData",
22781
+ "protected": true,
22782
+ "type": {
22783
+ "primitive": "boolean"
22784
+ }
22785
+ },
22794
22786
  {
22795
22787
  "docs": {
22796
22788
  "see": "https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-properties-glue-table-tableinput.html#cfn-glue-table-tableinput-parameters",
@@ -22800,7 +22792,7 @@
22800
22792
  "immutable": true,
22801
22793
  "locationInModule": {
22802
22794
  "filename": "lib/table-base.ts",
22803
- "line": 252
22795
+ "line": 269
22804
22796
  },
22805
22797
  "name": "parameters",
22806
22798
  "protected": true,
@@ -22821,7 +22813,7 @@
22821
22813
  "immutable": true,
22822
22814
  "locationInModule": {
22823
22815
  "filename": "lib/table-base.ts",
22824
- "line": 210
22816
+ "line": 227
22825
22817
  },
22826
22818
  "name": "tableArn",
22827
22819
  "overrides": "@aws-cdk/aws-glue-alpha.ITable",
@@ -22837,7 +22829,7 @@
22837
22829
  "immutable": true,
22838
22830
  "locationInModule": {
22839
22831
  "filename": "lib/table-base.ts",
22840
- "line": 209
22832
+ "line": 226
22841
22833
  },
22842
22834
  "name": "tableName",
22843
22835
  "overrides": "@aws-cdk/aws-glue-alpha.ITable",
@@ -22853,7 +22845,7 @@
22853
22845
  "immutable": true,
22854
22846
  "locationInModule": {
22855
22847
  "filename": "lib/table-base.ts",
22856
- "line": 208
22848
+ "line": 225
22857
22849
  },
22858
22850
  "name": "tableResource",
22859
22851
  "protected": true,
@@ -22869,7 +22861,7 @@
22869
22861
  "immutable": true,
22870
22862
  "locationInModule": {
22871
22863
  "filename": "lib/table-base.ts",
22872
- "line": 211
22864
+ "line": 228
22873
22865
  },
22874
22866
  "name": "partitionIndexes",
22875
22867
  "optional": true,
@@ -22890,7 +22882,7 @@
22890
22882
  "immutable": true,
22891
22883
  "locationInModule": {
22892
22884
  "filename": "lib/table-base.ts",
22893
- "line": 236
22885
+ "line": 253
22894
22886
  },
22895
22887
  "name": "partitionKeys",
22896
22888
  "optional": true,
@@ -22911,7 +22903,7 @@
22911
22903
  "immutable": true,
22912
22904
  "locationInModule": {
22913
22905
  "filename": "lib/table-base.ts",
22914
- "line": 246
22906
+ "line": 263
22915
22907
  },
22916
22908
  "name": "partitionProjection",
22917
22909
  "optional": true,
@@ -22932,7 +22924,7 @@
22932
22924
  "immutable": true,
22933
22925
  "locationInModule": {
22934
22926
  "filename": "lib/table-base.ts",
22935
- "line": 241
22927
+ "line": 258
22936
22928
  },
22937
22929
  "name": "storageParameters",
22938
22930
  "optional": true,
@@ -22953,7 +22945,7 @@
22953
22945
  "datatype": true,
22954
22946
  "docs": {
22955
22947
  "stability": "experimental",
22956
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\n\ndeclare const database: glue_alpha.Database;\ndeclare const dataFormat: glue_alpha.DataFormat;\ndeclare const partitionProjectionConfiguration: glue_alpha.PartitionProjectionConfiguration;\ndeclare const storageParameter: glue_alpha.StorageParameter;\nconst tableBaseProps: glue_alpha.TableBaseProps = {\n columns: [{\n name: 'name',\n type: {\n inputString: 'inputString',\n isPrimitive: false,\n },\n\n // the properties below are optional\n comment: 'comment',\n }],\n database: database,\n dataFormat: dataFormat,\n\n // the properties below are optional\n compressed: false,\n description: 'description',\n enablePartitionFiltering: false,\n parameters: {\n parametersKey: 'parameters',\n },\n partitionIndexes: [{\n keyNames: ['keyNames'],\n\n // the properties below are optional\n indexName: 'indexName',\n }],\n partitionKeys: [{\n name: 'name',\n type: {\n inputString: 'inputString',\n isPrimitive: false,\n },\n\n // the properties below are optional\n comment: 'comment',\n }],\n partitionProjection: {\n partitionProjectionKey: partitionProjectionConfiguration,\n },\n storageParameters: [storageParameter],\n storedAsSubDirectories: false,\n tableName: 'tableName',\n};",
22948
+ "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\n\ndeclare const database: glue_alpha.Database;\ndeclare const dataFormat: glue_alpha.DataFormat;\ndeclare const partitionProjectionConfiguration: glue_alpha.PartitionProjectionConfiguration;\ndeclare const storageParameter: glue_alpha.StorageParameter;\nconst tableBaseProps: glue_alpha.TableBaseProps = {\n columns: [{\n name: 'name',\n type: {\n inputString: 'inputString',\n isPrimitive: false,\n },\n\n // the properties below are optional\n comment: 'comment',\n }],\n database: database,\n dataFormat: dataFormat,\n\n // the properties below are optional\n compressed: false,\n description: 'description',\n enablePartitionFiltering: false,\n hasEncryptedData: false,\n parameters: {\n parametersKey: 'parameters',\n },\n partitionIndexes: [{\n keyNames: ['keyNames'],\n\n // the properties below are optional\n indexName: 'indexName',\n }],\n partitionKeys: [{\n name: 'name',\n type: {\n inputString: 'inputString',\n isPrimitive: false,\n },\n\n // the properties below are optional\n comment: 'comment',\n }],\n partitionProjection: {\n partitionProjectionKey: partitionProjectionConfiguration,\n },\n storageParameters: [storageParameter],\n storedAsSubDirectories: false,\n tableName: 'tableName',\n};",
22957
22949
  "custom": {
22958
22950
  "exampleMetadata": "fixture=_generated"
22959
22951
  }
@@ -23074,6 +23066,26 @@
23074
23066
  "primitive": "boolean"
23075
23067
  }
23076
23068
  },
23069
+ {
23070
+ "abstract": true,
23071
+ "docs": {
23072
+ "default": "true",
23073
+ "remarks": "This sets the `has_encrypted_data` table parameter. Athena reads it when\nquerying client-side (CSE-KMS) encrypted datasets; for server-side\nencrypted (SSE-S3 / SSE-KMS) or unencrypted data it has no effect, since\nAmazon S3 decrypts server-side encrypted objects transparently.\n\nDo not also set `has_encrypted_data` through `parameters` - use this\nproperty instead. A conflicting value in `parameters` is rejected.",
23074
+ "see": "https://docs.aws.amazon.com/athena/latest/ug/creating-tables-based-on-encrypted-datasets-in-s3.html",
23075
+ "stability": "experimental",
23076
+ "summary": "Whether the data stored in the table is encrypted."
23077
+ },
23078
+ "immutable": true,
23079
+ "locationInModule": {
23080
+ "filename": "lib/table-base.ts",
23081
+ "line": 181
23082
+ },
23083
+ "name": "hasEncryptedData",
23084
+ "optional": true,
23085
+ "type": {
23086
+ "primitive": "boolean"
23087
+ }
23088
+ },
23077
23089
  {
23078
23090
  "abstract": true,
23079
23091
  "docs": {
@@ -23158,7 +23170,7 @@
23158
23170
  "immutable": true,
23159
23171
  "locationInModule": {
23160
23172
  "filename": "lib/table-base.ts",
23161
- "line": 176
23173
+ "line": 193
23162
23174
  },
23163
23175
  "name": "partitionProjection",
23164
23176
  "optional": true,
@@ -23294,7 +23306,7 @@
23294
23306
  "datatype": true,
23295
23307
  "docs": {
23296
23308
  "stability": "experimental",
23297
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport { aws_kms as kms } from 'aws-cdk-lib';\nimport { aws_s3 as s3 } from 'aws-cdk-lib';\n\ndeclare const bucket: s3.Bucket;\ndeclare const database: glue_alpha.Database;\ndeclare const dataFormat: glue_alpha.DataFormat;\ndeclare const key: kms.Key;\ndeclare const partitionProjectionConfiguration: glue_alpha.PartitionProjectionConfiguration;\ndeclare const storageParameter: glue_alpha.StorageParameter;\nconst tableProps: glue_alpha.TableProps = {\n columns: [{\n name: 'name',\n type: {\n inputString: 'inputString',\n isPrimitive: false,\n },\n\n // the properties below are optional\n comment: 'comment',\n }],\n database: database,\n dataFormat: dataFormat,\n\n // the properties below are optional\n bucket: bucket,\n compressed: false,\n description: 'description',\n enablePartitionFiltering: false,\n encryption: glue_alpha.TableEncryption.S3_MANAGED,\n encryptionKey: key,\n parameters: {\n parametersKey: 'parameters',\n },\n partitionIndexes: [{\n keyNames: ['keyNames'],\n\n // the properties below are optional\n indexName: 'indexName',\n }],\n partitionKeys: [{\n name: 'name',\n type: {\n inputString: 'inputString',\n isPrimitive: false,\n },\n\n // the properties below are optional\n comment: 'comment',\n }],\n partitionProjection: {\n partitionProjectionKey: partitionProjectionConfiguration,\n },\n s3Prefix: 's3Prefix',\n storageParameters: [storageParameter],\n storedAsSubDirectories: false,\n tableName: 'tableName',\n};",
23309
+ "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nimport { aws_kms as kms } from 'aws-cdk-lib';\nimport { aws_s3 as s3 } from 'aws-cdk-lib';\n\ndeclare const bucket: s3.Bucket;\ndeclare const database: glue_alpha.Database;\ndeclare const dataFormat: glue_alpha.DataFormat;\ndeclare const key: kms.Key;\ndeclare const partitionProjectionConfiguration: glue_alpha.PartitionProjectionConfiguration;\ndeclare const storageParameter: glue_alpha.StorageParameter;\nconst tableProps: glue_alpha.TableProps = {\n columns: [{\n name: 'name',\n type: {\n inputString: 'inputString',\n isPrimitive: false,\n },\n\n // the properties below are optional\n comment: 'comment',\n }],\n database: database,\n dataFormat: dataFormat,\n\n // the properties below are optional\n bucket: bucket,\n compressed: false,\n description: 'description',\n enablePartitionFiltering: false,\n encryption: glue_alpha.TableEncryption.S3_MANAGED,\n encryptionKey: key,\n hasEncryptedData: false,\n parameters: {\n parametersKey: 'parameters',\n },\n partitionIndexes: [{\n keyNames: ['keyNames'],\n\n // the properties below are optional\n indexName: 'indexName',\n }],\n partitionKeys: [{\n name: 'name',\n type: {\n inputString: 'inputString',\n isPrimitive: false,\n },\n\n // the properties below are optional\n comment: 'comment',\n }],\n partitionProjection: {\n partitionProjectionKey: partitionProjectionConfiguration,\n },\n s3Prefix: 's3Prefix',\n storageParameters: [storageParameter],\n storedAsSubDirectories: false,\n tableName: 'tableName',\n};",
23298
23310
  "custom": {
23299
23311
  "exampleMetadata": "fixture=_generated"
23300
23312
  }
@@ -23492,7 +23504,7 @@
23492
23504
  "docs": {
23493
23505
  "stability": "experimental",
23494
23506
  "summary": "Represents a type of a column in a table schema.",
23495
- "example": "declare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'data',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'date',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n partitionProjection: {\n date: glue.PartitionProjectionConfiguration.date({\n min: '2020-01-01',\n max: '2023-12-31',\n format: 'yyyy-MM-dd',\n interval: 1, // optional, defaults to 1\n intervalUnit: glue.DateIntervalUnit.DAYS, // optional: YEARS, MONTHS, WEEKS, DAYS, HOURS, MINUTES, SECONDS\n }),\n },\n});",
23507
+ "example": "declare const myDatabase: glue.Database;\nnew glue.S3Table(this, 'MyTable', {\n database: myDatabase,\n columns: [{\n name: 'data',\n type: glue.Schema.STRING,\n }],\n partitionKeys: [{\n name: 'date',\n type: glue.Schema.STRING,\n }],\n dataFormat: glue.DataFormat.JSON,\n partitionProjection: {\n date: glue.PartitionProjectionConfiguration.date({\n min: 'NOW-3YEARS',\n max: 'NOW',\n format: 'yyyy-MM-dd',\n }),\n },\n});",
23496
23508
  "custom": {
23497
23509
  "exampleMetadata": "infused"
23498
23510
  }
@@ -23563,12 +23575,68 @@
23563
23575
  "name": "WeeklyScheduleTriggerOptions",
23564
23576
  "symbolId": "lib/triggers/trigger-options:WeeklyScheduleTriggerOptions"
23565
23577
  },
23578
+ "@aws-cdk/aws-glue-alpha.WorkerConfiguration": {
23579
+ "assembly": "@aws-cdk/aws-glue-alpha",
23580
+ "datatype": true,
23581
+ "docs": {
23582
+ "remarks": "The worker type and the number of workers are set together: providing this\nconfiguration requires both values, so a Spark job can never be given one\nwithout the other.",
23583
+ "stability": "experimental",
23584
+ "summary": "The worker configuration for a Spark job.",
23585
+ "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkEtlJob(stack, 'PySparkETLJob', {\n jobName: 'PySparkETLJobCustomName',\n description: 'This is a description',\n role,\n script,\n glueVersion: glue.GlueVersion.V5_1,\n continuousLogging: { enabled: false },\n workerConfiguration: {\n workerType: glue.WorkerType.G_2X,\n numberOfWorkers: 2,\n },\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n maxRetries: 2,\n});",
23586
+ "custom": {
23587
+ "exampleMetadata": "infused"
23588
+ }
23589
+ },
23590
+ "fqn": "@aws-cdk/aws-glue-alpha.WorkerConfiguration",
23591
+ "kind": "interface",
23592
+ "locationInModule": {
23593
+ "filename": "lib/jobs/spark-job.ts",
23594
+ "line": 101
23595
+ },
23596
+ "name": "WorkerConfiguration",
23597
+ "properties": [
23598
+ {
23599
+ "abstract": true,
23600
+ "docs": {
23601
+ "stability": "experimental",
23602
+ "summary": "The number of workers of the given `workerType` that are allocated when a job runs."
23603
+ },
23604
+ "immutable": true,
23605
+ "locationInModule": {
23606
+ "filename": "lib/jobs/spark-job.ts",
23607
+ "line": 112
23608
+ },
23609
+ "name": "numberOfWorkers",
23610
+ "type": {
23611
+ "primitive": "number"
23612
+ }
23613
+ },
23614
+ {
23615
+ "abstract": true,
23616
+ "docs": {
23617
+ "remarks": "Enum options: Standard, G_1X, G_2X, G_025X, G_4X, G_8X, Z_2X",
23618
+ "stability": "experimental",
23619
+ "summary": "The type of predefined worker that is allocated when a job runs."
23620
+ },
23621
+ "immutable": true,
23622
+ "locationInModule": {
23623
+ "filename": "lib/jobs/spark-job.ts",
23624
+ "line": 107
23625
+ },
23626
+ "name": "workerType",
23627
+ "type": {
23628
+ "fqn": "@aws-cdk/aws-glue-alpha.WorkerType"
23629
+ }
23630
+ }
23631
+ ],
23632
+ "symbolId": "lib/jobs/spark-job:WorkerConfiguration"
23633
+ },
23566
23634
  "@aws-cdk/aws-glue-alpha.WorkerType": {
23567
23635
  "assembly": "@aws-cdk/aws-glue-alpha",
23568
23636
  "docs": {
23569
23637
  "stability": "experimental",
23570
23638
  "summary": "The type of predefined worker that is allocated when a job runs.",
23571
- "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkEtlJob(stack, 'PySparkETLJob', {\n jobName: 'PySparkETLJobCustomName',\n description: 'This is a description',\n role,\n script,\n glueVersion: glue.GlueVersion.V5_1,\n continuousLogging: { enabled: false },\n workerType: glue.WorkerType.G_2X,\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n numberOfWorkers: 2,\n maxRetries: 2,\n});",
23639
+ "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nnew glue.PySparkEtlJob(stack, 'PySparkETLJob', {\n jobName: 'PySparkETLJobCustomName',\n description: 'This is a description',\n role,\n script,\n glueVersion: glue.GlueVersion.V5_1,\n continuousLogging: { enabled: false },\n workerConfiguration: {\n workerType: glue.WorkerType.G_2X,\n numberOfWorkers: 2,\n },\n maxConcurrentRuns: 100,\n timeout: cdk.Duration.hours(2),\n connections: [glue.Connection.fromConnectionName(stack, 'Connection', 'connectionName')],\n securityConfiguration: glue.SecurityConfiguration.fromSecurityConfigurationName(stack, 'SecurityConfig', 'securityConfigName'),\n tags: {\n FirstTagName: 'FirstTagValue',\n SecondTagName: 'SecondTagValue',\n XTagName: 'XTagValue',\n },\n maxRetries: 2,\n});",
23572
23640
  "custom": {
23573
23641
  "exampleMetadata": "infused"
23574
23642
  }
@@ -23684,9 +23752,9 @@
23684
23752
  "see": "https://docs.aws.amazon.com/glue/latest/dg/about-triggers.html\n\n## Usage Example\n\n```ts\nconst app = new App();\nconst stack = new Stack(app, 'TestStack');\n\n// Create a Glue Job\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\nconst job = new glue.PySparkStreamingJob(stack, 'ImportedJob', { role, script });\n\n// Create a Glue Workflow\nconst workflow = new glue.Workflow(stack, 'TestWorkflow');\n\n// Add an on-demand trigger to the Workflow\nworkflow.addOnDemandTrigger('OnDemandTrigger', {\nactions: [{ job: job }],\n});\n```",
23685
23753
  "stability": "experimental",
23686
23754
  "summary": "This module defines a construct for creating and managing AWS Glue Workflows and Triggers.",
23687
- "example": "// The code below shows an example of how to instantiate this type.\n// The values are placeholders you should change.\nimport * as glue_alpha from '@aws-cdk/aws-glue-alpha';\nconst workflow = new glue_alpha.Workflow(this, 'MyWorkflow', /* all optional props */ {\n defaultRunProperties: {\n defaultRunPropertiesKey: 'defaultRunProperties',\n },\n description: 'description',\n maxConcurrentRuns: 123,\n workflowName: 'workflowName',\n});",
23755
+ "example": "import * as cdk from 'aws-cdk-lib';\nimport * as iam from 'aws-cdk-lib/aws-iam';\ndeclare const stack: cdk.Stack;\ndeclare const role: iam.IRole;\ndeclare const script: glue.Code;\n\n// Create a job to run from the workflow\nconst job = new glue.PySparkEtlJob(stack, 'Job', { role, script });\n\n// Create a workflow and add a trigger that runs the job\nconst workflow = new glue.Workflow(stack, 'Workflow');\nworkflow.addOnDemandTrigger('OnDemandTrigger', {\n actions: [{ job }],\n});",
23688
23756
  "custom": {
23689
- "exampleMetadata": "fixture=_generated"
23757
+ "exampleMetadata": "infused"
23690
23758
  }
23691
23759
  },
23692
23760
  "fqn": "@aws-cdk/aws-glue-alpha.Workflow",
@@ -24476,6 +24544,6 @@
24476
24544
  "symbolId": "lib/storage-parameter:WriteParallel"
24477
24545
  }
24478
24546
  },
24479
- "version": "2.265.0-alpha.0",
24547
+ "version": "2.266.0-alpha.0",
24480
24548
  "fingerprint": "**********"
24481
24549
  }