(aws-s3-deployment): assets with deploy-time values

[johnameyer]

Allow for deploying of an inline file to an s3 bucket, like as can be done for Lambda functions.

Use Case

This is one way to allow for resources generated at synth time to be aware of resources that are created in a stack at deploy time, e.g. allowing a static frontend (with a config file that is deployed by this mechanism) to know the URIs of backend resources like a Cognito auth endpoint or an API Gateway. Since normally static resources would be built at the asset generation phase, it would normally be impossible for them to obtain these attributes of resources generated later in the lifecycle. Instead, you can build your code with the assumption of a config file being generated at deploy time and deployed to the bucket and postpone actually needing those URIs.

This is particularly helpful in CDK Pipelines, where it is desired to have all the assets generated up-front and then configure a stage at deploy time, to prevent extensive customization of the Pipeline.

Proposed Solution

From the description of how the s3-deployments module works under the hood, it's not exactly clear at what point the files are uploaded to the intermediary asset bucket, but it seems at least naively a custom resource could create a resource in a bucket from a parameter (resolved by cloudformation) from which a SourceConfig could be generated.

The workaround currently is to create a custom resource which deploys this file to the bucket, which seems to be how s3-deployments already work, but does not have the in-built feature of invalidating Cloudfront.

Other

S3 Deployments module: https://docs.aws.amazon.com/cdk/api/latest/docs/aws-s3-deployment-readme.html
Lambda InlineCode: https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_aws-lambda.InlineCode.html
StackOverflow question: https://stackoverflow.com/questions/60074546/aws-cdk-passing-api-gateway-url-to-static-site-in-same-stack/65799623#65799623
Example usage:

const sources = [
  s3deploy.Source.asset('./website-dist'),
  s3deploy.Source.inline('config.js', `exports.config = { cognitoUserPool: '${ cognitoUserPool.userPoolClientId }', apiURI: '${ api.apiId }'`)
];
new s3deploy.BucketDeployment(this, 'DeployWebsite', {
  sources,
  destinationBucket: websiteBucket,
  destinationKeyPrefix: 'web/static'
}); // my file should exist at web/static/config.js and have the tokens resolved

• 👋 I may be able to implement this feature request
• ⚠️ This feature might incur a breaking change

---

This is a 🚀 Feature Request

[johnameyer]

Implemented current workaround in https://stackoverflow.com/a/66084764/2088936 as proof of feasibility/starting point for discussion.

[jogold]

I'm offering a solution for this here https://github.com/jogold/cloudstructs/tree/master/src/static-website (static website with backend config saved to the bucket).

[iliapolo]

@johnameyer I'm not opposed to having this functionality, seems useful. So the current proposal is to:

1. Custom resource accept a string that is resolved at deploy time.
2. Custom resource lambda zips up the resolved string and uploads to an intermediary bucket.
3. That bucket is used to bind SourceConfig

Right?

P.S Is what @jogold referenced fully addresses your use-case?

[johnameyer]

That's the current workaround and I think it is the least disagreeable. However, it is not a particularly efficient solution since it's adding a custom resource and a bucket that just feeds into another custom resource and bucket under the hood. I think there a few ways to make it less heavy-handed.
One way (call it proposal B) is to allow passing a non-zip file to the ISource (since we just go and unzip on the other side), as it would make the inline source custom resource no longer need to zip up the file (no extra library needed, and now just a simple upload so it can just be an AwsCustomResource that puts to s3).
Potentially (proposal C), we could also add inline as an option onto ISource like Lambda does and then have the s3-deployment custom resource handle everything (which would require a little more work), but this then saves us making an intermediate bucket and another custom handler altogether.
There's potentially a few other options as well but curious as to what you think of those two.

@jogold's solution is pretty good, but it doesn't directly integrate with s3-deployments (and as such things like the cloudfront invalidation that I need).

[iliapolo]

@johnameyer Yeap, I think option 3 is the best, just to make sure we aligned, basically adding a new property in SourceConfig:

aws-cdk/packages/@aws-cdk/aws-s3-deployment/lib/source.ts

Lines 9 to 19 in 3b5ff05

	export interface SourceConfig {
	/**
	* The source bucket to deploy from.
	*/
	readonly bucket: s3.IBucket;
	/**
	* An S3 object key in the source bucket that points to a zip file.
	*/
	readonly zipObjectKey: string;
	}

And take it from there. Yes?

[johnameyer]

Oh yep yep SourceConfig is the one.

[iliapolo]

@johnameyer Sounds great. feel free to pick this up whenever you get a chance :)