Skip to content

update loadbalancer logs readme to be generic #21

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Nov 21, 2022
Merged

update loadbalancer logs readme to be generic #21

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
82161b0
update loadbalancer logs readme to be generic
mjayaram Nov 18, 2022
8212179
Merge branch 'main' into readme-updates
mjayaram Nov 21, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 14 additions & 9 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,26 +2,27 @@

[![CI](https://github.com/honeycombio/terraform-aws-integrations/actions/workflows/test-terraform-module.yml/badge.svg)](https://github.com/honeycombio/terraform-aws-integrations/actions?query=Test%20Terraform%20Module)

This repo contains a set of modules in the [modules folder](https://github.com/honeycombio/terraform-aws-integrations/tree/main/modules) for resources
in [AWS](https://aws.amazon.com/) using [Terraform](https://www.terraform.io/) to send observability data to [Honeycomb](https://www.honeycomb.io/).
This repo contains a set of modules in
the [modules folder](https://github.com/honeycombio/terraform-aws-integrations/tree/main/modules) for resources
in [AWS](https://aws.amazon.com/) using [Terraform](https://www.terraform.io/) to send observability data
to [Honeycomb](https://www.honeycomb.io/).

## How does this work?

![AWS Integrations architecture](docs/overview.png?raw=true)


**Integrations supported are:**

* [CloudWatch Logs](https://github.com/honeycombio/terraform-aws-integrations/tree/main/modules/cloudwatch-logs)


* [CloudWatch Metrics](https://github.com/honeycombio/terraform-aws-integrations/tree/main/modules/cloudwatch-metrics)


* [S3 Logfile](https://github.com/honeycombio/terraform-aws-integrations/tree/main/modules/s3-logfile)
* [RDS Logs](https://github.com/honeycombio/terraform-aws-integrations/tree/main/modules/rds-logs)


* [RDS Logs](https://github.com/honeycombio/terraform-aws-integrations/tree/main/modules/rds-logs)
* [Logs from a S3 Bucket](https://github.com/honeycombio/terraform-aws-integrations/tree/main/modules/s3-logfile)


## Use
Expand Down Expand Up @@ -62,7 +63,9 @@ Set the API key used by Terraform setting the HONEYCOMB_API_KEY environment vari
export TF_VAR_HONEYCOMB_API_KEY=$HONEYCOMB_API_KEY
```

Setup AWS credentials for the intended AWS account where the resources will be created and managed. Please see for more [details and options](https://registry.terraform.io/providers/hashicorp/aws/latest/docs#authentication-and-configuration).
Setup AWS credentials for the intended AWS account where the resources will be created and managed. Please see for
more [details and options](https://registry.terraform.io/providers/hashicorp/aws/latest/docs#authentication-and-configuration)
.

```bash
export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
Expand Down Expand Up @@ -93,7 +96,9 @@ in [`tests/`](https://github.com/honeycombio/terraform-aws-integrations/tree/mai
export TF_VAR_HONEYCOMB_API_KEY=$HONEYCOMB_API_KEY
```

3. Setup AWS credentials. Please see for more [details and options](https://registry.terraform.io/providers/hashicorp/aws/latest/docs#authentication-and-configuration).
3. Setup AWS credentials. Please see for
more [details and options](https://registry.terraform.io/providers/hashicorp/aws/latest/docs#authentication-and-configuration)
.

```bash
export AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
Expand Down Expand Up @@ -128,4 +133,4 @@ You can run `make terraform-format` to auto fix formatting issues.
Features, bug fixes and other changes to this module are gladly accepted. Please open issues or a pull request with your
change.

All contributions will be released under the Apache License 2.0.
All contributions will be released under the Apache License 2.0.
Binary file added docs/s3-logs-overview.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
20 changes: 10 additions & 10 deletions modules/s3-logfile/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,32 +1,32 @@
# TERRAFORM AWS S3 LOGS IN A BUCKET


This repo contains a module for resources in [AWS](https://aws.amazon.com/) using [Terraform](https://www.terraform.io/)
to send logs from AWS Load Balancers to [Honeycomb](https://www.honeycomb.io/).
to send logs from an AWS S3 Bucket to [Honeycomb](https://www.honeycomb.io/).

## How does this work?

![AWS CloudWatch Logs Integration overview](../../docs/s3-logfile-overview.png)
![AWS Logs from a bucket Integration overview](../../docs/s3-logs-overview.png)

All required resources to setup an integration pipelines to logs from a S3 bucket and send them to
Honeycomb can be created and managed via this module.
All required resources to setup an integration pipelines to take logs from a S3 bucket and send them to
Honecyomb can be created and managed via this module.


## Use

The minimal config is:

```hcl
module "alb_logs" {
source = "./modules/s3-logfile"
module "logs_from_a_bucket_integrations" {
source = "honeycombio/integrations/aws//s3-logfile"
name = var.logs_integration_name

name = var.lb_logs_integration_name
parser_type = var.parser_type # valid types are alb, elb, cloudfront, vpc-flow-log, s3-access, json, and keyval
s3_bucket_arn = var.s3_bucket_arn // The full ARN of the bucket storing the logs.

s3_bucket_arn = var.s3_bucket_arn // The full ARN of the bucket storing load balancer access logs.
parser_type = "alb" // alb, elb, cloudfront, vpc-flow-log, s3-access, json, or keyval

honeycomb_api_key = var.honeycomb_api_key // Your Honeycomb team's API key.
honeycomb_dataset_name = "alb-logs" // Your Honeycomb dataset name that will receive the metrics.

}
```

Expand Down