Skip to content

[RFC 0001] Connection Pooling #33

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Jul 26, 2021
Merged

[RFC 0001] Connection Pooling #33

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions rfcs/0001-connection-pooling.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
---
feature: Connection Pooling
start-date: 2021-02-04
author: kiwicopple
co-authors: steve-chavez, dragarcia
related-issues: (will contain links to implementation PRs)
---

# Summary
[summary]: #summary

We would like to explore connection pooling on Supabase. This RFC is intended to decide:

- Whether we should provide a pooler
- Which connection pooler we should use
- Where in the stack it would be installed - i.e. if should bundle it with the Postgres build


# Motivation
[motivation]: #motivation

In Postgres, every connection is a process. Because of this, a lot of connections to the database can be very expensive on memory.

When connecting to Postgres database from serverless functions, there is no connection pooling, and so the server needs to maintain hundreds/thousands of connections.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Transaction pooling and statement pooling(see pgbouncer types of pooling) might help there. The disadvantage of those is that they cannot use prepared statements for example, so we'd need to check for support on common client libs(too much work, maybe just document it).

Seems DO also recommends transaction pooling: https://assets.digitalocean.com/articles/managed_db_pools/connection_pool_blank.png

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed on this - when I look at the additional tooling around Postgres, pgbouncer seems to be the one supported most frequently (eg, pgmetrics, various postgres operators).



# Detailed design
[design]: #detailed-design

This is still in the "Gather Feedback" stage. To start the discussion:


### 1. Decide on a PG Pooler

- `pg_bouncer` - https://www.pgbouncer.org/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd vote for pgbouncer. Heroku(1) and DO(2) use it.

(1):https://www.digitalocean.com/community/tutorials/managed-databases-connection-pools-and-postgresql-benchmarking-using-pgbench (search for pgbouncer)
(2):https://devcenter.heroku.com/articles/best-practices-pgbouncer-configuration

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's also an AWS pgbouncer fork: https://github.com/awslabs/pgbouncer-rr-patch.

- `PG Pool II` - https://www.pgpool.net/mediawiki/index.php/Main_Page
- `odyssey` - https://github.com/yandex/odyssey
- others?

### 2. Decide on configuration

Most poolers allow different configurations. We would need to decide on how we would configure the pooler by default

### 3. Decide if the user should be able re-configure the pooler

Should a user be able to change the configuration? If so, how would they do it?


# Drawbacks
[drawbacks]: #drawbacks

- Security
- Not directly relevant to the "supabase" stack, so it's additional non-core support
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could add a dedicated GitHub discussion category for "External libraries" and mention it's low prio for us(plus reasoning).


# Alternatives
[alternatives]: #alternatives

1. Since we already offer [PostgREST](https://github.com/postgrest/postgrest) and [postgres-meta](https://github.com/supabase/pg-api), this isn't entirely necessary for the Supabase stack. Bundling this is only beneficial for connecting external tools.
2. We could hold back on this implementation until we move to a full Postgres Operator, which would include a pooler. It would be nice to have something for local development though.


# Unresolved questions
[unresolved]: #unresolved-questions

- Add any unresolved questions here


# Future work
[future]: #future-work

- Add any future work here