feat: add pg_egress_collect service

[burmecia]

What kind of change does this PR introduce?

This PR is to add service for postgres egress metric collection. See more context: https://www.notion.so/supabase/Fix-DB-egress-metric-190164a5c4e5444cbbe20c726cea241d

What is the current behavior?

Currently there is no reliable way to gather accurate database-only egress, so the whole server egress is used instead.

What is the new behavior?

The solution is using tcpdump to capture outgoing TCP packets on port 5432 and 6543, and a perl script extract packet length and sum up to one minute interval. The result is saved to text file at /tmp/pg_egress_collect.txt by default. Admin API then can read metric from that file and expose it to Victoria Metrics.

By using this approach, we can collect accurate network egress for postgres and expose it in admin api so Victoria Metrics can gather for downstream pipeline.

Additional context

This metric will also be as data source for admin api, see more details: https://github.com/supabase/supabase-admin-api/pull/130

[darora]

@burmecia have you collected data on performance overhead for this across a few projects with different utilizations?

[burmecia]

I tried several projects including my personal projects and RevOps project, not much overhead I can see. CPU usage is unnoticeable, memory is about 10M and only write one line to a tmp file per minute.

[burmecia]

Tested its performance on my project, below are the setups:

• Used sql to generate random serial data and read it to local using psql.

  SELECT time, random()*100 as cpu_usage
  FROM generate_series(now() - INTERVAL '12 months',now(),INTERVAL '1 minute') as time

• Kept monitoring egress for about 1 hour
• Total egress is about 5GB
• No other database activities

And the test result is:

• Average CPU usage is around 10% including postgres
• Average memory usage by the egress collector is about 15M (tcpdump + perl script)
• Disk space used: <10 bytes
• EBS IO balance: 99% all the time