Skip to content

Automate deploys to Pypi #48

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Sep 22, 2022
Merged

Automate deploys to Pypi #48

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
0e4ecc0
Introduce test deploy to Pypi (borrowed from dbsqlcli)
Sep 6, 2022
f8d9e0b
Add publish workflow
Sep 6, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 64 additions & 0 deletions .github/workflows/publish-test.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
name: Publish to PyPI [Test]
on: [push]
jobs:
test-pypi:
name: Create patch version number and push to test-pypi
runs-on: ubuntu-latest
steps:
#----------------------------------------------
# check-out repo and set-up python
#----------------------------------------------
- name: Check out repository
uses: actions/checkout@v2
- name: Set up python
id: setup-python
uses: actions/setup-python@v2
with:
python-version: 3.9
#----------------------------------------------
# ----- install & configure poetry -----
#----------------------------------------------
- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
installer-parallel: true
#----------------------------------------------
# load cached venv if cache exists
#----------------------------------------------
- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v2
with:
path: .venv
key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-${{ github.event.repository.name }}-${{ hashFiles('**/poetry.lock') }}
#----------------------------------------------
# install dependencies if cache does not exist
#----------------------------------------------
- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --no-interaction --no-root
#----------------------------------------------
# Get the current version and increment it (test-pypi requires a unique version number)
#----------------------------------------------
- name: Get next version
uses: reecetech/[email protected]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this auto increment the version after the release if so to which version?

how does it know to increment the minor version or the major version?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This step has no side-effects.

Understand this is for test-pypi where versions are ephemeral (periodically deleted). The only requirement for the version we push is that it must be unique. This step combines with the following step to generate a guaranteed unique version number for testing purposes.

id: version
with:
scheme: semver
increment: patch
#----------------------------------------------
# Tell poetry to update the version number
#----------------------------------------------
- name: Update pyproject.toml
run: poetry version ${{ steps.version.outputs.major-version }}.${{ steps.version.outputs.minor-version }}.dev$(date +%s)
#----------------------------------------------
# Attempt push to test-pypi
#----------------------------------------------
- name: Build and publish to pypi
uses: JRubics/[email protected]
with:
pypi_token: ${{ secrets.TEST_PYPI_TOKEN }}
repository_name: "testpypi"
repository_url: "https://test.pypi.org/legacy/"
64 changes: 64 additions & 0 deletions .github/workflows/publish.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
name: Publish to PyPI [Production]
on:
release:
types: [published]
jobs:
publish:
name: Publish
runs-on: ubuntu-latest
steps:
#----------------------------------------------
# check-out repo and set-up python
#----------------------------------------------
- name: Check out repository
uses: actions/checkout@v2
- name: Set up python
id: setup-python
uses: actions/setup-python@v2
with:
python-version: 3.9
#----------------------------------------------
# ----- install & configure poetry -----
#----------------------------------------------
- name: Install Poetry
uses: snok/install-poetry@v1
with:
virtualenvs-create: true
virtualenvs-in-project: true
installer-parallel: true
#----------------------------------------------
# load cached venv if cache exists
#----------------------------------------------
- name: Load cached venv
id: cached-poetry-dependencies
uses: actions/cache@v2
with:
path: .venv
key: venv-${{ runner.os }}-${{ steps.setup-python.outputs.python-version }}-${{ github.event.repository.name }}-${{ hashFiles('**/poetry.lock') }}
#----------------------------------------------
# install dependencies if cache does not exist
#----------------------------------------------
- name: Install dependencies
if: steps.cached-poetry-dependencies.outputs.cache-hit != 'true'
run: poetry install --no-interaction --no-root
#------------------------------------------------------------------------------------------------
# Here we use version-increment to fetch the latest tagged version (we won't increment it though)
#------------------------------------------------------------------------------------------------
- name: Get next version
uses: reecetech/[email protected]
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have this in the other publish yaml file to publish to test pypi.

How do we handle this scenario:

  1. we publish to test pypi, hence publish test action increments the version
  2. we want to publish the same binary in step 1 to pypi (do we have to roll back the version manually after step 1. incremented it?)

Copy link
Contributor Author

@susodapop susodapop Sep 20, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pypi does not allow rollbacks. test-pypi is entirely different from Pypi. Pushing a version to one has zero affect on the other.

Note that the output from this step is never used as described in the comments.

The "one true version" is derived from git tags. We're using the reecetech/version-increment action as a simple shortcut to parse the version tags.

id: version
with:
scheme: semver
increment: patch
#-----------------------------------------------------------------------------
# Tell poetry to use the `current-version` that was found by the previous step
#-----------------------------------------------------------------------------
- name: Update pyproject.toml
run: poetry version ${{ steps.version.outputs.current-version }}
#----------------------------------------------
# Attempt push to test-pypi
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

which action triggers a publish to test-pypi?
do we publish on every change on main branch? or does someone need to take a manual action to trigger publish to test-pypi?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's based on the release.published attribute. So the publish.yml only runs when we cut a release on GitHub.

The publish-test.yml runs on every push to a branch on this repository (main or even feature branches).

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we have a doc how we are doing the release?

So the publish.yml only runs when we cut a release on GitHub.
what constitutes cutting a release? what would you do to cut a release?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Internally, yes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The basic steps to create a release are:

  1. tag a commit with a semantic version number
  2. Create a github release using that tag

#----------------------------------------------
- name: Build and publish to pypi
uses: JRubics/[email protected]
with:
pypi_token: ${{ secrets.PROD_PYPI_TOKEN }}