GitHub Actions templates for deploying code to preview Deployments on Astro

The Astro GitHub integration can automatically deploy code from a GitHub repository to Astro without you needing to configure a GitHub action. In addition, the Astro UI shows Git metadata for each deploy on your Deployment information screen. See Deploy code with the Astro GitHub integration for setup steps.

The Astronomer deploy action includes several sub-actions that can be used together to create a complete Deployment preview pipeline, a configuration that allows you to test your code changes in an ephemeral development Deployment before promoting your changes to a production Astro Deployment.

The Deployment preview templates use GitHub secrets to manage the credentials needed for GitHub to authenticate to Astro. You can specify the credentials for your secrets backend so that preview Deployments have access to secret Airflow variables or connections during tests. See Deployment preview template with secrets backend implementation.

Deployment preview templates use Astronomer’s deploy-action to automates the deploy process, meaning it can selectively deploy parts of your project based on which files you changed. See Standard deploy templates for more information about the deploy-action.

Prerequisites

An Astro project hosted in a GitHub repository.
An Astro Deployment.
A Workspace API token or Organization API token.
Access to GitHub Actions.

Specific templates might have additional requirements.

Creating preview Deployments for Deployments that use a private image registry is currently unsupported.

If you use a self-hosted runner to execute jobs from GitHub Actions, the Astro CLI’s config.yaml file, which stores default deploy details, might be shared across your organization and hence multiple CI/CD pipelines. To reduce the risk of accidentally deploying to the wrong Deployment, ensure the following:

Add ASTRO_API_TOKEN to your repository and include a check in your GitHub workflow to verify that it exists.
Specify deployment-id or deployment-name in your action. For example, astro deploy <deployment-id> or astro deploy -n <deployment-name>.
Add the command astro logout at the end of your workflow to ensure that your authentication token is cleared from the config.yaml file.

Deployment preview template

The standard Deployment preview template uses GitHub secrets and an Astro Workspace or Organization API token to create a preview Deployment whenever you create a new feature branch off of your main branch.

Setup

Copy and save the Deployment ID for your Astro Deployment.

Replace <main-deployment-id> with this Deployment ID in all the scripts created in the following steps. Even though some scripts take action on the preview Deployment, the <main-deployment-id> should be same for each script.

Set the following GitHub secret in the repository hosting your Astro project:

Key: ASTRO_API_TOKEN
Secret: <your-token>

In your project repository, create a new YAML file in .github/workflows named deploy-to-preview.yml that includes the following configuration:

1 name: Astronomer CI - Deploy code to preview
2 
3 on:
4   pull_request:
5     branches:
6       - main
7 
8 env:
9   ## Set your API token as a GitHub secret
10   ASTRO_API_TOKEN: ${{ secrets.ASTRO_API_TOKEN }}
11 
12 jobs:
13   deploy:
14     runs-on: ubuntu-latest
15     steps:
16     - name: Create preview Deployment
17       uses: astronomer/deploy-action@v0.13.0
18       continue-on-error: true  # Create action fails if deploy preview already exist, which is expected for subsequent commits in the PR
19       with:
20         action: create-deployment-preview
21         deployment-id: <main-deployment-id>
22         wait-time: 10m # Max wait time for the preview deployment to be completed by the deploy action. The workflow will fail if the deployment fails to create within give time period.
23     - name: Deploy code to preview
24       uses: astronomer/deploy-action@v0.13.0
25       with:
26         action: deploy-deployment-preview
27         deployment-id: <main-deployment-id>

In the same folder, create a new YAML file named delete-preview-deployment.yml that includes the following configuration:

1 name: Astronomer CI - Delete Preview Deployment
2 
3 on:
4   delete:
5     branches:
6       - "**"
7 env:
8   ## Set your API token as a GitHub secret
9   ASTRO_API_TOKEN: ${{ secrets.ASTRO_API_TOKEN }}
10 
11 jobs:
12   deploy:
13     runs-on: ubuntu-latest
14     steps:
15     - name: Delete preview Deployment
16       uses: astronomer/deploy-action@v0.13.0
17       with:
18         action: delete-deployment-preview
19         deployment-id: <main-deployment-id>

In the same folder, create a new YAML file named deploy-to-main-deployment.yml that includes the following configuration:

1 name: Astronomer CI - Deploy code to main Deployment
2 
3 on:
4   push:
5     branches:
6       - main
7 
8 env:
9   ## Set your API token as a GitHub secret
10   ASTRO_API_TOKEN: ${{ secrets.ASTRO_API_TOKEN }}
11 
12 jobs:
13   deploy:
14     runs-on: ubuntu-latest
15     steps:
16     - name: Deploy code to main Deployment
17       uses: astronomer/deploy-action@v0.13.0
18       with:
19         deployment-id: <main-deployment-id>

(Optional) You can add optional configurations to customize your workflow.

All three workflow files must have the same Deployment ID specified. The actions use this Deployment ID to create and delete preview Deployments based on your main Deployment.

Deployment preview template with secrets backend implementation

If you use a secrets backend to manage Airflow objects such as variables and connections, you can configure your action to grant preview Deployments access to your secrets backend. This means that dags in the preview Deployment can access your secret Airflow objects for testing purposes.

This template makes use of the AIRFLOW__SECRETS__BACKEND_KWARGS environment variable to store information and credentials for your secrets backend.

Prerequisites

A secrets backend, such as Hashicorp Vault.

Setup

Copy and save the Deployment ID for your Astro deployment.

Set the following GitHub secrets in the repository hosting your Astro project. This includes your Astro API Token, so that GitHub has permissions to deploy code to your Deployments or Workspaces, and your secrets backend information stored in AIRFLOW__SECRETS__BACKEND_KWARGS. See Configure a secrets backend for more information about configuring your secrets backend as an environment variable.

Key 1: ASTRO_API_TOKEN
Secret 1: <your-token>
Key 2: AIRFLOW__SECRETS__BACKEND_KWARGS
Secret 2: <your-kwargs>

In your project repository, create a new YAML file in .github/workflows named create-deployment-preview.yml that includes the following configuration.

1 name: Astronomer CI - Create preview Deployment with Secrets Backend
2 
3 on:
4   create:
5     branches:
6       - "**"
7 
8 env:
9   ## Sets Deployment API token credentials as environment variables
10   ASTRO_API_TOKEN: ${{ secrets.ASTRO_API_TOKEN }}
11 
12 jobs:
13   deploy:
14     runs-on: ubuntu-latest
15     steps:
16     - steps:
17       - name: Create Deployment Preview
18         uses: astronomer/deploy-action@v0.13.0
19         with:
20           action: create-deployment-preview
21           deployment-name: "test"
22           id: create-dep-prev
23       - name: Create Secret Variables
24         run: |
25           astro deployment variable update --deployment-id ${{steps.create-dep-prev.outputs.preview-id}} AIRFLOW__SECRETS__BACKEND_KWARGS=${{ secrets.AIRFLOW__SECRETS__BACKEND_KWARGS }} --secret

In the same folder, create a new YAML file named deploy-to-preview.yml that includes the following configuration:

1 name: Astronomer CI - Deploy code to preview
2 
3 on:
4   pull_request:
5     branches:
6       - main
7 
8 env:
9   ## Set your API token as a GitHub secret
10   ASTRO_API_TOKEN: ${{ secrets.ASTRO_API_TOKEN }}
11 
12 jobs:
13   deploy:
14     runs-on: ubuntu-latest
15     steps:
16     - name: Deploy code to preview
17       uses: astronomer/deploy-action@v0.13.0
18       with:
19         action: deploy-deployment-preview
20         deployment-id: <main-deployment-id>

In the same folder, create a new YAML file named delete-preview-deployment.yml that includes the following configuration:

1 name: Astronomer CI - Delete Preview Deployment
2 
3 on:
4   delete:
5     branches:
6       - "**"
7 env:
8   ## Set your API token as a GitHub secret
9   ASTRO_API_TOKEN: ${{ secrets.ASTRO_API_TOKEN }}
10 
11 jobs:
12   deploy:
13     runs-on: ubuntu-latest
14     steps:
15     - name: Delete preview Deployment
16       uses: astronomer/deploy-action@v0.13.0
17       with:
18         action: delete-deployment-preview
19         deployment-id: <main-deployment-id>

In the same folder, create a new YAML file named deploy-to-main-deployment.yml that includes the following configuration:

1 name: Astronomer CI - Deploy code to main Deployment
2 
3 on:
4   push:
5     branches:
6       - main
7 
8 env:
9   ## Set your API token as a GitHub secret
10   ASTRO_API_TOKEN: ${{ secrets.ASTRO_API_TOKEN }}
11 
12 jobs:
13   deploy:
14     runs-on: ubuntu-latest
15     steps:
16     - name: Deploy code to main Deployment
17       uses: astronomer/deploy-action@v0.13.0
18       with:
19         deployment-id: <main-deployment-id>

All four workflow files must have the same Deployment ID specified. The actions use this Deployment ID to create and delete preview Deployments based on your main Deployment.

1	name: Astronomer CI - Deploy code to preview
2
3	on:
4	pull_request:
5	branches:
6	- main
7
8	env:
9	## Set your API token as a GitHub secret
10	ASTRO_API_TOKEN: ${{ secrets.ASTRO_API_TOKEN }}
11
12	jobs:
13	deploy:
14	runs-on: ubuntu-latest
15	steps:
16	- name: Create preview Deployment
17	uses: astronomer/deploy-action@v0.13.0
18	continue-on-error: true # Create action fails if deploy preview already exist, which is expected for subsequent commits in the PR
19	with:
20	action: create-deployment-preview
21	deployment-id: <main-deployment-id>
22	wait-time: 10m # Max wait time for the preview deployment to be completed by the deploy action. The workflow will fail if the deployment fails to create within give time period.
23	- name: Deploy code to preview
24	uses: astronomer/deploy-action@v0.13.0
25	with:
26	action: deploy-deployment-preview
27	deployment-id: <main-deployment-id>

1	name: Astronomer CI - Delete Preview Deployment
2
3	on:
4	delete:
5	branches:
6	- "**"
7	env:
8	## Set your API token as a GitHub secret
9	ASTRO_API_TOKEN: ${{ secrets.ASTRO_API_TOKEN }}
10
11	jobs:
12	deploy:
13	runs-on: ubuntu-latest
14	steps:
15	- name: Delete preview Deployment
16	uses: astronomer/deploy-action@v0.13.0
17	with:
18	action: delete-deployment-preview
19	deployment-id: <main-deployment-id>

1	name: Astronomer CI - Deploy code to main Deployment
2
3	on:
4	push:
5	branches:
6	- main
7
8	env:
9	## Set your API token as a GitHub secret
10	ASTRO_API_TOKEN: ${{ secrets.ASTRO_API_TOKEN }}
11
12	jobs:
13	deploy:
14	runs-on: ubuntu-latest
15	steps:
16	- name: Deploy code to main Deployment
17	uses: astronomer/deploy-action@v0.13.0
18	with:
19	deployment-id: <main-deployment-id>

1	name: Astronomer CI - Create preview Deployment with Secrets Backend
2
3	on:
4	create:
5	branches:
6	- "**"
7
8	env:
9	## Sets Deployment API token credentials as environment variables
10	ASTRO_API_TOKEN: ${{ secrets.ASTRO_API_TOKEN }}
11
12	jobs:
13	deploy:
14	runs-on: ubuntu-latest
15	steps:
16	- steps:
17	- name: Create Deployment Preview
18	uses: astronomer/deploy-action@v0.13.0
19	with:
20	action: create-deployment-preview
21	deployment-name: "test"
22	id: create-dep-prev
23	- name: Create Secret Variables
24	run: \|
25	astro deployment variable update --deployment-id ${{steps.create-dep-prev.outputs.preview-id}} AIRFLOW__SECRETS__BACKEND_KWARGS=${{ secrets.AIRFLOW__SECRETS__BACKEND_KWARGS }} --secret