Astronomer CI/CD templates are customizable, pre-built code samples that help you configure automated workflows with popular CI/CD tools, such as GitHub Actions or Jenkins. Use the templates to create a workflow that automates deploying code to Astro according to your team’s CI/CD requirements and strategy.
Template types differ based on the deploy method they use and how many branches or environments they require. This document contains information about the following template types:
/dbt, to make the code accessible in your Deployment.Astronomer maintains a dedicated guide with templates for select CI/CD tools. Most guides include image based templates for a single-branch implementation. Astronomer recommends reconfiguring the templates to work with your own directory structures, tools, and processes.
If you’re interested in documentation for a CI/CD tool or template type that does not exist, configure your own or contact Astronomer support. To learn more about single-branch and multiple-branch implementations and decide which template is right for you, see Choose a CI/CD strategy.
Dag deploy templates check the changes in your Astro project and trigger either a dag deploy or image deploy based on the files changed, allowing for faster deploys. This template deploys your dags when only the files in your dags folder are modified, and it deploys the rest of your Astro project as a Docker image when any other files or directories are modified. To learn more about the benefits of this workflow, see Deploy dags.
CI/CD templates that use the dag deploy workflow:
ASTRO_API_TOKEN environment variable.dags folder have changed, run astro deploy --dags. This pushes your dags folder to your Deployment.dags folder has changed, run astro deploy. This triggers two subprocesses. One that creates a Docker image for your Astro project, authenticates to Astro using your Deployment API token, and pushes the image to your Deployment. A second that pushes your dags folder to your Deployment.If you stage multiple commits to dag files and push them all at once to your remote branch, the template only deploys dag code changes from the most recent commit. It will miss any code changes made in previous commits.
To avoid this, either push commits individually or configure your repository to Squash commits for pull requests that merge multiple commits simultaneously.
This process is equivalent to the following shell script:
Image based templates build a Docker image and push it to Astro whenever you update any file in your Astro project. This type of template works well for development workflows that include complex Docker customization or logic.
CI/CD templates that use image based workflows:
ASTRO_API_TOKEN environment variable.astro deploy command. This creates a Docker image for your Astro project, authenticates to Astro using your Deployment API token, and pushes the image to your Deployment.This is equivalent to running the following shell script:
dbt deploy templates use dbt deploy to deploy just your dbt code to your Astro project, allowing for faster deploys. Or, it uses the Deploy Action with GitHub Actions to deploy dbt code. These templates deploy your dbt code when only the files in your dbt project folder, by default /dbt, are modified.
CI/CD templates that use image based workflows:
ASTRO_API_TOKEN environment variable.astro dbt deploy command from your dbt project directory. This bundles all files in your dbt project and pushes them to Astro, where they are mounted on your Airflow containers so that your dags can access them.This is equivalent to running the following shell script from your dbt directory:
Preview Deployment templates enable a CI/CD workflow that automates creating and deleting Deployments based on feature branches in your Git repository. A preview Deployment is automatically created when the temporary feature branch is created and the Deployment is deleted when the branch is deleted. Astronomer recommends using preview Deployments if you regularly need to test a small set of dags on Astro before promoting those dags to a base, production Deployment. This helps you lower the infrastructure cost of Deployments that are dedicated to development and testing.
To implement this feature, you need a CI/CD workflow that:
If you use GitHub Actions as your CI/CD tool, you can find preview Deployment templates as part of the Astronomer GitHub action in the GitHub Marketplace. This GitHub action includes sub-actions for each of these four steps. To learn more, see GitHub Actions.
To configure your own automated workflow for preview Deployments with another CI/CD tool, use the following scripts. Each of the following four shell scripts is equivalent to the steps required to implement this feature with GitHub Actions.
In a Deployment preview CI/CD pipeline, you run this script when you create a feature branch off of the main branch of your Astro project.
In a Deployment preview CI/CD pipeline, you run this script whenever you make changes in your feature branch.
In a Deployment preview CI/CD pipeline, you run this script when you delete your feature branch.
In a Deployment preview CI/CD pipeline, you run this script when you merge your feature branch into your main branch.