dbt Cloud is a SaaS product that runs SQL-first transformation workflows. Integrating dbt Cloud with Airflow allows you to trigger dbt cloud jobs and check their status from an Airflow DAG.
This guide provides the basic setup for creating a dbt Cloud Airflow connection. For a complete integration tutorial, see Orchestrate dbt Cloud jobs with Airflow. To run your dbt core jobs using Airflow, see Orchestrate dbt-core Jobs with Airflow.
A connection from Airflow to dbt Cloud requires the following information:
Complete the following steps to retrieve these values:
In the dbt Cloud UI, copy the URL of your dbt Cloud account. It should be formatted as https://cloud.getdbt.com. Your URL might be different based on the hosted region of your dbt Cloud. See dbt Cloud URIs for more details.
If you’re using a dbt Developer account, follow the dbt documentation to copy the API token for your user account. To generate a token for a service account, see Generating service account tokens.
To retrieve the Account ID of your dbt account, go to the settings page in dbt Cloud UI. Then, from the URL of the settings page, copy the account ID, which is an integer appearing after accounts. For example, in the URL https://cloud.getdbt.com/settings/accounts/88348, the account ID is 88348.
Astro users can also create connections using the Astro Environment Manager, which stores connections in an Astro-managed secrets backend. These connections can be shared across multiple deployed and local Airflow environments. See Create Airflow connections in the Astro UI.
Open your Astro project and add the following line to your requirements.txt file:
This will install the dbt Cloud provider package, which makes the dbt Cloud connection type available in Airflow.
Run astro dev restart to restart your local Airflow environment and apply your changes in requirements.txt.
In the Airflow UI for your local Airflow environment, go to Admin > Connections. Click + to add a new connection, then select the connection type as dbt Cloud.
Fill out the following connection fields using the information you retrieved from Get connection details:
Click Test. After the connection test succeeds, click Save.
Note that the connection test only tests the Tenant and API Token fields. It will not validate your Account ID.
Airflow uses Python’s requests library to connect to dbt Cloud through the dbtHook.