Migrate to Airflow 3 from Airflow 2 in Astro Private Cloud

This guide describes the steps required to migrate your Airflow deployments from Airflow 2 to Airflow 3 in Astro Private Cloud.

1. Enable Airflow 3 on the cluster

1. Navigate to the Cluster Configuration Override page in the Astro Private Cloud UI.
2. Enable the Airflow 3 flag:

1
astronomer:
2
    houston:
3
        deployments:
4
            airflowV3:
5
                enabled: true

3. Enable the mandatory Dag processor component for Airflow 3:

1
astronomer:
2
    houston:
3
        deployments:
4
            dagProcessorEnabled: true

4. Enable Sidecar Logging for Airflow tasks:

1
global:
2
    loggingSidecar:
3
        enabled: true

2. Configure registry access

1. Allowlist the Azure Container Registry (azurecr) repository and URL for pulling Airflow 3 images.
2. If using third-party registries, configure mirroring to ensure reliable image pulls.

3. Check Dag compatibility before migration

Before upgrading, follow the Airflow 2 → 3 upgrade guide to verify that your Dags are compatible with Airflow 3.

4. Update Deployment Dockerfile

1. Modify your Dockerfile to use the Airflow 3 image and updated registry URL:

1
FROM astrocrpublic.azurecr.io/runtime:3.1-2

2. Build and push your updated Docker image.
   Airflow 2 images are available on Quay (quay.io/astronomer/airflow), while Airflow 3 images are hosted on astrocr (astrocrpublic.azurecr.io).

5. Update Deployment

Perform the Deployment update using one of the following methods:

1. UI: Navigate to your Deployment and trigger an update with the new Airflow 3 image.
2. API: Use the UpsertDeployment API to update the Deployment image.
3. CLI: Use the Astro CLI to deploy the updated image.

Ensure the updated Deployment reflects the Airflow 3 runtime version.

6. Validate after migration

• Confirm that Dags are running as expected in Airflow 3.
• Monitor logs and metrics to ensure stability.
• Verify that the Dag processor component is active and functioning.