Export logs to a Secondary WASB Container

Export logs to a Secondary WASB Container

By forwarding Airflow task logs from your Astro Deployment to an additional, customer‑managed Azure Storage (WASB) container, you can keep redundant copies of your execution history, integrate with existing log‑processing pipelines, and satisfy compliance or retention requirements that extend beyond the built‑in Astro logs.

Prerequisites

• Your Deployment must run Astro Runtime 11.7.0 or later. See Upgrade Astro Runtime.
• Your image must include astronomer-providers-logging==1.5.4 or later. The secondary WASB logging feature was introduced in this version.
• An Azure Storage Account with a container created to hold the logs.
• An Azure Managed Identity (or Service Principal) that has read/write access to the container.

Required environment variables

Add these environment variables in the Deployment.

1
AIRFLOW__ASTRO_SECONDARY_LOGS__WASB_ENABLED=true
2
AIRFLOW__ASTRO_SECONDARY_LOGS__WASB_TENANT_ID="<YOUR_TENANT_ID>"
3
AIRFLOW__ASTRO_SECONDARY_LOGS__WASB_CLIENT_ID="<YOUR_CLIENT_ID>"
4
AIRFLOW__ASTRO_SECONDARY_LOGS__WASB_STORAGE_ACCOUNT="<YOUR_STORAGE_ACCOUNT_NAME>"
5
AIRFLOW__ASTRO_SECONDARY_LOGS__WASB_CONTAINER="<YOUR_CONTAINER_NAME>"

Step 1: Create Azure resources

1. Create a Storage Account.
2. Inside the account, create a blob container to hold the logs.

Step 2: Configure Astro Workload Identity

Follow Authorize Deployments to your Azure cloud to assign a Managed Identity to the Deployment.

Step 3: Grant storage permissions

In Azure, assign the Managed Identity the Storage Blob Data Contributor or equivalent scoped role to the Storage Account or container.

Step 4: Add environment variables and redeploy

Set the environment variables listed in required environment variables and redeploy.

Troubleshooting

If task logs do not appear in the container:

1. Confirm that all required environment variables are set and have no extra quotes or spaces.
2. Verify that the Managed Identity has read/write access to the Storage Account and container.
3. Check Deployment logs for authentication or authorization errors related to Azure.
4. Ensure the container exists, Astro will not create it automatically.

If issues persist, contact Astronomer Support with the Deployment ID and any relevant error output.