Whether you’re just starting to use Apache Airflow or your team is running it at scale, incorporating an alerting framework is critical to the health of your data. On Astronomer Software, users have access to three types of alerting solutions:
This guide focuses on configuring task and DAG-level alerts. For information on configuring platform and Deployment-level alerts, read Alerting on Astronomer Software.
As an Airflow user, you can configure event-based email alerts directly in your DAG code by leveraging Airflow’s email util. Depending on your use case, you may choose to be notified if a particular task or DAG fails, succeeds, retries, etc. On Astronomer, setting up task-level alerts requires configuring an SMTP service to handle the delivery of these emails.
If your team isn’t already using an SMTP service, we recommend the following:
Step-by-step instructions on how to integrate these two services with Astronomer are provided below, but you can use any SMTP service for this purpose.
SendGrid is an email delivery service that’s easy to set up for Airflow task-level alerts. A free SendGrid account grants users 40,000 free emails within the first 30 days of an account opening and 100 emails per day after that. This should be more than enough emails for most alerting use cases.
To get started with SendGrid:
Create a SendGrid account. Be prepared to disclose some standard information about yourself and your organization.
Verify a Single Sender Identity. Because you’re sending emails only for internal administrative purposes, a single sender identity is sufficient for integrating with Astronomer. The email address you verify here is used as the sender for your Airflow alert emails.
Create a Sendgrid API key. In SendGrid, go to Email API > Integration Guide. Follow the steps to generate a new API key using SendGrid’s Web API and cURL.
For more information, see Sendgrid documentation.
Skip the step for exporting your API key to your development environment. Instead, execute the generated curl code directly in your command line, making sure to replace $SENDGRID_API_KEY in the --header field with your copied key.
Verify your integration in SendGrid to confirm that the key was activated. If you get an error indicating that SendGrid can’t find the test email, try rerunning the cURL code in your terminal before retrying the verification.
In the Deployment view of the Astronomer Software UI, create an environment variable with the following values:
AIRFLOW__EMAIL__EMAIL_BACKENDairflow.providers.sendgrid.utils.emailer.send_emailFor more information on this environment variable, see Airflow documentation.
In the Airflow UI, create an Airflow connection with the following values:
smtp_defaultEmailsmtp.sendgrid.netapikey<your-sendgrid-api-key>587Click Save to finalize your configuration.
To begin receiving Airflow alerts via email for task failures within a given DAG, configure the following values in the DAG’s default_args:
This setup requires an AWS account and use of the AWS Management Console.
In the AWS Management Console, go to AWS Console > Simple Email Service > Email Addresses to add and verify the email addresses you want to receive alerts.
Open the inbox of each email address you specified and verify them through the emails sent by Amazon.
In the AWS Console, go to Simple Email Service > SMTP Settings and use the Create My SMTP Credentials button to generate a username and password. This will look similar to an access and secret access key. Write down this username and password for step 5, as well as the Server Name and Port.
Choose an Amazon EC2 region to use, then write down the code of this server for the next step. Refer to Amazon’s list of available regions and servers to determine which server best fits your needs.
In the Deployment view of the Astronomer Software UI, create an environment variable with the following values:
AIRFLOW__EMAIL__EMAIL_BACKENDairflow.providers.amazon.aws.utils.emailer.send_emailFor more information on this environment variable, see Airflow documentation.
In the Airflow UI, create an Airflow connection with the following values:
smtp_defaultEmail<your-smtp-host><your-aws-username><your-aws-password>587Click Save to finalize your configuration.
To begin receiving Airflow alerts via email for task failures within a given DAG, configure the following values in the DAG’s default_args:
By default, email alerts configured via the email_on_failure param (source) are handled at the task level. If some number of your tasks fail, you’ll receive an individual email for each of those failures.
If you want to limit failure alerts to the DAG-run level, you can instead set up your alerts using the on_failure_callback param (source). When you pass on_failure_callback directly in your DAG file, it defines a Python function that sends you one email per DAG failure, rather than multiple emails for each task that fails:
The code in your DAG might look something like this (source):