The Astronomer Certified Docker image for Apache Airflow extends the community-developed Airflow image in a way that makes running Airflow more secure, reliable, and extensible. It is the default image for Airflow Deployments on Astronomer.
This guide provides reference information for the building blocks of Astronomer Certified, as well as information on its release and distribution.
Astronomer Certified is distributed both as a Python wheel and a Debian-based Docker image. These distributions vary slightly in scope and dependencies.
The Python wheel and Docker image are functionally identical to open source Apache Airflow. Additionally, they both include performance and stability improvements, including critical bug fixes and security patches.
The Astronomer Certified Docker image is built from the Python wheel and incorporates additional logic that makes it easier for users to both get started and run Airflow at scale. This includes:
Every supported version of the Astronomer Certified Python wheel is available at pip.astronomer.io. The Dockerfiles for all supported Astronomer Certified images can be found in Astronomer’s ap-airflow GitHub repository.
Running Airflow with the Astronomer Certified Docker image requires specific versions for key system components.
While the Astronomer Certified Python Wheel supports Python versions 3.6, 3.7, and 3.8, Astronomer Certified Docker images have been tested and built only with Python 3.7. These requirements are slightly different for running only the Python wheel. For the Python wheel, you can use:
For more information on running a Python wheel installation of Astronomer Certified, read Install on a Virtual Machine.
When an Airflow service is started, it checks a file for runtime environment variables. These are equivalent to values defined in Airflow’s airflow.cfg file.
If you run the Astronomer Certified Docker image without the Astronomer platform, environment variables are defined in your Dockerfile. They can be overwritten with a runtime command, such as docker run.
If you’re running the Astronomer Certified Docker image with the Astronomer platform, there are a few ways you can configure environment variables. For more information, read Environment Variables.
Astronomer Certified supports the same environment variables as Apache Airflow. For a list of all configurable environment variables, read the Apache Airflow documentation.
The following table lists the essential environment variables used when running Airflow with Astronomer Certified. Only environment variables explicitly defined in the Dockerfile are listed here; all other environment variables have the same default value as they have in OSS Airflow.
Starting in version 2.0.0, the Astronomer Certified image includes provider packages that are utilized in some background processes, as well as packages which are commonly used by the Airflow community. The following table contains version information for each provider package installed as part of Astronomer Certified:
The Astronomer Certified Docker image includes a number of OS-level dependencies for running basic system processes. These dependencies can be installed in the Python Wheel as described in Install Packages.
Astronomer Certified includes a few packages that don’t have a corresponding provider. These packages are used for basic system functions or optional Airflow functionality. The following list contains all extra packages built into Astronomer Certified by default:
async: Provides asynchronous workers for Gunicornpassword: Adds support for user password hashingstatsd: Adds support for sending metrics to StatsDvirtualenv: Adds support for running Python tasks in local virtual environments