One of the main features of Apache Airflow is its user interface (UI), which provides insights into your DAGs and DAG runs. The UI is essential for understanding, monitoring, and troubleshooting your pipelines.
This guide is an overview of some of the most useful features and visualizations in the Airflow UI. If you’re not already using Airflow and want to get it up and running to follow along, see Install the Astro CLI to quickly run Airflow locally.
All images in this guide were taken from an Astronomer Runtime Airflow image. Other than some modified colors and an additional Astronomer tab, the UI is the same as when using OSS Airflow. The images in this guide are from Airflow version 2.10. If you are using an older version of Airflow, some UI elements might be slightly different or missing.
There are multiple resources for learning about this topic. See also:
To get the most out of this guide, you should have an understanding of:
The DAGs view is the landing page when you sign in to Airflow. It shows a list of all your DAGs, the status of recent DAG runs and tasks, the time of the last DAG run, and basic metadata about the DAG like the owner and the schedule. To see the status of the DAGs update in real time, toggle Auto-refresh.
In the DAGs view you can:
Actions section.Links section.To see more information about a specific DAG, click its name or use one of the links.
The DAG view gives you detailed insights into a specific DAG, including its DAG runs and task instances.
On the left side you can see a grid representation of the DAG’s previous runs, including their duration and the outcome of all individual task instances. Each column represents a DAG run, and each square represents a task instance in that DAG run. Task instances are color-coded according to their status. A small play icon on a DAG run indicates that a run was triggered manually, and a small dataset icon shows that a run was triggered by a dataset update. If no icon is shown, the DAG ran according to its schedule.
On the right side you can see further details about the item (DAG, DAG run or task instance) that is currently selected.
When a DAG run, task instance, or task group instance is selected in the DAG grid, several action buttons appear:
There are several tabs available within the DAG view:
In Airflow 2.7 and later, the DAG view includes keyboard shortcuts. You can see all available shortcuts by pressing shift + / while in the DAG view.
The Details tab displays information about the DAG, individual DAG runs, and in-depth information about each task instance. Here you can find information like total historic runs of a DAG, the data interval start of a DAG run, and the duration of a task instance.
To access the details of a specific DAG run or task instance, you first need to select it in the DAG grid as shown in the following gif:
If a task has been retried, you can also click on a task try button to get details about a specific attempt:
When you select a task instance in the DAG grid, two additional options appear underneath the tabs:
The Graph tab shows a graph visualization of the tasks and dependencies in your DAG, including Airflow datasets a DAG is scheduled on or updates. If you select a task or task group instance in a DAG grid column, the graph highlights and zooms to the selected task. You can also navigate complex DAGs using the Filter Tasks option and the minimap. This view is useful for exploring the DAG structure and task dependencies.
Earlier Airflow versions had a different Graph view that was not integrated into the DAG view. See the Airflow documentation of your version for more information.
The Gantt chart tab visualizes task duration and overlap, so you can quickly identify bottlenecks and costly tasks.
You can also find information about previous retries if a task has been retried:
Under the Code tab you can access the code that generates the DAG you are viewing. While your code should live in source control, the Code tab provides a quick insight into what is going on in the DAG. DAG code can’t be edited in the UI.
This tab shows code only from the file that generated the DAG. It does not show any code that may be imported in the DAG, such as custom hooks or operators or code in your /include directory.
The Audit Logs tab shows a list of events that have occurred in your Airflow environment relating to the DAG, DAG run or task instance you have selected.
The Run Duration tab shows a bar chart of the duration of each DAG run over time.
The K8s Pod Spec tab displays information about a task’s K8s pod if a task has been run on one.
The Task Duration tab shows a line chart of the duration of each task across all DAG runs.
To access the logs of a specific task instance, click on the Logs tab which appears as soon as you select a task instance. By default the logs of the task execution are shown, while the Pre task execution logs and the Post task execution logs are collapsed and can be expanded by clicking on the respective log item.
Click on the Logs tab to see the logs for a selected task. If a task has been retried, you can click on a task try button to see the logs for that attempt. Hover over a button for status and duration information at a glance.
The cluster activity tab shows aggregated metrics for the entire Airflow cluster. It includes live metrics, such as currently occupied slots in different pools, unpaused DAGs, and scheduler health. It also includes historical metrics like the states of past DAG runs and task instances, as well as how each DAG run was triggered.
The Dataset tab links to a page showing all datasets that have been produced in the Airflow environment, with additional tabs offering:
Click on a dataset to open the history of all updates to the dataset that were recorded in the Airflow environment. You can use the Play button to manually trigger an update to a Dataset.
Click on the Dependency graph tab to view a holistic graph of datasets and DAGs.
On Astro, role-based access control is managed at the platform level. As a result, the Security tab is not needed and is not available on Airflow deployments on Astro.
The Security tab links to multiple pages, including List Users and List Roles, that you can use to review and manage Airflow role-based access control (RBAC). For more information on working with RBAC, see Security.
If you are running Airflow on Astronomer, the Astronomer RBAC will extend into Airflow and take precedence. There is no need for you to use Airflow RBAC in addition to Astronomer RBAC. Astronomer RBAC can be managed from the Astronomer UI, so the Security tab might be less relevant for Astronomer users.
The Browse tab links to multiple pages that provide additional insight into and control over your DAG runs and task instances for all DAGs in one place.
The DAG runs and task instances pages are the easiest way to view and manipulate these objects in aggregate. If you need to re-run tasks in multiple DAG runs, you can do so from this page by selecting all relevant tasks and clearing their status.
The DAG Dependencies view shows a graphical representation of any cross-DAG and dataset dependencies in your Airflow environment.
Other views on the Browse tab include:
The Admin tab links to pages for content related to Airflow administration that are not specific to any particular DAG. Many of these pages can be used to both view and modify your Airflow environment.
For example, the Connections page shows all Airflow connections stored in your environment. Click + to add a new connection. For more information, see Managing your Connections in Apache Airflow.
Similarly, the XComs page shows a list of all XComs stored in the metadata database and allows you to easily delete them.
Other pages on the Admin tab include:
airflow.cfg file. Note that this can be disabled by your Airflow admin for security reasons.The Docs tab provides links to external Airflow resources including:
In the upper right-hand corner of the UI, you will find a light/dark theme toggle button, time zone selector with search functionality, and a login dropdown with a link to an editable user profile view.
This guide provided a basic overview of some of the most commonly used features of the Airflow UI.
The Airflow community is consistently working on improvements to the UI to provide a better user experience and additional functionality. Make sure you upgrade your Airflow environment frequently to ensure you are taking advantage of Airflow UI updates as the community releases them.