Run your Deployment's current Airflow image

Run the current Airflow image

You can run your Deployment’s current Airflow image with the KubernetesPodOperator by using the environment variable, ASTRONOMER_AIRFLOW_IMAGE. This environment variable allows you to run KPO tasks using the same Runtime image that your Deployment uses, in a way that won’t be affected by underlying cluster infrastructure changes that might change the base image repository URL. The ASTRONOMER_AIRFLOW_IMAGE environment variable allows you to ensure that your Deployment retrieves the correct image URL.

Setup

1

Add current Airflow image configuration

Add the ASTRONOMER_AIRFLOW_IMAGE environment variable to your KPO configuration. For example:

1 import os
2 from airflow.configuration import conf
3 from airflow.providers.cncf.kubernetes.operators.pod import KubernetesPodOperator
4 from kubernetes.client import models as k8s
5 KubernetesPodOperator(
6     namespace=conf.get("kubernetes", "NAMESPACE"),
7     image=os.getenv("ASTRONOMER_AIRFLOW_IMAGE"),
8     image_pull_secrets=[
9         k8s.V1LocalObjectReference("image-pull-secret"),
10         k8s.V1LocalObjectReference("per-dp-registry-pull-secret"),
11     ],
12     cmds=["<commands-for-image>"],
13     arguments=["<arguments-for-image>"],
14     labels={"<pod-label>": "<label-name>"},
15     name="<pod-name>",
16     task_id="<task-name>",
17     get_logs=True,
18     in_cluster=True,
19 )

1	import os
2	from airflow.configuration import conf
3	from airflow.providers.cncf.kubernetes.operators.pod import KubernetesPodOperator
4	from kubernetes.client import models as k8s
5	KubernetesPodOperator(
6	namespace=conf.get("kubernetes", "NAMESPACE"),
7	image=os.getenv("ASTRONOMER_AIRFLOW_IMAGE"),
8	image_pull_secrets=[
9	k8s.V1LocalObjectReference("image-pull-secret"),
10	k8s.V1LocalObjectReference("per-dp-registry-pull-secret"),
11	],
12	cmds=["<commands-for-image>"],
13	arguments=["<arguments-for-image>"],
14	labels={"<pod-label>": "<label-name>"},
15	name="<pod-name>",
16	task_id="<task-name>",
17	get_logs=True,
18	in_cluster=True,
19	)