Run images from Google Artifact Registry

Request the Compute Engine default service account ID

Contact Astronomer support to request the Compute Engine default service account ID for your cluster.

Add Google Artifact Registry repository permissions

1. Log in to Google Artifact Registry.
2. Click the checkbox next to the repository that you want to use.
3. In the Properties pane that appears, click ADD PRINCIPAL in the PERMISSIONS tab.
4. In the Add Principals text box, paste the Compute Engine default service account ID that was provided to you by Astronomer Support.
5. In the Assign Roles selector, search for Artifact Registry Reader and select the role that appears.
6. Click Save to grant read access for the registry to Astro.

Set up the KubernetesPodOperator

The following snippet is the minimum configuration you’ll need to create a KubernetesPodOperator task on Astro:

1
from airflow.configuration import conf
2
from airflow.providers.cncf.kubernetes.operators.pod import KubernetesPodOperator
3
4
namespace = conf.get("kubernetes", "NAMESPACE")
5
6
KubernetesPodOperator(
7
    namespace=namespace,
8
    image="<your-docker-image>",
9
    cmds=["<commands-for-image>"],
10
    arguments=["<arguments-for-image>"],
11
    labels={"<pod-label>": "<label-name>"},
12
    name="<pod-name>",
13
    task_id="<task-name>",
14
    get_logs=True,
15
    in_cluster=True,
16
)

For each instantiation of the KubernetesPodOperator, you must specify the following values:

• namespace = conf.get("kubernetes", "NAMESPACE"): Every Deployment runs on its own Kubernetes namespace within a cluster. Information about this namespace can be programmatically imported as long as you set this variable.
• image: This is the Docker image that the operator will use to run its defined task, commands, and arguments. Astro assumes that this value is an image tag that’s publicly available on Docker Hub. To pull an image from a private registry, see Pull images from a Private Registry.
• in_cluster: If a Connection object is not passed to the KubernetesPodOperator’s kubernetes_conn_id parameter, specify in_cluster=True to run the task in the Deployment’s Astro cluster.

Add the Google Artifact Registry URI

When you configure an instantiation of the KubernetesPodOperator, replace <your-docker-image> with the Google Artifact Registry image URI. To retrieve the URI:

• In the Google Artifact Registry, click the registry containing the image.
• Click the image you want to use.
• Click the copy icon next to the image in the top corner. The string you copy should be in the format <GCP Region>-docker.pkg.dev/<Project Name>/<Registry Name>/<Image Name>.