Configure probes | Astronomer Documentation

In Astro Private Cloud (APC), you can create liveness and readiness probes to assess whether your Kubernetes Pods or network are healthy and can process requests.

Some components in APC include liveness and readiness probes by default, but all components support adding and configuring them. APC allows you to use the Kubernetes liveness and readiness probe definitions so you can monitor the state of your Pods.

Liveness probes can be useful in all cases. However, readiness probes might be most useful for the following scenarios:

If you have network ports open on the container.
If a container doesn’t have open ports but has multiple processes.
If a process in a container might never reach a healthy state because it’s waiting for some state to be achieved.

Default probe behavior

You can use the following structure to define your probes in your values.yaml file. For example, you might want to adjust any default values by configuring the amount of time until a timeout.

You can add any definitions that are compatible with Kubernetes probes. However, because Kubernetes doesn’t allow having more than one handler of probes, you must be sure that you don’t define probes that use both exec and httpGet. For consistency, the examples shown in the Default Astronomer Helm probe configurations use httpGet, but you can use exec when appropriate.

Liveness probe templates

1 livenessProbe:
2   enabled: true
3   httpGet:
4     path: /index.html
5     port: 443

Readiness probe templates

1 readinessProbe:
2   enabled: true
3   httpGet:
4     path: /index.html
5     port: 444

Retrieve existing probe definitions

You can retrieve the default probe definitions from the Kubernetes manifest. The following example shows how to retrieve the definitions for the APC API.

$ kubectl -n "${NAMESPACE}" get deployment -l component=houston -o yaml

This command produces a large amount of YAML output describing your APC API configuration. Within this output is a section describing the livenessProbe, which looks like this:

1 livenessProbe:
2   failureThreshold: 10
3   httpGet:
4     path: /v1/healthz
5     port: 8871
6     scheme: HTTP
7   initialDelaySeconds: 30
8   periodSeconds: 10
9   successThreshold: 1
10   timeoutSeconds: 1

You can copy and paste this output into your values.yaml file for your APC API configuration, then adjust the values you want to customize. Then apply a platform config change.

Reference Helm values within your probes

The liveness and readiness probes specified in Helm values are passed through the Helm template function, which allows you to reference other Helm values within the probes. Specifically, the livenessProbe and readinessProbe values are rendered to YAML, then passed through the Helm template function, which renders any Helm template syntax into the produced YAML.

For example, instead of hardcoding values for your probes to match values defined by other configurations in your values.yaml file, you can use the configuration variable itself.

The following example, using the alertmanager YAML configuration, shows how the path and ports are defined by Values.ports.http and Values.prefixURL elsewhere in the values.yaml file.

1 readinessProbe:
2   httpGet:
3     path: {{ .Values.prefixURL }}/#/status
4     port: {{ .Values.ports.http }}
5   initialDelaySeconds: 30
6   timeoutSeconds: 30

Default Astronomer Helm probe configurations

The following components have their default probe configuration defined in the Astronomer Helm chart.

If a component doesn’t have probes defined by default, you can see which components support custom probe configurations below.

Airflow operator

The following can also be configured to include liveness and readiness probes:

1 airflow-operator:
2   livenessProbe: {}
3   readinessProbe: {}

Alert manager

The following can also be configured to include liveness and readiness probes:

1 alertmanager:
2   livenessProbe: {}
3   readinessProbe: {}

Astronomer

1 astronomer:
2   astroUI:
3     livenessProbe:
4       httpGet:
5         path: /
6         port: 8080
7       initialDelaySeconds: 10
8       periodSeconds: 10
9     readinessProbe:
10       httpGet:
11         path: /
12         port: 8080
13       initialDelaySeconds: 10
14       periodSeconds: 10
15   commander:
16     livenessProbe:
17       httpGet:
18         path: /healthz
19         port: 8880
20         scheme: HTTP
21       initialDelaySeconds: 10
22       periodSeconds: 10
23       failureThreshold: 5
24       successThreshold: 1
25       timeoutSeconds: 5
26     readinessProbe:
27       httpGet:
28         path: /healthz
29         port: 8880
30       initialDelaySeconds: 10
31       periodSeconds: 10
32   houston:
33     livenessProbe:
34       httpGet:
35         path: /v1/healthz
36         port: 8871
37       initialDelaySeconds: 30
38       periodSeconds: 10
39       failureThreshold: 10
40     readinessProbe:
41       httpGet:
42         path: /v1/healthz
43         port: 8871
44       initialDelaySeconds: 30
45       periodSeconds: 10
46       failureThreshold: 10
47   registry:
48     livenessProbe:
49       httpGet:
50         path: /
51         port: 5000
52       initialDelaySeconds: 10
53       periodSeconds: 10
54       timeoutSeconds: 5
55     readinessProbe:
56       httpGet:
57         path: /
58         port: 5000
59       initialDelaySeconds: 10
60       periodSeconds: 10
61       timeoutSeconds: 5

The following can also be configured to include liveness and readiness probes:

1 astronomer:
2   configSyncer:
3     livenessProbe: {}
4     readinessProbe: {}
5   houston:
6     bootstrapper:
7       livenessProbe: {}
8       readinessProbe: {}
9     cleanupAirflowDb:
10       livenessProbe: {}
11       readinessProbe: {}
12     cleanupClusterAudits:
13       livenessProbe: {}
14       readinessProbe: {}
15     cleanupDeployRevisions:
16       livenessProbe: {}
17       readinessProbe: {}
18     cleanupDeployments:
19       livenessProbe: {}
20       readinessProbe: {}
21     dbMigration:
22       livenessProbe: {}
23       readinessProbe: {}
24     syncDataplaneClusters:
25       livenessProbe: {}
26       readinessProbe: {}
27     taskUsageMetrics:
28       livenessProbe: {}
29       readinessProbe: {}
30     updateCheck:
31       livenessProbe: {}
32       readinessProbe: {}
33     updateResourceStrategy:
34       livenessProbe: {}
35       readinessProbe: {}
36     updateRuntimeCheck:
37       livenessProbe: {}
38       readinessProbe: {}
39     upgradeDeployments:
40       livenessProbe: {}
41       readinessProbe: {}
42     waitForDB:
43       livenessProbe: {}
44       readinessProbe: {}
45     worker:
46       livenessProbe: {}
47       readinessProbe: {}