Use this guide to deploy the Astro Private Cloud (APC) control plane with the Helm-based Astronomer platform charts. The control plane hosts central management services such as the APC UI, APC API, monitoring coordination, and authentication.
Astro Private Cloud supports two deployment patterns:
Choose the pattern that matches your reliability and compliance requirements. After selecting a pattern, determine how many APC environments you need. An environment refers to the pairing of control plane and its associated data planes. In unified mode, this maps to a single Kubernetes cluster, whereas in split mode, the environment is the combination of the control plane and all the registered data plane Kubernetes clusters.
Each APC environment can host multiple Airflow Deployments, potentially on multiple data planes. Common types include:
Create a project folder for every environment you plan to host to contain its configuration files. For example, if you want to install a development environment, create a folder named ~/astronomer-dev/control-plane.
eksctl for creating and managing your Astronomer cluster on EKS.Astro Private Cloud requires a Kubernetes Ingress controller to function and provides an integrated Ingress controller by default. Before installing, decide whether to use a third-party ingress controller or the Astronomer integrated ingress controller.
Astronomer generally recommends you use the integrated Ingress controller, but Astro Private Cloud also supports certain third-party ingress-controllers.
Ingress controllers typically need elevated permissions, including a ClusterRole, to function. Specifically, the Astro Private Cloud Ingress controller requires the ability to:
If you have complex regulatory requirements, you might need to use an Ingress controller that’s approved by your organization and disable the Astronomer integrated controller. You configure the Ingress controller during the installation.
values.yaml from a templateAstro Private Cloud uses Helm to apply platform-level configurations. Choose your cloud provider tab below to copy a ready-to-use values.yaml, then customize the template to your requirements.
As you work with the template configuration, keep the following in mind.
helm upgrade or upgrade.sh until instructed to do so in later steps.astronomer.houston.config.publicSignups: true so you can create the initial administrator account. You will lock down account creation in Disable anonymous account creation.When you install Astro Private Cloud, it creates a variety of services that your users access to manage, monitor, and run Airflow.
Choose a base domain such as astronomer.example.com, astro-sandbox.example.com, astro-prod.example.internal for which:
app.<base-domain>houston.<base-domain>alertmanager.<base-domain>prometheus.<base-domain>The base domain itself doesn’t need to be available and can point to another service not associated with Astronomer or Airflow.
When choosing a base domain, consider the following:
dp-01.<base-domain>, so ensure you can create DNS records and issue TLS certificates for subdomains of this base domain.*.example.com can provide service for app.example.com but not app.astronomer-dev.example.com.app and prometheus, are fixed and can’t be changed.The base domain is visible to end users. They can view the base domain in the following scenarios:
https://app.sandbox-astro.example.com.astro login sandbox-astro.example.com.app.apps.<OpenShift-domain>. Doing this requires permission to reconfigure the route admission policy for the standard ingress controller to InterNamespaceAllowed. See Third Party Ingress Controller - Configuration notes for OpenShift for additional information and options.Locate the global.baseDomain in your values.yaml file and change it to your base domain as shown in the following example:
In your Kubernetes cluster, create a Kubernetes namespace to contain the Astro Private Cloud platform. This guide refers to this namespace as <apc platform namespace> below. For example, if you chose apc-cp you would create the namespace as follows:
To install Astro Private Cloud, you need a TLS certificate that is valid for several domains. One of the domains is the primary name on the certificate, also known as the common name (CN). The additional domains are equally valid, supplementary domains known as Subject Alternative Names (SANs).
Astro Private Cloud requires a private certificate to be present in the Astro Private Cloud platform namespace, even if you use a third-party ingress controller that doesn’t otherwise require it.
Request a TLS certificate for the control plane from your security team for Astro Private Cloud. In your request, include the following:
app.<base-domain> as the CN instead.*.<base-domain> (plus an explicit SAN for <base-domain>) or list each hostname individually:
app.<base-domain> (omit if already used as the Common Name)houston.<base-domain>prometheus.<base-domain>alertmanager.<base-domain> (required if you keep the integrated Alertmanager enabled)*.<domainPrefix>.<base-domain> for the control-plane hosts (app.<base-domain>, houston.<base-domain>, and so on); request a certificate that explicitly matches the control-plane names listed earlier.key.pem containing the private key in pem formatfull-chain.pem (containing the public certificate and additional certificates required to validate it, in pem format) or a bare cert.pem and explicit affirmation that there are no intermediate certificates and that the public certificate is the full chain.private-root-ca.pem in pem format of the private Certificate Authority used to create your certificate or a statement that the certificate is signed by a public Certificate Authority.Ensure that you received each of the following three items:
key.pem containing the private key in pem format.full-chain.pem, in pem format, that contains the public certificate and additional certificates required to validate it or a bare cert.pem and explicit affirmation that there are no intermediate certificates and that the public certificate is the full chain.private-root-ca.pem in pem format of the private Certificate Authority used to create your certificate or a statement that the certificate is signed by public Certificate Authority.To validate that your security team generated the correct certificate, run the following command using the openssl CLI:
This command generates a report. If the X509v3 Subject Alternative Name section of this report includes either a single *.<base-domain> wildcard domain or all subdomains, then the certificate creation was successful.
Confirm that your full-chain certificate chain is ordered correctly. To determine your certificate chain order, run the following command using the openssl CLI:
The command generates a report of all certificates. Verify that the certificates are in the following order:
Determine whether or not your certificate was issued by an intermediate certificate-authority. If you don’t know, assume you use an intermediate certificate and attempt to obtain a full-chain.pem bundle from your certificate authority.
Certificates issued by operators of root certificate authorities, including but not limited to LetsEncrypt, are frequently issued from intermediate certificate authorities associated with a trusted root CA.
If, and only if, your certificate was issued directly by the root Certificate Authority of a universally trusted certificate authority, and not from one of their intermediaries, then the server.crt is also the full-chain certificate bundle.
Identify your full-chain public certificate .pem file and use it while storing and configuring the ingress controller TLS certificate.
Run the following command to store the public full-chain certificate in the Astro Private Cloud Platform Namespace in a tls-type Kubernetes secret. You can create a custom name for this secret. The following example uses the default name, astronomer-tls.
--cert parameter must reference your full-chain.pem, which includes the server certificate and any intermediate certificates, if any. Using the server cert directly causes Dag and image deploys to fail.Astronomer recommends naming the secret astronomer-tls with no substitutions when using a third-party ingress controller. If you use another name for the secret, you must uncomment and update the tlsSecret in your values.yaml file.
Skip this step if the control plane will keep using Astronomer’s built-in ingress controller. Configure a custom ingress only when this cluster must integrate with your organization’s ingress stack. The data plane guide includes its own instructions for data plane ingress changes.
If you need a third-party controller, follow the provider-specific guidance in Third-party Ingress-Controllers for the control plane cluster, then return here before continuing.
Skip this step if you don’t use a private Certificate Authority (private CA) to sign the certificate used by your ingress-controller. Or, if you don’t use a private CA for any of the following services that the Astro Private Cloud platform interacts with.
Astro Private Cloud trusts public Certificate Authorities automatically.
Astro Private Cloud must be configured to trust any private Certificate Authorities issuing certificates for systems Astro Private Cloud interacts with, including but not limited to:
Perform the procedure described in Configuring private CAs for each certificate authority used to sign TLS certificates. After creating the trust secret (for example astronomer-ca), add it to global.privateCaCerts in values.yaml so platform components trust the issuer.
Skip this step unless the Astro Private Cloud control plane will pull platform container images from an external container registry that uses a certificate signed by a private CA.
Kubernetes must be able to pull images from one or more container registries for Astro Private Cloud to function. By default, Kubernetes only trusts publicly signed certificates. This means that by default, Kubernetes doesn’t honor the list of certificates trusted by the Astro Private Cloud platform.
Many enterprises configure Kubernetes to trust additional certificate authorities as part of their standard cluster creation procedure. Contact your Kubernetes administrator to find out what, if any, private certificates are currently trusted by your Kubernetes cluster. Then, consult your Kubernetes administrator and Kubernetes provider’s documentation for instructions on configuring Kubernetes to trust additional CAs.
Follow procedures for your Kubernetes provider to configure Kubernetes to trust each CA associated with your container registries.
Certain clusters don’t provide a mechanism to configure the list of certificates trusted by Kubernetes.
While configuring the Kubernetes list of cluster certificates is a customer responsibility, Astro Private Cloud includes an optional component that can, for certain Kubernetes cluster configurations, add certificates defined in global.privateCaCerts to the list of certificates trusted by Kubernetes. Enable this by setting global.privateCaCertsAddToHost.enabled and global.privateCaCertsAddToHost.addToContainerd to true in your values.yaml file and setting global.privateCaCertsAddToHost.containerdConfigToml to:
For example, if your registry lives at my-registry.example.com and you store the CA certificate in a secret named my-private-ca, the global.privateCaCertsAddToHost section would be:
Astro Private Cloud requires the ability to send email to:
Astro Private Cloud sends all outbound email using SMTP.
Do Not Reply <donotreply@example.com>./ or any other escape character in your username or password, you may need to URL encode those characters.Ensure that your Kubernetes cluster has access to send outbound email to the SMTP server.
Change the configuration in values.yaml from noreply@your.domain to an email address that is valid to use with your SMTP credentials.
Construct an email connection string and store it in a secret in the Astro Private Cloud platform namespace. The following example shows how to store the connection in a secret called astronomer-smtp for a user my@user with a password my@pass. Make sure to url-encode the username and password if they contain special characters.
In general, an SMTP URI is formatted as smtps://USERNAME:PASSWORD@HOST/?pool=true. The following table contains examples of the URI for some of the most popular SMTP services:
If your SMTP provider isn’t listed, refer to the provider’s documentation for information on creating an SMTP URI.
Ensure this secret is referenced in the values.yaml file via an entry in the astronomer.houston.secret list. For example:
Skip this step if a single default storage class is sufficient for every control plane component. Otherwise, set the fields below to point at the storage classes you want to use. Astronomer recommends solid-state storage for all volumes.
Key fields to review in values.yaml:
global.storageClass: Fallback storage class for control plane components.postgresql.persistence.storageClass: Only required if you enable the bundled Postgres database (not recommended outside of testing environments).prometheus.persistence.storageClassName: Used by the control plane Prometheus when retaining metrics locally.alertmanager.persistence.storageClassName: Required if Alertmanager should keep state on disk.nats.jetstream.fileStorage.storageClassName: Only relevant if you enable JetStream persistence; most control plane deployments leave JetStream stateless.Example: to point Prometheus at a custom storage class called fast-storage, add:
When you have the desired values, merge them into values.yaml manually or by using a YAML merge tool of your choosing.
Astro Private Cloud requires a central Postgres database that acts as the backend for Astro Private Cloud’s APC API.
If, while evaluating Astro Private Cloud, you need to create a temporary environment where Postgres isn’t available, locate the global.postgresql.enabled option already present in your values.yaml and set it to true, then skip the remainder of this step.
Note that global.postgresql.enabled to true is an unsupported configuration, and should never be used on any development, staging, or production environment.
If you use Azure Database for either PostgreSQL or another Postgres instance that doesn’t enable the pg_trgm by default, you must enable the pg_trgm extension prior to installing Astro Private Cloud. If pg_trgm isn’t enabled, the installation fails. pg_trgm is enabled by default on Amazon RDS and Google Cloud SQL for PostgresQL.
For instructions on enabling the pg_trgm extension for Azure Flexible Server, see PostgreSQL extensions in Azure Database for PostgreSQL - Flexible Server.
Additional requirements apply to the following databases:
pg_trgm extension as per the advisory earlier in this section.global.ssl.mode to prefer in your values.yaml file.Create a Kubernetes Secret in the namespace chosen for the install, named astronomer-bootstrap, that points to your database. You must URL encode any special characters in your Postgres password.
global.postgresql.enabled: true) should only be used for short-lived testing. Always rely on an external Postgres instance for any persistent environment.To create this secret, run the following command replacing the APC platform namespace, username, password, database hostname, and database port with their respective values:
For example, for a username named bob with password abc@abc for the database dbname at hostname some.host.internal, you would run:
astronomer-bootstrap and must be present in the APC platform namespace before you install Astro Private Cloud.Skip this step if you are installing Astro Private Cloud onto a Kubernetes cluster that can pull container images from public image repositories and you don’t want to mirror these images locally.
Docker registry secrets will also need to be created in any data planes you register with this environment, which will be covered in the data plane installation guide.
If your registry can be reached without credentials, ensure the endpoint is restricted to trusted networks (for example private subnets or VPN access). Avoid exposing the platform image registry directly to the public internet. No additional APC configuration is required beyond setting the repository locations later in this step.
For additional examples (including per-deployment registry settings and air gapped workflows), see Configure a custom registry for Deployment images.
Astronomer recommends new Astro Private Cloud installations use the most recent version available in either the Stable or Long Term Support (LTS) release-channel. Keep this version number available for the following steps. For a separate control plane and data plane topology, at least version 1.0.0 of Astro Private Cloud is required.
See Astro Private Cloud’s lifecycle policy and version compatibility reference for more information.
If you have internet access to https://helm.astronomer.io, run the following command on the machine where you want to install Astro Private Cloud:
If you don’t have internet access to https://helm.astronomer.io, download the Astro Private Cloud Platform Helm chart file corresponding to the version of Astro Private Cloud you are installing or upgrading to from https://helm.astronomer.io/astronomer-<version number>.tgz. For example, for Astro Private Cloud v1.0.0 you would download https://helm.astronomer.io/astronomer-1.0.0.tgz. This file doesn’t need to be uploaded to an internal chart repository.
Create a file named upgrade.sh in your platform deployment project directory containing the following script. Specify the following values at the beginning of the script:
CHART_VERSION: Your Astro Private Cloud version, including patch and a v prefix. For example, v1.0.0.RELEASE_NAME: Your Helm release name. astronomer is strongly recommended.NAMESPACE: The namespace to install platform components into. astronomer is strongly recommended.CHART_NAME: Set to astronomer/astronomer if fetching platform images from the internet. Otherwise, specify the filename if you’re installing from a file (for example astronomer-1.0.0.tgz).Mac and Linux users with jq installed can set CHART_VERSION in the following snippet and run it to produce a list of images.
If you are installing Astro Private Cloud into an egress-controlled or air gapped environment, perform the following steps.
By default, Astro Private Cloud checks for Airflow updates, which are included in the Astro Runtime, once per day at midnight by querying https://updates.astronomer.io/astronomer-runtime. This returns a JSON file with details about the latest available Astro Runtime versions.
In an egress-controlled or air gapped environment, you need to store the JSON file in the cluster itself, avoiding the external check. To store the JSON file in the cluster, complete the following steps:
astro-runtime-base-images to your APC API configuration using the runtimeReleasesConfigMapName configuration:If you’re not installing Astro Private Cloud into an OpenShift Kubernetes cluster, skip this step.
Add the following values into values.yaml. You can do this manually or by using a YAML merge tool of your choosing.
Only Ingress objects with the annotation route.openshift.io/termination: "edge" are supported for generating routes in OpenShift 4.11 and later. Other termination types are no longer supported for automatic route generation.
If you’re on an older version of OpenShift, route creation should be done manually.
Astro Private Cloud on OpenShift is only supported when using a third-party ingress controller and the logging sidecar feature of Astro Private Cloud. The preceding configuration enables both of these items.
Astro Private Cloud includes integrations for several of the most popular OAUTH2 identity providers (IdPs), such as Okta and Microsoft Entra ID. Configuring an external IdP allows you to automatically provision and manage users in accordance with your organization’s security requirements. See Integrate an auth system to configure the identity provider of your choice in your values.yaml file.
Deploy the control plane using the upgrade.sh script you created earlier. Confirm RELEASE_NAME, NAMESPACE, and CHART_VERSION reflect your environment, then execute:
To review manifests before applying them, run ./upgrade.sh --dry-run or use helm template with the same flags defined in the script.
Whether you use the Astronomer integrated ingress controller or a third-party controller, publish the same set of DNS records so users can reach control plane services.
If you use the integrated controller, get the load balancer address directly:
If you use a third-party controller, ask your ingress administrator for the hostname or IP address that should front the Astronomer routes (refer back to Configure a third-party ingress controller).
Create either a wildcard record such as *.sandbox-astro.example.com or individual CNAME records for the following hostnames so that traffic routes through the chosen load balancer:
app.<base-domain> (required)houston.<base-domain> (required)prometheus.<base-domain> (required)alertmanager.<base-domain> (required if you keep the integrated Alertmanager enabled)<base-domain> (optional but recommended, provides a vanity redirect to app.<base-domain>)Astronomer generally recommends pointing the zone apex (@) directly to the load balancer address and mapping the remaining hostnames as CNAMEs to that apex. In lower environments, you can safely use a low TTL (for example 60 seconds) to speed up troubleshooting during the initial rollout.
After your DNS provider propagates the records, verify them with tools like dig <hostname> or getent hosts <hostname>. You can complete this DNS work after verifying the platform pods—Astronomer services stay healthy without external DNS, but end users need these records to sign in.
Upgrades from 0.x to 2.0 rename the control plane ingress Service from astronomer-nginx to astronomer-cp-nginx. This provisions a new LoadBalancer with a new public IP/hostname. If you’re upgrading, update DNS and firewall/allowlists and re-issue TLS/SSL certificates if they reference the previous LoadBalancer hostname. See the upgrade guide.
Visit https://app.<base-domain> in your web browser to view the Astro Private Cloud web interface. If any components aren’t ready, consult the debugging guide or contact Astronomer support with the relevant logs and events.
Congratulations, you have configured and installed an Astro Private Cloud platform instance - your new Airflow control plane!
From the Astro Private Cloud UI, you’ll be able to both invite and manage users as well as create and monitor Airflow Deployments on the platform.
Leave astronomer.houston.config.publicSignups: true only long enough to create your first administrator. Afterwards, secure the platform as follows:
astronomer.houston.config.email.enabled: true), specify a trusted domain list under astronomer.houston.config.allowedSystemLevelDomains, and verify that users can only join through an approved identity provider.astronomer.houston.config.publicSignups: false so new accounts require an invitation.helm upgrade targeting the control plane release.The following topics include optional information about one or multiple topics in the installation guide:
Add the data planes to the control plane to begin creating Airflow Deployments. See Register a data plane with the APC control plane for instructions on exchanging tokens, approving connectivity, and assigning deployments.