Install Python packages from private sources

Use the --build-secrets flag to pass .netrc credentials at build time. This approach lets you add private packages to your requirements.txt file without custom Dockerfile configuration.

Prerequisites

• The Astro CLI
• An Astro project
• Custom Python packages that are installable with pip
• A private GitHub repository for each of your custom Python packages
• The GitHub CLI or a GitHub personal access token

Step 1: Specify the private repository in your project

Add your private packages to your requirements.txt file using HTTPS URLs in the following format:

git+https://github.com/<your-github-organization-name>/<your-private-repository>.git

For example, to install mypackage1 and mypackage2 from myorganization:

git+https://github.com/myorganization/mypackage1.git
git+https://github.com/myorganization/mypackage2.git

Step 2: Configure credentials

Define the NETRC_CONTENT environment variable in your shell profile (.bashrc or .zshrc):

$
export NETRC_CONTENT="machine github.com login oauth2 password $(gh auth token)"

If you don’t use the GitHub CLI, replace $(gh auth token) with a GitHub personal access token that has access to your private repositories.

Step 3: Run with build secrets

Pass the --build-secrets flag when running Astro CLI commands:

$
astro dev start --build-secrets id=netrc,env=NETRC_CONTENT

To run tests:

$
astro dev pytest --build-secrets id=netrc,env=NETRC_CONTENT

The --build-secrets flag securely provides the .netrc content during the Docker build without storing credentials in the image.

Deploy with GitHub Actions

To build your image in a GitHub Actions workflow, pass the .netrc content as a build secret:

1
- name: Build image
2
  uses: docker/build-push-action@v4
3
  with:
4
    context: .
5
    secrets: |
6
      netrc=machine github.com login oauth2 password ${{ secrets.GITHUB_TOKEN }}

Deploy with the Astronomer deploy action

To deploy using the Astronomer deploy action, pass the .netrc content through build-secrets and set the NETRC_CONTENT environment variable:

1
- name: Deploy to Astro
2
  uses: astronomer/deploy-action@v0.x
3
  with:
4
    build-secrets: id=netrc,env=NETRC_CONTENT
5
    # ... deployment-id, etc.
6
  env:
7
    NETRC_CONTENT: "machine github.com login oauth2 password ${{ secrets.GITHUB_TOKEN }}"