How to build Ansible execution environment images for unconnected environments

Create a defined, consistent and portable environment for executing automation jobs on air-gapped or disconnected systems .

Ansible execution environments (EE) were introduced in Ansible Automation Platform 2 to provide a defined, consistent and portable environment for executing automation jobs. Execution environments are basically Linux container images that help execute Ansible playbooks.

Before learning how to build execution environment images for an unconnected, offline environment, you need to know some execution environment basic concepts.

This article was originally published in the Red Hat blog.

Execution environment components

The container images for the execution environments contain the necessary components to execute Ansible automation jobs. These include Python, Ansible (ansible-core), Ansible Runner, required Python libraries, and dependencies.

When you install Ansible Automation Platform, the installer deploys the following container images whether you’re in a connected or an unconnected installation:

• The ee-29-rhel8 image contains Ansible 2.9 to use with older Ansible playbooks.
• ee-minimal-rhel8 is the minimal container image with ansible-core and basic collections.
• ee-supported-rhel8 is the container image with ansible-core and automation content collections supported by Red Hat.

Use ansible-builder to create a custom execution environment

Ansible Automation Platform’s default container images let you start doing automation without any additional configurations. If your automation requires additional libraries or plugins, you can build custom container images and store them inside any supported container registry, such as a private automation hub .

You can follow the standard container image build process for building execution environment container images, but Ansible Automation Platform also includes a command-line utility called ansible-builder to build container images for custom execution environments. The ansible-builder tool can be installed from the upstream Python repository or the Red Hat RPM repository:

## Install ansible-builder utility
$ pip3 install ansible-builder

## Ansible Automation Platform repository subscription is required
$ sudo dnf install ansible-builder

The ansible-builder helps you build container images with the definition file execution-environment.yml . A typical execution-environment.yml contains the base container image ( EE_BASE_IMAGE ), ansible.cfg , and other dependency file details:

---
version: 1

build_arg_defaults:
  EE_BASE_IMAGE: 'aap25.lab.iamgini.com/ee-minimal-rhel8:latest'

ansible_config: 'ansible.cfg'

dependencies:
  galaxy: requirements.yml
  python: requirements.txt

additional_build_steps:
  append:
    - RUN microdnf install which

In the preceding code, aap25.lab.iamgini.com is the private automation hub, but you can use another container registry source, such as registry.redhat.io or Quay.io .

Once you’ve prepared the execution-environment.yml , execute the ansible-builder build command to create a build context that includes the Containerfile .

Now, create the container:

$ ansible-builder build --tag my_custom_ee
Running command:
  podman build -f context/Containerfile -t my_custom_ee context
Complete! The build context can be found at: /home/gmadappa/ansible-aap-demo/context

Refer to the official documentation to learn more about building container images using ansible-builder .

Use custom execution environments in unconnected environments

There are situations where the automation platform is inside an air-gapped environment or restricted network with limitations including:

• No access to external Python repositories or no internal PyPI repository servers available to use
• No access to required RPM repositories

In this case, you have two options to build and use custom execution environments with Ansible Automation Platform: building and transferring the container image or creating a custom environment.

Option 1. Build and transfer a container image

You can create a container image from a connected machine (for example, a developer workstation) with all the dependencies inside and transfer it to the private automation hub (or another supported registry).

Step 1. Create and archive the container image from a connected machine:

## build the container image
$ ansible-builder build --tag my_custom_ee

## Save the container image as archive file
$ podman save --quiet -o my_custom_ee-1.0.tar localhost/my_custom_ee:1.0

Step 2. Copy the archived container image (for example, my_custom_ee-1.0.tar ) to the target unconnected system using secure file transfer , a secure USB drive, or another method available to you.

Step 3. Load the container image from the tar file to the system on the unconnected machine:

$ podman load -i my_custom_ee-1.0.tar

Step 4. Follow the tag and push process for private automation hub .

Option 2. Create a custom execution environment in an unconnected environment

You can create a container image inside the unconnected machine after transferring the dependencies.

Identify the Python version on target machine

It is important to identify the target machine Python version and details before you download the libraries and packages.

Execute the following command inside the base EE image to check the Python details on the target machine.

$ python -m pip debug --verbose

• Identify Python version and ABI (examples: cp311, cp312).
• Platform tag (examples: manylinux2014_x86_64 for most glibc-based Linux; musllinux for Alpine; win_amd64 for Windows).

E.g:

...
Compatible tags: 714
  cp311-cp311-manylinux_2_28_x86_64
  cp311-cp311-manylinux_2_27_x86_64
  cp311-cp311-manylinux_2_26_x86_64
...

Download the package from machine with internet access

From a machine which is connected to internet, download the packages and dependencies using pip download command. (We will download the package and dependencies to a directory to avoid mix-up)

Create a directory as repository to keep the packages and dependencies.

user@node1 $ mkdir python-packages

Download the python libraries

# Download everything required by your requirements file
pip download -r ./requirements.txt \
  -d ./python-packages \
  --platform manylinux_2_17_x86_64 \
  --python-version 311 \
  --implementation cp \
  --abi cp311 \
  --only-binary=:all:

Transfer dependencies to the Ansible builder machine

Now transfer the directory and content (directly or as an archive file) to target machine (disconnected machine where we are running the ansible-builder command) – using any of the available methods. (a secure USB drive, scp, sftp, WinSCP etc)

For example, the following demonstration contains an archive file with the Python libraries required for my automation playbooks and collections:

$ tar -tf python-packages.tar
python-packages/
python-packages/pan_os_python-1.7.3-py2.py3-none-any.whl
python-packages/ipaddress-1.0.23-py2.py3-none-any.whl
python-packages/requirements.txt
python-packages/pan_python-0.17.0-py2.py3-none-any.whl

Modify the execution-environment.yaml with additional steps

# execution-environment.yml
---
version: 3
build_arg_defaults:
  ANSIBLE_GALAXY_CLI_COLLECTION_OPTS: '--pre --ignore-certs'

dependencies:
  galaxy: requirements.yml
  python: requirements.txt
  system: bindep.txt

images:
  base_image:
    name: registry.redhat.io/ansible-automation-platform-25/ee-minimal-rhel8:1.0.0-955

options:
  package_manager_path: /usr/bin/microdnf
  user: root

additional_build_files:
  - src: ansible.cfg
    dest: configs
  - src: ./python-packages
    dest: python-packages
  - src: ./requirements.txt
    dest: python-packages/requirements.txt


additional_build_steps:
  prepend_galaxy:
    - COPY _build/configs/ansible.cfg /etc/ansible/ansible.cfg
    - COPY _build/requirements.txt /opt/requirements.txt
    - COPY _build/python-packages /opt/python-wheels
    - RUN $PYCMD -m pip install --no-cache-dir -r /opt/requirements.txt --find-links=/opt/python-wheels --no-index

Step 4. Follow the tag and push process for Private Automation Hub .

I’m making these assumptions about this process:

• The ee-minimal-rhel8:latest and the ansible-builder-rhel8 container images are already on the unconnected machine (installed or configured as part of the Ansible Automation Platform deployment).
• No multistage build is involved.

Push the container image to the private automation hub

You can store the container image in Ansible’s private automation hub with other default execution environment images.

Step 1. Log in to the container registry:

$ podman login aap25.lab.iamgini.com
Username: pahadmin
Password:
Login Succeeded!

If you’re using self-signed Secure Socket Layer (SSL) certificate on the AAP or private automation hub nodes, then disable SSL validation by adding the --tls-verify=false option at the end.

Step 2. Tag the local container image with the private automation hub path:

$ podman tag localhost/network-ee:1.0 \
aap25.lab.iamgini.com/network-ee:1.0

Step 3. Push the image to the private automation hub (registry):

$ podman push \
aap25.lab.iamgini.com/network-ee:1.0

Step 4. Once the image is copied, verify the image on the private automation hub’s (Automation content) graphical user interface:

If you’re not using the private automation hub, then you need to copy the container image to all execution nodes and hybrid nodes manually using the save and load method mentioned earlier.

Refer to the official documentation to learn how to use an execution environment in jobs on automation controllers.

Building containers

You can use any possible supported method to build container images for an execution environment and store it in a supported container registry. You can also use the upstream images ( quay.io/ansible/ansible-runner:latest and quay.io/ansible/ansible-builder:latest ) but with community-only support for building custom execution environment images.

Building image using Podman and Containerfile

Prepare the Containerfile with instructions to build the container image for the execution environment:

## Containerfile for custom execution environment
ARG EE_BASE_IMAGE=registry.redhat.io/ansible-automation-platform-22/ee-minimal-rhel8:latest
ARG EE_BUILDER_IMAGE=registry.redhat.io/ansible-automation-platform-22/ansible-builder-rhel8

FROM $EE_BASE_IMAGE

ADD ansible.cfg ansible.cfg
ADD python-packages.tar python
RUN python3 -m pip install -r python/python-packages/requirements.txt --find-links=python/python-packages/ --no-index

Build the container image using Podman:

$ podman build -f Containerfile -t localhost/network-ee:1.0
[...]

Looking in links: python/python-packages/
Processing ./python/python-packages/pan_os_python-1.7.3-py2.py3-none-any.whl
Processing ./python/python-packages/pan_python-0.17.0-py2.py3-none-any.whl
Installing collected packages: pan-python, pan-os-python
[...]

Successfully tagged localhost/network-ee:1.0
01e210e05a60dcf49c1b4a2b1bf1e58c49a487823b585233a15d1ecd66910bab

The TAR file is copied, extracted, and the content is installed inside the image.