Skip to main content

AWX (Ansible Tower)

Add Persistent storage to EE

Add Persistent storage to EE

Sometimes you need to add persistent storage to your runtime environment to either keep some data that were processed, downloads or for many other reasons. Since the AWX is running in Kubernetes your ansible playbooks are executed inside container that is spawned for each job, and removed when its done.

The correct term is Execution Environment or EE and by default its this image: quay.io/ansible/awx-ee:latest so your playbook is downloaded from GIT to this "Docker" image and run inside.

This can cause two issues, one is that you might need some ansible modules or python modules that are required by your playbook and this image simply does not have it. It contains essentials but missing stuff like VMware related modules etc... we will address this issue in separate article.

Adding Storage

šŸ’”
This option is probably more suited for more than one node Kubernetes and with proper storage class. There is simpler method with single node Kubernetes down in the article.

First you need to create persistent storage on your Kubernetes instance. This usually depends on your storage solution but if you run single node Kubernetes you can create simple hostPath volume. Create following two files.

awx-ee-perist-pv.yaml

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: custom-awx-pv
spec:
  storageClassName: standard
  accessModes:
    - ReadWriteOnce
  capacity:
    storage: 5Gi
  hostPath:
    path: /data/custom-awx-pv
    type: DirectoryOrCreate
  • apiVersion: v1: Indicates the API version being used. In this case, it's the core Kubernetes API version.
  • kind: PersistentVolume: Specifies the kind of Kubernetes resource being defined, which is a PersistentVolume in this case.
  • metadata: Contains metadata about the PersistentVolume, such as its name.
    • name: custom-awx-pv: The name of the PersistentVolume, which is "custom-awx-pv" in this example.
  • spec: Defines the specifications of the PersistentVolume.
    • storageClassName: standard: Specifies the storage class name for this PersistentVolume. Storage classes define the provisioning policy and parameters for the underlying storage.
    • accessModes: Defines the access modes that the PersistentVolume supports. In this case, it supports the "ReadWriteOnce" mode, which means it can be mounted as read-write by a single node.
    • capacity: Specifies the capacity of the PersistentVolume. It allocates 5 gigabytes (5Gi) of storage space.
    • hostPath: Specifies the host path where the volume will be mounted.
      • path: /data/custom-awx-pv: Defines the path on the host machine where the volume will be mounted. In this case, it's /data/custom-awx-pv.
      • type: DirectoryOrCreate: Specifies the type of the host path. If the directory specified by path doesn't exist, it will be created. This ensures that the directory is available for use by the PersistentVolume.

awx-ee-perist-pvc.yaml

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: custom-awx-pvc
spec:
  volumeName: custom-awx-pv
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 5Gi

Apply both of them to your Kubernetes

kubectl apply -f awx-ee-perist-pv.yaml
kubectl apply -f awx-ee-perist-pvc.yaml

You can then go to the Instance Groups > default (or if you setup other groups) > Edit

Add this to bottom of the pod specification:

  volumes:
    - name: custom-ee-storage
      persistentVolumeClaim:
        claimName: custom-awx-pvc

Mounting local folder

This is more easy option, but this will mount local folder on node where the EE will run. Most advantageous in single node clusters as you will always know where it is šŸ˜„

Go to Settings > Jobs settings

Click Edit at the bottom of the opened page and look for box called Paths to expose to isolated jobs

Here you can add local folder and rw option, this fil mount the folder from host to container where your playbook will run. On your host change the permissions to the folder to 777 to avoid any issue with permissions and owners...

The Operator Way

Third option is to use the operator we used to install AWX and tell it to mount disk to EE.

More info can be found here: https://ansible.readthedocs.io/projects/awx-operator/en/latest/user-guide/advanced-configuration/custom-volume-and-volume-mount-options.html

šŸš§
Warning ! This method works only if you use default Execution Environment, if you start using custom EE this method will be ignored.

If you followed my guide how to install the AWX via operator you should have file called: awx-instance-deployment.yml

awx-instance-deployment.yml

---
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
spec:
  service_type: nodeport
  projects_persistence: true
  projects_storage_access_mode: ReadWriteOnce
  web_extra_volume_mounts: |
    - name: static-data
      mountPath: /var/lib/projects
  extra_volumes: |
    - name: static-data
      persistentVolumeClaim:
        claimName: public-static-data-pvc

You can add the disk specification here. From documentation we are looking into ee_extra_volume_mounts so lets add a folder from server to be in EE.

First create the local PersistentVolume

local-test-pv.yaml

apiVersion: v1
kind: PersistentVolume
metadata:
  name: local-test-pv
spec:
  capacity:
    storage: 1Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  storageClassName: local-storage
  local:
    path: /tmp/test

Create claim, local-test-pvc.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: local-test-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  volumeName: local-test-pv
  storageClassName: local-storage

Don't forget to apply them with kubectl apply -f <file>

Then you can use it in the operator file:

---
apiVersion: awx.ansible.com/v1beta1
kind: AWX
metadata:
  name: awx
spec:
  service_type: nodeport
  projects_persistence: true
  projects_storage_access_mode: ReadWriteOnce
  web_extra_volume_mounts: |
    - name: static-data
      mountPath: /var/lib/projects
  extra_volumes: |
    - name: static-data
      persistentVolumeClaim:
        claimName: public-static-data-pvc
    - name: local-test
      persistentVolumeClaim:
        claimName: local-test-pvc
  ee_extra_volume_mounts: |
    - name: local-test
      mountPath: /tmp/test

Note we added the PVC onto extra_volumes and then specified where it will be mounted under ee_extra_volume_mounts

šŸ’”
Volume and VolumeMount names cannot contain underscores(_)