How To Test Ansible Roles with Molecule on Ubuntu 18.04

Barry

Introduction

In Ansible Unit testing is very much important in order to make surethat roles perform accordingly. This entire process is made easy by Molecule, which helps by allowing you to particularize conditions that test roles against various environments .The use of Ansible, molecule implements roles to a provisionary that helps in deploying the role in a configured environment and therefore calls a verifierto check for configuration drift which will also assure that your role has done all the customization to the environment in that specific case. In this particular blog you will learn how to build an Ansible role that will implement Apache  to a host and configures firewalld  on CentOS 7. If you want to test whether this role works as expected, you need to create a test in Molecule using Docker like a driver and Testinfra. Then you need to create a Python library in order to test the condition of the servers. You will see how molecule will verify that whether the server has been configured as it was thought to be. Till the completion of the blog you will know how to create multiple test cases for builds all over the environments and then access those tests using Molecule.

Prerequisites

You will be needing the following before we start:

JOIN OUR NEWSLETTER
Not Every One Focuses On Your Requirements! Get What You Want- Revenue & Ranking Both. Make Money While Stepping Up The Ladder Of SERPs.
We hate spam. Your email address will not be sold or shared with anyone else.
  • You will need one Ubuntu 18.04 server.
  • You will need a server in which Docker will be installed.
  • Lastly , you will need Python 3 and venv installed and configured on your server. Familiarity with Ansible playbooks.

Preparing the Environment

You must have Python 3 venv, and Docker installed and correctly configured if you follow the prerequisites. Now you will have to create a virtual environment in order to test Ansible with Molecule.

now you need to begin by logging in as a non-root user and will have to create a virtual environment:

$ python3 -m venv my_env

Then you will have to activate it in order to assure that your actions are minimalized to that environment :

$ source my_env/bin/activate

Now, in your activated environment, you need to install the wheel package, which offers the bdist_wheelsetuptools extension that pip uses to install Ansible:

(my_env) sammy@ubuntu:~$ python3 -m pip install wheel

You can now install molecule and docker with pip. Ansible will be automatically installed as a dependency for Molecule:

(my_env) sammy@ubuntu:~$ python3 -m pip install molecule docker

This is what the packages will do:

  • molecule: If you install molecule Ansible will be automatically installed .This will also make the use of Ansible playbooks in order to execute roles and tests.You will have to use this package , if you want to test roles.
  • docker: Docker is generally a Python Library, which is used by Molecule to assemblage with Docker .This Python library is used by Molecule to interface with Docker and you will have to use it because you will use Docker as a driver.

Now you will have to move onto the next step that is the creation of a role in Molecule.

Creating a Role in Molecule

Now after you have your environment set up, you can use Molecule in order to create a basic role which you will have to use during the testing and the installation of Apache. This role will create a directory structure along with some initial tests and particularize Docker as the driver so that the moleculeis able to use the to run its tests.

Now you need to create a new role known as ansible-apache:

(my_env) sammy@ubuntu:~$ molecule init role -r ansible-apache -d docker

Here the -r flag particularizes the name of the role and -d specifies the driver. These provision the host for molecule for being used in testing.

Now you will have to shift into the directory of the exclusively created role:

(my_env) sammy@ubuntu:~$ cd ansible-apache

Now you need to test the default role to check if Molecule has been set up properly:

(my_env) sammy@ubuntu:~/ansible-apache$ molecule test

now you will see an output that will list each of the default test actions . Now before the test begins the molecule verifies the configuration file molecule.yml in order to make sure that everything is in order and this will also print the test matrix ,which will particularize the arrangement of the of the test actions.

Output
--> Validating schema /home/sammy/ansible-apache/molecule/default/molecule.yml.
Validation completed successfully.
--> Test matrix

└── default
    ├── lint
    ├── destroy
    ├── dependency
    ├── syntax
    ├── create
    ├── prepare
    ├── converge
    ├── idempotence
    ├── side_effect
    ├── verify
    └── destroy
...

Now you need to go through each and every test action in details after you have created your role and have edited your tests. Now you will have to concentrate towards the PLAY_RECAP for each test, and you need to be sure that none of the default action comes back with a failed status. Like for example, the PLAY_RECAP for the default ‘create’ action must look like this:

Output
...
PLAY RECAP *********************************************************************
localhost                  : ok=5    changed=4    unreachable=0    failed=0

Now moving on to the modification of the role in order to configure Apache and firewalld.

Configuring Apache and Firewalld

If you want to configure Apache and firewalld, you will have to make a tasks file for the role, particularizing packages to install and services to enable. These details will be taken from a variable extracted from a variables file and template that you will use to substitute the default Apache index page. Also in the ansible-apache directory, you need to create a task file for the role using nano, or you can also use your favorite text editor:

(my_env) sammy@ubuntu:~/ansible-apache$ nano tasks/main.yml

You will see that the file is already existing . So you will You’ll get to see that the file already exists. Now you will have to delete what is existing and will have to substitute it with the following code in order to install the required packages and start the correct services , firewall settings and HTML defaults.

~/ansible-apache/tasks/main.yml
---
- name: "Ensure required packages are present"
  yum:
    name: "{{ pkg_list }}"
    state: present

- name: "Ensure latest index.html is present"
  template:
    src: index.html.j2
    dest: /var/www/html/index.html

- name: "Ensure httpd service is started and enabled"
  service:
    name: "{{ item }}"
    state: started
    enabled: true
  with_items: "{{ svc_list }}"

- name: "Whitelist http in firewalld"
  firewalld:
    service: http
    state: enabled
    permanent: true
    immediate: true

Here are the 4 tasks that this play book includes:

  • “Ensure required packages are present”: This task will install the packages listed in the variables file under pkg_list. The variables file will be located at ~/ansible-apache/vars/main.yml and you will create it at the end of this step.
  • “Ensure latest index.html is present”: This task will copy a template page, index.html.j2, and paste it over the default index file, /var/www/html/index.html, generated by Apache. You will also create the new template in this step.
  • “Ensure httpd service is started and enabled”: This task will start and enable the services listed in svc_list in the variables file.
  • “Whitelist http in firewalld”: This task will whitelist the http service in firewalld. Firewalld is a complete firewall solution present by default on CentOS servers. For the http service to work, you will need to expose the required ports. Instructing firewalld to whitelist a service ensures that it whitelists all of the ports that the service requires.

After you complete you will have to save and close the file.

then you will have to create a templates directory for the index.html.j2 template page:

(my_env) sammy@ubuntu:~/ansible-apache$ mkdir templates

Then you will have to create the page itself:

(my_env) sammy@ubuntu:~/ansible-apache$ nano templates/index.html.j2

Now you will have to paste the following boilerplate code:

~/ansible-apache/templates/index.html.j2
<div style="text-align: center">
<h2>Managed by Ansible</h2>
</div>

Then you will have to save and close the file.

The final step you need to take is that you will have to complete the role and that is writing the variables file also which provides the names of packages and services to your main role playbook:

(my_env) sammy@ubuntu:~/ansible-apache$ nano vars/main.yml

Now you will have to paste over the default content which will particularize , with the following code pkg_list and svc_list:

~/ansible-apache/vars/main.yml
---
pkg_list:
- httpd
- firewalld
svc_list:
- httpd
- firewalld

These lists contain the following information:

pkg_list: This contains the names of the packages that the role will install: httpd and firewalld.

svc_list: This contains the names of the services that the role will start and enable: httpd and firewalld.

Note: One thing you need to keep a watch on is that whether your variables file doesn’t have any blank lines or else your test will fail during linting.

Now that you’ve finished creating your role, let’s configure Molecule to test if it works as intended.

Modifying the Role for Running Tests

The configuring Molecule includes modifying the Molecule configuration file molecule.yml to add platform specifications, because you’re testing a role that configures and begins the httpd systemd service, you will need to use an image with systemd configured and privileged mode enabled. You can also use the milcom/centos7-systemd image available on Docker Hub . Privileged mode allows containers to run with almost all of the capabilities of their host machine.

Then you will have to customize molecule.yml to show these changes:

(my_env) sammy@ubuntu:~/ansible-apache$ nano molecule/default/molecule.yml

Next you need to add on the highlighted platform information:

~/ansible-apache/molecule/default/molecule.yml
---
dependency:
  name: galaxy
driver:
  name: docker
lint:
  name: yamllint
platforms:
  - name: centos7
    image: milcom/centos7-systemd
    privileged: true
provisioner:
  name: ansible
  lint:
    name: ansible-lint
scenario:
  name: default
verifier:
  name: testinfra
  lint:
    name: flake8

Now you will have to save and close the file when you are done.

Now that you’ve successfully configured the test environment, you need to go on to the next level that is writing the test cases that Molecule will run against your container after executing the role.

Writing Test Cases

You need to check the following conditions in the test for this role:

  • That the httpd and firewalld packages are installed.
  • That the httpd and firewalld services are running and enabled.
  • That the http service is enabled in your firewall settings.
  • That index.html contains the same data specified in your template file.

If you see that all of these test cases passes then also know that the role will work as it was intended.

In order to write the test cases for these conditions , you need to edit the default tests in
~/ansible-apache/molecule/default/tests/test_default.py. Now with the use of Testinfra, you will have to write the test cases as python functions using Molecule classes.

Now you will have to open test_default.py:

(my_env) sammy@ubuntu:~/ansible-apache$ nano molecule/default/tests/test_default.py

Then you will have to delete the contents of the file so that you can write the tests from the beginning.

Note: Also you need to keep in mind that as you write the tests, you need to make sure that they are separated by two new lines or else they will fail.

Start by importing the required Python modules:

~/ansible-apache/molecule/default/tests/test_default.py
import os
import pytest

import testinfra.utils.ansible_runner

These modules include:

  • os: This built-in Python module enables operating-system-dependent functionality, making it possible for Python to interface with the underlying operating system.
  • pytest: The pytest module enables test writing.
  • testinfra.utils.ansible_runner: This Testinfra module uses Ansible as the backend  for command execution.

Under the module imports, add the following code, which uses the Ansible backend to return the current host instance:

~/ansible-apache/molecule/default/tests/test_default.py
...
testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
    os.environ['MOLECULE_INVENTORY_FILE']).get_hosts('all')

With your test file configured to use the Ansible backend, let’s write unit tests to test the state of the host.

The first test will ensure that httpd and firewalld are installed:

~/ansible-apache/molecule/default/tests/test_default.py
...

@pytest.mark.parametrize('pkg', [
  'httpd',
  'firewalld'
])
def test_pkg(host, pkg):
    package = host.package(pkg)

    assert package.is_installed

The test begins with the pytest.mark.parametrize decorator, which allows us to parameterize the arguments for the test. This first test will take test_pkg as a parameter to test for the presence of the httpd and firewalld packages.

The next test will check whether or not httpd and firewalld are running and enabled. It takes test_svcas a parameter:

~/ansible-apache/molecule/default/tests/test_default.py
...

@pytest.mark.parametrize('svc', [
  'httpd',
  'firewalld'
])
def test_svc(host, svc):
    service = host.service(svc)

    assert service.is_running
    assert service.is_enabled

The last test checks that the files and contents passed to parametrize() exist. If the file isn’t created by your role and the content isn’t set properly, assert will return False:

~/ansible-apache/molecule/default/tests/test_default.py
...

@pytest.mark.parametrize('file, content', [
  ("/etc/firewalld/zones/public.xml", ""),
  ("/var/www/html/index.html", "Managed by Ansible")
])
def test_files(host, file, content):
    file = host.file(file)

    assert file.exists
    assert file.contains(content)

In each test, assert will return True or False depending on the test result.

The finished file will look like this:

~/ansible-apache/molecule/default/tests/test_default.py
import os
import pytest

import testinfra.utils.ansible_runner

testinfra_hosts = testinfra.utils.ansible_runner.AnsibleRunner(
    os.environ['MOLECULE_INVENTORY_FILE']).get_hosts('all')


@pytest.mark.parametrize('pkg', [
  'httpd',
  'firewalld'
])
def test_pkg(host, pkg):
    package = host.package(pkg)

    assert package.is_installed


@pytest.mark.parametrize('svc', [
  'httpd',
  'firewalld'
])
def test_svc(host, svc):
    service = host.service(svc)

    assert service.is_running
    assert service.is_enabled


@pytest.mark.parametrize('file, content', [
  ("/etc/firewalld/zones/public.xml", ""),
  ("/var/www/html/index.html", "Managed by Ansible")
])
def test_files(host, file, content):
    file = host.file(file)

    assert file.exists
    assert file.contains(content)

Now after you’ve specified your test cases, now you will have to test the role.

Testing the Role with Molecule

Once you initiate the test, Molecule will execute the actions you defined in your scenario. Let’s now run the default molecule scenario again, executing the actions in the default test sequence while looking more closely at each.

Run the test for the default scenario again:

(my_env) sammy@ubuntu:~/ansible-apache$" molecule test

Which will initiate the test run. The initial output prints the default test matrix:

Output
--> Validating schema /home/sammy/ansible-apache/molecule/default/molecule.yml.
Validation completed successfully.
--> Test matrix

└── default
    ├── lint
    ├── destroy
    ├── dependency
    ├── syntax
    ├── create
    ├── prepare
    ├── converge
    ├── idempotence
    ├── side_effect
    ├── verify
    └── destroy

Let’s go through each test action and the expected output, starting with linting.

The linting action executes yamllintflake8, and ansible-lint:

yamllint: This linter is executed on all YAML files present in the role directory.

flake8: This Python code linter checks tests created for Testinfra.

ansible-lint: This linter for Ansible playbooks is executed in all scenarios.

Output
...
--> Scenario: 'default'
--> Action: 'lint'
--> Executing Yamllint on files found in /home/sammy/ansible-apache/...
Lint completed successfully.
--> Executing Flake8 on files found in /home/sammy/ansible-apache/molecule/default/tests/...
Lint completed successfully.
--> Executing Ansible Lint on /home/sammy/ansible-apache/molecule/default/playbook.yml...
Lint completed successfully.

The next action, destroy, is executed using the destroy.yml file. This is done to test our role on a newly created container.

By default, destroy is called twice: at the start of the test run, to delete any pre-existing containers, and at the end, to delete the newly created container:

Output
...
--> Scenario: 'default'
--> Action: 'destroy'

    PLAY [Destroy] *****************************************************************

    TASK [Destroy molecule instance(s)] ********************************************
    changed: [localhost] => (item=None)
    changed: [localhost]

    TASK [Wait for instance(s) deletion to complete] *******************************
    ok: [localhost] => (item=None)
    ok: [localhost]

    TASK [Delete docker network(s)] ************************************************
    skipping: [localhost]

    PLAY RECAP *********************************************************************
    localhost                  : ok=2    changed=1    unreachable=0    failed=0

After the destroy action is completed, the test will move on to dependency. This action allows you to pull dependencies from ansible-galaxy if your role requires them. In this case, our role does not:

Output
...
--> Scenario: 'default'
--> Action: 'dependency'
Skipping, missing the requirements file.

The next test action is a syntax check, which is executed on the default playbook.yml playbook. It works in a similar way to the –syntax-check flag in the command ansible-playbook –syntax-check playbook.yml:

Output
...
--> Scenario: 'default'
--> Action: 'syntax'

    playbook: /home/sammy/ansible-apache/molecule/default/playbook.yml

Next, the test moves on to the create action. This uses the create.yml file in your role’s Molecule directory to create a Docker container with your specifications:

Output
...

--> Scenario: 'default'
--> Action: 'create'

    PLAY [Create] ******************************************************************

    TASK [Log into a Docker registry] **********************************************
    skipping: [localhost] => (item=None)
    skipping: [localhost]

    TASK [Create Dockerfiles from image names] *************************************
    changed: [localhost] => (item=None)
    changed: [localhost]

    TASK [Discover local Docker images] ********************************************
    ok: [localhost] => (item=None)
    ok: [localhost]

    TASK [Build an Ansible compatible image] ***************************************
    changed: [localhost] => (item=None)
    changed: [localhost]

    TASK [Create docker network(s)] ************************************************
    skipping: [localhost]

    TASK [Create molecule instance(s)] *********************************************
    changed: [localhost] => (item=None)
    changed: [localhost]

    TASK [Wait for instance(s) creation to complete] *******************************
    changed: [localhost] => (item=None)
    changed: [localhost]

    PLAY RECAP *********************************************************************
    localhost                  : ok=5    changed=4    unreachable=0    failed=0

After the creation, the test shifts on to the prepare action. This action executes the prepare playbook, which will bring the host to a specific state before running converge. This is useful if your role requires a pre-configuration of the system before the role is executed. This doesn’t apply to your role:

Output
...
--> Scenario: 'default'
--> Action: 'prepare'
Skipping, prepare playbook not configured

After prepare, the coverge action executes your role on the container by running the playbook.yml playbook. If multiple platforms are configured in the molecule.yml file, Molecule will converge on all of these:

Output
...
--> Scenario: 'default'
--> Action: 'converge'

    PLAY [Converge] ****************************************************************

    TASK [Gathering Facts] *********************************************************
    ok: [centos7]

    TASK [ansible-apache : Ensure required packages are present] *******************
    changed: [centos7]

    TASK [ansible-apache : Ensure latest index.html is present] ********************
    changed: [centos7]

    TASK [ansible-apache : Ensure httpd service is started and enabled] ************
    changed: [centos7] => (item=httpd)
    changed: [centos7] => (item=firewalld)

    TASK [ansible-apache : Whitelist http in firewalld] ****************************
    changed: [centos7]

    PLAY RECAP *********************************************************************
    centos7                    : ok=5    changed=4    unreachable=0    failed=0

After coverge, the test will move on to idempotence. This action tests the playbook for idempotence to make sure no unexpected changes are made in multiple runs:

Output
...
--> Scenario: 'default'
--> Action: 'idempotence'
Idempotence completed successfully.

Now going onto the next test action which is the side-effect action which helps you in producing scenarios in which you can easily test more things like HA failover and by default molecule doesn’t configure a side-effect playbook and the task is skipped.

Output
...
--> Scenario: 'default'
--> Action: 'side_effect'
Skipping, side effect playbook not configured.

Now Molecule will run the verifier action with the help of the default verifier, Testinfra. This particular action will execute the tests that you have wrote previously in test_default.py If you see that all of these tests pass successfully, you will get to see a success message and Molecule will proceed to the next level:

Output
...
--> Scenario: 'default'
--> Action: 'verify'
--> Executing Testinfra tests found in /home/sammy/ansible-apache/molecule/default/tests/...
    ============================= test session starts ==============================
    platform linux -- Python 3.6.5, pytest-3.7.3, py-1.5.4, pluggy-0.7.1
    rootdir: /home/sammy/ansible-apache/molecule/default, inifile:
    plugins: testinfra-1.14.1
collected 6 items

    tests/test_default.py ......                                             [100%]

    ========================== 6 passed in 41.05 seconds ===========================
Verifier completed successfully.

Molecule finally destroys the instances that were completed during the test and deletes the network assigned to those instances:

Output
...
--> Scenario: 'default'
--> Action: 'destroy'

    PLAY [Destroy] *****************************************************************

    TASK [Destroy molecule instance(s)] ********************************************
    changed: [localhost] => (item=None)
    changed: [localhost]

    TASK [Wait for instance(s) deletion to complete] *******************************
    changed: [localhost] => (item=None)
    changed: [localhost]

    TASK [Delete docker network(s)] ************************************************
    skipping: [localhost]

    PLAY RECAP *********************************************************************
    localhost                  : ok=2    changed=2    unreachable=0    failed=0

Therefore these test actions are now complete, verifying that your role worked as intended.

Conclusion

Therefore in this blog you thus learn to create an Ansible role to install and configure Apache and firewalld. You learned how to write unit tests with Testinfra that Molecule used to assert that the role ran successfully. You can use the same basic method for highly complex roles, and automate testing using a CI pipeline as well. Molecule is a highly configurable tool that can be used to test roles with any providers that Ansible supports, not just Docker. It’s also possible to automate testing against your own infrastructure, making sure that your roles are always up-to-date and functional.

Header Image Source: https://do.co/2rdS8iv

mm

Barry Davis is a Technology Evangelist who is joined to Webskitters for more than 5 years. A specialist in Website design, development & planning online business strategy. He is passionate about implementing new web technologies that makes websites perform better.

Facebooktwittergoogle_pluspinterestlinkedin

Interested in working with us?

We'd love to hear from you
Webskitters LLC
7950 NW 53rd St #337 Miami, Florida 33166
Phone: 732.218.7686