Part 1: The Ansible.utils Collection for Playbook Creators

The Ansible ansible.utils collection includes a variety of plugins that aid in the management, manipulation and visibility of data for the Ansible playbook developer. The most common use case for this collection is when you want to work with the complex data structures present in an Ansible playbook, inventory, or returned from modules. See each plugin documentation page for detailed examples for how these utilities can be used in tasks. In this two-part blog we will overview this collection in part one and see an example use case of using the utils collection in detail in part two.

Plugins inside ansible.utils

Plugins are code which will augment ansible core functionality. This code executes on control node.it and gives options and extensions for the core features of Red Hat Ansible Automation Platform. This ansible.utils plugin collection includes:

Filter plugins
Lookup plugins
Test plugins
Modules

Filter plugins

Filter plugins manipulate data. With the right filter you can extract a particular value, transform data types and formats, perform mathematical calculations, split and concatenate strings, insert dates and times, and do much more. Ansible Automation Platform uses the standard filters shipped with Jinja2 and adds some specialized filter plugins. You can create custom Ansible filters as plugins. Please refer to the docs for more information.

The ansible.utils filter plugins include the following:

ansible.utils.from_xml - Convert a given XML string to native python dictionary.
ansible.utils.get_path - Retrieve the value in a variable using a path
ansible.utils.index_of - Find the indices of items in a list matching some criteria
ansible.utils.param_list_compare - Generate the final param list combining/comparing base and provided parameters.
ansible.utils.to_paths - Flatten a complex object into a dictionary of paths and values
ansible.utils.to_xml - Convert given JSON string to XML
ansible.utils.usable_range - Expand the usable IP addresses
ansible.utils.validate - Validate data with provided criteria

Lookup plugins

Lookup plugins are an Ansible-specific extension to the Jinja2 templating language. You can use lookup plugins to access data from outside sources (files, databases, key/value stores, APIs, and other services) within your playbooks. Like all templating, lookups execute and are evaluated on the Ansible Automation Platform control machine. Ansible makes the data returned by a lookup plugin available using the standard templating system. You can use lookup plugins to load variables or templates with information from external sources. You can alsocreate custom lookup plugins. Please refer to the docs for more information.

The ansible.utils lookup plugins include:

ansible.utils.get_path - Retrieve the value in a variable using a path
ansible.utils.index_of - Find the indices of items in a list matching some criteria
ansible.utils.to_paths - Flatten a complex object into a dictionary of paths and values
ansible.utils.validate - Validate data with provided criteria

Note: In ansible.utils some plugins were implemented as both filter and lookup plugins to give the playbook developer flexibility depending on their use case.

Test plugins

Test plugins evaluate template expressions and return a value of True or False. With test plugins you can create conditionals to implement the logic of your tasks, blocks, plays, playbooks, and roles. Ansible Automation Platform uses the standard tests shipped as part of Jinja, and adds some specialized test plugins. Please refer to the docs for more information.

The ansible.utils test plugins include:

ansible.utils.in_any_network - Test if an IP or network falls in any network
ansible.utils.in_network - Test if IP address falls in the network
ansible.utils.in_one_network - Test if IP address belongs in any one of the networks in the list
ansible.utils.ip - Test if something in an IP address or network
ansible.utils.ip_address - Test if something in an IP address
ansible.utils.ipv4 - Test if something is an IPv4 address or network
ansible.utils.ipv4_address - Test if something is an IPv4 address
ansible.utils.ipv4_hostmask - Test if an address is a valid hostmask
ansible.utils.ipv4_netmask - Test if an address is a valid netmask
ansible.utils.ipv6 - Test if something is an IPv6 address or network
ansible.utils.ipv6_address - Test if something is an IPv6 address
ansible.utils.ipv6_ipv4_mapped - Test if something appears to be a mapped IPv6 to IPv4 mapped address
ansible.utils.ipv6_sixtofour - Test if something appears to be a 6to4 address
ansible.utils.ipv6_teredo - Test if something appears to be an IPv6 teredo address
ansible.utils.loopback - Test if an IP address is a loopback
ansible.utils.mac - Test if something appears to be a valid MAC address
ansible.utils.multicast - Test for a multicast IP address
ansible.utils.private - Test if an IP address is private
ansible.utils.public - Test if an IP address is public
ansible.utils.reserved - Test for a reserved IP address
ansible.utils.resolvable - Test if an IP or name can be resolved via /etc/hosts or DNS
ansible.utils.subnet_of - Test if a network is a subnet of another network
ansible.utils.supernet_of - Test if a network is a supernet of another network
ansible.utils.unspecified - Test for an unspecified IP address
ansible.utils.validate - Validate data with provided criteria

Modules

Modules are the main building blocks of Ansible playbooks. Although we do not generally speak of "module plugins", a module is a type of plugin. For a developer-focused description of the differences between modules and other plugins, see Modules and plugins: what is the difference?. Please refer to the docs for more information.

The ansible.utils modules include:

ansible.utils.cli_parse - Parse cli output or text using a variety of parsers
ansible.utils.fact_diff - Find the difference between currently set facts
ansible.utils.update_fact - Update currently set facts
ansible.utils.validate - Validate data with provided criteria

Accessing and using the ansible.utils Collection

To download the utils Collection, refer to Automation hub (fully supported, requires a Red Hat Ansible Automation Platform subscription) or Ansible Galaxy (upstream):

Automation hub Collection: ansible.utils
Ansible Galaxy Collection: ansible.utils

Ansible.utils is also available in the Supported Execution environment along with its required python libraries. Please refer to docs for more details about Execution Environments.

Different use cases of Utils

As we know, ansible.utils has a variety of plugins and it has various use cases. The following are the most common use cases of ansible.utils:

Validating business logic before pushing configurations using validate and test plugins
Auditing architectural deposition and layouts using test plugins
Managing complex data structure in ansible playbook using get_path, to_path plugins
Conducting minor checks related to network address using test plugins
Operational state assessment using cli_parse, validate plugins

Future scope

Here are some additional ansible.utils capabilities that are on the horizon:

Ipaddr filter plugin supports:
- The Ipaddr filter is designed to provide an interface to the netaddr Python package from within Ansible.
- It can operate on strings or lists of items, test various data to check if they are valid IP addresses, and manipulate the input data to extract requested information.
- ipaddr() works with both IPv4 and IPv6 addresses in various forms.
- There are also additional functions available to manipulate IP subnets and MAC addresses.
- We are currently working on supporting the ipaddr filter as part of ansible.utils collection.
Support of more number of validate engines in ansible.utils.validation plugin:
- Currently the validate plugin is supporting only ansible.utils.jsonschema validation engines, but there is plan to add more validation engines.
Support different filter plugins to manipulate input data:
- Recursive plugins: remove_keys, replace_keys, keep_keys

Contributing to this collection

This collection is intended for plugins that are not platform or discipline specific. Simple plugin examples should be generic in nature. More complex examples can include real world platform modules to demonstrate the utility of the plugin in a playbook.

We welcome community contributions to this collection. If you find problems, please open an issue or create a PR against the ansible.utils collection repository. See Contributing to Ansible-maintained collections for complete details. See the Ansible Community Guide for details on contributing to Ansible.

Takeaways and next steps

ansible.utils plugins makes playbook writing experience simple and smooth
Implementation of ansible.utils plugins is very fast as they executed locally
Easy to understand, code, use, and integrate with other modules
As its plugins ecosystem, it is so easy to add new plugins in ansible.utils