Streamline Your Ansible Module Development with Content Builder

Photo by Hush Naidoo Jade Photography on Unsplash

Are you tired of manually creating Ansible modules for every new tool, cloud service, or appliance you need to manage? Look no further than the Ansible Content Builder. This powerful Python tool can generate Ansible modules for any appliance or service with a network CLI, NETCONF, or OpenAPI. In this blog post, we'll show you how to use the Content Builder to scaffold your Ansible modules to create good, consistent content for your cloud platform of choice. We'll also explore how the Content Builder can help you onboard new tools and services into the Ansible ecosystem with ease. So if you're looking to supercharge your Ansible workflow and automate any cloud platform, this post is for you!

The Ansible Content Collections for hybrid cloud automation house multiple modules. These modules handle the creation, deletion, modification (and more) of cloud resources. While developing individual modules for different resources of the cloud platform can lead to modularity and ease of use, reinventing the wheel can be error-prone and time-consuming. Standardizing the common steps that are similar but differ based on limited parameters can solve many problems encountered while developing these modules. This is achieved by a scaffolding tool that sets up a baseline skeleton for the modules we intend to develop. 

Content Builder is a tool developed to generate modules and plugins for different content verticals of Ansible. It supports networking, security, and cloud content. In this blog, we are going to look into cloud content generation in specific. 

Amazon.cloud modules are generated from the CloudFormation Resource Type Definition Schema or meta-schema. Vmware.vmware_rest modules are generated using the vSphere REST API. The schema generated by these APIs is fed as input to this tool to generate the content.

Before the Content Builder tool,  VMware generator, and Amazon cloud generator were used to generate the respective Collections. These generators are archived now.

Why Content Builder for cloud modules

The cloud Collections that this tool can generate are amazon.cloud and vmware.vmware_rest. Both these Collections are generated based on OpenAPI-based REST schemas. All the modules of amazon.cloud collection use the AWS Cloud Control API to interact with the AWS services. This tool supports the generation of modules that are supported by the Cloud Control APIs. They follow the same code template the Content Builder scaffolds in a flash. The tool takes care of formatting the modules using Black. It is the same for VMware. VMware modules are based on the VMware vSphere REST API interface. Whenever a new version of the REST APIs is released, which may include support for new services for the platform, the Content Builder tool can be run to generate the new modules. Once the code is scaffolded, we can start with the automation almost immediately using the new modules. 

How to use the Ansible Content Builder

The content_builder tool takes a manifest file as input. This YAML file provides the necessary arguments for the tool to generate the cloud content. The manifest file for amazon.cloud content generation looks like this:

manifest.yaml
---
collection:
  path: collections/ansible_collections/amazon/cloud
  namespace: amazon
  name: cloud
plugins:
  - type: module_openapi
    name: "amazon_cloud"
    content: cloud
    api_object_path: api_spec
    resource: config
    action: generate_all
    unique_key: ""
    rm_swagger_json: ""
    module_version: "1.0.0"
    author: "Ansible Cloud Team"

For the VMware content, a few values from the above YAML file are changed to suit the VMware content generation.

manifest.yaml
---
collection:
  path: collections/ansible_collections/VMware/vmware_rest
  namespace: vmware
  name: vmware_rest
plugins:
  - type: module_openapi
    name: "vmware_rest"
    content: cloud
    api_object_path: api_spec
    resource: config
    action: generate_all
    unique_key: ""
    rm_swagger_json: ""
    module_version: "1.0.0"
    author: "Ansible Cloud Team"

The following table explains what these key, value pairs in the manifest.yaml mean.

Both plugins:unique_key and plugins:rm_swagger_json keys are irrelevant for the cloud content generation, but mandatory. In the future, these arguments will be made optional.

To generate the content, we use the following command:

ansible-playbook build.yaml -e manifest_file=manifest.yaml

build.yaml
---
- hosts: localhost
  gather_facts: yes
  roles:
    - ansible.content_builder.run

Generation of amazon. cloud

The amazon.cloud modules can be generated by listing the modules in modules.yaml. Setting plugins: action to generate_all will result in the execution of generate_schema, generate_modules, and generate_examples. These actions can be performed individually by specifying the respective action in the plugins: action parameter. For the generate_modules to be performed separately, the API specification JSON files should be present and the api_object_path input argument should be set to the path. Similarly for the generate_examples to be executed separately, the modules should be available, and the path to these modules should be given through the collection: path parameter.

Following is the snippet from the execution of the content_builder with the above-mentioned manifest file (amazon.cloud) as input.

TASK [Generate cloud content] ******************************************************************************************************************************************
TASK [ansible.content_builder.module_openapi_cloud : Generate schema for "cloud"] **************************************************************************************
Collecting Schema
AWS::Backup::BackupVault
Collecting Schema
AWS::Backup::Framework
Collecting Schema
AWS::Backup::ReportPlan
Collecting Schema
AWS::EKS::Cluster
ok: [localhost]
TASK [ansible.content_builder.module_openapi_cloud : Generate modules for "cloud"] *************************************************************************************
Generating modules AWS_Backup_BackupVault
Generating modules AWS_Backup_Framework
Generating modules AWS_Backup_ReportPlan
Generating modules AWS_EKS_Cluster
ok: [localhost]
TASK [ansible.content_builder.module_openapi_cloud : Generate examples for "cloud"] ************************************************************************************
…….
<output trimmed>
………..
Updating eks_cluster
Updating eks_fargate_profile
Updating iam_server_certificate
Updating logs_log_group
Updating s3_bucket
ok: [localhost]
TASK [ansible.content_builder.module_openapi_cloud : Generate ignore files for "cloud"] ********************************************************************************
skipping: [localhost]
TASK [ansible.content_builder.module_openapi_cloud : Format the files in the collection using black] *******************************************************************
ok: [localhost -> 127.0.0.1]

The tool generates all the modules mentioned in modules.yaml under collection:path.

The JSON schema files will be generated under ./api_spec (based on the given manifest file).

Generation of vmware.vmware_rest

Similar to amazon.cloud, VMware modules can be generated by listing the modules in modules.yaml and using the appropriate actions (generate_all, generate_modules, generate_examples, and genrate_ignore_files). For the VMware collection, the schema files are pre-generated using https://github.com/vmware/vmware-openapi-generator. The location of these schema files is provided to the content_builder tool by setting api_spec key to the respective value.

With the following pre-generated schema files, the generation of VMware modules using the Content Builder tool and the execution will look like the following. 

Schema files:

api.json  appliance.json  cis.json  content.json  esx.json  hvc.json  __init__.py  stats.json  vapi.json  vcenter.json

Execution output:

TASK [Generate cloud content] ******************************************************************************************************************************************
TASK [ansible.content_builder.module_openapi_cloud : Generate schema for "vmware_rest"] ********************************************************************************
skipping: [localhost]
TASK [ansible.content_builder.module_openapi_cloud : Generate modules for "vmware_rest"] *******************************************************************************
Generating modules from vcenter.json
- do not build: vcenter_authentication_token
….
<output trimmed>
………….
Generating modules from appliance.json
No parameters for get /api/appliance/system/time
- do not build: appliance_health_applmgmt
- do not build: appliance_health_database
- do not build: appliance_recovery_backup_schedules_update
- do not build: appliance_recovery_backup_validate
- do not build: appliance_recovery_restore_validate
ok: [localhost]
TASK [ansible.content_builder.module_openapi_cloud : Generate examples for "vmware_rest"] ******************************************************************************
….
<output trimmed>
……….
Hiding register _result.
Hiding register _result.
Hiding register _result.
Hiding register _vm_hardware_ethernet_2.
Hiding register _result.
Updating appliance_access_consolecli_info
Updating appliance_access_consolecli
Updating appliance_access_dcui_info
Updating appliance_access_dcui
….
<output trimmed>
……….
Updating appliance_networking_dns_hostname_info
Updating appliance_networking_dns_hostname
Updating appliance_networking_dns_servers_info
Updating appliance_networking_dns_servers
Updating appliance_networking_firewall_inbound_info
Updating appliance_networking_firewall_inbound
Updating appliance_networking_interfaces_ipv4_info
Updating appliance_networking_interfaces_ipv4
Updating appliance_networking_interfaces_ipv6_info
Updating appliance_networking_interfaces_ipv6
….
<output trimmed>
……….
Updating vcenter_vm_guest_networking_routes_info
Updating vcenter_vm_power_info
Updating vcenter_vm_guest_identity_info
Updating vcenter_vm_tools
ok: [localhost]
TASK [ansible.content_builder.module_openapi_cloud : Generate ignore files for "vmware_rest"] **************************************************************************
ok: [localhost]
TASK [ansible.content_builder.module_openapi_cloud : Format the files in the collection using black] *******************************************************************
ok: [localhost -> 127.0.0.1]

The vmware_rest modules for the given schema files are generated under collection:path.

This tool does not support the generation of ‘RETURN’ blocks for the VMware modules for now. These blocks can be generated by following these steps.

What’s next?

The plan for this tool is to extend the code scaffolding capabilities across different content verticals. In particular, for the REST API-based content generation, the plan is to extend the module_openapi role to support the generation of other REST API-based Collections.

For more details on amazon.cloud and vmware.vmware_rest content generation, refer to

1. https://github.com/ansible-collections/amazon.cloud
2. https://github.com/ansible-collections/vmware.vmware_rest
3. https://github.com/ansible-community/ansible.content_builder

What can we do to make the content_builder tool more usable? What type of content generation do you want to add to this tool? We would love to hear from you. Please do give your feedback and suggestions by submitting an issue or pull requests here. You can also join us on the #ansible-aws channel on Libera chat and #aws:ansible.com on Matrix.

Where to go next

• Come visit us at AnsibleFest, now a part of Red Hat Summit 2023.
• Missed out on AnsibleFest 2022? Check out the Best of AnsibleFest 2022.
• Self-paced lab exercises - We have interactive, in-browser exercises to help you get started with the Ansible Automation Platform.
• Try the Ansible Automation Platform for free for 60 days.