Installing and using collections on Ansible Tower

Installing and using collections on Ansible Tower

Ansible Collections are the new way to distribute and manage content. Ansible content can be modules, roles, plugins and even Ansible Playbooks. In my previous blog, I provide a walkthrough of using Ansible Collections from Ansible Galaxy and Automation Hub.  Ansible Galaxy is the upstream community for sharing Ansible Collections.  Any community user can create a namespace and share content with anyone. Access to Automation Hub is included with a Red Hat Ansible Automation Platform subscription. Automation Hub only contains fully supported and certified content from Red Hat and our partners.

In this blog post we'll walk through using Ansible Collections with Ansible Tower, part of the Red Hat Ansible Automation Platform.  There are a few differences between using command-line Ansible for syncing with Ansible Galaxy or the Automation Hub versus using Ansible Tower. However, it is really easy and I will show you how!

Accessing collections content from Automation Hub and Galaxy from Ansible Tower.

If the Ansible Collections are included in your project you do not need to authenticate to Automation Hub. This method is where you are downloading dynamically using a requirements file. In general there are three strategies for using Ansible Collections with your environment:

  1. Install the collection into your runtime environment or virtual environment
  2. Provide the collection as part of your SCM tree 
  3. Use a requirements file

When accessing content from Automation Hub, the authentication token and authentication URL configuration has to be made in Ansible Tower's settings. 

Note: Even if you have the authorization details in your ansible.cfg file within the project repo, it will not be picked up by Ansible Tower. You will need to enter these details in the Ansible Tower settings

In order to do this, navigate to the Settings > Jobs  sidebar link from the Ansible Tower administration section.

tower jobs ui

Click on Jobs and update the following fields:

  1. PRIMARY GALAXY SERVER URL: https://cloud.redhat.com/api/automation-hub/
  2. PRIMARY GALAXY AUTHENTICATION URL: https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/token
  3. PRIMARY GALAXY SERVER TOKEN: xxxxxxxxxxxxxxxxxxxxxxxxx......

Red Hat recommends using Automation Hub for your primary Galaxy Server URL to ensure you are using certified content that is fully supported via your Red Hat Ansible Automation Platform subscription

tower ui screenshot

This is the only step you need to do in Ansible Tower in order to download the certified collection from Automation Hub which was defined in the collections/requirements.yml file.

The playbook is invoked via a job template as usual.

tower ui screenshot

Launching this job template results in the collections being pulled down from Automation Hub and Galaxy and invoking modules made available through those collections.

ajay collections

Conclusion

Ansible Collections introduce a way to modularize and package automation content effectively. The Red Hat Automation Hub hosts certified collections that are validated and supported by Red Hat. Ansible Galaxy hosts community contributed collections. Customers can access collections from both content repositories. I think of collections as superchargers to the "batteries included" approach that Ansible takes. It up-levels the nuances involved in building out automation, allowing users to plug-n-play the latest and greatest automation content being built by certified partners and the community.




Ansible security automation resource modules

Ansible security automation resource modules

Security professionals are increasingly adopting automation as a way to help unify security operations into structured workflows that can reduce operational complexity, human error, time to respond and can be integrated into existing SIEM (Security Information and Event Management) or SOAR (Security Orchestration Automation and Response) platforms.

In October of 2019 the Ansible network automation team introduced the concept of resource modules.

So what exactly is a "resource module?" Sections of a device's configuration can be thought of as a resource provided by that device. Network resource modules are intentionally scoped to configure a single resource and can be combined as building blocks to configure complex network services.

Keep in mind that the first network automation modules could either execute arbitrary commands on target devices, or read in the device configuration from a file and deploy it. These modules were quite generic and provided no fine-tuning of certain services or resources.

In contrast, resource modules can make network automation easier and more consistent for those automating multiple platforms in production by avoiding large configuration file templates covering all kinds of configuration. Instead they focus on the task at hand, providing separate building blocks which can be used to describe complex configurations.

The same principle can be applied in the security space, and we started exploring the possibility with Ansible security automation.

Resource modules in security automation

In security automation, many Collections already have more refined modules targeting use cases or workflows of the corresponding target environment. Therefore, there is little standardization or generic abstraction in terms of product agnostic resources.

For example, if you have a closer look at our investigation enrichment blog post, you will see that while we used a certain amount of modules, those were usually very product specific and didn't offer much in terms of generic resources.

At the same time, security automation does cover many tasks where resource modules can add a lot of value. Whether it is granting and denying access to networks via Access Control Lists (ACLs) or policies, the management of rules in IDPS systems or the log forwarding of nodes to a central SIEM: all those tasks are often executed on a well-defined resource across multiple products, which makes these tasks good candidates to be helped by resource modules.

Security automation resource modules for Access Control Lists

Following this line of thought we have started to introduce ACLs resource modules within Ansible. ACLs can help provide a first layer of security when applied to interfaces, or globally as access rules, as they permit or deny traffic flows in firewalls.

Within an ACL the order of Access Control Entries (ACEs) are crucial since based on the ACEs sequence/order, appliances decide whether traffic is allowed or not. Given this background, an ACL resource module provides the same level of functionality that a user can achieve when manually changing the configuration on a corresponding device. However, the ACL resource module comes with the advantages of Ansible:

  • Automating things using Ansible can accelerate the time to become productive.
  • Ansible is powerful and users can automate a wide variety of tasks,  at both the user or enterprise level. This helps to orchestrate the complete app lifecycle including the ACLs, and makes the security automation part of the app deployment process and the entire technical business process.
  • Ansible has agentless architecture which uses the native communication protocols of the managed target nodes. This avoids the need to introduce and install new software and new security protocols in the managed environments.
  • Last but not least, with the help of Ansible's fact gathering, the data structures of managed nodes can be collected and made accessible in an efficient manner.

Please note that the naming convention for the new ACL resource modules uses the plural form instead of singular: "acls" instead of "acl". If the platform you're automating has modules with both names, the plural form of the module is the newer one corresponding to the resource module initiative. The singular form of the module will likely be deprecated in a future release. This distinction was introduced to ease the transition to resource modules and avoid disruption of the current automated workflows.

Example: Cisco ASA ACLs

A good way to understand the new ACLs resource modules is via an example. For this, let's have a look at the Cisco ASA Collection which targets the Cisco Adaptive Security Appliance family of security devices. In this Collection you will find a module called asa_acls which is the resource module to manage named or numbered ACLs on ASA devices.

As an example, let's first check the current documentation. For that, we can use the capability of the module to gather the existing ACLs configuration:

---
- name: Get structured data
  hosts: cisco
  gather_facts: false
  collections:
   - cisco.asa

  tasks:
  - name: Gather facts
    asa_acls:
      state: gathered
      register: gather
  - name: output data
    debug:
      vars: “{{ gather }}”

The output will be something along the lines of:

- acls:
   - aces:
       - destination:
           address: 192.0.3.0
           netmask: 255.255.255.0
           port_protocol:
                 eq: www
         grant: deny

Note that the output generated this way is purely focused on the resource at hand - ACLs. This is in contrast to a generic fact gathering wheremore data is provided, making it difficult to keep an overview and handle the data subsequently.

Given the configuration at hand, let's assume for the sake of this example that we analyze the gathered configuration, and want to make a change to it. The next configuration looks like:

- acls:
  - name: global_access
    acl_type: extended
    aces:
    - grant: deny
      line: 1
      protocol_options:
        tcp: true
      source:
        address: 192.0.4.0
        netmask: 255.255.255.0
        port_protocol:
          eq: telnet
      destination:
        address: 192.0.5.0
        netmask: 255.255.255.0
        port_protocol:
          eq: www

This configuration describes that access from a defined source to a target is denied. Note that this entire definition is mostly product agnostic and can be used with other systems as well.

Given this description is available as the variable acls this can be deployed with the cisco_acls module:

---
- name: Replace ACLs device configuration
  hosts: cisco
  gather_facts: false
  collections:
   - cisco.asa

  tasks:
  - name: Replaces device configuration of listed acls
    asa_acls:
      config: “{{ acls }}”
      state: replaced

As you see it is possible to apply an existing resource description to an existing device. Resource modules allow the user to read in existing configuration and convert that into a structured data model. These data models can be used as a base to further deploy changed configuration on the target nodes.

Takeaways

Security professionals are in need of unification of their operational workflows. Automation helps - even more so if the platform it is running on provides a simpler means to control otherwise rather complex structures. The Ansible security automation resource modules provided are a building block in standardizing automation actions.




Bullhorn #5

Ansible Bullhorn banner

The Bullhorn

A Newsletter for the Ansible Developer Community

Welcome to The Bullhorn, our newsletter for the Ansible developer community. If you have any questions or content you’d like to share, please reach out to us at the-bullhorn@redhat.com.
 

ANSIBLE-BASE 2.10 BETA 1 RELEASED

On June 17th, the Ansible Core team released the first beta of ansible-base-2.10.

The ansible-base package consists of only the Ansible execution engine, related tools (e.g. ansible-galaxy, ansible-test), and a very small set of built-in plugins; the remaining content has been moved into Ansible collections, which can now be installed separately. For more details on what’s new, installation instructions, and a link to the full changelog, read Matt Davis’s announcement to the ansible-devel mailing list.
 

ANSIBLE-2.10 ALPHA 1 RELEASED

On June 18th, the Ansible Community team released the first alpha of ansible-2.10. This Ansible package is comparable with, and intended to be a drop-in replacement for, previous versions of Ansible; collections with all previously available modules have been reintegrated into this package. For more details on installation instructions, known issues, and upcoming release plans, read Toshio Kuratomi’s announcement to the ansible-devel mailing list.
 

NEW ANSIBLE CONTENT COLLECTIONS RELEASED

The initial release of Red Hat-maintained Ansible Content Collections have been published to Automation Hub for automating select platforms from Arista, AWS, Cisco, IBM, Juniper, Splunk and more. Community versions are also available from Ansible Galaxy. For more information, read Andrius Benokraitis’s blog post on ansible.com.
 

NEW ANSIBLE COMMUNITY COLLECTIONS RELEASED

Two key Ansible community collections, community.general and community.network, reached 0.2.0 last week, which is an important milestone in their development. See https://github.com/ansible-collections/community.general/issues/507 and https://github.com/ansible-collections/community.network/issues/64 for more information on their release cycles.
 

REMINDER: VIRTUAL ANSIBLE CONTRIBUTOR SUMMIT

The next Ansible Contributor Summit is less than two weeks away! It’s scheduled for Monday, July 6th, with hackathon days to follow on the 7th and 8th. Please register on Eventbrite if you plan to attend any part of the event.

For the agenda, check out the “Potential Topics” section on the etherpad, add topics you’d like to see discussed or vote on existing topics.
 

COMMUNITY WORKING GROUP HIGHLIGHTS

The community working group meets on Wednesdays at 18:00 UTC in #ansible-community on Freenode IRC. You can import this URL community.ics to your own calendar if you’d like to receive reminders. To see what's being discussed and add your own topics see the agenda.

One of the aims of the Bullhorn is to keep people informed about important changes to contributors. With the move to a distributed codebase, in the form of collections and their repos, it's even more important that we improve communication. So, we will be including a summary of the recent changes that you as (potential) contributors or collection maintainers should be aware of.

Changes impacting contributors:

  • As ansible-base 2.10 beta1 has been released, now is the time to ensure that your repos are running ansible-test sanity from the ansible/ansible:stable-2.10 branch. If you don't yet have CI setup you can copy the "Sanity" section from ansible-test GitHub Action example.
  • Also on the CI front, you should copy tests/sanity/ignore-2.10.txt to tests/sanity/ignore-2.11.txt in preparation for running sanity tests from stable-2.10 and devel branches of ansible-base.
  • We are happy to announce that the way to define and document deprecations has been agreed on. Ansible-test (included in ansible-base 2.10) has been updated to check these, so you should review and update as needed.
  • As deprecations are now defined in a collections meta/runtime.yml, ANSIBLE_METADATA is no longer needed, so you can bulk delete this from all modules in your collection, see dropping ANSIBLE_METADATA for details.
  • To speed up your unit and integration tests you may wish to review the Python dependencies you are pulling in, check out part A and part B for more information.
  • There is now a process for automatically generating changelogs, for information on how to achieve this check out this guide.

We recommend that you subscribe (with the button on the right hand side) to https://github.com/ansible-collections/overview/issues/45 which is where changes will be announced in advance.

If you need help with any of this, please join us in #ansible-devel on Freenode.
 

ANSIBLE PULL REQUEST REVIEW DAY

On Wednesday the 17th July we hosted the first "Big PR review day" since the move to collections. It was a good opportunity for people to find out about the collections work and how things are progressing. We had 15 people join in, of which two were brand new contributors to Ansible.

The next PR review day will be in August, subscribe to Big PR Review Day to find out when.
 

ANSIBLE VIRTUAL MEETUPS

The following virtual meetups are being held in the Ansible community over the next month:

Ansible India Meetup: Ansible 2.10, Collections, and more!
Sat, Jun 27 · 10:00 AM UTC+5:30
RSVP with one of the meetup groups “near” you: Bangalore, Kolkata, Delhi, Mumbai, Aurangabad, Chennai, Hyderabad, or Pune! (They will link to the same virtual event.)

Ansible NOVA: Multicloud Networking leveraging Ansible and Pureport!
Tue, Jun 30 · 4:00 PM EDT
https://www.meetup.com/Ansible-NOVA/events/271232565/

Ansible New Zealand: Ansible for Windows. How, Why and When?
Thu, Jul 9 · 12:00 PM GMT+12
https://www.meetup.com/Ansible-New-Zealand/events/271155213/

Note: For these virtual events, the links to participate will be visible once you RSVP to attend. If you’re interested in the topics presented, you can join from anywhere in the world as long as the time zone and language works for you!

Interested in presenting at an Ansible virtual meetup? Sign up here: https://forms.gle/aG5zpVkXDVMHLERX9
 

FEEDBACK

Have any questions you’d like to ask, or issues you’d like to see covered? Please send us an email at the-bullhorn@redhat.com.

 

 




Bullhorn #4

Ansible Bullhorn banner

The Bullhorn

A Newsletter for the Ansible Developer Community

Welcome to The Bullhorn, our newsletter for the Ansible developer community. If you have any questions or content you’d like to share, please reach out to us at the-bullhorn@redhat.com.
 

VIRTUAL ANSIBLE CONTRIBUTOR SUMMIT

The next Ansible Contributor Summit will be on Monday, July 6th, with hackathon days to follow on the 7th and 8th. Please register on Eventbrite if you plan to attend any part of the event, and add agenda items for potential discussion on the etherpad.

 

ANSIBLE BIG PR DAYS

The awesome Ansible community contributes lots of pull requests (PRs) every month. Sometimes the volume of incoming PRs can mean contributors having to wait a long time for them to be merged, or even reviewed. We are going to change that by having dedicated PR review days.

The next one will be Wednesday 17th June 2020 from 14:00 UTC (what's this in my timezone). We’re really excited about this, as it will be the first one to focus on Collections.

What we want to achieve:
  • Give potential new community contributors a place to learn and experiment with Ansible's review process

  • Improve the process and documentation by getting feedback from you

    • Feedback from people starting their journey with open source is particularly important to us, as it helps to identify and improve processes and documentation that may have evolved over time to become complicated. People that have been contributing for a while have already built up this knowledge and have forgotten what it was like to simply not know how to do something!

  • Identify PRs that can be merged or closed

Help PRs that are not covered by one of the existing Ansible Working Groups, and therefore may accidentally fall through the cracks

If you have some time during the day to join us, please feel free to take part or even just listen in.

For more information on how to join see https://github.com/ansible/community/issues/407

 

NEW WEEKLY COMMUNITY MEETING

Based on community feedback, we have regular weekly meetings on Wednesdays at 18:00 UTC in #ansible-collections on Freenode IRC. You can import this URL community.ics to your own calendar if you’d like to receive reminders.

This is a great opportunity to find out about the progress being made towards Ansible 2.10 and specifically moving from the monorepo (github.com/ansible/ansible) to the many Collection repositories.

To see what's been discussed so far, take a look at  https://github.com/ansible/community/issues/539

 

COMMUNITY DATA ANALYSIS

Greg Sutcliffe, our community data scientist, has a new blog post on the real statistics of community.general and why it pays to properly test your code. It turns out the graph we published in issue #2 was incorrect, and he details what happened, what the real graphs are, and where we're taking this. Here's a teaser:

 

 

COMMUNITY CONTENT

Every month we notice the community posting great content. Although not all strictly developer focussed, maybe there’s an article or two here that piques your interest? Let us know if you’d like to see more of this.
 

Make integration tests for your Ansible Collection fast! A story of how Steampunks made Sensu Go Ansible Collection’s test times as short as possible.
 

Serge van Ginderachter has a gist with example code for an inventory plugin.
 

Testing Ansible roles using Molecule and Podman. Ilkka Tengvall is an Ansible Solutions Architect based in Helsinki, Finland. Here he talks through his role testing setup, and even has a short video for your enjoyment too.

Mint Security have written about Installing Splunk with Ansible.
 

Stuart Cunliffe from IBM has a lengthy discussion on Ansible on YouTube. He talks about how it relates to IBM Power Systems and its integration with IBM PowerVC. In this video he demonstrates Ansible modules, playbooks and roles, as well as showing Ansible deploying new PowerVM LPARs.

 

ANSIBLE VIRTUAL MEETUPS

The following virtual meetups are being held in the Ansible community over the next month:

 

Ansible Paris Meetup #19

Fri, Jun 12 · 6:30 PM UTC+2

https://www.meetup.com/Ansible-Paris/events/271033303/

 

Ansible Minneapolis: Self-Service Infrastructure Provisioning in the Cloud

Thu, Jun 18 · 6:30 PM CDT

https://www.meetup.com/Ansible-Minneapolis/events/sbqkgrybcjbxb/

 

Ansible Toronto June 2020

Fri, Jun 19 · 12:00 PM EDT

https://www.meetup.com/Ansible-Toronto/events/271081384/ 

 

Ansible India Meetup

Sat, Jun 27 · 10:00 AM UTC+5:30

RSVP with one of the meetup groups “near” you: Bangalore, Kolkata, Delhi, Mumbai, Aurangabad, Chennai, Hyderabad, or Pune! (They will link to the same virtual event.)


Note: For these virtual events, the links to participate will be visible once you RSVP to attend. If you’re interested in the topics presented, you can join from anywhere in the world as long as the time zone and language works for you!

We are planning more virtual meetups to reach a broader audience, and we want to hear from you! Have you started using Ansible recently, or are you a long-time user? How has Ansible improved your workflow or scaled up your automation? What are some of the challenges you’ve faced and lessons learned? Share your experience by presenting at a Virtual Ansible Meetup: https://forms.gle/aG5zpVkXDVMHLERX9

You’ll have the option to pre-record the presentation, and be available during the meetup for live Q&A, or deliver the presentation live. We will work with you on the optimal set up, and share with you some cool Ansible swag!
 

FEEDBACK

Have any questions you’d like to ask, or issues you’d like to see covered? Please send us an email to the-bullhorn@redhat.com.

 

 




Bullhorn #3

Ansible Bullhorn banner

The Bullhorn

A Newsletter for the Ansible Developer Community

Welcome to The Bullhorn, our newsletter for the Ansible developer community. If you have any questions or content you’d like to share, please reach out to us at the-bullhorn@redhat.com.

 

ANSIBLEFEST 2020 IS NOW A VIRTUAL EXPERIENCE

The AnsibleFest Virtual Experience will be a free, immersive multi-day event the week of October 12th, 2020, that will deliver timely and useful customer keynotes, breakout sessions, direct access to Ansible experts, and more. Read more details at the Ansible blog.
 

NEXT VIRTUAL ANSIBLE CONTRIBUTOR SUMMIT: MONDAY, JULY 6TH


The voting is complete, and the majority of contributors who participated have chosen Monday, July 6th as the date for our next Ansible Virtual Contributor Summit, with virtual hackfest days to follow on the 7th and 8th. To sign up or to add agenda items for potential discussion, please go to the Etherpad for the event. 
 

UPCOMING PREVIEW RELEASES FOR ANSIBLE 2.10

The first beta release of Ansible Base 2.10 is currently scheduled for this Friday, May 29th, and nightly builds of Ansible 2.10 will begin on Tuesday, June 2nd. These nightly builds will be an excellent opportunity to begin testing collections and general functionality, and we encourage collection maintainers in particular to use this time to begin validation of their modules and content.

** We’ll be broadcasting loud and clear when the first nightly build is available, along with where and how to get it; stay tuned! **
 

FIFTY K

This week saw the Ansible project roll over a significant milestone, 50,000 commits!



ANSIBLE LINTING WITH GITHUB ACTIONS

Want to make sure that your Ansible content is checked with every GitHub commit? There’s now a GitHub action to run Ansible Lint. Colin McCarthy walks you through the process in his blog post at ansible.com.

 

AN INTRO TO ANSIBLE WITH NETWORK CHUCK

Have friends who need a quick easy-to-follow introduction to Ansible? NetworkChuck has posted a new Ansible video to his YouTube channel with an easy-to-follow tutorial that takes users from installation through the first playbook. 
 

ANSIBLE VIRTUAL MEETUPS

The following virtual meetups are being held in the Ansible community over the next month:

Ansible London: Virtual meetup – 3rd June
Wed, Jun 03 · 6:00 PM GMT+1
https://www.meetup.com/Ansible-London/events/270746783/
 

Ansible in DevOps Torun-Bydgoszcz: #6 Meeting AiDO -> Terraform
Wed, Jun 10 · 5:00 PM GMT+2

https://www.meetup.com/Ansible-in-DevOps-Torun-Bydgoszcz/events/270881163/

We are planning more virtual meetups to reach a broader audience, and we want to hear from you! Have you started using Ansible recently, or are you a long-time user? How has Ansible improved your workflow or scaled up your automation? What are some of the challenges you’ve faced and lessons learned? Share your experience by presenting at a Virtual Ansible Meetup: https://forms.gle/aG5zpVkXDVMHLERX9

You’ll have the option to pre-record the presentation, and be available during the meetup for live Q&A, or deliver the presentation live. We will work with you on the optimal set up, and share with you some cool Ansible swag!
 

FEEDBACK

Have any questions you’d like to ask, or issues you’d like to see covered? Please send us an email to the-bullhorn@redhat.com.