In November 2013, a major government office announced that it was undertaking a strategic review of one of the mission critical systems that it owns and operates. The assessment was initiated to consider the Agency and industry requirements of those systems in anticipation of substantial upgrades that will be necessary over the next 2-5 years.
Datacom secured the tender to host, build, support and operate the infrastructure for the replacement project that included complex multi-tiered applications with strict security protocols and high-availability. The actual implementation started at the beginning of 2017.
The system requires high availability and reliable architecture with geographically separated redundancy and high level of security and integrity. The infrastructure build and all the other underlying application requirements are automated wherever possible. Datacom explored possible automation tools available and Ansible was the selected automation engine of choice. It was chosen primarily for orchestration purposes to allow Automation to be developed in native languages in addition to direct Ansible automation. The project is delivered using Agile methodologies (i.e. Scrum and Sprints) alongside other CI/CD kits for the development and maintenance of the codes, roles and playbooks.
Ansible is chosen for its radical simplicity and efficiency for multi-tier enterprise level deployments. It is a powerful enabler for the wide area of expertise that Datacom already has across technology platforms. A number of teams across the company have already proven Ansible’s benefit in executing repeatable tasks while maintaining consistency and standards with remarkable speed.
Our intent was to ensure all roles were standardised and re-useable to allow us to leverage this automation within other customer accounts.
Datacom engaged RedHat at the beginning of the project on consultations regarding best practices on host inventories, variables structures and Ansible Tower queries alongside the licensing and costs involved. RedHat’s existing Technical Account manager, collaborated on key design decisions during his time as Datacom architect in the commencement phase of the project. Datacom, as a company has embarked on a major Ansible journey several years leading to this project which gave them a head start with the initial codes and set of engaged staff enthusiastic on automation technologies. They have come across numerous use-cases and proof of concepts which yielded good results.
Apart from the governance and architectural employees involved in the project, the technical working group is composed of SMEs across technologies encompassed in the solution. The universality and simplicity of Ansible as the chosen automation and orchestration tool enabled all SMEs with varied levels of automation coding experience to speak a single language and translate their expertise to Ansible builds and operations. A prime example is where a junior SME was able to automate Windows server builds and server patching amongst other tasks. Engineers and BAU operations staff translated their experiences on manual tedious operational tasks and builds into an automated solution. Database Administrators were able to build and operate databases more efficiently than before. The initial focus was to automate the build and deployments but because of the advantages brought about by Ansible the focus then shifted to automated operations and management of the system using Ansible and Ansible Tower.
After 2 years, tens of thousands of lines of codes and the creation of hundreds of playbooks and roles, the team has coded the build requirements and stood up all necessary environments using Ansible – including servers and applications installed with ongoing performance benchmarking, fine-tuning and extensive testing by a dedicated test team.
Operational tasks such as OS patching, applications restarts and DR/site switching also have working job templates in Ansible Tower. Certain tasks for non-technical business end-users such as synthetic transaction tests and performance loads were also created and are now being utilized via Ansible Tower.
Joseph Tejal, Datacom New Zealand