Agile & DevOps Are Not Just a Fad

The Agile methodology represents a flexible and adaptive approach to software development. The basic concept of agile is to deliver software in small chunks, instead of a complete application, to get feedback and continuously evolve the product.

The spirit of agile lies in welcoming the change, encouraging collaboration between team members, making decisions on-the-go, and continuously planning and testing the artifact. The main target of an agile team is to satisfy the needs of the customer effectively and to reduce the time to market.

Agile is not a technique in itself; instead, it is driven by a set of core values. It is an umbrella term which covers Scrum, Extreme Programming, Lean Development, Feature Driven Development and other variants.

Agile enhanced the communication between development teams, quality assurance teams and business analysts. However, it could not successfully bring operations into the picture. This gap paved the way for the next step in the evolution of software development - i.e. DevOps.

This e-book is inspired by the webinar ‘Agile and DevOps are not just a FAD – Mastering Test Automation for Agile and DevOps’ delivered by Reginald Moore and Jeffrey Martin. In the webinar, both veterans shared a real-life experience of how they overcame the challenges encountered when practicing Agile and DevOps methodologies, while keeping aligned with their organizational business goals.

Who this e-book is for?

This e-book is a guideline for any individual who wants to develop a system where development teams and operations work together to deliver quality code to the clients, meeting the business goals in a fast-paced environment.

What is in the e-book?

This e-book is a step-by-step journey of developing a framework that could enable you to deliver quality code in less time without compromising the quality. The main objective of this e-book is to enlighten you on how you can successfully automate the testing process in your organization to ensure faster delivery of software artifacts to cope with a rapidly changing business environment. Be prepared to learn the following topics by tuning into an easy-to-understand real life journey of our veterans:

1. A brief introduction of Agile and DevOps
2. Agile and DevOps are not just a F.A.D
3. Understanding the components of F.A.D
4. Real life example of building a test automation framework for delivery – FAD; using Test Complete
5. Insights from the experience of building F.A.D

What is the output of e-book?

By the end of this e-book, you will be able to clearly understand the concepts that lay the foundation of a successful and reusable framework. You will be able to follow the example and develop a test automation framework tailored for your organization. Thus, enabling your organization to set and meet higher targets for the business.

You can enable Agile and DevOps in your organization by understanding how to create a framework that can be used for all types of automation and can work with your delivery model. You can take your test automation practices to the next level with distributed testing techniques and building upon a Selenium based test framework.

In the rapidly changing business environments of today, it has been a daunting challenge for the software industry to deliver products that satisfy the needs of the customer, yet maintain the reduced time to market to stay competitive.

About two decades ago, the veterans of the software industry realized that traditional methods of software development cannot be used to address the challenge. They realized that some other methodology would be required to deliver the software products in an efficient, more adaptive, and flexible manner. This idea gave birth to the ‘Agile’ and ‘DevOps’ methodologies.

DevOps is a relatively new concept in the software industry. It can be considered a step forward from Agile. In DevOps, the development engineering team and the operations team collaborate to deliver an improved product to satisfy customer needs with a significantly shorter time to market. DevOps encompasses the entire service lifecycle, from the design of the product and development, to testing, release and production support.

The main difference between Agile and DevOps is that the operations team also tries to use technology, like the development team, to enhance the overall delivery and post delivery process. The aim of DevOps is to build a system in which software artifacts can be quickly developed, tested, released, and deployed to production with the help of automation techniques.

DevOps can be effectively implemented in an organization with the help of several tools such as code control repositories, tools for the automation of testing such as TestComplete, Continuous Integration (CI) servers for automatic deployment, and through the integration with reporting tools, like QAComplete.

DevOps sounds challenging to implement, especially if your entire team is not familiar with the concept. But not to worry! You are going to learn how to build a test automation framework for continuous delivery.

The team put a lot of effort in making a framework which could interact in the bigger ecosystem. They did choose two tools, TestComplete and QAComplete, of SmartBear.

However, the framework made use of tools from other vendors as well. For example, Cucumber was used by developers to do unit testing and initiated acceptance test driven model. Gradually, business analysts also started to write test cases using cucumber and played a role in implementation of F.A.D.

For implementing DevOps model, Octopus was used for continuous integration process. It had a plug in that provides capability to run the scripts from command line easily.

A Cultural Shift - Speaking the same Language

You know you have been successful in your endeavors when there is a visible change of culture in your team. It is the point where every team member starts to speak the same language. Whenever a requirement comes in, team thinks about adjusting framework to update the code base, they know the tools that will be used in the due process of test automation and the tools to support delivery of quality code. In other words, the team starts to think ‘What is the F.A.D around it?’

Ultimately, it becomes a routine to add things to your framework as everyone understands the enterprise level picture and know what is going on.

How Test Complete Helped – Value Story

F.A.D is an acronym for Framework, Automation, and Delivery. Using the F.A.D methodology, a comprehensive and reusable framework can be built. This framework can be organized and customized easily to meet the changing business needs of the organization.

Framework

The organizational goal is to create a flexible framework which can be used for all types of testing such as user interface testing, functional testing, regression testing, and smoke testing. It should have the capability to plug and play with any type of automation. Realistically, your application dictates your framework.

The framework should have real objectives and add value to the business of the organization. You need to identify your problems, think how you are going to address them, and understand why you need to solve them.

You need to come up with certain Key Performance Indicators (KPIs) to measure the success of your framework. Business stakeholders and management should be able to comprehend and make informed decisions based on these metrics and KPIs.

Our objective is to create a framework that assures faster delivery at reduced costs that follows a DevOps delivery methodology.

The Golden Circle Technique

The Golden Circle technique can be used to create an effective framework for your organization. It propounds the idea of building the framework out from the inside. It has three sequential components: Why, how and what.

Understanding the What

The first step in the golden circle is to understand, ‘What needs to be done?’ In our case, we need to reduce costs without compromising quality. It is the process of identification of the correct framework that corresponds to the automation tools used in the organization.

Another facet of ‘What’ is to achieve what Reginald Moore termed the blue-chip value (i.e., making the best use of an individual’s time by reducing the boring, repetitive tasks and spend time on things which are valuable).

Understanding the How

The second step is to ask, ‘How are you going to do it?’ In our case, we are going to create a framework that can be quickly implemented, updated, and used by the team for continuous delivery.

To implement ‘How’, it is important to identify what things are needed to maintain the code, such as a code repository.

Another aspect is to understand the skill set of your team members. Get answers to questions like, ‘Who has automation skills?’ Or, ‘Who can learn quickly?’ Who is going to be the primary trainer and teacher?’ An insight of the skill sets of the team will help you in planning how you can implement the framework and who you can use as your helping hands.

Understanding the Why

The third but the most important part is answering the ‘Why’. The management and business stakeholders are mainly interested in the ‘Why’ of the golden circle. A convincing answer to ‘Why’ will get you permission and resources from upper-level management to develop and use the framework.

In our case, we are building a framework to help deliver improved quality code for the business partners of the organization. Automation will reduce costs and enable faster delivery.

Moreover, it will enable your team to enjoy a positive work-life balance. This is an important factor to keep them motivated and productive at their jobs.

Automation

Automation is the key process in F.A.D methodology. A lot of people confuse themselves by doing automation for the sake of automation. However, we aim to build an automated process that complements the delivery model and matches the expectations of organization at a higher level.

After a clear understanding of the ‘What, How, and Why’ of the framework, you can figure out how you want to implement automated testing in a way that best serves your client and delivery channels. You can choose between behavior-driven automation or module driven automation.

In short, you can build the right automation system once you have a visualization of your targets at an organizational level.

Base automation on Product Knowledge

Your team should have the product knowledge as the basic requirement when they start their work on building the framework. It is favorable if your team has test automation knowledge as well.

It is essential to base the automated testing process on the product so that it can add value to the business. When your team understands the product and its business case values, it will be able to build a re-usable framework.

Identify Teachers and Trainers

Identify teachers and trainers for sharing the knowledge of automation with other team members. They will build momentum towards the objective. They should teach your team about how to use the framework effectively and enable the team to get to the execution stage of F.A.D.

Delivery

It is vital to understand your delivery model while building a comprehensive framework using F.A.D.

If your organization is following DevOps, you need to build an automation framework that corresponds with the continuous integration development pipeline.

If you are following an agile methodology, you don’t need the advanced continuous delivery mechanism. Note, that you can always steer towards a DevOps model to take your organization to the next level.

A framework needs to be built for a real environment. It should be flexible enough to work with different applications that are developed by different people in the organization. The framework should be able to work with the business that is running in a non-ideal environment.

Let’s take a real-life example of a situation shared by Reginald Moore. He was given the task to automate test cases for a windows- based product, MS Access 2000.

Topography of a Framework

The topography of our framework included:

1. TestComplete, an automated UI testing tool was used to base the automation code. It was linked to the test management tool QAComplete and Cucumber to initiate Acceptance Test Driven Development (ATTD).
2. Selenium Web Driver was used to write automated test cases for the web applications. It was also used to update automation reporting tool.
3. Power BI tool was used to generate automation testing results in the form of reports.

In an organization, it is critical to be able to measure the results and demonstrate the success to management.

In our framework, Power BI was used as a reporting tool to prepare reports of automation testing results. A class was built within Selenium Java Framework that called Power BI to produce data and then, Power BI would produce specific reports. These reports were used to answer all the ‘Why’ questions from the upper management and receive credit for the successful efforts.

To automate a windows-based product, selenium web driver cannot be used. Hence, TestComplete was chosen to base automation code that would link into the framework to run automated tests. In this scenario, TestComplete had three main benefits to offer:

• It had re-usable components.
• It had a plug-in to integrate with the QAComplete, which was already being used by the team.
• TestComplete was easy to use. There are several tutorials, interactive videos, and webinars available on the official site of SmartBear to help users in understanding how things can be done using Test Complete.

Unit of Automation

You can automate a feature or the user story, as required by your organization. The terminology may vary in different organizations. In an Agile environment, a requirement comes in as a ‘User Story’, then it becomes a ‘Feature’ and a set of features form ‘Epic’.

In this case, feature files were used as the unit of automation. The feature files correspond to using anything that is cucumber and is also known as ‘SpecFlow’.

Structure

The structure of the test repository depends on how your organization wants to define and maintain it. If there is a scrum team, each tester might get an assignment. A generic term of ‘requirement’ can be used to understand the concept. As it is the basic thing against which development and automation of testing is being done.

TestComplete enables you to organize folders corresponding to different features. Each folder can contain test cases associated with that feature.

In this case, different agile teams were made. Whenever a feature or user story came in, it was assigned to a specific agile team. Each agile team had their respective folders. The features were maintained in the sub-folders of the respective team.

Statistical Analysis to set up Environments

A statistical analysis of the customer base enables you to focus on the environments which are mainly used by the people who interact your application.

This information can help you spend less time on maintaining the environments; thus, sparing you more time to automate the actual requirements. TestComplete allows you to create virtual machines and maintain different environments to meet your testing needs.

Integration with a Test Case Management Tool

TestComplete can be well integrated with QAComplete, which was already in use to store the test case repository.

It is a key factor to be considered while building framework that the tools can be integrated and used with other tools. This will keep your options open and your organization would not be locked in with a specific vendor.

Integraion of Unit Tests

Although unit tests are written and executed by developers, unit tests written in IDEs such as Visual Studio can be integrated in the TestComplete.

Continuous Integration and Continuous Development

Eventually, TestComplete was used as part of the Continuous Integration and Continuous Development (CI/CD) pipeline. CI/CD is process automation for automated and non-automated testing. Any good framework should, by itself, be able to be automated.

There are several options to do so, such as calling jobs from command line or use Jenkins.

Selenium WebDriver was used to provide support for web applications in our framework. The test scripts recorded and created using Selenium can be exported into TestComplete.

Selenium Web Driver was also used to call different classes and produce reports using Power BI.

Let us wrap up by taking a quick overview of what we have learned so far. We can list down the qualities and functions of a good framework as:

1. It is designed around your application.
2. It automates the testing process and support manual testing too.
3. It is made of reusable pieces to build automation of new requirements, quickly.
4. It is easier to train team members and pass on the knowledge about how to use framework and its constituent tools correctly.
5. It supports the delivery model of organization.
6. It is scalable to the needs of product delivery.
7. It can measure results and produce specific reports for management use.
8. It is flexible to work effectively in a non-ideal environment.
9. It drives business value from the communication between everybody.
10. Everyone owns the quality and lay efforts to successfully follow DevOps model.