A discussion on automated testing

The annual Drupal conference (known as DrupalCon) includes lively discussion sessions called Birds of a Feather, or BoF for short. At the 2023 European DrupalCon in Lille, SystemSeed Full Stack Engineer, Branimir Juranić, hosted a BoF on the topic of automated testing, including the why and how of creating automated test suites.

At SystemSeed, we employ automated acceptance testing to help us minimize errors and speed up the acceptance process across our client projects.

What is automated acceptance testing?

An automated acceptance test is a scripted test that runs automatically within a testing tool. These tests are simulating user behavior, so we can see whether there are any problems with our code. For example, a test can be created for a user to click on the site’s login button, enter their username and password, click login, and then be redirected to the login welcome page. 

Why employ automated tests?

Testing is a vital part of good Drupal development and one of the best ways to do that is through automated tests. However, automated testing is new to a lot of teams, so we wanted to use this discussion as an opportunity to share experiences with a wider group of people and ask questions to each other. 

I shared with the group that I have only been doing automated testing for 18 months myself, and wanted to discuss:

• The benefits of automated acceptance testing in Drupal
• How to do automated testing, including the install module
• How tools like Codeception can help with automated testing

The benefits of automated testing:

Rather than testing everything manually, your code can be verified through automated acceptance testing. 

To do this, you need to define the user behavior for each step in your test, and once the test has run through all the steps, you will receive a verification that the code is fully functional.

There are a few key benefits of automating acceptance tests:

• Saving costs - More often than not, one hears how expensive automation can be to set up and implement initially. This is true, but here’s what most testers, developers or managers ignore: manual testing costs far more in the long run, simply in terms of the work-hours required and the additional cost of fixing human errors.
• Faster feedback loop - With faster tests and quicker results, teams can get feedback more frequently, which helps them to detect bugs earlier and improve functionality.
• Better allocation of resources - QAs (quality assurers) can devote their time to activities that actually require human discernment and intellectual engagement. For example, they can spend more time on formulating new test cases, rather than actioning old ones. Additionally, with fewer monotonous tasks to deal with, testers are likely to be more engaged in their day-to-day work. This keeps them happier and raises team morale – all of which leads to a better quality of work. 
• Higher accuracy - Given that manual testers are human, they will make mistakes. No matter how meticulous they are, manual testers cannot be expected to deliver 100% accurate results. People get tired, miss steps, or make other errors. However when automated, the testing tool can run a script perfectly every single time.

How to do automated testing - an example

In our work with Concern Worldwide, we run automated tests on new modules in development environments before they are pushed to production. One of the main ways we do this is by using Codeception.

How tools like Codeception can help

What is Codeception?

Codeception is an open-source test automation framework for PHP that allows you to write tests in different formats, including: 

• BDD (behavior-driven development)-style scenarios, 
• PHP code, or 
• YAML files. 

Codeception also supports different types of tests such as: unit, functional, acceptance, and AP tests. 

Codeception has a modular architecture, which means you can enable or disable different modules depending on your needs.

How we write tests at Systemseed

We use a straightforward process to write automated tests:

1. Each page or feature has its own file in the development environment. 
2. Inside the file, we have separate functions for different tests we do for each feature or page. 
3. We divide tests into smaller fragments to be more readable and for easier maintenance.

How to test certain elements on a page - examples

If you have multiple items of the same kind to test, you can target them with XPath (XPath is an expression language designed to support the query or transformation of XML documents).

We use CircleCI to build automated pipelines, from testing to deployment. Engineers can automate their entire test suite for new commits, thereby reducing human errors. 

We have 20 containers and each container has its own set of tests. In the past we had long tests, and we decided to break them down into smaller fragments for manageability, and all tests are run in a parallel process. 

My overall experience so far

When I first heard of automated acceptance testing, it looked interesting. Then, as I started to take a deeper look and my mentors explained it, it was love at first sight! Because we use Codeception (which uses PHP) it was much easier for me to understand it from the beginning. What I really like is that you use the tests to mimic user behavior and test functionality. 

You also get a response at once, so it is very easy to debug or upgrade your code. And most importantly, if you want to be good at something it helps if you love what you are doing too.

Feedback from the group

Our discussion showed that most of the participants aren’t currently doing automated testing and have manual testing processes instead. In fact, some of them said they were only doing content testing.

If I were to run a group discussion on this topic again, I’d like to spend more time on how to set up automated tests, so that they can leave the session understanding where they can begin with automated testing on their own projects.

Interested in seeing how automated testing can help you?

We’d love to help you implement automated testing for your own sites, and can even begin with an initial audit to discover any issues. Contact us to start the conversation here: systemseed.com/contact.