Test activities in Scrum

Lately I've been absorbed and emerged into the REAL agile development life cycle and how to do do Agile properly. To do so, I got my scrum master certificate, started reading about scrum, worked as a scrum master and went to work in the company that hired Arie van Bennekum, one of the co-founders of the Agile Manifesto.

On my path to learn about agile and scrum,  I learned a lot about Unit testing and integration testing, Test Driven Design (TDD) Design and eXtreme Programming (XP), The definition of done and ready, continuous build and integration... I also earned about acceptance criteria and even Acceptance Test Driven Development. Those terms were not entirely new for me, but let's say I understand them better now since I saw them being put in practice by the teams.

But what about exploratory testing, non functional testing, front-end automation, test data preparation, test environment management for example? And how do testers get their test-related jobs done when they require considerable extra effort? I found, next to the very structured applications of unit testing and TDD, manual testing and test automation aspects as well as test related tasks and estimations were not yet very mature.

As I'm now only 2 years active in Agile development and more particularly in Scrum, I do not claim to own Agile development. But I do claim to know something about testing and where I would take proper testing tasks into consideration while working in for example Scrum. This is how I did it:

I divided the test related tasks into 3 parts

- Taks to be taken up during product backlog definition

- Tasks to be taken up during sprint backlog definition

- Tasks to be taken up during the sprint

Product Backlog definition

Production issues

It is important that production issues ( and maybe also issues coming out of acceptance testing if this is not taken into the definition of done of a sprint) get a proper description and impact and become part of the product backlog.

Test related features and testability items

Features needed to accommodate testing (stubs, drivers, datasets, monitoring plugins, maybe even a button and input field within the application to provide feedback for beta testers) need to be defined and developed in time.

Also product supporting elements like test automation, performance tests, security tests, exploratory sessions are items that could be put on the product backlog.

Sprint backlog definition

Test estimation

During the sprint backlog definition, testing has to make sure that all testing efforts are estimated for every user story that goes into a sprint. Things that are easily forgotten are test data preparation and often even testing of negative paths and exploration sessions.

Acceptance Criteria

Making sure the user stories have adequate acceptance criteria defined.

Evaluation

Proof-reading user stories - checking if they are concrete and elaborated enough to go into development. Check them on corner cases and look for inconsistencies and interdependencies between stories.

Sprint

Daily scrums

Testing is or should be part of the definition of done of a sprint. Therefore testing is an activity taking place within the sprint and testing should be part of every daily scrum.

Manual testing

Manual testing, checking on acceptance criteria but also exploratory testing is a very important part of the sprint. Whatever piece of code that is test-ready should be tested.

Exploratory testing is not always.. or let's say almost never related to one user story. Exploratory sessions could focus on company history, comparable products, internal consistency, browser and device compatibility... and often is a trans-feature test activity. This means that exploration sessions might need individual tasks, next to the user story related checks that need to be part of the user stories definition of done.

Automated testing

Best is to automate tests that on stable code of the previous sprint and run a test set of automated regression tests on top of the unit and integration test framework

Test data preparation

During the sprint, testers or developers need to make sure the required test data is available. sometimes this is part of release management, some times it needs to be prepared in files, directly in the database or in the application, using the front end.

This needs to be a well-coordinated.

Test environment

As agile development equals constant build and and constant integration, the test environment needs to be ready for testing when it needs to be ready and needs to stay in a certain situation maybe for a small period of time before a new version of the product is released. This requires intensive communication. We need to know at all times which version is under test and under which version bugs are being reported.

User coaching

Another duty of testing is coaching of users in acceptance testing or even in the definition of requirements and acceptance criteria. Users need coaching. Often they don't understand the principles of scrum to much and a scrum master can not go into details involving and coaching all users. A tester has the competences and the adequate product knowledge to take this role.

Why testing is NOT a "standard process"

In the beginning there was consumption...

It all starts with the consumption society in which we, software testers, live.
An average consumer, willing to spend some money, asks for a certain custom product.
Being a consumer for quite a while, and having the experience of buying standard products, we know a price and an exact scope of  what we're going to get for our money.

When I for example go and buy a car, I know exactly what I'm going to pay, how my car will look like and when I'm going to receive it. The same goes by the way for a book, cd or even a software package.

So our problem, of testing being a process, begins exactly here. We are all very familiar with the trade of standard products. But there is a big difference between standard products and custom made products. Many clients and suppliers are not very competent in buying or selling such a non-standard, custom product.
And even though you think your car is customized when you have carefully chosen your options and colour - you just configured your car with the standard options and colour a supplier provides for this car. If you disagree, go to the garage and try to have an integrated GPS with bluetooth module, traffic board recognition and rear-driving camera, ESP and ABS system in a 1992 Opel Corsa and you'll see what I'm talking about.

When testing is required, we're talking mostly about new product development. We're going to build or integrate something that we never built or integrate before. So this is NOT a standard product that we can just take off a shelf.

What we seem to like to omit, is the fact that in custom software production, we're going to make something for the first time. We might integrate with components that aren't there yet, using technologies we haven't been using yet, working with people we have never seen before. There are a lot of unknown factors here.


So how do we need to buy or sell a custom product?

When we sell or buy something that does not yet exist, we don't want to find out that what we wanted is not really what we needed on the moment we spent all our money. We want to verify step by step and have the possibility to learn and steer and even more, get a good understanding about the real cost of the complete product we're asking for.

What we need to do is learn quickly, as we will be making false assumptions on cost or complexity, forgot about important things and maybe had some misunderstandings on all different levels. We have to keep our eyes open, and our focus sharp. Is this product that we're building fit for purpose? Are we on the right track? We better find out that we made a mistake before we spent all those millions.

So ideally we plan, build and act in small iterations. Each iteration gives us time to act and learn so that we can adopt in order to do better during the next iteration. Testing is a part of acting and is taking place in close cooperation with all stakeholders and is therefore a important driver of the learning and improvement of the team.

We buy and sell by building a solid trust relation between buyer and seller that gives room for continuity and results after a couple of iterations in a very clear view on budget, scope, planning and their  respective interferences.


Process VS Learning

Now what is a process?
A process is a series of predefined actions that produce something or that lead to a particular result.
Processes tend to always start from the same starting points, with known input and the output of a process is also known.

Processes are used to streamline factories and service companies that offer standard products or services. We don't want those people, building cars, delivering lettres or producing cd's to learn and improve. They have to follow the process. We want them to work effectively and at a minimal cost.

When building custom products, it is close to impossible to have a concrete view on any standard process to follow right from the start as there is no routine yet, that could be the base for a process to be put in place. On the contrary, there are many unknown factors in input, actions to be taken and even outputs.

This does not mean that no agreements can be made about communication and cooperation.
By learning and improving, processes and procedures might be put in place during software development. But those processes  and procedures will be very specific and tailored to the people and the context of this piece of software that is being built.


Conclusion

I would like to claim that testing can not be performed successfully by only following any standard process. Quality of any custom made software product can not be guaranteed by following any process. Good and effective testing requires expertise, knowledge, experience and skill.

During testing, it is the learning curve that can lead to creation of specific processes that can be applied within projects, programs or cooperating teams. But it is the learning curve and close stakeholder cooperation that leed to less issues in production, not any process.

If you read this and you agree with me, please go and sign here to support the testing community.