Why testers know best...

Because testers ask questions when they don't understand
Because testers think about what can go wrong and not how it should work
Because testers are not dedicated to go live in time
Because testers still want to go live in time more than finding bugs that don't matter
Because testers love finding bugs that matter
Because testers don't have much advantage in telling 'I told you so'
Because in the end, if a problem is not fixed in time, the tester suffers from that so they are driven to find issues and defects early
Because testers don't have 'babies' to protect and therefore are the least biased people in the project
Because testers have a critical technical eye for detail and understand what business wants in the same time

Forming Storming Norming Performing

We kicked off with a team of completely diverse people, starting a project that was going to be the biggest challenge most of us ever faced.

The first months of the project where more difficult than any previous other project I had been working on. Finding out our places in the project, how we would fit to the puzzle, was for everyone one of the biggest challenges. At first conflicts were avoided, and specific tasks were not performed. Later on, some of us got caught up in quarrels and fights and after months of hard work, as a team, we contributed to late and poor quality delivery.


After a couple of months of suffering, we learned on a team building event that this is so very normal. After that we found that by good leadership, team members are being carried trough different phases in a model of a team-building process.
The model I'm talking about is the model of Tuckman and considers the Forming, Storming, Norming and Performing phases in team building.

If you ever get a chance to look at your own team from a distance, you'll definately recognize this model and if you're manager of a leader of a newly created team, you definately need to know these things if you want to improve your team atmosphere and quality of delivery.

Forming

The team is assembled. Team members don't know each other well and tend to act independently. There is no real group-feeling.  Conflict is avoided. Team members have a need for guidance.

Storming

Team members start to take their positions in the team. This process often leads tentions between team members and contradictive ideas.

Norming

Team members are aware of the need of rules and methods. Those rules and methods are defined within the team. The common objectives become clear and the required roles are divided over the team members. The team with all team members take responsibility for the delivery of the objectives.  The risk during the Norming phase is that the team loses either their creative spirit or the drive that brought them to the Norming phase.

Performing

The team is working together harmoniously to achieve the common objectives. The team is working independently and makes collaborate decisions. Performing teams are identified by high levels of independence, motivation, knowledge and competence.



Unfortunately, many teams never make it past the Storming phase. However I dear to say that we made it at least to the Norming phase as a team, yet we still have a lot to deliver...

Bug Fix Bingo

How to dynamically turn frustration of testers into positive energy?


Bug Fix Bingo
Developers seem to return with interesting reasons why a bug is not 'really' a bug.
Testers on the other hand, need to understand that developers don't produce code but create art-work.

Now, tester, when you come back from a soft-skill-battle with the developer where you finally, without hurting their feelings, were able to make them aware of this little mistake they made, you can return and have a little prize as a reward for your empathy and communication skills.
In stead of getting frustrated and going back to your PC with a head-ache, you will be allowed to participate in the Bug Fix Bingo.

To the developers: Please don't take it too personal. This Bug Fix Bingo keeps your testers sane and in the end, those bugs really need fixing.
The game was created by K. J. Ross & Associate ( http://www.kjross.com.au/page/Resources/Testing_Games/)

Let's get rid of the safety net

 I couldn't possibly say it better than Gojko did... but I can write about it.

What Gojko came to tell us in the Eurostar conference of 2011 was that with software testing as we see it most - A phase of executing test cases after the product development phase has ended - does not contribute to quality.
It contributes to building a safety net for developers, managers, and even testers.


Since testing is done after development, developers get encouraged to become lazy as they don't need to make sure that their development still really works.

Testing should not be about

• Logging defects that developers actually already know about. When for example simple client side field validations don't work, developers could know before a tester knows.
• Waiting for a product to start testing and designing tests against a formalized analysis, to cover our asses instead of the product
• Answering business on their requests instead of providing what they really need
• Assume that all requirements and designs are clear and correctly defined

Testing should be about

• Helping analysts and developers by showing them how things can go wrong during design, development and delivery
• Providing what business needs instead of what business wants
• Help non-testers be better at testing
• Test the complex and critical parts of a product
• Find the real requirements and share them with all team-members
• Being part of a team that is jointly responsible for the quality of the delivered product

 

View more presentations from gojkoadzic

Beware of inattentional blindness while testing

What's "Inattentional blindness"? and what does it have to do with testing?
I got advised about inattentional blindness by a colleague tester, who got it from another colleague tester, who got it from another colleague...

This turns "Inattentional blindness" into a Test-Myth!

Before I say anything further about it, I would like you to watch the very short video below and carefully follow it's instructions.

(Don't read below the movie yet, as it will contain spoilers)

Have you been surprised? There are different explanations for the reason behind it. (http://en.wikipedia.org/wiki/Inattentional_blindness )

• Conspicuity - The ball draws your attention.
• Mental Workload - You need to count so you focus on counting only
• Expectation - You expect only passes to see and focus on the passes only and block other content
• Capacity - You see this movie for the first time and you're not used to these exercises. You're not used to do this.

Now, what is in it for a tester? The key message is:

When you focus, DON'T FORGET TO DE-FOCUS!


This is what you do when you execute pre-scripted test cases:

• The test needs your attention - if you pass it, you better be sure about it! - Conspicuity
• You need to focus on preparing the correct data and executing the correct steps - Mental Workload
• You expect only a defect against what you are testing - Expectation
• Especially during the first run, you're not yet used to go trough the procedural steps - Capacity

It's important to achieve a goal, to pay attention and to be focused on what you're doing.
It is equally important to look around at the things you did not prepare test scripts for and find the bugs.


How can we do this?

• If you feel the urge to prepare test scripts - don't detail them into descriptive test steps. Leave yourself some space to find alternative paths. Describe your paths on the moment of execution.
• Per functionality, give yourself a time-boxed moment of time to look around in the application and find out about what you forgot about.
• Put a post-it on your screen saying "de-focus" as you will forget from the moment you try to remember.

Things you can do while waiting for a testable product

A project manager gets most nervous on the moment a product (part) is about to be released to testing. Finally someone will tell them something about the quality of the product that will be delivered. In the time before the big moment arrives, they want you to carefully prepare, doing useful things like for example

• Prepare (preferably thousands of) detailed test scripts based on functional designs 
• Deliver a detailed (re)planning of all test activities over different phases and environments 

There are however, other useful things you can do while waiting for a product to be released.
Those might not be directly required by your PM, but you'll benefit from them later on the critical path.
This list is not exhaustive as exhaustive lists would be exhausting.

• Talk to business users and analysts. Ask them about what they think is important, ask them how business works. Create your high level business End to End test scenario's in a workshop. Compare those End to End scenario's with functional designs and requirements and deliver your first gap analysis results. Explain your test approach to business users and analysts. The end to end flows can give you a nice way later on to report back to business on your test progress in a way they will all understand. I'll come back to this later on in a specific blogpost. 
• Talk to developers. Ask them about the programming issues and re-used components. This can save you some time later on. There is a reason why development will deliver late... Agree with them about the defect template and the procedure... that you want to keep as pragmatic as possible. 
• Read trough the analysis documents and identify high level test cases. Make note of inconsistencies and communicate about them on a regular base. These are very likely places where bugs can be found if not taken into account. It's perfectly sane behavior for a tester to make some commotion if you're not listened to during this moment in time. 
• Challenge the Architectural picture with what you learned from the business users and developers and read in the documentation. Make again note of inconsistencies and communicate them. 
• Create a high level overview of all test activities that need to take place. Agree with all parties that will take part in testing and find inter-dependencies over the available test environments. Find out which test activities can be run in parallel. Find out when, most likely, data resets will be required. Verify when other test activities in the environments are planned. Get to know your testing neighbors. 
• Define your reporting structure and agree with your PM and business. Use some visible examples. Make sure everyone understands that metrics are not the only thing they want to receive. Try to be creative and present visual, simple reports and arrange the possibility to have meeting to explain your status. 
• Train your BA's and other non-testers who will be joining in testing at some moment in time. Your mission is to make them become better testers. Tell them why writing all possible variations for a field validation test is an endless activity. A tester will have endless amounts of test cases to test one field but might only execute some for which the product will likely manifest bugs. You could tell them something about Boundary value analysis, data combination, paths, about focussing and defocussing, the tea-test and the shoe-test. 
• Make sure you obtain access to the test environment(s) with the correct user roles and assess if all infrastructural components, required to start testing, are in place. 
• Test the test data backup and restore process. If you have the possibility, make a proof of concept. 
• Prepare your assessment on entry criteria if required and give your development team a heads up on where they are in function of your entry criteria. In this way they won't be surprised when you need to give them a no-go to deploy in the test environment. 
• Test the deployment and release procedures. 
• Implement required test tools. 
• Reassure the PM, they are nervous and need some support.

The Kick-off

Dear Reader,



Kick-starting myself into the world of possible readers, I find myself ready to write on what I think software testing is absolutely not about, definitely should be about and share how I experience my attempts to contribute to change. It’s time for the era of better software-testing.

As a kid I started testing on my first assignment with the TMap® and later the TMap NEXT® at my side. I wanted to learn all about the black box of software development and as a good boy, I absorbed, without a lot of questioning. Moreover, how could I think about questioning when being taught by Michiel Vroon and Rob Baarda themselves? I was genuinely impressed and learned mostly about the test design techniques and test phases.

It all made perfect sense to me: First there are requirements which get broken down into specifications. In a very good project they will even be broken down into a technical design. When this is all done, coding can start and then everything gets tested in different test levels, each of them having their test cases designed according to the documents available. When the testing phases are over, we go live... Doesn’t that sound like a V-model? (http://www.waterfall-model.com/v-model-waterfall-model/)

It’s like building a house, in fact. First you come up with a plan, the architect makes a blue print, the master builder makes a plan and the house is built in phases, starting with the foundations…

… but wait a minute… when is the house tested? After the last construction part is done? Actually while building a house, testing takes place within every development phase. For example, the foundation is verified when walls are being build. Then why do testers only test when it’s actually too late? And what if the living room didn’t really look like we hoped it to look like during construction? What if we, living in France, bought cheaper power plugs in UK, who will notice and when? … You know where I’m getting at, I suppose. Software development is not like building a house, nor it is like building a car, it’s mostly like software.

Things are often not as easy as they look like. Therefore the V-model, like any other model has disadvantages

• In the V-model requirements don’t change within the development life cycle – Unfortunately even if they don’t change, our language is still too primitive to formulate an unambiguous complete requirement that is understood in a perfectly similar way by all stakeholders and involved parties. Or in other words, let’s accept we’re human.
• Mistakes are only found when all development is already done – No requirements or any other spec is verified up until the moment that all development is done and a go for testing is given. That's too late.

Now, unless you work in the army or for the public sector, the currently most recognized documented best practices and methodologies for testing are difficult to put to actual use. That is mostly because in your project you don’t have a complete industrialized process and/or an immense budget available. The V-model doesn’t really offer the possibility to change your mind or make mistakes during the development lifecycle and requires high discipline on top of that.


I do have to recon on the other hand that it’s easy to plan a test phase 3 months in advance, hire 15 testers to write test cases. Most Project or Program Managers understand that those test cases always go straight to the recycle bin before they actually get executed. They do however like a pro-forma report with execution rates, preferably linked to the amount reopened defects of priority 1, to go to the developers and show them, having the proof in hands that their development is crap and that they are buying time.


Now I hope to all find you nodding heads, agreeing with what I’m saying, but guess what, lots of us are testing in this kind of structure that really does not make sense.
So why do lots of us work in projects that are run like this?
- I know: Because people want things that make sense, that are easy to explain.
Then why does this non-sense make sense?
- It doesn’t. What makes sense is that a V-model makes a complex world look easy and people want easy things
So what can we do make this testing (and project) world a better place?
- Find a way to test that is easy to explain and understand, does make sense and, most of all, really works.


In this blog, mostly, for now, I will describe my frustrations of testing in the European Hemisphere, my inventions and ideas to improve testing and relate with other professions in the project development landscape.
And most of all, I will try to contribute to the common sense in testing.