tag:blogger.com,1999:blog-23140196140947212902024-03-27T07:37:41.108+01:00Common Test SenseA contribution to common sense... while testingMike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.comBlogger26125tag:blogger.com,1999:blog-2314019614094721290.post-27134113998836465372015-02-21T17:22:00.000+01:002015-02-21T17:25:56.110+01:00Test activities in ScrumLately I've been absorbed and emerged into the REAL agile development life cycle and how to do do Agile properly. To do so, I got my scrum master certificate, started reading about scrum, worked as a scrum master and went to work in the company that hired Arie van Bennekum, one of the co-founders of the Agile Manifesto.<br />
<br />
On my path to learn about agile and scrum, I learned a lot about Unit testing and integration testing, Test Driven Design (TDD) Design and eXtreme Programming (XP), The definition of done and ready, continuous build and integration... I also earned about acceptance criteria and even Acceptance Test Driven Development. Those terms were not entirely new for me, but let's say I understand them better now since I saw them being put in practice by the teams.<br />
<br />
But what about exploratory testing, non functional testing, front-end automation, test data preparation, test environment management for example? And how do testers get their test-related jobs done when they require considerable extra effort? I found, next to the very structured applications of unit testing and TDD, manual testing and test automation aspects as well as test related tasks and estimations were not yet very mature.<br />
<br />
As I'm now only 2 years active in Agile development and more particularly in Scrum, I do not claim to own Agile development. But I do claim to know something about testing and where I would take proper testing tasks into consideration while working in for example Scrum. This is how I did it:<br />
<br />
<div class="separator" style="clear: both;">
I divided the test related tasks into 3 parts</div>
<div class="separator" style="clear: both;">
- Taks to be taken up during product backlog definition</div>
<div class="separator" style="clear: both;">
- Tasks to be taken up during sprint backlog definition</div>
<div class="separator" style="clear: both;">
- Tasks to be taken up during the sprint</div>
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlEpRzBujA_E79PH6y4x82HTxxTIVIx-g7nhH0toub_CmMH4NynshZqINIt3-BEeDiHlk0_HSQdQk9OlicGbLXDjIxlfYdvqEGc0i9ucrc_IRv5Pitni0K_olfEyeOozcGjwY14VsmLYr0/s1600/Screen+Shot+2015-02-21+at+15.22.13.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjlEpRzBujA_E79PH6y4x82HTxxTIVIx-g7nhH0toub_CmMH4NynshZqINIt3-BEeDiHlk0_HSQdQk9OlicGbLXDjIxlfYdvqEGc0i9ucrc_IRv5Pitni0K_olfEyeOozcGjwY14VsmLYr0/s1600/Screen+Shot+2015-02-21+at+15.22.13.png" height="377" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<h3 style="clear: both; text-align: left;">
<b>Product Backlog definition</b></h3>
<div>
<b><br /></b></div>
<div>
<b>Production issues</b></div>
<div class="separator" style="clear: both; text-align: left;">
It is important that production issues ( and maybe also issues coming out of acceptance testing if this is not taken into the definition of done of a sprint) get a proper description and impact and become part of the product backlog.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<b>Test related features and testability items</b></div>
<div class="separator" style="clear: both; text-align: left;">
Features needed to accommodate testing (stubs, drivers, datasets, monitoring plugins, maybe even a button and input field within the application to provide feedback for beta testers) need to be defined and developed in time.</div>
<div class="separator" style="clear: both; text-align: left;">
Also product supporting elements like test automation, performance tests, security tests, exploratory sessions are items that could be put on the product backlog.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<h3 style="clear: both; text-align: left;">
Sprint backlog definition</h3>
<div>
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<b>Test estimation</b></div>
<div class="separator" style="clear: both; text-align: left;">
During the sprint backlog definition, testing has to make sure that all testing efforts are estimated for every user story that goes into a sprint. Things that are easily forgotten are test data preparation and often even testing of negative paths and exploration sessions.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<b>Acceptance Criteria</b></div>
<div class="separator" style="clear: both; text-align: left;">
Making sure the user stories have adequate acceptance criteria defined.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<b>Evaluation</b></div>
<div class="separator" style="clear: both; text-align: left;">
Proof-reading user stories - checking if they are concrete and elaborated enough to go into development. Check them on corner cases and look for inconsistencies and interdependencies between stories.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<h3 style="clear: both; text-align: left;">
Sprint</h3>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<b>Daily scrums</b></div>
<div class="separator" style="clear: both; text-align: left;">
Testing is or should be part of the definition of done of a sprint. Therefore testing is an activity taking place within the sprint and testing should be part of every daily scrum.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<b>Manual testing</b></div>
<div class="separator" style="clear: both; text-align: left;">
Manual testing, checking on acceptance criteria but also exploratory testing is a very important part of the sprint. Whatever piece of code that is test-ready should be tested.</div>
<div class="separator" style="clear: both; text-align: left;">
Exploratory testing is not always.. or let's say almost never related to one user story. Exploratory sessions could focus on company history, comparable products, internal consistency, browser and device compatibility... and often is a trans-feature test activity. This means that exploration sessions might need individual tasks, next to the user story related checks that need to be part of the user stories definition of done.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<b>Automated testing</b></div>
<div class="separator" style="clear: both; text-align: left;">
Best is to automate tests that on stable code of the previous sprint and run a test set of automated regression tests on top of the unit and integration test framework</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<b>Test data preparation</b></div>
<div class="separator" style="clear: both; text-align: left;">
During the sprint, testers or developers need to make sure the required test data is available. sometimes this is part of release management, some times it needs to be prepared in files, directly in the database or in the application, using the front end.</div>
<div class="separator" style="clear: both; text-align: left;">
This needs to be a well-coordinated.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<b>Test environment</b></div>
<div class="separator" style="clear: both; text-align: left;">
As agile development equals constant build and and constant integration, the test environment needs to be ready for testing when it needs to be ready and needs to stay in a certain situation maybe for a small period of time before a new version of the product is released. This requires intensive communication. We need to know at all times which version is under test and under which version bugs are being reported.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<b>User coaching</b></div>
<div class="separator" style="clear: both; text-align: left;">
Another duty of testing is coaching of users in acceptance testing or even in the definition of requirements and acceptance criteria. Users need coaching. Often they don't understand the principles of scrum to much and a scrum master can not go into details involving and coaching all users. A tester has the competences and the adequate product knowledge to take this role.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<br />Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com49tag:blogger.com,1999:blog-2314019614094721290.post-15922319441136215392014-08-24T01:38:00.000+02:002014-10-31T11:57:32.208+01:00Why testing is NOT a "standard process" <b>In the beginning there was consumption...</b><br />
<b><br /></b>
It all starts with the consumption society in which we, software testers, live.<br />
An average consumer, willing to spend some money, asks for a certain custom product.<br />
Being a consumer for quite a while, and having the experience of buying standard products, we know a price and an exact scope of what we're going to get for our money.<br />
<br />
When I for example go and buy a car, I know exactly what I'm going to pay, how my car will look like and when I'm going to receive it. The same goes by the way for a book, cd or even a software package.<br />
<br />
So our problem, of testing being a process, begins exactly here. We are all very familiar with the trade of standard products. But there is a big difference between<b> standard products</b> and <b>custom made products</b>. Many clients and suppliers are not very competent in buying or selling such a non-standard, custom product.<br />
And even though you think your car is customized when you have carefully chosen your options and colour - you just configured your car with the standard options and colour a supplier provides for this car. If you disagree, go to the garage and try to have an integrated GPS with bluetooth module, traffic board recognition and rear-driving camera, ESP and ABS system in a 1992 Opel Corsa and you'll see what I'm talking about.<br />
<br />
When testing is required, we're talking mostly about new product development. We're going to build or integrate something that we never built or integrate before. So this is NOT a standard product that we can just take off a shelf.<br />
<br />
What we seem to like to omit, is the fact that in custom software production, we're going to make something for the first time. We might integrate with components that aren't there yet, using technologies we haven't been using yet, working with people we have never seen before. There are a lot of unknown factors here.<br />
<br />
<br />
<b>So how do we need to buy or sell a custom product?</b><br />
<b><br /></b>
When we sell or buy something that does not yet exist, we don't want to find out that what we wanted is not really what we needed on the moment we spent all our money. We want to verify step by step and have the possibility to learn and steer and even more, get a good understanding about the real cost of the complete product we're asking for.<br />
<b><br /></b>
What we need to do is learn quickly, as we will be making false assumptions on cost or complexity, forgot about important things and maybe had some misunderstandings on all different levels. We have to keep our eyes open, and our focus sharp. Is this product that we're building fit for purpose? Are we on the right track? We better find out that we made a mistake before we spent all those millions.<br />
<br />
So ideally we plan, build and act in small iterations. Each iteration gives us time to act and learn so that we can adopt in order to do better during the next iteration. Testing is a part of acting and is taking place in close cooperation with all stakeholders and is therefore a important driver of the learning and improvement of the team.<br />
<br />
We buy and sell by building a solid trust relation between buyer and seller that gives room for continuity and results after a couple of iterations in a very clear view on budget, scope, planning and their respective interferences.<br />
<br />
<br />
<b>Process VS Learning</b><br />
<b><br /></b>
Now what is a process?<br />
A process is a series of predefined actions that produce something or that lead to a particular result.<br />
Processes tend to always start from the same starting points, with known input and the output of a process is also known.<br />
<br />
Processes are used to streamline factories and service companies that offer standard products or services. We don't want those people, building cars, delivering lettres or producing cd's to learn and improve. They have to follow the process. We want them to work effectively and at a minimal cost.<br />
<br />
When building custom products, it is close to impossible to have a concrete view on any standard process to follow right from the start as there is no routine yet, that could be the base for a process to be put in place. On the contrary, there are many unknown factors in input, actions to be taken and even outputs.<br />
<br />
This does not mean that no agreements can be made about communication and cooperation.<br />
By learning and improving, processes and procedures might be put in place during software development. But those processes and procedures will be very specific and tailored to the people and the context of this piece of software that is being built.<br />
<br />
<br />
<b>Conclusion</b><br />
<b><br /></b>
I would like to claim that testing can not be performed successfully by only following any standard process. Quality of any custom made software product can not be guaranteed by following any process. Good and effective testing requires expertise, knowledge, experience and skill.<br />
<br />
During testing, it is the learning curve that can lead to creation of specific processes that can be applied within projects, programs or cooperating teams. But it is the learning curve and close stakeholder cooperation that leed to less issues in production, not any process.<br />
<br />
If you read this and you agree with me, please go and sign <a href="http://www.ipetitions.com/petition/stop29119/utm_medium=social&utm_source=twitter&utm_campaign=button" target="_blank">here</a> to support the testing community.<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com4tag:blogger.com,1999:blog-2314019614094721290.post-34691661541500671402014-08-10T13:30:00.002+02:002014-08-10T13:35:38.099+02:00The role of QA in the Agile organization<div style="color: #222222; font-family: arial; font-size: small;">
<b>What is quality not (anymore)?</b></div>
<div style="color: #222222; font-family: arial; font-size: small;">
<b><br /></b></div>
<div style="color: #222222; font-family: arial; font-size: small;">
Some companies (still) think of quality as a standard set of KPI's that should be met for all projects in the company and reported on.</div>
<div style="color: #222222; font-family: arial; font-size: small;">
Those KPI's consist in the worst case for example of amount of analysis documents finished in time VS finished late, amount of test cases prepared per designed function, quotes on how well templates are being followed, bugs per test case, reopened bugs, bug fix times etc...</div>
<div style="color: #222222; font-family: arial; font-size: small;">
One role in the company is ultimately responsible for implementing quality in systems.</div>
<div style="color: #222222; font-family: arial; font-size: small;">
This role is the QA Lead or QA Manager who represent the project "head of police". They take their best-practice sheet out of the magical best practice box and start implementing it in the company.</div>
<div style="color: #222222; font-family: arial; font-size: small;">
The result of this way of implementing quality in a company producing software is a product that meets KPI's, was more costly to build and the people building it getting frustrated from working in a framework they don't support.</div>
<div style="color: #222222; font-family: arial; font-size: small;">
People on the floor don't seem to see the added value of the QA Manager and their KPI's, but those KPI's are sent way up, so they need to shine... in the worst case at the cost of product quality.</div>
<div style="color: #222222; font-family: arial; font-size: small;">
Maybe this was an acceptable way to approach quality when building products using the waterfall and V-model, and as a step to take in order to learn how to do better.<br />
Now there are better ways to do this.</div>
<div style="color: #222222; font-family: arial; font-size: small;">
<br /></div>
<div style="color: #222222; font-family: arial; font-size: small;">
<br /></div>
<div style="color: #222222; font-family: arial; font-size: small;">
<br /></div>
<div style="color: #222222; font-family: arial; font-size: small;">
<b>What is quality and how to pursue it?</b></div>
<div style="color: #222222; font-family: arial; font-size: small;">
<br /></div>
<div style="color: #222222; font-family: arial; font-size: small;">
"Quality" is like any software requirement. The idea of what an application needs to and may not do, changes over time and not all quality aspects can or should be fulfilled in order to deliver a successful project.</div>
<div style="color: #222222; font-family: arial; font-size: small;">
Each and every project member needs to understand the goal of what they are building and why they are building it. That's the first step in delivering software quality - a clear common goal.</div>
<div style="color: #222222; font-family: arial; font-size: small;">
<br /></div>
<div style="color: #222222; font-family: arial; font-size: small;">
The team with all their individual members are the owners of quality, the QA lead is their coach. </div>
<div style="color: #222222; font-family: arial; font-size: small;">
The definition of what quality is, is formulated by it's stakeholders and can change over time.</div>
<div style="color: #222222; font-family: arial; font-size: small;">
What it means in reality is that good software quality (how stakeholders perceive quality) can only be reached when the common understanding of quality is carried by the team.</div>
<div style="color: #222222; font-family: arial; font-size: small;">
Stakeholders in this case are not only the business, but also the developers, testers, end users, release team, support teams etc...</div>
<div style="color: #222222; font-family: arial; font-size: small;">
<br /></div>
<div style="color: #222222; font-family: arial; font-size: small;">
The QA job in a project starts by making sure all stakeholders are identified in the first place and making sure they are aware they need to provide (continuous) input on what they need from a software product in order to reach the level of quality they want to achieve.</div>
<div style="color: #222222; font-family: arial; font-size: small;">
The next thing QA needs to do, is coach the product owner (or project sponsor - who owns the budget) in deciding which of all identified aspects are more or less important considering the budget and timelines. </div>
<div style="color: #222222; font-family: arial; font-size: small;">
<br /></div>
<div style="color: #222222; font-family: arial; font-size: small;">
Then QA talks with the team on how these quality aspects will be put in place. The team will decide, together with QA on the definition of done (What are the measurable aspects of the software product that need to be fulfilled before we can ever say any component of this software has been "released".)</div>
<div style="color: #222222; font-family: arial; font-size: small;">
<br /></div>
<div style="color: #222222; font-family: arial; font-size: small;">
By doing so, the team defines how they will reach quality standards to ensure development speed, release efficiency, product maintenance, product aesthetics, acceptance criteria etc...</div>
<div style="color: #222222; font-family: arial; font-size: small;">
Based on the team's "definition of done" that evolves over time, the individual project will gain more maturity in understanding what quality means for their project and will start reporting on measurable project related KPI's.</div>
<div style="color: #222222; font-family: arial; font-size: small;">
The QA Lead can assist in setting up and maintaining those reports, but needs to take care not to become the owner of those reports.</div>
<div style="color: #222222; font-family: arial; font-size: small;">
<br /></div>
<div style="color: #222222; font-family: arial; font-size: small;">
Projects are further monitored by testers (or test engineers, QA engineers or whatever you call the people who look for flaws in products and processes). They assess process and product, and contribute to the definition of done by testing the product, but also by checking with the team and stakeholders on the completeness and correctness of the definition of done - that can be adapted after every iteration.</div>
<div style="color: #222222; font-family: arial; font-size: small;">
<br /></div>
<div style="color: #222222; font-family: arial; font-size: small;">
At the end of every iteration, it is important to look back on what went well and what went wrong in this project.</div>
<div style="color: #222222; font-family: arial; font-size: small;">
Lessons learned and incentives are the most valuable aspects a project can receive. Making sure those sessions take place and result in action points, can also be a QA role.</div>
<div style="color: #222222; font-family: arial; font-size: small;">
<br /></div>
<div style="color: #222222; font-family: arial; font-size: small;">
Results of implementing quality are not statistics in the means of the traditional overall KPI's anymore, but are project specific KPI's and even more importantly - they result in more successful projects, more accurate cost estimations, faster release cycles, faster delivery cycles, faster defect fix-rates, closer business cooperation and even higher retention levels of employees.</div>
<div style="color: #222222; font-family: arial; font-size: small;">
<br /></div>
Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com1tag:blogger.com,1999:blog-2314019614094721290.post-55376083257911377282014-05-30T23:11:00.002+02:002014-07-14T09:51:10.120+02:00Moving towards acceptanceFor an average professional software tester, testing is a merely rational concept.<br />
The tester discusses the test scope, creates a test plan, defines an approach, defines the test levels, entry and exit criteria. Then they install the tools, define working instructions, test the analysis, the code, the procedures and maybe even te project members. They register bugs, shout about risks ad issues that get solved or not after which they retest, retest and retest after which they finally advise about moving to acceptance testing or go-live, and (maybe after a couple of times going trough this procedure) start the hand over to the standing organisation to start testing for another project. For a tester, most of the time, there is no life beyond the project. No fear for production issues and how they will impact maybe thousands of people after the final Go-Live is given.<br />
<br />
For an acceptant, testing is a totally different matter. the acceptance testers are generally a representative set of constructive people that KNOW the organisation, operate or lead operations and therefore have a good influence on their business colleagues. They will have the last say about going live or not. (even when they are not formally asked - they can create a lot of commotion if their buy-in is not obtained by the time of go-live)<br />
It's the business users that will face the issues coming from poor design, architecture, forgotten parts from analysis and also from development and testing shortcomings.<br />
<br />
They care less about those numbers in those professional and colourful defect reports. They don't look into most of your risk assessments and don't even care about your scope descriptions.<br />
For them, the product needs to work on an acceptable level before go live and there is no number or colour that will convince them otherwise.<br />
Therefore those acceptants are the managers biggest nightmare as they are out of his control but will decide about the added value of the project team's work.<br />
<br />
It's those people that will have to work with the product for the next weeks/months/years until a possible update of the product will be made available. And that can take a while when you're not working in an agile structure.<br />
<br />
So how can we create this buy-in that managers are so desperately looking for?<br />
<br />
It's not all that hard. It all comes down to basic things. The project needs to be able to listen. They should not abandon all those "extra feature requirements" that are constantly coming in as defects at the end of the project. Instead the project should distill those extra features as soon as possible in the project, so that they can be estimated and built in time.<br />
<br />
It is useless to build a program that fits incorrect or incomplete requirements. The goal is to get a good understanding of what we are building and why we are building it, making sure that we deliver value. To get to a good understanding, feedback loops are of the highest importance.<br />
Is this what you wanted and is it working as you expected? Thats what we need to find out as soon as possible.<br />
<br />
So how can we do that?<br />
<br />
<br />
<b>1. Agile product delivery</b><br />
Running short sprints of software delivery, focussing on the most important parts of the most important features, is one of the best ways I know to get early feedback from your stakeholders. They get involved in setting up priorities and backlog creation and can be involved in testing very early on. They need to be coached in testing though as they might expect more then they will receive in the first deliveries and are looking for the wrong defects.<br />
<br />
If agile product delivery is not an option for you, there are still other ways to get a good and in time understanding of what your business actually really wants.<br />
<br />
<b>2. Dry-run</b><br />
Testing for business is not only about the software, but about doing their jobs with this software.<br />
A dry-run can be seen as a role-playing game where all business that is involved in accepting the product makes a simulation of their day to day operational work with the new software package.<br />
This can already be done based on analysis material as for example wire frames or a prototype.<br />
Every part that is analysed and prototyped or wire-framed, can pass a dry-run session - which can eventually lead to proper documentation of the business processes and analysis and re-used in all later testing.<br />
<br />
<b>3. Prototyping</b><br />
Before building the actual product, it might be a good idea to build a complete prototype. Prototypes can help people understand what the development team understood they need to build.<br />
Prototypes can be run trough from different angles<br />
- Does this prototype support the business process?<br />
- Do we capture all possible errors that can occur?<br />
- Does the screen layout and setup make sense?<br />
- Will the software be able to support the amount of users we estimate to have?<br />
- etc...<br />
They can be evaluated individually by the stakeholders and testers and/or used in a dry-run session.<br />
<br />
<b>4. Demo's</b><br />
As soon as you have a feature ready that can be shown, the developers take the initiative to give a demo. A demo is a forum to receive constructive feedback. The invitees should not be the managers high up in the chain but a representative set of users.<br />
After the developer finished the demo, the stakeholders ask the developer to show other relevant scenario's of the same feature. The developer can find and solve defects and the project leader might receive a set of changes in features or new features.<br />
<br />
Probably there are more ways to accomplish a good relation with your stakeholders, but I found the previous four very fruitful. Engaging your business early on in the project, making them part of the project delivery, is key to successful project delivery.<br />
<br />
<br />
<br />
<br />Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com0tag:blogger.com,1999:blog-2314019614094721290.post-74899755283942371042013-11-24T16:22:00.000+01:002013-12-06T13:30:39.262+01:00Scripting, checking, testing - enemies or friends?<br />
"If you don't script test cases, your testing is not structured" or "Testing is checking if the software works as designed" are statements made by people who like to call themselves "Structured testers".<br />
I don't completely disagree with statements like those, but I also would call myself a structured tester, even though I suppose I see things in a slightly wider context...<br />
<br />
Scripting, checking and testing can live seemlessly together as we can seemlessly live together in multicultural societies.<br />
It is not because we don't do it too often... we effectively can't. A good tester knows when to check, test and/or script.<br />
<br />
<b>What do I check?</b><br />
Checking is what I do when I have an exact understanding of what a function or process should have as an outcome.<br />
This can be at a very low level for example:<br />
- a field should contain no more than 15 characters<br />
- an email should contain an @ sign, a "dot", in this order, and should contain some characters in between<br />
It can be outcomes of a calculation engine that contains the rules on when to advise to grant an insurance or not, based on a defined set of conditions.<br />
It can be a set of acceptance criteria or business rules.<br />
Checking is verification and is a part of software testing. Whether or not they have to be all executed by a professional software tester is another question.<br />
When there is a lot to check - the pairwise method can help, putting different parameters together to make checking more efficient.<br />
<br />
<b>What do I test?</b><br />
Testing is what I do when I don't have an exact understanding of how functions and processes should co-exist or work together.<br />
The software parts I tend to test are the complex combinations of possibilities an application offers.<br />
In this case, I don't only look at a field, a screen or a function (sometimes I do) but at functions or flows that contain complex steps, and possibilities. I try to emulate an end user, taking into account the business processes, the company history, the development method and team's knowledge and skills... this combined with my own knowledge and tools... and go searching for the steepest and largest pitfalls that developers or analysts or business may have overlooked.<br />
Knowing what a function under certain circumstances is supposed to do, I try to find out if it's consistent with itself under relevant conditions.<br />
Questions I may ask myself:<br />
- How will the system behave if the interface is not available? - This could have a big impact on business function and is a case that happened a lot with the previous software package... and a reason why they developed a new package that was supposed to handle these issues.<br />
- What will happen if I push the back button instead of going back using the described functionality. The previous application in use - made use of the standard browser buttons. Many users are used to those, so I will check them all the time.<br />
<br />
<br />
<b>What do I script?</b><br />
Scripting is what I do to make sure that I remember what to <b>check</b> before I start checking or to remember what I have tested. Scripts can possibly be used as evidence for bugs found during testing or as a working tool to follow up on all that needs verification.<br />
I script what I should not forget.<br />
I could script all those things that need checking - but most of the time that already has been done.<br />
I could script the test cases that resulted in issues or bugs.<br />
I could script the test cases that will help me find regression issues later on. Those I would prefer to automate.<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com0tag:blogger.com,1999:blog-2314019614094721290.post-86507159370143194122013-11-16T13:22:00.001+01:002013-11-16T16:28:14.781+01:00Agile reporting<div class="separator" style="clear: both; text-align: center;">
<br /></div>
<div class="" style="clear: both; text-align: left;">
<b>How do we report and what do we report on?</b></div>
<div class="separator" style="clear: both; text-align: left;">
We often try to fall back on "facts" in order to provide test reports.</div>
<div class="separator" style="clear: both; text-align: left;">
Those facts are mostly represented in bars or curves, giving a view on the "reality" of the product quality and test progress.</div>
<div class="separator" style="clear: both; text-align: left;">
I put reality in quotes because reality is different for every viewer.</div>
<div class="separator" style="clear: both; text-align: left;">
I also put facts in quotes because facts are just a single dimension of showing a single aspect of a reality, seen by a person.</div>
<div class="separator" style="clear: both; text-align: left;">
It becomes clear that "facts" are becoming less solid as they may seem when pronounced by experienced managers.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<b>Traffic in bars</b></div>
<div class="" style="clear: both; text-align: left;">
Imagine you need to get traffic information in order to find out whether it is worth taking off from your destination. You go to your favourite traffic website and suddenly you see something like this:</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjL1YjyUz3UJ2pLjGB5uGl17KU1uj2x3taRn72v9Ta3Pc39rESVLv7JYaXJZDOr6E8Wyla8KqIaynfqr9UstMVKKrujN1Lox59_pu8sl9MtTn19W1MtlHt1kF56y0ey-mtEUFgpXYWtIokz/s1600/traffic+in+bars.PNG" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em; text-align: left;"><img border="0" height="372" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjL1YjyUz3UJ2pLjGB5uGl17KU1uj2x3taRn72v9Ta3Pc39rESVLv7JYaXJZDOr6E8Wyla8KqIaynfqr9UstMVKKrujN1Lox59_pu8sl9MtTn19W1MtlHt1kF56y0ey-mtEUFgpXYWtIokz/s640/traffic+in+bars.PNG" width="640" /></a></div>
<div style="text-align: left;">
<br /></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
</div>
<div style="text-align: left;">
<br /></div>
<div style="margin-left: 1em; margin-right: 1em; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQLu47oyCKNak3XbWe6SajFqAwEUMBrrojMV7nbfDq4GI0GPsaGeMmKJ5LA0gBjbo8CQdtESNkycGzhKodi0lhGRWeezMiaH0byRl95GregYtuvUX5o7EXnqcyLvSRFc6UECOwLqCGCjUz/s1600/traffic.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQLu47oyCKNak3XbWe6SajFqAwEUMBrrojMV7nbfDq4GI0GPsaGeMmKJ5LA0gBjbo8CQdtESNkycGzhKodi0lhGRWeezMiaH0byRl95GregYtuvUX5o7EXnqcyLvSRFc6UECOwLqCGCjUz/s1600/traffic.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="183" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiQLu47oyCKNak3XbWe6SajFqAwEUMBrrojMV7nbfDq4GI0GPsaGeMmKJ5LA0gBjbo8CQdtESNkycGzhKodi0lhGRWeezMiaH0byRl95GregYtuvUX5o7EXnqcyLvSRFc6UECOwLqCGCjUz/s400/traffic.PNG" width="400" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Would you be able to find out, based on this information where traffic is situated and if you are able to get home in time?</div>
<div class="separator" style="clear: both; text-align: left;">
I would not.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Same goes for a test report.</div>
<div class="separator" style="clear: both; text-align: left;">
A test report can give you a lot of valid information. But sometimes it lacks that what is really important to understand like for example</div>
<div class="separator" style="clear: both; text-align: left;">
- the context and underlying risks of the bugs that are still open?</div>
<div class="separator" style="clear: both; text-align: left;">
- what has been tested, and what not and what are the underlying risks?</div>
<div class="separator" style="clear: both; text-align: left;">
- how testing was conducted and which issues were found during testing that prevent us from testing efficiently or from testing at all?</div>
<div class="separator" style="clear: both; text-align: left;">
- what can be improved in the testing activities?</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
I got inspired by James Bach's website (https://www.developsense.org) and by the team I'm working with to find an alternative for the current reporting in bars... and when I started looking at the traffic information, I realized how silly we actually are, only making use of graphs and bars as 'facts' to present 'reality".</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
Let's for example have a look an example in software development </div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<b>Building of a Web Shop</b></div>
<div class="separator" style="clear: both; text-align: left;">
This conceptual web shop has a product search, a shopping basket, a secure login, and some cross- and up-selling features, a back end with an admin tool, a customer support tool and it interfaces with a logistics application, a CRM application, a BI application and a payments application.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjl2BbRwENDi_x-Ti_HGSezfeAJOwsnBkRiiI9yhnCSnJz8ECcHAy17QdnAfmll-3SsxqFY-LYJ7dwUOiJ7l-cjnX9BUYdRN9pg2N9TdxktasTOCCTsBPFw_d0EPP_kMYpvLpkRsOLlUakD/s1600/the+webshop.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="448" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjl2BbRwENDi_x-Ti_HGSezfeAJOwsnBkRiiI9yhnCSnJz8ECcHAy17QdnAfmll-3SsxqFY-LYJ7dwUOiJ7l-cjnX9BUYdRN9pg2N9TdxktasTOCCTsBPFw_d0EPP_kMYpvLpkRsOLlUakD/s640/the+webshop.PNG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The web shop is build in bi-weekly iterations of development and test results need to be reported.</div>
<div class="separator" style="clear: both; text-align: left;">
The standard test report looks like this on the moment the go/no-go decision has to be made.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<b>A standard report</b></div>
<div class="separator" style="clear: both; text-align: left;">
Test cases were planned and prepared up front (460 of them).</div>
<div class="separator" style="clear: both; text-align: left;">
</div>
<ol>
<li>In <b>Phase1</b> of the project, the testers were motivated to find relevant bugs in the system and realized that they would not find them by only following the prepared test cases.They were not allowed to create extra test cases, so a lot of bugs were registered,without a trace to a test.</li>
<li>In <b>Phase 2</b> of the project, the project manager extrapolated the amount of test cases executed to the future and found that testing will delay his project if it is continued like this. The test manager was instructed to speed up testing. A lot of unstructured testing had been going on and this was not budgeted for. The testers anticipated, an started failing all test cases they could, to get back on track with test execution. A lot less defects were registered as the testers were not really testing anymore, but rather executing test cases. Project management was happy. Less defects got raised so development was able to get back on track and testing was on track - the delivery date was safe again.</li>
<li>In <b>Phase 3</b> End-to-end tests started and business got involved. By doing so, they started also looking around in the application and registered a lot of extra bugs again. Flagging off those E2E test cases went very slowly as business did not want to sign off on a test case, if they saw still other things not working well. It took Project Management up to 3 weeks to convince them to finalize the tests. In the meanwhile the testers got instructed to pass as many test cases as they could - because the release had to go trough and now everything was depending on testing</li>
</ol>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgrTtD37BaZvjoXfa7gcwgAe8sncOW7AkdNxn4qTvmCLtA_IkXqa5HmRSglZgTxJ_B9v2fy2347RphhV2U4LpWb0Z6yko4qedNBGG3YDVEMwaFID8BDyyygAJaIPQvDGh0GDjiKmxDLkN5v/s1600/report+trends.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="390" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgrTtD37BaZvjoXfa7gcwgAe8sncOW7AkdNxn4qTvmCLtA_IkXqa5HmRSglZgTxJ_B9v2fy2347RphhV2U4LpWb0Z6yko4qedNBGG3YDVEMwaFID8BDyyygAJaIPQvDGh0GDjiKmxDLkN5v/s640/report+trends.PNG" width="640" /></a>Finally 3 blocking bugs were not resolved. However, the overview of test progress and defect progress looked pretty well. We also can see that development reacts always fast on blocking and critical issues, so if we would find a blocking issue, we can be sure it will be fixed in a day. However, that turn-around time was measured when a developer stated the defect was resolved, and not when it was confirmed to be fixed indeed. A lot of critical and blocking defects went trough the mill for some iterations before they finally got resolved.</div>
<br />
<div class="" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEgrTtD37BaZvjoXfa7gcwgAe8sncOW7AkdNxn4qTvmCLtA_IkXqa5HmRSglZgTxJ_B9v2fy2347RphhV2U4LpWb0Z6yko4qedNBGG3YDVEMwaFID8BDyyygAJaIPQvDGh0GDjiKmxDLkN5v/s1600/report+trends.PNG" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em; text-align: left;"></a><a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhu8X66ulPrOilO0nwBjYMqmV14IfUx92WcyhefyQavV6EexwJVitvr606q42qSYYlbCUnjyjmgTRS_lOavdX5h_Ihaa_Fhoa2pYlFIcml-vliUTU4dlBXSsL9D_EEA_bcVGC_2lqPYTyNI/s1600/report+stats.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em; text-align: center;"><img border="0" height="400" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhu8X66ulPrOilO0nwBjYMqmV14IfUx92WcyhefyQavV6EexwJVitvr606q42qSYYlbCUnjyjmgTRS_lOavdX5h_Ihaa_Fhoa2pYlFIcml-vliUTU4dlBXSsL9D_EEA_bcVGC_2lqPYTyNI/s640/report+stats.PNG" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<b><br /></b></div>
<div class="separator" style="clear: both; text-align: left;">
Clearly something went really wrong here. This report is unclear and seems fabricated. But it is only when we look at it with an eagle's eye, that we're able to notice these oddities.</div>
<div class="separator" style="clear: both; text-align: left;">
Based on these results it is impossible to make a profound decision whether or not to release this software.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
The pitfall of what we call "Goodhart's law" was stumbled into: <b>"When a measure becomes a target, it ceases to be a good measure"</b></div>
<div class="separator" style="clear: both; text-align: left;">
<b><br /></b></div>
<div class="separator" style="clear: both; text-align: left;">
So what can be done to avoid these kind of "facts" to represent "reality"?</div>
<div class="separator" style="clear: both; text-align: left;">
The answer to this is:</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<b>A visual, agile report.</b></div>
<div class="separator" style="clear: both; text-align: left;">
So how would this report look like? Well, it would look very much like a traffic map would look like.</div>
<div class="separator" style="clear: both; text-align: left;">
We could, for example, use functional blocks in an architectural model. </div>
<div class="separator" style="clear: both; text-align: left;">
Coming back to the web shop, we see the different functions with an indication wheter they were tested or not, if there were bugs found and if there were, how severe they are and how many blocking and severe issues we may have found in each module (visualized in the testing progress circles showing first the amount of blocking issues and then the amount of criticals).</div>
<div class="separator" style="clear: both; text-align: left;">
We also clearly indicate which of the existing functions are taken out of scope or come in scope of the test project - so that everyone has a same understanding of the scope we are really talking about.</div>
<div class="separator" style="clear: both; text-align: left;">
<br /></div>
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdzwdJjyWAsip6chldbYpbrd6ODVGPGaYEmGmzOvo6TrJKwCSADoQisMvn2TJUeXH_7ABSVp2ERDu0OWMclXPHQYC23gEd3udQmZ6uPDUv_naC0VO5uN0k_a0RSJ2oKY6gASEStnOAl6JB/s1600/Overview.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="448" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjdzwdJjyWAsip6chldbYpbrd6ODVGPGaYEmGmzOvo6TrJKwCSADoQisMvn2TJUeXH_7ABSVp2ERDu0OWMclXPHQYC23gEd3udQmZ6uPDUv_naC0VO5uN0k_a0RSJ2oKY6gASEStnOAl6JB/s640/Overview.PNG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
We take a look one level deeper into the functions, present in each functional block with blocking issues.</div>
<div class="separator" style="clear: both; text-align: left;">
For example the shopping basket. In the shopping basket we see that deleting items from the shopping basket is not possible. This is an annoying one to go live with, but if this one is solved one day after go-live, we can still agree to go live. Let's have a look at the transactions</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3mZTXktHGoV6_xN3bNZ8RQCHSQiyCYRSYGsfNF06oiMKVUxo5nfvh36sqFls9OJSGhoraJjOBcvmlWG1i1hyphenhyphenxtrZZF70hxVQ1BVA-_65LsFRSMlqSCyYRfr9oycf71cBZikHW_M3El91O/s1600/Detail+1.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="394" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj3mZTXktHGoV6_xN3bNZ8RQCHSQiyCYRSYGsfNF06oiMKVUxo5nfvh36sqFls9OJSGhoraJjOBcvmlWG1i1hyphenhyphenxtrZZF70hxVQ1BVA-_65LsFRSMlqSCyYRfr9oycf71cBZikHW_M3El91O/s640/Detail+1.PNG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
In the transactions section, we see that we can not really pay with the money transfer option and there are still some relevant issues in the Visa payment area. We can go live with Pay-pal, have Visa money transfer working in a day and have those Visa and money transfer issues that are left, fixed in a weeks time. We're still going live...</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTUMpT8KNX-xawSPxjTegDBKaiUslFTFJ1Il2BHMO05zaJZIq_yQcsbft9enQw_YMurhKgL0Cumnh5NQ2Rc-uS2aAklqRQ7l9B18Y_hg8DFDKJgTvdCwhWCY3DT_x4qBwpogBDGlHn6geX/s1600/Detail+2.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="404" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjTUMpT8KNX-xawSPxjTegDBKaiUslFTFJ1Il2BHMO05zaJZIq_yQcsbft9enQw_YMurhKgL0Cumnh5NQ2Rc-uS2aAklqRQ7l9B18Y_hg8DFDKJgTvdCwhWCY3DT_x4qBwpogBDGlHn6geX/s640/Detail+2.PNG" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: left;">
In the interface with logistics, we see something more worrying. The stock order isn't working and actually none of the functions behind have been ever tested. In the standard report those tests were noted as failed. Now we see that we won't be able to go live yet with this software. This defect needs to be resolved and further testing is required.</div>
<br />
<div class="separator" style="clear: both; text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGuYHaOuZo97EmaUcF87YTofe1PBkYoATSyWxmyuDi5dwrWZESh2LG4J90b_qb9Vz21Rl5ygLGQgdYvj5k-M7um61vSLCea5C2syLxoeJGcJHa_GbDlpEqb0D8zeAB8mA-hg6tmsnDpRuQ/s1600/Detail+3.PNG" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="396" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEiGuYHaOuZo97EmaUcF87YTofe1PBkYoATSyWxmyuDi5dwrWZESh2LG4J90b_qb9Vz21Rl5ygLGQgdYvj5k-M7um61vSLCea5C2syLxoeJGcJHa_GbDlpEqb0D8zeAB8mA-hg6tmsnDpRuQ/s640/Detail+3.PNG" width="640" /></a></div>
In general we can conclude that this kind of reporting, can maybe not completely replace the classical reporting. But it can be a good addition. Reporting on test progress, based on prepared tests however, seems tempting as it may give a fake feeling of control. However this control is not real. So this is always a very bad idea.<br />
<br />
The report is made to reflect and cope with agile development, but can also be used for a classical phased delivery approach. Also different color indications can be used or different "maps" (eg: business processes or functional flows might also do the job )<br />
<br />
This dashboard view still has to be accompanied with the test approach and efficiency of the testing that was conducted. That's gonna be for another post.<br />
<br />
This post has been created with the help of some of my <a href="http://www.realdolmen.com/">RealDolmen</a> colleages (Bert Jagers, Sarah Teugels, Stijn Van Cutsem, Danny Bollaert and and the inspiration of my wife) Thanks!Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com3tag:blogger.com,1999:blog-2314019614094721290.post-86879620821522959062013-06-16T19:08:00.006+02:002013-06-16T19:08:39.367+02:00Creating an effective defect report<br />
It's easy to monitor on defects, but when you start reaching your 1000th defect and you keep track of all kinds of data, you might start loosing control on what the actual quality of your application might look like. You start to lose overview on where the issues lie and what actions need to be taken to control product quality within delivery time. You drown in the graphs you need to present and in stead of reading your report yourself and making the correct conclusions, you'll use your energy creating the shiny report that nobody reads.<br />
<br />
<br />
Many test coordinators and test managers let themselves drive by all the information available and recorded in the defect registration tool, presenting all kinds of exotic graphs but at the same time lose focus on the actual reason of the defect report -<b> Giving insight in product quality</b>.<br />
<br />
So how do we provide insight in product quality, making use of a defect report?<br />
<br />
<br />
1. First of all, before starting to report, you need to get data in the defect registration system.<br />
You only want to capture data that is useful to show product quality. All other data is waste.<br />
<br />
<ul>
<li>Divide your software into logical blocks to report defects on (Using functional and technical blocks) to report defects on eg: Intake; Contract; Interfaces System A; Interfaces System B,... Make sure you don't have more than 9 of those. If you have more, regroup.</li>
<li>Define defect priorities and/or severities : 'Urgent - High - Medium - Low' will do for starters</li>
<li>Defect statuses to report on do not equal your actual defect statuses - simplify and group them for a report. Use:</li>
<ul>
<li>Open (For the developer to pick up or to resolve)</li>
<li>Closed - Rejected (Closed - Agreed not to fix)</li>
<li>Resolved (Needs a retest)</li>
<li>Closed - Fixed ( Closed and fixed)</li>
<li>Escalated (No agreement found - needs to be taken up in a defect board)</li>
</ul>
<li>You want to know what the actual issue was of defect when it is being resolved. This can be data, test, design, front end code, back end code, business rule, migration... again make sure you don't have more than 9</li>
</ul>
<div>
<br /></div>
<div>
2. Present your data in a way that anyone can understand what you want to show.<br />
<br />
<ul>
<li>The titles of the graphs you present, should be meaningful and unambiguous</li>
<li>The report should be printable on maximum 2 pages and still be readable</li>
<li>Use the correct chart types to display what you want to show. Trend views are no luxury. </li>
<li>The colors you present need to be in line with their meanings</li>
<li>Always provide an interpretation. A report that only shows metrics is can be easily misunderstood.</li>
</ul>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
An example of how a defect report could look like, is shown below:</div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
<div>
<br /></div>
</div>
<div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrTcGU8tIHzZzdyB6AEN7wHfG6yYEkKaDTdmDZMtpeTxyaivZYb945aj-_GtmeMlK9Z4mJKHwfC4TmMT16_B6OpddQCViyJ53gbyfvTW6TZPZtQoN8ijg6NtPkysBA43BeUcTI769R2RYI/s1600/Picture+5.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="343" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhrTcGU8tIHzZzdyB6AEN7wHfG6yYEkKaDTdmDZMtpeTxyaivZYb945aj-_GtmeMlK9Z4mJKHwfC4TmMT16_B6OpddQCViyJ53gbyfvTW6TZPZtQoN8ijg6NtPkysBA43BeUcTI769R2RYI/s640/Picture+5.png" width="640" /></a></div>
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLLHmL4xH73M0i4cQbthJ2h6ABmd8YyxXF9aaF6cSVz1upLwN6IsHBDZsG0Q6nCvWdssCllxyFSRQbEhtcd5rSC-Ba4kW4RD7hhbFvWNbtLnYsMxA3pBLgnWTU4mFTOWUS6SQiZmFT9IOM/s1600/Picture+6.png" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="366" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEhLLHmL4xH73M0i4cQbthJ2h6ABmd8YyxXF9aaF6cSVz1upLwN6IsHBDZsG0Q6nCvWdssCllxyFSRQbEhtcd5rSC-Ba4kW4RD7hhbFvWNbtLnYsMxA3pBLgnWTU4mFTOWUS6SQiZmFT9IOM/s640/Picture+6.png" width="640" /></a></div>
<br /></div>
Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com3tag:blogger.com,1999:blog-2314019614094721290.post-53148871367487707922013-01-09T22:50:00.000+01:002013-01-09T22:50:24.287+01:00How to organise a performance test without having the experienceWhat do you do when you need to organize a load and/ or stress test of a web application but you've never done it before? The purpose of this article is to provide a light checklist of things to keep in mind when organizing a performance test.<br />
<br />
<br />
<b>1. Define some high level performance indicators</b><br />
Check the competition - how long do they take to load a page in average?Which pages load fast and which load slower - and when is a slower load acceptable?<br />
Who needs to be involved? Client, Technical Architect, Sponsor, PM<br />
Example:<br />
On average, a screen needs to load in 2 seconds<br />
Overview pages need to load in 5 seconds<br />
Detail pages need to load within a second<br />
<b><br /></b>
<b>2. Obtain the business critical scenario's</b><br />
Common sense will get you far. Think of<br />
- How many concurrent users/transactions do you expect to be used by the functions?<br />
- Which functions are heavy or complex? (think of calculation engines)<br />
- Think about peak times and regular time simulations. (You might not want to pay for infrastructure that you will only use once a year)<br />
- Think about back ground concurrent (batch) processes that need to be simulated<br />
Who needs to be involved? Business, Technical architect, Oracle or Expert (someone the clients business and or technical processes)<br />
Example: Definition of the risk class of a prospect based upon a set of defined parameters<br />
Rule of thumb: Try not to have more than 10 scenario's<br />
<br />
<b></b><br />
<b>3. Create the detailed test scenario's</b><br />
During functional testing, make screenshots and exact steps on how to reproduce each scenario<br />
Interesting additions are<br />
- Think times: How long does it take to fill out a page before you can proceed<br />
- Expected response times per screen or per request<br />
- Define load profiles: Out of 100% total users, how many will use which scenario - in comparison to expected business behavior.<br />
<br />
<b style="background-color: white;">4. Define the environmental needs</b><br />
An environment needs to be as close to the production environment as possible to obtain comparable results. If you have the opportunity to use the Prod environment only once (before go live) then do the exact same tests on the non-sized test environment. You might be able to find a relation that will allow you to predict the future.<br />
Let your performance team assess the environment<br />
<br />
<b></b><br />
<b>5. Find a specialized company to run the load tests or select/setup your tool</b><br />
When you don't have experience in performance testing, this is the part where you find yourself a skilled person or company.<br />
Any company providing a decent performance test will require at least the previous 4 as an input and might even assist you to obtain results.<br />
Other things to take into consideration when selecting a third party or tool<br />
- Measuring points: where do you want to measure (web server, application server, load balancer, database layer,...) Note that every measuring point has impact on performance. A measuring point needs to be available in order to isolate a possible bottle neck.<br />
- Required automation work-arounds. The use of personalized security tokens might for example make life slightly more difficult...<br />
<br />
<b>6. Run the performance tests </b><br />
- Performance intake<br />
With the first functional code available, early analysis can be done at code and architecture/ design level to identify possible performance issues or risks. Performance defects found in this stage might still get prevented in stead of fixed and will therefore be less costly to implement.<br />
<br />
<span style="background-color: white;">Stress tests</span><br />
=>When and where does the system break? Identify the bottle necks and implement fixes where required.<br />
<br />
<span style="background-color: white;">Load tests</span><br />
=>To verify that you can run the different business processes, with an expected load of people using the expected on a production like environment<br />
<br />
<b>7. Result interpretation and presentation</b><br />
- Isolate issues and investigate any issue before reporting it as a performance issue (many issues reside in the test, measuring points,...)<br />
- Provide recommendations based on your results<br />
<br />
<br />Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com0tag:blogger.com,1999:blog-2314019614094721290.post-34630078943195143112012-11-30T08:44:00.001+01:002012-11-30T08:44:46.984+01:00The intake testBefore we want to start testing a product, it needs to be testable for the level of testing we want to perform. To ensure the subject under test is testable, we could require a whole set of measurable requirements as for example:<br />
<ul>
<li>The product (part) is delivered with all it's functional dependencies</li>
<li>All interfacing products are in place, issues are known</li>
<li>Amount of defects of certain severities closed in previous test levels VS total are met</li>
<li>Amount of elements planned to test in previous test levels VS tested are met</li>
<li>Test environment is setup and tested</li>
<li>Test data is prepared and consistently scrambled over the whole integrated platform</li>
<li>Batches are running</li>
<li>Configuration is setup and tested</li>
<li>Test preparation is done</li>
<li>Testers are trained</li>
<li>Licenses for tools are handed out and confirmed</li>
<li>... </li>
</ul>
When facing reality, these entry criteria are formulated at the very start of the project, but often barely maintained and applied as the delivery pressure becomes higher.<br />
<br />
But how often do we end up in situations where at least one of those entry criteria are not met, which blocks us from proceeding with testing? And how many times did we assess entry critria as passed but in fact they were not? A defect could be wrongly allocated, users have access to the tools, but have the incorrect user rights, the environment is build but it can't be accessed... Releasing untestable chunks of code to testers or releasing in an environment that is not fit for testing, is despite many project managers opinion a waste of time and resources.<br />
Environments need to be maintained longer, defects are discussed with more stakeholders and have longer turn-around times.<br />
<br />
<b>In short: Bad entry criteria management costs the project time and money.</b><br />
<br />
A pragmatic and easy way to keep a decent check on the status of your entry criteria, is to define an intake test for every substantial module under test. <br />
This intake test should be executable in a very limited amount of time and describe the positive flow(s) trough your modules under test. Based on the result of this intake test, delivery issues can be found instantly. And which statistical report can compare with "showing that it works". From the moment of ordering the environment, the teams focus should be on making this test work and all reasons why it doesn't will return into critical issues or defects that require high priority to solve.<br />
<br />
When the intake test for a module is successful, it can be recorded and stored in a library as a proof and testing for this module can start.Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com0tag:blogger.com,1999:blog-2314019614094721290.post-28227239925144019352012-11-20T21:18:00.001+01:002012-11-20T21:18:20.577+01:00An example of a simple defect life cycleThis blog post is providing a practical example of what I think is a minimal defect flow, accompanied with the roles and responsibilities<br />
<br />
In general the flow has 3 end statuses.<br />
<ul>
<li><b>Rejected - Closed:</b> This will tell something about the quality of defects that are raised. Rejects occur most often when registering duplicates or when the desired functionality is not well understood. </li>
<li><b>Tolerated - Closed:</b> Defects that end in this status are defects that are known. When going to production, these are the known issues that the product will be shipped with. We need to keep them separated from the rejected and fixed defects.</li>
<li><b>Fixed - Closed:</b> Defects that end in this status are unambiguously fixed defects that have been found resolved.</li>
</ul>
Some roles have been defined, regarding the defect flow<br />
<ul>
<li><b>Issuer</b>: Anyone who registers a defect. This is not necesarily a tester.</li>
<li><b>Tester</b>: Someone who is trained in testing, knows the functionality of the product to be delivered and has a brief technical insight into development, infrastructure and architecture. The tester can challenge the not accepted defects and is able to retest defects in their context.</li>
<li><b>Defect manager</b>: Someone who has development insight and decision power to accept, not accept and assign defects. This can role often maps on a development management role. </li>
<li><b>Developer</b>: A wide role for anyone who can fix defects in the code, infrastructure, parametrization or even in the design documentation. A defect does not necessarily reside only in code. A defect can also be an inconsistency in the design documentation.</li>
</ul>
<br />
<br />
On top of this it is advised to agree on SLA's on defect resolution times and agree on defect severity levels and/or priority levels. They can be combined and interrelated.<br />
I mostly use the following basic levels:<br />
<br />
<ul>
<li><b>Urgent - Prio 1:</b> Defect needs a patch - testers or users are blocked and no workaround can be put in place - Patch required within 2 working days</li>
<li><b>High - Prio 2: </b>Defect might need a patch but a workaround can be put in place or a less substantial amount of functionality is blocked. - Update required in a week</li>
<li><b>Medium - Prio 3:</b> Defect needs to be resolved, but there is a workaround possible. - All defects need to be resolved before go-live</li>
<li><b>Low - Prio 4:</b> Cosmetic defects. - 70 percent of defects need to be resolved before go-live</li>
</ul>
Sometimes cosmetic defects increase in ranking, when for example they occur on documents that are sent out to customers.<br />
<br />
The status flow and its description below depicts the minimum requirements for a defect flow to be implemented.<br />
<br />
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7eXfRT9zTC3wCwpzx2Rb_gVytwqQM2TSqtXjka1y3cuxH8Cqx5zLQGvhcgwXosuWdbCZr9_qYbIuFWOdUT9YuTydXysZ3gd0gZafSt6_p1getIaH_3p0TWJbRAMYFgXzgG_o7hE97XIrH/s1600/Basic+defect+flow.jpeg" style="margin-left: 1em; margin-right: 1em;"><img border="0" height="640" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7eXfRT9zTC3wCwpzx2Rb_gVytwqQM2TSqtXjka1y3cuxH8Cqx5zLQGvhcgwXosuWdbCZr9_qYbIuFWOdUT9YuTydXysZ3gd0gZafSt6_p1getIaH_3p0TWJbRAMYFgXzgG_o7hE97XIrH/s640/Basic+defect+flow.jpeg" width="452" /></a> <br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7eXfRT9zTC3wCwpzx2Rb_gVytwqQM2TSqtXjka1y3cuxH8Cqx5zLQGvhcgwXosuWdbCZr9_qYbIuFWOdUT9YuTydXysZ3gd0gZafSt6_p1getIaH_3p0TWJbRAMYFgXzgG_o7hE97XIrH/s1600/Basic+defect+flow.jpeg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;">
<style>
<!--table
{mso-displayed-decimal-separator:"\,";
mso-displayed-thousand-separator:"\.";}
@page
{margin:1.0in .75in 1.0in .75in;
mso-header-margin:.5in;
mso-footer-margin:.5in;}
td
{padding-top:1px;
padding-right:1px;
padding-left:1px;
mso-ignore:padding;
color:black;
font-size:12.0pt;
font-weight:400;
font-style:normal;
text-decoration:none;
font-family:Calibri, sans-serif;
mso-font-charset:0;
mso-number-format:General;
text-align:general;
vertical-align:bottom;
border:none;
mso-background-source:auto;
mso-pattern:auto;
mso-protection:locked visible;
white-space:nowrap;
mso-rotate:0;}
.xl63
{font-size:9.0pt;
font-weight:700;
font-family:Calibri;
mso-generic-font-family:auto;
mso-font-charset:0;
border-top:none;
border-right:.5pt solid windowtext;
border-bottom:.5pt solid windowtext;
border-left:none;
background:#D9D9D9;
mso-pattern:black none;}
.xl64
{font-size:9.0pt;
font-weight:700;
font-family:Calibri;
mso-generic-font-family:auto;
mso-font-charset:0;
border-top:none;
border-right:.5pt solid windowtext;
border-bottom:.5pt solid windowtext;
border-left:.5pt solid windowtext;
background:#D9D9D9;
mso-pattern:black none;}
.xl65
{font-size:9.0pt;
font-weight:700;
font-family:Calibri;
mso-generic-font-family:auto;
mso-font-charset:0;
border-top:none;
border-right:none;
border-bottom:.5pt solid windowtext;
border-left:.5pt solid windowtext;
background:#D9D9D9;
mso-pattern:black none;}
.xl66
{font-size:9.0pt;
font-family:Calibri;
mso-generic-font-family:auto;
mso-font-charset:0;
text-align:center;
vertical-align:middle;
border-top:.5pt solid windowtext;
border-right:.5pt solid windowtext;
border-bottom:.5pt solid windowtext;
border-left:none;
background:yellow;
mso-pattern:black none;}
.xl67
{font-size:9.0pt;
font-family:Calibri;
mso-generic-font-family:auto;
mso-font-charset:0;
text-align:center;
vertical-align:middle;
border:.5pt solid windowtext;
background:yellow;
mso-pattern:black none;}
.xl68
{font-size:9.0pt;
font-family:Calibri;
mso-generic-font-family:auto;
mso-font-charset:0;
text-align:center;
vertical-align:middle;
border:.5pt solid windowtext;
background:yellow;
mso-pattern:black none;
white-space:normal;}
.xl69
{font-size:9.0pt;
font-family:Calibri;
mso-generic-font-family:auto;
mso-font-charset:0;
text-align:center;
vertical-align:middle;
border-top:.5pt solid windowtext;
border-right:none;
border-bottom:.5pt solid windowtext;
border-left:.5pt solid windowtext;
background:yellow;
mso-pattern:black none;}
.xl70
{font-size:9.0pt;
font-family:Calibri;
mso-generic-font-family:auto;
mso-font-charset:0;
text-align:center;
vertical-align:middle;
border-top:.5pt solid windowtext;
border-right:.5pt solid windowtext;
border-bottom:.5pt solid windowtext;
border-left:none;
background:green;
mso-pattern:black none;}
.xl71
{font-size:9.0pt;
font-family:Calibri;
mso-generic-font-family:auto;
mso-font-charset:0;
text-align:center;
vertical-align:middle;
border:.5pt solid windowtext;
background:green;
mso-pattern:black none;}
.xl72
{font-size:9.0pt;
font-family:Calibri;
mso-generic-font-family:auto;
mso-font-charset:0;
text-align:center;
vertical-align:middle;
border:.5pt solid windowtext;
background:green;
mso-pattern:black none;
white-space:normal;}
.xl73
{font-size:9.0pt;
font-family:Calibri;
mso-generic-font-family:auto;
mso-font-charset:0;
text-align:center;
vertical-align:middle;
border-top:.5pt solid windowtext;
border-right:none;
border-bottom:.5pt solid windowtext;
border-left:.5pt solid windowtext;
background:green;
mso-pattern:black none;}
.xl74
{font-size:9.0pt;
font-family:Calibri;
mso-generic-font-family:auto;
mso-font-charset:0;
text-align:center;
vertical-align:middle;
border-top:.5pt solid windowtext;
border-right:.5pt solid windowtext;
border-bottom:.5pt solid windowtext;
border-left:none;
background:#76933C;
mso-pattern:black none;}
.xl75
{font-size:9.0pt;
font-family:Calibri;
mso-generic-font-family:auto;
mso-font-charset:0;
text-align:center;
vertical-align:middle;
border:.5pt solid windowtext;
background:#76933C;
mso-pattern:black none;}
.xl76
{font-size:9.0pt;
font-family:Calibri;
mso-generic-font-family:auto;
mso-font-charset:0;
text-align:center;
vertical-align:middle;
border:.5pt solid windowtext;
background:#76933C;
mso-pattern:black none;
white-space:normal;}
.xl77
{font-size:9.0pt;
font-family:Calibri;
mso-generic-font-family:auto;
mso-font-charset:0;
text-align:center;
vertical-align:middle;
border-top:.5pt solid windowtext;
border-right:none;
border-bottom:.5pt solid windowtext;
border-left:.5pt solid windowtext;
background:#76933C;
mso-pattern:black none;}
.xl78
{font-size:9.0pt;
font-family:Calibri;
mso-generic-font-family:auto;
mso-font-charset:0;
text-align:center;
vertical-align:middle;
border-top:.5pt solid windowtext;
border-right:.5pt solid windowtext;
border-bottom:none;
border-left:none;
background:yellow;
mso-pattern:black none;}
.xl79
{font-size:9.0pt;
font-family:Calibri;
mso-generic-font-family:auto;
mso-font-charset:0;
text-align:center;
vertical-align:middle;
border-top:.5pt solid windowtext;
border-right:.5pt solid windowtext;
border-bottom:none;
border-left:.5pt solid windowtext;
background:yellow;
mso-pattern:black none;}
.xl80
{font-size:9.0pt;
font-family:Calibri;
mso-generic-font-family:auto;
mso-font-charset:0;
text-align:center;
vertical-align:middle;
border-top:.5pt solid windowtext;
border-right:.5pt solid windowtext;
border-bottom:none;
border-left:.5pt solid windowtext;
background:yellow;
mso-pattern:black none;
white-space:normal;}
.xl81
{font-size:9.0pt;
font-family:Calibri;
mso-generic-font-family:auto;
mso-font-charset:0;
text-align:center;
vertical-align:middle;
border-top:.5pt solid windowtext;
border-right:none;
border-bottom:none;
border-left:.5pt solid windowtext;
background:yellow;
mso-pattern:black none;}
-->
</style>
</a></div>
<table border="0" cellpadding="0" cellspacing="0" style="border-collapse: collapse; width: 484px;">
<colgroup><col style="mso-width-alt: 2730; mso-width-source: userset; width: 64pt;" width="64"></col>
<col style="mso-width-alt: 2986; mso-width-source: userset; width: 70pt;" width="70"></col>
<col style="mso-width-alt: 11520; mso-width-source: userset; width: 270pt;" width="270"></col>
<col style="mso-width-alt: 3413; mso-width-source: userset; width: 80pt;" width="80"></col>
</colgroup><tbody>
<tr height="46" style="height: 46.0pt; mso-height-source: userset;">
<td class="xl63" height="46" style="background: #D9D9D9; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 700; height: 46.0pt; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none; width: 64pt;" width="64">From status</td>
<td class="xl64" style="background: #D9D9D9; border-left: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 700; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none; width: 70pt;" width="70">Responsible</td>
<td class="xl64" style="background: #D9D9D9; border-left: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 700; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none; width: 270pt;" width="270">Description</td>
<td class="xl65" style="background: #D9D9D9; border-left: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 700; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none; width: 80pt;" width="80">To status</td>
</tr>
<tr height="47" style="height: 47.0pt; mso-height-source: userset;">
<td class="xl66" height="47" style="background: yellow; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; height: 47.0pt; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">-</td>
<td class="xl67" style="border-left: none; border-top: none;">Anyone / Issuer</td>
<td class="xl68" style="border-left: medium none; border-top: medium none; text-align: left; width: 270pt;" width="270">Anyone
who finds a defect, can register it in the system. A defect has an unambiguous title, a clear description of what is expected and what is
observed and is evidenced.</td>
<td class="xl69" style="background: yellow; border-left: none; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">Open</td>
</tr>
<tr height="63" style="height: 63.0pt; mso-height-source: userset;">
<td class="xl70" height="63" style="background: green; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; height: 63.0pt; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">Open</td>
<td class="xl71" style="border-left: none; border-top: none;">Defect manager</td>
<td class="xl72" style="border-left: medium none; border-top: medium none; text-align: left; width: 270pt;" width="270">Assesses
the opened defect and decides that the defect is valid. The defect is not yet
in the pipeline of being resolved but it has been agreed with a developer to
start working on, taking priorities and timelines into consideration</td>
<td class="xl73" style="background: green; border-left: none; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">Accepted</td>
</tr>
<tr height="42" style="height: 42.0pt; mso-height-source: userset;">
<td class="xl70" height="42" style="background: green; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; height: 42.0pt; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">Open</td>
<td class="xl71" style="border-left: none; border-top: none;">Defect manager</td>
<td class="xl72" style="border-left: medium none; border-top: medium none; text-align: left; width: 270pt;" width="270">Assesses
the opened defect and decides that the defect is not valid. The defect is
updated with the rationale behind the disagreement.</td>
<td class="xl73" style="background: green; border-left: none; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">Not accepted</td>
</tr>
<tr height="64" style="height: 64.0pt; mso-height-source: userset;">
<td class="xl66" height="64" style="background: yellow; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; height: 64.0pt; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">Not accepted</td>
<td class="xl67" style="border-left: none; border-top: none;">Tester</td>
<td class="xl68" style="border-left: medium none; border-top: medium none; text-align: left; width: 270pt;" width="270">The
Tester assesses the non accepted defects and where they do not agree, they
discuss with the defect manager and project manager. TOGETHER, they agree
that the defect is valid and move the defect to the accepted status and
directly assign it to a developer.</td>
<td class="xl69" style="background: yellow; border-left: none; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">Accepted</td>
</tr>
<tr height="37" style="height: 37.0pt; mso-height-source: userset;">
<td class="xl66" height="37" style="background: yellow; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; height: 37.0pt; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">Not accepted</td>
<td class="xl67" style="border-left: none; border-top: none;">Tester</td>
<td class="xl68" style="border-left: medium none; border-top: medium none; text-align: left; width: 270pt;" width="270">The
tester assesses the<span style="mso-spacerun: yes;"> </span>non accepted
defects and where they agree, they consult with the issuer of the defect
that it can be rejected.</td>
<td class="xl69" style="background: yellow; border-left: none; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">Closed Rejected</td>
</tr>
<tr height="40" style="height: 40.0pt; mso-height-source: userset;">
<td class="xl74" height="40" style="background: #76933C; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; height: 40.0pt; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">Accepted</td>
<td class="xl75" style="border-left: none; border-top: none;">Developer</td>
<td class="xl76" style="border-left: medium none; border-top: medium none; text-align: left; width: 270pt;" width="270">The
developer indicates that they start working on the defect assigned to them.</td>
<td class="xl77" style="background: #76933C; border-left: none; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">In progress</td>
</tr>
<tr height="46" style="height: 46.0pt; mso-height-source: userset;">
<td class="xl74" height="46" style="background: #76933C; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; height: 46.0pt; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">In progress</td>
<td class="xl75" style="border-left: none; border-top: none;">Developer</td>
<td class="xl76" style="border-left: medium none; border-top: medium none; text-align: left; width: 270pt;" width="270">The
developer agrees with the project manager that other defects get priority
over this particular defect under fix. The defect fix is put on hold.</td>
<td class="xl77" style="background: #76933C; border-left: none; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">On hold</td>
</tr>
<tr height="37" style="height: 37.0pt; mso-height-source: userset;">
<td class="xl74" height="37" style="background: #76933C; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; height: 37.0pt; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">In progress</td>
<td class="xl75" style="border-left: none; border-top: none;">Developer</td>
<td class="xl76" style="border-left: medium none; border-top: medium none; text-align: left; width: 270pt;" width="270">The
developer indicates that the fix has been made and updates the defect with
information on the fix that has been applied</td>
<td class="xl77" style="background: #76933C; border-left: none; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">Fixed</td>
</tr>
<tr height="65" style="height: 65.0pt; mso-height-source: userset;">
<td class="xl70" height="65" style="background: green; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; height: 65.0pt; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">On hold</td>
<td class="xl71" style="border-left: none; border-top: none;">Defect manager</td>
<td class="xl72" style="border-left: medium none; border-top: medium none; text-align: left; width: 270pt;" width="270">The
defect manager or development manager propose defects that can be tolerated
in the release of the software to production. The defects that are accepted
by all stakeholders, move to the status closed-tolerated so that they can be
addressed in a later release.</td>
<td class="xl73" style="background: green; border-left: none; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">Closed Tolerated</td>
</tr>
<tr height="52" style="height: 52.0pt; mso-height-source: userset;">
<td class="xl70" height="52" style="background: green; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; height: 52.0pt; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">Fixed</td>
<td class="xl71" style="border-left: none; border-top: none;">Defect manager</td>
<td class="xl72" style="border-left: medium none; border-top: medium none; text-align: left; width: 270pt;" width="270">When
a build or fix is released in a test environment, the defects that were
fixed, will be re-testable. The defect manager updates those defects to the
retest status.</td>
<td class="xl73" style="background: green; border-left: none; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">Retest</td>
</tr>
<tr height="67" style="height: 67.0pt; mso-height-source: userset;">
<td class="xl66" height="67" style="background: yellow; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; height: 67.0pt; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">Retest</td>
<td class="xl67" style="border-left: none; border-top: none;">Tester/issuer</td>
<td class="xl68" style="border-left: medium none; border-top: medium none; text-align: left; width: 270pt;" width="270">Any
defect to be retested is taken up by a tester to verify if the defect has
been really fixed in all instances. When the defect has been partially fixed
or not fixed (not when another regression defect occurs) the defect is set to
the status Open with the according evidence and comments.</td>
<td class="xl69" style="background: yellow; border-left: none; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">Open</td>
</tr>
<tr height="55" style="height: 55.0pt; mso-height-source: userset;">
<td class="xl78" height="55" style="background: yellow; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; height: 55.0pt; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">Retest</td>
<td class="xl79" style="background: yellow; border-left: none; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">Issuer</td>
<td class="xl80" style="background: none repeat scroll 0% 0% yellow; border: 0.5pt solid windowtext; color: black; font-family: Calibri; font-size: 9pt; font-weight: 400; text-align: left; text-decoration: none; width: 270pt;" width="270">Any defect to be retested needs
ultimately retesting of the issuer of the defect. When the fix is confirmed,
the issuer brings the defect to the final status of Closed-Fixed</td>
<td class="xl81" style="background: yellow; border-left: none; border-top: none; border: .5pt solid windowtext; color: black; font-family: Calibri; font-size: 9.0pt; font-weight: 400; mso-pattern: black none; text-decoration: none; text-line-through: none; text-underline-style: none;">Closed-Fixed</td>
</tr>
</tbody></table>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj7eXfRT9zTC3wCwpzx2Rb_gVytwqQM2TSqtXjka1y3cuxH8Cqx5zLQGvhcgwXosuWdbCZr9_qYbIuFWOdUT9YuTydXysZ3gd0gZafSt6_p1getIaH_3p0TWJbRAMYFgXzgG_o7hE97XIrH/s1600/Basic+defect+flow.jpeg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><br /></a></div>
<br />
<br />
<br />Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com61tag:blogger.com,1999:blog-2314019614094721290.post-39799188626015248692012-11-09T09:31:00.000+01:002012-11-09T09:33:10.281+01:00Introducing proper defect management<br />
A defect process, a process where project related defects (bugs) are addressed and resolved, can be run efficient or can become frustrating and very time consuming. Certain people don't take up their responsibilities, certain procedures are not clear and therefore not followed. Defects start flying up and down between parties and start revealing some people's deepest emotional frustrations or interesting dialogues, sometimes they even carry the only analysis of a certain topic available in the whole program... <br />
A defect is not an information exchange process, nor is it an analysis addendum or personal ventilation tool. A defect should be clear and unambiguous. The turn-around time of a defect should be quick. Therefore a defect needs to be light and to the point.<br />
<br />
<br />
<h4>
Process</h4>
The defect status flow is most often discussed. You might be using the companies existing process flow. However, often even if the process is defined, it is not always unambiguously explained. When your company is less mature in testing and test process implementation, it is advised to keep status flows as simple as possible. More statuses require more overhead and result in more confusion.<br />
The 3 main questions you need to clarify in order to describe a defect process are:<br />
<ol>
<li>What are the circumstances for a defect to arrive in a certain status?</li>
<li>Which role is responsible to treat a defect when it arrives in a certain status? </li>
<li>What are the next possible statuses a defect can arrive in?</li>
</ol>
<br />
<br />
<h4>
Defect procedures</h4>
Defect procedures look into your particular instance of the maybe generic defect flow. Who will take up certain roles? What are the steps to be undertaken in certain situations? Agree on clear responsibilities.<br />
These are, which I found vital defect procedures
<br />
<ul>
<li>Defect coordination board with as main purpose treating defects that are not agreed upon. </li>
<li>Defect retest procedures. Who retests where and how do we know a defect fix is deployed in which environment.</li>
</ul>
<div>
<br /></div>
<h4>
Defect guidelines or instructions </h4>
<h4>
<span style="font-weight: normal;">Defect guidelines or instructions are the good practices you share with all testers and developers when registering a defect. A developer needs to know how to reproduce a defect and a manager needs to see what a defect is exactly about at a first glance. </span></h4>
<h4>
<span style="font-weight: normal;">These are defect guidelines that I find most important:</span></h4>
<ul>
<li>Summary - One line issue description, following a convention of giving the location where the defect occurred and a short and clear description of the defect that occurs: e.g. Billing - Field 'Street' should also accept numeric values </li>
<li>Comments - Comments are added to a defect after them being discussed. No changes are made to a defect without a complete and concise explanation.</li>
<li>Prioritization - Define severity, impact and priority parameters unambiguously. The more statuses you invent, the more complex it will become to keep an overview. Keep it simple.</li>
</ul>
<br />
<h4>
Defect coordination is mainly communication</h4>
<h4>
<span style="font-weight: normal;">You can introduce a flawless
defect process, defect procedures and guidelines, if they are not understood and
carried by all involved parties, your defect coordination is doomed. When you are to become a defect coordinator, make sure you involve your main stakeholders. The easiest way to do so is to involve them in a workshop and start from a decent proposal. In environments where defect management is new, it often gets confused with production issue management. If defect management is a completely new item in the company, it is recommended to do a dry run with your stakeholders. </span></h4>
<br />
<ul>
</ul>
Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com2tag:blogger.com,1999:blog-2314019614094721290.post-46087711085042238202012-10-03T22:35:00.000+02:002012-10-03T22:35:02.833+02:00How software testing can contribute to a decrease in software qualityTime after time, project, program and line managers tried to impress me, telling they will increase the budget for testing, hoping to boost my interest and commitment as a test manager.<br />
Hallelujah!, I thought, when this happened to me for the first time.<br />
I felt the pressure as well as my growing set of responsibilities. Doubling a test team in the heat of the moment with only a couple of months left to deliver a massively buggy program, is not easy.<br />
<br />
Especially when at the same time, analysts are being fired, development delays, entry criteria are used to mop the vermin floor and the deadline resides, carved in stone. <br />
<br />
As we all know project delivery is always in fragile balance between cost, time and quality. <br />
While delivering within time and budget constraints trying to aim at quality, in reality quality is often decreasing despite of the increased focus on testing.<br />
<br />
How can this possibly happen?<br />
<br />
<b>1. Start testing too early</b><br />
<br />
It's the project managers first responsibility to deliver in time and budget. To do so the test manager easily gets overruled when entry criteria are not met. Starting to test when a product is in fact not yet testable, means that you start registering defects that are probably already known and are being taken care of. Looking into known defects is a waste of development time and budget which will without exception be rewarded with decreased quality.<br />
<br />
<b>2. Replace analysts with testers when analysis is done</b><br />
The analysts are too expensive to keep them on the project but are also most aware of the difficulties and issues. Replacing them by testers is not only cheaper, but also increases the team freshness. The real knowledge about the product and the issues walked away together with the analysts. The quality of the product inevitably decreases.<br />
<br />
<b>3. Make testers and client responsible for product quality</b><br />
Test cases are the full responsibility of the testers. Testers get demotivated to receive answers to questions as the analysts have disappeared. As an extra security, it is often practiced that the client validates the test cases, as they are responsible to accept. This altogether easily turns into ass-covering exercise.<br />
The issues have smaller chances to be found during testing and when they are found, they are more easily accepted. <br />
<br />
<b>4. CMMI Levels and TPI levels are used as silver bullets</b><br />
The process is sacred. Making sure that everything is done by the book makes a project easy to manage. Following predefined processes seems to add quality. When complexity increases, it is more tempting to fall back on the known processes and procedures, avoiding failure. However, following processes without identification of anomalies, decreases product quality.<br />
<br />
<br />
<br />
<b>5. Use test management tools to control testing activities</b><br />
Test management tools can easily contribute to 'report-driven testing'. Tests for example need to be created before execution and can not be increased or run twice because we need to be able to plan how many test cases there will be run per day per person and report against this plan. Test management tools make management give the illusion that testing is defined and measurable and tends to get micromanaged easily. This can incapacitate testers in doing their actual jobs (finding defects as soon as possible) and turn them into mindless test script executors.<br />
<br />
<br />
<br />Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com0tag:blogger.com,1999:blog-2314019614094721290.post-51279718323621670172012-09-18T22:56:00.000+02:002012-09-18T23:05:51.569+02:00Quality Insurance<div>
<br />
How does a change in the software affect your business process? <br />
What's the risk of having an issue in production? What's the occurrence of - and damage on failure?<br />
<br />
<br />
Most business fear lies in signing analysis documentation to send it off to development. They suddenly become responsible for knowing what they want and how they want it, before ever trying it out. Making mistakes is human, not being able to see the future is human as well. So why would you sign a document, knowing mistakes are most probably inside?<br />
What else could we do, except for agile product delivery to deliver the clients explicit and implicit changing needs? And even there, how can we prevent a business going broke as a result of a software defect that was not found during development?<br />
<br />
For example:<br />
<a href="http://www.reuters.com/article/2012/08/02/knightcapital-loss-idUSL2E8J27QE20120802">http://www.reuters.com/article/2012/08/02/knightcapital-loss-idUSL2E8J27QE20120802</a><br />
<br />
The answer might be.. QUALITY INSURANCE (or QI)<br />
<b><br /></b>
<br />
<h4>
<b>Why the term QI?</b></h4>
Quality insurance is a term used to oppose the often wrongly used term 'Quality Assurance'<br />
<br />
<h4>
<b>How would that conceptually work?</b></h4>
<br />
We would make a difference between soft- and hardware insurance.<br />
By default, this insurance system would not focus on the underlying infrastructure. Neither is it explicitly excluded from the insured scope.<br />
<br />
A software insurance policy could be taken after the Go/No-go decision was positive and would start at the end of supplier warranty. <br />
For starters, the business processes in scope of the insurance policy would be described by a team of experienced analysts and testers.<br />
An intake is done on the delivered product in a technical go live in a production-like environment.<br />
This intake would consist of an exploratory test. <br />
<br />
Possible product issues and risks are reported. Per identified risk, an insurance fee is mapped to an insured amount.<br />
The interesting risks to insure are the ones with a very low probability of occurrence, but having a very high impact on failure. </div>
<div>
<br />
The company can then decide to reduce risks (and insurance costs) by fixing defects if the fix cost is lower than the fee or decide to insure themselves for this risk.<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6Wo946pUzJ55RJmFbLAZ_jRhoUDfe-nw-Kn_U7sovzg2ESlnzDDoRPHtkFC1gQxC7RIdE6MY8ZIg2J8GoAIrlpqoj_Zai5INo0TuwK9U2Fa5OFw3yDkzA_0Z3qtTj9pa-dBfcbwy8rWw4/s1600/Slide1.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="480" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEj6Wo946pUzJ55RJmFbLAZ_jRhoUDfe-nw-Kn_U7sovzg2ESlnzDDoRPHtkFC1gQxC7RIdE6MY8ZIg2J8GoAIrlpqoj_Zai5INo0TuwK9U2Fa5OFw3yDkzA_0Z3qtTj9pa-dBfcbwy8rWw4/s640/Slide1.jpg" width="640" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
</div>
<b> </b><b> </b><br />
<b>What would be the Strengths?</b><br />
<ul>
<li>Increase the business attention on product risks. Issues can be resolved before go-live.</li>
<li>The business can go live with a product they are not sure about resulting an overall decreased project failure.</li>
<li>Direct linking of risks and issues with costs. This confronting exercise increases awareness.</li>
</ul>
<h4>
</h4>
<h4>
What would be the Weaknesses?</h4>
<ul>
<li>Resolving issues late in the development cycle is still more expensive. The direct cost of software delivery is not reduced but rather increased due to the additional insurance policy.</li>
</ul>
<b>What would be the Opportunities?</b><br />
<ul>
<li>Level out the current overrated importance of certification - a tester will need to prove what they're worth. Also development quality will stand more in the picture. </li>
<li>The real value of a good tester becomes visible. The tester's function would get more appreciation and direct value. </li>
<li>The value of the Exploratory Testing method can be proven. Only by exploring, focusing and de-focusing, the dangerous underlying issues, inconsistencies and process gaps can be found... and insured.</li>
<li>Increase software quality by increasing the awareness of the cost on failure.</li>
</ul>
<b>What would be the Threats?</b><br />
<ul>
<li>Decrease of software quality and user satisfaction due to competitive insurance policies. </li>
<li>Decrease of insurance quality due to competitive insurance policies </li>
</ul>
</div>
Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com0tag:blogger.com,1999:blog-2314019614094721290.post-40788647226898631252012-09-04T22:56:00.001+02:002012-09-05T08:04:48.070+02:0010 Signs that you're not cut to be a tester<div>
I had quite some fun reading trough signs indicating why you would be not cut for IT, not cut to be a developer or an IT consultant on the Tech republic blog:<br />
<a href="http://www.techrepublic.com/blog/10things/10-signs-that-you-arent-cut-out-for-it/3072">http://www.techrepublic.com/blog/10things/10-signs-that-you-arent-cut-out-for-it/3072</a><br />
<br />
I don't mean to say that it made a whole bunch of sense to me, but I did notice that one particular blog post was still outstanding... so I decided to suggest this one myself:<br />
<br />
These are my top 10 signs that you're not cut to be a tester:<br />
<br />
<h3>
1. You get nervous from buggy software</h3>
The software you're facing on a day to day base is going to be full of bugs and will look like a poor Picasso imitation. You'll need to find workarounds. And when a bug has been resolved, you'll be facing the next problem. You'll have to go trough again. <br />
<br />
<h3>
2. You're unaware of business expectations</h3>
If you're not aware of business expectation, you'll look over half of the defects that are in front of you. You'll read over inconsistencies and you won't question the rationale behind an architectural solution that has been proposed. An overview of what business wants and why they need an application, gives you structure and focus on finding the defects that potentially are of high impact.<br />
<br />
<h3>
3. You get tired from explaining defects occurrences</h3>
Not everyone knows the system like you do. Not everyone understands the reason why a bug is a bug. You'll need to explain it... again and again. Even when it works on their machine...<br />
<br />
<h3>
4. You don't read blogs, books or attend conferences about software testing</h3>
A tester keeps up to date on the latest evolutions on tools, techniques, methods and can learn from practical experience of other testers. It's no use to invent what already has been invented for you. It is even less useful not to learn from others mistakes.<br />
<br />
<h3>
5. You're ashamed of your role in the project</h3>
A tester could also be an analyst, a developer, a designer or an architect, but chooses to be a tester. A tester is not a developer or analyst, coming close to retirement. A tester is proud of their job and aims on improving software quality. They aim on challenging development, analysis and architecture on their correctness, completeness and coherence. A tester is proud of and dedicated to find the human flaws in a program before it is released to production.<br />
<br />
<br />
<h3>
6. You know how to check but you don't know how to explore</h3>
Checking against expectations is a given. This needs to be done. After this, the software needs exploration. How else are we going to find those undefined features and prove that if you click on the left mouse button 6 times, then do a page refresh, while keeping your right mouse button clicked and then releasing the right mouse button after which you disconnect the keyboard, your money has gone from your account, but has not been transfered?...<br />
<div>
<br /></div>
<br />
<h3>
7. You are not keeping up to date on the technical aspects of IT</h3>
IT terms are not just hype words for a tester. A tester has to have an understanding of OO, has to know what a web service is or a message queue, what the function is of an isolation layer or why server and client side validations can be implemented while building a website. A tester needs to constantly improve and understand what they are technically dealing with as every technicality has it's weakness. That's where most probably the defects will be.<br />
<br />
<h3>
8. You don't like to communicate</h3>
Communication, communication, communication. Testing is about finding bugs and inconsistencies and communicating about them.<br />
<br />
<h3>
9. You are not aware of the application development life cycle</h3>
A tester needs to know about architecture, design, analysis, development, infrastructure, release management, ... and also how they fit together. A testers could even carefully address flaws in the application life cycle, since they might result in software bugs later on.<br />
<br />
<h3>
10. You can't get over defects you found and don't get fixed</h3>
Face it. You'll find those defects that are obvious and you would never want to see in your software. However, you're not paying for every single defect to be fixed. Decisions will be made that might not make you happy. Your job is to point to the risks and issues of not fixing a defect. Your job is not to get them resolved.</div>
Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com4tag:blogger.com,1999:blog-2314019614094721290.post-31952350253939220662012-09-01T13:47:00.001+02:002012-09-03T08:07:30.483+02:00Becoming an Inspiring Test CoordinatorMany starting testers would like to become test coordinators in stead of better testers. However, many good test coordinators are good all-around testers with extra coordination skills. I don't know for myself, how good of a tester I would make but I know I always liked testing and I keep improving my testing skills, even though officially, I'm not testing anymore for the last 3 years. <br />
<br />
Seeing Test Coordinators, and working with and for them, I realized there are only few inspiring leaders among them. The biggest pitfalls a test coordinator has in front of them, that keep them from becoming inspiring leaders are the following:<br />
<br />
<h4>
</h4>
<h4>
"The common enemy" or "The common goal"?</h4>
As a coordinator, you'll need to run test activities like clockwork. The best way to do so is by making your individual testers work as a team. Bringing people together can be done the easy way, by finding the common enemy, or the hard way, by identifying the common goal. The common goal sounds a bit naive maybe. Facing a goal that you might not reach also requires some sort of bravery. Defining achievable targets towards the common goal, is the key to move on and increase the team spirit on success and even on failure.<br />
<br />
<br />
<h4>
"Driving the heard" or "Leading by example"?</h4>
Telling people what to do, often results in the opposite effect. The best way to get respect, is to take part in the test activities. So you better aim to be a good tester, if you ever want to become a good coordinator. Coaching is part of the job. If a tester doesn't know how to get started, you will have to come up with something that works. If your testers miss out on something, it's your job to find out about it and make them aware. <br />
<br />
<h4>
<b>"Responsibilization" or "Taking the problems away"?</b></h4>
Your testers will face issues constantly. Unstable environments, data restore issues, changing user expectations, defect discussions... Issues will overcome them on a daily base. You have 2 options. Try to let your testers solve the issues by making them responsible or taking their problems away. Taking problems away is often a hard job and might not be very rewarding. You'll need to make your team aware of how you manage their problems, you can even ask them for advice.<br />
<br />
<br />
<h4>
"Push pressure further" or "Push pressure back"?</h4>
<span style="font-weight: normal;">Project managers like to hear that all is going well and tend to become a pain in the ass when testing is not proceeding as planned. Mostly that is because all the buffers they placed in the project are used and your poor test activities are getting squeezed in timelines. Your job is to make sure that testing continues like clockwork and keep management changing ideas, controls and unnecessary reports as far away from your testers as possible. </span><br />
<br />
<h4>
</h4>
<h4>
</h4>
<h4>
<b>"Mushroom coordination" or "Transaparant coordination"?</b></h4>
<br />
Knowledge is power. As a test coordinator, you know more than your team members. You take part in steering and coordination meetings. You can use this knowledge either to outsmart your team members and stay one step ahead of them, or you can use it to inform your team members for which they will reward you with trust.<br />
<br />
<br />
<br />
<br />Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com0tag:blogger.com,1999:blog-2314019614094721290.post-17763314772564922412012-08-26T15:54:00.001+02:002012-08-26T15:54:59.663+02:00How to start a new mission as a Test ManagerStarting to work in a new company or starting a new mission that sometimes even looked pretty similar to the previous one, I always find every mission to turn out quite different in it's objectives and challenges.<br />
<br />
In this blog post I will focus on how I learnt to orient in a new job as a test coordinator or test manager.<br />
<br />
<b></b>
<h4>
<b>Who will I be working for? </b></h4>
This might be a Project or Programme manager or Line manager - but there might be other important stakeholders out there. Find out who they are. Who will need/want information from you? They can be persons trusted by higher management, key business users, release managers, operation teams...These are your stakeholders. Know who they are and what they want, as you will be working for them and reporting to them (this can be directly or indirectly). <br />
<br />
<br />
<b>Who will I be working with?</b><br />
<ul>
<li>Who are your team members that will be reporting to you? What are their skills? What do you expect and what can they deliver? Find out what you can learn from them and what you will need to teach them. Also here, involve your stakeholders so that you can align the stakeholders' wishes with the budget they need to set aside for it. <br />Your team can be on shore or off shore, external or internal, and work under different contractual situations. You need to get a hold of each of them and find a way to make all of them work together as ONE team.</li>
<li>Who are your direct colleagues that you need to share information with? Release management or deployment management, delivery management, analysts, designers, architects, project managers, development, infrastructure... Get to know them and find out for each of them how you will exchange information and what you will require or what might be required from you. You'll need a project planning, a methodology, an architectural blueprint, a data model, designs, analysis or a test environment in a specific way so that you can use it for testing. You'll need to address questions, issues, defects and in this way that they accept them and understand your team's role. </li>
</ul>
<i>Watch and observe. Treat your colleagues and team members with respect. Understand what they need from you. Make them understand what you need from them. Communicate with them.</i><br />
<br />
<h4>
<b>How does the project team structure look like?</b></h4>
You
can be working in many different kinds of structures using horizontal
divisions or vertical divisions, flat structures, resource pools, very hierarchical structures or even complete chaos. A
project manager might in reality be an issue manager or a project
management officer. A business analyst might be an architect and a
designer might be an analyst. Get to an understanding of the project
structure and your responsibilities. Find out how the informal and formal roles and responsibilities are defined and where you find yourself in this structure. You can even test the
project structure, identify the gaps and bring them up to your
management. It's always your job to consult in favor of qualitative
delivery.<br />
<br />
<br />
<b>How is testing implemented and what are my responsibilities? </b><br />
<br />
<ul>
<li>Look at<u>
test levels </u>- Are you responsible for unit, system testing, system integration testing, acceptance testing? Who will be performing the tasks? Define your
scope and check with your stakeholders. </li>
<li>Look at <u>reference material for testing</u> - What will you need to perform testing and to prepare for testing? Assess the quality of the reference material and inform about possible risks or issues.</li>
<li>Look at <u>entry<i> </i>and exit criteria</u> - Are you responsible to report on entry criteria or to even monitor them up front?</li>
<li>Look
at <u>the testing approach</u> - How is testing performed currently? Can it be
improved? Point to gaps in the used methods and to their risks to your
stakeholders and find out if those risks are accepted or if you suddenly
increased your job scope. </li>
<li>Look at <u>test environments and infrastructure</u> - Are you
responsible for defining, ordering, maintaining test environments or test
infrastructure, defining or improving release processes? </li>
<li>Look at<u> test types</u>. Are you responsible for functional testing only or also for usability testing, security testing, regression testing, performance testing... ? Do you have the required skills in your team to address the required test types to be performed?</li>
<li>Look at <u>test tools</u>. Which test tools will you require or are standard in the company? Do you need to define or adhere to procedures? Make sure all your team members are mastering and if required contributing to the tool selection and setup.</li>
<li>Look at <u>communication and reporting patterns</u> - What is discussed in which structural meetings? Which deliverables, reports and metrics are expected at which times? What would you think would be required? Come forward with proposals.</li>
</ul>
Refer to existing, representative products and processes in the company. Find out how testing has been performed in the past. What was expected, what was delivered and how it was the result perceived by stakeholders.<br />
<br />
<i>Don't go in and re-invent the wheel. Gather the existing knowledge. If there is one, adopt the existing way of working. Bring your colleagues, team members and stakeholders towards improvement step by step.</i><br />
<br />
<br />
<b>Some tips and tricks that help me</b><br />
<ul>
<li>Make a list of your colleagues' names. You can make a drawing of the desks in the office and put their names on it.</li>
<li>Document what you learn as you learn it (if not yet documented) and store it in a central place. Let other team members contribute to that. (special phone numbers, creation of test users,...) </li>
<li>Hear people out. Ask for their opinions. Show that you can listen - and learn from their skills and experience.</li>
<li>Wear clean, washed clothes. Don't go meeting people directly after smoking.</li>
<li>Say good morning and good evening to everyone on the floor.</li>
<li>Don't say directly YES or NO if you don't understand the request or question completely, but say you will look into it and come back on the next day.</li>
<li>Set-up workshops with your business users and identify the important end to end scenario's with them as soon as possible. This will keep you focused during project delivery.</li>
<li>Always get to an agreement with at least your key stakeholders separately before proposing new elements to them together in a meeting.</li>
<li>Estimate and plan, using techniques and common sense. Make plans with
input from your team members. Make them all agree on and commit to the
plans before presenting and committing a plan to your management.</li>
<li>As a people manager, try to make a positive difference for your people. </li>
</ul>
<ul>
</ul>
Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com0tag:blogger.com,1999:blog-2314019614094721290.post-31432814224330013282012-08-13T08:24:00.000+02:002012-08-13T14:46:30.964+02:00List of requirements for a new test environmentIt was the first time we needed an integration test environment that we didn't have yet. <br />
There was no infrastructure, no procedures, no version control, no data restore process,...<br />
What we needed was a controlled test environment where we could test integration for a project of considerable size. <br />
<br />
As a tester, I know that knowing what you want, is not obvious and putting it all together to make it work is even more difficult. So, a long brainstorm session with all stakeholders, some Google sessions and blog-scans came afterwards.<br />
It lead to the following list of requirements<b> </b>that we used as a base, to start building our integration test environment.<b><br /></b><br />
<br />
<h4>
<b>Testability requirements</b></h4>
<ul>
<li><span style="background-color: white;">All applications that
exist or are to be delivered in the production environment have to
exist or simulated in the test environment. </span></li>
<li>List of settings and configurations is available together with the
comparison of the settings and configurations of the production
environment.
</li>
<li>Required tools and licenses are available for testers (<i>If for any reason tools can not be provided, a work-around is in place to facilitate testing)</i></li>
<li>Front end devices to access software through different channels are available <i>(card readers, GPS Units, desk tops, laptops, pads, phones,...)</i></li>
<li>Batch procedures are internally controllable and automated as in a production environment (<i>manual trigger, start, pause, stop batch procedures)</i></li>
<li>Measuring tools are in place to enable monitoring the communication between applications and application layers.</li>
<li>External Connectivity to test environments for support, deployment and testing is required <i>(as different vendors need to deploy and test their software and integration on the environment)</i></li>
<li>Time traveling is possible in at least one of the following means: "<span style="background-color: white;">time goes by faster (eg: 1 week = 1 hour)", "</span><span style="background-color: white;">move forward and reset from a defined time in the future", "</span><span style="background-color: white;">coherent data creation/update to a time stamp in the past"</span></li>
<li><span style="background-color: white;">Input and/or output simulators are in place and documented for every interface type</span></li>
<li><span style="background-color: white;">All configurations of the different interfacing systems within the test environment have to be centrally managed</span> </li>
<li>At least 1 test user exists for every defined role in the application under test</li>
<li>Database read access is required for all testers<span style="background-color: white;"> </span></li>
<li>The test infrastructure has an availability of 95% (planned deployments are not included)</li>
<li>Defects are centrally managed, using 1 agreed defect managing system and flow </li>
</ul>
<br />
<h4>
<b>Test data requirements</b></h4>
<ul>
<li>Any production data loaded into the test environment is scrambled in
a way that it still fits logically together but is impossible to link
back to actual clients in the production database</li>
<li>A representative set of production(-like) test data can be unloaded in the test environment</li>
<li>Back-up and restore procedures exist to back-up and release a partial or full data extract</li>
<li>Back-up and restore procedure does not take longer than 4 working hours</li>
<li>Back-up and restore procedure should be possible at least on a weekly basis with a lead time of 1 working day</li>
<li>Full Data refresh with scrambled production data is possible on request with a lead time of at least 1 working week. </li>
<li>A procedure to request for a data is implemented</li>
</ul>
<h4>
<span style="background-color: white;"> </span><b>Deployment/Release management requirements</b></h4>
<ul>
<li>Version control is applied to all development products and system documentation</li>
<li>Deployments are always done from 1 agreed central data store </li>
<li>Roll-back scenario's are in place for every deployment cycle</li>
<li>Users are trained and informed about deployment management agreements</li>
<li>A patch procedure is known, documented and implemented</li>
<li>Deployment tasks follow formalized checklists - verified by the test team</li>
<li>Builds or data refreshes in a test environment take only place after approval of the impacted test teams </li>
<li>A deployment/release management role is assigned</li>
<li>Every defect fixed for a new deployment in the test environment is at least communicated within an agreed defect management tool </li>
<li>Every deployed, fixed component is at least communicated trough a build note</li>
<li>Builds are subject to formal and/or informal entry criteria </li>
</ul>
<br />
<div>
</div>
<h4>
<b>Maintainability requirements</b></h4>
<ul>
<li>Procedures to add/delete or update hardware components are existing and followed</li>
<li>The test environment components are sized in relation to the production environment</li>
<li>Log files are available for the different applications deployed on the test environment</li>
<li>Procedure exists to request and add test users for any application in the test environment</li>
<li>Disaster recovery plan is known, documented and implemented</li>
<li>Unexpected downtime of the (partial) test environment is communicated to all stakeholders (preferably using a central communication channel - for instance an intranet website)</li>
<li>Physical locations of different infrastructural components are identical to the production environment and documented. Any deviation is clearly identified and approved by the project team </li>
</ul>
<br />
<br />
<br />
I definitely forgot some requirements and not all of them are actually in place. But but based on these we set up an environment we can control with a handful of people and test in properly. We are agile enough to patch and refresh data properly which improved the test experience drastically.<br />
Any suggestions to improve this list are welcome.<br />
<br />Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com28tag:blogger.com,1999:blog-2314019614094721290.post-3834016545101681282012-08-08T22:31:00.001+02:002012-08-09T13:08:28.505+02:00How to deliver a qualitative product<br />
The mind-map I'm posting, has been created by looking back on different projects and consulting different colleagues, including my wife, the best Business Analyst I've ever seen. It is most probably not complete. But I think it is ready to start sharing it.<br />
<br />
The idea behind this mind-map is to focus on key aspects required to deliver a qualitative software product of considerable size and how testers can contribute to obtain a higher level of quality of the product to be delivered.<br />
<br />
<div style="text-align: left;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEjekbMS-6P14j3CZcPrfv_wVxQlim46a5SVv0hhsrf_a3ukgMWU4Litg3WKnRveqtS8r3VLbQ0ZaPrfs0KysBlZsCFnn5XFqr7KbnAQoStXxOnkkCtFfrx0Z7Zzh-XndqBR4nM1VWzWfsvr/s1600/Deliver+a+qualitative+product.jpeg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><br /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglJEGUvh-mvKgRhC-2Lwej0F7sUyZstgRXd4F9mczowdm-82T04GYNxubpim_LCuBzBet67KxD46Ot5muVxH1fjyjaadwrkYZ1cHp2rGXdVhJtRAX2HoMdy5Qgh7ydS_Z1HKzyQeoSkMPq/s1600/Deliver+a+qualitative+product.jpeg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img border="0" height="198" src="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEglJEGUvh-mvKgRhC-2Lwej0F7sUyZstgRXd4F9mczowdm-82T04GYNxubpim_LCuBzBet67KxD46Ot5muVxH1fjyjaadwrkYZ1cHp2rGXdVhJtRAX2HoMdy5Qgh7ydS_Z1HKzyQeoSkMPq/s400/Deliver+a+qualitative+product.jpeg" width="400" /></a></div>
<div class="separator" style="clear: both; text-align: center;">
<a href="https://blogger.googleusercontent.com/img/b/R29vZ2xl/AVvXsEh5I-b65zKYuwCiiCCYts0xFkEZl6uQhnZ0UwjJqbNOsgvSdLzpTXXidzAPGQcBOt0qbrS5q-qCwiaVtuCFflLNjjG5fQ6gSRslGsZGal9Y0Ic_VwJoLmkFc4HmMx2qwes8fcS77r3K4EPq/s1600/Deliver+a+qualitative+product.jpeg" imageanchor="1" style="margin-left: 1em; margin-right: 1em;"><br /></a></div>
<br />
<br />
If a tester wants to contribute to quality, they can do more than testing a product when it is delivered. Testers have many other duties during software development. They know the product and project in- and outputs and are aware of their risks. They understand the changing business needs and their explicit and implicit requirements. They challenge the architecture and analysis, coach developers and acceptants to check their delivered product effectively on standards. They facilitate in environment and data preparation. A tester understands and contributes to the release and deployment mechanisms. <br />
<br />
<br />
<h4>
<b>Planning & Control</b></h4>
It is the testers job to:<br />
<ul>
<li>Take part in the project planning and approach; challenge the planning </li>
<li>Look into the processes and challenge them in function of quality delivery</li>
<li>Find dependencies and flag them up to project/programme management</li>
<li>Understand the test requirements and build a log of them </li>
<li>Advise on test tool selection in function of the budget, timeline, experience and test requirements.</li>
</ul>
<br />
<h4>
<b> Design</b></h4>
It is the testers job to:<br />
<ul>
<li>Understand the product needs and rationale behind the project </li>
<li>Test the designs and architectural models and point to their weaknesses</li>
<li>Find and report flaws in communication patterns and stakeholder involvement</li>
<li>Coach acceptants in the definition of E2E prototyping test cases; lead them and help them in the E2E test preparation. </li>
</ul>
<br />
<h4>
<b>Delivery</b></h4>
It is the testers job to:<br />
<ul>
<li>Test, test, test, and automate tests</li>
<li>Coach developers in checking against standards</li>
<li>Contribute to and cooperate closely with deployment and release management</li>
<li>Coordinate proper and clean bug reporting and the bug life cycle </li>
<li>Provide and manage the required test infrastructure.</li>
</ul>
<br />
<h4>
<b>Acceptance</b></h4>
It is the testers job to:<br />
<ul>
<li>Coach the acceptants in their verification and testing process</li>
<li>Analyze and report test results in simple and understandable language</li>
<li>Provide release advise</li>
<li>Coordinate E2E test execution.</li>
</ul>
<br />
<h4>
<b>Communication</b></h4>
A tester is a great communicator. They understand the business needs, reason with analysts, challenge architects and managers and coach users and developers.<br />
<br />
<ul></ul>Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com0tag:blogger.com,1999:blog-2314019614094721290.post-27953111359431496402012-08-02T11:57:00.000+02:002012-08-02T12:07:52.735+02:00Why TMap should be called CMap<style>
<!--
/* Font Definitions */
@font-face
{font-family:"Courier New";
panose-1:2 7 3 9 2 2 5 2 4 4;
mso-font-charset:0;
mso-generic-font-family:auto;
mso-font-pitch:variable;
mso-font-signature:3 0 0 0 1 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;
mso-font-charset:2;
mso-generic-font-family:auto;
mso-font-pitch:variable;
mso-font-signature:0 268435456 0 0 -2147483648 0;}
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;
mso-font-charset:0;
mso-generic-font-family:auto;
mso-font-pitch:variable;
mso-font-signature:3 0 0 0 1 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;
mso-font-charset:0;
mso-generic-font-family:auto;
mso-font-pitch:variable;
mso-font-signature:3 0 0 0 1 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-parent:"";
margin-top:0cm;
margin-right:0cm;
margin-bottom:10.0pt;
margin-left:0cm;
line-height:115%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:Calibri;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:FR-BE;}
p.MsoHeader, li.MsoHeader, div.MsoHeader
{mso-style-priority:99;
mso-style-link:"Header Char";
margin:0cm;
margin-bottom:.0001pt;
mso-pagination:widow-orphan;
tab-stops:center 8.0cm right 16.0cm;
font-size:11.0pt;
font-family:Calibri;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:FR-BE;}
p.MsoFooter, li.MsoFooter, div.MsoFooter
{mso-style-priority:99;
mso-style-link:"Footer Char";
margin:0cm;
margin-bottom:.0001pt;
mso-pagination:widow-orphan;
tab-stops:center 8.0cm right 16.0cm;
font-size:11.0pt;
font-family:Calibri;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:FR-BE;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
mso-themecolor:hyperlink;
text-decoration:underline;
text-underline:single;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-noshow:yes;
mso-style-priority:99;
color:purple;
mso-themecolor:followedhyperlink;
text-decoration:underline;
text-underline:single;}
p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
margin-top:0cm;
margin-right:0cm;
margin-bottom:10.0pt;
margin-left:36.0pt;
mso-add-space:auto;
line-height:115%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:Calibri;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:FR-BE;}
p.MsoListParagraphCxSpFirst, li.MsoListParagraphCxSpFirst, div.MsoListParagraphCxSpFirst
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
mso-add-space:auto;
line-height:115%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:Calibri;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:FR-BE;}
p.MsoListParagraphCxSpMiddle, li.MsoListParagraphCxSpMiddle, div.MsoListParagraphCxSpMiddle
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:0cm;
margin-left:36.0pt;
margin-bottom:.0001pt;
mso-add-space:auto;
line-height:115%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:Calibri;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:FR-BE;}
p.MsoListParagraphCxSpLast, li.MsoListParagraphCxSpLast, div.MsoListParagraphCxSpLast
{mso-style-priority:34;
mso-style-unhide:no;
mso-style-qformat:yes;
mso-style-type:export-only;
margin-top:0cm;
margin-right:0cm;
margin-bottom:10.0pt;
margin-left:36.0pt;
mso-add-space:auto;
line-height:115%;
mso-pagination:widow-orphan;
font-size:11.0pt;
font-family:Calibri;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:FR-BE;}
span.HeaderChar
{mso-style-name:"Header Char";
mso-style-priority:99;
mso-style-unhide:no;
mso-style-locked:yes;
mso-style-link:Header;}
span.FooterChar
{mso-style-name:"Footer Char";
mso-style-priority:99;
mso-style-unhide:no;
mso-style-locked:yes;
mso-style-link:Footer;}
.MsoChpDefault
{mso-style-type:export-only;
mso-default-props:yes;
font-size:11.0pt;
mso-ansi-font-size:11.0pt;
mso-bidi-font-size:11.0pt;
font-family:Calibri;
mso-ascii-font-family:Calibri;
mso-ascii-theme-font:minor-latin;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-hansi-font-family:Calibri;
mso-hansi-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;
mso-ansi-language:FR-BE;}
.MsoPapDefault
{mso-style-type:export-only;
margin-bottom:10.0pt;
line-height:115%;}
@page WordSection1
{size:595.3pt 841.9pt;
margin:70.85pt 70.85pt 70.85pt 70.85pt;
mso-header-margin:35.4pt;
mso-footer-margin:35.4pt;
mso-paper-source:0;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:1466660044;
mso-list-type:hybrid;
mso-list-template-ids:1953762444 -149113972 135004185 135004187 135004175 135004185 135004187 135004175 135004185 135004187;}
@list l0:level1
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-weight:bold;}
@list l0:level2
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level3
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level4
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level5
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level6
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l0:level7
{mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level8
{mso-level-number-format:alpha-lower;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;}
@list l0:level9
{mso-level-number-format:roman-lower;
mso-level-tab-stop:none;
mso-level-number-position:right;
text-indent:-9.0pt;}
@list l1
{mso-list-id:2095780611;
mso-list-type:hybrid;
mso-list-template-ids:202144528 750414430 135004163 135004165 135004161 135004163 135004165 135004161 135004163 135004165;}
@list l1:level1
{mso-level-number-format:bullet;
mso-level-text:-;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Calibri;
mso-fareast-font-family:Calibri;
mso-fareast-theme-font:minor-latin;
mso-bidi-font-family:"Times New Roman";
mso-bidi-theme-font:minor-bidi;}
@list l1:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Courier New";}
@list l1:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l1:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l1:level5
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Courier New";}
@list l1:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
@list l1:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Symbol;}
@list l1:level8
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Courier New";}
@list l1:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:none;
mso-level-number-position:left;
text-indent:-18.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
-->
</style>
<br />
<div class="MsoNormal">
<span style="mso-ansi-language: EN-US;">As I told before, I’ve
been raised with TMap.<span style="mso-spacerun: yes;"> </span>And since I
arrived in, what one could call ‘test puberty’, I start to make up my mind about
testing and find my own way.<span style="mso-spacerun: yes;"> </span>My adventure
lead me to a great blog post that made me think about testing and checking (<a href="http://www.ministryoftesting.com/2012/07/mindmaptesting-and-checking/">http://www.ministryoftesting.com/2012/07/mindmaptesting-and-checking/</a>)
which lead me to Michael Bolton’s blog (<a href="http://www.developsense.com/blog/2009/08/testing-vs-checking/">http://www.developsense.com/blog/2009/08/testing-vs-checking/</a>)
and made me realize that TMap (Test Management Approach) should actually be
called CMap (Check Management Approach)<span style="mso-spacerun: yes;"> </span></span></div>
<div class="MsoNormal">
<span style="mso-ansi-language: EN-US;">If the above 2 links
are new to you, I would recommend to go and read them. </span></div>
<div class="MsoNormal">
<span style="mso-ansi-language: EN-US;">I don’t have the time
or will to get into every detail of TMap to make my point, as this "bible" counts
over 700 pages of valuable information about testing. Therefore, I’m about to
touch the most important parts of the book and explain how I want to propose to
apply changes to the book in the next version(s).</span></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span style="mso-ansi-language: EN-US;">Let’</span></b><b style="mso-bidi-font-weight: normal;"><span lang="EN-GB" style="mso-ansi-language: EN-GB;">s start with the
definition of testing.</span></b></div>
<div class="MsoNormal">
<span style="mso-ansi-language: EN-US;">TMap<sup>®</sup>:
Testing is a process that provides
insight into, and advice on, quality and the related risks</span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">I would
like to split it out into 2 new definitions</span></div>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;">
<b style="mso-bidi-font-weight: normal;"><span style="mso-ansi-language: EN-US; mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">1.<span style="font: 7.0pt "Times New Roman";">
</span></span></span></b><b style="mso-bidi-font-weight: normal;"><span style="mso-ansi-language: EN-US;">Checking</span></b><span style="mso-ansi-language: EN-US;"> is a process that <b style="mso-bidi-font-weight: normal;">possibly </b>provides
insight into, and advice on, quality and the related risks</span></div>
<div class="MsoListParagraphCxSpLast" style="mso-list: l0 level1 lfo1; text-indent: -18.0pt;">
<b style="mso-bidi-font-weight: normal;"><span style="mso-ansi-language: EN-US; mso-bidi-font-family: Calibri; mso-bidi-theme-font: minor-latin;"><span style="mso-list: Ignore;">2.<span style="font: 7.0pt "Times New Roman";">
</span></span></span></b><span style="mso-ansi-language: EN-US;">Testing
is an <b style="mso-bidi-font-weight: normal;">investigation </b>that<b style="mso-bidi-font-weight: normal;"> </b>provides insight into, and advice on,
quality and the related risks.</span></div>
<div class="MsoNormal">
<span style="mso-ansi-language: EN-US;">Why is testing not a
process?</span></div>
<div class="MsoNormal">
<span lang="FR-BE">A <b>process</b> is a set of interrelated
tasks that, together, transform inputs into outputs (<a href="http://en.wikipedia.org/wiki/Process_%28engineering%29%29">http://en.wikipedia.org/wiki/Process_(engineering))</a>
</span></div>
<div class="MsoNormal">
<span lang="FR-BE">So we should have:</span></div>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l1 level1 lfo2; text-indent: -18.0pt;">
<span lang="FR-BE" style="mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-fareast-font-family: Calibri; mso-hansi-font-family: Calibri;"><span style="mso-list: Ignore;">-<span style="font: 7.0pt "Times New Roman";">
</span></span></span><span lang="FR-BE">Input: Test base, test cases,
expected results</span></div>
<div class="MsoListParagraphCxSpLast" style="mso-list: l1 level1 lfo2; text-indent: -18.0pt;">
<span lang="FR-BE" style="mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-fareast-font-family: Calibri; mso-hansi-font-family: Calibri;"><span style="mso-list: Ignore;">-<span style="font: 7.0pt "Times New Roman";">
</span></span></span><span lang="FR-BE">Output: Defects, test report</span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">In reality,
testing can not be a process. <span style="mso-spacerun: yes;"> </span>Why not?
Because we can’t plan where we will find the bugs, so we can’t plan were we
will need to look and for how long. Was the analysis/development consistent
with the company’s history, is compliant with the business process? What were
the real user expectations? In order to do so, we have to plan an investigation
and re-plan, based upon our findings.</span></div>
<div class="MsoNormal">
<span lang="FR-BE">Checking can be mapped out in a process and
can be easily planned. What do we check ? When do we check ? How much
re-checking do we estimate ? Which bugs did we find, checking which
requirements ? We can check if requirements are implemented correctly. We
can check if field validations are representing the analyzed error messages, we
can verify the load and stress requirements...</span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">Check
specification (scripting of checks) can only be done for the parts that we can check against
expectations or a predefined standard. Testing we can do out of those boundaries.
What else can the system do that was not specified? Do we want the application
to be able to do this? Is it important to test this?</span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span lang="EN-GB" style="mso-ansi-language: EN-GB;">What is testing?</span></b></div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">When we
elaborate further on testing, according to TMap, each definition of testing
compares a product with a predefined standard. </span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">Of course
this is the ultimate way to deliver a set of beautiful metrics later on. But
what about those expectations that were not predefined? What about those orphan
features, those implicit requirements or poorly communicated company standards…
or those gaps that nobody wanted to add to the predefined standard? Are those
out of scope for testing because they were not defined?</span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">For
Checking, I would agree. Checking is this very important task that great developers
do and that is of high importance during the acceptance phase of product
delivery.</span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span lang="EN-GB" style="mso-ansi-language: EN-GB;">The 4 Essentials</span></b></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span lang="EN-GB" style="mso-ansi-language: EN-GB;">1. Business driven test management</span></b></div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">This area
of TMap offers a great method and toolset to prepare for checking and testing
with a correct business focus and performing proper product risk analysis. From
the moment the risks are defined in the BDTM life cycle, testing and checking in
my opinion part ways.</span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">Checking
continues perfectly conform to the TMap methodology, where at a certain moment in
time check goals are consolidated, product risks are defined, test and check thoroughness
is defined and check-scripts are being written and executed.</span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">Testing
however will not consolidate. Risk areas might be reduced or increased, new
goals might be introduced and other goals will disappear. Testing will be
focused on those risks and goals within a changing environment. There will be
more cycles passing trough business, there will be less script-preparation but
a lot of testing will take place and will get documented.</span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">I would
propose to separate BDTM from TMap as this toolset and method is a very
valuable one for testing as well as for checking. I can only recommend it to anyone who
hasn’t used it before.</span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span lang="EN-GB" style="mso-ansi-language: EN-GB;">2. The structured test process</span></b></div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">See
definition of testing. Testing is not a process, it is an investigation.</span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">Checking
however can follow a process. The one described in TMap offers a great way to
do so. My proposal is to change testing into checking here for the sake of
terminology and testing. </span><br />
<br />
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">I</span><span lang="EN-GB" style="mso-ansi-language: EN-GB;"> have to
add here that there is one lovely part in the TMap test process. Specification
of the test infrastructure and the checklists provided to do so, helped me
quite a lot to specify and not to forget about all artefacts I had to order to
enable testing.</span></div>
<div class="MsoNormal">
<br /></div>
<div class="MsoNormal">
<b style="mso-bidi-font-weight: normal;"><span lang="EN-GB" style="mso-ansi-language: EN-GB;">3. Complete toolbox</span></b></div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">For
starters, a toolbox in my opinion is never complete. Only a sales department will try to
convince me that they can offer me a complete toolbox…. Not an expert, using
tools. I have to be fair TMap doesn’t repeat the statement further in the book.
It is mentioned that there is a large set of tools present, which cannot be
denied.</span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">I would still
request for the addition the tea-test and the shoe-test for example. Those
tools, I tend to use on every occasion I get to test software and sad enough,
stay very rewarding.</span><br />
<br />
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">TMap offers a great bunch of checklists and templates on <a href="http://www.tmap.net/">http://www.tmap.net/ </a></span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">I have seen
and used almost all of the TMap test tools but always seem to come to the same
conclusion that they are check-tools or used as check tools when they really
could be test tools.</span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">For
example: The well known BVA (Boundary Value Analysis)</span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">BVA is a
great test tool but if you are looking for boundaries only in the predefined standard,
you turn your test tool into a check tool. How do you know for sure if you know
the real boundaries or if you know all boundaries? Checking applies BVA to the
information provided in the standard. Testing is looking for the real
boundaries in the system.</span></div>
<div class="MsoNormal">
</div>
<div class="MsoNormal">
<h4>
<b style="mso-bidi-font-weight: normal;"><span lang="EN-GB" style="mso-ansi-language: EN-GB;">4. Adaptive method</span></b></h4>
</div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">I have to
admit that TMap is an adaptive method. You don’t have to use any of it if you
don’t want to and you can pick and choose the useful things that are relevant for your context.</span></div>
<div class="MsoNormal">
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">I propose
to use the following great features TMap offers</span></div>
<div class="MsoListParagraphCxSpFirst" style="mso-list: l1 level1 lfo2; text-indent: -18.0pt;">
<span lang="EN-GB" style="mso-ansi-language: EN-GB; mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-fareast-font-family: Calibri; mso-hansi-font-family: Calibri;"><span style="mso-list: Ignore;">-<span style="font: 7.0pt "Times New Roman";">
</span></span></span><span lang="EN-GB" style="mso-ansi-language: EN-GB;">BDTM (taking into account that testing scope changes)</span></div>
<div class="MsoListParagraphCxSpMiddle" style="mso-list: l1 level1 lfo2; text-indent: -18.0pt;">
<span lang="EN-GB" style="mso-ansi-language: EN-GB; mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-fareast-font-family: Calibri; mso-hansi-font-family: Calibri;"><span style="mso-list: Ignore;">-<span style="font: 7.0pt "Times New Roman";">
</span></span></span><span lang="EN-GB" style="mso-ansi-language: EN-GB;">All
provided checklists and techniques</span></div>
<div class="MsoListParagraphCxSpLast" style="mso-list: l1 level1 lfo2; text-indent: -18.0pt;">
<span lang="EN-GB" style="mso-ansi-language: EN-GB; mso-ascii-font-family: Calibri; mso-bidi-font-family: Calibri; mso-fareast-font-family: Calibri; mso-hansi-font-family: Calibri;"><span style="mso-list: Ignore;">-<span style="font: 7.0pt "Times New Roman";">
</span></span></span><span lang="EN-GB" style="mso-ansi-language: EN-GB;">Test
Infrastructure part of the test process</span></div>
<div class="MsoNormal">
<h4>
<b><span lang="EN-GB" style="mso-ansi-language: EN-GB;"> </span></b></h4>
<h4>
<b><span lang="EN-GB" style="mso-ansi-language: EN-GB;">Conclusion</span></b></h4>
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">TMap is a great method for checking and offers a bunch of useful checklists and templates. Those are downloadable from <a href="http://www.tmap.net/">http://www.Tmap.net. </a></span><br />
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">BDTM is a good check Management method and offers a good base for a test management method. </span><br />
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">I would therefore extract BDTM out of TMap and re-brand TMap to CMap.</span><br />
<span lang="EN-GB" style="mso-ansi-language: EN-GB;">I would refer to for example </span><span style="mso-ansi-language: EN-US;"><a href="http://www.ministryoftesting.com/2012/07/mindmaptesting-and-checking/">http://www.ministryoftesting.com</a> or </span><span style="mso-ansi-language: EN-US;"><a href="http://www.developsense.com/blog/2009/08/testing-vs-checking/">http://www.developsense.com</a></span><span style="mso-ansi-language: EN-US;"> to get to know what you're missing out on if you only check and don't test.<a href="http://www.developsense.com/blog/2009/08/testing-vs-checking/"></a></span></div>
<div class="MsoNormal">
<br /></div>Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com0tag:blogger.com,1999:blog-2314019614094721290.post-89226446168757709932012-07-26T20:38:00.003+02:002012-07-26T20:38:34.679+02:00Why testers know best...Because testers ask questions when they don't understand<br />
Because testers think about what can go wrong and not how it should work<br />
Because testers are not dedicated to go live in time<br />
Because testers still want to go live in time more than finding bugs that don't matter<br />
Because testers love finding bugs that matter<br />
Because testers don't have much advantage in telling 'I told you so'<br />
Because in the end, if a problem is not fixed in time, the tester suffers from that so they are driven to find issues and defects early<br />
Because testers don't have 'babies' to protect and therefore are the least biased people in the project<br />
Because testers have a critical technical eye for detail and understand what business wants in the same time <br />
<br />
<br />Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com0tag:blogger.com,1999:blog-2314019614094721290.post-60516496115859570292012-07-24T20:56:00.001+02:002012-08-01T22:40:59.829+02:00Forming Storming Norming Performing<br />
We kicked off with a team of completely diverse people, starting a project that was going to be the biggest challenge most of us ever faced.<br />
<br />
The first months of the project where more difficult than any previous other project I had been working on. Finding out our places in the project, how we would fit to the puzzle, was for everyone one of the biggest challenges. At first conflicts were avoided, and specific tasks were not performed. Later on, some of us got caught up in quarrels and fights and after months of hard work, as a team, we contributed to late and poor quality delivery.<br />
<br />
<br />
After a couple of months of suffering, we learned on a team building event that this is so very normal. After that we found that by good leadership, team members are being carried trough different phases in a model of a team-building process.<br />
The model I'm talking about is the model of Tuckman and considers the Forming, Storming, Norming and Performing phases in team building. <br />
<br />
If you ever get a chance to look at your own team from a distance, you'll definately recognize this model and if you're manager of a leader of a newly created team, you definately need to know these things if you want to improve your team atmosphere and quality of delivery.<br />
<br />
<h2>
</h2>
<h3>
<a href="http://www.vadino.com/images.scaled/eJwlxksKgCAQANAbOZUQ0QFaRtAJJPxMpWPjgBsP38LVe0EkrwC1VlU-86C6KMIIG3HE5E_p7p3DsuuDcrG1qQQS5dE1rec2LUO7s2-ZKRMLUjLvD7qhJHo=.jpg" imageanchor="1" style="clear: right; float: right; margin-bottom: 1em; margin-left: 1em;"><img border="0" height="280" id="il_fi" src="http://www.vadino.com/images.scaled/eJwlxksKgCAQANAbOZUQ0QFaRtAJJPxMpWPjgBsP38LVe0EkrwC1VlU-86C6KMIIG3HE5E_p7p3DsuuDcrG1qQQS5dE1rec2LUO7s2-ZKRMLUjLvD7qhJHo=.jpg" style="padding-bottom: 8px; padding-right: 8px; padding-top: 8px;" width="311" /></a>Forming</h3>
The team is assembled. Team members don't know each other well and tend to act independently. There is no real group-feeling. Conflict is avoided. Team members have a need for guidance.<br />
<br />
<h3>
Storming</h3>
Team members start to take their positions in the team. This process often leads tentions between team members and contradictive ideas.<br />
<br />
<h3>
</h3>
<h3>
Norming</h3>
Team members are aware of the need of rules and methods. Those rules and methods are defined within the team. The common objectives become clear and the required roles are divided over the team members. The team with all team members take responsibility for the delivery of the objectives. The risk during the Norming phase is that the team loses either their creative spirit or the drive that brought them to
the Norming phase.<br />
<br />
<h3>
Performing</h3>
The team is working together harmoniously to achieve the common objectives. The team is working independently and makes collaborate decisions. Performing teams are identified by high levels of independence, motivation, knowledge and competence. <br />
<br />
<br />
<br />
Unfortunately, many teams never make it past the Storming phase. However I dear to say that we made it at least to the Norming phase as a team, yet we still have a lot to deliver... <br />
<br />Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com0tag:blogger.com,1999:blog-2314019614094721290.post-63357690384586119192012-07-16T08:35:00.001+02:002012-07-19T20:31:50.218+02:00Bug Fix BingoHow to dynamically turn frustration of testers into positive energy?<br />
<br />
<br />
<b>Bug Fix Bingo</b><br />
Developers seem to return with interesting reasons why a bug is not 'really' a bug.<br />
Testers on the other hand, need to understand that developers don't produce code but create art-work. <br />
<br />
Now, tester, when you come back from a soft-skill-battle with the developer where you finally, without hurting their feelings, were able to make them aware of this little mistake they made, you can return and have a little prize as a reward for your empathy and communication skills.<br />
In stead of getting frustrated and going back to your PC with a head-ache, you will be allowed to participate in the Bug Fix Bingo.<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<a href="http://blog-cdn.futtta.be/wp-content/uploads/2007/12/bugfixbingo.jpg" imageanchor="1" style="clear: left; float: left; margin-bottom: 1em; margin-right: 1em;"><img alt="bug fix bingo screenshot" border="0" height="423" src="http://blog-cdn.futtta.be/wp-content/uploads/2007/12/bugfixbingo.jpg" title="" width="600" /></a></div>
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
<br />
To the developers: Please don't take it too personal. This Bug Fix Bingo keeps your testers sane and in the end, those bugs really need fixing.<br />
<b> </b>The game was created by K. J. Ross & Associate ( http://www.kjross.com.au/page/Resources/Testing_Games/)<br />
<br />
<br />Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com2tag:blogger.com,1999:blog-2314019614094721290.post-87231858881179451292012-07-07T00:02:00.000+02:002012-07-11T13:53:38.676+02:00Let's get rid of the safety net<br />
I couldn't possibly say it better than Gojko did... but I can write about it.<br />
<br />
What Gojko came to tell us in the Eurostar conference of 2011 was that with software testing as we see it most - A phase of executing test cases after the product development phase has ended - does not contribute to quality.<br />
It contributes to building a safety net for developers, managers, and even testers.<br />
<br />
<br />
Since testing is done after development, developers get encouraged to become lazy as they don't need to make sure that their development still really works.<br />
<br />
Testing should <u><b>not</b></u> be about<br />
<ul>
<li>Logging defects that developers actually already know about. When for example simple client side field validations don't work, developers could know before a tester knows.</li>
<li>Waiting for a product to start testing and designing tests against a formalized analysis, to cover our asses instead of the product</li>
<li>Answering business on their requests instead of providing what they really need</li>
<li>Assume that all requirements and designs are clear and correctly defined</li>
</ul>
<br />
Testing <b><u>should</u></b> be about<br />
<ul>
<li>Helping analysts and developers by showing them how things can go wrong during design, development and delivery</li>
<li>Providing what business needs instead of what business wants</li>
<li>Help non-testers be better at testing</li>
<li>Test the complex and critical parts of a product </li>
<li>Find the real requirements and share them with all team-members</li>
<li>Being part of a team that is jointly responsible for the quality of the delivered product</li>
</ul>
<br />
<br />
<br />
<div id="__ss_10279850" style="width: 425px;">
<iframe allowfullscreen="" frameborder="0" height="355" marginheight="0" marginwidth="0" scrolling="no" src="http://www.slideshare.net/slideshow/embed_code/10279850" style="border-bottom: #ccc 1px solid; border-left: #ccc 1px solid; border-right: #ccc 1px solid; border-top: #ccc 1px solid;" width="425"></iframe> <br />
<div style="padding-bottom: 12px; padding-left: 0px; padding-right: 0px; padding-top: 5px;">
View more presentations from <a href="http://www.slideshare.net/gojkoadzic" target="_blank">gojkoadzic</a> </div>
</div>
<br />
<br />
<br />Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com0tag:blogger.com,1999:blog-2314019614094721290.post-80453522249834775682012-07-05T23:24:00.001+02:002012-07-06T13:43:05.761+02:00Beware of inattentional blindness while testing<br />
What's "Inattentional blindness"? and what does it have to do with testing?<br />
I got advised about inattentional blindness by a colleague tester, who got it from another colleague tester, who got it from another colleague...<br />
<br />
This turns "Inattentional blindness" into a Test-Myth!<br />
<br />
Before I say anything further about it, I would like you to watch the very short video below and carefully follow it's instructions.<br />
<br />
(Don't read below the movie yet, as it will contain spoilers)<br />
<br />
<br />
<div class="separator" style="clear: both; text-align: center;">
<iframe allowfullscreen='allowfullscreen' webkitallowfullscreen='webkitallowfullscreen' mozallowfullscreen='mozallowfullscreen' width='320' height='266' src='https://www.youtube.com/embed/Ahg6qcgoay4?feature=player_embedded' frameborder='0'></iframe></div>
<br />
<br />
Have you been surprised? There are different explanations for the reason behind it. (<a href="http://en.wikipedia.org/wiki/Inattentional_blindness">http://en.wikipedia.org/wiki/Inattentional_blindness</a> )<br />
<ul></ul>
<br />
<ul>
<li>Conspicuity - The ball draws your attention. </li>
<li>Mental Workload - You need to count so you focus on counting only </li>
<li>Expectation - You expect only passes to see and focus on the passes only and block other content </li>
<li>Capacity - You see this movie for the first time and you're not used to these exercises. You're not used to do this.</li>
</ul>
<ul></ul>
<br />
Now, what is in it for a tester? The key message is:<br />
<br />
When you focus, DON'T FORGET TO DE-FOCUS!<br />
<br />
<br />
This is what you do when you execute pre-scripted test cases:<br />
<ul></ul>
<ul>
<li>The test needs your attention - if you pass it, you better be sure about it! - Conspicuity </li>
<li>You need to focus on preparing the correct data and executing the correct steps - Mental Workload </li>
<li>You expect only a defect against what you are testing - Expectation </li>
<li>Especially during the first run, you're not yet used to go trough the procedural steps - Capacity</li>
</ul>
<ul></ul>
<br />
It's important to achieve a goal, to pay attention and to be focused on what you're doing.<br />
It is equally important to look around at the things you did not prepare test scripts for and find the bugs.<br />
<br />
<br />
How can we do this?<br />
<ul></ul>
<ul>
<li>If you feel the urge to prepare test scripts - don't detail them into descriptive test steps. Leave yourself some space to find alternative paths. Describe your paths on the moment of execution. </li>
<li>Per functionality, give yourself a time-boxed moment of time to look around in the application and find out about what you forgot about. </li>
<li>Put a post-it on your screen saying "de-focus" as you will forget from the moment you try to remember.</li>
</ul>
<ul></ul>
<br />Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com0tag:blogger.com,1999:blog-2314019614094721290.post-63872367606205959552012-06-29T17:08:00.003+02:002012-06-29T17:15:16.261+02:00Things you can do while waiting for a testable productA project manager gets most nervous on the moment a product (part) is about to be released to testing. Finally someone will tell them something about the quality of the product that will be delivered. In the time before the big moment arrives, they want you to carefully prepare, doing useful things like for example <br />
<ul>
<li>Prepare (preferably thousands of) detailed test scripts based on functional designs </li>
<li>Deliver a detailed (re)planning of all test activities over different phases and environments </li>
</ul>
<br />
There are however, other useful things you can do while waiting for a product to be released.<br />
Those might not be directly required by your PM, but you'll benefit from them later on the critical path. <br />
This list is not exhaustive as exhaustive lists would be exhausting. <br />
<div>
<ul>
<li>Talk to business users and analysts. Ask them about what they think is important, ask them how business works. Create your high level business End to End test scenario's in a workshop. Compare those End to End scenario's with functional designs and requirements and deliver your first gap analysis results. Explain your test approach to business users and analysts. The end to end flows can give you a nice way later on to report back to business on your test progress in a way they will all understand. I'll come back to this later on in a specific blogpost. </li>
<li>Talk to developers. Ask them about the programming issues and re-used components. This can save you some time later on. There is a reason why development will deliver late... Agree with them about the defect template and the procedure... that you want to keep as pragmatic as possible. </li>
<li>Read trough the analysis documents and identify high level test cases. Make note of inconsistencies and communicate about them on a regular base. These are very likely places where bugs can be found if not taken into account. It's perfectly sane behavior for a tester to make some commotion if you're not listened to during this moment in time. </li>
<li>Challenge the Architectural picture with what you learned from the business users and developers and read in the documentation. Make again note of inconsistencies and communicate them. </li>
<li>Create a high level overview of all test activities that need to take place. Agree with all parties that will take part in testing and find inter-dependencies over the available test environments. Find out which test activities can be run in parallel. Find out when, most likely, data resets will be required. Verify when other test activities in the environments are planned. Get to know your testing neighbors. </li>
<li>Define your reporting structure and agree with your PM and business. Use some visible examples. Make sure everyone understands that metrics are not the only thing they want to receive. Try to be creative and present visual, simple reports and arrange the possibility to have meeting to explain your status. </li>
<li>Train your BA's and other non-testers who will be joining in testing at some moment in time. Your mission is to make them become better testers. Tell them why writing all possible variations for a field validation test is an endless activity. A tester will have endless amounts of test cases to test one field but might only execute some for which the product will likely manifest bugs. You could tell them something about Boundary value analysis, data combination, paths, about focussing and defocussing, the tea-test and the shoe-test. </li>
<li>Make sure you obtain access to the test environment(s) with the correct user roles and assess if all infrastructural components, required to start testing, are in place. </li>
<li>Test the test data backup and restore process. If you have the possibility, make a proof of concept. </li>
<li>Prepare your assessment on entry criteria if required and give your development team a heads up on where they are in function of your entry criteria. In this way they won't be surprised when you need to give them a no-go to deploy in the test environment. </li>
<li>Test the deployment and release procedures. </li>
<li>Implement required test tools. </li>
<li>Reassure the PM, they are nervous and need some support.</li>
</ul>
<ul>
</ul>
</div>Mike Meurshttp://www.blogger.com/profile/15184386727383901043noreply@blogger.com3