Monday, August 13, 2012

List of requirements for a new test environment

It was the first time we needed an integration test environment that we didn't have yet.
There was no infrastructure, no procedures, no version control, no data restore process,...
What we needed was a controlled test environment where we could test integration for a project of considerable size.

As a tester, I know that knowing what you want, is not obvious and putting it all together to make it work is even more difficult. So, a long brainstorm session with all stakeholders, some Google sessions and blog-scans came afterwards.
It lead to the following list of requirements that we used as a base, to start building our integration test environment.


Testability requirements

  • All applications that exist or are to be delivered in the production environment have to exist or simulated in the test environment. 
  • List of settings and configurations is available together with the comparison of the settings and configurations of the production environment. 
  • Required tools and licenses are available for testers (If for any reason tools can not be provided, a work-around is in place to facilitate testing)
  • Front end devices to access software through different channels are available (card readers, GPS Units, desk tops, laptops, pads, phones,...)
  • Batch procedures are internally controllable and automated as in a production environment (manual trigger, start, pause, stop batch procedures)
  • Measuring tools are in place to enable monitoring the communication between applications and application layers.
  • External Connectivity to test environments for support, deployment and testing is required (as different vendors need to deploy and test their software and integration on the environment)
  • Time traveling is possible in at least one of the following means: "time goes by faster (eg: 1 week = 1 hour)", "move forward and reset from a defined time in the future", "coherent data creation/update to a  time stamp in the past"
  • Input and/or output simulators are in place and documented for every interface type
  • All configurations of the different interfacing systems within the test environment have to be centrally managed 
  • At least 1 test user exists for every defined role in the application under test
  • Database read access is required for all testers
  • The test infrastructure has an availability of 95% (planned deployments are not included)
  • Defects are centrally managed, using 1 agreed defect managing system and flow 

Test data requirements

  • Any production data loaded into the test environment is scrambled in a way that it still fits logically together but is impossible to link back to actual clients in the production database
  • A representative set of  production(-like) test data can be unloaded in the test environment
  • Back-up and restore procedures exist to back-up and release a partial or full data extract
  • Back-up and restore procedure does not take longer than 4 working hours
  • Back-up and restore procedure should be possible at least on a weekly basis with a lead time of 1 working day
  • Full Data refresh with scrambled production data is possible on request  with a lead time of at least 1 working week. 
  • A procedure to request for a data is implemented
 

 Deployment/Release management requirements

  • Version control is applied to all development products and system documentation
  • Deployments are always done from 1 agreed central data store
  • Roll-back scenario's are in place for every deployment cycle
  • Users are trained and informed about deployment management agreements
  • A patch procedure is known, documented and implemented
  • Deployment tasks follow formalized checklists - verified by the test team
  • Builds or data refreshes in a test environment take only place after approval of the impacted test teams
  • A deployment/release management role is assigned
  • Every defect fixed for a new deployment in the test environment is at least communicated within an agreed defect management tool 
  • Every deployed, fixed component is at least communicated trough a build note
  • Builds are subject to formal and/or informal entry criteria

Maintainability requirements

  • Procedures to add/delete or update hardware components are existing and followed
  • The test environment components are sized in relation to the production environment
  • Log files are available for the different applications deployed on the test environment
  • Procedure exists to request and add test users for any application in the test environment
  • Disaster recovery plan is known, documented and implemented
  • Unexpected downtime of the (partial) test environment is communicated to all stakeholders (preferably using a central communication channel - for instance an intranet website)
  • Physical locations of different infrastructural components are identical to the production environment and documented. Any deviation is clearly identified and approved by the project team 



I definitely forgot some requirements and not all of them are actually in place. But but based on these we set up an environment we can control with a handful of people and test in properly. We are agile enough to patch and refresh data properly which improved the test experience drastically.
Any suggestions to improve this list are welcome.

28 comments:

  1. This is Great and very useful advice with in this post. Thank you.
    c,c++training in chennai

    ReplyDelete
  2. Thanks for sharing such a great information..Its really nice and informative..
    R training in chennai

    ReplyDelete
  3. Greens Technology's the leading software Training & placement centre Chennai & ( Adyar)
    SCM training in chennai

    ReplyDelete
  4. Thank you a lot for providing individuals with a very spectacular possibility to read critical reviews from this site.


    selenium training in bangalore

    ReplyDelete
  5. Thank you a lot for providing individuals with a very spectacular possibility to read critical reviews from this site.


    selenium training in bangalore

    ReplyDelete
  6. Ciitnoida provides Core and java training institute in noida. We have a team of experienced Java professionals who help our students learn Java with the help of Live Base Projects. The object-oriented, java training in noida , class-based build of Java has made it one of most popular programming languages and the demand of professionals with certification in Advance Java training is at an all-time high not just in India but foreign countries too.

    By helping our students understand the fundamentals and Advance concepts of Java, we prepare them for a successful programming career. With over 13 years of sound experience, we have successfully trained hundreds of students in Noida and have been able to turn ourselves into an institute for best Java training in Noida.

    java training institute in noida
    java training in noida
    best java training institute in noida
    java coaching in noida
    java institute in noida

    ReplyDelete
  7. BCA Colleges in Noida

    CIIT Noida provides Sofracle Specialized B Tech colleges in Noida based on current industry standards that helps students to secure placements in their dream jobs at MNCs. CIIT provides Best B.Tech Training in Noida. It is one of the most trusted B.Tech course training institutes in Noida offering hands on practical knowledge and complete job assistance with basic as well as advanced B.Tech classes. CIITN is the best B.Tech college in Noida, greater noida, ghaziabad, delhi, gurgaon regoin .

    At CIIT’s well-equipped Sofracle Specialized M Tech colleges in Noida aspirants learn the skills for designing, analysis, manufacturing, research, sales, management, consulting and many more. At CIIT B.Tech student will do practical on real time projects along with the job placement and training. CIIT Sofracle Specialized M.Tech Classes in Noida has been designed as per latest IT industry trends and keeping in mind the advanced B.Tech course content and syllabus based on the professional requirement of the student; helping them to get placement in Multinational companies (MNCs) and achieve their career goals.

    MCA colleges in Noida we have high tech infrastructure and lab facilities and the options of choosing multiple job oriented courses after 12th at Noida Location. CIIT in Noida prepares thousands of engineers at reasonable B.Tech course fees keeping in mind training and B.Tech course duration and subjects requirement of each attendee.

    Engineering College in Noida"

    ReplyDelete
  8. Wonderful thanks for sharing an amazing idea. keep it...

    Get Web Methods Training in Bangalore from Real Time Industry Experts with 100% Placement Assistance in MNC Companies. Book your Free Demo with Softgen Infotech.

    ReplyDelete
  9. Thanks for one marvelous posting! I enjoyed reading it; you are a great author. I will make sure to bookmark your blog and may come back someday. I want to encourage that you continue your great posts.

    sap hybris training in bangalore

    sap scm training in bangalore

    sap pm training in bangalore

    sap crm training in bangalore

    sap ewm training in bangalore

    ReplyDelete
  10. Simple Test setup. Hope this works well. Thank you for posting it.

    data science institute in indore

    ReplyDelete
  11. Very interesting to read this article.I would like to thank you for the efforts you had made for writing this awesome article. oracle training in chennai

    ReplyDelete
  12. Im really impressed by it.I am happy for sharing on this blog its awesome blog I really impressed. thanks for sharing. Great efforts. data science training

    ReplyDelete
  13. Really Nice Information It's Very Helpful All courses Checkout Here.
    Sap Apo training in bangalore


    ReplyDelete
  14. Hızlı takipçi almak için takipçi satın al
    Organik takipçi almak için takipçi satın al
    Bilgisayardan takipçi almak için takipçi satın al
    Mobil cihazdan takipçi almak için takipçi satın al
    Gerçek ve orijinal takipçi almak için takipçi satın al
    Yazarkasa ile takipçi almak için takipçi satın al
    Bitcoin takipçi almak için takipçi satın al
    Pos ile takipçi almak için takipçi satın al
    EFT ile takipçi almak için takipçi satın al
    Havale ile takipçi almak için takipçi satın al
    Mobil ödeme ile takipçi almak için takipçi satın al
    Tamamı orijinal takipçi almak için takipçi satın al
    Organik ile takipçi almak için takipçi satın al
    Türkiye takipçi almak için takipçi satın al
    Global takipçi almak için takipçi satın al
    En hızlı instagram takipçi satın al
    En uygun instagram takipçi satın al
    En telafili instagram takipçi satın al
    En gerçek spotify takipçi satın al
    En ucuz instagram takipçi satın al
    En otomatik instagram takipçi satın al
    En sistematik tiktok takipçi satın al
    En otantik instagram takipçi satın al
    En opsiyonel instagram takipçi satın al
    En güçlü instagram takipçi satın al
    En kuvvetli instagram takipçi satın al
    En seri instagram takipçi satın al
    En akıcı instagram takipçi satın al
    En akıcı instagram takipçi satın al
    En akıcı instagram takipçi satın al
    En akıcı instagram takip etmeyenler vrfv

    ReplyDelete