Software Testing Outsourcing Guide
If a company is convinced that its skilled and abundant testing resources are ready to meet any deadline and cover a project of any scope, that it has access to the best testing tools and practices, then this company must be unrealistic. The reality is companies often face the inability to cover the whole scope of their testing needs.
Here’s where software QA and testing outsourcing comes into play. If you manage to choose a reliable QA outsourcing vendor, you will get a team of an optimal number and high competency as well as meet your project deadlines, improve the general QA maturity level, and easily reach your quality management objectives. In this article, we are ready to share our QA outsourcing success formula.
- QA outsourcing roadmap
- Getting ready for QA outsourcing
- Choosing the vendor
- Vendor measurement and management
- QA outsourcing problems' resolution
The road to accessible and fruitful QA outsourcing may be long and bumpy, but following this plan, you may avoid most of the obstacles. Here’s the roadmap we offer:
- Prepare your staff and workflows.
- Streamline workflows.
- Assign a QA professional to manage your future vendor.
- Ensure development and testing alignment.
- Choose the right software testing vendor.
- Consider reliability and expertise, service and resource capacity, and self-management.
- Perform ongoing vendor assessment and management.
- Resolve possible QA outsourcing problems.
Let’s see what stands behind each milestone.
Before looking for a perfect match among QA outsourcing vendors, make sure to be ready for future cooperation.
Streamline your workflows
The major touchpoints between you and your future QA outsourcing provider during the testing process are normally software requirements specification, existing and updated code, and found and fixed bugs. Before starting to work with any vendor, you should clear up the following:
- Where the needed information on each of these three points is stored.
- How often such information is updated.
- What relevant tools your developers use (development environment, source code management and continuous integration software, etc.).
- What bug reporting procedure you find convenient.
Ideal development processes and their full compliance with QA aspects are not possible in the real world and are not required. Still, the more information you provide to a testing outsourcing vendor, the less time the initial partnership formalization will take. In other words, the less expensive your project quality ramp-up will get.
Assign a senior-level QA manager
You need in-house management (at least one senior-level QA professional to manage a team of 20 to 50 outsourced testers) to cooperate with a vendor more effectively by:
- Reviewing their test strategy, test plans, test cases, bugs, and test reports.
- Organizing updates of software requirements.
- Assessing and managing vendor performance and providing relevant feedback.
- Making balanced judgments in conflict situations and, ideally, defusing them at the earliest.
- Considering risks thoroughly and preparing a mitigation plan.
Motivate your development team to cooperate with an external test team
Development and testing alignment is one of the most crucial problems in software testing outsourcing. Quite often, external test teams have to handle the reluctance of development teams to communicate openly and freely. The lack of inter-team communication may result in poor understanding of software requirements, late changes of a test plan, unclear bug reports and, consequently, missed iteration goals and deadlines. So, the assigned QA manager needs to persuade the stakeholders that the only way to create a smoothly operating solution implies cooperation and mutual understanding.
Here are some key points to look for while choosing a vendor.
Reliability and expertise
To choose a reliable vendor, it’s a good practice to pay attention to vendors’ references and success stories. A good vendor respects test documentation (a general test strategy and a plan, test scenarios, cases, and reports) and gives you easy access to their test artefacts templates. A vendor’s competence can be proved by QA software testing certificates (International Software Testing Qualifications Board (ISTQB) or Quality Assurance Institute (QAI) of outsourced test team members. In addition, a trustworthy company is to work by data security regulations and standards (HIPAA, PCI DSS), quality and testing standards (ISO9001:2015, ISO/IEC/IEEE 29119).
Sufficient resources and services
While choosing a vendor, mind that an outsourcing partner should have at least 5 times more testers than you need at the moment. Your software testing vendor should be able to perform all testing types (automated or manual; web, mobile or desktop; functional, performance, usability, compatibility, API, security testing, etc.). Even if there is no need in some of them now, when it comes to scaling up, your vendor will easily ramp up and provide a wider testing scope.
The main idea of QA outsourcing is to assign the time-consuming and scrupulous testing process, QA issue management, and its improvement to a qualified external team and focus on high-level quality management, the code creation, software design and integration. So, a vendor is to be self-managed to cover everything coming within efficient QA process: from the assistance with a requirements specification development to continuous testing during the whole application lifecycle. Again, a vendor’s success stories and test artifacts templates speak better for this QA outsourcing aspect. In addition, there should be an assigned QA manager on the vendor’s side too, so that you can address all related questions to a responsible person.
According to the current statistics, less than a half (39%) of world leading IT companies feel that they are effective in getting all the benefits of managing their outsourcing relations. So, here are some tips to help you to manage the chosen vendors effectively.
Understand your vendor
The core of vendor management lies in understanding what drives your vendors. Like all market players, a QA vendor experiences constant pressure: operational costs, talent acquisition challenges, problems with other projects. Thus, you should consider that excessive demands for price discounts may result in insufficient resource allocation to your project as a vendor may try to reduce the price by cutting their actual expenses (assigning insufficient or junior staff).
Another thing to mind is that you shouldn’t dread to share the information with your testing vendor (within your NDA). To render high-quality testing services, an outsourced team needs to understand your software, business objectives, specific technical requirements, and user expectations.
Regularly assess the vendor
Assessing a QA outsourcing vendor systematically helps to ensure that your company gets exactly what you’ve aimed at. Thus, if the quality of service provided by your vendor doesn’t live up to your expectations, you can promptly start searching for another one or find possibilities to enhance your existing partnership.
Your testing vendor evaluation is to include the most relevant performance and billing criteria. This will give you a multidimensional overview of how effective the vendor is. Here is the sample list of assessment criteria:
- The number of missed bugs.
- The quality of defect description.
- The correlation between testing efforts and outcomes.
- The quality of test documentation.
- The capacity and availability of resources.
- The efficiency of testing tools.
Your QA manager needs to extend this set of KPIs with more specific ones relevant to your software testing projects and potential problem points.
Manage vendor performance
For your QA outsourcing vendor to continue rendering their services at the same level or step up, your dedicated QA manager has to supervise their performance. A QA manager needs to review testing documentation at least once a month. They also have to provide the test team with regular feedback, detect hidden wasteful steps and cost drivers, align your business practices with the vendor’s work processes and uncover the root causes of QA process complications.
To meet your actual QA outsourcing concerns, vendor management should be based on continuous communication between your QA professional and the vendor’s dedicated manager. For example, if a lot of bugs are missed or unclearly reported, your QA professional should provide the vendor with the relevant feedback for them to eliminate the real reason for these faults. The most common cause of missed bugs and unclear bug reports is business requirements and software requirements specification misunderstood by the test team. So, to be sure the outsourced test team does understand the intended software functions and business requirements, the QA manager can organize ad hoc interviews among them.
Consider multivendor strategy
Vendor performance assessment results may turn out to be alarming: your only vendor’s testing quality doesn’t meet your expectations, they allocate incompetent test staff, use outdated bug reporting tools, their capacity is insufficient to meet your needs or the test documentation is poor. Getting locked in with such a vendor puts the entire project or even a number of projects at risk. Thus, it may be relevant to consider a multivendor strategy. This strategy implies involving a number of vendors and making each of them responsible for a separate project or a project part. This makes vendor replacement much easier, but the strategy is not a universal lifesaver.
The multivendor strategy requires a certain project and business scale. A project has to comprise at least 30 outsourced testers to be worth the time spent on the search and management of several vendors.
Another thing to mind is that one in-house QA manager is not enough to manage multiple vendors. The dedicated person will have to deal with numerous test strategies, plans, bugs, reports, requirements as well as different amendment procedures. So, you will need a number of in-house QA professionals able to manage vendors separately and a head of the in-house team to coordinate them and make sure all test teams work in sync.
Even after you’ve got the processes ready for QA outsourcing, chosen a reliable vendor or a number of them, and your QA manager assesses and manages their performance regularly, some cooperation issues still may remain unaddressed. Often such challenges are driven by the Agile methodology of software development.
Unexpected software requirements changes
The most typical stumbling block with outsourced QA in Agile projects is that software requirements are regularly updated and changed. Imagine your vendor has prepared a comprehensive and detailed iteration test plan, test environment, test cases, and it turns out that everything should be done again from scratch, as software requirements were revised.
For your outsourced testing vendor to quickly adapt to changes and not to carry out testing blindfolded, the communication between your in-house project and external test teams should be seamless. Such flexibility can be reached with the help of conventional project management and continuous integration tools. Additionally, your dedicated QA manager should organize daily goals’ setting in a common chat and weekly video conferences for actualizing and synchronizing development and testing efforts.
The loss of testing focus
During a complex software development project, an outsourced QA vendor may get perplexed with constantly changing priorities and occasionally leave the most problematic and critical software features out of the testing scope.
To deal with this issue, your QA manager should assist outsourced test teams with the development of a clear test plan for each iteration. Besides, risk-based and exploratory testing can help to prioritize testing activities so that they will cover business critical and flawed software features in the first place.
A superficial contract with a QA vendor
A QA outsourcing contract should serve as a legal foundation both for the whole cooperation with a software testing vendor and for each stage of testing services’ delivery. It’s critical for your vendor performance management to be based on a clear set of testing metrics and milestones. However, a typical QA outsourcing contract lasts from 5 to 10 years. So, your QA manager may find it complicated to make a constantly changing Agile testing process meet general QA outsourcing contract conditions.
This way, during long-lasting QA outsourcing, it’s handy to include several Service Level Agreements (SLAs) into your contract to divide the whole collaboration process into manageable parts. Each SLA should define the subset of services to be rendered, the time limits for their execution, the priority for each testing type, and clear testing KPIs. This way, SLAs help to formalize even highly flexible Agile testing.
Software testing outsourcing is not a tricky process if you know what to expect from it. The whole scope of testing outsourcing can be put as simple as that: preparation, making a prudent choice, regular assessment and management, coping with common challenges of outsourced testing. This is to be your success formula spiced up with an appropriate risk mitigation plan and an opportunity to use the multivendor strategy.