•
Drizz raises $2.7M in seed funding •
•
Featured on Forbes
•
Drizz raises $2.7M in seed funding •
•
Featured on Forbes
The lack of budget is never your problem. The wrong QA model, however, is.
For your QA budget, there are 3 main options:
A lot of solutions combine these options, and still end up with a surprise break in production, with no explanation for why testing failed to identify it.
Each model accomplishes something unique.
Having QA in house provides product knowledge. Outsourced QA offers flexibility in both capacity and skill, as well as validation and broader coverage of services. Automation offers speed and consistency for stable and well-structured tests.
Each of these options approaches unique issues related to QA, including control, capacity, depth of testing, team costs, device coverage, cost of automation maintenance, and the demand for releases.
Automation does not disrupt QA. A suite of 500 regression tests is of limited value if the tests are unstable, poorly designed, or unable to locate issues in combination with screen and device specific issues. A device farm provides you with thousands of devices, but does not decide which tests are important. An outsourced team is able to perform many tasks, but may lack the depth of product knowledge of an embedded QA team.
The reality is simpler: Product critical release flows must stay in your control, as must the automation of checks. Outside QA must be used to bridge the gap between flexibility in capacity, independent validation, and tool coverage. A hybrid model is more likely to balance coverage of risks where one model dominates.
This article discusses in-house QA, outsourced QA, and mobile test automation in terms of cost, control, speed, expertise, device coverage, scalability, maintenance, and release risk. It also provides examples of how to implement mobile QA strategies, where the models fail, and where each model fits best.
Knowing the primary purpose of each model can help users determine the best fit:
Almost always, a mobile team wants a combination of all three models. It comes down to cost, frequency of releases, how complex the app is, which devices support the app, how the app is regulated, and how expensive a fault in production would be.
A team of in-house QA testers integrates with product management and development teams as well as support and customer service teams. Testers gain context around the product and the customer, and develop a sense of the app's user journeys, pain points, known failures, and customer expectations.
This context gives the testers the ability to make more informed decisions around the quality of the product, when a failure should be considered a blocker to performing a release, and the importance of a defect. A payment failure that impacts all customers is more critical than an onboarding failure that impacts a single user.
In-house QA is highly beneficial for:
In-house QA tends to increase fixed costs in the long run as employees require varying levels of investment of both time and money in order to be effectively managed. In addition, they also require specialized testing equipment that can be costly and difficult to manage when hiring needs increase/decrease.
A small in-house team can struggle to test major product releases that involve numerous operating system versions, offline behavior, payment integration and notification systems.
By outsourcing your QA, you gain access to test management, testers, devices, and domain knowledge, for which you only pay the agreed scope, service, or period of testing.
You should consider outsourcing QA when you require:
A team sourced externally can also provide a better view of your software due to lack of context regarding how software has been developed. This can help internal teams find unintended and critical bugs.
Outsourcing has its limitations. Among other things, external testers need clearly stated requirements, test data, access credentials, consistency in defect definition, and a product orientation. Without these, outsourced testing teams can report a plethora of low-value defects and can potentially miss high-value defects that link to risks covering the business's priority items.
Clarification of roles, as well as distance and time zones between testers and developers, impact the quality of the work product. A poorly defined developer-tester hand off can result in a tester receiving a test report that does not include clear direction on whether or not a release should occur.
The hallmarks of a good outsourced QA partnership include direct access to product and engineering, real-time visibility into the status and results of testing, clear definitions of severity levels, and short feedback cycles.
Mobile automation uses tools such as Appium, Espresso, XCUITest, and/or framework-specific solutions to run automated checks against an application. Automated tests work well for stable workflows with clearly anticipated outcomes. They're particularly well suited toward account establishment, payments and checkout workflows, and navigation. Automation of data validation, permission handling, and smoke tests after each build or prior to a release is common practice.
Automation minimizes the need to perform the same task repeatedly. Having a test suite available provides development teams with feedback prior to performing code changes that can impact a workflow.
Automation is an investment that requires ongoing efforts. The mobile platform becomes obsolete, application screens change, test data expires, and the behavior of third-party integrations also changes. A test failure can be indicative of a variety of issues including a product defect, a changed locator, an issue with the network, an invalid test account, or an issue with the test environment.
Costs associated with flaky tests quickly add up. If tests cannot be trusted, the team stops addressing failed builds. Time continues to be spent maintaining the suite, but confidence during a release decreases.
Automated tests lose context compared to human testers. These tools overlook issues regarding aesthetics, copy, layouts, animations, touch targets, various screen designs, and user interactions that fall outside the "normal" patterns.
“The cheapest hourly option” may not be the cheapest way to address the issue of QA testing. Don’t forget to consider the complete cost of all bug reporting, bug prevention, and bug management for each QA model.
Of all the models, what should we consider the cost of missing a defect? What is the cost to the company in terms of customer support, emergency releases, refunds, missed opportunities/goals, and strained relationships? A model with a high operating cost may be the most cost effective if the risk of failure is great.
When considering control, speed and coverage, one can see trade-offs.
For control, in-house QA is better for prioritization, gate release and product understanding. For work volume, outsourced QA is the better option.
The fastest coverage is achieved through automation, but there is an element of risk. Running automation across ten device profiles is not a substitute for mobile coverage. Differences in screens, OS, memory, manufacturer, permissions and battery states etc. affect mobile coverage.
A mobile coverage plan is defined by:
Informed priorities are given by: your analytics, support and usage tickets, crash reports and user demographics. Each of these will provide much better direction than a generic device list.
Choose in-house QA for situations where product context, quick collaboration, and ownership of releases are most critical. This option is well suited to products that have complex business logic, frequently change features, or have a high impact on the customer.
With outsourced QA, you can have more capacity, independent feedback, expert coverage, or a greater range of devices over a set time.
Choose automation when the same checks have to be done repeatedly, and the expected outcome is always the same, as long as the process is stable enough to maintain.
Adopt a hybrid model when your app has a stable regulation path and a high-risk area that requires human assessment. Most mobile teams get to this point as they grow. Internal testers make risk assessments and product decisions. Automation fills in the gaps for checks that are done repeatedly. External testers come in and provide increased coverage during important releases.
Drizz fills the gap when mobile teams want a better way to organize real device testing.
Drizz is useful when mobile teams need to scale real-device validation without taking the human element out of QA. It can automate repeatable regression and cross-device testing, while QA teams remain responsible for test strategy, exploratory testing, and areas that require human judgment.
"Internal risk ranking" refers to dividing workflow risks along the following dimensions: test automation (vs manual testing), devices, and customer impact.
Once this is done, each of these ranges can be delegated in a find-repeat manner. Automated smoke and regression tests can take care of validation.
Besides this, Drizz also aids in the testing of these workflows across mobile environments. Therefore, your team will have a wider range of evidence before a release.
This process makes clear what belongs to whom. The risk of a workflow is the concern of Product and QA teams. Automated tests take care of repetitive work. A mobile testing platform reduces the work needed to extend device coverage and coordinate runs and results.
This is far more impactful than the traditional approach of prioritizing a large number of tests because your team is able to understand the behavior of the workflow across the mobile devices that the customers employ.
List the highest risk user journeys.
Add payment, authentication, data sync, offline behavior, notifications, deep links, permissions as relevant.
Now, distribute each journey to some testing methods:
Check this plan after each major incident. If a defect escapes, determine the absence of the control. The answer may be a new automated test and/or device, exploratory testing, more extensive user stories, or the inclusion of QA earlier in the development.
The best QA strategy doesn’t give a single answer. It matches the right level of control with the appropriate testing method and owner, and it uses people for judgment and product understanding, automation for repeatability, and external support/mobile testing platforms when independence, capacity, or device coverage are the most important.
©Drizz 2026 • All Rights Reserved
