For a delivery app, faster delivery is part of the business model. 

New restaurants, deals, payment options, checkout flows and experiments reach users within a week. The product must evolve with the needs of the customer and the market.

Each release has its own workload for the test suite. Test suite maintenance builds over time after a release, whereas work on a feature ends after a release.

Maintaining a mobile test suite of 200 tests with a delivery app that is rapidly changing, can cost 200+ engineering hours each year to simply maintain the existing automation. In fact, it matches the workload of a full-time QA engineer by 500 tests.

So there is a real cost of test automation. Larger test suite along with faster product change cycle creates high maintenance costs.

Delivery apps require frequent updates to the UI, implement personalization for each user, handle the complexity of the checkout and payment flows and require constant experiments with weekly releases. So it is not a decision of test coverage alone. You need to measure the engineering time required to keep the test coverage working.

If your test suite breaks on every iteration to the product, it is already costing you a lot in engineering hours. Therefore, you need a better approach.

What Test Suite Maintenance Includes

Any work required to maintain the reliability of automated tests after modifications to the product or environment is covered under test suite maintenance.

Some examples are:

• Modifying resource IDs, accessibility labels, XPath expressions, or other selectors after a screen redesign.
• Modifying waits after changes in network/animation behavior.
• Fixing tests after changes in the operating system or device image.
• Diagnosing test failures due to test data, APIs, or test environments.
• Removing obsolete tests.
• Re-recording flows after a new checkout or change in onboarding experience.

These tasks don't add new application coverage. They just maintain the coverage you've already paid to build.

In fact, a test suite creates maintenance work outside the test code too. Engineers have to review CI logs, reproduce failures, identify whether a failure is due to the app or the environment, and confirm the same across devices.

Estimating the Annual Cost of Test Maintenance

The cost is dependent on the app, test framework, device matrix, and team structure. However, a simple estimate allows the teams to understand how the cost of test maintenance scales.

For a selector-based mobile suite running against a product with weekly UI changes, a 200-test suite might require 16 to 28 hours of maintenance each week. Over 52 weeks, this equals roughly 832 to 1,456 hours per year. At an engineer’s fully loaded cost of $60 per hour, this annual maintenance cost would amount to $49,920 to $87,360.

A 500-test suite would require 2,080 to 3,640 hours per year, and amount to a cost of $124,800 to $218,400 for the year. These figures exclude things like new test creation, exploratory testing, defect investigation, infrastructure tests and release validation work. This is the cost of keeping existing test automation operational.

To build your own estimate, you should use data from the last 4 to 6 release cycles. 

The AMC formula is simple:

Annual maintenance cost = Maintenance hours per release x Releases per year x Loaded hourly cost of engineers

It is important that you track time spent on repairing existing tests, writing new tests and investigating product defects separately. A team that spends 60 hours on test automation might have spent 45 hours repairing failed tests and only 15 hours on test improvements.

How Much Time Does Test Maintenance Actually Take?

For teams shipping weekly UI updates with selector-based automation tools (Appium, Espresso, XCUITest, Detox, Maestro), test maintenance consumes the following proportion of QA engineering time:

These numbers represent maintenance only. Not test creation, not exploratory testing, not bug investigation. At 500 tests, one to two full-time engineers spend their entire working year keeping existing tests alive.

Why Delivery Apps Experience More Breakages

Delivery apps bring together a number of complexities in one integrated product.

A customer journey includes taking location permissions, searching for restaurants and personalizing order menu options, and post-order options such as payment selection, order status tracking and requesting a refund.

Each step depends on changing data, network calls, third-party services, and different user states.

The app serves more than one user group. Customer, restaurant, courier and support workflows access different interfaces of the same backend services. A change to the pricing, promotions, or order status can affect multiple user journeys.

Additionally, doing personalization in the app may mean different users see different offers, delivery times, etc. A test that assumes a consistent screen layout can become fragile when the product introduces variation based on the location, account, time or experiment group.

The most common reasons for failure can be classified into 4 broad categories. 

These are:

1. Selector Drift (50-60% of All Breakages)

Selector Drift occurs when a developer alters an internal identifier during a redesign or refactor. The button is still functional for users, but the test no longer finds it. A single identifier change can break many tests.

Let’s say a developer changes a checkout button name from checkout_btn to purchase_button. The button is still functional and is still tap-able and carries out the same action. Tests which use the old ID fail for every flow which even references it.

// checkout_fragment.xml
- com.app:id/checkout_btn
+ com.app:id/purchase_button

// The button still exists.
// It still says "Place Order".
// It still works perfectly.

2. Timing and Synchronisation Failures (20-25%)

Timing failures occur when tests interact with the app before the app reaches the desired state. Network delays, animations, loading indicators, and navigation transitions create inconsistent results.

Fixed waits solve the problem by hiding the real issue and make the suite slow. A wait time of 3 seconds may be too short on a slow CI device and too long on a device where the app is immediately responsive.

3. Environment and Infrastructure Changes (10-15%)

Environment failures happen due to changes in the underlying system or the infrastructure including operating systems, emulators, device farms, CI runners, APIs and even test data. These failures can occur even without a product change. 

New versions of Android and iOS, or a new API can cause existing tests to fail. Engineers have the responsibility to determine whether the failure attributes to a bug in the product, a bug in the test or a failure because of the infrastructure.

4. Intentional Product Changes (10-15%)

Intentional product changes require corresponding changes to the test. For example, if the product provides additional payment methods, new checkout steps, modified copy or provides new design screens, the test should be updated accordingly.

A failure on the first 3 categories creates noise, whereas the 4th actually indicates real product advancement. Differentiating between these cases is important because they require different responses by the team.

What Is the Per-Test Lifetime Cost?

A single automated test case has a predictable cost lifecycle:

Creation cost: 2-4 hours (identify elements, write script, configure waits, validate across devices).

Monthly maintenance cost: 30-60 minutes per sprint (averaged across all tests  some sprints require zero maintenance for a given test, some require hours after a UI change).

Annual maintenance cost: 13-26 hours per test per year (at bi-weekly sprint cadence).

Lifetime maintenance cost: 4-8x the original creation cost over 12 months.

A test that took 3 hours to create accumulates 13-26 hours of maintenance over a year. The test you're proudest of writing becomes the test that quietly consumes your team's capacity for the next 12 months.

The Opportunity Cost of Unstable Automation

Maintenance isn't just about the automation budget. It impacts how QA teams spend their time. When engineers spend the majority of a sprint fixing automated tests, new features get less coverage.

New flows are the least tested due to the remaining capacity being used by the existing automated tests. A failing test also decreases the confidence in the entire pipeline.

Engineers begin ignoring the failing tests, marking the unstable tests as non-blocking, and even run tests in the background. Eventually, the suite reports having hundreds of tests, but fewer of those results actually support release decisions.

This is especially risky in checkout and payment flows. These flows receive many changes and have a direct revenue impact. If tests in these flows stay failing, the product team loses automated feedback when it is most needed.

Flaky tests also create the same problem. When engineers have to decide if a failure is a defect, the test suite isn't a valuable resource anymore.

Practices That Reduce Test Maintenance

Reducing the maintenance of tests is still the best practice. It helps contain the cost of failures and the underlying causes.

1. Use Stable Accessibility Identifiers

Use accessibility identifiers for important interactive elements. Avoid relying on XPaths and fragile positions in the view hierarchy.

2. Centralise Selectors

Keep a single selector in a page object or screen object. Centralizing selectors decreases the number of updates engineers have to make.

3. Replace Fixed Waits with Conditions

Instead of using fixed sleeps, use condition based synchronisation. Wait for a meaningful state. Eg. a button becoming enabled, a loading indicator disappearing, or a network response completing etc.

4. Keep Tests Focused

Keep tests shorter as they are easier to diagnose and less expensive to repair.

5. Improve Test Data Reliability

Ensure that you create reliable test data and isolate external services when needed. A payment test should not fail because a third-party sandbox is unavailable.

6. Remove Obsolete Tests

A growing suite with redundant cases provides less value than a smaller suit covering distinct risks.

The above steps can help improve maintainability of the test-suite but they do not remove coupling between the test and the app's internal structure. Any identifier change or a component migration can still break tests even when nothing has changed visibly on the screen or from a product behaviour standpoint.

Evaluating Visual and AI-Assisted Testing

You can reduce selector-related maintenance by using visual or AI-assisted element recognition. Instead of figuring out a button through an internal id, the test identifies it through visible text, appearance and screen context.

Drizz is a vision AI based mobile testing platform. You can simply write the tests in plain english and Drizz figures out the steps to do it and uses vision AI to interact with elements instead of relying on resource ids, XPath or accessibility labels. Even if a developer changes an internal identifier, the visible experience remains the same, so the test requires virtually no maintenance work.

For a delivery app that requires frequent UI updates, redesigns and refactors, this is useful. Be in restaurant cards, search results, promotional banners, checkout buttons, and payment flows - Drizz can handle the tests as long as you can describe it like “tap Place Order”. This helps teams spend less time repairing selector failures and spend more time expanding test coverage, building and testing new features and investigating deeper product defects. To see this in action, watch Drizz testing the Licious app  - a complete order flow on India's leading D2C meat delivery platform, automated in plain English without a single selector.

However, visual testing still needs careful evaluation. Teams should check how the system handles different screen sizes, themes, languages, accessibility settings, dynamic content, and ambiguous elements.

A practical pilot test would involve selecting 10 to 20 tests with the highest repair work. Run the existing selector tests and then run the tests through Drizz in parallel for several release cycles. Measure:

• Maintenance hours
• Failure rate
• False failure rate
• Time to diagnose a failure
• Time required to update a test after a product change
• Coverage of critical journeys
• CI execution time

You’ll realize that Drizz reduces the maintenance burden caused by selector drift without reducing the quality of the test results. Such a pilot would be perfect for you to see and measure the meaningful savings it makes for your app, devices, and release process. 

The Maintenance Comparison

The 0.4-0.7 FTEs reclaimed from maintenance at 200 tests redirect into writing new tests, expanding coverage, exploratory testing, and strategic QA work  the activities that actually prevent defects from reaching users.

How to Run a Test Maintenance Audit

1. Review the last 4 weeks of CI failures and classify each failure as being related to selector drift, timing, environment, product change, test data, or as a real defect.
   
   
2. Prioritize tests based on the total estimated time to fix. Often, the highest value targets are not the oldest tests. Rather they are the frequently executed tests that cover critical, high value paths, and break after changes to common elements.
   
   
3. Compare the total time spent on maintenance to the time spent on new value addition. If the total maintenance time exceeds 40% of the total QA time available, the team must consider this as a structural issue. Increasing the headcount will not reduce the cost to fix each test.

Finally, set a maintenance budget for each release. A suite should not grow without concrete plans for ownership, cleanup, failure triage and retirement. Trying to build the largest possible test suite is not an optimal choice.

If 10 Drizz tests require zero maintenance hours while 10 Appium tests require 8+ hours of fixes over 4 sprints, the math speaks for itself.

The goal isn’t simply to have more tests. It’s to have a test suite that keeps delivering value as the app changes. When tests can keep up with UI changes, run reliably across devices, and require less ongoing maintenance, teams can expand coverage without expanding the cost of maintaining it. That’s the difference between having test coverage and having usable coverage and it’s where Drizz helps teams turn mobile test automation into a dependable safety net for every release.

Frequently Asked Questions

What is the average cost of maintaining a mobile test suite?

For a 200-test suite using selector-based tools (Appium, Espresso, XCUITest) at a company shipping weekly, annual maintenance costs range from $49,920 to $87,360 in QA engineering time (832-1,456 hours at $60/hour). This covers selector updates, timing fixes, environment debugging, and re-recording after UI changes.

Why does test maintenance increase with release frequency?

Each release that modifies UI elements creates potential selector breakages across every test touching modified screens. A monthly release cadence means 12 potential breakage events per year. A weekly cadence means 52. The maintenance burden scales roughly proportionally with release frequency because each release is an independent source of breakages.

Can test maintenance be eliminated entirely?

Maintenance from intentional UI changes (new flows, changed copy, added steps) always requires test updates regardless of the testing approach. This accounts for 10-15% of total maintenance. The remaining 85-90% (selector drift, timing, environment) can be eliminated or dramatically reduced by removing the coupling between tests and internal element identifiers. Drizz's Vision AI testing achieves this by identifying elements visually rather than through selectors.

How does Vision AI testing reduce maintenance costs?

Drizz's Vision AI identifies screen elements visually by their appearance, text, and position on the rendered screen rather than through internal identifiers… When developers change internal identifiers during refactoring, the visual appearance remains the same, so tests continue passing. This eliminates selector drift (50-60% of all maintenance) and reduces synchronization issues by evaluating the rendered screen state rather than element tree state.

‍At what team size does maintenance become unsustainable?

For a 3-person QA team, maintenance typically becomes unsustainable at 150-200 tests. At that point, 40-70% of total QA capacity goes to maintenance, leaving insufficient time for new test creation, coverage expansion, or exploratory testing. The team enters a maintenance trap where coverage plateaus and bugs ship in the newest, most-changed flows that lack test coverage.

Is migrating from Appium to Vision AI an all-or-nothing decision?

No. Most teams migrate incrementally by rewriting their highest-maintenance tests first (typically 10-20 tests) and running them in parallel with the existing suite. This provides a direct maintenance cost comparison without risking existing coverage. Full migration happens over weeks or months as teams validate the approach on progressively more flows.