Does self-healing hide real bugs?

Well, it can definitely do that when I "self-healing" becomes a a retry loop or a silent selector swap where the developers or testers have no record or context of what changed. However, It shouldn't, and doesn't, when healing is confined to how a step finds an element and never touches what the test asserts. A failed validation must always fail the test. That single design line separates self-healing from self-deception.

If you've been burned by retry-until-green, your skepticism is earned. Let's take the question seriously instead of waving it away.

Where the fear comes from

Early "self-healing" deserved its reputation. Some tools implemented it as automatic retries. All it did was run the step three times, report the pass, mention the two failures nowhere. Others silently loosened matchers: the exact selector died, so the tool grabbed the nearest similar element and moved on. And a few would quietly skip or weaken steps that kept failing.

All three produce the same outcome: a green dashboard and a bug in production. Your users become the assertion.

So the question isn't paranoid. It's the right question, you're really asking what exactly is my tool allowed to change without telling me? The answer varies more than vendors admit, we've broken down how every tool actually implements self-healing, but you can audit any of them with what follows.

The line that matters: locating vs. asserting

Every test step does two jobs. It finds something on the screen, and it verifies something about the app's behavior.

• Finding is mechanical. It is easy for a tool to understand that "Continue to Payment" moved 40 pixels down, got a new colour, or its underlying identifier changed in a refactor. The button still exists and still works. A test that fails here isn't reporting a bug, it's reporting its own brittleness.
• Verifying is the actual test. Understanding that the order total is correct. The confirmation screen appears. The error message shows when it should.

Legitimate self-healing operates only on the first job. It repairs the finding, but no the verifying. If a validation fails because of wrong total, missing screen, changed state, that is a real failure, and it must surface as one, loudly, with evidence.

A tool that "heals" a failing assertion isn't healing anything. It's deleting your test while leaving the row in your dashboard.

How Drizz draws the line

Drizz approaches the problem from a different starting point: its Vision AI reads the screen the way a person does, no selectors, no XPaths, no accessibility-tree lookups. That means the most common category of "broken test" (identifier changed, layout shifted, element restyled) mostly doesn't need healing at all, because there was never a selector to break. [screenshot: same step passing before/after a UI refresh]

Where behavior genuinely changes, the roles stay separate. The Self-Healer applies to locating and interacting; validations are never auto-modified, a failed check fails the run. And instead of a bare red X, Drizz's failure reasoning tells you why it failed: wrong screen, missing element, changed copy, with a full video recording of the run as evidence. You review the diagnosis; you decide whether it's UI drift or a real regression.

The honest version of "zero script maintenance" is not "nothing ever fails." It's "the only failures you see are real ones." That's the contract codeless test automation has to honor before it belongs in a release pipeline.

Three signs your tool is hiding bugs

Auditing this takes ten minutes, whatever platform you run:

1. No heal log. If you can't see exactly which steps were healed, when, and what changed, you have no way to distinguish healing from hiding. Every heal should be inspectable after the run.
2. Pass rates that jump after a UI change. Ship a redesign and watch the suite. If everything stays green with zero healed-step entries to review, the tool didn't adapt, it stopped looking.
3. Assertions that get "updated" without a human. Expected values, validation text, success criteria: if any of these can change without someone approving the change, your suite is verifying whatever the app currently does, which is the same as verifying nothing.

When a healed test should still fail

Worth stating plainly, because this is the trust contract:

• The element is genuinely gone → fail.
• The flow changed (a new screen appears mid-checkout) → fail, with reasoning that says so.
• The validation text or expected state changed → fail, until a human confirms the new expected behavior.

Self-healing that respects those three rules removes noise. Self-healing that overrides them removes coverage.

FAQs

What's the difference between self-healing and a retry?

‍A retry runs the same step again and hopes. A heal changes how the step locates its target, records what it changed, and still subjects the result to the original assertion. If a tool can't show you the record, assume it's retrying.

Should healed steps be reviewed?

Skim them, yes, especially after releases. A cluster of heals on one flow is an early warning that the flow changed more than cosmetically. The review takes minutes; it's the audit trail retries never gave you.

Can self-healing fix a failing assertion?

‍By design, no. An assertion is the test's opinion about correct behavior; only a human should change an opinion. Any tool that edits assertions automatically has inverted the purpose of the suite.

‍