What Happens When Your Test Suite Has a Planner, a Generator, and a Runner

Something shifted in test automation over the past year. Vendors stopped pitching "an AI agent" and started pitching several agents working together, each with one narrow job. Tricentis rebuilt its product line around this idea in March 2026. Autify frames its natural language testing the same way. Playwright, the open source framework most teams already run, shipped the pattern into the tool itself. If your test suite has never had a planner, a generator, and a runner working as separate roles, it probably will soon.

The pattern shows up in three very different places

Playwright added three named agents starting in version 1.56: planner, generator, and healer. According to Playwright's own documentation, the planner explores a running application and writes a Markdown test plan, the generator turns that plan into Playwright test files, and the healer runs the suite and repairs failing tests. You install all three with one command and point them at a client such as VS Code or Claude Code. The Playwright automation course covers this workflow end to end if you want a guided walkthrough.

Tricentis built a whole platform around the same split. Its AI Workspace, launched in March 2026, coordinates specialized agents for quality intelligence, test creation, automation, and performance across the software lifecycle rather than shipping one general purpose bot. Tricentis states on its product page that the platform can cut testing cycle time by up to 90 percent and lift productivity by up to 60 percent. Those figures come from the vendor, not an independent audit, so treat them as a ceiling rather than an average.

Autify approaches the same problem from the perception side. According to Autify's own writeup, its agents read an application visually, recognizing buttons and fields the way a person would rather than relying on brittle DOM selectors, then reason about what action to take next. The planning and the acting stay separate concerns even when the product markets them as a single assistant.

Three companies, three starting points, and they converged on the same division of labor. That convergence is the real story here, more than any single feature release.

What each role is actually on the hook for

Planner. Reads a goal, a requirement, or just the live application, and decides what should be tested. The output is a plan, not code. A planner that is doing its job well surfaces edge cases a human would have skipped on a tight sprint.

Generator. Takes an approved plan and writes the executable version. It picks locators, wires up assertions, and handles setup and teardown. This is the "how" stage, and it produces most of the code your team will actually maintain.

Runner, often called a healer. Executes the suite, watches what fails, and either repairs the test or flags it for a human. A locator that moved gets a new one. A test whose underlying flow broke should get skipped, not silently patched into a false pass.

Treating these as one job is how you end up with a single overloaded agent trying to explore, code, and debug in the same context window, doing all three worse than three focused agents would.

Why the split works better than one big agent

Large language models tend to be more reliable on a narrow, well scoped task than on one long open ended job that mixes planning, coding, and debugging at once. Splitting the work keeps each prompt tractable and each output easier to check on its own terms.

It also creates natural review points. A plan gets reviewed before any code exists. Generated code gets reviewed before it merges. A healer's patch gets reviewed before it ships to CI. Tricentis builds approval gates and a decision log into every step for exactly this reason, so regulated teams get an audit trail without slowing delivery down. Building a pipeline like this yourself instead of buying a platform? The Agentic AI for QA and SDET course walks through assembling a multi agent workflow role by role with an open framework.

Isolation matters too. When a nightly run turns red, you want to know immediately whether the plan was wrong, the generated code was wrong, or the healer's patch was wrong, instead of digging through one tangled transcript to find out which stage lied to you.

What changes for the person running the suite

The job moves from writing every line yourself to reviewing what each agent produced. That is a different skill than typing locators by hand, and for some teams it takes longer to adjust to than the tooling itself does.

You still need to write a clear test intent for the planner. You still need to read a generated plan critically enough to catch a missing negative case. You still need to decide whether a healer's locator fix is safe to merge or whether it just papered over a real regression. None of the three agents removes judgment from the process. They move where that judgment gets applied, from typing to reviewing.

Frequently asked questions

Do I need three separate tools, or does one platform cover all three roles?

Both exist. Playwright ships all three roles inside one open source framework you likely already use. Tricentis and Autify package the same roles inside a commercial platform. The role split matters more than which product owns it.

Does a planner, generator, runner setup replace scripted end to end tests?

No. Most current stacks still run fast deterministic unit and API tests as the base layer, keep scripted regression coverage for critical paths, and add the agentic layer on top for breadth and resilience to UI churn.

What happens if the runner heals a test that was actually catching a real bug?

That is the central risk here. A healer that finds a workaround, such as extending a timeout or swapping in a new locator, can quietly turn a legitimate failure into a false pass. Treat every automated patch as a proposal that needs a human look before it merges, not a fix that is already safe to trust.

Can a small team without a platform budget build something like this?

Yes. Playwright's agents are free and open source, and the three roles can be adopted one at a time, starting with whichever stage is costing your team the most hours right now.

If this looks like the direction your own test strategy is heading, the Generative AI and AI Agents for QA and SDETs masterclass covers the prompt writing and review habits this setup depends on, so the judgment stays with you as the typing moves to the agents.