Docs
Regression suite
Every flow the agent walked successfully, in an audit or an issue check, is kept as plain Playwright steps. The suite replays them without AI, so a later change that breaks one is caught within minutes of the deploy.
How it works
- Flows join the suite on their own: a completed audit flow, or an issue criterion that passed. A newer recording of the same flow replaces the older one.
- Every issue you confirm by moving its card to Fixing becomes a flow too: the steps that led to it. It fails while the issue is there and passes once it is fixed, so the issue cannot come back unnoticed. Not an issue removes it.
- Replays use the element's role and name first, then its test id, label or text, so a renamed button does not break a flow on its own.
- When an element is gone for good, the AI may point at the one that now does the same, and the replay goes on. The flow is marked Needs review and nothing changes in it until you accept. A wrong page or title after a step is never healed: that stays a failure, so a real regression is not hidden.
- A flow that no longer works becomes a card on the issue board, with the step that failed, its screenshot and logs.
- Replays follow safe mode: a recorded Delete is not pressed on production, and nothing is paid.
- Checkouts are left out of the suite, so a replay never pays again.
- A flow that fails is replayed once more from a fresh page. If it passes then, it is marked flaky: no card and no alarm, and the suite lists it apart so you can look into it.
Run it after every deploy
-
Create a deploy hook
On the project page, under Regression suite, click Create deploy hook. Copy the URL: it is shown only once. Anyone with it can start the suite, and nothing else.
-
Vercel
In your Vercel team settings, open Webhooks, add the URL and pick the deployment.succeeded event. Other deployment events are ignored, so picking more does no harm.
-
Netlify
In Site configuration → Notifications → Deploy notifications, add an outgoing webhook for Deploy succeeded with the URL.
-
GitHub Actions or any CI
Store the URL as a secret and call it after the deploy step.
# .github/workflows/deploy.yml, after your deploy step - name: Run the WitnessQA regression suite run: curl -fsS -X POST "${{ secrets.WITNESSQA_DEPLOY_HOOK }}"Anywhere else, one request is enough:
curl -X POST "$WITNESSQA_DEPLOY_HOOK"
On a schedule, or now
Pick Every day or Every week and the environment. Run now starts the suite at once.
Take your tests with you
Export as Playwright downloads the whole suite as a Playwright project that runs without WitnessQA: one page
object per page with role-based locators, one test per flow, and web-first assertions instead of fixed waits. Point
BASE_URL at any environment with the same paths and run npm test.
When a flow breaks
Paste a Slack incoming webhook, or any URL that accepts {"text": "..."}, under Notification webhook.
WitnessQA posts once per run that finds a problem, with the flows that broke and a link to the issue board.