Qampanion Logo
Qampanion

Self-healing

A test case can fail when the screen changes or an element moves. Self-healing sends the failed step to an AI agent before Qampanion reports the result. The agent reviews the failure and picks one of three verdicts. Qampanion applies the fix and runs the test case again.

Self-healing works with any OpenAI-compatible API.

Setup

  1. Click Settings in the toolbar and open the Project tab.
  2. In the AI section, fill in the connection fields.
  3. Turn on Enable self-healing.
  4. Click Save.
Field Description
Base URL API address with the version path, for example https://api.openai.com/v1. Qampanion adds /chat/completions.
Token API key of your provider. The app saves it on this Mac, outside the project package.
Model Model name as the provider writes it, for example gpt-4o. The model must accept images and support structured output (JSON schema).
Self-healing prompt Optional. Extra instructions for the agent, for example hints about your app.
Max agent calls per test case Default: 5.
Max agent calls per step Default: 2.

⚠️ Qampanion sends screenshots and step data to the AI provider you configure.

How it works

Self-healing starts when a required default step fails during a test case run or a test plan run. Two failures start it: a screenshot mismatch and an action error. Debug mode does not use self-healing.

Qampanion sends the agent:

  • the failed step and the previous step, with their actions and excluded zones
  • the expected screenshot, the actual screenshot, and the diff
  • the expected screenshot of the previous step
  • the UI tree of the current screen
  • the test case name, description, and list of steps
  • the results of earlier attempts on this step

If the previous step is a linked, network request, or script step, the agent sees only its type. The agent cannot change it.

Verdicts

  • update - the agent found a fix. It can replace the expected screenshot of the failed step and the actions of the failed step and the previous step. Qampanion saves the changes to the project and runs the test case again from the start.
  • skip - the failure does not matter, for example a loading animation. Qampanion marks the step as skipped and runs the remaining steps.
  • terminated - the agent found a real bug or no safe fix. Qampanion stops the test case and shows the original failure.

Limits and rollback

When a call limit is reached, or the agent call fails, the test case fails with the original failure.

If the test case does not pass after the fixes, Qampanion restores the changed steps to their original state. If it passes, the fixes stay in the project. Review the changed steps after the run.

⚠️ A small model can pick a wrong fix. If the fixes are wrong, use a stronger model or add hints to the prompt.

Status and logs

While the agent works, the test case and the step show the Self-healing status with a pulsing wand icon. Click Stop to cancel the run. Qampanion saves no fix after a stop.

The logs panel shows three kinds of lines:

  • self-healing started for a step
  • the verdict
  • the final action, for example fixes applied, restarting test case

The log file ~/Library/Caches/Qampanion/logs/ has more detail for each agent call: duration, tokens, verdict, reason, and what changed. If the provider returns the cost (OpenRouter does), the file also shows the cost per call and the total for the run.

CLI

run-test-case and run-test-plan use the self-healing settings from the project package. After a run with fixes, the CLI saves the project package and prints the self-healing events.

The CLI reads the token from the QAMPANION_AI_TOKEN environment variable. Without it, the CLI uses the token saved by the app on this Mac.

QAMPANION_AI_TOKEN="..." QampanionCLI run-test-case ...