Content
# Aethr /ˈiːθər/
Natural language test scenario runner agent with MCP tools.
## Overview
Aethr is a command-line AI agent that runs natural language test scenarios with LLM and MCP, with real-time terminal feedback. It bridges the gap between plain natural language test descriptions and automated testing.
## Demo
### Success example
### Failure example
## Features
- Natural language test file definition
- AI agent for test plan and execution
- MCP tool calling to automate test execution
- CLI-based i.e. CI friendly
## Getting started
Assuming you have Node.js installed and an OpenAI key that can access your `gpt-4.1-mini` model on macOS or Linux:
```sh
export OPENAI_API_KEY=sk-....
export OPENAI_MODEL=gpt-4.1-mini
mkdir aethr-test && cd aethr-test
# The 'aethr-test' directory is created to serve as a workspace to store aethr.config.mjs and 'logs'
# You can replace 'aethr-test' with any directory name of your choice.
npx -y aethr@latest run https://raw.githubusercontent.com/autifyhq/aethr/main/demo.md
```
> [!NOTE]
> See how to setup for the other LLM providers in Prerequisites section.
## Prerequisites
- Install Node.js 22+ by any method you like
- Setup API keys or credentials for one of LLM providers below:
- OpenAI, OpenRouter, Anthropic, Amazon Bedrock, Google GenAI, Google VertexAI, Azure OpenAI, Groq, Cohere, Ollama
- Put model name and keys/credentials in environment variables (or `.env` file)
<details>
<summary>Environment variable setup</summary>
```shell
# Model names here are just examples.
# As long as the model supports tool calling, you can try it out with Aethr.
OPENAI_MODEL=gpt-4o-mini
OPENAI_API_KEY=
# or
OPENROUTER_MODEL=openai/gpt-4o-mini
OPENROUTER_API_KEY=
# or
ANTHROPIC_MODEL=claude-3-7-sonnet-20250219
ANTHROPIC_API_KEY=
# or
BEDROCK_MODEL=us.amazon.nova-pro-v1:0
AWS_PROFILE=
AWS_DEFAULT_REGION=
# or
GOOGLE_MODEL=gemini-1.5-flash-8b
GOOGLE_API_KEY=
# or
VERTEXAI_MODEL=gemini-2.0-flash-exp
GOOGLE_CLOUD_PROJECT=
# or
AZURE_OPENAI_MODEL=gpt-4o-mini
AZURE_OPENAI_API_KEY=
AZURE_OPENAI_API_INSTANCE_NAME=
AZURE_OPENAI_API_DEPLOYMENT_NAME=
AZURE_OPENAI_API_VERSION=
# or
GROQ_MODEL=llama-3.1-8b-instant
GROQ_API_KEY=
# or
COHERE_MODEL=command-r7b-12-2024
COHERE_API_KEY=
# or
OLLAMA_MODEL=llama3.2
```
</details>
## Usage
Create a test scenario file (see below or examples directory) and run it:
```bash
npx -y aethr@latest run <your_test_file>
```
> [!NOTE]
> The default MCP tools is `@aethr/playwright-mcp` that is a patched version of `@playwright/mcp` with a few new features:
>
> - Assertion tool (`browser_assert_contain_text`)
> - Environment variable replacement (`browser_navigate`, `browser_type`)
> - Trace file output
>
> You can use any MCP tools for test automation by modifying `aethr.config.mjs` created by the first execution.
Along with the CLI output logs, you can also check Playwright Trace data for debugging:
```bash
npx -p @aethr/playwright-mcp@latest playwright show-trace ./trace.zip
```
### Example Test Scenario
Test scenario can be any text format as long as LLM can understand.
Here is an example using Markdown:
```markdown
# Test login feature
Follow the steps below.
## Steps
1. Visit ${URL}
2. Enter ${MY_USERNAME} to the Username
3. Enter ${MY_PASSWORD} to the Password
4. Click login button
5. Assert the button shows "Logout"
6. Assert "You logged into a secure area!" text exists
```
Note: `${...}` notation is replaced by the environment variables available on the CLI execution (currently it's done by MCP actually). This is useful for sensitive input like password as well as data driven tests.
## Configuration
Aethr uses `aethr.config.mjs` in the current directory as a configuration file (you can change it by `--config-file` option):
```js
/** @type {import("aethr/dist/config/schema.js").ConfigSchema} */
export default {
profiles: {
default: {
mcpServers: {
playwright: {
command: "npx",
args: [
"-y",
"@aethr/playwright-mcp@latest",
...["--browser", "chromium"],
...["--headless"],
...["--user-data-dir", "${TEMP_DIR}"], // ${TEMP_DIR} is given by Aethr per run and cleared after the run
],
env: {
FORCE_COLOR: "0", // To avoid unnecessary color codes in the assertion failure message
TRACE: "./trace.zip", // Location to store the trace file
},
},
},
runOptions: {
recursionLimit: 50,
temperature: 0.7,
thinkTool: false,
reasoning: false,
},
},
},
};
```
You can tweak any configuration here.
## CI example
> [!NOTE]
> We're preparing to provide a GitHub Actions to simplify this setup. Stay tuned.
If you want to run Aethr in GitHub Actions:
```yaml
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: "22"
- run: npx -y -p @aethr/playwright-mcp playwright install chromium --with-deps --only-shell
- run: npx -y aethr run ./example/login.md
env:
OPENROUTER_MODEL: ${{ vars.OPENROUTER_MODEL}}
OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }}
URL: ${{ vars.URL }}
MY_USERNAME: ${{ secrets.MY_USERNAME }}
MY_PASSWORD: ${{ secrets.MY_PASSWORD }}
- run: mkdir -p trace && unzip trace.zip -d trace
- uses: actions/upload-artifact@v4
with:
name: trace
path: trace
- uses: actions/upload-artifact@v4
with:
name: logs
path: logs
```
## Development
```bash
# Run in development mode
npm run dev -- run examples/login.test.md
# Lint code
npm run lint
# Format code
npm run format
```
## Author
Autify Inc.
## License
Apache-2.0
