Content
# Workable Evaluation MCP Server
An MCP server for evaluating AI Product Manager candidates — reads interview transcripts, scores candidates against a scorecard template, and submits evaluations to Workable.
## Prerequisites
- Python 3.12+
- [Poetry](https://python-poetry.org/)
- Workable API token (admin or recruiter scope)
- A scorecard/interview kit configured in Workable
- Interview transcripts stored as `data/interview-{name}.md`
## Setup
```bash
cd eval-mcp
poetry install
# Ensure WORKABLE_API_TOKEN and WORKABLE_EMAIL are set in your shell
```
## Configuration
| Variable | Required | Default | Description |
|---|---|---|---|
| `WORKABLE_API_TOKEN` | Yes | — | Workable API token |
| `WORKABLE_EMAIL` | Yes | — | Evaluator email (used by all tools, no manual input needed) |
| `WORKABLE_SUBDOMAIN` | No | auto-discovered | Workable account subdomain |
| `WORKABLE_TRANSCRIPTS_DIR` | No | `<project-root>/data` | Interview transcripts directory |
## Running
```bash
cd eval-mcp && poetry run python server.py
```
## opencode Integration
Add to `opencode.json`:
```json
{
"mcp": {
"workable": {
"type": "local",
"command": ["poetry", "-C", "eval-mcp", "run", "python", "server.py"],
"env": {
"WORKABLE_API_TOKEN": "${WORKABLE_API_TOKEN}",
"WORKABLE_TRANSCRIPTS_DIR": "${WORKABLE_TRANSCRIPTS_DIR}",
"WORKABLE_EMAIL": "${WORKABLE_EMAIL}"
}
}
}
}
```
## Workflow
```
1. evaluate_candidate(name) → guided: read transcript, score traits, preview, submit
2. edit_draft(name) → view and update a saved draft
3. list_pending_evaluations → see who needs evaluation
4. get_scorecard_template → see the 8 traits and their questions
5. read_transcript(candidate) → read the interview transcript
6. preview_evaluation(...) → save a draft (re-call to update)
7. list_drafts → review saved drafts
8. submit_evaluation(candidate) → submit to Workable
```
## Tools
| Tool | Description |
|---|---|
| `list_pending_evaluations` | Candidates needing your evaluation |
| `get_scorecard_template` | Show the scorecard structure and trait questions |
| `read_transcript` | Read the interview transcript for a candidate |
| `preview_evaluation` | Create or update a draft evaluation, returns a markdown summary |
| `submit_evaluation` | Submit a saved draft to Workable |
| `list_drafts` | Show saved draft evaluations |
| `list_evaluations` | Show past submitted evaluations (`detail=true` for full trait breakdown) |
| `get_evaluation` | Get the full evaluation for one candidate |
## Prompts
| Prompt | Description |
|---|---|
| `evaluate_candidate` | Guided end-to-end workflow — reads transcript, scores all 8 traits, writes a comment, previews the draft, and asks for confirmation before submitting |
| `edit_draft` | Loads a saved draft, displays current values, and guides updating the grade, comment, or trait scores |
## Commands
| Command | Description |
|---|---|
| `/eval-past` | List your past submitted evaluations |
| `/eval-pending` | List candidates pending your evaluation |
| `/eval-view` [name] | View a submitted evaluation for a candidate; prompts for name if omitted |
| `/eval-draft` [name] | Draft or edit an evaluation using the guided workflow; prompts for candidate if omitted |
## Project Structure
```
eval-mcp/
server.py — MCP server entry point (8 tools, 2 prompts, stdio)
workable.py — Workable API client with rate-limit retry
scorecard.py — Scorecard template loader and payload assembler
scorecard.json — AI PM scorecard template (4 sections, 8 traits)
draft_store.py — Disk persistence for drafts and submitted evaluations
drafts/ — Runtime draft files (gitignored)
submitted/ — Local archive of submitted evaluations (gitignored)
```
## Notes
- **Editing drafts:** The `edit_draft` prompt loads a saved draft and guides updating it. Under the hood, `preview_evaluation` is called again with the same candidate name to overwrite the draft.
- **Rate limit:** Workable API throttles at ~7-9 requests/minute. The client retries with exponential backoff (4 attempts, 2s/4s/8s).
- **Scorecard:** The `scorecard.json` is a local copy — Workable's API does not expose the scorecard template. Swap this file to change the evaluation template for a different role.
- **Trait storage:** Workable's API does not return per-trait scores for submitted evaluations. Detailed trait data is stored locally in `submitted/` and served via `list_evaluations(detail=true)` and `get_evaluation`.
- **Transcripts:** Interview transcripts are plain Markdown files in `data/`. The `read_transcript` tool reads them by name.
