Content
# 大模型黄历 · Astro-LLM
> **The LLM almanac.** One dashboard that tells you, every day, **(1)** which model is
> *sharp today* and whether you should spend or save your tokens, and **(2)** which
> model actually *shares your values* — all wrapped in an astrology / weather-report
> skin that's fun on the surface and a real personal eval harness underneath.
It looks like a fortune-telling almanac. It's actually a benchmark.
**▶ Live demo:** https://2-actual-hack.vercel.app · **Repo:** https://github.com/EthanPany/astro-llm
> Five completely different themed layouts (almanac · oracle · court · ink · aurora), a step-by-step value-match quiz, and a live model-fortune board. Full walkthrough: [`docs/showcase.mp4`](docs/showcase.mp4).
---
## Why
Coding plans (Claude Code, Codex, …) feel slightly different day to day — sometimes
the same model thinks longer, sometimes it's sharper, sometimes worse. And which model
"gets" your taste is deeply personal. Astro-LLM turns both into a daily reading:
| Organ | Astrology name | What it really measures |
|------|----------------|--------------------------|
| **Capability probe** | 运势 *fortune* | Is the model sharp today? — 18 deterministic, auto-graded tasks → an IQ, latency, cost, refusal rate, trend. |
| **Value matcher** | 缘分 *affinity* | Does it judge like *you*? — a bank of subjective dilemmas; we measure agreement (footrule + verdict match). |
| **MCP server** | — | How an agent *runs* the test on itself. |
| **Harmonize loop** | 调和 *harmony* | Can we nudge a model toward you? — inject your taste, re-test on **held-out** dilemmas. |
The "luck" is the skin; the numbers underneath are real. See the honesty note at the
bottom of the dashboard.
---
## Quickstart (local)
Requirements: Python 3.11+, [uv](https://docs.astral.sh/uv/), Node 20+.
```bash
cp .env.example .env # paste your API keys (all optional — see below)
make install # backend venv (uv) + frontend deps
make seed # probe live models, write committed seed JSON (optional)
make dev-backend # API on http://localhost:8000 (terminal 1)
make dev-frontend # UI on http://localhost:5173 (terminal 2)
```
Open **http://localhost:5173**.
**No keys?** It still works. The repo ships committed seed data, so the dashboard,
verdict grid and value-match all render offline. Keys unlock live re-probes
(`↻ 实时巡检`), the live custom-dilemma panel, and the harmonize loop.
### Providers
`ANTHROPIC_API_KEY`, `OPENAI_API_KEY`, `DEEPSEEK_API_KEY` are wired and verified.
`GEMINI_API_KEY` / `QWEN_API_KEY` are implemented; without them those models show
clearly-labelled simulated data. Adding a key flips them to live on the next `make seed`.
---
## Deploy (one command)
```bash
make up # docker compose: builds the frontend, serves SPA + API on :8000
# or:
docker build -t astro-llm . && docker run -p 8000:8000 --env-file .env astro-llm
```
In production FastAPI serves the built SPA and the API from the **same origin**, so
there's nothing else to configure. The image bakes in the seed data, so it renders
even with zero keys.
---
## Architecture
```
backend/ FastAPI + httpx + pydantic + SQLite (uv-managed)
app/
providers/ one adapter per provider (OpenAI/Anthropic/DeepSeek/Gemini/Qwen)
benchmarks/ dilemma bank, capability tasks, judge + capability runners,
matching (footrule + agreement), scoring (IQ + suggestion)
astrology/ deterministic zodiac / numerology / fortune / compatibility
loop/ harmonize (train/test guardrail)
storage/ SQLite DAL + seed generator
service.py assembly: runs + astrology -> AlmanacCards + headline
api/routes.py REST API | mcp_server.py MCP tools | scheduler.py
frontend/ React + Vite + TypeScript, handwritten CSS, GSAP, hand-drawn SVG charts
```
Full data contracts: [`ARCHITECTURE.md`](./ARCHITECTURE.md).
### API
`GET /api/dashboard` · `GET /api/models` · `GET /api/dilemmas` ·
`POST /api/match` · `POST /api/harmonize` · `POST /api/judge` · `GET /api/health`
### MCP
```bash
make mcp # python -m app.mcp_server (stdio)
```
Tools: `list_models`, `get_almanac`, `judge_dilemma`, `match_me`,
`score_model_against_user`. Point any MCP client (e.g. Claude Desktop) at it and an
agent can score itself against the benchmark.
### Hourly almanac
Set `ENABLE_SCHEDULER=true` to probe available models on an interval (`PROBE_INTERVAL_MINUTES`),
persisting real runs to SQLite so the trend becomes measured data over time.
---
## Tests
```bash
make test # 34 tests: astrology, matching, scoring, graders, judge parsing,
# harmonize, API (offline via TestClient + monkeypatched seed)
```
## Design
Handwritten CSS only — no Tailwind, no component library, no chart library. Charts are
hand-drawn SVG. Type: Fraunces (display) + Space Grotesk (UI) + Space Mono (numbers).
GSAP drives entrance reveals, number count-ups, and chart draw-on.
---
## Honesty note
The capability score is a real stopwatch + autograder. The value-match measures how a
model was *tuned to present itself*, not a hidden soul. The harmonize loop is scored on
**held-out** dilemmas the model never saw during tuning — the guardrail that keeps
"alignment" from collapsing into a flattery machine. The astrology is for fun; we say so.
> ⚠ Rotate any API keys that were shared in plaintext before making this repo public.
