Content
# Tool List
This directory is a set of LiteLLM agent skills for students. The positioning is "teaching and practical guidance," not directly replacing the official live proxy management skills.
## Official Situation
LiteLLM officially provides `litellm-skills`, which allows tools that support Agent Skills standards to manage live LiteLLM proxies through `curl`. Official skills cover users, teams, keys, orgs, models, MCP servers, agents, and usage queries.
Official sources:
- LiteLLM Skills documentation: https://docs.litellm.ai/docs/tutorials/claude_code_skills
- Official GitHub repository: https://github.com/BerriAI/litellm-skills
- Skills Gateway: https://docs.litellm.ai/docs/skills_gateway
## Skill Set
| Skill | Teaching Purpose |
| --- | --- |
| `litellm-sdk-basics` | Practice building completion, streaming, structured output, and tool calling using LiteLLM Python SDK |
| `litellm-proxy-gateway` | Design OpenAI-compatible LiteLLM Proxy and `config.yaml` |
| `litellm-routing-reliability` | Practice Router, load balancing, fallback, retry, timeout, and health checks |
| `litellm-cost-governance` | Establish spend tracking, virtual keys, budgets, tags, and cost attribution processes |
| `litellm-observability` | Configure callbacks, logging, OpenTelemetry, and cost/request tracking |
| `litellm-guardrails-safety` | Configure guardrails, tool permission, MCP guardrails, and proxy call hooks |
| `litellm-mcp-skills-gateway` | Practice MCP Gateway, MCP permissions, toolsets, Skills Gateway, and official skills evaluation |
## Recommended Teaching Order
1. `litellm-sdk-basics`
2. `litellm-proxy-gateway`
3. `litellm-routing-reliability`
4. `litellm-cost-governance`
5. `litellm-observability`
6. `litellm-guardrails-safety`
7. `litellm-mcp-skills-gateway`
## Usage Scenarios and Prompt Examples
The following are three usage scenarios and corresponding AI prompt examples for each skill:
### 1. `litellm-sdk-basics` (LiteLLM SDK Basics)
* **Scenario 1: Basic Completion and Exception Handling**
* *Prompt example*: "Help me write a basic completion script using LiteLLM Python SDK with `openai/gpt-4o-mini`, including rate limit and authentication error exception handling. Do not hardcode API keys in the code."
* **Scenario 2: Structured Outputs**
* *Prompt example*: "I need to design a structured output example using LiteLLM SDK, defining a Pydantic schema to parse news summaries (including `title`, `summary`, `tags`), and ensuring the model output conforms to the schema and verifying it at the Python level."
* **Scenario 3: Function Calling**
* *Prompt example*: "Please help me write a complete exercise on LiteLLM function calling. Define a `get_weather` tool that can query the weather and demonstrate the model's decision to call the tool, execute the tool, and return the result to obtain the final model response."
### 2. `litellm-proxy-gateway` (LiteLLM Proxy Gateway)
* **Scenario 1: Basic `config.yaml` and Multi-Model Alias**
* *Prompt example*: "Help me design a `config.yaml` for LiteLLM Proxy. I need to set up two model aliases: `course-gpt` pointing to `openai/gpt-4o-mini` and `course-claude` pointing to `anthropic/claude-3-5-haiku`. All API keys must be read from environment variables."
* **Scenario 2: Testing Proxy Connection with OpenAI SDK**
* *Prompt example*: "I have started a local LiteLLM Proxy (`http://localhost:4000`). Please help me write a Python test script using the standard `openai` SDK with a virtual key (`test-key`) to call the `course-gpt` model configured in our proxy."
* **Scenario 3: Virtual Keys and Access Control**
* *Prompt example*: "I want to enable virtual key functionality in LiteLLM Proxy. Please help me write a tutorial on how to enable database storage for spend tracking in config and create a virtual key with admin key through API that can only use `course-gpt` with a daily budget of $1 USD."
### 3. `litellm-routing-reliability` (Routing and Reliability)
* **Scenario 1: Load Balancing and Router Configuration**
* *Prompt example*: "I want to design a load balancing (Load Balancing) router configuration for `gpt-4o-mini`. Please help me write a `config.yaml` that configures two different API keys (in the form of environment variables) corresponding to the same model and explains how LiteLLM performs request distribution."
* **Scenario 2: Automatic Failover**
* *Prompt example*: "Please help me design a LiteLLM Proxy configuration with a failover mechanism. When the primary model `openai/gpt-4o` encounters a Rate Limit or Service Unavailable, it can automatically fallback to the backup model `anthropic/claude-3-5-sonnet` and provide a Python example to test this backup mechanism."
* **Scenario 3: Retries and Timeout Control**
* *Prompt example*: "I need to add global retries and timeout limits to the LiteLLM Router configuration. Please modify my `config.yaml` to set the connection timeout to 10 seconds and automatically retry 3 times when encountering a 5xx error, and demonstrate how to set a separate timeout for individual calls on the Python SDK side."
### 4. `litellm-cost-governance` (Cost Management and Governance)
* **Scenario 1: Project and Team Cost Attribution (Spend Tracking by Tags / Teams)**
* *Prompt example*: "Please teach me how to track API call costs for different student members or projects through `tags` or `team_id` in LiteLLM Proxy. I need to see how to pass these metadata when calling API and how to perform cost statistics on the Proxy side."
* **Scenario 2: Setting Budget Limits and Quota Alerts (Budgets & Rate Limiting)**
* *Prompt example*: "I want to set a weekly budget limit for a specific virtual key. Please write the relevant `config.yaml` configuration or use the management API to set the virtual key budget and explain what error code LiteLLM will return when the budget exceeds."
* **Scenario 3: Basic Cost Estimation and Callback Integration**
* *Prompt example*: "I don't want to use a database, but I want to print the token usage and estimated cost of each call directly in the terminal after the Python SDK call is completed. Please help me implement an example using `completion_cost()` and explain how it calculates the price of different provider models."
### 5. `litellm-observability` (Observability and Monitoring)
* **Scenario 1: Configuring Global Logging and Webhook Callbacks**
* *Prompt example*: "I want to add a custom logging callback to LiteLLM Proxy. When each API call succeeds or fails, it can send the request and response token usage and model name to my specified Slack webhook. Please help me design this config and implementation logic."
* **Scenario 2: Integration with OpenTelemetry / Langfuse**
* *Prompt example*: "For teaching demonstrations, I want to synchronize the call traces (Traces) of LiteLLM Proxy to Langfuse for observability analysis. Please provide a complete `config.yaml` configuration example, including how to securely pass Langfuse's public key and secret key."
* **Scenario 3: Using Built-in Prometheus Metrics for Monitoring**
* *Prompt example*: "I want to deploy LiteLLM Proxy in Kubernetes and use Prometheus to monitor request latency and error rate. Please teach me how to enable Prometheus telemetry metrics in `config.yaml` and list common Prometheus query syntax."
### 6. `litellm-guardrails-safety` (Safety and Guardrails)
* **Scenario 1: Configuring Input Filtering and Content Moderation**
* *Prompt example*: "I want to block student input of sensitive words at the LiteLLM Proxy level. Please help me configure Llama Guard or OpenAI Moderation as a guardrail to review content before requests reach LLM and reject non-compliant requests."
* **Scenario 2: Model Access Control with Virtual Keys**
* *Prompt example*: "To avoid student abuse of expensive models, I need to design a security guardrail: restrict a virtual key to only call `gpt-4o-mini`, and directly return a permission error if the key attempts to call `gpt-4o`. Please write out the specific configuration and testing method."
* **Scenario 3: Custom Guard Logic with Proxy Call Hooks**
* *Prompt example*: "I want to write a custom Python Hook (e.g., `pre_call_hook`) in LiteLLM Proxy to check the length of the request prompt. If it exceeds 2000 characters, directly intercept and reject the call. Please provide the writing method and mounting configuration of this Python Hook."
### 7. `litellm-mcp-skills-gateway` (MCP and Skills Gateway)
* **Scenario 1: Configuring MCP Servers in LiteLLM**
* *Prompt example*: "I want to integrate LiteLLM Proxy with Model Context Protocol (MCP). Please configure an MCP server in `config.yaml` (e.g., a server connected to a local sqlite database) and demonstrate how models under Proxy can dynamically use the MCP tool."
* **Scenario 2: Setting MCP Tool Permissions and Filtering (MCP Permissions)**
* *Prompt example*: "We have many MCP tools, but I want to restrict specific virtual keys to only use read-type MCP tools and cannot execute write or delete operations. Please teach me how to configure MCP tool permission control in LiteLLM Gateway."
* **Scenario 3: Skills Gateway and Official Skills Evaluation**
* *Prompt example*: "Please help me analyze the difference between LiteLLM's official Skills Gateway and a general MCP gateway and write out how to use the official `litellm-skills` to enable an agent to autonomously manage users and keys in Proxy and evaluate its architecture and security."
## Usage Principles
- When needing version, model, price, and functional support status, must re-check LiteLLM official documentation or official GitHub.
- Do not submit API keys, proxy master keys, virtual keys, or database connection passwords in teaching materials, slides, or repositories.
- Student exercises should use local proxy, test keys, low-cost models, and minimal budgets.
- If managing live proxies, prioritize reference to official `litellm-skills` and include admin key permissions, operation auditing, and response processes in classroom requirements.
