Content
# Custom Databricks MCP Server for Claude Desktop
A local read-only Model Context Protocol (MCP) server that connects Claude Desktop to an Azure Databricks workspace through the Databricks REST API.
The goal of this project is to let Claude inspect Databricks workspace metadata, notebooks, jobs, job runs, and clusters through safe read-only tools.
## Overview
Databricks Genie MCP is useful for asking natural-language questions over configured data spaces, but it does not inspect notebooks, source code, jobs, clusters, or workspace history.
This custom MCP server fills that gap by exposing selected Databricks REST API calls as Claude Desktop tools.
## Features
* List Databricks workspace folders and objects
* Export Databricks notebooks as source code
* List Databricks jobs
* List recent job runs
* List Databricks clusters
* Provide Claude Desktop with read-only Databricks workspace inspection tools
## Tools Exposed to Claude
The server exposes the following MCP tools:
| Tool | Description |
| --------------------------- | ---------------------------------------------------------- |
| `list_workspace` | Lists Databricks workspace objects under a given path |
| `export_notebook` | Exports a Databricks notebook as source code |
| `list_jobs` | Lists Databricks jobs |
| `list_job_runs` | Lists recent Databricks job runs |
| `list_clusters` | Lists Databricks clusters |
| `summarize_workspace_start` | Provides a quick summary of workspace objects under a path |
## Project Structure
```text
databricks-mcp/
├── server.py
├── .env
├── .env.example
├── .gitignore
└── README.md
```
Important: `.env` must not be committed to GitHub.
## Requirements
* Windows, macOS, or Linux
* Python 3.10 or newer
* Claude Desktop
* Azure Databricks workspace
* Databricks Personal Access Token
* Git, if publishing the project to GitHub
Python packages:
```text
mcp
requests
python-dotenv
```
## Setup
### 1. Clone or Create the Project Folder
Example on Windows:
```powershell
cd C:\
mkdir Eli_Projects
cd C:\Eli_Projects
mkdir databricks-mcp
cd databricks-mcp
```
If the folder already exists:
```powershell
cd C:\Eli_Projects\databricks-mcp
```
### 2. Create a Python Virtual Environment
```powershell
python -m venv .venv
```
Activate the environment:
```powershell
.\.venv\Scripts\activate
```
After activation, the terminal should look similar to this:
```powershell
(.venv) PS C:\Eli_Projects\databricks-mcp>
```
### 3. Install Dependencies
```powershell
pip install mcp requests python-dotenv
```
Optional: create a `requirements.txt` file:
```powershell
pip freeze > requirements.txt
```
Then future users can install dependencies with:
```powershell
pip install -r requirements.txt
```
## Databricks Authentication
Create a `.env` file in the project root:
```powershell
notepad .env
```
Add the following content:
```env
DATABRICKS_HOST=https://your-workspace.azuredatabricks.net
DATABRICKS_TOKEN=your_databricks_personal_access_token
```
Example host format:
```env
DATABRICKS_HOST=https://adb-xxxxxxxxxxxxxxxx.x.azuredatabricks.net
```
Do not share or commit your Databricks token.
## Example `.env.example`
Create `.env.example` so others know which variables are required:
```env
DATABRICKS_HOST=https://your-workspace.azuredatabricks.net
DATABRICKS_TOKEN=your_databricks_personal_access_token
```
## Test the Databricks Connection
Because an MCP server communicates through JSON-RPC, running `python server.py` directly may appear to wait for input. For a quick Databricks API self-test, temporarily replace the bottom of `server.py`:
```python
if __name__ == "__main__":
mcp.run()
```
with:
```python
if __name__ == "__main__":
if os.getenv("MCP_SELF_TEST") == "1":
print(summarize_workspace_start("/Workspace"))
else:
mcp.run()
```
Then run:
```powershell
$env:MCP_SELF_TEST="1"
python server.py
```
Expected output shape:
```text
{'path': '/Workspace', 'count': 3, 'items': [...]}
```
After the test, reset the variable:
```powershell
Remove-Item Env:\MCP_SELF_TEST
```
Then change the bottom of `server.py` back to:
```python
if __name__ == "__main__":
mcp.run()
```
## Claude Desktop Configuration
Open the Claude Desktop configuration file.
Windows location:
```text
C:\Users\<YourUserName>\AppData\Roaming\Claude\claude_desktop_config.json
```
Example for this project path:
```json
{
"mcpServers": {
"databricks-workspace": {
"command": "C:\\Eli_Projects\\databricks-mcp\\.venv\\Scripts\\python.exe",
"args": [
"C:\\Eli_Projects\\databricks-mcp\\server.py"
]
}
}
}
```
If you already have other MCP servers configured, add `databricks-workspace` inside the existing `mcpServers` object.
Example with another existing server:
```json
{
"mcpServers": {
"databricks-genie": {
"command": "cmd",
"args": [
"/c",
"npx",
"mcp-remote",
"https://your-workspace.azuredatabricks.net/api/2.0/mcp/genie",
"--header",
"Authorization: Bearer your_databricks_token"
]
},
"databricks-workspace": {
"command": "C:\\Eli_Projects\\databricks-mcp\\.venv\\Scripts\\python.exe",
"args": [
"C:\\Eli_Projects\\databricks-mcp\\server.py"
]
}
}
}
```
After editing the config:
1. Save `claude_desktop_config.json`.
2. Fully close Claude Desktop.
3. Open Task Manager and end any remaining Claude processes.
4. Start Claude Desktop again.
5. Go to `Settings -> Developer -> Local MCP Servers`.
6. Confirm that `databricks-workspace` is running.
## Example Claude Prompts
After the MCP server is running, ask Claude:
```text
Use databricks-workspace to summarize my Databricks workspace from /Workspace.
```
```text
Use databricks-workspace to list /Workspace/Users.
```
```text
Use databricks-workspace to list my Databricks jobs and recent job runs.
```
```text
Use databricks-workspace to list my Databricks clusters.
```
```text
Use databricks-workspace to export this notebook and summarize the code: /Workspace/Users/example@example.com/my_notebook
```
## GitHub Setup
### 1. Create `.gitignore`
Create `.gitignore`:
```powershell
notepad .gitignore
```
Add:
```gitignore
.venv/
.env
__pycache__/
*.pyc
.mypy_cache/
.pytest_cache/
.DS_Store
```
### 2. Initialize Git
```powershell
git init
git status
```
Make sure `.env` is not staged or committed.
### 3. Add Files
```powershell
git add server.py README.md .gitignore .env.example
```
If you created `requirements.txt`, add it too:
```powershell
git add requirements.txt
```
### 4. Commit
```powershell
git commit -m "Initial custom Databricks MCP server"
```
### 5. Create GitHub Repository
Create a new empty GitHub repository, for example:
```text
databricks-mcp-server
```
Do not initialize the GitHub repository with a README, `.gitignore`, or license if you already created them locally.
Then connect the local repository:
```powershell
git branch -M main
git remote add origin https://github.com/YOUR_USERNAME/databricks-mcp-server.git
git push -u origin main
```
Replace `YOUR_USERNAME` with your GitHub username.
## Security Notes
* Do not commit `.env`.
* Do not commit Databricks tokens.
* Use a Databricks token with the minimum required permissions.
* Start with read-only tools only.
* Do not add tools for deleting, importing, running jobs, modifying permissions, or terminating clusters unless you explicitly need them.
* If a token is exposed in logs, screenshots, or GitHub, revoke it immediately and create a new one.
## Limitations
* This server only exposes the tools implemented in `server.py`.
* It does not automatically understand every Databricks asset unless the relevant API call is added.
* It does not inspect notebook code unless `export_notebook` is called with a specific notebook path.
* It does not replace Databricks Genie. Genie is still useful for natural-language questions over configured data spaces.
* It is intended as a local development and learning project, not a production service.
## Possible Future Improvements
* Add pagination support for large workspaces and job lists
* Add a recursive workspace scanner
* Add notebook summarization helpers
* Add SQL warehouse listing
* Add Unity Catalog metadata listing
* Add support for Databricks repos
* Add structured error messages
* Add logging without exposing secrets
* Add tests
* Add Docker support
## License
This project can be licensed under the MIT License or another license of your choice.
