Content
# AI-Powered Multilingual Document Summarizer
A complete, production-ready Multilingual Document Summarizer built with Python, Streamlit, Hugging Face Transformers (`google/mt5-small`), and FastMCP. The application automatically extracts text from PDF documents, detects the written language (English + 9 Indian languages), cleans the content, generates a concise summary, and visualizes compression statistics. It also runs as a Model Context Protocol (MCP) server so AI clients like Cursor can invoke it as a tool.
---
## 🌟 Key Features
* **High-Fidelity Text Extraction**: Reusable page-by-page PDF reader that handles errors, empty pages, and Unicode characters.
* **Multilingual Support**: Supports automatic language detection and summarization for **English** and **9 Indian languages**: Hindi, Marathi, Tamil, Telugu, Bengali, Gujarati, Kannada, Malayalam, and Punjabi.
* **Advanced NLP Summarization**: Utilizes the Google `mT5-small` Transformer model (300M parameters) hosted and cached locally.
* **Token-based Chunking Pipeline**: Automatically segments long PDF documents into safe-sized token chunks to bypass the model's sequence-length limitations, summarizing sections individually before performing a final refining pass.
* **Interactive Web UI**: Modern, glassmorphic Streamlit interface with sidebar controls, language badges, and a statistics dashboard displaying original/summary word counts, compression reduction ratios, and processing times.
* **Cursor-Ready MCP Server**: Implements a standard `FastMCP` server exposing tools to summarize, detect languages, and save summaries of documents directly from Cursor or other AI editors.
---
## 📁 Project Structure
```text
multilingual-document-summarizer/
├── app.py # Streamlit Frontend Web Application
├── mcp_server.py # FastMCP Server (Stdio transport)
│
├── summarizer/ # Core NLP and processing package
│ ├── pdf_reader.py # Reusable PDF text extraction module
│ ├── language_detector.py # Text language identification module
│ ├── text_preprocessor.py # Whitespace/Unicode normalization module
│ ├── mt5_model.py # Local model downloading and caching module
│ └── summarizer_service.py # Token chunking and mT5 summarizer orchestration
│
├── uploads/ # Storage for temporary uploaded PDFs
├── summaries/ # Saved summaries (as text files)
├── models/ # Local cached model weights (google/mt5-small)
│
├── requirements.txt # Dependency configuration
└── README.md # Project Documentation
```
---
## 🚀 Installation & Setup
Ensure you have **Python 3.10+** installed on your system.
1. Clone or navigate to the project workspace directory:
```powershell
cd c:\Users\Asus\OneDrive\Desktop\mcpDocSummarizer
```
2. Install the required dependencies:
```powershell
pip install -r requirements.txt
```
*Note: On first run, Hugging Face will automatically download the ~1.2 GB `google/mt5-small` model and cache it inside the local `models/` directory for offline reuse.*
---
## 🖥️ Running the Streamlit App
Launch the Streamlit web application:
```powershell
streamlit run app.py
```
This will open the interface in your default browser (usually at `http://localhost:8501`).
### How to use the Web UI:
1. **Configure Length**: In the sidebar, select your target summary length (`Short`, `Medium`, or `Long`).
2. **Upload PDF**: Drag and drop your PDF file in the upload zone.
3. **Generate**: Click **⚡ Generate Summary**. The app will show a status card tracking extraction, language identification, preprocessing, and model inference steps.
4. **Download**: Once finished, view the summary text, inspect the statistics (word count reduction percentage and processing speed), and click **💾 Download Summary** to save it.
---
## 🤖 Integrating with Cursor (MCP Server Setup)
The FastMCP server enables Cursor (or other MCP-supporting clients) to read, detect, and summarize PDFs using your local `mT5` model.
### 1. Configure Cursor
To add the summarizer tools to Cursor:
1. Open Cursor and go to **Settings** (Gear Icon in the top right) -> **Features** -> **MCP**.
2. Click **+ Add New MCP Server**.
3. Fill in the details:
* **Name**: `Multilingual Summarizer`
* **Type**: `command`
* **Command**: `python "c:\Users\Asus\OneDrive\Desktop\mcpDocSummarizer\mcp_server.py"`
4. Click **Save**. The status indicator should turn green, indicating a successful connection.
### 2. Available MCP Tools
Once configured, Cursor can call the following tools:
* `detect_pdf_language(pdf_path: str)`: Returns the detected ISO code and human-readable name of the PDF language.
* `summarize_pdf(pdf_path: str, summary_length: str)`: Generates a summary for the given PDF using the local mT5 model and returns structured metrics.
* `save_summary(pdf_path: str, summary_length: str)`: Summarizes the PDF and saves the resulting text to `summaries/summary_<filename>_<timestamp>.txt`.
### Example Cursor Prompt:
> *"Can you detect the language and summarize this document for me? Path: c:\Users\Asus\OneDrive\Desktop\mcpDocSummarizer\uploads\sample.pdf"*
---
## 🛠️ Verification & Verification Commands
To verify the individual sub-modules without running the full Streamlit UI or MCP server:
### 1. Test Language Detection
Run a short python command to verify `langdetect` is operating:
```powershell
python -c "from summarizer.language_detector import detect_language; print(detect_language('नमस्ते, यह एक परीक्षण है।'))"
# Output should be: ('hi', 'Hindi')
```
### 2. Test PDF Reader
Verify that text can be read page-by-page from a sample PDF:
```powershell
python -c "from summarizer.pdf_reader import extract_text; print(extract_text('path/to/your/pdf.pdf')[:100])"
```
