langchain-masterclass

<p align="center"> <h1 align="center">LangChain MasterClass</h1> <p align="center">A comprehensive, hands-on course for mastering the LangChain ecosystem — from first model call to agents.</p> </p> <div align="center"> [](https://www.python.org/) [](https://www.langchain.com/) [](LICENSE) </div> <div align="center"> []() []() []() []() </div> --- ## Table of Contents - [Introduction](#-introduction) - [Features](#-features) - [Course Roadmap](#-course-roadmap) - [Prerequisites](#-prerequisites) - [Installation](#-installation) - [Configuration](#%EF%B8%8F-configuration) - [Running Labs](#%EF%B8%8F-running-labs) - [Curriculum](#-curriculum) - [Skills Acquired](#-skills-acquired) - [Best Practices Demonstrated](#-best-practices-demonstrated) - [Troubleshooting / FAQ](#-troubleshooting--faq) - [Contributing](#-contributing) - [Acknowledgements](#-acknowledgements) - [License](#-license) --- ## 🦜 Introduction **LangChain MasterClass** is a structured, code-first curriculum that teaches you how to build sophisticated AI applications using the LangChain ecosystem. Every concept is grounded in a self-contained Python lab — no slides, no fluff, no theoretical hand-waving. The repository spans 18 modules and 100+ individual labs, taking you from the absolute basics (invoking a chat model) all the way to building autonomous agents that call live APIs, query databases, discover tools over the Model Context Protocol, and reason across multi-step tool chains. Every lab is executable as a standalone script with a single command. ### Target Audience This course is designed for **students, Python developers, backend engineers, and AI practitioners** who want to build real LangChain applications — not just copy-paste demos. Whether you are transitioning into AI engineering, deepening an existing LangChain skill set, or building the knowledge base needed to build RAG and agent systems, this curriculum provides a clear, practical path. | Audience | Why This Course Fits | |----------|---------------------| | **Students** | Bridging academic theory with industry-standard AI engineering | | **Python Developers** | Familiar with Python; want to add AI/LLM capabilities to their skill set | | **Backend Engineers** | Building APIs or services that incorporate LLM-powered features | | **AI/ML Practitioners** | Know the theory; want hands-on LangChain implementation experience | | **Data Engineers** | Interested in RAG pipelines, document processing, and semantic search | ### What Makes It Different? Unlike tutorial repositories that focus on a single feature or borrow examples directly from LangChain's documentation, this course builds every concept from the ground up using a consistent codebase, shared helper utilities, and progressively harder challenges. A multi-provider **ModelFactory** abstraction means you can run every lab against OpenAI, Anthropic Claude, Google Gemini, Ollama, or HuggingFace and more by changing two lines in a `.env` file. --- ## ✨ Features - 📦 **18 progressive modules** — from your first model call to full autonomous agents - 🧪 **100+ runnable labs** — every concept is a standalone, executable Python script - 🔌 **Multi-provider support** — OpenAI, Anthropic, Google Gemini, Ollama, HuggingFace and more - 🧬 **Structured outputs** — Pydantic, TypedDict, and JSON-schema-driven extraction - 🗄️ **Vector databases** — InMemoryVectorStore, Chroma, and FAISS, each with full CRUD coverage - 📚 **Retrieval-Augmented Generation (RAG)** — from basic pipelines to conversational chatbots and agentic RAG - 🔗 **Model Context Protocol (MCP)** — local and remote tool discovery and aggregation - 🤖 **Agents** — custom agentic loops, ReAct-style agents, SQL agents, and RAG agents - ⚡ **Async, Streaming** — non-blocking pipelines with token-by-token delivery - 📡 **Callbacks** — logging, cost tracking, and real-time event hooks --- <p align="center"> 💡 If you find this repo useful, don't forget to <a href="https://github.com/zainulabidin1/langchain-masterclass/stargazers">star (🌟)</a> — it helps others discover it too! </p> --- ## 📚 Course Roadmap | Module | Topic | Skills Learned | |--------|-------|----------------| | [`01_models`](./01_models) | Models & Embeddings | ChatModel, temperature, max_tokens, stop sequences, multimodal (image input), LLM caching, embedding vectors, embedding dimensions | | [`02_prompts`](./02_prompts) | Prompt Engineering | Message types (System/Human/AI/Tool/Chat), PromptTemplate, ChatPromptTemplate, multi-variable prompts, few-shot prompting, length-based example selection | | [`03_chains`](./03_chains) | LCEL Chains | Simple LCEL chains, output parsers in chains, multi-step sequential chains, batch execution, sequential vs batch performance benchmarking | | [`04_runnables`](./04_runnables) | LCEL Runnables | RunnableSequence, pipe operator `\|`, RunnableParallel, RunnableBranch, RunnableLambda (dynamic routing), RunnablePassthrough, `.assign()`, fallback chains, runtime ConfigurableField & ConfigurableAlternatives | | [`05_output_parsers`](./05_output_parsers) | Output Parsers | StrOutputParser, CommaSeparatedListOutputParser, JsonOutputParser, XMLOutputParser, PydanticOutputParser, custom BaseOutputParser | | [`06_structured_output`](./06_structured_output) | Structured Output | JSON schema binding, TypedDict schema, Pydantic schema, nested Pydantic models, `with_structured_output()` | | [`07_async_streaming`](./07_async_streaming) | Async & Streaming | `ainvoke`, `astream` (token-by-token), `astream_events` (v2 event filtering by run name) | | [`08_callbacks`](./08_callbacks) | Callbacks | StdOutCallbackHandler, OpenAI token & cost tracking, custom sync BaseCallbackHandler, custom async AsyncCallbackHandler | | [`09_memory`](./09_memory) | Memory & Persistence | Stateless chat (demonstrates forgetting), manual message list, RunnableWithMessageHistory, SQLite via SQLChatMessageHistory, JSON file custom history, `trim_messages`, interactive in-memory chatbot, interactive SQLite chatbot | | [`10_document_loaders`](./10_document_loaders) | Document Loaders | TextLoader, CSVLoader, PyPDFLoader, WebBaseLoader, DirectoryLoader, SQLDatabaseLoader, WikipediaLoader | | [`11_text_splitters`](./11_text_splitters) | Text Splitters | CharacterTextSplitter, RecursiveCharacterTextSplitter, TokenTextSplitter (tiktoken), MarkdownHeaderTextSplitter, language-aware code splitter, SemanticChunker | | [`12_document_transformers`](./12_document_transformers) | Document Transformers | HTML cleaning & tag extraction with BeautifulSoupTransformer, semantic duplicate filtering with EmbeddingsRedundantFilter | | [`13_vector_stores`](./13_vector_stores) | Vector Stores | InMemoryVectorStore (CRUD), Chroma (persistent, CRUD), FAISS (save/load, CRUD), similarity search types (standard, MMR, scored, threshold) | | [`14_retrievers`](./14_retrievers) | Retrievers | WikipediaRetriever, VectorStoreRetriever, BM25Retriever, EnsembleRetriever (hybrid), MultiQueryRetriever, ContextualCompressionRetriever (LLM extractor), ParentDocumentRetriever, MultiVectorRetriever (summary index), FlashRank reranking | | [`15_rag`](./15_rag) | RAG Pipelines | Basic RAG (PDF ingestion), Wikipedia RAG (LCEL), RAG + message history + query rewriting, RAG + Pydantic citations, conversational RAG chatbot (CLI), long-context reordering | | [`16_tools`](./16_tools) | Tools & Toolkits | DuckDuckGo/Wikipedia/PythonREPL/Requests built-in tools, `@tool` decorator, StructuredTool, BaseTool class, manual tool-call execution loop, FileManagementToolkit, custom BaseToolkit, retriever-as-tool | | [`17_mcp`](./17_mcp) | Model Context Protocol (MCP) | FastMCP server, `@mcp.tool()`, STDIO transport, MultiServerMCPClient, local & remote (Streamable HTTP) tool discovery, multi-server aggregation | | [`18_agents`](./18_agents) | Agents | Custom agentic reasoning loop with tool dispatch, `create_agent` (built-in ReAct-style), `create_sql_agent` (natural language to SQL), MCP-powered agent via `MultiServerMCPClient`, retriever-as-tool RAG agent via `create_retriever_tool` | --- ## 📋 Prerequisites - **Python 3.10 or higher** - **Basic Python proficiency** (functions, classes, decorators, `async/await`) - **Familiarity with REST APIs and JSON** (used in tool labs) - **At least one LLM API key** (OpenAI, Anthropic, Google) — or [Ollama](https://ollama.com) install - **No prior LangChain experience required** --- ## 🚀 Installation ### 1. Clone the repository ```bash git clone https://github.com/zainulabidin1/langchain-masterclass.git cd langchain-masterclass ``` ### 2. Create and activate a virtual environment ```bash python -m venv .venv # Windows .venv\Scripts\activate # macOS / Linux source .venv/bin/activate ``` ### 3. Install all dependencies ```bash pip install -r requirements.txt ``` ### 4. Configure your environment ```bash # macOS / Linux / Windows (PowerShell) cp .env.example .env # Windows (CMD) copy .env.example .env ``` Then open `.env` and fill in your credentials (see [Configuration](#%EF%B8%8F-configuration) below). --- ## ⚙️ Configuration All configuration lives in the `.env` file at the project root. The two most important settings are `LLM_MODEL_PROVIDER` and `LLM_MODEL_NAME`. ```env # Choose your LLM provider: "openai", "google", "ollama", "anthropic", "huggingface" LLM_MODEL_PROVIDER="ollama" LLM_MODEL_NAME="gpt-oss:120b-cloud" # Choose your embeddings provider: "openai", "google", "ollama", "huggingface" EMBEDDINGS_MODEL_PROVIDER="ollama" EMBEDDINGS_MODEL_NAME="nomic-embed-text" # API Keys — fill in only the ones you plan to use OPENAI_API_KEY="" ANTHROPIC_API_KEY="" GOOGLE_API_KEY="" HUGGINGFACEHUB_API_TOKEN="" # Set if not running Ollama on the default localhost. Otherwise leave blank OLLAMA_BASE_URL="" # Identifies your requests to external services (e.g. Wikipedia, WebBaseLoader) # Some APIs reject or rate-limit requests with no User-Agent — safe to leave as default USER_AGENT="LangChainMasterClass/1.0" ``` ### How `model_factory.py` Works The `_common/model_factory.py` module is the backbone of the entire course. It reads `LLM_MODEL_PROVIDER` and `LLM_MODEL_NAME` from the environment at startup and exposes three factory functions: | Function | Returns | Notes | |----------|---------|-------| | `get_chat_model(**kwargs)` | `BaseChatModel` | Supports OpenAI, Google, Ollama, Anthropic, HuggingFace | | `get_embedding_model(**kwargs)` | `Embeddings` | Supports OpenAI, Google, Ollama, HuggingFace | All `**kwargs` are forwarded directly to the underlying provider class, so you can pass `temperature`, `max_tokens`, `streaming`, or any provider-specific parameter without modifying the factory. This is why every lab can override the model behavior (e.g., `get_chat_model(temperature=0.9)`) without touching the provider configuration. **In practice, every lab starts the same way:** ```python from _common.model_factory import get_chat_model # The entire codebase adapts to your .env file automatically model = get_chat_model(temperature=0) response = model.invoke("Hello, LangChain!") ``` Switch `LLM_MODEL_PROVIDER` in `.env` from `ollama` to `openai` or `anthropic` and every lab — no code changes required — runs against a different provider. --- ## ▶️ Running Labs Each lab is a self-contained Python script. To run any lab, activate your virtual environment and execute the file directly: ```bash # Example: Run from module folder cd 01_models python lab_01_chat_model.py # Example: Run from project root folder python 01_models/lab_01_chat_model.py ``` ### Recommended Progression Work through the modules in order. Within each module, labs are numbered sequentially (`lab_01`, `lab_02`, etc.) and each one introduces a specific concept that later labs build on. > **Tip:** If you are new to LangChain, start with [Module 1](./01_models) (Models) and follow the roadmap table top-to-bottom. If you are already familiar with LCEL and structured output, jump directly to [Module 9](./09_memory) (Memory). --- ## 🧩 Curriculum Every module includes a purpose statement, a lab-by-lab breakdown, and a "Key APIs" summary. 📖 **See [CURRICULUM](CURRICULUM.md) for the complete, module-by-module lab breakdown** For a quick overview, see the [Course Roadmap](#-course-roadmap) table above. --- ## 🎓 Skills Acquired After completing this curriculum you will be able to: - Architect and implement multi-provider LLM pipelines that hot-swap between OpenAI, Anthropic, Google, and Ollama models - Compose complex data transformation and reasoning pipelines with LCEL using parallel, sequential, conditional, and fallback patterns - Extract structured data from unstructured text using Pydantic schemas, TypedDict, and JSON via the native model function-calling API - Build non-blocking, token-streaming AI services using Python's async ecosystem - Instrument LLM pipelines with custom callbacks for logging, cost tracking, and real-time WebSocket delivery - Implement multi-session, multi-backend conversation memory (in-memory, SQLite, custom JSON) - Ingest, chunk, and index documents from PDFs, databases, the web, and file systems into semantic vector stores - Design and evaluate retrieval systems using vector search, BM25, hybrid ensemble, multi-query, contextual compression, parent-document, multi-vector, and cross-encoder reranking strategies - Build production RAG pipelines with query rewriting, citation extraction, and long-context mitigation - Define and compose tools using three different abstraction levels (`@tool`, `StructuredTool`, `BaseTool`) and organize them into reusable toolkits - Stand up FastMCP servers, discover tools dynamically over local STDIO and remote Streamable HTTP transports, and aggregate multiple MCP servers behind a single `MultiServerMCPClient` - Build autonomous agents from first principles and leverage LangChain's built-in agent constructors, including the SQL agent and MCP-powered agents --- ## ✅ Best Practices Demonstrated - **LCEL-first composition**: every pipeline uses the `|` pipe operator or explicit Runnable wrappers rather than legacy chain classes - **Provider abstraction**: `model_factory.py` prevents provider lock-in across all labs - **Pydantic for structured output**: schemas defined with field descriptions drive accurate model extraction - **Session-scoped memory**: `RunnableWithMessageHistory` with separate `session_id` keys correctly isolates users - **Token budget management**: `trim_messages()` integrated via `RunnablePassthrough.assign()` protects context windows in long conversations - **Fallback resilience**: `.with_fallbacks()` prevents hard failures when a primary provider is unavailable - **Retrieval precision**: child-chunk indexing + parent-document return in `ParentDocumentRetriever` balances search accuracy with full context - **Reranking for quality**: FlashRank cross-encoder reranking applied after broad vector retrieval improves final result relevance - **Citation grounding**: Pydantic-enforced citations with `source_id` and verbatim `quote` fields eliminate hallucinated references - **Lazy loading**: `DirectoryLoader.lazy_load()` avoids loading entire corpora into memory - **Cross-platform paths**: `path_helper.py` uses `pathlib.Path` throughout, never hardcoded strings --- ## ❓ Troubleshooting / FAQ **`ModuleNotFoundError` when running a lab** Your virtual environment isn't activated, or dependencies weren't installed into it. Re-run `source .venv/bin/activate` (for mac/linux) or `.venv\Scripts\activate` (for windows) followed by `pip install -r requirements.txt`. See [Installation](#-installation) for details. **`AuthenticationError` / `401` from OpenAI, Anthropic, or Google** The corresponding API key in `.env` is missing, empty, or invalid. Double-check you copied `.env.example` to `.env` (not just edited the example) and that there are no stray quotes or trailing spaces around the key. **"Connection refused" or timeout errors when `LLM_MODEL_PROVIDER="ollama"`** Ollama isn't running, or isn't serving on the expected host/port. Start it with `ollama serve`, check `OLLAMA_BASE_URL` in `.env` if you're not running it on `localhost`. **A lab hangs or is very slow with `LLM_MODEL_PROVIDER="ollama"`** Local models are compute-bound by your hardware. Try a smaller model or use ollama cloud hosted. **`NotImplementedError` or schema-validation errors in the structured output / tool-calling labs** Not every model supports native structured output or function/tool calling — this is a model limitation, not a bug in the labs. If you hit this, switch `LLM_MODEL_NAME` in `.env` to a model with native tool-calling support (e.g. `gpt-4o-mini`, `claude-*`, or an Ollama model explicitly tagged for tool use, such as `llama3.1` or `qwen2.5`) rather than a smaller/general-purpose model. **Errors or garbled responses in the multimodal / image-input labs** Not every model accepts image input — text-only models will error out or silently ignore the image. Make sure `LLM_MODEL_NAME` is set to a vision-capable model (e.g. `gpt-4o`, `claude-*`, `gemini-*`, or an Ollama vision model such as `llama3.2-vision`). **Embeddings and chat model providers don't have to match** `LLM_MODEL_PROVIDER`/`LLM_MODEL_NAME` and `EMBEDDINGS_MODEL_PROVIDER`/`EMBEDDINGS_MODEL_NAME` are configured independently — for example, you can chat with Anthropic Claude while embedding with a local Ollama model. Still stuck? Open an issue with the lab number, your `.env` provider settings (redact your keys), and the full error traceback. --- ## 🤝 Contributing Contributions are welcome and appreciated. Useful ways to contribute: - 🐛 **Bug fixes** — a lab throwing an error, a broken import, an outdated API call - ✏️ **Typo / clarity fixes** — corrections to explanations, comments, or documentation - 🧪 **New labs** — additional examples for existing modules, or edge cases the current labs don't cover - 🌍 **Translations** — localized versions of the README - 💡 **New modules** — proposals for topics not yet covered (open an issue to discuss scope first) See [CONTRIBUTING](CONTRIBUTING.md) for setup details, coding conventions, and the PR process. --- ## 🙏 Acknowledgements This course would not exist without the excellent work of the open-source community: - [**LangChain**](https://github.com/langchain-ai/langchain) and [**LangGraph**](https://github.com/langchain-ai/langgraph) — for the framework and abstractions this entire curriculum is built on - [**Model Context Protocol**](https://modelcontextprotocol.io) — for the open standard powering the MCP modules - [**Ollama**](https://ollama.com/) , [**HuggingFace**](https://huggingface.co/), and other model providers — for the models and APIs used throughout the labs - The broader **open-source Python and AI community**, whose libraries (FAISS, Chroma, FlashRank, and more) make labs like these possible --- ## 👤 Author Built and maintained by **Zain** [](https://www.linkedin.com/in/zainulabidin1/) --- ## 📄 License This project is licensed under the MIT License. See [LICENSE](LICENSE) for details. --- <p align="center"> ⭐ Found this helpful? <a href="https://github.com/zainulabidin1/langchain-masterclass/stargazers">Star the repo</a> to help more people find it. </p>