MCP Context Server

Overview

MCP Context Server is a high-performance Model Context Protocol server that gives LLM agents a durable, searchable memory layer beyond their context window. Agents working on the same task share context through thread-scoped storage backed by SQLite or PostgreSQL, search it with full-text, semantic, or hybrid retrieval, and let a cross-encoder reranker sharpen the top results — all behind a standard MCP interface that Claude Code, LangGraph, and other MCP-compatible clients understand out of the box.

The full source is on GitHub under the MIT license, published on PyPI as mcp-context-server, and listed in the official MCP Registry.

The Problem

While building a multi-agent system on top of Claude Code, I kept running into the same three friction points around context:

• The orchestrator lost details passing the user's request to subagents. Orchestrators naturally summarize, reformulate, and compress what they receive; important specifics can go missing before a subagent ever reads them. Subagents needed a way to reach the original, verbatim user message, not a paraphrase of it.
• Subagents struggled to hand their results back. A planning agent that produced a detailed plan might return only a summary; even when it returned everything, the orchestrator then compressed it before passing anything downstream. Information degraded at every hop.
• Agent-to-agent context exchange was effectively impossible. The orchestrator sat between every pair of agents and trimmed whatever went through it, so two subagents could not share rich context without going through a bottleneck that stripped it.

Initial Attempt: Markdown Files

The obvious first attempt — the one most people reach for — was Markdown files on disk: each agent writes its notes, plans, and results into a shared folder, and downstream agents read them. In practice, it didn't scale. Agents wandered off naming conventions the moment the task shape changed. Drafts, scratch notes, and intermediate reports accumulated and nobody cleaned them up. Two agents writing to similar-looking filenames started tripping over each other. There was no query layer — a downstream agent looking for "the most recent plan for task X" had to read a directory listing and hope the filenames told the truth. And there was still no way to address a single entry precisely (by id, by metadata, by date) without reinventing half of a database on top of the filesystem.

That's when it became clear the right shape was a server with a database behind it, exposing a familiar MCP interface — so agents could use context storage the same way they already use any other MCP tool, and so the storage itself could enforce structure, identity, and queryability instead of relying on convention.

The Solution

MCP Context Server treats agent context as first-class durable data. Entries are grouped by thread_id (one thread per task, project, or session), so agents working on the same task share a common context pool; cross-thread discovery is available when you want it. Each entry carries text, optional images, metadata, tags, and a timestamp, and is addressable by id for exact retrieval.

The search story is layered. A full-text branch indexes content with SQLite FTS5 or PostgreSQL tsvector (stemming, boolean operators, phrase matching, prefix queries, ranked with BM25 or ts_rank). A semantic branch embeds content with pluggable providers — Ollama, OpenAI, Azure OpenAI, HuggingFace, or Voyage — stored in sqlite-vec or pgvector, ranked with L2 distance. A hybrid branch runs both in parallel and fuses results with Reciprocal Rank Fusion, so documents that rank well on both signals float to the top. A cross-encoder reranker (FlashRank by default, provider-pluggable) then re-scores the candidates the way a reranker actually wants to — on the passage that matters, not the first N characters of the document.

Two pieces of the retrieval design are worth calling out explicitly, because they go a bit further than what I've seen documented in public RAG libraries:

• Pointer-based chunking for embeddings. The common parent-child retrieval pattern stores child chunks as text in the vector database alongside the parent document — effectively duplicating content for every chunk. Here I keep only embeddings plus character-offset pointers (start_index, end_index) into the parent text; the child chunk text is sliced on demand from the parent when reranking needs it. This saves both storage and time, and it's an approach I haven't seen documented elsewhere as a default in the RAG libraries I've surveyed.
• Highlight-driven passage formation for FTS reranking. Cross-encoder rerankers work on short passages, so you have to feed them good ones. Instead of handing the reranker an arbitrary prefix of the document, I parse the FTS engine's native highlight output (the <mark>...</mark> tags from SQLite FTS5 or PostgreSQL's ts_headline), expand each match to the nearest natural-language boundary (paragraph → line → sentence → word, in that priority), merge overlapping or nearby windows, and hand the resulting clean passage to the reranker. Every reranked FTS candidate arrives at the cross-encoder with a passage that actually contains the reason it matched — a specific bridging design I haven't seen documented elsewhere in public RAG literature.

For the complete tool surface and semantics, see the API reference — 13 MCP tools across core operations, search, and batch.

Features and Capabilities

• Thread-scoped storage with exact-by-id retrieval and optional cross-thread discovery.
• Multimodal entries — text plus optional images — stored uniformly.
• Metadata filtering with 16 operators (equality, comparison, membership, existence, substring, array-contains, null checks, and more) over arbitrary JSON fields on each entry.
• Date-range filtering on creation timestamps using ISO 8601.
• Tag organization with normalized indexed tags.
• Batch operations for high-throughput store, update, and delete.
• Full-text search on SQLite FTS5 or PostgreSQL tsvector with stemming, boolean operators, phrase matching, and prefix queries, ranked with BM25 or ts_rank.
• Semantic search backed by sqlite-vec or pgvector, with five pluggable embedding providers (Ollama, OpenAI, Azure OpenAI, HuggingFace, Voyage) — swap providers with one environment variable.
• Hybrid search fusing full-text and semantic results with Reciprocal Rank Fusion so documents that rank well on both signals surface first.
• Cross-encoder reranking across every search mode, with pluggable providers (FlashRank by default).
• LLM-powered summary generation with three pluggable providers (Ollama, OpenAI, Anthropic) to compress long entries on demand.
• Bearer-token authentication for HTTP transport deployments.
• Two database backends — SQLite (zero-config, local) or PostgreSQL (production-grade, high-concurrency) — selectable by one environment variable.
• Three deployment shapes — direct stdio for local Claude Code, Docker Compose for HTTP deployments, and a Helm chart for Kubernetes.

Capabilities Unlocked

Putting this in front of a multi-agent system changed what was practical:

• Lossless context transfer between user and agents, and between agents. A Claude Code hook now captures every user message verbatim into the server, so subagents can retrieve the original request rather than the orchestrator's paraphrase of it. Agents store their work products the same way, so downstream agents read what was actually produced, not what survived the orchestrator's compression.
• Durable storage in place of fragile files. Entries live in SQLite or PostgreSQL with WAL mode or MVCC, strategic indexing, and async operations. Nothing depends on a contributor sticking to a Markdown naming convention.
• Search over accumulated experience. With thread-scoped queries and cross-thread discovery, agents can look up how a similar problem was handled last time — across a single project, or across every project the same server has served. Good decisions and hard-won patterns compound instead of getting forgotten.
• A clean substrate for context engineering. Retrieval primitives (metadata filters, date ranges, tag logic, hybrid search, reranking, summarization) land in one place behind a stable MCP tool surface, so agents don't reimplement them and projects don't drift.
• Broad client compatibility. Works with Claude Code, LangGraph, and any MCP-compatible client — the server exposes standard MCP tools that clients use the same way they use any other MCP server, so integration cost is effectively zero.

Full documentation lives in the project's docs directory.

Origin Story

The direct trigger was the multi-agent system I was building on top of Claude Code. Every round of work surfaced the same three pain points — orchestrators lost details when passing the user request to subagents, subagents lost details when handing results back, and agent-to-agent context simply didn't flow without going through a bottleneck that stripped it. Markdown files failed as a workaround, for the reasons in "Initial Attempt" above. What the system actually needed was a queryable, durable, structured context store that every agent could reach through a standard protocol.

MCP was the natural interface. Claude Code and other MCP-compatible clients already treat MCP tools as first-class citizens, so exposing context storage as MCP tools meant agents could use it without any custom plumbing. A small set of storage and search primitives (store, retrieve, update, delete, FTS, semantic, hybrid, summarize, batch) covered every context-handoff scenario the multi-agent system ran into. Once the primitives were in place, context engineering stopped being "paste it in and hope for the best" and became a real, queryable, durable substrate — good enough that I now rely on it as the memory layer for AEGIS, the multi-agent Claude Code environment I build on top of it.

The project lives on my personal GitHub under an MIT license because the MCP and LLM-agent ecosystem benefits from shared infrastructure. The more people who can reach for a well-tested persistent-memory primitive instead of reinventing Markdown-files-on-disk, the healthier the agent-tooling landscape becomes.

My Role

I designed, built, and maintain MCP Context Server end-to-end as a solo open-source project — the storage schema for SQLite and PostgreSQL, the chunking and pointer-based embedding design, the passage-formation algorithm for FTS reranking, the Reciprocal Rank Fusion implementation, the metadata-filter engine, the pluggable embedding, summary, and reranking provider abstractions, the async MCP tool layer on FastMCP, the test suite and strict type-checking, and the release automation. It lives on my personal GitHub under an MIT license, and I accept issues and contributions from the wider MCP and LLM-agent community.