Last updated: 2026-06-05

The Short Version: Your RAG Pipeline Is Probably Phoning Home

If you stood up a RAG pipeline this week using LangChain or LlamaIndex tutorials, your documents, user queries, and chain execution traces are very likely leaving your infrastructure right now. Not because of a breach. Because that is the default.

Both frameworks are optimized for developer velocity, which means they default to cloud APIs, managed observability platforms, and hosted vector stores. Privacy requires configuration, and configuration requires knowing where to look. Most tutorials never tell you.

This audit covers six concrete leakage vectors, where each one lives in the code, and a drop-in replacement for each.

Leakage Vector 1: LangSmith Tracing (LangChain)

This is the most aggressive default and the one most developers miss entirely.

When the environment variable LANGCHAIN_API_KEY is present, LangChain v0.2+ automatically enables LangSmith tracing. Every chain invocation, every LLM call, every tool call, and every prompt template rendered — including the full text of those prompts — is shipped to LangSmith's cloud at api.smith.langchain.com. This happens silently in the background.

If you are building a RAG system over sensitive documents (legal, medical, financial), and you set LANGCHAIN_API_KEY to test something once, you are now continuously streaming query+context pairs to a third-party service.

The fix:

```python

Explicitly disable tracing regardless of env vars

import os

os.environ["LANGCHAIN_TRACING_V2"] = "false"

os.environ["LANGCHAIN_API_KEY"] = "" # belt-and-suspenders

Or use the SDK-level toggle

from langchain_core.callbacks import CallbackManager

Pass an empty callback manager to individual chains

chain = your_chain.with_config({"callbacks": []})

```

For teams that want observability without cloud egress, self-host LangSmith's open-source langsmith-server Docker image or use Langfuse — a privacy-respecting LangSmith alternative you can run on your own hardware.

Affiliate Disclosure: This article may contain affiliate links. If you make a purchase through these links, we may earn a small commission at no extra cost to you. We only recommend products we genuinely believe in. This helps support our work and allows us to continue providing free content.

The Privacy-Hardened Baseline Configuration

Here is a minimal, fully local RAG pipeline that eliminates all six leakage vectors. No data leaves your machine.

```python

import os

Block LangSmith before any imports

os.environ["LANGCHAIN_TRACING_V2"] = "false"

os.environ["LANGCHAIN_API_KEY"] = ""

os.environ["LLAMA_INDEX_DISABLE_TELEMETRY"] = "1"

from langchain_ollama import ChatOllama, OllamaEmbeddings

from langchain_chroma import Chroma

from langchain.chains import RetrievalQA

from langchain.callbacks import CallbackManager

Local LLM — no API calls

llm = ChatOllama(model="llama3.2:3b", temperature=0)

Local embeddings — no API calls

embeddings = OllamaEmbeddings(model="nomic-embed-text")

Local vector store — no cloud

vectorstore = Chroma(

collection_name="private_docs",

embedding_function=embeddings,

persist_directory="./chroma_db",

)

No callback handlers registered

qa_chain = RetrievalQA.from_chain_type(

llm=llm,

retriever=vectorstore.as_retriever(),

callbacks=[], # explicit empty — no globals

)

```

Run python -m mitmproxy in a companion terminal and verify zero outbound HTTPS connections during a query. That is your acceptance test.

Verification Checklist

Before deploying any RAG pipeline handling sensitive data, run through this list:

[ ] LANGCHAIN_TRACING_V2=false confirmed in environment
[ ] LLAMA_INDEX_DISABLE_TELEMETRY=1 confirmed in environment
[ ] Embedding model is local (HuggingFace or Ollama) — no OpenAI/Cohere calls during indexing
[ ] Vector store is local (Chroma persistent, Qdrant local, FAISS) — no Pinecone/Weaviate Cloud
[ ] Callback handler list inspected and contains only explicitly added handlers
[ ] LLM is local (Ollama) or is a provider with a DPA and data retention policy you have reviewed
[ ] Network traffic audited with mitmproxy or equivalent during a test query

The defaults in these frameworks are not adversarial — they exist because cloud services are easier to demo. But easy defaults and private defaults are not the same thing. Audit once, harden at initialization, and your pipeline stays yours.

Stay Ahead of the Next Leak

New integrations ship weekly in both ecosystems. The fastest way to catch a new default sending data somewhere unexpected is to subscribe to dependency changelogs and re-run your network audit after every pip install --upgrade.

Get our RAG Privacy Hardening Checklist — updated whenever a new leakage vector surfaces in either framework:

Get the RAG Privacy Checklist →

No noise. One email when something important changes.