You Switched to Ollama. Your Data Is Still Leaving — Here's Where

You installed Ollama. Downloaded Mistral 7B or Llama 3. Watched the first token stream locally on your machine, no API key, no cloud, no Terms of Service checkbox. You felt the quiet satisfaction of a problem solved.

It wasn't solved.

The privacy threat in AI has migrated up the stack, and most tech workers who went local stopped looking too soon. The inference layer is private — yes. But inference is only one node in a chain that touches your filesystem, your IDE, your search results, and your document store. At every other node, data is leaving your machine, and odds are good you have no idea which nodes or where it's going.

This is the threat that doesn't show up in the "ChatGPT vs. local LLM" comparison articles. The conversation about AI privacy got stuck at the model layer and never moved.

The Self-Satisfaction Trap

Privacy-conscious engineers who run local LLMs tend to share a mental model: cloud AI bad, local AI good, therefore I am done. This binary is the problem.

It mirrors what happened with VPNs five years ago. People bought a VPN, checked the mental box marked "private browsing," and went on to log into every service they used — nullifying 80% of the protection. The VPN handled the transport layer. Everything above it was unchanged. The threat moved up the stack, but the user's attention stayed at the layer they'd just fixed.

Local LLMs handle one layer: the inference computation. Your prompt goes in, tokens come out, and none of it touches a third-party API. That's real, and it matters. But a modern AI workflow has six or seven more layers. Fixing one and calling it done is security theater — it feels like security without providing it.

Where Your "Private" AI Is Actually Sending Data

Let's work through a typical setup: VS Code with the Continue.dev extension, Ollama running Mistral locally, a small RAG pipeline indexing your project files. This is a reasonable, privacy-conscious stack. Here is what it actually does.

IDE telemetry. VS Code itself has telemetry enabled by default and sends usage metrics — including extension activity data — to Microsoft. Continue.dev's telemetry is opt-out, not opt-in. If you didn't explicitly set "telemetry.telemetryLevel": "off" in VS Code settings and toggle the Continue telemetry flag in its config file, your IDE usage patterns are being reported. This doesn't include your prompts, but it includes which features you use and how often — a behavioral fingerprint of your workflow.

Embedding APIs. This one catches nearly everyone. When you set up a RAG pipeline to give your local LLM context over your codebase or documents, the pipeline needs to turn text chunks into vector embeddings. Ollama can generate embeddings locally with ollama embeddings. Most tutorials, LangChain defaults, and LlamaIndex starter configs do not use local embeddings. They call OpenAI's text-embedding-3-small or a similar cloud API. Every document chunk you index — your code, your internal notes, your client contracts — gets sent to that API. The LLM never saw it, so you felt private. The embedding model, which is running in a datacenter, processed every word.

RAG retrieval-augmented search. Even if your embeddings are local, your retrieval step may trigger external calls. Web-browsing tools built into some orchestration frameworks will fire off search requests, pass your query to a web search API (Bing, SerpAPI, Brave Search), and return results. Your query — which may contain proprietary terminology or project names — just went to a third party.

Model download telemetry and update checks. Ollama phones home on startup to check for model updates. LM Studio has a similar behavior. These calls are minimal and don't contain prompt data, but they confirm to the endpoint that you are running specific models — a usage fingerprint.

Clipboard and context manager leaks. Many AI setups use clipboard-aware features — copy text, paste into prompt, get answer. If you run a clipboard manager that syncs to cloud (the macOS Clipboard sync via iCloud, Alfred Powerpack sync, or similar), your prompts are being copied to cloud storage as a side effect of your clipboard workflow. This is entirely outside the AI stack, and almost nobody audits it.

The RAG Pipeline Is the Real Problem

Of the leaks above, the embedding API issue deserves the most attention because it's the most comprehensive and the least obvious.

RAG works by splitting documents into chunks, generating a vector for each chunk, storing those vectors, and then at query time finding the chunks most similar to your question and injecting them into the prompt. The LLM only ever sees the retrieved chunks and your question — so a local LLM genuinely does keep inference private. But the entire document had to be embedded first.

If you indexed your company's product roadmap to give your AI assistant context, and you used the default OpenAI embedding call, OpenAI processed the roadmap. OpenAI's API terms of service as of early 2026 do not use API data for training by default — that's a genuine improvement — but the data was transmitted to and processed on their infrastructure. You made a decision about that data without realizing it.

The same is true for code. Engineers frequently index entire repositories to enable AI-assisted navigation and refactoring. Every function, every comment, every TODO with an internal system name went through the embedding API.

Auditing Your Actual Data Flow

The fix starts with knowing what's actually happening on your machine. Here is a practical audit protocol.

Step 1: Run a traffic capture during AI operations. Use mitmproxy or Charles Proxy to intercept your own traffic. Start a capture session, run a typical AI workflow — open your IDE, run a RAG query, let the assistant answer a question — then stop the capture and read the log. Every outbound connection during that session is visible. This takes twenty minutes and will surprise you.

Step 2: Read your framework's default config. If you use LangChain, LlamaIndex, or Haystack, open the docs for whatever embedder and retriever you initialized. If the class name contains "OpenAI", "Cohere", "HuggingFaceInferenceAPI", or "Pinecone", it is calling a cloud API. The local alternative classes are usually named with "Local", "Ollama", or "FAISS" (for local vector storage).

Step 3: Audit IDE extension permissions and telemetry. In VS Code: File → Preferences → Settings, search for "telemetry". Set telemetry.telemetryLevel to "off". Check each AI extension's documentation for its own telemetry setting — Continue, Codeium, Cursor, and others each have independent controls.

Step 4: Inventory clipboard sync. Check System Preferences → iCloud → iCloud Drive and look for clipboard sharing. Check any clipboard manager applications (Alfred, Raycast, Paste) for cloud sync settings and disable them if your clipboard will contain AI prompts with sensitive content.

Building a Stack That Stays Local

Once you know what's leaking, you can fix it without abandoning the functionality. Here is what a genuinely air-gapped AI workflow looks like.

Local embeddings: Replace cloud embedding calls with ollama embeddings using nomic-embed-text or mxbai-embed-large. Both run on consumer hardware and produce embeddings comparable in quality to small cloud models. In LangChain, swap OpenAIEmbeddings for OllamaEmbeddings. In LlamaIndex, use OllamaEmbedding.

Local vector store: Use Chroma or FAISS as your vector database, both of which run entirely in-process or as a local server. No external calls.

Local web search fallback: If your agent needs to search the web, SearXNG is a self-hostable meta-search engine you can run on your local network. It queries public search engines without logging queries to a third-party service. For cloud searches where you must use a service, Perplexity AI with its privacy mode enabled is the most defensible external option — it supports incognito-style queries that aren't tied to your account.

Encrypted file storage for AI context documents: If you maintain a personal knowledge base, context documents, or exported conversations that you store and sync across machines, those files deserve encryption at rest. Tresorit provides end-to-end encrypted cloud storage with client-side encryption that the provider cannot read — meaningfully different from Dropbox or Google Drive. This is the right home for AI context files, RAG source documents, and exported conversations you want to keep.

Affiliate Disclosure: This article may contain affiliate links. If you make a purchase through these links, we may earn a small commission at no extra cost to you. We only recommend products we genuinely believe in. This helps support our work and allows us to continue providing free content.

Disable telemetry at the IDE layer: Beyond VS Code settings, create a ~/.continue/config.json entry: "allowAnonymousTelemetry": false. For Cursor, check Settings → Privacy → Telemetry. Do this as a one-time setup step, not something to revisit — then it's done.

The Deeper Reframe

Here is the Fox angle worth sitting with: the privacy conversation around AI has been captured by the model vendors. Every discussion about ChatGPT vs. Claude vs. Gemini vs. local models focuses on whose inference you trust. That's a useful question. But it let the tools and frameworks around AI — the orchestration layer, the embedding layer, the IDE layer — accumulate data access without scrutiny.

Privacy-conscious engineers who correctly identified inference as a risk fixed inference and stopped there. The vendors of everything else — Anthropic, OpenAI, the embedding APIs, the IDE extension makers — benefited from that attention being focused on their competitors rather than on them.

Data sovereignty isn't a single toggle. It's an audit practice. The question isn't "is my LLM local?" — it's "where does data flow in my entire AI pipeline, and do I consent to each hop?"

Most people who think they've solved AI privacy have solved 20% of the problem at the layer that was easiest to fix and best-marketed to them. The remaining 80% is in the plumbing nobody told them to check.

Run the traffic capture. Read the framework defaults. Audit the clipboard. The threats you can see, you can fix.

---

Last updated: 2026-05-29

---

Take back your AI stack.

Get our private AI setup checklist — a step-by-step guide to auditing and hardening your local LLM environment, including the mitmproxy audit script, LangChain config snippets for fully local RAG, and a telemetry disable checklist for the twelve most popular AI dev tools.

---