You switched to open-source AI because you didn't want your prompts fed into a training pipeline. Smart move. But here's the part most guides skip: open source means the code is visible, not that your data is private.

There's a meaningful gap between those two things — and inside that gap, a surprising amount of information leaks out of setups that look air-tight on the surface. This article is about closing that gap.

The Fox angle here isn't "open source is bad." It's that the privacy community has conflated code transparency with usage privacy. They're different properties, and treating them as equivalent leaves real holes in otherwise careful setups.

Last updated: 2026-06-25

"Open Source" Describes the Model Weights, Not Your Infrastructure

When Llama 3, Mistral, or Qwen releases under an open license, what's open is the model architecture and the trained weights. You can inspect them, modify them, redistribute them. That transparency is genuinely valuable.

What open source does not describe:

How your inference software handles your queries
Where your model runner reports telemetry
What your Python dependencies do with network access
Whether your outputs stay local after generation

Most people running "local" open-source AI are actually running it through a layer of software they've never audited. That software — Ollama, LM Studio, Open WebUI, LangChain, HuggingFace transformers — is the actual surface area that matters for privacy.

Open source weights running on leaky infrastructure is not a private setup. It's a private model with a surveillance wrapper.

Ollama Has Telemetry — And It's Opt-Out, Not Opt-In

Ollama is the most popular way to run local models. It's excellent software. It also phones home by default.

Ollama collects aggregate usage metrics — model names pulled, request counts, hardware info. The data is anonymized, and Ollama is honest about it in their docs. But "anonymized aggregate telemetry" reported from a home IP address tied to a professional email account is meaningfully different from "no data leaves your machine."

To disable it:

```bash

Add to your shell profile

export OLLAMA_TELEMETRY=false

```

If you're running Ollama as a system service, set it in the service definition:

```bash

macOS launchd plist

EnvironmentVariables

OLLAMA_TELEMETRY

false

```

LM Studio has similar telemetry controls buried in preferences. Open WebUI, if self-hosted, has no outbound telemetry by default — but verify your deployment isn't behind a reverse proxy that logs request bodies.

The broader point: every layer of your stack is a potential leak. Default-on telemetry in inference runners is the most common one that privacy-conscious users miss because they stopped checking once they confirmed the model weights were local.

HuggingFace Tracks Every Model Pull

When you run huggingface-cli download or use the transformers library to pull a model, HuggingFace logs that request. They see your IP, the model name, the timestamp, and the version. This is standard CDN behavior, but it means your model usage history exists in HuggingFace's infrastructure.

For most people, this is acceptable. For threat models that include "don't let anyone know I'm working with [specific model]," it's a real exposure.

Mitigations:

Download once over a VPN, then work fully offline. HuggingFace only logs the pull, not your inference runs.
Use a local mirror. huggingface-mirror lets you host your own model registry. Overkill for individuals, appropriate for teams.
Download models from direct releases (model cards often link to direct weights on GitHub or the researcher's own CDN) rather than through the HuggingFace Hub API.

```bash

Pull a model once, then disable HF network access for your inference environment

export HF_HUB_OFFLINE=1

export TRANSFORMERS_OFFLINE=1

```

These environment variables force transformers and diffusers to use only locally cached models — no network calls during inference.

Python Dependencies Are Not Audited

If your AI pipeline is built with Python, you're running code from dozens of packages you haven't reviewed. Some of those packages have legitimate telemetry. Some have been compromised via supply chain attacks. A few are just badly written and accidentally leak data.

LangChain, as one example, had a period where its tracing features — designed to send execution traces to LangSmith — were enabled by default in certain configurations. If you were running LangChain without explicitly disabling LANGCHAIN_TRACING_V2, your prompt chains were potentially being sent to Langsmith's servers.

This isn't a LangChain-specific problem. It's a "modern Python ML tooling" problem. The ecosystem moves fast, defaults change between minor versions, and "opt-out telemetry" appears in package changelogs that nobody reads.

Practical mitigations:

Run inference in a network-isolated environment — a VM with no internet access, or unshare -n on Linux to sandbox a process from the network:

```bash

Run your inference script with no network access

unshare -n python your_inference_script.py

```

Pin your dependencies and review changelogs before upgrading
Use pip install --no-index --find-links=./wheels/ to run entirely from a local wheel cache

Your Outputs Are the Biggest Leak

Here's the one that trips up experienced engineers: even with a fully local, telemetry-free inference setup, your outputs go where you send them.

If you're using a local LLM to draft emails and you paste the output into Gmail, that draft is now in Google's infrastructure. If you're using it to summarize documents and you save the summaries to Dropbox, those summaries — and potentially the patterns that reveal the source documents — are in Dropbox's infrastructure.

The model running locally is not the end of the data pipeline. The end is wherever the output lands.

For genuinely sensitive work:

Draft and review entirely in local editors (Obsidian with local vault, VS Code with no telemetry extensions)
Use encrypted storage for both inputs and outputs before they touch any sync service

Proton Drive provides end-to-end encrypted cloud storage, which means even Proton can't read what you upload. For AI-generated content that needs to leave your machine — summaries, research notes, client deliverables built with local AI assistance — E2EE cloud storage is the correct boundary, not "cloud storage from a company I trust."

Affiliate Disclosure: This article may contain affiliate links. If you make a purchase through these links, we may earn a small commission at no extra cost to you. We only recommend products we genuinely believe in. This helps support our work and allows us to continue providing free content.

When Cloud AI Is the Honest Choice

Not every use case belongs on a local model. If you're running a 70B parameter model on consumer hardware for latency-sensitive tasks, you're going to be frustrated. Sometimes cloud inference is genuinely the right call.

If you're going to use cloud AI, the meaningful privacy differentiators are:

No training on your data (contractual guarantee, not a feature)
Minimal logging with short retention windows
No requirement to create an account to run queries

Perplexity offers a privacy-focused search and AI mode where queries are not used for training and are processed with reduced logging. For research tasks where local models underperform — multi-source synthesis, up-to-date information retrieval, citation-heavy research — it's a more honest choice than defaulting to ChatGPT or Gemini and assuming the privacy posture is equivalent.

The Fox reframe: using cloud AI with real privacy controls is often more private than running open-source AI through a telemetry-laden inference stack on your home machine. The model's license doesn't determine your privacy posture. Your infrastructure does.