Air-Gapped Deployment

DocsGPT runs without internet access once three things are inside your network: the software, the embedding models, and an LLM you host yourself. This guide covers bringing them in, the settings that switch off features that need the internet, blocking outbound traffic, and checking that nothing gets out.

What needs the internet

With a self-hosted LLM, chat, document ingestion and retrieval work offline. These parts of DocsGPT contact external services by default and need a setting:

Features whose purpose is to fetch outside content cannot work offline: URL, crawler, sitemap, GitHub and Reddit sources, the Google Drive, SharePoint and Confluence connectors, web search and other internet-facing tools, and hosted LLM or embedding providers.

1. Bring the software in

Docker

The backend image already contains everything the default configuration loads at run time: both default embedding models (granite for new installs, mpnet for upgrades), their tokenizers and the token-counting encoding. A container downloads nothing on first use.

On a machine with internet access, pull the images and save them to one file:

TAG=latest   # or a release, e.g. 0.19.0
docker pull arc53/docsgpt:$TAG
docker pull arc53/docsgpt-fe:$TAG
docker pull redis:6-alpine
docker pull postgres:16-alpine
docker save -o docsgpt-images.tar \
  arc53/docsgpt:$TAG arc53/docsgpt-fe:$TAG redis:6-alpine postgres:16-alpine

Copy docsgpt-images.tar and the standalone Compose file into the air-gapped network, then load the images (or push them to your internal registry):

docker load -i docsgpt-images.tar

Set DOCSGPT_IMAGE_TAG to the same tag when you start the stack. Bring your model server’s image and weights in the same way, following its own documentation.

To use an embedding model other than the two defaults, build your own backend image with it baked in: docker build -f docsgpt/Dockerfile --build-arg EMBEDDINGS_PREFETCH=<model> .

pip

On a connected machine with the same operating system, CPU architecture and Python version as the target, download the packages and the models:

python -m venv .venv && source .venv/bin/activate
pip download docsgpt -d wheelhouse
pip install --no-index --find-links wheelhouse docsgpt
DOCSGPT_HOME=./docsgpt-home docsgpt prefetch-models

prefetch-models stores both default embedding models and their tokenizers under models/ in the data home. Pass model names to fetch a different set.

Copy wheelhouse/ and docsgpt-home/models/ to the target, install from the wheelhouse, and put models/ in the target’s data home (or point EMBEDDINGS_CACHE_DIR at it):

pip install --no-index --find-links wheelhouse docsgpt

2. Point DocsGPT at your own LLM

Run an OpenAI-compatible model server inside the network, such as vLLM, Ollama, a llama.cpp server or Text Generation Inference, and set:

LLM_PROVIDER=openai
OPENAI_BASE_URL=http://llm.internal:8000/v1
LLM_NAME=your-model-name
API_KEY=None

Setting OPENAI_BASE_URL also removes DocsGPT’s hosted model from the model list. See Local inference engines for the base URL of each server.

3. Switch off the features that call out

Add these to .env:

VERSION_CHECK=0
TTS_PROVIDER=none
STT_PROVIDER=none
DEFAULT_CHAT_TOOLS=["memory","scheduler"]
HF_HUB_OFFLINE=1

• VERSION_CHECK=0 stops the worker’s anonymous version check.
• TTS_PROVIDER=none hides the Speak button on answers.
• STT_PROVIDER=none hides the microphone button, and audio files fail to ingest with a message naming the setting. If your model server offers an OpenAI-compatible /v1/audio/transcriptions endpoint, keep STT_PROVIDER=openai instead: speech-to-text uses OPENAI_BASE_URL too.
• DEFAULT_CHAT_TOOLS defaults to memory, read_webpage and scheduler; this list drops read_webpage.
• HF_HUB_OFFLINE=1 makes the Hugging Face libraries read models from the local cache only. A model that was never fetched fails at once with an error, instead of waiting on a connection that never opens. The libraries read it from the process environment: Docker Compose’s env_file provides that, and for pip installs export it in the environment that starts docsgpt api and docsgpt worker.

Leave the document parsing and OCR settings at their defaults.

4. Block outbound traffic in the network

DocsGPT doesn’t block outbound connections itself. Enforce that in the network, where the same rule also covers Postgres, Redis, your model server and anything you add later.

Hosts with no route out. The simplest setup: servers whose firewall allows only internal address ranges and your internal DNS.

Docker. Attach the services to a network created with internal: true, which has no route out of the Docker host. A container on an internal network can’t publish ports, so give only a reverse proxy (nginx, Caddy, Traefik) a second, regular network and serve the UI and API through it:

networks:
  airgap:
    internal: true
  edge: {}
 
services:
  backend:
    networks: [airgap]
  worker:
    networks: [airgap]
  frontend:
    networks: [airgap]
  postgres:
    networks: [airgap]
  redis:
    networks: [airgap]
  proxy:
    image: nginx:stable
    networks: [airgap, edge]
    ports: ["443:443"]

Docker’s published ports bypass host firewalls such as ufw. If you rely on host rules instead of internal networks, filter container traffic in the DOCKER-USER iptables chain.

Kubernetes. With a network plugin that enforces NetworkPolicy, deny egress for the namespace except to its own pods and cluster DNS:

apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: deny-external-egress
  namespace: docsgpt
spec:
  podSelector: {}
  policyTypes: [Egress]
  egress:
    - to:
        - podSelector: {}
    - to:
        - namespaceSelector: {}
          podSelector:
            matchLabels:
              k8s-app: kube-dns
      ports:
        - protocol: UDP
          port: 53
        - protocol: TCP
          port: 53

Add a rule for your model server if it runs in another namespace.

Egress proxy. When some traffic must leave the network, for example to a hosted LLM, send it through a forward proxy that allows only those hosts. The Python HTTP libraries DocsGPT uses honour the standard variables; exclude internal services in NO_PROXY:

HTTPS_PROXY=http://proxy.internal:3128
HTTP_PROXY=http://proxy.internal:3128
NO_PROXY=localhost,127.0.0.1,backend,llm.internal

• Features that fetch user-supplied URLs connect to the resolved IP address, so a hostname allowlist can’t match them. Keep those features off.
• If the proxy inspects TLS, add its CA certificate with SSL_CERT_FILE and REQUESTS_CA_BUNDLE. For model downloads through such a proxy, also set HF_HUB_DISABLE_XET=1: Hugging Face’s transfer client ignores SSL_CERT_FILE.

5. Verify

Check that the models load with networking disabled:

docker run --rm --network none arc53/docsgpt:$TAG python -m docsgpt.scripts.verify_offline

On a pip install, run docsgpt verify-offline on the air-gapped host. Both check token counting, each default embedding model’s tokenizer, and an embedding; pass a model name to check a different model.

Then use the app: upload a document and ask a question about it. Your firewall or proxy logs should show no connection attempts from DocsGPT to outside hosts. The web UI loads its fonts and scripts from the DocsGPT server only.