Setting Up a Development Environment

This guide will walk you through setting up a development environment for DocsGPT. This setup allows you to modify and test the application’s backend and frontend components.

1. Spin Up Postgres and Redis

For development purposes, you can quickly start Postgres and Redis containers. Postgres is the user-data store for DocsGPT (conversations, agents, prompts, sources, attachments, workflows, logs, and token usage), and Redis is used as the cache and Celery broker. We provide a dedicated Docker Compose file, docker-compose-dev.yaml, located in the deployment directory, that includes only these essential services. Once POSTGRES_URI points at it (step 1 of section 2 below), the backend applies the Alembic schema automatically on first boot (AUTO_MIGRATE=true / AUTO_CREATE_DB=true ship enabled), so no separate migration step is required. You can still run python scripts/db/init_postgres.py or docsgpt migrate (python -m docsgpt migrate without installing) explicitly if you prefer.

You can find the docker-compose-dev.yaml file here .

Steps to start Postgres and Redis:

1. Navigate to the root directory of your DocsGPT repository in your terminal.

2. Run the following command to start the containers defined in docker-compose-dev.yaml:

   docker compose -f deployment/docker-compose-dev.yaml up -d

   This starts Postgres and Redis in detached mode, running in the background. When the API boots against the fresh Postgres instance, it will automatically create the database (if missing) and apply the current Alembic schema.

2. Run the Backend

To run the DocsGPT backend locally, you’ll need to set up a Python environment and install the necessary dependencies.

Prerequisites:

• Python 3.12: pyproject.toml requires 3.12 or newer, and CI tests on 3.12. Check your version with python --version or python3 --version.
• uv  (recommended): it installs the locked dependencies and the docsgpt command in one step. Plain pip works too.

Steps to run the backend:

1. Configure Environment Variables:

   DocsGPT reads its settings from environment variables and from a .env file in the repository root. Start from the template:

   cp .env-template .env

   Then edit .env and set the two values a development setup needs:

   # The Postgres started in section 1 (credentials from deployment/docker-compose-dev.yaml)
   POSTGRES_URI=postgresql://docsgpt:docsgpt@localhost:5432/docsgpt
   # Any random string; generate one with: openssl rand -hex 32
   INTERNAL_KEY=<paste the generated value>

   POSTGRES_URI is commented out in the template, and without it the backend skips its database setup. INTERNAL_KEY is how the worker hands a finished index back to the API, so without it every upload fails after parsing. The API and the worker read the same .env, so they share the key.

   To pick a model provider, set LLM_PROVIDER and API_KEY as described in Cloud providers or Local inference. To work without any provider or key, run the backend with docsgpt dev --mock-llm (step 4), which answers from a local mock.

   You can also export any setting in your shell instead of writing it to .env. The App Configuration guide explains the common settings, and the Settings Reference lists every one, generated from the definitions in docsgpt/core/settings/.

2. Install the Backend:

   From the repository root, with uv:

   uv sync

   This creates .venv, installs the dependencies pinned in uv.lock together with the test tools, and installs DocsGPT itself in editable mode, which puts the docsgpt command in .venv/bin. Activate the environment (source .venv/bin/activate, or .venv\Scripts\activate on Windows) or prefix commands with uv run, as in uv run docsgpt dev. uv uses a Python 3.12 or newer that it finds on your system, or downloads one; uv sync --python 3.12 pins the version CI uses.

   With pip instead, create a virtual environment, install the exported requirements, then install the checkout itself so the docsgpt command exists:

   python -m venv .venv
   source .venv/bin/activate          # Windows: .venv\Scripts\activate
   pip install -r docsgpt/requirements.txt
   pip install -e .

   Without the last line, run the commands in this guide as python -m docsgpt … (for example python -m docsgpt dev).

   Optional extras are not installed by default. Add them when you need the feature (each file is the core set plus the extra):

   uv sync --extra docling --extra milvus            # name every extra you want: uv sync removes the others
   # or with pip:
   pip install -r docsgpt/requirements-docling.txt   # docling parser engine: OCR backend, read_document structured output
   pip install -r docsgpt/requirements-milvus.txt    # VECTOR_STORE=milvus

   The docling file adds the PyTorch CPU index; pip handles that as-is, while uv pip install -r needs UV_INDEX_STRATEGY=unsafe-best-match (or use uv sync --extra docling, which reads the lock).

   A feature whose extra is missing fails with the exact command to run. Dependencies are declared in pyproject.toml and locked in uv.lock, and the requirements*.txt files are exported from that lock. After changing pyproject.toml, run uv lock and bash scripts/export_requirements.sh so the exported files stay in sync (CI checks this).

   With Postgres and Redis running, check the setup:

   docsgpt doctor

   It reports whether PostgreSQL and Redis answer, whether a model provider is configured, and whether the API port is free. A missing POSTGRES_URI shows up here as a failure.

3. Embedding Model (no action needed):

   The embedding model is downloaded automatically the first time you ingest a document, and cached for subsequent runs under models/ in the data home: the repository root, unless DOCSGPT_HOME points elsewhere. Set EMBEDDINGS_CACHE_DIR to use another directory.

   For an offline or air-gapped machine, fetch it ahead of time instead:

   docsgpt prefetch-models

4. Run the Backend:

   One command runs the API and the worker from this checkout, each restarting when you save a file:

   docsgpt dev

   Both run as children of that terminal, with their output interleaved and labelled, and Ctrl-C stops them together. Useful flags:

   Flag	What it does
   --ui	also start the Vite dev server, so the whole app runs from one command
   --mock-llm	run scripts/mock_llm.py and point DocsGPT at it, so no API key is needed
   --no-worker	leave the worker to you, for instance when debugging it in your editor; start one as in step 5, or searches and scheduled tasks stop working
   --no-reload	do not restart anything on save
   --port	serve the API somewhere other than 7091

   docsgpt dev is for a checkout. docsgpt up --native, by contrast, installs supervised services that outlive the shell — see Run it as services.

   docsgpt doctor checks the things that usually break a new setup: whether PostgreSQL answers and its schema matches this version, whether Redis answers, whether a model provider is configured, and whether the port is free. Run it first when something does not start.

   To run the two processes yourself instead, start the ASGI composition under uvicorn. It serves the whole application, hot-reloads on source changes, and matches the production runtime:

   uvicorn docsgpt.asgi:asgi_app --host 0.0.0.0 --port 7091 --reload

   This makes the backend accessible on http://localhost:7091. Production uses gunicorn -k docsgpt.gunicorn_worker.BoundedDrainUvicornWorker against the same docsgpt.asgi:asgi_app target; docsgpt/Dockerfile has the full command.

   A plain Flask run is a faster inner loop (quick startup, the Werkzeug interactive debugger):

   flask --app docsgpt/app.py run --host=0.0.0.0 --port=7091

   But it serves only the WSGI Flask app, so the ASGI-only features return 404 under it. Chat still works (POST /stream is a Flask route). Use flask run only when you don’t need those features.

5. Start the Celery Worker (not needed if you used docsgpt dev):

   Open a new terminal window (and activate your virtual environment if you used one), then start the worker from the repository root:

   docsgpt worker

   Use python -m docsgpt worker if the docsgpt command is not on your PATH. The worker parses and embeds uploaded documents, embeds every search query for the API, and runs background agent tasks. It also embeds the Celery beat scheduler (-B).

   Chat retrieval needs a running worker. The API sends each query embedding to it (EMBEDDINGS_DELEGATE_TO_WORKER, on by default). Without a worker, the first search waits up to EMBEDDINGS_DELEGATE_TIMEOUT (60 seconds), searches in the next 30 seconds fail at once, and every answer comes back with no retrieved context, without an error. To run the API alone, set EMBEDDINGS_DELEGATE_TO_WORKER=false to load the model in the API process, or point EMBEDDINGS_BASE_URL at an embeddings service.

   The beat scheduler fires scheduled agent runs, source syncs, the reconciliation of stuck tasks, the background job sweeps, retention cleanups and the version check. If no scheduler runs, none of these happen and nothing reports it. At least one scheduler must run. Running more than one is safe, because a lock in Redis lets only one of them schedule at a time. docsgpt worker --no-beat leaves the scheduler out, and docsgpt beat runs it on its own. On Windows the scheduler cannot run inside the worker, so run docsgpt beat in a third terminal. Background jobs lists every periodic task and what it trims.

   To run Celery directly, add -B yourself:

   celery -A docsgpt.app.celery worker -l INFO -B

   Celery rejects -B on Windows. There, drop it and run the scheduler in a separate terminal with celery -A docsgpt.app.celery beat -l INFO (or docsgpt beat).

   macOS note: due to a threading issue with the default prefork pool, use the solo pool on macOS. docsgpt worker picks it automatically on macOS and Windows. With raw Celery, pass it yourself:

   python -m celery -A docsgpt.app.celery worker -l INFO -B --pool=solo

   The solo pool makes each query embedding take roughly 350 ms instead of about 55 ms on prefork, almost all of it spent waiting for the worker to pick up the message. That only affects local development; production runs prefork.

Running in Debugger (VSCode):

For easier debugging, you can launch the API and the Celery worker directly from VSCode’s debugger.

• Press Shift + Cmd + D (macOS) or Shift + Windows + D (Windows) to open the Run and Debug view.
• You should see configurations named “API (uvicorn)” and “Celery worker”, and a compound “DocsGPT: Full Stack” that starts them with the frontend. Select one and click the “Start Debugging” button (green play icon).

The API configuration runs the same ASGI app as production, so the ASGI-only features work under the debugger. It deliberately runs without --reload: the reloader restarts the server in a child process, which your breakpoints would not be attached to. The worker configuration uses the solo pool and embeds the beat scheduler (-B), except on Windows, where it leaves -B out; start docsgpt beat there.

3. Start the Frontend

To run the DocsGPT frontend locally, you’ll need Node.js and npm (Node Package Manager). docsgpt dev --ui starts the frontend dev server together with the backend once its dependencies are installed (steps 1 and 2 below).

Prerequisites:

• Node.js 22: the frontend declares node >=22 <23 and pins 22 in frontend/.nvmrc, so with nvm  you can run nvm use in frontend/. Check your version with node -v. npm is bundled with Node.js.

Steps to start the frontend:

1. Navigate to the Frontend Directory:

   In your terminal, change the current directory to the frontend folder within your DocsGPT repository:

   cd frontend

2. Install Frontend Dependencies:

   Install the project’s frontend dependencies using npm:

   npm install --include=dev

   This installs everything in package.json, development dependencies included. Vite runs from this local install, and the install also sets up the Husky pre-commit hook, so neither needs a global install.

3. Run the Frontend App:

   Start the frontend development server:

   npm run dev

   This command will start the Vite development server. The frontend application will typically be accessible at http://localhost:5173/ . The terminal will display the exact URL where the frontend is running.

With both the backend and frontend running, you should now have a fully functional DocsGPT development environment. You can access the application in your browser at http://localhost:5173/  and start developing!

ASGI-only features

A few routes hold a response open for a long time, so they are native-async routes mounted only on the ASGI app, docsgpt.asgi:asgi_app, and not on the Flask app:

docsgpt api, docsgpt dev, uvicorn docsgpt.asgi:asgi_app, gunicorn with a uvicorn worker and the Docker images all serve the ASGI app. flask --app docsgpt/app.py run serves only the Flask app, so these routes return 404 under it.

Working on two branches at once

Each install keeps its own directory and its own services, so a second branch can run beside the first as long as it gets its own port:

docsgpt dev --port 7092                      # a second checkout, second terminal
docsgpt up --native --dir ~/.docsgpt/review --port 7092   # or a second installed copy

A native install in another directory gets its own service names, so the two never write over each other’s units. docsgpt status --dir ~/.docsgpt/review reports on that one alone.

Troubleshooting

Run docsgpt doctor first. It checks PostgreSQL, Redis, the model provider and the API port.

• Searches hang for up to a minute or fail at once, and answers ignore your documents. No worker is consuming the embeddings queue. Start one as in step 5. A worker started with an explicit -Q must list embeddings.
• Scheduled agent runs and source syncs never fire. No beat scheduler is running. Start the worker with docsgpt worker or add -B to the celery command. On Windows, also run docsgpt beat.
• Notifications never appear, a dropped answer doesn’t resume, a paired device never receives commands, or artifact downloads return 404. The API is running under flask run. Serve it through the ASGI app instead; see ASGI-only features.