Connecting DocsGPT to Cloud LLM Providers

DocsGPT is designed to seamlessly integrate with a variety of Cloud Large Language Model (LLM) providers, giving you access to state-of-the-art AI models for document question answering.

Configuration via .env file

The primary method for configuring your LLM provider in DocsGPT is through the .env file. For a comprehensive understanding of all available settings, please refer to the detailed DocsGPT Settings Guide.

To connect to a cloud LLM provider, you will typically need to configure the following basic settings in your .env file:

• LLM_PROVIDER: This setting is essential and identifies the specific cloud provider you wish to use (e.g., openai, google, anthropic).
• LLM_NAME: Specifies the exact model you want to utilize from your chosen provider (e.g., gpt-4o, gemini-2.0-flash, claude-3-5-sonnet-latest). Refer to your provider's documentation for a list of available models.
• API_KEY: Almost all cloud LLM providers require an API key for authentication. Obtain your API key from your chosen provider's platform and securely store it in your .env file.

Explicitly Supported Cloud Providers

DocsGPT offers direct, streamlined support for the following cloud LLM providers, making configuration straightforward. The table below outlines the LLM_PROVIDER and example LLM_NAME values to use for each provider in your .env file.

Connecting to OpenAI-Compatible Cloud APIs

DocsGPT's flexible architecture allows you to connect to any cloud provider that offers an API compatible with the OpenAI API standard. This opens up a vast ecosystem of LLM services.

To connect to an OpenAI-compatible cloud provider, you will still use LLM_PROVIDER=openai in your .env file. However, you will also need to specify the API endpoint of your chosen provider using the OPENAI_BASE_URL setting. You will also likely need to provide an API_KEY and LLM_NAME as required by that provider.

Example for DeepSeek (OpenAI-Compatible API):

To connect to DeepSeek, which offers an OpenAI-compatible API, your .env file could be configured as follows:

LLM_PROVIDER=openai
API_KEY=YOUR_API_KEY # Your DeepSeek API key
LLM_NAME=deepseek-chat # Or your desired DeepSeek model name
OPENAI_BASE_URL=https://api.deepseek.com/v1 # DeepSeek's OpenAI API URL

Remember to consult the documentation of your chosen OpenAI-compatible cloud provider for their specific API endpoint, required model names, and authentication methods.

Adding Support for Other Cloud Providers

If you wish to connect to a cloud provider that is not explicitly listed above or doesn't offer OpenAI API compatibility, you can extend DocsGPT to support it. Within the DocsGPT repository, navigate to the application/llm directory. Here, you will find Python files defining the existing LLM integrations. You can use these files as examples to create a new module for your desired cloud provider. After creating your new LLM module, you will need to register it within the llm_creator.py file. This process involves some coding, but it allows for virtually unlimited extensibility to connect to any cloud-based LLM service with an accessible API.