SurfSense Documentation

Docker Installation

Setting up SurfSense using Docker

Known Limitations

⚠️ Important Note: Currently, the following features have limited functionality when running in Docker:

  • Ollama integration: Local Ollama models do not work when running SurfSense in Docker. Please use other LLM providers like OpenAI or Gemini instead.
  • Web crawler functionality: The web crawler feature currently doesn't work properly within the Docker environment.

We're actively working to resolve these limitations in future releases.

Docker Installation

This guide explains how to run SurfSense using Docker Compose, which is the preferred and recommended method for deployment.

Prerequisites

Before you begin, ensure you have:

  • Docker and Docker Compose installed on your machine
  • Git (to clone the repository)
  • Completed all the prerequisite setup steps including:
    • PGVector setup
    • File Processing ETL Service (choose one):
      • Unstructured.io API key (Supports 34+ formats)
      • LlamaIndex API key (enhanced parsing, supports 50+ formats)
    • Other required API keys

Installation Steps

  1. Configure Environment Variables Set up the necessary environment variables:

    Linux/macOS:

    # Copy example environment files
    cp surfsense_backend/.env.example surfsense_backend/.env
    cp surfsense_web/.env.example surfsense_web/.env
    cp .env.example .env  # For Docker-specific settings

    Windows (Command Prompt):

    copy surfsense_backend\.env.example surfsense_backend\.env
    copy surfsense_web\.env.example surfsense_web\.env
    copy .env.example .env

    Windows (PowerShell):

    Copy-Item -Path surfsense_backend\.env.example -Destination surfsense_backend\.env
    Copy-Item -Path surfsense_web\.env.example -Destination surfsense_web\.env
    Copy-Item -Path .env.example -Destination .env

    Edit all .env files and fill in the required values:

Docker-Specific Environment Variables

ENV VARIABLEDESCRIPTIONDEFAULT VALUE
FRONTEND_PORTPort for the frontend service3000
BACKEND_PORTPort for the backend API service8000
POSTGRES_PORTPort for the PostgreSQL database5432
PGADMIN_PORTPort for pgAdmin web interface5050
POSTGRES_USERPostgreSQL usernamepostgres
POSTGRES_PASSWORDPostgreSQL passwordpostgres
POSTGRES_DBPostgreSQL database namesurfsense
PGADMIN_DEFAULT_EMAILEmail for pgAdmin loginadmin@surfsense.com
PGADMIN_DEFAULT_PASSWORDPassword for pgAdmin loginsurfsense
NEXT_PUBLIC_API_URLURL of the backend API (used by frontend)http://backend:8000

Backend Environment Variables:

ENV VARIABLEDESCRIPTION
DATABASE_URLPostgreSQL connection string (e.g., postgresql+asyncpg://postgres:postgres@localhost:5432/surfsense)
SECRET_KEYJWT Secret key for authentication (should be a secure random string)
NEXT_FRONTEND_URLURL where your frontend application is hosted (e.g., http://localhost:3000)
AUTH_TYPEAuthentication method: GOOGLE for OAuth with Google, LOCAL for email/password authentication
GOOGLE_OAUTH_CLIENT_ID(Optional) Client ID from Google Cloud Console (required if AUTH_TYPE=GOOGLE)
GOOGLE_OAUTH_CLIENT_SECRET(Optional) Client secret from Google Cloud Console (required if AUTH_TYPE=GOOGLE)
EMBEDDING_MODELName of the embedding model (e.g., mixedbread-ai/mxbai-embed-large-v1)
RERANKERS_MODEL_NAMEName of the reranker model (e.g., ms-marco-MiniLM-L-12-v2)
RERANKERS_MODEL_TYPEType of reranker model (e.g., flashrank)
TTS_SERVICEText-to-Speech API provider for Podcasts (e.g., openai/tts-1). See supported providers
TTS_SERVICE_API_KEYAPI key for the Text-to-Speech service
TTS_SERVICE_API_BASE(Optional) Custom API base URL for the Text-to-Speech service
STT_SERVICESpeech-to-Text API provider for Podcasts (e.g., openai/whisper-1). See supported providers
STT_SERVICE_API_KEYAPI key for the Speech-to-Text service
STT_SERVICE_API_BASE(Optional) Custom API base URL for the Speech-to-Text service
FIRECRAWL_API_KEYAPI key for Firecrawl service for web crawling
ETL_SERVICEDocument parsing service: UNSTRUCTURED (supports 34+ formats) or LLAMACLOUD (supports 50+ formats including legacy document types)
UNSTRUCTURED_API_KEYAPI key for Unstructured.io service for document parsing (required if ETL_SERVICE=UNSTRUCTURED)
LLAMA_CLOUD_API_KEYAPI key for LlamaCloud service for document parsing (required if ETL_SERVICE=LLAMACLOUD)

Optional Backend LangSmith Observability:

ENV VARIABLEDESCRIPTION
LANGSMITH_TRACINGEnable LangSmith tracing (e.g., true)
LANGSMITH_ENDPOINTLangSmith API endpoint (e.g., https://api.smith.langchain.com)
LANGSMITH_API_KEYYour LangSmith API key
LANGSMITH_PROJECTLangSmith project name (e.g., surfsense)

Backend Uvicorn Server Configuration:

ENV VARIABLEDESCRIPTIONDEFAULT VALUE
UVICORN_HOSTHost address to bind the server0.0.0.0
UVICORN_PORTPort to run the backend API8000
UVICORN_LOG_LEVELLogging level (e.g., info, debug, warning)info
UVICORN_PROXY_HEADERSEnable/disable proxy headersfalse
UVICORN_FORWARDED_ALLOW_IPSComma-separated list of allowed IPs127.0.0.1
UVICORN_WORKERSNumber of worker processes1
UVICORN_ACCESS_LOGEnable/disable access log (true/false)true
UVICORN_LOOPEvent loop implementationauto
UVICORN_HTTPHTTP protocol implementationauto
UVICORN_WSWebSocket protocol implementationauto
UVICORN_LIFESPANLifespan implementationauto
UVICORN_LOG_CONFIGPath to logging config file or empty string
UVICORN_SERVER_HEADEREnable/disable Server headertrue
UVICORN_DATE_HEADEREnable/disable Date headertrue
UVICORN_LIMIT_CONCURRENCYMax concurrent connections
UVICORN_LIMIT_MAX_REQUESTSMax requests before worker restart
UVICORN_TIMEOUT_KEEP_ALIVEKeep-alive timeout (seconds)5
UVICORN_TIMEOUT_NOTIFYWorker shutdown notification timeout (sec)30
UVICORN_SSL_KEYFILEPath to SSL key file
UVICORN_SSL_CERTFILEPath to SSL certificate file
UVICORN_SSL_KEYFILE_PASSWORDPassword for SSL key file
UVICORN_SSL_VERSIONSSL version
UVICORN_SSL_CERT_REQSSSL certificate requirements
UVICORN_SSL_CA_CERTSPath to CA certificates file
UVICORN_SSL_CIPHERSSSL ciphers
UVICORN_HEADERSComma-separated list of headers
UVICORN_USE_COLORSEnable/disable colored logstrue
UVICORN_UDSUnix domain socket path
UVICORN_FDFile descriptor to bind to
UVICORN_ROOT_PATHRoot path for the application

For more details, see the Uvicorn documentation.

Frontend Environment Variables

ENV VARIABLEDESCRIPTION
NEXT_PUBLIC_FASTAPI_BACKEND_URLURL of the backend service (e.g., http://localhost:8000)
NEXT_PUBLIC_FASTAPI_BACKEND_AUTH_TYPESame value as set in backend AUTH_TYPE i.e GOOGLE for OAuth with Google, LOCAL for email/password authentication
NEXT_PUBLIC_ETL_SERVICEDocument parsing service (should match backend ETL_SERVICE): UNSTRUCTURED or LLAMACLOUD - affects supported file formats in upload interface
  1. Build and Start Containers

    Start the Docker containers:

    Linux/macOS/Windows:

    docker compose up --build

    To run in detached mode (in the background):

    Linux/macOS/Windows:

    docker compose up -d

    Note for Windows users: If you're using older Docker Desktop versions, you might need to use docker compose (with a space) instead of docker compose.

  2. Access the Applications

    Once the containers are running, you can access:

Using pgAdmin

pgAdmin is included in the Docker setup to help manage your PostgreSQL database. To connect:

  1. Open pgAdmin at http://localhost:5050
  2. Login with the credentials from your .env file (default: admin@surfsense.com / surfsense)
  3. Right-click "Servers" > "Create" > "Server"
  4. In the "General" tab, name your connection (e.g., "SurfSense DB")
  5. In the "Connection" tab:
    • Host: db
    • Port: 5432
    • Maintenance database: surfsense
    • Username: postgres (or your custom POSTGRES_USER)
    • Password: postgres (or your custom POSTGRES_PASSWORD)
  6. Click "Save" to connect

Useful Docker Commands

Container Management

  • Stop containers:

    Linux/macOS/Windows:

    docker compose down
  • View logs:

    Linux/macOS/Windows:

    # All services
    docker compose logs -f
     
    # Specific service
    docker compose logs -f backend
    docker compose logs -f frontend
    docker compose logs -f db
  • Restart a specific service:

    Linux/macOS/Windows:

    docker compose restart backend
  • Execute commands in a running container:

    Linux/macOS/Windows:

    # Backend
    docker compose exec backend python -m pytest
     
    # Frontend
    docker compose exec frontend pnpm lint

Troubleshooting

  • Linux/macOS: If you encounter permission errors, you may need to run the docker commands with sudo.
  • Windows: If you see access denied errors, make sure you're running Command Prompt or PowerShell as Administrator.
  • If ports are already in use, modify the port mappings in the docker-compose.yml file.
  • For backend dependency issues, check the Dockerfile in the backend directory.
  • For frontend dependency issues, check the Dockerfile in the frontend directory.
  • Windows-specific: If you encounter line ending issues (CRLF vs LF), configure Git to handle line endings properly with git config --global core.autocrlf true before cloning the repository.

Next Steps

Once your installation is complete, you can start using SurfSense! Navigate to the frontend URL and log in using your Google account.