Prerequisites

PGVector installation Guide

SurfSense requires the pgvector extension for PostgreSQL:

Linux and Mac

Compile and install the extension (supports Postgres 13+)

cd /tmp
git clone --branch v0.8.0 https://github.com/pgvector/pgvector.git
cd pgvector
make
make install # may need sudo

See the installation notes if you run into issues

Windows

Ensure C++ support in Visual Studio is installed, and run:

call "C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat"

Note: The exact path will vary depending on your Visual Studio version and edition

Then use nmake to build:

set "PGROOT=C:\Program Files\PostgreSQL\16"
cd %TEMP%
git clone --branch v0.8.0 https://github.com/pgvector/pgvector.git
cd pgvector
nmake /F Makefile.win
nmake /F Makefile.win install

See the installation notes if you run into issues

Google OAuth Setup

SurfSense user management and authentication works on Google OAuth. Lets set it up.

Login to your Google Developer Console
Enable People API.
Set up OAuth consent screen.
Create OAuth client ID and secret.
It should look like this.

File Upload's

Files are converted to LLM friendly formats using Unstructured

Get an Unstructured.io API key from Unstructured Platform
You should be able to generate API keys once registered

LLM Observability (Optional)

This is not required for SurfSense to work. But it is always a good idea to monitor LLM interactions. So we do not have those WTH moments.

Get a LangSmith API key from smith.langchain.com
This helps in observing SurfSense Researcher Agent.

Crawler

SurfSense have 2 options for saving webpages:

SurfSense Extension (Overall better experience & ability to save private webpages, recommended)
Crawler (If you want to save public webpages)

NOTE: SurfSense currently uses Firecrawl.py for web crawling. If you plan on using the crawler, you will need to create a Firecrawl account and get an API key.

Next Steps

Once you have all prerequisites in place, proceed to the installation guide to set up SurfSense.