Local Development Setup

This guide walks you through running Yew Search on your local machine. It is written for a developer who wants to get the app running quickly without needing a homelab, a server, or any cloud accounts.

For a printable progress checklist see Local Deployment Checklist.

What You Will End Up With

By the end of this guide you will have:

The backend API running at http://localhost:8443
The frontend running at http://localhost:5173
A user account you can log in with
The hello-world integration running so you can see the full polling and search pipeline working

You will not have Gmail or any real OAuth integration working. That requires a publicly-accessible callback URL and is covered at the bottom of this guide.

What Gets Deployed

Four things run locally. Two are managed by Docker Compose, two run natively via Node.

Component	How it runs	Port	What it does
PostgreSQL	Docker Compose	5432	Primary database — stores users, integrations, and all synced content
Redis	Docker Compose	6379	Session store and Bull task queue for the integration polling system
NestJS backend	`pnpm run start:dev`	8443	REST API, polling scheduler, OAuth handling, search
Svelte frontend	`pnpm run dev`	5173	Browser UI — login, search, integration management

Docker Compose (backend/docker-compose.yaml) starts and manages PostgreSQL and Redis. Data is persisted in named Docker volumes so it survives container restarts.

The backend runs natively (not in Docker) so you get fast TypeScript watch-mode reloading. It connects to PostgreSQL and Redis on localhost. On first boot it auto-creates all database tables — no migration commands needed.

The frontend is a Svelte SPA served by Vite's dev server. It talks directly to the backend at http://localhost:8443. The backend URL is injected at startup via frontend/public/config.js.

The hello-world integration is what stands in for OAuth integrations locally. It runs inside the backend process, generates dummy content every polling cycle, and proves the full pipeline works: scheduler → Redis queue → worker → PostgreSQL → search.

Prerequisites

Install these before you start. If you already have them, skip ahead.

Node.js 20 — check with node --version
pnpm — install with npm install -g pnpm, then check with pnpm --version
Docker — used to run Postgres and Redis via Docker Compose. Check with docker --version

Step 1 — Start PostgreSQL and Redis

The backend requires both of these. A docker-compose.yaml in backend/ defines both services with credentials that match the default .env.example values.

From the backend/ directory, run:

docker compose up -d

Verify both containers are running:

docker compose ps

You should see yew-postgres and yew-redis listed with status running.

Step 2 — Configure the Backend

The docker compose up -d command in Step 1 already ran from backend/. Stay there.

Copy the example environment file:

cp .env.example .env

Open .env in your editor. The default values already match the Docker containers you just started, so you only need to make two changes:

Change 1 — Enable integrations so the polling system runs:

INTEGRATIONS_ENABLED=true

Change 2 — Make sure CORS allows the Vite dev server port:

CORS_ORIGINS=http://localhost:5173,http://127.0.0.1:5173

Everything else in .env can stay as-is for local development. You do not need to fill in GOOGLE_CLIENT_ID or GOOGLE_CLIENT_SECRET for this guide.

Step 3 — Install Backend Dependencies

Still inside backend/:

pnpm install

This installs dependencies for the main API and all integrations. It will take a minute or two the first time.

Step 4 — Start the Backend

pnpm run start:dev

This starts NestJS in watch mode. It will recompile and restart automatically whenever you change a source file.

The first time it starts, TypeORM will automatically create all the database tables because NODE_ENV is set to development. You do not need to run any migrations.

Watch the terminal output. A successful start looks like this near the end:

[NestApplication] Nest application successfully started

Leave this terminal running. Open a new terminal for the next steps.

Check the API is up:

curl http://localhost:8443

You should get a JSON response (even just {} is fine).

Then open http://localhost:8443/docs in your browser to confirm the Swagger UI loads.

Step 5 — Create Your User Account

The app has no registration page. Users are created through the CLI. Run this from inside the backend/ directory:

pnpm run cli user:create --email="you@example.com" --name="Your Name"

The CLI will prompt you for a password. Type it (nothing will appear on screen — that is normal) and press Enter.

You should see:

✓ User created successfully
  ID: ...
  Email: you@example.com
  Name: Your Name

For all available CLI commands see List of CLI Commands.

Step 6 — Set up the Frontend

Open a new terminal and navigate to the frontend:

cd frontend
pnpm install

The backend URL is already configured. The file frontend/public/config.js already exists in the repo and points to the correct local backend address:

window.APP_CONFIG = {
  BACKEND_URL: 'http://localhost:8443',
};

You do not need to create or edit this file. Vite serves files from public/ automatically during development.

Step 7 — Start the Frontend

pnpm run dev

Vite will start on port 5173. Open http://localhost:5173 in your browser.

If you see a "Blocked host" error in the browser: This is caused by allowedHosts in frontend/vite.config.ts being set to a production hostname. To fix it, open vite.config.ts and change:
allowedHosts: ['app-dev.yewsearch.com'],
to:
allowedHosts: ['app-dev.yewsearch.com', 'localhost'],

You should see the login page.

Step 8 — Log In

On the login page at http://localhost:5173, enter the email and password you used in Step 5.

After a successful login you will be redirected to the search page. The search bar will be empty because no content has been synced yet. That is expected — you will fix that in the next step.

Step 9 — Connect an Integration (no OAuth required)

The app ships with a hello-world integration. It requires no credentials, no OAuth, and no external services. It simply generates small text records every time the polling system runs. It is perfect for confirming that the full pipeline — scheduling, task queue, content storage, and search — all work correctly.

Link it to your user account using the CLI. From the backend/ directory:

pnpm run cli user-integration:create \
  --email="you@example.com" \
  --domain="hello-world"

You should see:

✓ User integration created successfully
  Integration ID: ...
  Domain: hello-world
  Status: active
  Sync Status: active
  Next Sync: <timestamp>

What happens next:

The polling scheduler runs every minute. Within 60 seconds it will:

Pick up the hello-world integration because its nextSyncAt is in the past
Push a start task onto the Redis-backed Bull queue
A worker will execute the task, generate content, and save it to PostgreSQL

After about a minute, go to the search page and search for hello. You should see results appear.

What to Check if Something is Not Working

Backend will not start

Are Postgres and Redis running? cd backend && docker compose ps
Did you copy .env.example to .env? Check with ls -la backend/.env
Look at the error in the terminal — TypeORM connection errors usually say "Connection refused" and tell you exactly which host/port it tried

Is the backend running? Try curl http://localhost:8443
Check that CORS_ORIGINS in backend/.env includes http://localhost:5173
Restart the backend after changing .env

Re-run the user:create CLI command — it will tell you if the user already exists
Use pnpm run cli user:update-password --email="you@example.com" to reset your password

No search results after connecting hello-world

Wait a full minute — the scheduler only runs every 60 seconds
Check the backend terminal for log lines mentioning PollingScheduler and hello-world
Verify INTEGRATIONS_ENABLED=true is set in backend/.env
Restart the backend after changing .env

Stopping and restarting

If you stop the backend and start it again later, the database tables will still be there — TypeORM's synchronize mode only creates what is missing, it does not wipe existing data. Your user account and any synced content will persist across container restarts because data is stored in named Docker volumes.

To stop the containers:

cd backend
docker compose stop

To start them again:

cd backend
docker compose start

To wipe everything and start completely fresh (this deletes all database data):

cd backend
docker compose down -v
docker compose up -d

What does not work locally

OAuth integrations (Gmail, Slack, etc.)

OAuth requires a publicly-accessible callback URL. When you click "Connect Gmail", Google sends the user back to a URL like http://localhost:8443/v1/user-integration/oauth/gmail/callback. Google will refuse to redirect to a localhost address for security reasons.

To test OAuth locally you have two options:

Option A — Use ngrok (quickest)

Install ngrok: brew install ngrok or download from ngrok.com
Expose your local backend: ngrok http 8443
Copy the HTTPS URL ngrok gives you, e.g. https://abc123.ngrok.io

Update backend/.env:

OAUTH_REDIRECT_BASE_URL=https://abc123.ngrok.io
BACKEND_URL=https://abc123.ngrok.io
CORS_ORIGINS=http://localhost:5173,http://127.0.0.1:5173
GOOGLE_CLIENT_ID=your_client_id_here
GOOGLE_CLIENT_SECRET=your_client_secret_here

In your Google Cloud Console, add https://abc123.ngrok.io/v1/user-integration/oauth/gmail/callback as an authorized redirect URI
Restart the backend

Note: the ngrok URL changes every time you restart ngrok (unless you have a paid plan).

Option B — Skip OAuth entirely for local dev

Use the hello-world integration (Step 9) to verify the full pipeline works. Use the real OAuth integrations only in a deployed environment where you have a stable public URL.

Observability (Loki, Tempo, Prometheus)

The backend is configured with LOGGING_ENABLED=true by default, but this only does anything if Loki is actually running. If it is not, the backend will log to the console instead and continue working normally. Tracing and metrics are disabled by default (TRACING_ENABLED=false, METRICS_ENABLED=false) so nothing breaks without them.

If you want to run the full observability stack locally, see Observability for a Docker Compose setup with Loki, Tempo, Prometheus, and Grafana.

Daily workflow (once set up)

Once everything is set up, here is how to start it each day:

# Start infrastructure (if not already running)
cd backend && docker compose start

# Terminal 1 — backend
cd backend
pnpm run start:dev

# Terminal 2 — frontend
cd frontend
pnpm run dev

Open http://localhost:5173.

Verifying the environment with lab scripts

Once the backend is running, use the scripts in lab/alan/ to confirm each layer of the stack is reachable and working. These scripts are the approved way to verify connectivity — do not test by clicking through the UI.

Check service connectivity

Confirms that Postgres, Redis, and other services are reachable from your machine:

cd lab/alan/testing-connection
npm install
node postgres.js
node redis.js

Check the queue

Confirms that Bull can enqueue and process jobs via Redis:

cd lab/alan/testing-queuing
npm install
npm run test:combined

You should see jobs added, processed, and a final success rate in the output.

Check logging

Confirms that structured log output is flowing correctly from the backend:

cd lab/alan/testing-logging
npm install
node index.js

If all three pass, your local environment is fully operational. See Testing for the full testing philosophy and how to write your own lab scripts.

What You Will End Up With​

What Gets Deployed​

Prerequisites​

Step 1 — Start PostgreSQL and Redis​

Step 2 — Configure the Backend​

Step 3 — Install Backend Dependencies​

Step 4 — Start the Backend​

Step 5 — Create Your User Account​

Step 6 — Set up the Frontend​

Step 7 — Start the Frontend​

Step 8 — Log In​

Step 9 — Connect an Integration (no OAuth required)​

What to Check if Something is Not Working​

Backend will not start​

Login fails with a network error​

Login fails with "invalid credentials"​

No search results after connecting hello-world​

Stopping and restarting​

What does not work locally​

OAuth integrations (Gmail, Slack, etc.)​

Observability (Loki, Tempo, Prometheus)​

Daily workflow (once set up)​

Verifying the environment with lab scripts​

Check service connectivity​

Check the queue​

Check logging​