innercontext/docs/DEPLOYMENT.md

# Deployment Guide (LXC + systemd + nginx)

This project deploys from an external machine (developer laptop or CI runner) to a Debian LXC host over SSH.

Deployments are push-based, release-based, and atomic:

- Build and validate locally
- Upload to `/opt/innercontext/releases/<timestamp>`
- Run backend dependency sync and migrations in that release directory
- Promote once by switching `/opt/innercontext/current`
- Restart services and run health checks
- Auto-rollback on failure

Environment files have exactly two persistent locations on the server:

- `/opt/innercontext/shared/backend/.env`
- `/opt/innercontext/shared/frontend/.env.production`

Each release links to those files from:

- `/opt/innercontext/current/backend/.env` -> `../../../shared/backend/.env`
- `/opt/innercontext/current/frontend/.env.production` -> `../../../shared/frontend/.env.production`

## Architecture

```
external machine (manual now, CI later)
  |
  | ssh + rsync
  v
LXC host
  /opt/innercontext/
    current -> releases/<timestamp>
    releases/<timestamp>
    shared/backend/.env
    shared/frontend/.env.production
    scripts/
```

Services:

- `innercontext` (FastAPI, localhost:8000)
- `innercontext-node` (SvelteKit Node, localhost:3000)
- `innercontext-pricing-worker` (background worker)

nginx routes:

- `/api/*` -> `http://127.0.0.1:8000/*`
- `/*` -> `http://127.0.0.1:3000/*`

## Run Model

- Manual deploy: run `./deploy.sh ...` from repo root on your laptop.
- Optional CI deploy: run the same script from a manual workflow (`workflow_dispatch`).
- The server never builds frontend assets.

## One-Time Server Setup

Run on the LXC host as root.

### 1) Install runtime dependencies

```bash
apt update && apt upgrade -y
apt install -y git nginx curl ca-certificates libpq5 rsync python3 python3-venv

curl -LsSf https://astral.sh/uv/install.sh | UV_INSTALL_DIR=/usr/local/bin sh

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.4/install.sh | bash
. "$HOME/.nvm/nvm.sh"
nvm install 24
cp --remove-destination "$(nvm which current)" /usr/local/bin/node

curl -fsSL "https://github.com/pnpm/pnpm/releases/latest/download/pnpm-linux-x64" \
  -o /usr/local/bin/pnpm
chmod 755 /usr/local/bin/pnpm
```

### 2) Create app user and directories

```bash
useradd --system --create-home --shell /bin/bash innercontext

mkdir -p /opt/innercontext/releases
mkdir -p /opt/innercontext/shared/backend
mkdir -p /opt/innercontext/shared/frontend
mkdir -p /opt/innercontext/scripts
chown -R innercontext:innercontext /opt/innercontext
```

### 3) Create shared env files

```bash
cat > /opt/innercontext/shared/backend/.env <<'EOF'
DATABASE_URL=postgresql+psycopg://innercontext:change-me@<pg-ip>/innercontext
GEMINI_API_KEY=your-key

# OIDC Configuration
OIDC_ISSUER=https://auth.example.com
OIDC_CLIENT_ID=innercontext-backend
OIDC_DISCOVERY_URL=https://auth.example.com/.well-known/openid-configuration
OIDC_ADMIN_GROUPS=admins
OIDC_MEMBER_GROUPS=members

# Bootstrap Admin (Optional, used for initial setup)
# BOOTSTRAP_ADMIN_OIDC_ISSUER=https://auth.example.com
# BOOTSTRAP_ADMIN_OIDC_SUB=user-sub-from-authelia
# BOOTSTRAP_ADMIN_EMAIL=admin@example.com
# BOOTSTRAP_ADMIN_NAME="Admin User"
# BOOTSTRAP_HOUSEHOLD_NAME="My Household"
EOF

cat > /opt/innercontext/shared/frontend/.env.production <<'EOF'
PUBLIC_API_BASE=http://127.0.0.1:8000
ORIGIN=http://innercontext.lan

# Session and OIDC
SESSION_SECRET=generate-a-long-random-string
OIDC_ISSUER=https://auth.example.com
OIDC_CLIENT_ID=innercontext-frontend
OIDC_DISCOVERY_URL=https://auth.example.com/.well-known/openid-configuration
EOF
```

## OIDC Setup (Authelia)

This project uses OIDC for authentication. You need an OIDC provider like Authelia.

### Authelia Client Configuration

Add the following to your Authelia `configuration.yml`:

```yaml
identity_providers:
  oidc:
    clients:
      - id: innercontext-frontend
        description: InnerContext Frontend
        secret: '$pbkdf2-sha512$...' # Not used for public client, but Authelia may require it
        public: true
        authorization_policy: one_factor
        redirect_uris:
          - http://innercontext.lan/auth/callback
        scopes:
          - openid
          - profile
          - email
          - groups
        userinfo_signed_response_alg: none

      - id: innercontext-backend
        description: InnerContext Backend
        secret: '$pbkdf2-sha512$...'
        public: false
        authorization_policy: one_factor
        redirect_uris: []
        scopes:
          - openid
          - profile
          - email
          - groups
        userinfo_signed_response_alg: none
```

### Bootstrap Admin

To create the first user and household, set the `BOOTSTRAP_ADMIN_*` environment variables in the backend `.env` file and restart the backend. The backend will automatically create the user and household on startup if they don't exist. After the first successful login, you can remove these variables.

## Health Checks

- Backend: `http://127.0.0.1:8000/health-check` (returns 200)
- Frontend: `http://127.0.0.1:3000/` (returns 200 or 302 redirect to login)
- Worker: `systemctl is-active innercontext-pricing-worker`

Manual checks:

```bash
curl -sf http://127.0.0.1:8000/health-check
curl -sf http://127.0.0.1:3000/
systemctl is-active innercontext
systemctl is-active innercontext-node
systemctl is-active innercontext-pricing-worker
```

## Troubleshooting

### Lock exists

```bash
cat /opt/innercontext/.deploy.lock
rm -f /opt/innercontext/.deploy.lock
```

Only remove the lock if no deployment is running.

### Sudo password prompt during deploy

Re-check `/etc/sudoers.d/innercontext-deploy` and run:

```bash
visudo -c -f /etc/sudoers.d/innercontext-deploy
sudo -u innercontext sudo systemctl is-active innercontext
```

### Backend migration failure

Validate env file and DB connectivity:

```bash
ls -la /opt/innercontext/shared/backend/.env
grep '^DATABASE_URL=' /opt/innercontext/shared/backend/.env
```

### Service fails after deploy

```bash
journalctl -u innercontext -n 100
journalctl -u innercontext-node -n 100
journalctl -u innercontext-pricing-worker -n 100
```

## Manual CI Deploy (Optional)

Use the manual Forgejo workflow (`workflow_dispatch`) to run the same `./deploy.sh all` path from CI once server secrets and SSH trust are configured.