innercontext/docs/DEPLOYMENT.md

Deployment Guide (LXC + systemd + nginx)

This project deploys from an external machine (developer laptop or CI runner) to a Debian LXC host over SSH.

Deployments are push-based, release-based, and atomic:

• Build and validate locally
• Upload to /opt/innercontext/releases/<timestamp>
• Run backend dependency sync and migrations in that release directory
• Promote once by switching /opt/innercontext/current
• Restart services and run health checks
• Auto-rollback on failure

Environment files have exactly two persistent locations on the server:

• /opt/innercontext/shared/backend/.env
• /opt/innercontext/shared/frontend/.env.production

Each release links to those files from:

• /opt/innercontext/current/backend/.env -> ../../../shared/backend/.env
• /opt/innercontext/current/frontend/.env.production -> ../../../shared/frontend/.env.production

Architecture

external machine (manual now, CI later)
  |
  | ssh + rsync
  v
LXC host
  /opt/innercontext/
    current -> releases/<timestamp>
    releases/<timestamp>
    shared/backend/.env
    shared/frontend/.env.production
    scripts/

Services:

• innercontext (FastAPI, localhost:8000)
• innercontext-node (SvelteKit Node, localhost:3000)
• innercontext-pricing-worker (background worker)

nginx routes:

• /api/* -> http://127.0.0.1:8000/*
• /* -> http://127.0.0.1:3000/*

Run Model

• Manual deploy: run ./deploy.sh ... from repo root on your laptop.
• Optional CI deploy: run the same script from a manual workflow (workflow_dispatch).
• The server never builds frontend assets.

One-Time Server Setup

Run on the LXC host as root.

1) Install runtime dependencies

apt update && apt upgrade -y
apt install -y git nginx curl ca-certificates libpq5 rsync python3 python3-venv

curl -LsSf https://astral.sh/uv/install.sh | UV_INSTALL_DIR=/usr/local/bin sh

curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.4/install.sh | bash
. "$HOME/.nvm/nvm.sh"
nvm install 24
cp --remove-destination "$(nvm which current)" /usr/local/bin/node

curl -fsSL "https://github.com/pnpm/pnpm/releases/latest/download/pnpm-linux-x64" \
  -o /usr/local/bin/pnpm
chmod 755 /usr/local/bin/pnpm

2) Create app user and directories

useradd --system --create-home --shell /bin/bash innercontext

mkdir -p /opt/innercontext/releases
mkdir -p /opt/innercontext/shared/backend
mkdir -p /opt/innercontext/shared/frontend
mkdir -p /opt/innercontext/scripts
chown -R innercontext:innercontext /opt/innercontext

3) Create shared env files

cat > /opt/innercontext/shared/backend/.env <<'EOF'
DATABASE_URL=postgresql+psycopg://innercontext:change-me@<pg-ip>/innercontext
GEMINI_API_KEY=your-key

# OIDC Configuration
OIDC_ISSUER=https://auth.example.com
OIDC_CLIENT_ID=innercontext-backend
OIDC_DISCOVERY_URL=https://auth.example.com/.well-known/openid-configuration
OIDC_ADMIN_GROUPS=admins
OIDC_MEMBER_GROUPS=members

# Bootstrap Admin (Optional, used for initial setup)
# BOOTSTRAP_ADMIN_OIDC_ISSUER=https://auth.example.com
# BOOTSTRAP_ADMIN_OIDC_SUB=user-sub-from-authelia
# BOOTSTRAP_ADMIN_EMAIL=admin@example.com
# BOOTSTRAP_ADMIN_NAME="Admin User"
# BOOTSTRAP_HOUSEHOLD_NAME="My Household"
EOF

cat > /opt/innercontext/shared/frontend/.env.production <<'EOF'
PUBLIC_API_BASE=http://127.0.0.1:8000
ORIGIN=http://innercontext.lan

# Session and OIDC
SESSION_SECRET=generate-a-long-random-string
OIDC_ISSUER=https://auth.example.com
OIDC_CLIENT_ID=innercontext-frontend
OIDC_DISCOVERY_URL=https://auth.example.com/.well-known/openid-configuration
EOF

OIDC Setup (Authelia)

This project uses OIDC for authentication. You need an OIDC provider like Authelia.

Authelia Client Configuration

Add the following to your Authelia configuration.yml:

identity_providers:
  oidc:
    clients:
      - id: innercontext-frontend
        description: InnerContext Frontend
        secret: '$pbkdf2-sha512$...' # Not used for public client, but Authelia may require it
        public: true
        authorization_policy: one_factor
        redirect_uris:
          - http://innercontext.lan/auth/callback
        scopes:
          - openid
          - profile
          - email
          - groups
        userinfo_signed_response_alg: none

      - id: innercontext-backend
        description: InnerContext Backend
        secret: '$pbkdf2-sha512$...'
        public: false
        authorization_policy: one_factor
        redirect_uris: []
        scopes:
          - openid
          - profile
          - email
          - groups
        userinfo_signed_response_alg: none

Bootstrap Admin

To create the first user and household, set the BOOTSTRAP_ADMIN_* environment variables in the backend .env file and restart the backend. The backend will automatically create the user and household on startup if they don't exist. After the first successful login, you can remove these variables.

Health Checks

• Backend: http://127.0.0.1:8000/health-check (returns 200)
• Frontend: http://127.0.0.1:3000/ (returns 200 or 302 redirect to login)
• Worker: systemctl is-active innercontext-pricing-worker

Manual checks:

curl -sf http://127.0.0.1:8000/health-check
curl -sf http://127.0.0.1:3000/
systemctl is-active innercontext
systemctl is-active innercontext-node
systemctl is-active innercontext-pricing-worker

Troubleshooting

Lock exists

cat /opt/innercontext/.deploy.lock
rm -f /opt/innercontext/.deploy.lock

Only remove the lock if no deployment is running.

Sudo password prompt during deploy

Re-check /etc/sudoers.d/innercontext-deploy and run:

visudo -c -f /etc/sudoers.d/innercontext-deploy
sudo -u innercontext sudo systemctl is-active innercontext

Backend migration failure

Validate env file and DB connectivity:

ls -la /opt/innercontext/shared/backend/.env
grep '^DATABASE_URL=' /opt/innercontext/shared/backend/.env

Service fails after deploy

journalctl -u innercontext -n 100
journalctl -u innercontext-node -n 100
journalctl -u innercontext-pricing-worker -n 100

Manual CI Deploy (Optional)

Use the manual Forgejo workflow (workflow_dispatch) to run the same ./deploy.sh all path from CI once server secrets and SSH trust are configured.