DigitalOcean Production Deployment Runbook

Recommended MVP host

Start with a dedicated Ubuntu 24.04 LTS droplet sized around 8 vCPU / 16 GB RAM / 200+ GB NVMe if you want the chain indexer, PostgreSQL, Redis, Next.js, API, WebSocket gateway and worker all on one box with comfortable headroom. A smaller 4 vCPU / 8 GB box can work for development/low traffic, but real-time indexing + builds + database + AI proxy traffic are easier to operate with 16 GB.

Use a volume/snapshot/backup policy appropriate to the value and data involved. Never use the server as the sole copy of the repo or database backups.

1. Create non-root operator

bash

adduser forge
usermod -aG sudo forge
rsync --archive --chown=forge:forge ~/.ssh /home/forge

Then log in as forge and disable password/root SSH after verifying key login.

2. Firewall

bash

sudo ufw allow OpenSSH
sudo ufw allow 'Nginx Full'
sudo ufw enable

Do not expose PostgreSQL, Redis, API internal ports, indexer health ports or PM2 RPC publicly.

3. Install base packages

bash

sudo apt update && sudo apt upgrade -y
sudo apt install -y git curl ca-certificates build-essential nginx redis-server \
  postgresql postgresql-contrib jq unzip fail2ban certbot python3-certbot-nginx

Install Node.js 22 LTS through NodeSource/nvm/your standard controlled package method. Then:

bash

corepack enable
corepack prepare pnpm@latest --activate
npm i -g pm2

Pin the Node and pnpm versions in the repo after the first successful build.

4. PostgreSQL

bash

sudo -u postgres psql

sql

CREATE USER forge WITH PASSWORD 'replace-with-long-random-password';
CREATE DATABASE forge OWNER forge;
\q

Keep PostgreSQL listening on localhost unless you intentionally move DB to a private managed service.

5. Redis

Ensure Redis binds to localhost and protected mode is enabled. For local-only deployment, firewall should block 6379 externally regardless.

bash

sudo systemctl enable --now redis-server postgresql nginx

6. Repository layout

bash

sudo mkdir -p /opt/forge/releases /opt/forge/shared
sudo chown -R forge:forge /opt/forge
cd /opt/forge

git clone git@github.com:YOURORG/forge.git releases/initial
ln -sfn /opt/forge/releases/initial /opt/forge/current
cd current
pnpm install --frozen-lockfile
cp ENV.example /opt/forge/shared/.env
chmod 600 /opt/forge/shared/.env
ln -sfn /opt/forge/shared/.env .env

Use /opt/forge/shared for environment and other persistent deployment data, never commit .env.

7. Migrations and build

bash

cd /opt/forge/current
pnpm db:migrate
pnpm build

Contracts are deployed separately and addresses are entered into the shared env only after source verification.

8. PM2

Copy ops/ecosystem.config.cjs into the repo and start:

bash

cd /opt/forge/current
pm2 start ops/ecosystem.config.cjs
pm2 save
pm2 startup systemd -u forge --hp /home/forge

Run the exact command PM2 prints for startup with sudo, then:

bash

pm2 save
systemctl status pm2-forge

This provides reboot recovery. Each process also has autorestart/backoff/memory ceilings.

9. Nginx + TLS

Copy and edit ops/nginx-forge.conf:

bash

sudo cp ops/nginx-forge.conf /etc/nginx/sites-available/forge
sudo nano /etc/nginx/sites-available/forge
sudo ln -s /etc/nginx/sites-available/forge /etc/nginx/sites-enabled/forge
sudo nginx -t
sudo systemctl reload nginx

Before adding final TLS paths, you may temporarily use a normal port-80 host, then:

bash

sudo certbot --nginx -d forge.example -d www.forge.example

If Cloudflare proxy is used, still keep origin TLS valid. Do not use insecure “Flexible” SSL for a production app.

10. Log rotation

bash

sudo cp ops/logrotate-forge /etc/logrotate.d/forge-pm2
sudo logrotate -d /etc/logrotate.d/forge-pm2

You can also install pm2-logrotate, but do not depend on two competing rotation strategies without testing.

11. Health and recovery

Run:

bash

/opt/forge/current/ops/healthcheck.sh
pm2 status
pm2 logs --lines 100

Indexer health response should include:

provider connected
chain ID
latest RPC block
last indexed block
lag
latest processed block hash

Alert if lag exceeds threshold or DB/Redis is unavailable.

12. RPC failover

Configure at least:

primary HTTP
secondary HTTP
primary WS
secondary WS

On WS disconnect, indexer does not trust that no events were missed. It backfills from the stored checkpoint over HTTP and only then resumes the live tail.

13. Deploy script pattern

Use timestamped releases:

bash

release=/opt/forge/releases/$(date +%Y%m%d%H%M%S)
git clone --depth 1 git@github.com:YOURORG/forge.git "$release"
cd "$release"
ln -sfn /opt/forge/shared/.env .env
pnpm install --frozen-lockfile
pnpm db:migrate
pnpm build
ln -sfn "$release" /opt/forge/current
cd /opt/forge/current
pm2 reload ops/ecosystem.config.cjs --update-env

Keep at least the prior known-good release for rollback. Schema migrations must be backward-compatible when using zero/low-downtime reload.

14. Backup

Example daily DB dump:

bash

pg_dump "$DATABASE_URL" | gzip > /var/backups/forge-$(date +%F).sql.gz

A local dump is not a real backup by itself. Replicate encrypted backups to separate storage and test restore.

15. Initial launch order

DB/Redis
deploy/verify AgentFeeVaultFactory
configure live PONS addresses and chain RPCs
start indexer and backfill from PONS deployment block
start API/WS/web
confirm live factory events
run CLI dry run for $FORGE
launch $FORGE
confirm indexer ingests it
register platform metadata
run smoke test
enable public Explore
enable creator Launch only after end-to-end creator test and security checklist passes

DigitalOcean Production Deployment Runbook

Recommended MVP host

Use a volume/snapshot/backup policy appropriate to the value and data involved. Never use the server as the sole copy of the repo or database backups.

1. Create non-root operator

bash

adduser forge
usermod -aG sudo forge
rsync --archive --chown=forge:forge ~/.ssh /home/forge

Then log in as forge and disable password/root SSH after verifying key login.

2. Firewall

bash

sudo ufw allow OpenSSH
sudo ufw allow 'Nginx Full'
sudo ufw enable

Do not expose PostgreSQL, Redis, API internal ports, indexer health ports or PM2 RPC publicly.

3. Install base packages

bash

sudo apt update && sudo apt upgrade -y
sudo apt install -y git curl ca-certificates build-essential nginx redis-server \
  postgresql postgresql-contrib jq unzip fail2ban certbot python3-certbot-nginx

Install Node.js 22 LTS through NodeSource/nvm/your standard controlled package method. Then:

bash

corepack enable
corepack prepare pnpm@latest --activate
npm i -g pm2

Pin the Node and pnpm versions in the repo after the first successful build.

4. PostgreSQL

bash

sudo -u postgres psql

sql

CREATE USER forge WITH PASSWORD 'replace-with-long-random-password';
CREATE DATABASE forge OWNER forge;
\q

Keep PostgreSQL listening on localhost unless you intentionally move DB to a private managed service.

5. Redis

Ensure Redis binds to localhost and protected mode is enabled. For local-only deployment, firewall should block 6379 externally regardless.

bash

sudo systemctl enable --now redis-server postgresql nginx

6. Repository layout

bash

sudo mkdir -p /opt/forge/releases /opt/forge/shared
sudo chown -R forge:forge /opt/forge
cd /opt/forge

git clone git@github.com:YOURORG/forge.git releases/initial
ln -sfn /opt/forge/releases/initial /opt/forge/current
cd current
pnpm install --frozen-lockfile
cp ENV.example /opt/forge/shared/.env
chmod 600 /opt/forge/shared/.env
ln -sfn /opt/forge/shared/.env .env

Use /opt/forge/shared for environment and other persistent deployment data, never commit .env.

7. Migrations and build

bash

cd /opt/forge/current
pnpm db:migrate
pnpm build

Contracts are deployed separately and addresses are entered into the shared env only after source verification.

8. PM2

Copy ops/ecosystem.config.cjs into the repo and start:

bash

cd /opt/forge/current
pm2 start ops/ecosystem.config.cjs
pm2 save
pm2 startup systemd -u forge --hp /home/forge

Run the exact command PM2 prints for startup with sudo, then:

bash

pm2 save
systemctl status pm2-forge

This provides reboot recovery. Each process also has autorestart/backoff/memory ceilings.

9. Nginx + TLS

Copy and edit ops/nginx-forge.conf:

bash

sudo cp ops/nginx-forge.conf /etc/nginx/sites-available/forge
sudo nano /etc/nginx/sites-available/forge
sudo ln -s /etc/nginx/sites-available/forge /etc/nginx/sites-enabled/forge
sudo nginx -t
sudo systemctl reload nginx

Before adding final TLS paths, you may temporarily use a normal port-80 host, then:

bash

sudo certbot --nginx -d forge.example -d www.forge.example

If Cloudflare proxy is used, still keep origin TLS valid. Do not use insecure “Flexible” SSL for a production app.

10. Log rotation

bash

sudo cp ops/logrotate-forge /etc/logrotate.d/forge-pm2
sudo logrotate -d /etc/logrotate.d/forge-pm2

You can also install pm2-logrotate, but do not depend on two competing rotation strategies without testing.

11. Health and recovery

Run:

bash

/opt/forge/current/ops/healthcheck.sh
pm2 status
pm2 logs --lines 100

Indexer health response should include:

provider connected
chain ID
latest RPC block
last indexed block
lag
latest processed block hash

Alert if lag exceeds threshold or DB/Redis is unavailable.

12. RPC failover

Configure at least:

primary HTTP
secondary HTTP
primary WS
secondary WS

On WS disconnect, indexer does not trust that no events were missed. It backfills from the stored checkpoint over HTTP and only then resumes the live tail.

13. Deploy script pattern

Use timestamped releases:

bash

release=/opt/forge/releases/$(date +%Y%m%d%H%M%S)
git clone --depth 1 git@github.com:YOURORG/forge.git "$release"
cd "$release"
ln -sfn /opt/forge/shared/.env .env
pnpm install --frozen-lockfile
pnpm db:migrate
pnpm build
ln -sfn "$release" /opt/forge/current
cd /opt/forge/current
pm2 reload ops/ecosystem.config.cjs --update-env

Keep at least the prior known-good release for rollback. Schema migrations must be backward-compatible when using zero/low-downtime reload.

14. Backup

Example daily DB dump:

bash

pg_dump "$DATABASE_URL" | gzip > /var/backups/forge-$(date +%F).sql.gz

A local dump is not a real backup by itself. Replicate encrypted backups to separate storage and test restore.

15. Initial launch order

DB/Redis
deploy/verify AgentFeeVaultFactory
configure live PONS addresses and chain RPCs
start indexer and backfill from PONS deployment block
start API/WS/web
confirm live factory events
run CLI dry run for $FORGE
launch $FORGE
confirm indexer ingests it
register platform metadata
run smoke test
enable public Explore
enable creator Launch only after end-to-end creator test and security checklist passes