Art-gallery/Documentation/deploy-dev-to-prod.md

Deploy dev → prod (release runbook)

Step-by-step guide for promoting the development version of Gallery to production. Covers code, database schema, database data, thumbnail generation, and image files.

Default is dev. This runbook is for a scheduled release (~weekly, or when you explicitly decide to ship). Day-to-day work stays on dev — see environments.md. If the one-time prod install is not done yet, follow environments.md → One-time setup and infra/docker/DEPLOY-truenas.md first.

	Dev (source)	Prod (target)
Database	gallery_dev	gallery_prod
Images	repo data/images/	/mnt/BasePool/Applications/Gallery/data/images (SMB share Gallery)
App	npm run dev:web on dev PC :5173	gallery-web container on TrueNAS :5173
URL	https://devgallery.mysuperlab.netcraze.pro	https://gallery.mysuperlab.netcraze.pro
Env file	root .env (gallery_dev)	infra/docker/.env.prod (gallery_prod)

All commands run on the dev PC from the repo root (C:\Users\SNAP\Nextcloud\Personal\Repo\Gallery) unless a step says TrueNAS.

---

One-command release (automated)

For a typical weekly release, use the orchestrator instead of running each step manually. It reads settings from a local config file, runs the enabled steps in order, prints a SUCCESS/FAILED banner at the end of each sub-command, and stops on the first failure.

Setup (once)

1. Copy the example config:

   Copy-Item infra/deploy/devtoprod.config.example.json infra/deploy/devtoprod.config.json
   

2. Edit infra/deploy/devtoprod.config.json (gitignored — safe for SMB password):

   • profile: full | code | data (base step set; individual steps overrides win)
   • autoConfirm: true skips restore/image-sync prompts (CONFIRM_PROD=1, -SkipConfirm)
   • autoStartDevStack: true starts dev:server (or dev:web) automatically when validateBuild finds API down
   • git.message, git.branch — used when gitCommitPush is enabled
   • schemaChanged: true enables prod schema migration on full profile
   • smb.user / smb.password — optional; maps \\host\share before image sync
   • backupFile — optional fixed path; otherwise uses the newest gallery_dev_data_*.txt after backup

3. Prerequisites still apply: npm run dev:web running (Vite :5173 + API :3451), Docker Desktop running, docker login gitea.mysuperlab.netcraze.pro, infra/docker/.env.prod present.

   • Before migration/restore steps, verify prod DB target in .env.prod: DB_NAME=gallery_prod.

Run

npm run devtoprod:release

Or double-click deploy-dev-to-prod.cmd in the repo root.

Dry-run (print steps only, no changes):

npm run devtoprod:release -- -DryRun

What the orchestrator does

Config step	Maps to runbook
validateBuild	Step 0 — prod:build + dev /api/bounds check (API :3451 and/or Vite :5173 proxy)
gitCommitPush	Step 1 — git add, commit, push
thumbnails	Step 2 — devtoprod:thumbnails
backupDev	Step 3 — dev:db:backup
backupProd	Step 3 (rollback) — prod:db:backup
migrateProdSchema	Step 4 — dev:migrate on gallery_prod
restoreProd	Step 5 — devtoprod:db:restore
syncImages	Step 6 — devtoprod:images
dockerPublish	Step 7 — prod:docker:publish (auto tag: YYYYMMDD-HHMMSS-<gitsha> + push latest)
truenasRestartPause	Step 8 — manual pause; script waits for Enter after you restart gallery-web
verify	Step 9 — curl LAN + public /api/bounds

Final output is always a release banner:

================================================================
  RELEASE SUCCEEDED - production promoted
  Steps: validateBuild, gitCommitPush, ...
  Next:  open https://gallery.mysuperlab.netcraze.pro/
================================================================

Use the manual steps below when you need a partial release or want to inspect each step individually.

---

What changed → which steps to run

Run only the steps that match what you changed. Steps are independent except that code needs a rebuilt image and schema changes must be applied before a data restore.

You changed…	Required steps
Application code (client/server)	0 → 1 → 7 → 8 → 9
DB schema (new db/migrate-*.sql, schema.sql)	0 → 1 → 3 → 4 → (5 if data too) → 8 → 9
Catalog data (movements, artists, paintings, influences, bios)	0 → 2 → 3 → 5 → 9
Image files (new/replaced paintings, portraits)	0 → 2 → 3 → 5 → 6 → 9
Thumbnails only (rebuild from existing full images)	0 → 2 → 3 → 5 → 6 → 9
Everything (typical weekly release)	0 → 1 → 2 → 3 → 4 → 5 → 6 → 7 → 8 → 9

---

Prerequisites (per release)

• Prod already installed once on TrueNAS (gallery-web app exists).

• Docker Desktop running on the dev PC; logged in: docker login gitea.mysuperlab.netcraze.pro.

• Prod env file present: infra/docker/.env.prod with DB_NAME=gallery_prod (copy from .env.prod.example if missing).

• SMB share mapped for image sync (per Windows session):

  net use \\192.168.10.122\Gallery /user:YOUR_TRUENAS_USER
  

---

Step 0 — Validate on dev

Where: Dev PC + browser

1. Make sure the change works on https://devgallery.mysuperlab.netcraze.pro (or http://localhost:5173).

2. Build the client to catch type/compile errors before shipping:

   npm run prod:build
   

3. Sanity-check the dev API:

   curl.exe -sk https://devgallery.mysuperlab.netcraze.pro/api/bounds
   

Do not proceed until dev is correct — prod is a copy of dev.

---

Step 1 — Commit and push code

Where: Dev PC

Commit the release so prod is reproducible and the Docker build ships the intended code.

git status
git add .
git commit -m "Release: <summary of changes>"
git push origin main

---

Step 2 — Regenerate thumbnails on dev (before backup / image sync)

Where: Dev PC

Rebuilds thumbnail files under data/images/ and updates thumbnail_path / portrait_thumb_path in gallery_dev. Run this on dev before backing up the database and syncing images so prod receives both the files and the correct DB paths.

npm run devtoprod:thumbnails

This runs, in order:

Command	Output	DB column
dev:regenerate-thumbnails	data/images/paintings/thumbs/*_thumb.jpg	paintings.thumbnail_path
dev:regenerate-portrait-thumbs	data/images/portraits/thumbs/*_thumb.jpg	artists.portrait_thumb_path

Skip when no painting or portrait images changed since the last release and thumbs are already up to date.

Curator debug fixes on dev already regenerate thumbnails per work — no extra step for individual edits:

Debug action	Thumbnail regenerated
Fix it / More / Upload on painting detail or Checkup	paintings/thumbs/{Artist}_{Title}_thumb.jpg (~400px) + thumbnail_path in DB
Fix it / More / Upload on artist bio	portraits/thumbs/{Artist}_thumb.jpg (256px) + portrait_thumb_path in DB

Use devtoprod:thumbnails for bulk backfill (missing thumbs, imports, or releases where images changed outside the debug panel).

Order matters: thumbnail generation updates dev DB paths. If you regenerate after Step 3 (backup), run a fresh backup (Step 3 again) before Step 5 (restore), or prod will point at thumb paths that were not in the backup.

---

Step 3 — Back up the dev database

Where: Dev PC

Creates a data-only dump (INSERT statements) under db/DataBackup/.

npm run dev:db:backup

Output (note the exact path — you pass it to Step 5):

Backup written: db/DataBackup/gallery_dev_data_YYYYMMDD_HHMMSS.txt
Archive written: db/DataBackup/gallery_dev_data_YYYYMMDD_HHMMSS.zip

Also back up prod before overwriting it (optional but recommended): npm run prod:db:backup → db/DataBackup/gallery_prod_data_*.txt. Keep it as a rollback point (see Rollback).

---

Step 4 — Apply schema changes to prod (only if schema changed)

Where: Dev PC

The data restore in Step 5 only inserts rows — it does not create tables or add columns. If this release added a migration (db/migrate-*.sql) or changed db/schema.sql, the prod schema must be updated first so the new columns/tables exist.

server/migrate.js is idempotent (schema and migrations use IF NOT EXISTS / additive ALTERs). Point it at prod for one run by overriding DB_NAME (all other credentials come from .env; the gallery user has rights to both databases):

$env:DB_NAME = "gallery_prod"
npm run dev:migrate
Remove-Item Env:\DB_NAME      # restore dev default so later commands hit gallery_dev

Expect schema.sql + each migrate-*.sql to log OK. Re-running is safe.

If you prefer not to override env, temporarily set DB_NAME=gallery_prod in a scratch shell only — never edit the committed root .env to point at prod.

---

Step 5 — Restore dev data into prod (only if data/DB changed)

Where: Dev PC

Loads the Step 3 backup into gallery_prod. This TRUNCATES all public tables in prod first, then inserts dev's rows — prod becomes an exact copy of dev's data.

npm run devtoprod:db:restore -- --file db/DataBackup/gallery_dev_data_YYYYMMDD_HHMMSS.txt

Type yes at the confirmation prompt (or set CONFIRM_PROD=1 to skip it in automation).

The restore loads rows in two ways automatically:

• Fast path — if the DB role may set session_replication_role (superuser, or a PG 15+ GRANT SET ON PARAMETER), foreign-key checks are disabled for a single-pass load.
• Multi-pass fallback — the gallery role is not a superuser, so you will see note: ... using multi-pass insert. Rows whose foreign keys are not yet satisfied are retried on later passes until everything loads. No action needed.

Multi-line values (e.g. artist bios with embedded newlines) are parsed as whole statements, so long text restores correctly.

Caveats — prod tables are replaced by dev's contents:

• users and curator_audit_log are overwritten. The dev curator account and password become the prod login, and prod audit history is replaced. Make sure the dev curator credentials are the ones you want in prod.
• The session table is truncated, so any active prod curator sessions are logged out.
• The target is guarded: the restore refuses to run unless the database name ends with _prod and only reads infra/docker/.env.prod.

Optional — to use the faster single-pass load, have the postgres superuser run this once in pgAdmin (role-global, covers dev and prod): GRANT SET ON PARAMETER session_replication_role TO gallery;

---

Step 6 — Sync images to prod (only if image files changed)

Where: Dev PC (SMB share mapped — see Prerequisites)

Copies data/images/ (paintings, portraits, paintings/thumbs/, and portraits/thumbs/) from the repo to the TrueNAS volume.

Map the share in this Windows session first (a fresh terminal has no mapping — robocopy will fail with ERROR 5 Access is denied otherwise):

net use \\192.168.10.122\Gallery /user:YOUR_TRUENAS_USER   # prompts for password
Test-Path "\\192.168.10.122\Gallery"                        # must print True

Then sync:

npm run devtoprod:images

Type yes when prompted. Robocopy exit codes 0–7 = success. Destination: \\192.168.10.122\Gallery\data\images.

Alternatively, let the script map the share by exporting credentials first: $env:SMB_USER="…"; $env:SMB_PASSWORD="…" before running.

---

Step 7 — Build and push the Docker image (only if code changed)

Where: Dev PC — PowerShell as Administrator, Docker Desktop running

Builds the Express API + freshly built client SPA and pushes to Gitea.

docker login gitea.mysuperlab.netcraze.pro
npm run prod:docker:publish

• Already built and only need to push: npm run prod:docker:push-only
• Image tag: gitea.mysuperlab.netcraze.pro/danilka/gallery-web:latest
• No registry / offline: .\infra\docker\save-for-truenas.ps1 → copy gallery-web.tar via the Gallery SMB share → on TrueNAS sudo bash infra/docker/truenas-load-image.sh /path/to/gallery-web.tar.

---

Step 8 — Restart gallery-web on TrueNAS (after code or schema changes)

Where: TrueNAS — Web UI

1. Apps → gallery-web → Restart (pulls :latest when pull_policy: always, otherwise redeploy to force a pull).
2. Wait for status Running.

Data-only or image-only releases (no new image) do not require a restart, but a restart is harmless.

---

Step 9 — Verify production

Where: Dev PC / browser

Bypass the router first (should return JSON instantly), then check the public URL:

curl.exe -s  http://192.168.10.122:5173/api/bounds
curl.exe -sk https://gallery.mysuperlab.netcraze.pro/api/bounds

Then open https://gallery.mysuperlab.netcraze.pro/ and confirm:

• Timeline + movement flow load; portraits appear.
• Click a portrait / movement → 3D hall renders with painting images (/images/paintings/...).
• Curator login (top-right) works with the intended credentials.

Optional on TrueNAS shell: bash infra/docker/truenas-verify.sh.

---

Rollback

Problem	Action
Bad data release	Re-run Step 5 with the pre-release prod backup: npm run devtoprod:db:restore -- --file db/DataBackup/gallery_prod_data_<pre-release>.txt
Bad code release	Rebuild from the last good commit: git checkout <good-commit>, npm run prod:docker:publish, restart gallery-web
Bad images	Re-sync from a known-good dev copy, or pull prod back to dev with npm run prodto:dev:images to compare

Keep at least the most recent gallery_prod_data_*.txt from Step 3 so a data rollback is always possible.

---

Command quick reference

Step	Command	Runs on
All (config-driven)	npm run devtoprod:release or deploy-dev-to-prod.cmd	Dev PC
0 Validate	npm run prod:build	Dev PC
1 Ship code	git add . ; git commit -m "…" ; git push origin main	Dev PC
2 Thumbnails	npm run devtoprod:thumbnails	Dev PC
3 Backup dev	npm run dev:db:backup	Dev PC
3 Backup prod (rollback point)	npm run prod:db:backup	Dev PC
4 Migrate prod schema	$env:DB_NAME="gallery_prod"; npm run dev:migrate; Remove-Item Env:\DB_NAME	Dev PC
5 Restore data → prod	npm run devtoprod:db:restore -- --file <dev backup>.txt	Dev PC
6 Sync images → prod	npm run devtoprod:images	Dev PC (SMB)
7 Build + push image	npm run prod:docker:publish	Dev PC (Admin)
8 Restart app	Apps → gallery-web → Restart	TrueNAS UI
9 Verify	curl.exe -sk https://gallery.mysuperlab.netcraze.pro/api/bounds	Dev PC

---

Safety guards (built in)

• Prod DB scripts read only infra/docker/.env.prod; dev scripts refuse _prod database names.
• Prod restore/backup require typing yes (or CONFIRM_PROD=1 / autoConfirm in release config).
• npm run dev:migrate targets whatever DB_NAME is set — always Remove-Item Env:\DB_NAME after Step 4 so later commands stay on dev.
• Deploy-related npm scripts and PowerShell helpers print a final ===== SUCCESS: … ===== or ===== FAILED: … ===== banner as the last line of output.

Troubleshooting

Symptom	Cause	Fix
/api/bounds returns {"min_year":null,"max_year":null} on prod	gallery_prod has schema but no data	Run Step 5 (restore); confirm rows with a count query on art_movements / paintings
robocopy ... ERROR 5 (0x00000005) Access is denied on Step 6	SMB share not mapped/authenticated in this Windows session	net use \\192.168.10.122\Gallery /user:YOUR_TRUENAS_USER, verify Test-Path, then re-run. If still denied after auth, fix TrueNAS dataset ACL (chown -R 1001:1001, grant the SMB user write)
Restore: permission denied to set parameter "session_replication_role"	DB role is not superuser	Handled automatically (multi-pass insert). For the fast path, superuser runs GRANT SET ON PARAMETER session_replication_role TO gallery;
Restore: unterminated quoted string	Old parser split multi-line values (bios)	Fixed — statements are accumulated until quotes balance; update to latest scripts/restore-db-data.js
git push → Failed to authenticate user (Gitea)	Git Credential Manager cached an expired token	Clear it: "protocol=httpsnhost=gitea.mysuperlab.netcraze.pron" | git credential reject, then push again to re-prompt
Prod tables empty after a failed restore	Restore truncates before inserting; a mid-run error leaves tables empty	Dev is untouched — just re-run Step 5
Timeline portraits slow or missing on prod	Thumbs not regenerated or not synced	Run Step 2 on dev, then Step 3 (fresh backup), Step 5, and Step 6
502 / 504 on the public prod URL	Keenetic upstream wrong (IP/port/protocol)	192.168.10.122:5173, protocol http; verify LAN curl.exe -s http://192.168.10.122:5173/api/bounds first

Related docs

Document	Contents
environments.md	Full dev/prod walkthrough, one-time setup, Keenetic
infra/docker/DEPLOY-truenas.md	TrueNAS container install and troubleshooting
FAC.md	Command cheat sheet
data-and-images.md	Catalog and image pipeline