Art-gallery/Documentation/deploy-dev-to-prod.md

Deploy dev → prod (release runbook)

Step-by-step guide for promoting the development version of Gallery to production. Covers code, database schema, database data, thumbnail generation, and image files.

Default is dev. This runbook is for a scheduled release (~weekly, or when you explicitly decide to ship). Day-to-day work stays on dev — see environments.md.

Mid-week merge: If both dev and prod have catalog or image edits and you need last-write-wins sync instead of a full prod overwrite, use harmonize-dev-prod.md (npm run harmonize). Keep this runbook for releases where prod should exactly match dev.

If the one-time prod install is not done yet, follow environments.md → One-time setup and infra/docker/DEPLOY-truenas.md first.

	Dev (source)	Prod (target)
Database	gallery_dev	gallery_prod
Images	repo data/images/	/mnt/BasePool/Applications/Gallery/data/images (SMB share Gallery)
App	npm run dev:web on dev PC :5173	gallery-web container on TrueNAS :5173
URL	https://devgallery.mysuperlab.netcraze.pro	https://gallery.mysuperlab.netcraze.pro
Env file	root .env (gallery_dev)	infra/docker/.env.prod (gallery_prod)

All commands run on the dev PC from the repo root (C:\Users\SNAP\Nextcloud\Personal\Repo\Gallery) unless a step says TrueNAS.

---

One-command release (automated)

For a typical weekly release, use the orchestrator instead of running each step manually. It reads settings from a local config file, runs the enabled steps in order, prints a SUCCESS/FAILED banner at the end of each sub-command, and stops on the first failure.

Setup (once)

1. Copy the example config:

   Copy-Item infra/deploy/devtoprod.config.example.json infra/deploy/devtoprod.config.json
   

2. Edit infra/deploy/devtoprod.config.json (gitignored — safe for SMB password):

   • profile: full | code | data (base step set; individual steps overrides win)
   • steps — per-step overrides; see table below. On full profile, migrateProdSchema is on by default (runs dev:migrate on gallery_prod before restore).
   • autoConfirm: true skips restore/image-sync prompts (CONFIRM_PROD=1, -SkipConfirm)
   • autoStartDevStack: true starts dev:server (or dev:web) automatically when validateBuild finds API down
   • git.message, git.branch — used when gitCommitPush is enabled
   • smb.user / smb.password — optional; maps \\host\share before image sync
   • backupFile — optional fixed path; otherwise uses the newest gallery_dev_data_*.txt after backup

3. Prerequisites still apply: npm run dev:web running (Vite :5173 + API :3451), Docker Desktop running, docker login gitea.mysuperlab.netcraze.pro, infra/docker/.env.prod present.

   • Before migration/restore steps, verify prod DB target in .env.prod: DB_NAME=gallery_prod.

Run

npm run devtoprod:release

Or double-click deploy-dev-to-prod.cmd in the repo root.

Dry-run (print steps only, no changes):

npm run devtoprod:release -- -DryRun

What the orchestrator does

Config step	Maps to runbook
validateBuild	Step 0 — prod:build + dev /api/bounds check (API :3451 and/or Vite :5173 proxy)
gitCommitPush	Step 1 — git add, commit, push
thumbnails	Step 2 — devtoprod:thumbnails
backupDev	Step 3 — dev:db:backup
backupProd	Step 3 (rollback) — prod:db:backup
migrateProdSchema	Step 4 — dev:migrate on gallery_prod (enabled on full profile; auto-runs before restoreProd if you disabled it but left restore on)
restoreProd	Step 5 — devtoprod:db:restore
syncImages	Step 6 — devtoprod:images
dockerPublish	Step 7 — prod:docker:publish (auto tag: YYYYMMDD-HHMMSS-<gitsha> + push latest)
truenasRestartPause	Step 8 — manual pause; script waits for Enter after you restart gallery-web
verify	Step 9 — curl LAN + public /api/bounds

Final output is always a release banner:

================================================================
  RELEASE SUCCEEDED - production promoted
  Steps: validateBuild, gitCommitPush, ...
  Next:  open https://gallery.mysuperlab.netcraze.pro/
================================================================

Use the manual steps below when you need a partial release or want to inspect each step individually.

---

What changed → which steps to run

Run only the steps that match what you changed. Steps are independent except that code needs a rebuilt image and prod schema must match the dev backup before a data restore (Step 4 before Step 5).

You changed…	Required steps
Application code (client/server)	0 → 1 → 7 → 8 → 9
DB schema (new db/migrate-*.sql, schema.sql)	0 → 1 → 3 → 4 → (5 if data too) → 8 → 9
Catalog data (movements, artists, paintings, influences, bios)	0 → 2 → 3 → 4 → 5 → 9
Image files (new/replaced paintings, portraits)	0 → 2 → 3 → 5 → 6 → 9
Thumbnails only (rebuild from existing full images)	0 → 2 → 3 → 5 → 6 → 9
Everything (typical weekly release)	0 → 1 → 2 → 3 → 4 → 5 → 6 → 7 → 8 → 9

---

Prerequisites (per release)

• Prod already installed once on TrueNAS (gallery-web app exists).

• Docker Desktop running on the dev PC; logged in: docker login gitea.mysuperlab.netcraze.pro.

• Prod env file present: infra/docker/.env.prod with DB_NAME=gallery_prod (copy from .env.prod.example if missing).

• SMB share mapped for image sync (per Windows session):

  net use \\192.168.10.122\Gallery /user:YOUR_TRUENAS_USER
  

---

Step 0 — Validate on dev

Where: Dev PC + browser

1. Make sure the change works on https://devgallery.mysuperlab.netcraze.pro (or http://localhost:5173).

2. Build the client to catch type/compile errors before shipping:

   npm run prod:build
   

3. Sanity-check the dev API:

   curl.exe -sk https://devgallery.mysuperlab.netcraze.pro/api/bounds
   

Do not proceed until dev is correct — prod is a copy of dev.

---

Step 1 — Commit and push code

Where: Dev PC

Commit the release so prod is reproducible and the Docker build ships the intended code.

git status
git add .
git commit -m "Release: <summary of changes>"
git push origin main

---

Step 2 — Regenerate thumbnails on dev (before backup / image sync)

Where: Dev PC

Rebuilds thumbnail files under data/images/ and updates thumbnail_path / portrait_thumb_path in gallery_dev. Run this on dev before backing up the database and syncing images so prod receives both the files and the correct DB paths.

npm run devtoprod:thumbnails

This runs, in order:

Command	Output	DB column
dev:regenerate-thumbnails	data/images/paintings/thumbs/*_thumb.jpg	paintings.thumbnail_path
dev:regenerate-portrait-thumbs	data/images/portraits/thumbs/*_thumb.jpg	artists.portrait_thumb_path

Skip when no painting or portrait images changed since the last release and thumbs are already up to date.

Curator debug fixes on dev already regenerate thumbnails per work — no extra step for individual edits:

Debug action	Thumbnail regenerated
Fix it / More / Upload on painting detail or Checkup	paintings/thumbs/{Artist}_{Title}_thumb.jpg (~400px) + thumbnail_path in DB
Fix it / More / Upload on artist bio	portraits/thumbs/{Artist}_thumb.jpg (256px) + portrait_thumb_path in DB

Use devtoprod:thumbnails for bulk backfill (missing thumbs, imports, or releases where images changed outside the debug panel).

Order matters: thumbnail generation updates dev DB paths. If you regenerate after Step 3 (backup), run a fresh backup (Step 3 again) before Step 5 (restore), or prod will point at thumb paths that were not in the backup.

---

Step 3 — Back up the dev database

Where: Dev PC

Creates a data-only dump (INSERT statements) under db/DataBackup/.

npm run dev:db:backup

Output (note the exact path — you pass it to Step 5):

Backup written: db/DataBackup/gallery_dev_data_YYYYMMDD_HHMMSS.txt
Archive written: db/DataBackup/gallery_dev_data_YYYYMMDD_HHMMSS.zip

Also back up prod before overwriting it (optional but recommended): npm run prod:db:backup → db/DataBackup/gallery_prod_data_*.txt. Keep it as a rollback point (see Rollback).

---

Step 4 — Apply schema changes to prod (only if schema changed)

Where: Dev PC

The data restore in Step 5 only inserts rows — it does not create tables or add columns. If this release added a migration (db/migrate-*.sql) or changed db/schema.sql, the prod schema must be updated first so the new columns/tables exist.

server/migrate.js is idempotent (schema and migrations use IF NOT EXISTS / additive ALTERs). Point it at prod for one run by overriding DB_NAME (all other credentials come from .env; the gallery user has rights to both databases):

$env:DB_NAME = "gallery_prod"
npm run dev:migrate
Remove-Item Env:\DB_NAME      # restore dev default so later commands hit gallery_dev

Expect schema.sql + each migrate-*.sql to log OK. Re-running is safe.

If you prefer not to override env, temporarily set DB_NAME=gallery_prod in a scratch shell only — never edit the committed root .env to point at prod.

---

Step 5 — Restore dev data into prod (only if data/DB changed)

Where: Dev PC

Loads the Step 3 backup into gallery_prod. This TRUNCATES all public tables in prod first, then inserts dev's rows — prod becomes an exact copy of dev's data.

npm run devtoprod:db:restore -- --file db/DataBackup/gallery_dev_data_YYYYMMDD_HHMMSS.txt

Type yes at the confirmation prompt (or set CONFIRM_PROD=1 to skip it in automation).

The restore loads rows in two ways automatically:

• Fast path — if the DB role may set session_replication_role (superuser, or a PG 15+ GRANT SET ON PARAMETER), foreign-key checks are disabled for a single-pass load.
• Multi-pass fallback — the gallery role is not a superuser, so you will see note: ... using multi-pass insert. Rows whose foreign keys are not yet satisfied are retried on later passes until everything loads. No action needed.

Multi-line values (e.g. artist bios with embedded newlines) are parsed as whole statements, so long text restores correctly.

Caveats — prod tables are replaced by dev's contents:

• users is overwritten. The dev curator account and password become the prod login. curator_audit_log is not synced — prod keeps its existing audit history.
• The session table is truncated, so any active prod curator sessions are logged out.
• The target is guarded: the restore refuses to run unless the database name ends with _prod and only reads infra/docker/.env.prod.

Optional — to use the faster single-pass load, have the postgres superuser run this once in pgAdmin (role-global, covers dev and prod): GRANT SET ON PARAMETER session_replication_role TO gallery;

---

Step 6 — Sync images to prod (only if image files changed)

Where: Dev PC (SMB share mapped — see Prerequisites)

Copies data/images/ (paintings, portraits, paintings/thumbs/, and portraits/thumbs/) from the repo to the TrueNAS volume.

Map the share in this Windows session first (a fresh terminal has no mapping — robocopy will fail with ERROR 5 Access is denied otherwise):

net use \\192.168.10.122\Gallery /user:YOUR_TRUENAS_USER   # prompts for password
Test-Path "\\192.168.10.122\Gallery"                        # must print True

Then sync:

npm run devtoprod:images

Type yes when prompted. Robocopy exit codes 0–7 = success. Destination: \\192.168.10.122\Gallery\data\images.

Alternatively, let the script map the share by exporting credentials first: $env:SMB_USER="…"; $env:SMB_PASSWORD="…" before running.

---

Step 7 — Build and push the Docker image (only if code changed)

Where: Dev PC — PowerShell as Administrator, Docker Desktop running

Builds the Express API + freshly built client SPA and pushes to Gitea.

docker login gitea.mysuperlab.netcraze.pro
npm run prod:docker:publish

• Already built and only need to push: npm run prod:docker:push-only
• Image tag: gitea.mysuperlab.netcraze.pro/danilka/gallery-web:latest
• No registry / offline: .\infra\docker\save-for-truenas.ps1 → copy gallery-web.tar via the Gallery SMB share → on TrueNAS sudo bash infra/docker/truenas-load-image.sh /path/to/gallery-web.tar.

---

Step 8 — Restart gallery-web on TrueNAS (after code or schema changes)

Where: TrueNAS — Web UI

1. Apps → gallery-web → Restart (pulls :latest when pull_policy: always, otherwise redeploy to force a pull).
2. Wait for status Running.

Data-only or image-only releases (no new image) do not require a restart, but a restart is harmless.

---

Step 9 — Verify production

Where: Dev PC / browser

Bypass the router first (should return JSON instantly), then check the public URL:

curl.exe -s  http://192.168.10.122:5173/api/bounds
curl.exe -sk https://gallery.mysuperlab.netcraze.pro/api/bounds

Then open https://gallery.mysuperlab.netcraze.pro/ and confirm:

• Timeline + movement flow load; portraits appear.
• Click a portrait / movement → 3D hall renders with painting images (/images/paintings/...).
• Curator login (top-right) works with the intended credentials.

Optional on TrueNAS shell: bash infra/docker/truenas-verify.sh.

---

Rollback

Problem	Action
Bad data release	Re-run Step 5 with the pre-release prod backup: npm run devtoprod:db:restore -- --file db/DataBackup/gallery_prod_data_<pre-release>.txt
Bad code release	Rebuild from the last good commit: git checkout <good-commit>, npm run prod:docker:publish, restart gallery-web
Bad images	Re-sync from a known-good dev copy, or pull prod back to dev with npm run prodto:dev:images to compare

Keep at least the most recent gallery_prod_data_*.txt from Step 3 so a data rollback is always possible.

---

Command quick reference

Step	Command	Runs on
All (config-driven)	npm run devtoprod:release or deploy-dev-to-prod.cmd	Dev PC
0 Validate	npm run prod:build	Dev PC
1 Ship code	git add . ; git commit -m "…" ; git push origin main	Dev PC
2 Thumbnails	npm run devtoprod:thumbnails	Dev PC
3 Backup dev	npm run dev:db:backup	Dev PC
3 Backup prod (rollback point)	npm run prod:db:backup	Dev PC
4 Migrate prod schema	$env:DB_NAME="gallery_prod"; npm run dev:migrate; Remove-Item Env:\DB_NAME	Dev PC
5 Restore data → prod	npm run devtoprod:db:restore -- --file <dev backup>.txt	Dev PC
6 Sync images → prod	npm run devtoprod:images	Dev PC (SMB)
7 Build + push image	npm run prod:docker:publish	Dev PC (Admin)
8 Restart app	Apps → gallery-web → Restart	TrueNAS UI
9 Verify	curl.exe -sk https://gallery.mysuperlab.netcraze.pro/api/bounds	Dev PC

---

Safety guards (built in)

• Prod DB scripts read only infra/docker/.env.prod; dev scripts refuse _prod database names.
• Prod restore/backup require typing yes (or CONFIRM_PROD=1 / autoConfirm in release config).
• npm run dev:migrate targets whatever DB_NAME is set — always Remove-Item Env:\DB_NAME after Step 4 so later commands stay on dev.
• restoreProd safety: if migrateProdSchema is disabled in config but restoreProd is enabled, the orchestrator runs Step 4 first so prod has columns/tables present in the dev backup (e.g. updated_at, entity_translations).
• Deploy-related npm scripts and PowerShell helpers print a final ===== SUCCESS: … ===== or ===== FAILED: … ===== banner as the last line of output.

Troubleshooting

Symptom	Cause	Fix
/api/bounds returns {"min_year":null,"max_year":null} on prod	gallery_prod has schema but no data	Run Step 5 (restore); confirm rows with a count query on art_movements / paintings
robocopy ... ERROR 5 (0x00000005) Access is denied on Step 6	SMB share not mapped/authenticated in this Windows session	net use \\192.168.10.122\Gallery /user:YOUR_TRUENAS_USER, verify Test-Path, then re-run. If still denied after auth, fix TrueNAS dataset ACL (chown -R 1001:1001, grant the SMB user write)
Restore: permission denied to set parameter "session_replication_role"	DB role is not superuser	Handled automatically (multi-pass insert). For the fast path, superuser runs GRANT SET ON PARAMETER session_replication_role TO gallery;
Restore: column "updated_at" of relation … does not exist	Prod schema behind dev backup	Run Step 4 before Step 5; full profile enables migrateProdSchema by default; partial config with restoreProd alone also auto-migrates
Restore: unterminated quoted string	Old parser split multi-line values (bios)	Fixed — statements are accumulated until quotes balance; update to latest scripts/restore-db-data.js
git push → Failed to authenticate user (Gitea)	Git Credential Manager cached an expired token	Clear it: "protocol=httpsnhost=gitea.mysuperlab.netcraze.pron" | git credential reject, then push again to re-prompt
Prod tables empty after a failed restore	Restore truncates before inserting; a mid-run error leaves tables empty	Dev is untouched — just re-run Step 5
Timeline portraits slow or missing on prod	Thumbs not regenerated or not synced	Run Step 2 on dev, then Step 3 (fresh backup), Step 5, and Step 6
502 / 504 on the public prod URL	Keenetic upstream wrong (IP/port/protocol)	192.168.10.122:5173, protocol http; verify LAN curl.exe -s http://192.168.10.122:5173/api/bounds first

Related docs

Document	Contents
environments.md	Full dev/prod walkthrough, one-time setup, Keenetic
infra/docker/DEPLOY-truenas.md	TrueNAS container install and troubleshooting
FAC.md	Command cheat sheet
data-and-images.md	Catalog and image pipeline