Art-gallery/Documentation/deploy-dev-to-prod.md

# Deploy dev → prod (release runbook)

Step-by-step guide for promoting the **development** version of Gallery to **production**. Covers code, database schema, database data, thumbnail generation, and image files.

> **Default is dev.** This runbook is for a **scheduled release** (~weekly, or when you explicitly decide to ship). Day-to-day work stays on dev — see [environments.md](environments.md#development-first-workflow-default).
>
> **Mid-week merge:** If both dev and prod have catalog or image edits and you need **last-write-wins** sync instead of a full prod overwrite, use [harmonize-dev-prod.md](harmonize-dev-prod.md) (`npm run harmonize`). Keep this runbook for releases where prod should exactly match dev.
>
> If the one-time prod install is not done yet, follow [environments.md → One-time setup](environments.md#one-time-setup-full-walkthrough) and [infra/docker/DEPLOY-truenas.md](../infra/docker/DEPLOY-truenas.md) first.

| | Dev (source) | Prod (target) |
|---|---|---|
| Database | `gallery_dev` | `gallery_prod` |
| Images | repo `data/images/` | `/mnt/BasePool/Applications/Gallery/data/images` (SMB share **`Gallery`**) |
| App | `npm run dev:web` on dev PC `:5173` | `gallery-web` container on TrueNAS `:5173` |
| URL | https://devgallery.mysuperlab.netcraze.pro | https://gallery.mysuperlab.netcraze.pro |
| Env file | root [`.env`](../.env) (`gallery_dev`) | [`infra/docker/.env.prod`](../infra/docker/.env.prod) (`gallery_prod`) |

All commands run on the **dev PC** from the repo root (`C:\Users\SNAP\Nextcloud\Personal\Repo\Gallery`) unless a step says **TrueNAS**.

---

## One-command release (automated)

For a typical weekly release, use the orchestrator instead of running each step manually. It reads settings from a local config file, runs the enabled steps in order, prints a **SUCCESS/FAILED** banner at the end of each sub-command, and stops on the first failure.

### Setup (once)

1. Copy the example config:

   ```powershell
   Copy-Item infra/deploy/devtoprod.config.example.json infra/deploy/devtoprod.config.json
   ```

2. Edit [`infra/deploy/devtoprod.config.json`](../infra/deploy/devtoprod.config.json) (gitignored — safe for SMB password):
   - `profile`: `full` | `code` | `data` (base step set; individual `steps` overrides win)
   - `steps` — per-step overrides; see table below. On **`full`** profile, `migrateProdSchema` is **on by default** (runs `dev:migrate` on `gallery_prod` before restore).
   - `autoConfirm`: `true` skips restore/image-sync prompts (`CONFIRM_PROD=1`, `-SkipConfirm`)
   - `autoStartDevStack`: `true` starts `dev:server` (or `dev:web`) automatically when `validateBuild` finds API down
   - `git.message`, `git.branch` — used when `gitCommitPush` is enabled
   - `smb.user` / `smb.password` — optional; maps `\\host\share` before image sync
   - `backupFile` — optional fixed path; otherwise uses the newest `gallery_dev_data_*.txt` after backup

3. Prerequisites still apply: **`npm run dev:web` running** (Vite `:5173` + API `:3451`), Docker Desktop running, `docker login gitea.mysuperlab.netcraze.pro`, [`infra/docker/.env.prod`](../infra/docker/.env.prod) present.
   - Before migration/restore steps, verify prod DB target in `.env.prod`: `DB_NAME=gallery_prod`.

### Run

```powershell
npm run devtoprod:release
```

Or double-click [`deploy-dev-to-prod.cmd`](../deploy-dev-to-prod.cmd) in the repo root.

Dry-run (print steps only, no changes):

```powershell
npm run devtoprod:release -- -DryRun
```

### What the orchestrator does

| Config step | Maps to runbook |
|-------------|-----------------|
| `validateBuild` | Step 0 — `prod:build` + dev `/api/bounds` check (API `:3451` and/or Vite `:5173` proxy) |
| `gitCommitPush` | Step 1 — `git add`, commit, push |
| `thumbnails` | Step 2 — `devtoprod:thumbnails` |
| `backupDev` | Step 3 — `dev:db:backup` |
| `backupProd` | Step 3 (rollback) — `prod:db:backup` |
| `migrateProdSchema` | Step 4 — `dev:migrate` on `gallery_prod` (enabled on `full` profile; auto-runs before `restoreProd` if you disabled it but left restore on) |
| `restoreProd` | Step 5 — `devtoprod:db:restore` |
| `syncImages` | Step 6 — `devtoprod:images` |
| `dockerPublish` | Step 7 — `prod:docker:publish` (auto tag: `YYYYMMDD-HHMMSS-<gitsha>` + push `latest`) |
| `truenasRestartPause` | Step 8 — **manual** pause; script waits for Enter after you restart **gallery-web** |
| `verify` | Step 9 — curl LAN + public `/api/bounds` |

Final output is always a release banner:

```
================================================================
  RELEASE SUCCEEDED - production promoted
  Steps: validateBuild, gitCommitPush, ...
  Next:  open https://gallery.mysuperlab.netcraze.pro/
================================================================
```

Use the manual steps below when you need a **partial** release or want to inspect each step individually.

---

## What changed → which steps to run

Run only the steps that match what you changed. Steps are independent except that **code** needs a rebuilt image and **prod schema must match the dev backup** before a data restore (Step 4 before Step 5).

| You changed… | Required steps |
|--------------|----------------|
| Application code (client/server) | 0 → 1 → 7 → 8 → 9 |
| DB schema (new `db/migrate-*.sql`, `schema.sql`) | 0 → 1 → 3 → **4** → (5 if data too) → 8 → 9 |
| Catalog data (movements, artists, paintings, influences, bios) | 0 → 2 → 3 → **4** → 5 → 9 |
| Image files (new/replaced paintings, portraits) | 0 → 2 → 3 → 5 → 6 → 9 |
| Thumbnails only (rebuild from existing full images) | 0 → 2 → 3 → 5 → 6 → 9 |
| Everything (typical weekly release) | 0 → 1 → 2 → 3 → 4 → 5 → 6 → 7 → 8 → 9 |

---

## Prerequisites (per release)

- Prod already installed once on TrueNAS (`gallery-web` app exists).
- **Docker Desktop** running on the dev PC; logged in: `docker login gitea.mysuperlab.netcraze.pro`.
- Prod env file present: [`infra/docker/.env.prod`](../infra/docker/.env.prod) with `DB_NAME=gallery_prod` (copy from [`.env.prod.example`](../infra/docker/.env.prod.example) if missing).
- SMB share mapped for image sync (per Windows session):

  ```powershell
  net use \\192.168.10.122\Gallery /user:YOUR_TRUENAS_USER
  ```

---

## Step 0 — Validate on dev

**Where:** Dev PC + browser

1. Make sure the change works on https://devgallery.mysuperlab.netcraze.pro (or `http://localhost:5173`).
2. Build the client to catch type/compile errors before shipping:

   ```powershell
   npm run prod:build
   ```

3. Sanity-check the dev API:

   ```powershell
   curl.exe -sk https://devgallery.mysuperlab.netcraze.pro/api/bounds
   ```

Do not proceed until dev is correct — prod is a copy of dev.

---

## Step 1 — Commit and push code

**Where:** Dev PC

Commit the release so prod is reproducible and the Docker build ships the intended code.

```powershell
git status
git add .
git commit -m "Release: <summary of changes>"
git push origin main
```

---

## Step 2 — Regenerate thumbnails on dev (before backup / image sync)

**Where:** Dev PC

Rebuilds thumbnail **files** under `data/images/` and updates **`thumbnail_path`** / **`portrait_thumb_path`** in **`gallery_dev`**. Run this on dev **before** backing up the database and syncing images so prod receives both the files and the correct DB paths.

```powershell
npm run devtoprod:thumbnails
```

This runs, in order:

| Command | Output | DB column |
|---------|--------|-----------|
| `dev:regenerate-thumbnails` | `data/images/paintings/thumbs/*_thumb.jpg` | `paintings.thumbnail_path` |
| `dev:regenerate-portrait-thumbs` | `data/images/portraits/thumbs/*_thumb.jpg` | `artists.portrait_thumb_path` |

Skip when no painting or portrait images changed since the last release and thumbs are already up to date.

**Curator debug fixes on dev** already regenerate thumbnails per work — no extra step for individual edits:

| Debug action | Thumbnail regenerated |
|--------------|----------------------|
| **Fix it** / **More** / **Upload** on painting detail or Checkup | `paintings/thumbs/{Artist}_{Title}_thumb.jpg` (~400px) + `thumbnail_path` in DB |
| **Fix it** / **More** / **Upload** on artist bio | `portraits/thumbs/{Artist}_thumb.jpg` (256px) + `portrait_thumb_path` in DB |

Use `devtoprod:thumbnails` for **bulk** backfill (missing thumbs, imports, or releases where images changed outside the debug panel).

> **Order matters:** thumbnail generation updates dev DB paths. If you regenerate **after** Step 3 (backup), run a **fresh backup** (Step 3 again) before Step 5 (restore), or prod will point at thumb paths that were not in the backup.

---

## Step 3 — Back up the dev database

**Where:** Dev PC

Creates a data-only dump (INSERT statements) under `db/DataBackup/`.

```powershell
npm run dev:db:backup
```

Output (note the exact path — you pass it to Step 5):

```
Backup written: db/DataBackup/gallery_dev_data_YYYYMMDD_HHMMSS.txt
Archive written: db/DataBackup/gallery_dev_data_YYYYMMDD_HHMMSS.zip
```

> Also back up prod before overwriting it (optional but recommended): `npm run prod:db:backup` → `db/DataBackup/gallery_prod_data_*.txt`. Keep it as a rollback point (see [Rollback](#rollback)).

---

## Step 4 — Apply schema changes to prod (only if schema changed)

**Where:** Dev PC

The data restore in Step 5 only inserts rows — it does **not** create tables or add columns. If this release added a migration (`db/migrate-*.sql`) or changed `db/schema.sql`, the prod schema must be updated **first** so the new columns/tables exist.

`server/migrate.js` is idempotent (schema and migrations use `IF NOT EXISTS` / additive `ALTER`s). Point it at prod for one run by overriding `DB_NAME` (all other credentials come from `.env`; the `gallery` user has rights to both databases):

```powershell
$env:DB_NAME = "gallery_prod"
npm run dev:migrate
Remove-Item Env:\DB_NAME      # restore dev default so later commands hit gallery_dev
```

Expect `schema.sql` + each `migrate-*.sql` to log `OK`. Re-running is safe.

> If you prefer not to override env, temporarily set `DB_NAME=gallery_prod` in a scratch shell only — **never** edit the committed root `.env` to point at prod.

---

## Step 5 — Restore dev data into prod (only if data/DB changed)

**Where:** Dev PC

Loads the Step 3 backup into `gallery_prod`. This **TRUNCATES all public tables** in prod first, then inserts dev's rows — prod becomes an exact copy of dev's data.

```powershell
npm run devtoprod:db:restore -- --file db/DataBackup/gallery_dev_data_YYYYMMDD_HHMMSS.txt
```

Type `yes` at the confirmation prompt (or set `CONFIRM_PROD=1` to skip it in automation).

The restore loads rows in two ways automatically:

- **Fast path** — if the DB role may set `session_replication_role` (superuser, or a PG 15+ `GRANT SET ON PARAMETER`), foreign-key checks are disabled for a single-pass load.
- **Multi-pass fallback** — the `gallery` role is **not** a superuser, so you will see `note: ... using multi-pass insert`. Rows whose foreign keys are not yet satisfied are retried on later passes until everything loads. No action needed.

Multi-line values (e.g. artist bios with embedded newlines) are parsed as whole statements, so long text restores correctly.

**Caveats — prod tables are replaced by dev's contents:**

- `users` is overwritten. The **dev curator account and password become the prod login**. `curator_audit_log` is **not** synced — prod keeps its existing audit history.
- The `session` table is truncated, so any active prod curator sessions are logged out.
- The target is guarded: the restore refuses to run unless the database name ends with `_prod` and only reads `infra/docker/.env.prod`.

> **Optional** — to use the faster single-pass load, have the postgres superuser run this once in pgAdmin (role-global, covers dev and prod): `GRANT SET ON PARAMETER session_replication_role TO gallery;`

---

## Step 6 — Sync images to prod (only if image files changed)

**Where:** Dev PC (SMB share mapped — see Prerequisites)

Copies `data/images/` (paintings, portraits, `paintings/thumbs/`, and `portraits/thumbs/`) from the repo to the TrueNAS volume.

**Map the share in this Windows session first** (a fresh terminal has no mapping — robocopy will fail with `ERROR 5 Access is denied` otherwise):

```powershell
net use \\192.168.10.122\Gallery /user:YOUR_TRUENAS_USER   # prompts for password
Test-Path "\\192.168.10.122\Gallery"                        # must print True
```

Then sync:

```powershell
npm run devtoprod:images
```

Type `yes` when prompted. Robocopy exit codes **0–7** = success. Destination: `\\192.168.10.122\Gallery\data\images`.

Alternatively, let the script map the share by exporting credentials first: `$env:SMB_USER="…"; $env:SMB_PASSWORD="…"` before running.

---

## Step 7 — Build and push the Docker image (only if code changed)

**Where:** Dev PC — **PowerShell as Administrator**, Docker Desktop running

Builds the Express API + freshly built client SPA and pushes to Gitea.

```powershell
docker login gitea.mysuperlab.netcraze.pro
npm run prod:docker:publish
```

- Already built and only need to push: `npm run prod:docker:push-only`
- Image tag: `gitea.mysuperlab.netcraze.pro/danilka/gallery-web:latest`
- No registry / offline: `.\infra\docker\save-for-truenas.ps1` → copy `gallery-web.tar` via the `Gallery` SMB share → on TrueNAS `sudo bash infra/docker/truenas-load-image.sh /path/to/gallery-web.tar`.

---

## Step 8 — Restart gallery-web on TrueNAS (after code or schema changes)

**Where:** TrueNAS — Web UI

1. **Apps** → **gallery-web** → **Restart** (pulls `:latest` when `pull_policy: always`, otherwise redeploy to force a pull).
2. Wait for status **Running**.

Data-only or image-only releases (no new image) do not require a restart, but a restart is harmless.

---

## Step 9 — Verify production

**Where:** Dev PC / browser

Bypass the router first (should return JSON instantly), then check the public URL:

```powershell
curl.exe -s  http://192.168.10.122:5173/api/bounds
curl.exe -sk https://gallery.mysuperlab.netcraze.pro/api/bounds
```

Then open **https://gallery.mysuperlab.netcraze.pro/** and confirm:

- Timeline + movement flow load; portraits appear.
- Click a portrait / movement → 3D hall renders with painting images (`/images/paintings/...`).
- **Curator login** (top-right) works with the intended credentials.

Optional on TrueNAS shell: `bash infra/docker/truenas-verify.sh`.

---

## Rollback

| Problem | Action |
|---------|--------|
| Bad **data** release | Re-run Step 5 with the pre-release prod backup: `npm run devtoprod:db:restore -- --file db/DataBackup/gallery_prod_data_<pre-release>.txt` |
| Bad **code** release | Rebuild from the last good commit: `git checkout <good-commit>`, `npm run prod:docker:publish`, restart `gallery-web` |
| Bad **images** | Re-sync from a known-good dev copy, or pull prod back to dev with `npm run prodto:dev:images` to compare |

Keep at least the most recent `gallery_prod_data_*.txt` from Step 3 so a data rollback is always possible.

---

## Command quick reference

| Step | Command | Runs on |
|------|---------|---------|
| **All (config-driven)** | `npm run devtoprod:release` or `deploy-dev-to-prod.cmd` | Dev PC |
| 0 Validate | `npm run prod:build` | Dev PC |
| 1 Ship code | `git add . ; git commit -m "…" ; git push origin main` | Dev PC |
| 2 Thumbnails | `npm run devtoprod:thumbnails` | Dev PC |
| 3 Backup dev | `npm run dev:db:backup` | Dev PC |
| 3 Backup prod (rollback point) | `npm run prod:db:backup` | Dev PC |
| 4 Migrate prod schema | `$env:DB_NAME="gallery_prod"; npm run dev:migrate; Remove-Item Env:\DB_NAME` | Dev PC |
| 5 Restore data → prod | `npm run devtoprod:db:restore -- --file <dev backup>.txt` | Dev PC |
| 6 Sync images → prod | `npm run devtoprod:images` | Dev PC (SMB) |
| 7 Build + push image | `npm run prod:docker:publish` | Dev PC (Admin) |
| 8 Restart app | Apps → gallery-web → Restart | TrueNAS UI |
| 9 Verify | `curl.exe -sk https://gallery.mysuperlab.netcraze.pro/api/bounds` | Dev PC |

---

## Safety guards (built in)

- Prod DB scripts read **only** `infra/docker/.env.prod`; dev scripts refuse `_prod` database names.
- Prod restore/backup require typing `yes` (or `CONFIRM_PROD=1` / `autoConfirm` in release config).
- `npm run dev:migrate` targets whatever `DB_NAME` is set — always `Remove-Item Env:\DB_NAME` after Step 4 so later commands stay on dev.
- **`restoreProd` safety:** if `migrateProdSchema` is disabled in config but `restoreProd` is enabled, the orchestrator runs Step 4 first so prod has columns/tables present in the dev backup (e.g. `updated_at`, `entity_translations`).
- Deploy-related npm scripts and PowerShell helpers print a final **`===== SUCCESS: … =====`** or **`===== FAILED: … =====`** banner as the last line of output.

## Troubleshooting

| Symptom | Cause | Fix |
|---------|-------|-----|
| `/api/bounds` returns `{"min_year":null,"max_year":null}` on prod | `gallery_prod` has schema but no data | Run Step 5 (restore); confirm rows with a count query on `art_movements` / `paintings` |
| `robocopy ... ERROR 5 (0x00000005) Access is denied` on Step 6 | SMB share not mapped/authenticated in this Windows session | `net use \\192.168.10.122\Gallery /user:YOUR_TRUENAS_USER`, verify `Test-Path`, then re-run. If still denied after auth, fix TrueNAS dataset ACL (`chown -R 1001:1001`, grant the SMB user write) |
| Restore: `permission denied to set parameter "session_replication_role"` | DB role is not superuser | Handled automatically (multi-pass insert). For the fast path, superuser runs `GRANT SET ON PARAMETER session_replication_role TO gallery;` |
| Restore: `column "updated_at" of relation … does not exist` | Prod schema behind dev backup | Run Step 4 before Step 5; `full` profile enables `migrateProdSchema` by default; partial config with `restoreProd` alone also auto-migrates |
| Restore: `unterminated quoted string` | Old parser split multi-line values (bios) | Fixed — statements are accumulated until quotes balance; update to latest `scripts/restore-db-data.js` |
| `git push` → `Failed to authenticate user` (Gitea) | Git Credential Manager cached an expired token | Clear it: `"protocol=https`nhost=gitea.mysuperlab.netcraze.pro`n" \| git credential reject`, then push again to re-prompt |
| Prod tables empty after a failed restore | Restore truncates **before** inserting; a mid-run error leaves tables empty | Dev is untouched — just re-run Step 5 |
| Timeline portraits slow or missing on prod | Thumbs not regenerated or not synced | Run Step 2 on dev, then Step 3 (fresh backup), Step 5, and Step 6 |
| `502 / 504` on the public prod URL | Keenetic upstream wrong (IP/port/protocol) | `192.168.10.122:5173`, protocol **http**; verify LAN `curl.exe -s http://192.168.10.122:5173/api/bounds` first |

## Related docs

| Document | Contents |
|----------|----------|
| [environments.md](environments.md) | Full dev/prod walkthrough, one-time setup, Keenetic |
| [infra/docker/DEPLOY-truenas.md](../infra/docker/DEPLOY-truenas.md) | TrueNAS container install and troubleshooting |
| [FAC.md](FAC.md#promote-dev--prod-scheduled-release) | Command cheat sheet |
| [data-and-images.md](data-and-images.md) | Catalog and image pipeline |