Danilka/Art-gallery
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add one-command dev-to-prod release with clear SUCCESS/FAILED banners.

Browse Source 
Introduce devtoprod:release orchestrator, config file, CLI result footers on deploy scripts, auto-thumb regeneration on curator fixes, and updated deploy documentation.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Danila Khodjaef
2026-07-08 14:54:56 +03:00
co-authored by Cursor
parent 313666a4ab
commit 21e3e41e48
20 changed files with 839 additions and 86 deletions
Download Patch File Download Diff File Expand all files Collapse all files
Documentation/deploy-dev-to-prod.md
+135 -34
View File
@@ -1,6 +1,6 @@
# Deploy dev → prod (release runbook)
Step-by-step guide for promoting the **development** version of Gallery to **production**. Covers code, database schema, database data, and image files.
Step-by-step guide for promoting the **development** version of Gallery to **production**. Covers code, database schema, database data, thumbnail generation, and image files.
> **Default is dev.** This runbook is for a **scheduled release** (~weekly, or when you explicitly decide to ship). Day-to-day work stays on dev — see [environments.md](environments.md#development-first-workflow-default). If the one-time prod install is not done yet, follow [environments.md → One-time setup](environments.md#one-time-setup-full-walkthrough) and [infra/docker/DEPLOY-truenas.md](../infra/docker/DEPLOY-truenas.md) first.
@@ -16,17 +16,84 @@ All commands run on the **dev PC** from the repo root (`C:\Users\SNAP\Nextcloud\
---
## One-command release (automated)
For a typical weekly release, use the orchestrator instead of running each step manually. It reads settings from a local config file, runs the enabled steps in order, prints a **SUCCESS/FAILED** banner at the end of each sub-command, and stops on the first failure.
### Setup (once)
1. Copy the example config:
```powershell
Copy-Item infra/deploy/devtoprod.config.example.json infra/deploy/devtoprod.config.json
```
2. Edit [`infra/deploy/devtoprod.config.json`](../infra/deploy/devtoprod.config.json) (gitignored — safe for SMB password):
- `profile`: `full` | `code` | `data` (base step set; individual `steps` overrides win)
- `autoConfirm`: `true` skips restore/image-sync prompts (`CONFIRM_PROD=1`, `-SkipConfirm`)
- `git.message`, `git.branch` — used when `gitCommitPush` is enabled
- `schemaChanged`: `true` enables prod schema migration on `full` profile
- `smb.user` / `smb.password` — optional; maps `\\host\share` before image sync
- `backupFile` — optional fixed path; otherwise uses the newest `gallery_dev_data_*.txt` after backup
3. Prerequisites still apply: Docker Desktop running, `docker login gitea.mysuperlab.netcraze.pro`, [`infra/docker/.env.prod`](../infra/docker/.env.prod) present.
### Run
```powershell
npm run devtoprod:release
```
Or double-click [`deploy-dev-to-prod.cmd`](../deploy-dev-to-prod.cmd) in the repo root.
Dry-run (print steps only, no changes):
```powershell
npm run devtoprod:release -- -DryRun
```
### What the orchestrator does
| Config step | Maps to runbook |
|-------------|-----------------|
| `validateBuild` | Step 0 — `prod:build` + dev `/api/bounds` check |
| `gitCommitPush` | Step 1 — `git add`, commit, push |
| `thumbnails` | Step 2 — `devtoprod:thumbnails` |
| `backupDev` | Step 3 — `dev:db:backup` |
| `backupProd` | Step 3 (rollback) — `prod:db:backup` |
| `migrateProdSchema` | Step 4 — `dev:migrate` on `gallery_prod` |
| `restoreProd` | Step 5 — `devtoprod:db:restore` |
| `syncImages` | Step 6 — `devtoprod:images` |
| `dockerPublish` | Step 7 — `prod:docker:publish` |
| `truenasRestartPause` | Step 8 — **manual** pause; script waits for Enter after you restart **gallery-web** |
| `verify` | Step 9 — curl LAN + public `/api/bounds` |
Final output is always a release banner:
```
================================================================
RELEASE SUCCEEDED - production promoted
Steps: validateBuild, gitCommitPush, ...
Next: open https://gallery.mysuperlab.netcraze.pro/
================================================================
```
Use the manual steps below when you need a **partial** release or want to inspect each step individually.
---
## What changed → which steps to run
Run only the steps that match what you changed. Steps are independent except that **code** needs a rebuilt image and **schema changes** must be applied before a **data** restore.
| You changed… | Required steps |
|--------------|----------------|
| Application code (client/server) | 0 → 1 → 6 → 7 → 8 |
| DB schema (new `db/migrate-*.sql`, `schema.sql`) | 0 → 1 → 2 → **3** → (4 if data too) → 7 → 8 |
| Catalog data (movements, artists, paintings, influences, bios) | 0 → 2 → 4 → 8 |
| Image files (new/replaced paintings, portraits, thumbs) | 0 → 5 → 8 |
| Everything (typical weekly release) | 0 → 1 → 2 → 3 → 4 → 5 → 6 → 7 → 8 |
| Application code (client/server) | 0 → 1 → 7 → 8 → 9 |
| DB schema (new `db/migrate-*.sql`, `schema.sql`) | 0 → 1 → 3 → **4** → (5 if data too) → 8 → 9 |
| Catalog data (movements, artists, paintings, influences, bios) | 0 → 2 → 3 → 5 → 9 |
| Image files (new/replaced paintings, portraits) | 0 → 2 → 3 → 5 → 6 → 9 |
| Thumbnails only (rebuild from existing full images) | 0 → 2 → 3 → 5 → 6 → 9 |
| Everything (typical weekly release) | 0 → 1 → 2 → 3 → 4 → 5 → 6 → 7 → 8 → 9 |
---
@@ -79,7 +146,39 @@ git push origin main
---
## Step 2 — Back up the dev database
## Step 2 — Regenerate thumbnails on dev (before backup / image sync)
**Where:** Dev PC
Rebuilds thumbnail **files** under `data/images/` and updates **`thumbnail_path`** / **`portrait_thumb_path`** in **`gallery_dev`**. Run this on dev **before** backing up the database and syncing images so prod receives both the files and the correct DB paths.
```powershell
npm run devtoprod:thumbnails
```
This runs, in order:
| Command | Output | DB column |
|---------|--------|-----------|
| `dev:regenerate-thumbnails` | `data/images/paintings/thumbs/*_thumb.jpg` | `paintings.thumbnail_path` |
| `dev:regenerate-portrait-thumbs` | `data/images/portraits/thumbs/*_thumb.jpg` | `artists.portrait_thumb_path` |
Skip when no painting or portrait images changed since the last release and thumbs are already up to date.
**Curator debug fixes on dev** already regenerate thumbnails per work — no extra step for individual edits:
| Debug action | Thumbnail regenerated |
|--------------|----------------------|
| **Fix it** / **More** / **Upload** on painting detail or Checkup | `paintings/thumbs/{Artist}_{Title}_thumb.jpg` (~400px) + `thumbnail_path` in DB |
| **Fix it** / **More** / **Upload** on artist bio | `portraits/thumbs/{Artist}_thumb.jpg` (256px) + `portrait_thumb_path` in DB |
Use `devtoprod:thumbnails` for **bulk** backfill (missing thumbs, imports, or releases where images changed outside the debug panel).
> **Order matters:** thumbnail generation updates dev DB paths. If you regenerate **after** Step 3 (backup), run a **fresh backup** (Step 3 again) before Step 5 (restore), or prod will point at thumb paths that were not in the backup.
---
## Step 3 — Back up the dev database
**Where:** Dev PC
@@ -89,7 +188,7 @@ Creates a data-only dump (INSERT statements) under `db/DataBackup/`.
npm run dev:db:backup
```
Output (note the exact path — you pass it to Step 4):
Output (note the exact path — you pass it to Step 5):
```
Backup written: db/DataBackup/gallery_dev_data_YYYYMMDD_HHMMSS.txt
@@ -100,11 +199,11 @@ Archive written: db/DataBackup/gallery_dev_data_YYYYMMDD_HHMMSS.zip
---
## Step 3 — Apply schema changes to prod (only if schema changed)
## Step 4 — Apply schema changes to prod (only if schema changed)
**Where:** Dev PC
The data restore in Step 4 only inserts rows — it does **not** create tables or add columns. If this release added a migration (`db/migrate-*.sql`) or changed `db/schema.sql`, the prod schema must be updated **first** so the new columns/tables exist.
The data restore in Step 5 only inserts rows — it does **not** create tables or add columns. If this release added a migration (`db/migrate-*.sql`) or changed `db/schema.sql`, the prod schema must be updated **first** so the new columns/tables exist.
`server/migrate.js` is idempotent (schema and migrations use `IF NOT EXISTS` / additive `ALTER`s). Point it at prod for one run by overriding `DB_NAME` (all other credentials come from `.env`; the `gallery` user has rights to both databases):
@@ -120,11 +219,11 @@ Expect `schema.sql` + each `migrate-*.sql` to log `OK`. Re-running is safe.
---
## Step 4 — Restore dev data into prod (only if data/DB changed)
## Step 5 — Restore dev data into prod (only if data/DB changed)
**Where:** Dev PC
Loads the Step 2 backup into `gallery_prod`. This **TRUNCATES all public tables** in prod first, then inserts dev's rows — prod becomes an exact copy of dev's data.
Loads the Step 3 backup into `gallery_prod`. This **TRUNCATES all public tables** in prod first, then inserts dev's rows — prod becomes an exact copy of dev's data.
```powershell
npm run devtoprod:db:restore -- --file db/DataBackup/gallery_dev_data_YYYYMMDD_HHMMSS.txt
@@ -149,11 +248,11 @@ Multi-line values (e.g. artist bios with embedded newlines) are parsed as whole
---
## Step 5 — Sync images to prod (only if image files changed)
## Step 6 — Sync images to prod (only if image files changed)
**Where:** Dev PC (SMB share mapped — see Prerequisites)
Copies `data/images/` (paintings, portraits, and `thumbs/`) from the repo to the TrueNAS volume.
Copies `data/images/` (paintings, portraits, `paintings/thumbs/`, and `portraits/thumbs/`) from the repo to the TrueNAS volume.
**Map the share in this Windows session first** (a fresh terminal has no mapping — robocopy will fail with `ERROR 5 Access is denied` otherwise):
@@ -172,11 +271,9 @@ Type `yes` when prompted. Robocopy exit codes **0–7** = success. Destination:
Alternatively, let the script map the share by exporting credentials first: `$env:SMB_USER="…"; $env:SMB_PASSWORD="…"` before running.
> Thumbnails are files, generated on dev (`npm run dev:regenerate-thumbnails` / `dev:regenerate-portrait-thumbs`) and shipped here — prod does **not** regenerate them. Regenerate on dev **before** this step if needed.
---
## Step 6 — Build and push the Docker image (only if code changed)
## Step 7 — Build and push the Docker image (only if code changed)
**Where:** Dev PC — **PowerShell as Administrator**, Docker Desktop running
@@ -193,7 +290,7 @@ npm run prod:docker:publish
---
## Step 7 — Restart gallery-web on TrueNAS (after code or schema changes)
## Step 8 — Restart gallery-web on TrueNAS (after code or schema changes)
**Where:** TrueNAS — Web UI
@@ -204,7 +301,7 @@ Data-only or image-only releases (no new image) do not require a restart, but a
---
## Step 8 — Verify production
## Step 9 — Verify production
**Where:** Dev PC / browser
@@ -229,11 +326,11 @@ Optional on TrueNAS shell: `bash infra/docker/truenas-verify.sh`.
| Problem | Action |
|---------|--------|
| Bad **data** release | Re-run Step 4 with the pre-release prod backup: `npm run devtoprod:db:restore -- --file db/DataBackup/gallery_prod_data_<pre-release>.txt` |
| Bad **data** release | Re-run Step 5 with the pre-release prod backup: `npm run devtoprod:db:restore -- --file db/DataBackup/gallery_prod_data_<pre-release>.txt` |
| Bad **code** release | Rebuild from the last good commit: `git checkout <good-commit>`, `npm run prod:docker:publish`, restart `gallery-web` |
| Bad **images** | Re-sync from a known-good dev copy, or pull prod back to dev with `npm run prodto:dev:images` to compare |
Keep at least the most recent `gallery_prod_data_*.txt` from Step 2 so a data rollback is always possible.
Keep at least the most recent `gallery_prod_data_*.txt` from Step 3 so a data rollback is always possible.
---
@@ -241,35 +338,39 @@ Keep at least the most recent `gallery_prod_data_*.txt` from Step 2 so a data ro
| Step | Command | Runs on |
|------|---------|---------|
| **All (config-driven)** | `npm run devtoprod:release` or `deploy-dev-to-prod.cmd` | Dev PC |
| 0 Validate | `npm run prod:build` | Dev PC |
| 1 Ship code | `git add . ; git commit -m "…" ; git push origin main` | Dev PC |
| 2 Backup dev | `npm run dev:db:backup` | Dev PC |
| 2 Backup prod (rollback point) | `npm run prod:db:backup` | Dev PC |
| 3 Migrate prod schema | `$env:DB_NAME="gallery_prod"; npm run dev:migrate; Remove-Item Env:\DB_NAME` | Dev PC |
| 4 Restore data → prod | `npm run devtoprod:db:restore -- --file <dev backup>.txt` | Dev PC |
| 5 Sync images → prod | `npm run devtoprod:images` | Dev PC (SMB) |
| 6 Build + push image | `npm run prod:docker:publish` | Dev PC (Admin) |
| 7 Restart app | Apps → gallery-web → Restart | TrueNAS UI |
| 8 Verify | `curl.exe -sk https://gallery.mysuperlab.netcraze.pro/api/bounds` | Dev PC |
| 2 Thumbnails | `npm run devtoprod:thumbnails` | Dev PC |
| 3 Backup dev | `npm run dev:db:backup` | Dev PC |
| 3 Backup prod (rollback point) | `npm run prod:db:backup` | Dev PC |
| 4 Migrate prod schema | `$env:DB_NAME="gallery_prod"; npm run dev:migrate; Remove-Item Env:\DB_NAME` | Dev PC |
| 5 Restore data → prod | `npm run devtoprod:db:restore -- --file <dev backup>.txt` | Dev PC |
| 6 Sync images → prod | `npm run devtoprod:images` | Dev PC (SMB) |
| 7 Build + push image | `npm run prod:docker:publish` | Dev PC (Admin) |
| 8 Restart app | Apps → gallery-web → Restart | TrueNAS UI |
| 9 Verify | `curl.exe -sk https://gallery.mysuperlab.netcraze.pro/api/bounds` | Dev PC |
---
## Safety guards (built in)
- Prod DB scripts read **only** `infra/docker/.env.prod`; dev scripts refuse `_prod` database names.
- Prod restore/backup require typing `yes` (or `CONFIRM_PROD=1`).
- `npm run dev:migrate` targets whatever `DB_NAME` is set — always `Remove-Item Env:\DB_NAME` after Step 3 so later commands stay on dev.
- Prod restore/backup require typing `yes` (or `CONFIRM_PROD=1` / `autoConfirm` in release config).
- `npm run dev:migrate` targets whatever `DB_NAME` is set — always `Remove-Item Env:\DB_NAME` after Step 4 so later commands stay on dev.
- Deploy-related npm scripts and PowerShell helpers print a final **`===== SUCCESS: … =====`** or **`===== FAILED: … =====`** banner as the last line of output.
## Troubleshooting
| Symptom | Cause | Fix |
|---------|-------|-----|
| `/api/bounds` returns `{"min_year":null,"max_year":null}` on prod | `gallery_prod` has schema but no data | Run Step 4 (restore); confirm rows with a count query on `art_movements` / `paintings` |
| `robocopy ... ERROR 5 (0x00000005) Access is denied` on Step 5 | SMB share not mapped/authenticated in this Windows session | `net use \\192.168.10.122\Gallery /user:YOUR_TRUENAS_USER`, verify `Test-Path`, then re-run. If still denied after auth, fix TrueNAS dataset ACL (`chown -R 1001:1001`, grant the SMB user write) |
| `/api/bounds` returns `{"min_year":null,"max_year":null}` on prod | `gallery_prod` has schema but no data | Run Step 5 (restore); confirm rows with a count query on `art_movements` / `paintings` |
| `robocopy ... ERROR 5 (0x00000005) Access is denied` on Step 6 | SMB share not mapped/authenticated in this Windows session | `net use \\192.168.10.122\Gallery /user:YOUR_TRUENAS_USER`, verify `Test-Path`, then re-run. If still denied after auth, fix TrueNAS dataset ACL (`chown -R 1001:1001`, grant the SMB user write) |
| Restore: `permission denied to set parameter "session_replication_role"` | DB role is not superuser | Handled automatically (multi-pass insert). For the fast path, superuser runs `GRANT SET ON PARAMETER session_replication_role TO gallery;` |
| Restore: `unterminated quoted string` | Old parser split multi-line values (bios) | Fixed — statements are accumulated until quotes balance; update to latest `scripts/restore-db-data.js` |
| `git push` → `Failed to authenticate user` (Gitea) | Git Credential Manager cached an expired token | Clear it: `"protocol=https`nhost=gitea.mysuperlab.netcraze.pro`n" \| git credential reject`, then push again to re-prompt |
| Prod tables empty after a failed restore | Restore truncates **before** inserting; a mid-run error leaves tables empty | Dev is untouched — just re-run Step 4 |
| Prod tables empty after a failed restore | Restore truncates **before** inserting; a mid-run error leaves tables empty | Dev is untouched — just re-run Step 5 |
| Timeline portraits slow or missing on prod | Thumbs not regenerated or not synced | Run Step 2 on dev, then Step 3 (fresh backup), Step 5, and Step 6 |
| `502 / 504` on the public prod URL | Keenetic upstream wrong (IP/port/protocol) | `192.168.10.122:5173`, protocol **http**; verify LAN `curl.exe -s http://192.168.10.122:5173/api/bounds` first |
## Related docs