Fix harmonize prod DB targeting and large-hall loading hangs.

Prod env files now win over root .env so harmonize:db can open gallery_prod after gallery_dev; tour_stops gains updated_at; halls no longer block the overlay on every painting texture. Docs cover the fixes and one-artist image pull from prod; Duccio painting assets synced from TrueNAS.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Danila Khodjaef
2026-07-26 01:26:51 +03:00
co-authored by Cursor
parent f6c73c1792
commit 8e444fa461
115 changed files with 124 additions and 37 deletions
+2 -1
View File
@@ -161,8 +161,9 @@ Curated guided tours. See [tours.md](tours.md).
| `painting_id` | FK → `paintings` | ON DELETE CASCADE; UNIQUE with `tour_id` |
| `sort_order` | INTEGER | Visitor / editor order |
| `body` | TEXT | English tour notes for the stop (v1) |
| `updated_at` | TIMESTAMPTZ | Trigger on UPDATE; required for `npm run harmonize:db` |
Applied by `npm run dev:migrate` (`db/migrate-tours.sql`).
Applied by `npm run dev:migrate` (`db/migrate-tours.sql`; `updated_at` also covered by `migrate-sync-timestamps.sql` when the table already exists).
### `painting_influences`
+10 -1
View File
@@ -243,11 +243,20 @@ net use \\192.168.10.122\Gallery /user:YOUR_TRUENAS_USER
npm run devtoprod:release # full promote from infra/deploy/devtoprod.config.json
npm run devtoprod:thumbnails # rebuild thumb files + DB paths on dev (before backup)
npm run devtoprod:images # dev repo → TrueNAS (promote / first deploy)
npm run prodto:dev:images # TrueNAS → dev repo
npm run prodto:dev:images # TrueNAS → dev repo (all images)
npm run harmonize # bidirectional merge (newer wins) — see harmonize-dev-prod.md
npm run harmonize:dry-run # preview DB + image changes only
```
**One artist only (prod → dev):** after `net use`, robocopy the artist file prefix (example: Duccio):
```powershell
$src = "\\192.168.10.122\Gallery\data\images\paintings"
$dst = "T:\Repo\Gallery\data\images\paintings"
robocopy $src $dst "Duccio*" /XO /R:2 /W:3
robocopy "$src\thumbs" "$dst\thumbs" "Duccio*" /XO /R:2 /W:3
```
Type `yes` when prompted (or set `autoConfirm: true` in release config). Robocopy exit codes **07** = success. Deploy scripts print a final **`===== SUCCESS =====`** or **`===== FAILED =====`** banner.
---
+2 -2
View File
@@ -210,7 +210,7 @@ Pan, zoom, and era/event click-to-zoom only update **local** `viewStart` / `view
| Overlay **“Loading art history…”** | Until the first catalog fetch (`bounds` + `timeline` + `artists`) completes |
| Bottom banner **“Loading portraits…”** | While artist portrait thumbnails are still downloading on the movement flow (timeline stays interactive) |
| Overlay **“Opening artist/movement gallery…”** | Between clicking a portrait/movement and the 3D hall data being ready |
| Overlay **“Loading gallery…”** | While the 3D canvas initializes, HDR Environment settles, or door/hall shaders warm up after the hall opens |
| Overlay **“Loading gallery…”** | While the 3D canvas initializes, HDR Environment settles, or door/hall shaders warm up after the hall opens (painting images continue loading after the overlay dismisses) |
| Overlay **“Loading paintings…”** | While wall painting textures are still downloading / uploading to the GPU |
| Overlay **“Restoring gallery…”** | Briefly after WebGL context loss while the canvas remounts |
@@ -374,7 +374,7 @@ Full guide: [tours.md](tours.md).
**3D images** use locally cached files only (`galleryImageUrl` in `client/src/api/client.ts`). Remote fetches are too slow for realtime WebGL textures; the client calls `POST /api/artists/:id/preload-images` automatically when entering an **artist** hall (public route — links disk files only). Movement galleries load painting lists from the API without a separate preload step. While a texture is loading, the frame shows the canvas cover instead of a white placeholder.
**Boot overlay** (`VirtualGallery.tsx`): the center shows **“Loading gallery…”** / **“Loading paintings…”** until all of the following settle — WebGL canvas ready, painting textures downloaded and GPU-uploaded (`gl.initTexture` via `GalleryTextureLoadContext`), HDR `Environment` ready (or timed out / failed), and a one-shot `gl.compileAsync` warm-up so entrance doors / passages (often frustum-culled at spawn) do not hitch on the first turn. Env and shader warm-up have short timeouts so a slow CDN cannot leave the overlay up forever. The 3D hall stays mounted while painting detail or bio overlays are open; returning remounts the canvas when the hall becomes active again.
**Boot overlay** (`VirtualGallery.tsx`): the center shows **“Loading gallery…”** until the WebGL canvas is ready, HDR `Environment` has settled (or timed out / failed), and a one-shot `gl.compileAsync` warm-up finishes so entrance doors / passages (often frustum-culled at spawn) do not hitch on the first turn. Painting textures keep loading in the background (prefer thumbnails; per-image timeouts; GPU upload after decode) so large halls (50+ works) are not stuck on the overlay. Env and shader warm-up also have short timeouts. The 3D hall stays mounted while painting detail or bio overlays are open; returning remounts the canvas when the hall becomes active again.
**WebGL context-loss recovery:** on some GPUs/drivers (notably certain Chrome setups) the browser can drop the WebGL context right after entering a hall, which would otherwise leave a permanent dark window. `VirtualGallery.tsx` listens for `webglcontextlost` / `webglcontextrestored`, calls `preventDefault()` so the browser can restore the context, and remounts the `<Canvas>` with a fresh context (a **“Restoring gallery…”** overlay shows briefly). The network-loaded HDR `Environment` map is wrapped in an error boundary so, if it fails to load, the hall still renders without reflections instead of unmounting the whole scene.
+1 -1
View File
@@ -31,7 +31,7 @@ File names are sanitised `{Artist}_{Title}.{ext}`. The image service can redisco
| **Development** | `./data/images/` in repo | Working copy on dev PC |
| **Production** | `/mnt/BasePool/Applications/Gallery/data/images` on TrueNAS | SMB `\\192.168.10.122\Gallery\data\images` |
**One-direction promote:** `npm run devtoprod:images` (dev → prod, skip older). **Refresh dev from prod:** `npm run prodto:dev:images`. **Bidirectional merge** (newer file wins): `npm run harmonize:images` or full `npm run harmonize` — see [harmonize-dev-prod.md](harmonize-dev-prod.md). General sync reference: [environments.md](environments.md).
**One-direction promote:** `npm run devtoprod:images` (dev → prod, skip older). **Refresh all images on dev from prod:** `npm run prodto:dev:images`. **One artist only:** map the SMB share, then `robocopy` that artists `ArtistName*` files under `paintings/` and `paintings/thumbs/` (see [FAC.md](FAC.md#dev--prod-image-sync-smb)). **Bidirectional merge** (newer file wins): `npm run harmonize:images` or full `npm run harmonize` — see [harmonize-dev-prod.md](harmonize-dev-prod.md). General sync reference: [environments.md](environments.md).
## Scripts overview
+17 -2
View File
@@ -45,7 +45,7 @@ For refreshing dev from prod entirely, use `npm run prodto:dev:db` (destructive
net use \\192.168.10.122\Gallery /user:YOUR_TRUENAS_USER
```
5. Copy harmonize config:
5. Copy **dedicated** harmonize config (do not rely on `devtoprod.config.json` alone — that file is a release profile and its step flags are ignored for harmonize):
```powershell
Copy-Item infra/deploy/harmonize.config.example.json infra/deploy/harmonize.config.json
@@ -53,6 +53,8 @@ For refreshing dev from prod entirely, use `npm run prodto:dev:db` (destructive
Edit `harmonize.config.json` (gitignored) — optional `smb.user` / `smb.password`, `prefer` for tie-breaks (`dev` | `prod`), `schemaChanged: true` when new migrations shipped.
6. Prod DB targeting uses [`infra/docker/.env.prod`](../infra/docker/.env.prod). `scripts/db-env.js` loads that file with **file values winning** over the root `.env`, so `harmonize:db` can open `gallery_dev` and `gallery_prod` in the same process without the safety check rejecting `gallery_dev` as a fake prod target.
---
## One-command harmonize
@@ -107,7 +109,9 @@ Reports are written to `db/SyncReports/harmonize_db_*.json` and `harmonize_image
Processed in FK order:
`historical_eras` → `art_movements` → `artists` → `artist_periods` → `paintings` → `painting_influences` → `painting_influence_sources` → `painting_annotations` → **`entity_translations`**
`historical_eras` → `art_movements` → `artists` → `artist_periods` → `paintings` → `painting_influences` → `painting_influence_sources` → `painting_annotations` → **`entity_translations`** → **`tours`** → **`tour_stops`**
Every synced table needs an `updated_at` column (including `tour_stops` — added via `migrate-tours.sql` / `migrate-sync-timestamps.sql`).
---
@@ -134,6 +138,7 @@ Harmonize reports conflicts in the JSON report and skips those rows:
| New migration in repo | `harmonize:schema` or deploy step 4 |
| Only images changed on one side | `npm run harmonize:images` |
| Only DB metadata changed | `npm run harmonize:db` |
| Pull **one artists** images from prod → dev | Map SMB, then robocopy `Duccio*` (or the artist prefix) under `paintings/` and `paintings/thumbs/` — see [FAC.md — Dev ↔ prod image sync](FAC.md#dev--prod-image-sync-smb) |
---
@@ -163,6 +168,16 @@ npm run harmonize
---
## Troubleshooting
| Symptom | Likely cause | Fix |
|---------|--------------|-----|
| `Refusing prod operation on database "gallery_dev"` | Old `db-env` left `DB_NAME` from root `.env` when loading prod | Update to current `scripts/db-env.js` (`.env.prod` wins); confirm `infra/docker/.env.prod` has `DB_NAME=gallery_prod` |
| `Table tour_stops missing updated_at` | Prod/dev schema behind | `npm run dev:migrate` and migrate prod (`$env:DB_NAME="gallery_prod"; npm run dev:migrate; Remove-Item Env:\DB_NAME`) |
| Orchestrator lists release steps (`restoreProd`, …) | Missing `harmonize.config.json`; fallback to release config | Copy `harmonize.config.example.json` → `harmonize.config.json` (current `harmonize.ps1` ignores release-only step keys) |
---
## Safety
- Pre-flight backups of dev and prod DB (configurable; on by default)