From a2263ca1864c670f6ebea6d71ef7740a40eb55c4 Mon Sep 17 00:00:00 2001 From: Danila Khodjaef Date: Tue, 7 Jul 2026 12:36:34 +0300 Subject: [PATCH] Document gallery loading markers, WebGL recovery, and movement filtering. Update basics.md for the new loading indicators, 3D WebGL context-loss recovery, and the corrected movement-visibility rule (span overlap plus any catalogued artist). Refresh remaining docs and config for the current dev-first workflow and npm script names. Co-authored-by: Cursor --- .cursor/rules/dev-first-workflow.mdc | 8 +- .env.example | 2 +- Documentation/API.md | 4 +- Documentation/DB_structure.md | 32 ++--- Documentation/FAC.md | 126 +++++++++---------- Documentation/basics.md | 35 ++++-- Documentation/data-and-images.md | 108 ++++++++-------- Documentation/environments.md | 42 +++---- Documentation/setup.md | 176 +++++++++++++-------------- README.md | 30 ++--- 10 files changed, 288 insertions(+), 275 deletions(-) diff --git a/.cursor/rules/dev-first-workflow.mdc b/.cursor/rules/dev-first-workflow.mdc index 9386619..dd0882f 100644 --- a/.cursor/rules/dev-first-workflow.mdc +++ b/.cursor/rules/dev-first-workflow.mdc @@ -15,16 +15,16 @@ alwaysApply: true | Database | `gallery_dev` via root `.env` (`DB_NAME=gallery_dev`) | | Data / images | `data/images/` in the repo | | Public URL | https://devgallery.mysuperlab.netcraze.pro | -| Migrations / seeds / scripts | `npm run migrate`, `npm run seed`, etc. against **dev** | +| Migrations / seeds / scripts | `npm run dev:migrate`, `npm run dev:seed`, etc. against **dev** | ## Production is scheduled, not daily Prod (`gallery_prod`, TrueNAS Docker, https://gallery.mysuperlab.netcraze.pro) is updated **roughly weekly** (or when the user explicitly asks for prod): 1. Validate on dev -2. `npm run db:backup` → `npm run db:restore:prod` (if DB/data changed) -3. `npm run images:sync-to-prod` (if images changed) -4. `npm run docker:publish` (if code changed) +2. `npm run dev:db:backup` → `npm run devtoprod:db:restore` (if DB/data changed) +3. `npm run devtoprod:images` (if images changed) +4. `npm run prod:docker:publish` (if code changed) 5. Restart **gallery-web** on TrueNAS Do **not** edit `infra/docker/.env.prod`, run `db:restore:prod`, `images:sync-to-prod`, or `docker:publish` unless the user clearly targets prod. diff --git a/.env.example b/.env.example index c7604f8..776df1c 100644 --- a/.env.example +++ b/.env.example @@ -15,7 +15,7 @@ TRUST_PROXY=true IMAGE_DIR=./data/images -# Curator auth (run npm run migrate after setting CURATOR_PASSWORD) +# Curator auth (run npm run dev:migrate after setting CURATOR_PASSWORD) SESSION_SECRET=change-me-to-a-long-random-string SESSION_COOKIE_SECURE=false CURATOR_USERNAME=curator diff --git a/Documentation/API.md b/Documentation/API.md index 8c9f910..2baf037 100644 --- a/Documentation/API.md +++ b/Documentation/API.md @@ -279,9 +279,9 @@ Each painting includes: | `checkup_checked` | Reviewed in checkup / debug workflow (gold frame in 3D when true) | | `checkup_fixed` | Image replaced via **Fix it** | -Populate biographies with `npm run fetch-artist-bios` (see [data-and-images.md](data-and-images.md)). +Populate biographies with `npm run dev:fetch-artist-bios` (see [data-and-images.md](data-and-images.md)). -Artist objects also include `checkup_checked` and `checkup_fixed` (same semantics as paintings; gold portrait border when reviewed). After `npm run import-painter-palette`, **`palette_metadata`** holds PainterPalette enrichment (nationality, styles, occupations, raw influence fields, etc.). Run `npm run migrate:artist-checkup-flags` and `npm run migrate:artist-palette` on existing databases. +Artist objects also include `checkup_checked` and `checkup_fixed` (same semantics as paintings; gold portrait border when reviewed). After `npm run dev:import-painter-palette`, **`palette_metadata`** holds PainterPalette enrichment (nationality, styles, occupations, raw influence fields, etc.). Run `npm run dev:migrate:artist-checkup-flags` and `npm run dev:migrate:artist-palette` on existing databases. --- diff --git a/Documentation/DB_structure.md b/Documentation/DB_structure.md index b27a13b..344a261 100644 --- a/Documentation/DB_structure.md +++ b/Documentation/DB_structure.md @@ -1,12 +1,12 @@ # Art Gallery — database structure -PostgreSQL schema for the virtual gallery. Canonical DDL lives in **`db/schema.sql`**; **`server/migrate.js`** (`npm run migrate`) applies that file plus idempotent incremental scripts in `db/migrate-*.sql`. This document describes the logical model. +PostgreSQL schema for the virtual gallery. Canonical DDL lives in **`db/schema.sql`**; **`server/migrate.js`** (`npm run dev:migrate`) applies that file plus idempotent incremental scripts in `db/migrate-*.sql`. This document describes the logical model. Connection settings come from `.env` (dev) or `infra/docker/.env.prod` (prod scripts). See [environments.md](environments.md) and [setup.md](setup.md). ### One-time split (legacy `Gallery` → `gallery_prod` + `gallery_dev`) -Run [`db/split-dev-prod-pgadmin.sql`](../db/split-dev-prod-pgadmin.sql) in **pgAdmin** on the dev PC (postgres superuser). Alternative: `npm run db:split-databases` with `PGUSER=postgres`. +Run [`db/split-dev-prod-pgadmin.sql`](../db/split-dev-prod-pgadmin.sql) in **pgAdmin** on the dev PC (postgres superuser). Alternative: `npm run infra:db:split-dev-prod` with `PGUSER=postgres`. ## Overview @@ -73,14 +73,14 @@ Finer-grained styles (Impressionism, Cubism, Suprematism, …). | `birth_year`, `death_year` | INTEGER | Nullable; used for timeline portrait placement | | `movement_id` | FK → `art_movements` | Primary movement | | `portrait_path` | VARCHAR(500) | Relative to `data/images/`; nullable after debug **Clear** | -| `bio_short`, `bio_full` | TEXT | Wikipedia lead section (`npm run fetch-artist-bios`) | +| `bio_short`, `bio_full` | TEXT | Wikipedia lead section (`npm run dev:fetch-artist-bios`) | | `wikipedia_title` | VARCHAR(300) | Source page title | | `century` | INTEGER | Rounded century bucket for seeding limits | | `checkup_checked` | BOOLEAN NOT NULL DEFAULT false | Portrait reviewed in debug workflow (gold border on bio when true) | | `checkup_fixed` | BOOLEAN NOT NULL DEFAULT false | Portrait replaced, cleared, or uploaded via debug | -| `palette_metadata` | JSONB | Enrichment from `Inputs/PainterPalette.csv` (`npm run import-painter-palette`) | +| `palette_metadata` | JSONB | Enrichment from `Inputs/PainterPalette.csv` (`npm run dev:import-painter-palette`) | -Applied by `npm run migrate:artist-checkup-flags` (`db/migrate-artist-checkup-flags.sql`) and `npm run migrate:artist-palette` (`db/migrate-artist-palette.sql`). +Applied by `npm run dev:migrate:artist-checkup-flags` (`db/migrate-artist-checkup-flags.sql`) and `npm run dev:migrate:artist-palette` (`db/migrate-artist-palette.sql`). ### `artist_periods` @@ -114,7 +114,7 @@ Phases within an artist’s career (e.g. “Blue Period”, “Roman Period”). When `checkup_fixed` is true, `checkup_checked` is set automatically and cannot be cleared until **Fixed** is off. A cleared painting (`image_path` and `thumbnail_path` both null, `checkup_fixed` true) is shown as an empty frame in detail view and is not refetched on demand. -Applied by `npm run migrate:checkup-flags` (`db/migrate-checkup-flags.sql`). +Applied by `npm run dev:migrate:checkup-flags` (`db/migrate-checkup-flags.sql`). ### `painting_annotations` @@ -134,7 +134,7 @@ Short art-history notes shown on painting detail (`PaintingAnnotations.tsx`). | `sort_order` | INTEGER | Display order within the painting | | `confidence` | VARCHAR(20) | Default `curated`; Wikipedia pass uses `wikipedia` | -Applied by `npm run migrate:painting-annotations` (`db/migrate-painting-annotations.sql`). Load data with `npm run update-painting-annotations` (curated entries in `scripts/painting-annotations-data.js`; add `--wikipedia` for intro sentences from each work’s `wikipedia_title`). +Applied by `npm run dev:migrate:painting-annotations` (`db/migrate-painting-annotations.sql`). Load data with `npm run dev:update-painting-annotations` (curated entries in `scripts/painting-annotations-data.js`; add `--wikipedia` for intro sentences from each work’s `wikipedia_title`). ### `painting_influences` @@ -154,7 +154,7 @@ Directed edges: *this painting* was influenced by *that painting*. Unique constraint on `(painting_id, influenced_by_painting_id)`. -**Legacy mirror table.** `npm run update-influences` still inserts painting-to-painting rows here when curating data. The same edges are copied into `painting_influence_sources`. The **REST API does not read this table** for painting detail or hall navigation — use `painting_influence_sources` as the source of truth for display. +**Legacy mirror table.** `npm run dev:update-influences` still inserts painting-to-painting rows here when curating data. The same edges are copied into `painting_influence_sources`. The **REST API does not read this table** for painting detail or hall navigation — use `painting_influence_sources` as the source of truth for display. ### `painting_influence_sources` @@ -196,7 +196,7 @@ Curator accounts (named logins). Anonymous site visitors do not have rows here. | `created_at` | TIMESTAMPTZ | | | `last_login_at` | TIMESTAMPTZ | Updated on successful login | -First curator is bootstrapped on `npm run migrate` when `users` is empty and `CURATOR_USERNAME` / `CURATOR_PASSWORD` are set in env. +First curator is bootstrapped on `npm run dev:migrate` when `users` is empty and `CURATOR_USERNAME` / `CURATOR_PASSWORD` are set in env. ### `curator_audit_log` @@ -240,7 +240,7 @@ PostgreSQL session store for `express-session` (`connect-pg-simple`). Not applic ## First-time setup -The `gallery` database user needs `CREATE` on schema `public` for migrations. If tables cannot be created, run **`db/setup-admin.sql`** as PostgreSQL superuser (or the grants below) before `npm run migrate`: +The `gallery` database user needs `CREATE` on schema `public` for migrations. If tables cannot be created, run **`db/setup-admin.sql`** as PostgreSQL superuser (or the grants below) before `npm run dev:migrate`: ```sql GRANT CREATE ON SCHEMA public TO gallery; @@ -253,10 +253,10 @@ ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON SEQUENCES TO gallery; Then: ```bash -npm run migrate -npm run seed -npm run fetch-artist-bios -npm run expand-catalog +npm run dev:migrate +npm run dev:seed +npm run dev:fetch-artist-bios +npm run dev:expand-catalog ``` ## Data conventions @@ -264,5 +264,5 @@ npm run expand-catalog - **Year zero** is not used; BCE years are negative integers. - **Image paths** are relative to `IMAGE_DIR` (default `./data/images`), e.g. `portraits/Claude_Monet.jpg`, `paintings/thumbs/Raphael_The_School_of_Athens_thumb.jpg`. - **Seeding cap**: curated ingest targets at most ~100 artists per century to keep the catalog manageable. -- **Catalog expansion**: `scripts/famous-paintings-data.js` plus `npm run expand-catalog` raises thin artist catalogs to at least six notable paintings (`MIN_PAINTINGS`, default 6). -- **Biographies**: `bio_short` and `bio_full` are populated by `npm run fetch-artist-bios` from English Wikipedia lead sections; `wikipedia_title` on the artist row is the source article. +- **Catalog expansion**: `scripts/famous-paintings-data.js` plus `npm run dev:expand-catalog` raises thin artist catalogs to at least six notable paintings (`MIN_PAINTINGS`, default 6). +- **Biographies**: `bio_short` and `bio_full` are populated by `npm run dev:fetch-artist-bios` from English Wikipedia lead sections; `wikipedia_title` on the artist row is the source article. diff --git a/Documentation/FAC.md b/Documentation/FAC.md index 1b25b2c..eff097a 100644 --- a/Documentation/FAC.md +++ b/Documentation/FAC.md @@ -2,7 +2,9 @@ Quick cheat sheet for daily operations. All `npm` commands run from the **repository root** unless noted. -**Default target: dev.** Day-to-day commands use `gallery_dev`, repo `data/images/`, and https://devgallery.mysuperlab.netcraze.pro. Prod (`gallery_prod`, Docker, `docker:publish`) is for **scheduled releases** (~weekly) — see [Promote dev → prod](#promote-dev--prod-scheduled-release) below. +**Default target: dev.** Day-to-day commands use `gallery_dev`, repo `data/images/`, and https://devgallery.mysuperlab.netcraze.pro. Prod (`gallery_prod`, Docker, `prod:docker:publish`) is for **scheduled releases** (~weekly) — see [Promote dev → prod](#promote-dev--prod-scheduled-release) below. + +**Script prefixes:** `dev:` → `gallery_dev` / dev PC · `prod:` → production / TrueNAS · `devtoprod:` → promote dev → prod · `prodto:dev:` → refresh dev from prod · `infra:` → one-time setup **Environments:** @@ -39,10 +41,10 @@ npm run dev:client # Vite on :5173 ### Start — production-style (single Node process, built SPA) ```powershell -npm run start:prod # build client + serve on PORT from .env +npm run prod:start # build client + serve on PORT from .env # or: -npm run build -npm run start +npm run prod:build +npm run dev:start ``` ### Stop dev servers @@ -68,7 +70,7 @@ Stop-Process -Id -Force |--------|-------| | **Stop** | TrueNAS Web UI → Apps → **gallery-web** → Stop | | **Restart** | Same → Restart (after `docker:publish` or config change) | -| **Update image** | Dev PC: `npm run docker:publish` → restart app on TrueNAS | +| **Update image** | Dev PC: `npm run prod:docker:publish` → restart app on TrueNAS | --- @@ -80,11 +82,11 @@ copy .env.example .env # edit DB credentials, PUBLIC_URL npm install cd client; npm install; cd .. -npm run migrate # schema + incremental SQL (+ auth tables, bootstrap curator) -npm run setup # migrate + seed (fresh empty DB only) +npm run dev:migrate # schema + incremental SQL (+ auth tables, bootstrap curator) +npm run dev:setup # migrate + seed (fresh empty DB only) ``` -**Curator auth (after migrate):** set in `.env` before first `npm run migrate` if the DB has no curator yet: +**Curator auth (after migrate):** set in `.env` before first `npm run dev:migrate` if the DB has no curator yet: ```env SESSION_SECRET=your-long-random-secret @@ -120,7 +122,7 @@ CURATOR_USERNAME=curator CURATOR_PASSWORD=your-secure-password ``` -Run `npm run migrate` against prod DB after first deploy with auth vars set (creates tables + bootstrap curator if `users` is empty). +Run `npm run dev:migrate` against prod DB after first deploy with auth vars set (creates tables + bootstrap curator if `users` is empty). --- @@ -128,14 +130,14 @@ Run `npm run migrate` against prod DB after first deploy with auth vars set (cre | Command | Description | |---------|-------------| -| `npm run migrate` | Apply `db/schema.sql` + incremental migrations (safe to re-run) | -| `npm run setup` | `migrate` + `seed` — fresh catalog from Wikipedia data | -| `npm run db:split-databases` | One-time: legacy `Gallery` → `gallery_prod` + `gallery_dev` (needs `PGUSER=postgres`) | -| `npm run db:sync-from-prod` | Clone `gallery_prod` → `gallery_dev` (TEMPLATE); also runs image sync from prod | -| `npm run db:backup` | Dev data-only backup → `db/DataBackup/*.txt` + `.zip` | -| `npm run db:backup:prod` | Prod backup (reads `infra/docker/.env.prod`) | -| `npm run db:restore -- --file ` | Restore backup into **dev** (truncates tables first; prompts `yes`) | -| `npm run db:restore:prod -- --file ` | Restore into **prod** (requires confirmation) | +| `npm run dev:migrate` | Apply `db/schema.sql` + incremental migrations (safe to re-run) | +| `npm run dev:setup` | `migrate` + `seed` — fresh catalog from Wikipedia data | +| `npm run infra:db:split-dev-prod` | One-time: legacy `Gallery` → `gallery_prod` + `gallery_dev` (needs `PGUSER=postgres`) | +| `npm run prodto:dev:db` | Clone `gallery_prod` → `gallery_dev` (TEMPLATE); also runs image sync from prod | +| `npm run dev:db:backup` | Dev data-only backup → `db/DataBackup/*.txt` + `.zip` | +| `npm run prod:db:backup` | Prod backup (reads `infra/docker/.env.prod`) | +| `npm run dev:db:restore -- --file ` | Restore backup into **dev** (truncates tables first; prompts `yes`) | +| `npm run devtoprod:db:restore -- --file ` | Restore into **prod** (requires confirmation) | **One-time split (recommended):** pgAdmin on dev PC → open [`db/split-dev-prod-pgadmin.sql`](../db/split-dev-prod-pgadmin.sql) → run each STEP on database `postgres`, then verify on `gallery_dev`. @@ -159,52 +161,52 @@ Run in this order on a **new** or **re-seeded** database: | # | Command | What it does | |---|---------|----------------| -| 1 | `npm run seed` | Eras, movements, artists, one flagship painting per artist | -| 2 | `npm run sync-image-paths` | Import paintings from `data/images/paintings/`; link paths in DB | -| 3 | `npm run fetch-artist-images` | Portraits → `data/images/portraits/`, set `portrait_path` | -| 4 | `npm run fetch-artist-bios` | Wikipedia bios → `bio_short` / `bio_full` | -| 5 | `npm run expand-catalog` | Add famous works per artist (below `MIN_PAINTINGS`) | -| 6 | `npm run update-influences` | Influence graph (detail panels, 3D hall lamps) | -| 7 | `npm run fetch-images -- --limit=50` | Download missing painting files (batch) | +| 1 | `npm run dev:seed` | Eras, movements, artists, one flagship painting per artist | +| 2 | `npm run dev:sync-image-paths` | Import paintings from `data/images/paintings/`; link paths in DB | +| 3 | `npm run dev:fetch-artist-images` | Portraits → `data/images/portraits/`, set `portrait_path` | +| 4 | `npm run dev:fetch-artist-bios` | Wikipedia bios → `bio_short` / `bio_full` | +| 5 | `npm run dev:expand-catalog` | Add famous works per artist (below `MIN_PAINTINGS`) | +| 6 | `npm run dev:update-influences` | Influence graph (detail panels, 3D hall lamps) | +| 7 | `npm run dev:fetch-images -- --limit=50` | Download missing painting files (batch) | -**One-shot bootstrap:** `npm run setup` = steps 1 + migrate only; still run 2–7 for a full catalog. +**One-shot bootstrap:** `npm run dev:setup` = steps 1 + migrate only; still run 2–7 for a full catalog. ### Useful flags ```powershell -npm run sync-image-paths -- --dry-run -npm run fetch-artist-images -- --force -npm run fetch-artist-bios -- --force -npm run expand-catalog -- --fetch-images -npm run fetch-images -- --artist="Claude Monet" -npm run fetch-images -- --limit=50 --max-wait=120 -npm run update-influences -- --discover -npm run discover-influences # discovery only, no curated insert +npm run dev:sync-image-paths -- --dry-run +npm run dev:fetch-artist-images -- --force +npm run dev:fetch-artist-bios -- --force +npm run dev:expand-catalog -- --fetch-images +npm run dev:fetch-images -- --artist="Claude Monet" +npm run dev:fetch-images -- --limit=50 --max-wait=120 +npm run dev:update-influences -- --discover +npm run dev:discover-influences # discovery only, no curated insert ``` ### Optional migrations / imports | Command | Description | |---------|-------------| -| `npm run migrate:thumbnails` | Add thumbnail columns | -| `npm run migrate:influence-sources` | `painting_influence_sources` table + backfill | -| `npm run migrate:checkup-flags` | Review flags on `paintings` | -| `npm run migrate:artist-checkup-flags` | Review flags on `artists` | -| `npm run migrate:painting-annotations` | Art-history notes table | -| `npm run migrate:artist-palette` | `palette_metadata` JSONB on artists | -| `npm run import-painter-palette` | Enrich from `Inputs/PainterPalette.csv` | -| `npm run update-painting-annotations` | Load curated notes | -| `npm run update-painting-annotations -- --wikipedia` | Add Wikipedia intro sentences | +| `npm run dev:migrate:thumbnails` | Add thumbnail columns | +| `npm run dev:migrate:influence-sources` | `painting_influence_sources` table + backfill | +| `npm run dev:migrate:checkup-flags` | Review flags on `paintings` | +| `npm run dev:migrate:artist-checkup-flags` | Review flags on `artists` | +| `npm run dev:migrate:painting-annotations` | Art-history notes table | +| `npm run dev:migrate:artist-palette` | `palette_metadata` JSONB on artists | +| `npm run dev:import-painter-palette` | Enrich from `Inputs/PainterPalette.csv` | +| `npm run dev:update-painting-annotations` | Load curated notes | +| `npm run dev:update-painting-annotations -- --wikipedia` | Add Wikipedia intro sentences | ### Audit / export | Command | Description | |---------|-------------| -| `npm run audit-painting-images` | Thumb vs full aspect-ratio mismatches | -| `npm run find-duplicates` | Duplicate / near-duplicate painting rows | -| `npm run audit-influence-duplicates` | Duplicate influence edges | -| `npm run analyze-painter-palette` | CSV ↔ artist name match report | -| `npm run export-paintings` | Write `Output/paintings.csv` | +| `npm run dev:audit-painting-images` | Thumb vs full aspect-ratio mismatches | +| `npm run dev:find-duplicates` | Duplicate / near-duplicate painting rows | +| `npm run dev:audit-influence-duplicates` | Duplicate influence edges | +| `npm run dev:analyze-painter-palette` | CSV ↔ artist name match report | +| `npm run dev:export-paintings` | Write `Output/paintings.csv` | --- @@ -212,10 +214,10 @@ npm run discover-influences # discovery only, no curated insert | Command | Description | |---------|-------------| -| `npm run fetch-images` | Search/download missing painting files (alias: `search-missing-paintings`) | -| `npm run fetch-artist-images` | Download or link artist portraits | -| `npm run sync-image-paths` | Align DB paths with files on disk; import new rows | -| `npm run regenerate-thumbnails` | Rebuild all thumbs from full images | +| `npm run dev:fetch-images` | Search/download missing painting files (alias: `search-missing-paintings`) | +| `npm run dev:fetch-artist-images` | Download or link artist portraits | +| `npm run dev:sync-image-paths` | Align DB paths with files on disk; import new rows | +| `npm run dev:regenerate-thumbnails` | Rebuild all thumbs from full images | **Local paths:** `data/images/portraits/`, `data/images/paintings/`, `data/images/paintings/thumbs/` @@ -228,8 +230,8 @@ SMB share **`Gallery`** → `/mnt/BasePool/Applications/Gallery` on TrueNAS. ```powershell net use \\192.168.10.122\Gallery /user:YOUR_TRUENAS_USER -npm run images:sync-to-prod # dev repo → TrueNAS (promote / first deploy) -npm run images:sync-from-prod # TrueNAS → dev repo +npm run devtoprod:images # dev repo → TrueNAS (promote / first deploy) +npm run prodto:dev:images # TrueNAS → dev repo ``` Type `yes` when prompted. Robocopy exit codes **0–7** = success. @@ -240,13 +242,13 @@ Type `yes` when prompted. Robocopy exit codes **0–7** = success. | Command | Where | Description | |---------|-------|-------------| -| `npm run docker:publish` | Dev PC, **Admin** PowerShell, Docker Desktop running | Build + push `gallery-web:latest` to Gitea | -| `npm run docker:publish:push-only` | Same | Push only (skip rebuild) | +| `npm run prod:docker:publish` | Dev PC, **Admin** PowerShell, Docker Desktop running | Build + push `gallery-web:latest` to Gitea | +| `npm run prod:docker:push-only` | Same | Push only (skip rebuild) | | `.\infra\docker\save-for-truenas.ps1` | Dev PC | Save image to `gallery-web.tar` (offline deploy) | -**Before first deploy:** `npm run docker:publish` → TrueNAS Custom App from `infra/docker/compose.truenas.yaml`. +**Before first deploy:** `npm run prod:docker:publish` → TrueNAS Custom App from `infra/docker/compose.truenas.yaml`. -**After code changes:** `npm run docker:publish` → restart **gallery-web** on TrueNAS. +**After code changes:** `npm run prod:docker:publish` → restart **gallery-web** on TrueNAS. ```powershell docker login gitea.mysuperlab.netcraze.pro @@ -257,7 +259,7 @@ docker login gitea.mysuperlab.netcraze.pro ## Build frontend ```powershell -npm run build # client → client/dist/ +npm run prod:build # client → client/dist/ cd client && npm run build && cd .. ``` @@ -290,10 +292,10 @@ Expect JSON with `min_year` / `max_year`. HTML shell only from `curl` on `/` is ~Weekly (or when explicitly releasing to prod). Not part of daily dev. 1. Test on https://devgallery.mysuperlab.netcraze.pro -2. `npm run db:backup` -3. `npm run db:restore:prod -- --file db/DataBackup/gallery_dev_data_....txt` (if DB/catalog changed) -4. `npm run images:sync-to-prod` (if images changed) -5. `npm run docker:publish` (if code changed) +2. `npm run dev:db:backup` +3. `npm run devtoprod:db:restore -- --file db/DataBackup/gallery_dev_data_....txt` (if DB/catalog changed) +4. `npm run devtoprod:images` (if images changed) +5. `npm run prod:docker:publish` (if code changed) 6. Restart **gallery-web** on TrueNAS 7. Verify https://gallery.mysuperlab.netcraze.pro diff --git a/Documentation/basics.md b/Documentation/basics.md index 063766c..3d06213 100644 --- a/Documentation/basics.md +++ b/Documentation/basics.md @@ -31,7 +31,8 @@ Gallery/ ├── server/ # Express API, DB pool, image service ├── client/ # React/Vite frontend │ ├── src/ # Source (components, pages, 3D scene) -│ │ ├── components/VirtualGallery.tsx # 3D hall (artist + movement modes) +│ │ ├── components/VirtualGallery.tsx # 3D hall (artist + movement modes; WebGL context-loss recovery) +│ │ ├── components/GalleryLoadingMarker.tsx # Loading spinner overlay/banner (catalog, portraits, halls) │ │ ├── components/GalleryWindows.tsx # Side-wall daylight windows (movement) │ │ ├── components/HallPassage.tsx # Open archway between movement wings │ │ ├── components/MovementHallDetails.tsx # Period architectural details @@ -70,7 +71,7 @@ Gallery/ ├── Output/ # Generated exports (e.g. paintings.csv) ├── data/images/ # Local portraits and paintings (+ thumbs/) ├── db/ # schema.sql, setup-admin.sql, migrate-*.sql -├── server/migrate.js # npm run migrate — schema + incremental migrations +├── server/migrate.js # npm run dev:migrate — schema + incremental migrations ├── deploy/ # Legacy nginx + systemd examples (optional) ├── infra/docker/ # Production Dockerfile, TrueNAS compose, deploy scripts ├── Documentation/ # This folder @@ -92,12 +93,12 @@ Uses database **`gallery_dev`** on the same PostgreSQL host. ### Production (TrueNAS Docker) — scheduled releases -Production runs in **`gallery-web`** on TrueNAS port **5173**, database **`gallery_prod`**, public URL **https://gallery.mysuperlab.netcraze.pro**. Images: `/mnt/BasePool/Applications/Gallery/data/images` (SMB share **`Gallery`**). Deploy via `npm run docker:publish` and the promote checklist in [environments.md](environments.md). +Production runs in **`gallery-web`** on TrueNAS port **5173**, database **`gallery_prod`**, public URL **https://gallery.mysuperlab.netcraze.pro**. Images: `/mnt/BasePool/Applications/Gallery/data/images` (SMB share **`Gallery`**). Deploy via `npm run prod:docker:publish` and the promote checklist in [environments.md](environments.md). ### Production-style single process (local) ```bash -npm run start:prod # build client + serve on PORT from .env +npm run prod:start # build client + serve on PORT from .env ``` Serves `/api/*`, `/images/*`, and the built SPA from `client/dist`. @@ -159,7 +160,15 @@ On first visit, `HomePage.tsx` fetches the full catalog once: 2. `GET /api/timeline?start=…&end=…` — all eras and movements for that range. 3. `GET /api/artists?timeline=1` — lightweight artist rows (portraits, lifespan, movement colour; no full biography text). -Pan, zoom, and era/event click-to-zoom only update **local** `viewStart` / `viewEnd` state. `MovementBands.tsx` and `Timeline.tsx` filter what is visible for the current window — they do not trigger new API calls. The “Loading art history…” message appears only until the first successful load completes. +Pan, zoom, and era/event click-to-zoom only update **local** `viewStart` / `viewEnd` state. `MovementBands.tsx` and `Timeline.tsx` filter what is visible for the current window — they do not trigger new API calls. + +**Loading indicators** (`GalleryLoadingMarker.tsx`) keep the user informed while data is still arriving: + +| Marker | When | +|--------|------| +| Overlay **“Loading art history…”** | Until the first catalog fetch (`bounds` + `timeline` + `artists`) completes | +| Bottom banner **“Loading portraits…”** | While artist portrait thumbnails are still downloading on the movement flow (timeline stays interactive) | +| Overlay **“Opening artist/movement gallery…”** | Between clicking a portrait/movement and the 3D hall data being ready | View updates are **batched to one commit per animation frame** via `createViewChangeScheduler()` in `timelineView.ts` (`HomePage.tsx` → `handleViewChange`), so rapid scroll-wheel events do not flood React with separate renders. @@ -196,7 +205,7 @@ Each visible movement is drawn as a **portrait-width curved stream** (~54 px str | Vertical depth | Successor movements sit on rows below their deepest parent; sibling movements at the same depth are spread into lanes to limit overlap | | Branch connectors | Smooth curves from the **centre** of a parent stream to the **centre** of each child stream (siblings fan out along the parent’s length) | | Visual blending | Path-aligned SVG gradients with transparent fades at stream ends and branch junctions; streams draw on top of branches so overlap brightness stays uniform | -| Filtering | Same rule as the API: only movements with at least one artist active in the visible year range (filtered client-side after initial load) | +| Filtering | A movement is drawn when its **span overlaps** the visible year range **and** it has at least one catalogued artist — artists whose lifespan falls outside the window still keep their movement visible (their portraits simply do not render). Filtered client-side after initial load | | Viewport layout | Row height and stream width scale from measured canvas size so every visible movement row fits in the remaining screen space | ### Artists on movement streams @@ -237,7 +246,7 @@ Each artist has **exactly one hall**. The hall is a rectangular room sized to fi | Rule | Implementation | |------|----------------| | One hall per artist | `VirtualGallery.tsx` builds a single room from that artist’s paintings | -| Catalog depth | Most artists target **≥ 6** notable works via `npm run expand-catalog` and `famous-paintings-data.js`; some masters have larger museum dumps | +| Catalog depth | Most artists target **≥ 6** notable works via `npm run dev:expand-catalog` and `famous-paintings-data.js`; some masters have larger museum dumps | | Paintings on walls | Works hang on the **back, left, and right** walls in **one row per wall**; room **depth grows** when the catalog is large | | Corridor layout | **15+ paintings:** short back wall (up to 8 works), remaining works on extended **left/right** side walls — a long gallery corridor | | Wall order | On each wall, left → right: **later works on the left**, **earlier works on the right**; undated works sort toward the left | @@ -265,7 +274,7 @@ Each artist has **exactly one hall**. The hall is a rectangular room sized to fi | Click painting | Open detail view | | Exit doorway / `E` / **Exit →** header button | Open path picker | -Predecessors and successors come from **`painting_influence_sources`** (painting and artist sources). Empty lists mean no influence edges are recorded yet for that artist — run `npm run update-influences` or extend `art-influences-data.js`. +Predecessors and successors come from **`painting_influence_sources`** (painting and artist sources). Empty lists mean no influence edges are recorded yet for that artist — run `npm run dev:update-influences` or extend `art-influences-data.js`. ### Movement galleries @@ -302,7 +311,9 @@ Movement galleries do **not** use the predecessor/successor influence picker — ### Shared 3D behaviour -**3D images** use locally cached files only (`galleryImageUrl` in `client/src/api/client.ts`). Remote fetches are too slow for realtime WebGL textures; the client calls `POST /api/artists/:id/preload-images` automatically when entering an **artist** hall (public route — links disk files only). Movement galleries load painting lists from the API without a separate preload step. While a texture is loading, the frame shows the canvas cover instead of a white placeholder. The 3D hall stays mounted while painting detail or bio overlays are open; returning remounts the canvas when the hall becomes active again. +**3D images** use locally cached files only (`galleryImageUrl` in `client/src/api/client.ts`). Remote fetches are too slow for realtime WebGL textures; the client calls `POST /api/artists/:id/preload-images` automatically when entering an **artist** hall (public route — links disk files only). Movement galleries load painting lists from the API without a separate preload step. While a texture is loading, the frame shows the canvas cover instead of a white placeholder, and a **“Loading paintings…”** overlay is shown until every wall texture has resolved (tracked through `GalleryTextureLoadContext`). The 3D hall stays mounted while painting detail or bio overlays are open; returning remounts the canvas when the hall becomes active again. + +**WebGL context-loss recovery:** on some GPUs/drivers (notably certain Chrome setups) the browser can drop the WebGL context right after entering a hall, which would otherwise leave a permanent dark window. `VirtualGallery.tsx` listens for `webglcontextlost` / `webglcontextrestored`, calls `preventDefault()` so the browser can restore the context, and remounts the `` with a fresh context (a **“Restoring gallery…”** overlay shows briefly). The network-loaded HDR `Environment` map is wrapped in an error boundary so, if it fails to load, the hall still renders without reflections instead of unmounting the whole scene. ## Painting detail view @@ -346,13 +357,15 @@ Next to the toggle, **Show more** (checkbox, persisted in `localStorage`) opens - **Timeline bounds** derive from the earliest art movement start year, not ancient-era metadata alone, so the default view opens where catalogued content begins. - **Timeline catalog** loads once from the API; pan/zoom is client-side only, with per-frame batching via `createViewChangeScheduler()`. - **Movement flow interaction** uses simplified SVG and hides portrait DOM during active scroll/drag so zoom stays responsive over dense portrait fields. -- **Movement filtering** on zoom only shows movements that have at least one artist active in the visible year range. +- **Movement filtering** on zoom shows movements whose span overlaps the visible year range and that have at least one catalogued artist; a movement (e.g. Byzantine or Gothic viewed at 311–1231 CE) still appears even when all its artists lived outside the current window. - **Movement lineage** (`movement-lineage.ts`) documents art-historical predecessor→successor links for the flow diagram; extend that file to add or correct branches. - **One hall per artist** keeps navigation predictable: enter from the timeline or bio, leave via the single exit or back button. - **Movement galleries** complement artist halls: full movement corpus in period-themed wings, entered from the flow diagram. - **Influence-based hall links** connect artists through documented painting relationships, grouped by movement at the exit. - **3D gallery images** use locally cached files only; slow remote fetches would break realtime rendering. The client calls `POST /api/artists/:id/preload-images` automatically when entering an **artist** hall (public route — links disk files only). - **3D gallery session** stays mounted while painting detail or bio overlays are open; returning to the hall remounts the WebGL canvas when it becomes active again. +- **3D gallery resilience:** a lost WebGL context is recovered by remounting the canvas with a fresh context (rather than showing a dark window), and the HDR environment map is isolated behind an error boundary so its failure never blanks the scene. +- **Loading feedback:** `GalleryLoadingMarker` surfaces catalog load, portrait download, gallery entry, painting-texture load, and context-restore states so the user always knows work is still in progress. - **Influence data** is stored in **`painting_influence_sources`** (directed links from paintings to source paintings, artists, or movements), with optional period fields and citation metadata. Sources include curated scholarship (`art-influences-data.js`) and **PainterPalette** (`discovered_via = painter-palette`). Legacy `painting_influences` mirrors painting-to-painting edges for scripts only. ## User roles and access @@ -404,7 +417,7 @@ Influence side-panel thumbnails use **letterboxing** (`object-fit: contain`) so **Search visible** runs image search only for rows currently shown after text/filter — not automatically on page load. Fixing an image sets **Fixed** and **Reviewed**. -Run `npm run migrate:checkup-flags`, `npm run migrate:artist-checkup-flags`, and `npm run migrate:painting-annotations` once on existing databases. Load notes with `npm run update-painting-annotations` (add `--wikipedia` for overview lines from Wikipedia intro text). After server code changes, restart `npm run start` (or `npm run dev:server`) so new routes are registered. JSON body limit for uploads is **20 MB** (`express.json` in `server/index.js`); individual files are capped at **15 MB** after decode. +Run `npm run dev:migrate:checkup-flags`, `npm run dev:migrate:artist-checkup-flags`, and `npm run dev:migrate:painting-annotations` once on existing databases. Load notes with `npm run dev:update-painting-annotations` (add `--wikipedia` for overview lines from Wikipedia intro text). After server code changes, restart `npm run dev:start` (or `npm run dev:server`) so new routes are registered. JSON body limit for uploads is **20 MB** (`express.json` in `server/index.js`); individual files are capped at **15 MB** after decode. Server-side auth lives in `server/middleware/session.js`, `server/middleware/auth.js`, `server/routes/auth.js`, and `server/audit-log.js`. Client auth context: `client/src/context/AuthContext.tsx`. diff --git a/Documentation/data-and-images.md b/Documentation/data-and-images.md index bdc9da0..dca2079 100644 --- a/Documentation/data-and-images.md +++ b/Documentation/data-and-images.md @@ -31,29 +31,29 @@ File names are sanitised `{Artist}_{Title}.{ext}`. The image service can redisco | **Development** | `./data/images/` in repo | Working copy on dev PC | | **Production** | `/mnt/BasePool/Applications/Gallery/data/images` on TrueNAS | SMB `\\192.168.10.122\Gallery\data\images` | -Promote dev → prod files: `npm run images:sync-to-prod` (after `net use \\192.168.10.122\Gallery`). Refresh dev from prod: `npm run images:sync-from-prod`. See [environments.md](environments.md). +Promote dev → prod files: `npm run devtoprod:images` (after `net use \\192.168.10.122\Gallery`). Refresh dev from prod: `npm run prodto:dev:images`. See [environments.md](environments.md). ## Scripts overview | Script | npm command | Role | |--------|-------------|------| -| `seed-wikipedia.js` | `npm run seed` | Initial eras, movements, artists, one flagship painting per artist | +| `seed-wikipedia.js` | `npm run dev:seed` | Initial eras, movements, artists, one flagship painting per artist | | `seed-catalog-data.js` | *(data only)* | Eras, movements, artist metadata consumed by seed | -| `sync-image-paths.js` | `npm run sync-image-paths` | Import painting rows from disk; set `image_path` / `thumbnail_path` | -| `fetch-artist-images.js` | `npm run fetch-artist-images` | Download or link artist portraits under `data/images/portraits/` | -| `fetch-artist-bios.js` | `npm run fetch-artist-bios` | Wikipedia intros → `bio_short` / `bio_full` | +| `sync-image-paths.js` | `npm run dev:sync-image-paths` | Import painting rows from disk; set `image_path` / `thumbnail_path` | +| `fetch-artist-images.js` | `npm run dev:fetch-artist-images` | Download or link artist portraits under `data/images/portraits/` | +| `fetch-artist-bios.js` | `npm run dev:fetch-artist-bios` | Wikipedia intros → `bio_short` / `bio_full` | | `famous-paintings-data.js` | *(data only)* | Curated list of notable works per artist | -| `expand-paintings.js` | `npm run expand-catalog` | Inserts works from data file for thin catalogs | +| `expand-paintings.js` | `npm run dev:expand-catalog` | Inserts works from data file for thin catalogs | | `art-influences-data.js` | *(data only)* | Curated influence edges (painting / artist / movement) | -| `update-influences.js` | `npm run update-influences` | Applies influence graph; creates missing artists/works | -| `fetch-missing-images.js` | `npm run fetch-images` | Downloads files for paintings missing on disk | +| `update-influences.js` | `npm run dev:update-influences` | Applies influence graph; creates missing artists/works | +| `fetch-missing-images.js` | `npm run dev:fetch-images` | Downloads files for paintings missing on disk | | `image-fetcher.js` | *(library)* | Wikimedia / museum resolution used by fetch scripts and API | -| `sync-images-to-prod.ps1` / `sync-images-from-prod.ps1` | `npm run images:sync-*` | Robocopy via SMB `\\192.168.10.122\Gallery` | -| `regenerate-thumbnails.js` | `npm run regenerate-thumbnails` | Rebuild painting thumbs from full images via `sharp` | -| `regenerate-portrait-thumbs.js` | `npm run regenerate-portrait-thumbs` | Rebuild timeline portrait thumbs (~256px) and set `portrait_thumb_path` | -| `audit-painting-images.js` | `npm run audit-painting-images` | Detect thumb/full aspect-ratio mismatches | -| `find-duplicate-paintings.js` | `npm run find-duplicates` | Report exact and near-duplicate catalog rows | -| `migrate-checkup-flags.js` | `npm run migrate:checkup-flags` | Add `checkup_checked` / `checkup_fixed` columns | +| `sync-images-to-prod.ps1` / `sync-images-from-prod.ps1` | `npm run devtoprod:images` / `npm run prodto:dev:images` | Robocopy via SMB `\\192.168.10.122\Gallery` | +| `regenerate-thumbnails.js` | `npm run dev:regenerate-thumbnails` | Rebuild painting thumbs from full images via `sharp` | +| `regenerate-portrait-thumbs.js` | `npm run dev:regenerate-portrait-thumbs` | Rebuild timeline portrait thumbs (~256px) and set `portrait_thumb_path` | +| `audit-painting-images.js` | `npm run dev:audit-painting-images` | Detect thumb/full aspect-ratio mismatches | +| `find-duplicate-paintings.js` | `npm run dev:find-duplicates` | Report exact and near-duplicate catalog rows | +| `migrate-checkup-flags.js` | `npm run dev:migrate:checkup-flags` | Add `checkup_checked` / `checkup_fixed` columns | ## Typical workflow @@ -71,13 +71,13 @@ migrate → seed → sync-image-paths → fetch-artist-images → fetch-artist-b ## Seeding pipeline -`npm run seed` runs `scripts/seed-wikipedia.js`, which: +`npm run dev:seed` runs `scripts/seed-wikipedia.js`, which: 1. Inserts **historical eras** and **art movements** (curated date ranges and colours from `seed-catalog-data.js`). 2. For each curated **artist**: - Creates **artist periods** and one **flagship painting**. - May download portraits and painting images when run with `--fetch-images`. -3. Does **not** insert influence edges — run `npm run update-influences` after seed (see [Painting influence graph](#painting-influence-graph)). +3. Does **not** insert influence edges — run `npm run dev:update-influences` after seed (see [Painting influence graph](#painting-influence-graph)). Those influence edges power **3D hall navigation** and painting detail panels via **`painting_influence_sources`** (see `GET /api/artists/:id/navigation` and `GET /api/paintings/:id` in [API.md](API.md)). @@ -85,7 +85,7 @@ Artists are grouped by movement and century; the seed list targets at most ~100 ## Artist biographies -`npm run fetch-artist-bios` reads each artist’s `wikipedia_title`, fetches the English Wikipedia **lead section**, and stores: +`npm run dev:fetch-artist-bios` reads each artist’s `wikipedia_title`, fetches the English Wikipedia **lead section**, and stores: | Field | Content | |-------|---------| @@ -110,15 +110,15 @@ The bio page (`ArtistBio.tsx`) shows lifespan, movement, summary, full text, and ## Expanding thin catalogs -Many seed artists arrive with only one famous painting. `npm run expand-catalog` runs `scripts/expand-paintings.js`, which: +Many seed artists arrive with only one famous painting. `npm run dev:expand-catalog` runs `scripts/expand-paintings.js`, which: 1. Finds artists with fewer than `MIN_PAINTINGS` (default **6**). 2. Inserts rows from `scripts/famous-paintings-data.js` that are not already present (normalized title matching skips duplicates). 3. Sets `wikipedia_title` on each new painting for image resolution. ```bash -npm run expand-catalog # DB rows only -npm run expand-catalog -- --fetch-images # also download images (very slow) +npm run dev:expand-catalog # DB rows only +npm run dev:expand-catalog -- --fetch-images # also download images (very slow) ``` To add more works, append entries to `famous-paintings-data.js`: @@ -137,7 +137,7 @@ Renaissance and medieval masters with large museum catalog dumps (e.g. Raphael, When the repository includes a full `data/images/paintings/` tree but the database was seeded fresh (one row per artist), run: ```bash -npm run sync-image-paths +npm run dev:sync-image-paths ``` `scripts/sync-image-paths.js`: @@ -157,9 +157,9 @@ Typical result on a full clone: ~1,000+ paintings linked from ~1,000 on-disk fil ## Artist portraits -Timeline movement flow loads **`portrait_thumb_path`** (~256px JPEG under `portraits/thumbs/{Artist}_thumb.jpg`) when available; biography and 3D exit navigation use full `portrait_path`. After adding portraits, run `npm run regenerate-portrait-thumbs` to backfill thumbs on dev. +Timeline movement flow loads **`portrait_thumb_path`** (~256px JPEG under `portraits/thumbs/{Artist}_thumb.jpg`) when available; biography and 3D exit navigation use full `portrait_path`. After adding portraits, run `npm run dev:regenerate-portrait-thumbs` to backfill thumbs on dev. -`npm run fetch-artist-images` runs `scripts/fetch-artist-images.js`: +`npm run dev:fetch-artist-images` runs `scripts/fetch-artist-images.js`: 1. For each artist, checks `data/images/portraits/{Artist}.jpg` (or other extensions) and sets `portrait_path` when a local file exists. 2. Otherwise downloads from Wikipedia / search fallbacks via `image-fetcher.js`. @@ -175,7 +175,7 @@ Run after seed when portrait files exist on disk but the DB still has null `port Directed influence links are stored in **`painting_influence_sources`**. Each row connects a painting to a **source** of type `painting`, `artist`, or `movement`, with optional period context (e.g. influence during the work’s creation year). -The legacy **`painting_influences`** table (painting-to-painting only) is still written alongside sources when running `npm run update-influences` — it keeps script compatibility and matches the backfill migration. **The API reads only `painting_influence_sources`**, so each edge appears once in the UI. +The legacy **`painting_influences`** table (painting-to-painting only) is still written alongside sources when running `npm run dev:update-influences` — it keeps script compatibility and matches the backfill migration. **The API reads only `painting_influence_sources`**, so each edge appears once in the UI. Influence data drives: @@ -188,7 +188,7 @@ Influence data drives: Painting-to-painting edges exist in both tables by design. To confirm the database has no stray duplicates and that the API model is clean: ```bash -npm run audit-influence-duplicates +npm run dev:audit-influence-duplicates ``` Reports: edges present in both tables, duplicate rows within either table (should be 0), and legacy-only / sources-only mismatches. If *Influenced* ever shows the same successor twice, restart the server after pulling API fixes — responses must not union legacy and sources tables. @@ -196,19 +196,19 @@ Reports: edges present in both tables, duplicate rows within either table (shoul ### One-time migration ```bash -npm run migrate:influence-sources # create table + backfill legacy painting edges +npm run dev:migrate:influence-sources # create table + backfill legacy painting edges ``` ### Curated updates -`npm run update-influences` runs `scripts/update-influences.js` against `scripts/art-influences-data.js`: +`npm run dev:update-influences` runs `scripts/update-influences.js` against `scripts/art-influences-data.js`: ```bash -npm run update-influences # insert curated edges -npm run update-influences -- --fetch-images # also download images for newly created works -npm run update-influences -- --discover # curated + web discovery pass -npm run discover-influences # discovery only (no curated file pass) -npm run update-influences -- --discover --limit=20 # cap discovery to N works +npm run dev:update-influences # insert curated edges +npm run dev:update-influences -- --fetch-images # also download images for newly created works +npm run dev:update-influences -- --discover # curated + web discovery pass +npm run dev:discover-influences # discovery only (no curated file pass) +npm run dev:update-influences -- --discover --limit=20 # cap discovery to N works ``` Each entry defines a later `work` and one or more `influencedBy` sources. Legacy single-object form is still supported: @@ -290,14 +290,14 @@ Short curator-style notes on the painting detail page — separate from the infl | UI | `PaintingAnnotations.tsx` — numbered markers on the image (when `pos_x` / `pos_y` set) plus an “Art history notes” list | | API | Included as `annotations[]` on `GET /api/paintings/:id` | | Curated data | `scripts/painting-annotations-data.js` — artist/title keys matched via `influence-resolver.js` | -| Load | `npm run update-painting-annotations` (replaces existing rows per painting by default) | -| Wikipedia pass | `npm run update-painting-annotations -- --wikipedia` — one intro sentence per work from `wikipedia_title`; use `--wiki-delay=3000` if rate-limited; `--no-replace` to append without clearing curated rows | +| Load | `npm run dev:update-painting-annotations` (replaces existing rows per painting by default) | +| Wikipedia pass | `npm run dev:update-painting-annotations -- --wikipedia` — one intro sentence per work from `wikipedia_title`; use `--wiki-delay=3000` if rate-limited; `--no-replace` to append without clearing curated rows | Categories include `subject`, `technique`, `context`, and `symbolism`. Sources cite Gombrich, museum catalogs, and Wikipedia as appropriate. ## Batch image fetch -`npm run fetch-images` (alias: `npm run search-missing-paintings`) runs `scripts/fetch-missing-images.js`. It searches multiple sources for paintings without local files: +`npm run dev:fetch-images` (alias: `npm run dev:search-missing-paintings`) runs `scripts/fetch-missing-images.js`. It searches multiple sources for paintings without local files: | Source | Notes | |--------|--------| @@ -315,13 +315,13 @@ Categories include `subject`, `technique`, `context`, and `symbolism`. Sources c | Harvard Art Museums | Optional (`HARVARD_ART_API_KEY` in `.env`) | ```bash -npm run fetch-images # all missing, catalog order (~hours) -npm run fetch-images -- --limit=50 # random sample of 50; 10s max per painting -npm run fetch-images -- --limit=250 # random sample up to N (caps at current missing count) -npm run fetch-images -- --limit=50 --max-wait=120 # slower, more thorough lookup per painting -npm run fetch-images -- --artist="Albrecht Dürer" # one artist, catalog order -npm run fetch-images -- --discover-only --limit=20 # fix wikipedia_title only -npm run fetch-images -- --web-search-only --limit=50 # DuckDuckGo + Commons + multilingual Wikipedia +npm run dev:fetch-images # all missing, catalog order (~hours) +npm run dev:fetch-images -- --limit=50 # random sample of 50; 10s max per painting +npm run dev:fetch-images -- --limit=250 # random sample up to N (caps at current missing count) +npm run dev:fetch-images -- --limit=50 --max-wait=120 # slower, more thorough lookup per painting +npm run dev:fetch-images -- --artist="Albrecht Dürer" # one artist, catalog order +npm run dev:fetch-images -- --discover-only --limit=20 # fix wikipedia_title only +npm run dev:fetch-images -- --web-search-only --limit=50 # DuckDuckGo + Commons + multilingual Wikipedia ``` Each run prints **`Missing local files: N`** at startup — that is the current count of catalogued paintings with no full-size or thumbnail file under `data/images/`. A painting counts as present if **either** file exists on disk. @@ -377,10 +377,10 @@ These live in `client/public/` (and `client/dist/` after build). 1. Insert rows into `artists`, `artist_periods`, `paintings` (or extend the seed script). 2. Place image files under `data/images/` using the naming convention. -3. Run `npm run fetch-artist-bios` for the new artist’s biography. -4. Add entries to `famous-paintings-data.js` and run `npm run expand-catalog` if needed. -5. Run `npm run fetch-images -- --artist="…"` or rely on preload / on-demand sync. -6. Add influence rows via `npm run update-influences` / `art-influences-data.js` (writes both `painting_influence_sources` and legacy painting edges). +3. Run `npm run dev:fetch-artist-bios` for the new artist’s biography. +4. Add entries to `famous-paintings-data.js` and run `npm run dev:expand-catalog` if needed. +5. Run `npm run dev:fetch-images -- --artist="…"` or rely on preload / on-demand sync. +6. Add influence rows via `npm run dev:update-influences` / `art-influences-data.js` (writes both `painting_influence_sources` and legacy painting edges). ## PainterPalette external dataset @@ -389,8 +389,8 @@ These live in `client/public/` (and `client/dist/` after build). ### One-time setup ```bash -npm run migrate:artist-palette # adds artists.palette_metadata JSONB -npm run import-painter-palette # enrich + influence links +npm run dev:migrate:artist-palette # adds artists.palette_metadata JSONB +npm run dev:import-painter-palette # enrich + influence links ``` ### What gets imported @@ -409,10 +409,10 @@ Name matching uses normalized strings plus aliases in `scripts/painter-palette-l ### Commands ```bash -npm run analyze-painter-palette # match report -npm run import-painter-palette -- --dry-run -npm run import-painter-palette -- --metadata-only -npm run import-painter-palette -- --influences-only +npm run dev:analyze-painter-palette # match report +npm run dev:import-painter-palette -- --dry-run +npm run dev:import-painter-palette -- --metadata-only +npm run dev:import-painter-palette -- --influences-only ``` Re-run `import-painter-palette` after adding gallery artists or updating the CSV; existing palette influence rows are skipped if already present. @@ -422,7 +422,7 @@ Re-run `import-painter-palette` after adding gallery artists or updating the CSV Export the full painting catalog as CSV: ```bash -npm run export-paintings +npm run dev:export-paintings ``` Writes **`Output/paintings.csv`** with columns `artist`, `painting`, `year` (sorted by artist, year, title). The `Output/` folder is git-ignored by convention; regenerate after catalog changes. @@ -454,7 +454,7 @@ The catalog can contain the same work more than once — usually from a **double ### Find duplicates ```bash -npm run find-duplicates +npm run dev:find-duplicates ``` Runs `scripts/find-duplicate-paintings.js`, which reports: diff --git a/Documentation/environments.md b/Documentation/environments.md index 52d1cab..64bab5f 100644 --- a/Documentation/environments.md +++ b/Documentation/environments.md @@ -32,10 +32,10 @@ Unless a task explicitly says **prod** or **production**, assume the target is * |------|---------|-----------|-----------|------| | Stop dev servers | Dev PC `192.168.10.70` | PowerShell (normal) | your user | Close `dev:web` / `dev:server` / `dev:client` terminals | | **DB split (one-time)** | Dev PC | **pgAdmin** → Query Tool on `postgres` | **postgres superuser** | [`db/split-dev-prod-pgadmin.sql`](../db/split-dev-prod-pgadmin.sql) step by step | -| Dev migrate / dev:web | Dev PC | PowerShell (normal) | your user | `npm run migrate`, `npm run dev:web` | +| Dev migrate / dev:web | Dev PC | PowerShell (normal) | your user | `npm run dev:migrate`, `npm run dev:web` | | Image dir on TrueNAS | TrueNAS `192.168.10.122` | **Shell** (SSH or UI → System Settings → Shell) | root / sudo | `bash infra/docker/truenas-setup.sh` | -| Copy images to prod | Dev PC | PowerShell (normal) | SMB `\\192.168.10.122\Gallery` | `net use` then `npm run images:sync-to-prod` | -| Build + push Docker image | Dev PC | **PowerShell as Administrator** | admin (for LAN hosts entry) | `npm run docker:publish` | +| Copy images to prod | Dev PC | PowerShell (normal) | SMB `\\192.168.10.122\Gallery` | `net use` then `npm run devtoprod:images` | +| Build + push Docker image | Dev PC | **PowerShell as Administrator** | admin (for LAN hosts entry) | `npm run prod:docker:publish` | | Install prod app | TrueNAS | **Web UI** → Apps → Custom App | admin | Paste `infra/docker/compose.truenas.yaml` | | Verify prod | Dev PC or TrueNAS | PowerShell / browser | any | `curl.exe -sk https://gallery.mysuperlab.netcraze.pro/api/bounds` | @@ -94,7 +94,7 @@ psql -h 192.168.10.122 -U postgres -d postgres -f db/split-dev-prod.sql ```powershell cd C:\Users\SNAP\Nextcloud\Personal\Repo\Gallery $env:PGHOST="192.168.10.122"; $env:PGUSER="postgres"; $env:PGPASSWORD="YOUR_POSTGRES_PASSWORD" -npm run db:split-databases +npm run infra:db:split-dev-prod ``` ### Step C — Configure dev PC and test @@ -113,13 +113,13 @@ npm run db:split-databases CURATOR_PASSWORD=your-secure-password ``` - `npm run migrate` creates auth tables and bootstraps the first curator when `users` is empty. + `npm run dev:migrate` creates auth tables and bootstraps the first curator when `users` is empty. 2. Run: ```powershell cd C:\Users\SNAP\Nextcloud\Personal\Repo\Gallery - npm run migrate + npm run dev:migrate npm run dev:web ``` @@ -152,7 +152,7 @@ Requires SMB share **`Gallery`** → `/mnt/BasePool/Applications/Gallery` on Tru net use \\192.168.10.122\Gallery /user:YOUR_TRUENAS_USER cd C:\Users\SNAP\Nextcloud\Personal\Repo\Gallery -npm run images:sync-to-prod +npm run devtoprod:images ``` UNC destination: `\\192.168.10.122\Gallery\data\images` @@ -170,7 +170,7 @@ Prerequisites: **Docker Desktop running**, logged in to Gitea. ```powershell cd C:\Users\SNAP\Nextcloud\Personal\Repo\Gallery docker login gitea.mysuperlab.netcraze.pro -npm run docker:publish +npm run prod:docker:publish ``` Use normal PowerShell with `-SkipHosts` if you already added `192.168.10.122 gitea.mysuperlab.netcraze.pro` to `C:\Windows\System32\drivers\etc\hosts`. @@ -185,7 +185,7 @@ Use normal PowerShell with `-SkipHosts` if you already added `192.168.10.122 git 4. Replace `REPLACE_WITH_LONG_RANDOM_SECRET` and `REPLACE_WITH_SECURE_PASSWORD` for `SESSION_SECRET` and `CURATOR_PASSWORD` 5. **Apps** → **Settings** → register Gitea registry (`gitea.mysuperlab.netcraze.pro`, token with `read:package`) 6. Deploy → wait for **gallery-web** to show **Running** -7. Run `npm run migrate` against `gallery_prod` if auth tables are not yet applied (or migrate from dev PC with prod env) +7. Run `npm run dev:migrate` against `gallery_prod` if auth tables are not yet applied (or migrate from dev PC with prod env) ### Step H — Verify production @@ -298,8 +298,8 @@ See also [Drunkmeyou gitea-https-keenetic-npm-setup.md](../../Drunkmeyou/Documen |------|---------| | Public dev URL (Keenetic) | `npm run dev:web` | | Fast local HMR (no Keenetic) | `npm run dev:server` + `npm run dev:client` | -| Refresh dev DB from prod | `npm run db:sync-from-prod` | -| Pull prod images to dev | `npm run images:sync-from-prod` | +| Refresh dev DB from prod | `npm run prodto:dev:db` | +| Pull prod images to dev | `npm run prodto:dev:images` | --- @@ -310,10 +310,10 @@ Run this when you are ready to ship dev to production — **not** after every sm **Where:** Dev PC unless noted 1. Finish and test on https://devgallery.mysuperlab.netcraze.pro -2. `npm run db:backup` -3. `npm run db:restore:prod -- --file db/DataBackup/gallery_dev_data_....txt` (type `yes`) — skip if only code changed and prod DB should stay as-is -4. `npm run images:sync-to-prod` — skip if no new/changed images -5. `npm run docker:publish` — required when application code changed +2. `npm run dev:db:backup` +3. `npm run devtoprod:db:restore -- --file db/DataBackup/gallery_dev_data_....txt` (type `yes`) — skip if only code changed and prod DB should stay as-is +4. `npm run devtoprod:images` — skip if no new/changed images +5. `npm run prod:docker:publish` — required when application code changed 6. **TrueNAS Web UI** → restart **gallery-web** 7. Verify https://gallery.mysuperlab.netcraze.pro @@ -324,17 +324,17 @@ Run this when you are ready to ship dev to production — **not** after every sm | Command | Where | Purpose | |---------|-------|---------| | pgAdmin + `split-dev-prod-pgadmin.sql` | Dev PC pgAdmin | One-time split (recommended) | -| `npm run db:split-databases` | Dev PC PowerShell | Same split (needs `PGUSER=postgres`) | -| `npm run db:sync-from-prod` | Dev PC PowerShell | Clone prod → dev | -| `npm run db:backup` | Dev PC PowerShell | Dev backup | -| `npm run db:restore:prod` | Dev PC PowerShell | Restore into prod | +| `npm run infra:db:split-dev-prod` | Dev PC PowerShell | Same split (needs `PGUSER=postgres`) | +| `npm run prodto:dev:db` | Dev PC PowerShell | Clone prod → dev | +| `npm run dev:db:backup` | Dev PC PowerShell | Dev backup | +| `npm run devtoprod:db:restore` | Dev PC PowerShell | Restore into prod | ## Image sync | Command | Where | Direction | |---------|-------|-----------| -| `npm run images:sync-to-prod` | Dev PC PowerShell | Dev → TrueNAS volume | -| `npm run images:sync-from-prod` | Dev PC PowerShell | TrueNAS → dev repo | +| `npm run devtoprod:images` | Dev PC PowerShell | Dev → TrueNAS volume | +| `npm run prodto:dev:images` | Dev PC PowerShell | TrueNAS → dev repo | ## Safety guards diff --git a/Documentation/setup.md b/Documentation/setup.md index 5e7f4d9..a9dbb27 100644 --- a/Documentation/setup.md +++ b/Documentation/setup.md @@ -54,23 +54,23 @@ cd client && npm install && cd .. One-shot setup (migrate + seed): ```bash -npm run setup +npm run dev:setup ``` Or step by step: ```bash -npm run migrate # db/schema.sql + db/migrate-*.sql via server/migrate.js -npm run seed # eras, movements, artists, flagship paintings (one per artist) +npm run dev:migrate # db/schema.sql + db/migrate-*.sql via server/migrate.js +npm run dev:seed # eras, movements, artists, flagship paintings (one per artist) ``` -`npm run migrate` is safe to re-run on existing databases (uses `IF NOT EXISTS` / `ADD COLUMN IF NOT EXISTS`). +`npm run dev:migrate` is safe to re-run on existing databases (uses `IF NOT EXISTS` / `ADD COLUMN IF NOT EXISTS`). If migration fails with permission errors, grant schema rights to the app user first (see [DB_structure.md](DB_structure.md)). ### Curator accounts (auth migration) -`npm run migrate` applies `db/migrate-auth.sql` (`users`, `curator_audit_log`, `session` tables). When the `users` table is empty and `CURATOR_USERNAME` / `CURATOR_PASSWORD` are set in `.env`, the first curator account is created automatically. +`npm run dev:migrate` applies `db/migrate-auth.sql` (`users`, `curator_audit_log`, `session` tables). When the `users` table is empty and `CURATOR_USERNAME` / `CURATOR_PASSWORD` are set in `.env`, the first curator account is created automatically. After migrate, sign in from the site header (**Curator login**). Debug mode, Checkup, and all mutating debug APIs require an active curator session. Anonymous visitors browse the timeline and 3D halls without logging in. @@ -81,50 +81,48 @@ See [API.md — Authentication](API.md#authentication) and [basics.md — Develo After a fresh seed, run these to match a fully populated local install: ```bash -npm run sync-image-paths # import paintings from data/images/paintings/ (clone with image files) -npm run fetch-artist-images # link local portraits or download from Wikipedia -npm run fetch-artist-bios # bio_short / bio_full from Wikipedia -npm run expand-catalog # famous works for artists below MIN_PAINTINGS -npm run update-influences # painting influence graph for detail view + hall exits -npm run migrate:checkup-flags # optional: review/fixed flags for Checkup page (paintings) -npm run migrate:artist-checkup-flags # optional: same flags for artist portraits (bio debug) -npm run migrate:painting-annotations # optional: art-history notes table -npm run update-painting-annotations # optional: load curated notes (+ --wikipedia for Wikipedia intros) -npm run fetch-images -- --limit=50 # random sample; 10s max per painting (default) -npm run fetch-images -- --limit=50 --max-wait=120 # same batch size, longer lookup per work +npm run dev:sync-image-paths # import paintings from data/images/paintings/ (clone with image files) +npm run dev:fetch-artist-images # link local portraits or download from Wikipedia +npm run dev:fetch-artist-bios # bio_short / bio_full from Wikipedia +npm run dev:expand-catalog # famous works for artists below MIN_PAINTINGS +npm run dev:update-influences # painting influence graph for detail view + hall exits +npm run dev:migrate:checkup-flags # optional: review/fixed flags for Checkup page (paintings) +npm run dev:migrate:artist-checkup-flags # optional: same flags for artist portraits (bio debug) +npm run dev:migrate:painting-annotations # optional: art-history notes table +npm run dev:update-painting-annotations # optional: load curated notes (+ --wikipedia for Wikipedia intros) +npm run dev:fetch-images -- --limit=50 # random sample; 10s max per painting (default) +npm run dev:fetch-images -- --limit=50 --max-wait=120 # same batch size, longer lookup per work cd client && npm run build && cd .. ``` -**Order matters:** `sync-image-paths` should run when the repo already contains painting files under `data/images/paintings/` but the database only has one flagship work per artist (typical after `npm run setup` on a clone). `update-influences` is required for *Influenced By* / *Influenced* panels and golden lamps in the 3D hall — seed does not insert influence edges. +**Order matters:** `dev:sync-image-paths` should run when the repo already contains painting files under `data/images/paintings/` but the database only has one flagship work per artist (typical after `npm run dev:setup` on a clone). `dev:update-influences` is required for *Influenced By* / *Influenced* panels and golden lamps in the 3D hall — seed does not insert influence edges. -Image fetch can take hours if you run it for the entire catalog. The first line of each run reports **`Missing local files: N`**. Use **`npm run fetch-images -- --limit=N`** for random batches (10s per painting by default), **`--artist="…"`** for one artist in catalog order, or on-demand resolution when viewing a painting in the detail view. +Image fetch can take hours if you run it for the entire catalog. The first line of each run reports **`Missing local files: N`**. Use **`npm run dev:fetch-images -- --limit=N`** for random batches (10s per painting by default), **`--artist="…"`** for one artist in catalog order, or on-demand resolution when viewing a painting in the detail view. ## Run | Command | Purpose | |---------|---------| | `npm run dev:web` | **Public dev stack** — Vite `:5173`, API `:3451` (Keenetic → devgallery…) | -| `npm run docker:publish` | Build + push prod image to Gitea | -| `npm run images:sync-to-prod` | Copy `data/images/` → TrueNAS via SMB `Gallery` share | -| `npm run images:sync-from-prod` | Copy prod images → dev repo | -| `npm run db:sync-from-prod` | Clone `gallery_prod` → `gallery_dev` | -| `npm run db:backup` / `db:restore:prod` | Dev backup / promote to prod | -| `npm run build` | Build production SPA into `client/dist` | -| `npm run start` | API + static SPA on `HOST`:`PORT` | -| `npm run start:prod` | Build client, then start server | -| `npm run server` | Alias for `start` | -| `npm run dev:server` | API with nodemon reload (local `:3520` / `:5173` workflow) | +| `npm run prod:docker:publish` | Build + push prod image to Gitea | +| `npm run devtoprod:images` | Copy `data/images/` → TrueNAS via SMB `Gallery` share | +| `npm run prodto:dev:images` | Copy prod images → dev repo | +| `npm run prodto:dev:db` | Clone `gallery_prod` → `gallery_dev` | +| `npm run dev:db:backup` / `devtoprod:db:restore` | Dev backup / promote DB to prod | +| `npm run prod:build` | Build production SPA into `client/dist` | +| `npm run dev:start` | API + static SPA on `HOST`:`PORT` (uses root `.env`) | +| `npm run prod:start` | Build client, then start server | +| `npm run dev:server` | API with nodemon reload (local HMR workflow) | | `npm run dev:client` | Vite dev server on :5173 | -| `npm run dev` | Alias for `start` | See [environments.md](environments.md) for dev/prod URLs, database split, Docker deploy, and sync commands. Quick reference: [FAC.md](FAC.md). **Production frontend:** build the client, then start the server: ```bash -npm run build +npm run prod:build # or: cd client && npm run build && cd .. -npm run start +npm run dev:start ``` Open http://localhost:5173 (Vite) or http://localhost:3451 (API only). @@ -144,9 +142,9 @@ Quick deploy checklist: 1. One-time DB split in **pgAdmin** on dev PC: [`db/split-dev-prod-pgadmin.sql`](../db/split-dev-prod-pgadmin.sql) 2. Dev `.env` → `DB_NAME=gallery_dev`, `PORT=3451`, `PUBLIC_URL=https://devgallery.mysuperlab.netcraze.pro`, plus auth vars (`SESSION_SECRET`, `CURATOR_*`) -3. `npm run migrate` on dev and prod DBs (includes auth tables + bootstrap curator) -4. `net use \\192.168.10.122\Gallery` → `npm run images:sync-to-prod` -5. `npm run docker:publish` → TrueNAS Custom App from `infra/docker/compose.truenas.yaml` (set auth env in compose) +3. `npm run dev:migrate` on dev and prod DBs (includes auth tables + bootstrap curator) +4. `net use \\192.168.10.122\Gallery` → `npm run devtoprod:images` +5. `npm run prod:docker:publish` → TrueNAS Custom App from `infra/docker/compose.truenas.yaml` (set auth env in compose) 6. Keenetic: both domains → `:5173`, protocol to device **`http`**, correct IP per environment ### Legacy deployment (optional) @@ -157,48 +155,48 @@ Node on the dev PC at `:3520` with nginx → Vite `:5173` is superseded by TrueN | Command | Script | Purpose | |---------|--------|---------| -| `npm run seed` | `scripts/seed-wikipedia.js` | Reload curated Wikipedia data | -| `npm run fetch-artist-bios` | `scripts/fetch-artist-bios.js` | Wikipedia lead sections → `bio_short` / `bio_full` | -| `npm run fetch-artist-bios -- --force` | ↑ | Refresh bios even when already set | -| `npm run expand-catalog` | `scripts/expand-paintings.js` | Insert famous works from `famous-paintings-data.js` | -| `npm run expand-catalog -- --fetch-images` | ↑ | Also download images for new rows (slow) | -| `npm run fetch-images` | `scripts/fetch-missing-images.js` | Search Wikipedia, Commons, museums for missing files | -| `npm run search-missing-paintings` | ↑ (alias) | Same as `fetch-images` | -| `npm run fetch-images -- --artist="Name"` | ↑ | Limit to one artist (catalog order) | -| `npm run fetch-images -- --limit=50` | ↑ | Random sample of N missing paintings | -| `npm run fetch-images -- --limit=250` | ↑ | Larger random batch (processes min(N, missing count)) | -| `npm run fetch-images -- --limit=50 --max-wait=120` | ↑ | Random batch; 120s cap per painting | -| `npm run fetch-images -- --discover-only --limit=20` | ↑ | Fix `wikipedia_title` via search only | -| `npm run regenerate-thumbnails` | `scripts/regenerate-thumbnails.js` | Rebuild all thumbs from full local files | -| `npm run audit-painting-images` | `scripts/audit-painting-images.js` | List thumb/full aspect-ratio mismatches | -| `npm run migrate:thumbnails` | `db/migrate-thumbnails.sql` | Add thumbnail columns | -| `npm run fetch-artist-images` | `scripts/fetch-artist-images.js` | Download portraits or link existing files under `data/images/portraits/` | -| `npm run fetch-artist-images -- --force` | ↑ | Re-fetch even when `portrait_path` is set | -| `npm run sync-image-paths` | `scripts/sync-image-paths.js` | Link `image_path` / `thumbnail_path` and import missing painting rows from disk | -| `npm run sync-image-paths -- --dry-run` | ↑ | Report only, no DB writes | -| `npm run migrate:influence-sources` | `scripts/migrate-influence-sources.js` | Create `painting_influence_sources` + backfill legacy edges | -| `npm run migrate:checkup-flags` | `scripts/migrate-checkup-flags.js` | Add `checkup_checked` / `checkup_fixed` on `paintings` | -| `npm run migrate:artist-checkup-flags` | `scripts/migrate-artist-checkup-flags.js` | Add `checkup_checked` / `checkup_fixed` on `artists` (bio debug) | -| `npm run migrate:painting-annotations` | `scripts/migrate-painting-annotations.js` | Create `painting_annotations` table | -| `npm run migrate:artist-palette` | `scripts/migrate-artist-palette.js` | Add `palette_metadata` JSONB on `artists` | -| `npm run import-painter-palette` | `scripts/import-painter-palette.js` | Enrich artists + influence links from `Inputs/PainterPalette.csv` | -| `npm run analyze-painter-palette` | `scripts/analyze-painter-palette.js` | Report CSV ↔ gallery artist name matches | -| `npm run export-paintings` | `scripts/export-paintings-csv.js` | Write `Output/paintings.csv` (artist, painting, year) | -| `npm run update-painting-annotations` | `scripts/update-painting-annotations.js` | Load curated notes from `painting-annotations-data.js` | -| `npm run update-painting-annotations -- --wikipedia` | ↑ | Add intro sentences from each work’s Wikipedia page | -| `npm run update-painting-annotations -- --wikipedia --wiki-delay=3000` | ↑ | Slower Wikipedia pass when rate-limited (429) | -| `npm run find-duplicates` | `scripts/find-duplicate-paintings.js` | Report duplicate and near-duplicate painting rows | -| `npm run audit-influence-duplicates` | `scripts/audit-influence-duplicates.js` | Report mirrored legacy/sources edges and duplicate influence rows | -| `npm run update-influences` | `scripts/update-influences.js` | Insert influence links (painting / artist / movement) from `art-influences-data.js` | -| `npm run update-influences -- --fetch-images` | ↑ | Also download images for newly created works | -| `npm run update-influences -- --discover` | ↑ | Curated pass + web discovery (Wikipedia, Wikidata, Met, art-history sites) | -| `npm run discover-influences` | `update-influences.js --discover-only` | Discovery pass only | +| `npm run dev:seed` | `scripts/seed-wikipedia.js` | Reload curated Wikipedia data | +| `npm run dev:fetch-artist-bios` | `scripts/fetch-artist-bios.js` | Wikipedia lead sections → `bio_short` / `bio_full` | +| `npm run dev:fetch-artist-bios -- --force` | ↑ | Refresh bios even when already set | +| `npm run dev:expand-catalog` | `scripts/expand-paintings.js` | Insert famous works from `famous-paintings-data.js` | +| `npm run dev:expand-catalog -- --fetch-images` | ↑ | Also download images for new rows (slow) | +| `npm run dev:fetch-images` | `scripts/fetch-missing-images.js` | Search Wikipedia, Commons, museums for missing files | +| `npm run dev:search-missing-paintings` | ↑ (alias) | Same as `fetch-images` | +| `npm run dev:fetch-images -- --artist="Name"` | ↑ | Limit to one artist (catalog order) | +| `npm run dev:fetch-images -- --limit=50` | ↑ | Random sample of N missing paintings | +| `npm run dev:fetch-images -- --limit=250` | ↑ | Larger random batch (processes min(N, missing count)) | +| `npm run dev:fetch-images -- --limit=50 --max-wait=120` | ↑ | Random batch; 120s cap per painting | +| `npm run dev:fetch-images -- --discover-only --limit=20` | ↑ | Fix `wikipedia_title` via search only | +| `npm run dev:regenerate-thumbnails` | `scripts/regenerate-thumbnails.js` | Rebuild all thumbs from full local files | +| `npm run dev:audit-painting-images` | `scripts/audit-painting-images.js` | List thumb/full aspect-ratio mismatches | +| `npm run dev:migrate:thumbnails` | `db/migrate-thumbnails.sql` | Add thumbnail columns | +| `npm run dev:fetch-artist-images` | `scripts/fetch-artist-images.js` | Download portraits or link existing files under `data/images/portraits/` | +| `npm run dev:fetch-artist-images -- --force` | ↑ | Re-fetch even when `portrait_path` is set | +| `npm run dev:sync-image-paths` | `scripts/sync-image-paths.js` | Link `image_path` / `thumbnail_path` and import missing painting rows from disk | +| `npm run dev:sync-image-paths -- --dry-run` | ↑ | Report only, no DB writes | +| `npm run dev:migrate:influence-sources` | `scripts/migrate-influence-sources.js` | Create `painting_influence_sources` + backfill legacy edges | +| `npm run dev:migrate:checkup-flags` | `scripts/migrate-checkup-flags.js` | Add `checkup_checked` / `checkup_fixed` on `paintings` | +| `npm run dev:migrate:artist-checkup-flags` | `scripts/migrate-artist-checkup-flags.js` | Add `checkup_checked` / `checkup_fixed` on `artists` (bio debug) | +| `npm run dev:migrate:painting-annotations` | `scripts/migrate-painting-annotations.js` | Create `painting_annotations` table | +| `npm run dev:migrate:artist-palette` | `scripts/migrate-artist-palette.js` | Add `palette_metadata` JSONB on `artists` | +| `npm run dev:import-painter-palette` | `scripts/import-painter-palette.js` | Enrich artists + influence links from `Inputs/PainterPalette.csv` | +| `npm run dev:analyze-painter-palette` | `scripts/analyze-painter-palette.js` | Report CSV ↔ gallery artist name matches | +| `npm run dev:export-paintings` | `scripts/export-paintings-csv.js` | Write `Output/paintings.csv` (artist, painting, year) | +| `npm run dev:update-painting-annotations` | `scripts/update-painting-annotations.js` | Load curated notes from `painting-annotations-data.js` | +| `npm run dev:update-painting-annotations -- --wikipedia` | ↑ | Add intro sentences from each work’s Wikipedia page | +| `npm run dev:update-painting-annotations -- --wikipedia --wiki-delay=3000` | ↑ | Slower Wikipedia pass when rate-limited (429) | +| `npm run dev:find-duplicates` | `scripts/find-duplicate-paintings.js` | Report duplicate and near-duplicate painting rows | +| `npm run dev:audit-influence-duplicates` | `scripts/audit-influence-duplicates.js` | Report mirrored legacy/sources edges and duplicate influence rows | +| `npm run dev:update-influences` | `scripts/update-influences.js` | Insert influence links (painting / artist / movement) from `art-influences-data.js` | +| `npm run dev:update-influences -- --fetch-images` | ↑ | Also download images for newly created works | +| `npm run dev:update-influences -- --discover` | ↑ | Curated pass + web discovery (Wikipedia, Wikidata, Met, art-history sites) | +| `npm run dev:discover-influences` | `update-influences.js --discover-only` | Discovery pass only | ### Scripts in the repository These are checked in and maintained: -- `seed-wikipedia.js`, `seed-catalog-data.js` — initial catalog (`npm run seed`) +- `seed-wikipedia.js`, `seed-catalog-data.js` — initial catalog (`npm run dev:seed`) - `image-fetcher.js` — Wikimedia / museum image resolution - `fetch-artist-images.js` — artist portrait download / disk linking - `sync-image-paths.js` — import paintings and align paths from `data/images/paintings/` @@ -232,44 +230,44 @@ After clone: copy `.env.example` → `.env`, install dependencies, run [one-time | Symptom | Likely cause | Fix | |---------|--------------|-----| -| Empty timeline | DB not seeded | `npm run seed` | +| Empty timeline | DB not seeded | `npm run dev:seed` | | 502 / 504 on public URL | Keenetic rule wrong (IP, port, or `https` to device) | Dev → `192.168.10.70:5173`; prod → `192.168.10.122:5173`; protocol **`http`** — see [environments.md](environments.md) | | 503 on public URL | Dev servers not running | `npm run dev:web` (or `dev:server` + `dev:client` for local HMR) | -| **`manifest unknown`** on TrueNAS deploy | Image not in Gitea | `npm run docker:publish` on dev PC first | +| **`manifest unknown`** on TrueNAS deploy | Image not in Gitea | `npm run prod:docker:publish` on dev PC first | | 500 on all `/api/*` | Wrong `.env` or Postgres down | Check connection, logs | -| “Biographical information not yet available” | Bios not fetched | `npm run fetch-artist-bios` | -| Placeholder portraits on timeline | `portrait_path` not set | `npm run fetch-artist-images` | -| Artist hall has only 1–2 paintings | DB not expanded / disk not imported | `npm run sync-image-paths` then `npm run expand-catalog`; extend `famous-paintings-data.js` | +| “Biographical information not yet available” | Bios not fetched | `npm run dev:fetch-artist-bios` | +| Placeholder portraits on timeline | `portrait_path` not set | `npm run dev:fetch-artist-images` | +| Artist hall has only 1–2 paintings | DB not expanded / disk not imported | `npm run dev:sync-image-paths` then `npm run dev:expand-catalog`; extend `famous-paintings-data.js` | | Movement flow shows “Loading art history…” repeatedly | Stale client refetching on every pan/zoom | Pull latest client; timeline loads catalog once — hard-refresh | | Movement flow slow on first load | Large artist payload | Client uses `GET /api/artists?timeline=1` (no full bios); rebuild client | | Movement flow zoom sluggish / dead over portraits | Stale client build | Rebuild client — wheel uses capture listener + interaction-mode rendering; hard-refresh | | Timeline year labels overlap when zoomed out | Stale client | Rebuild client — `chooseTimelineTickInterval()` adapts step to span and bar width | -| Black frames / canvas covers in 3D gallery | No local image for painting | `npm run fetch-images -- --limit=50` or `--artist="…"`; then `POST …/preload-images` | +| Black frames / canvas covers in 3D gallery | No local image for painting | `npm run dev:fetch-images -- --limit=50` or `--artist="…"`; then `POST …/preload-images` | | Many `⏱ timeout` lines in fetch batch | Default 10s cap too short for hard works | `--max-wait=120` or raise `FETCH_MAX_WAIT_SEC` | | White/grey flicker on frames | Texture loading or z-fighting with wall | Rebuild client (`cd client && npm run build`); ensure latest `VirtualGallery.tsx` | -| Wrong painting in 3D gallery frame | Stale or mismatched thumbnail file | `npm run regenerate-thumbnails`; gallery prefers full `image_path` | +| Wrong painting in 3D gallery frame | Stale or mismatched thumbnail file | `npm run dev:regenerate-thumbnails`; gallery prefers full `image_path` | | Wrong painting image (fetch) | Bad museum / search match on first download | Add entry to `DIRECT_IMAGE_OVERRIDES` in `scripts/image-fetcher.js`, re-fetch file | -| Empty exit navigation lists | No influence edges for artist | `npm run update-influences`; check `painting_influence_sources` | -| Empty *Influenced By* / *Influenced* on painting detail | No edges for that work | `npm run migrate:influence-sources` then `npm run update-influences`; extend `art-influences-data.js` or run `--discover` | -| Same work listed twice under *Influenced* | Stale server merging legacy + sources tables | Restart server; API reads `painting_influence_sources` only — run `npm run audit-influence-duplicates` to verify DB | -| No golden lamps above frames in 3D hall | Stale API process or no influence edges | Restart server after API changes; run `npm run update-influences` | +| Empty exit navigation lists | No influence edges for artist | `npm run dev:update-influences`; check `painting_influence_sources` | +| Empty *Influenced By* / *Influenced* on painting detail | No edges for that work | `npm run dev:migrate:influence-sources` then `npm run dev:update-influences`; extend `art-influences-data.js` or run `--discover` | +| Same work listed twice under *Influenced* | Stale server merging legacy + sources tables | Restart server; API reads `painting_influence_sources` only — run `npm run dev:audit-influence-duplicates` to verify DB | +| No golden lamps above frames in 3D hall | Stale API process or no influence edges | Restart server after API changes; run `npm run dev:update-influences` | | Default Vite page instead of gallery | `client/dist` missing or stale | `cd client && npm run build` | | Permission denied creating tables | `gallery` user lacks CREATE | Run admin grants, then migrate | | Wikipedia API rate limit during fetch | Too many requests in a row | Wait and re-run; scripts retry with backoff | -| Checkup **Reviewed** toggle returns 404 | Stale server process missing new routes | Restart `npm run dev` after pulling API changes | -| Debug **More** / **Clear** / **Upload** / **Remove entry** returns 404 | Stale server process | Restart `npm run start` or `npm run dev:server`; routes in `server/index.js` + `server/image-service.js` | +| Checkup **Reviewed** toggle returns 404 | Stale server process missing new routes | Restart `npm run dev:web` or `npm run dev:server` after pulling API changes | +| Debug **More** / **Clear** / **Upload** / **Remove entry** returns 404 | Stale server process | Restart `npm run dev:start` or `npm run dev:server`; routes in `server/index.js` + `server/image-service.js` | | Debug **Remove entry** — button stuck or missing on next painting | Stale client build | `cd client && npm run build`; hard-refresh — detail view remounts per painting id | | Debug **Upload** returns 413 Payload Too Large | Base64 JSON exceeds body limit | Server allows 20 MB JSON / 15 MB decoded image; compress file or resize before upload | -| No art-history notes on painting detail | Annotations not migrated or loaded | `npm run migrate:painting-annotations` then `npm run update-painting-annotations` | +| No art-history notes on painting detail | Annotations not migrated or loaded | `npm run dev:migrate:painting-annotations` then `npm run dev:update-painting-annotations` | | **Fix it** fails with `read ECONNRESET` | Remote host dropped connection | Restart server; client sends `searchUrl` / `source`; retry or use Commons URL in overrides | | Fixed image not shown in 3D gallery | Stale gallery session or cached texture | Rebuild client; fix updates session + `?v=` revision — use **Back to Gallery** (not browser back) | | Frame still black after **Checked** | Gallery session not synced | Re-enter hall or toggle debug **Checked** from detail with gallery open behind overlay | -| Duplicate works in gallery / timeline | Double import or variant Wikipedia titles | `npm run find-duplicates`; merge or delete spare rows manually | -| **Failed to load movement gallery** / `Cannot GET /api/movements/:id/gallery` | Stale server process missing route | Restart `npm run dev` or `npm run dev:server` after pulling API changes | +| Duplicate works in gallery / timeline | Double import or variant Wikipedia titles | `npm run dev:find-duplicates`; merge or delete spare rows manually | +| **Failed to load movement gallery** / `Cannot GET /api/movements/:id/gallery` | Stale server process missing route | Restart `npm run dev:web` or `npm run dev:server` after pulling API changes | | Movement gallery shows generic cream walls | Stale client build | `cd client && npm run build`; hard-refresh browser | | Windows overlap paintings in movement wing | Stale client | Rebuild client — windows are placed only on side walls in gaps between frames | -| Influence thumbnails cropped on painting detail | Stale client build | `npm run build` — panels use `object-fit: contain` for full image | -| **Curator login** fails / always guest | Auth tables missing or wrong password | Set `SESSION_SECRET` + `CURATOR_PASSWORD` in `.env`, run `npm run migrate`, restart server | +| Influence thumbnails cropped on painting detail | Stale client build | `npm run prod:build` — panels use `object-fit: contain` for full image | +| **Curator login** fails / always guest | Auth tables missing or wrong password | Set `SESSION_SECRET` + `CURATOR_PASSWORD` in `.env`, run `npm run dev:migrate`, restart server | | Debug / Checkup returns **401** | Not signed in as curator | **Curator login** (top-right); session cookie `gallery.sid` must be sent (`credentials: include`) | | Debug works in UI but API rejects | Stale server without auth middleware | Restart `npm run dev:web` or `npm run dev:server` after pulling auth changes | | **Empty screen** entering 3D hall (header missing) | Stale client before gallery-session fix | Hard-refresh; pull latest client — hall renders from `view` state, not only `gallerySession` | diff --git a/README.md b/README.md index 53ba965..bfb5c51 100644 --- a/README.md +++ b/README.md @@ -32,26 +32,26 @@ Interactive virtual art gallery: zoomable historical timeline with era click-to- 3. Apply database schema (requires PostgreSQL superuser for first-time setup): ```bash - npm run migrate - npm run seed + npm run dev:migrate + npm run dev:seed ``` 4. Enrich the catalog (recommended after seed): ```bash - npm run sync-image-paths # import paintings from data/images/paintings/ when present - npm run fetch-artist-images # link or download artist portraits - npm run fetch-artist-bios # Wikipedia biographies for all artists - npm run expand-catalog # add famous works for artists with thin catalogs - npm run update-influences # art-history lineage links (detail panels + 3D hall) - npm run migrate:artist-palette # optional: JSONB column for PainterPalette enrichment - npm run import-painter-palette # optional: metadata + influence links from Inputs/PainterPalette.csv - npm run migrate:checkup-flags # review/fixed flags for Checkup + debug mode (paintings) - npm run migrate:artist-checkup-flags # same flags for artist portraits (bio debug) - npm run migrate:painting-annotations # art-history notes on painting detail - npm run update-painting-annotations # load curated notes (+ optional --wikipedia) - npm run fetch-images -- --limit=50 # random batch of missing images (10s per work) - npm run fetch-images -- --artist="Claude Monet" # one artist in catalog order + npm run dev:sync-image-paths # import paintings from data/images/paintings/ when present + npm run dev:fetch-artist-images # link or download artist portraits + npm run dev:fetch-artist-bios # Wikipedia biographies for all artists + npm run dev:expand-catalog # add famous works for artists with thin catalogs + npm run dev:update-influences # art-history lineage links (detail panels + 3D hall) + npm run dev:migrate:artist-palette # optional: JSONB column for PainterPalette enrichment + npm run dev:import-painter-palette # optional: metadata + influence links from Inputs/PainterPalette.csv + npm run dev:migrate:checkup-flags # review/fixed flags for Checkup + debug mode (paintings) + npm run dev:migrate:artist-checkup-flags # same flags for artist portraits (bio debug) + npm run dev:migrate:painting-annotations # art-history notes on painting detail + npm run dev:update-painting-annotations # load curated notes (+ optional --wikipedia) + npm run dev:fetch-images -- --limit=50 # random batch of missing images (10s per work) + npm run dev:fetch-images -- --artist="Claude Monet" # one artist in catalog order ``` 5. **Development (public URL):**