diff --git a/.cursor/rules/dev-first-workflow.mdc b/.cursor/rules/dev-first-workflow.mdc new file mode 100644 index 0000000..dd0882f --- /dev/null +++ b/.cursor/rules/dev-first-workflow.mdc @@ -0,0 +1,43 @@ +--- +description: Default to dev for all changes; prod only when explicitly requested +alwaysApply: true +--- + +# Development-first workflow + +**Default target is always development** unless the user explicitly names **prod** / **production**. + +## What "dev" means + +| Layer | Dev default | +|-------|-------------| +| Code | Edit in this repo; run `npm run dev:web` on the dev PC | +| Database | `gallery_dev` via root `.env` (`DB_NAME=gallery_dev`) | +| Data / images | `data/images/` in the repo | +| Public URL | https://devgallery.mysuperlab.netcraze.pro | +| Migrations / seeds / scripts | `npm run dev:migrate`, `npm run dev:seed`, etc. against **dev** | + +## Production is scheduled, not daily + +Prod (`gallery_prod`, TrueNAS Docker, https://gallery.mysuperlab.netcraze.pro) is updated **roughly weekly** (or when the user explicitly asks for prod): + +1. Validate on dev +2. `npm run dev:db:backup` → `npm run devtoprod:db:restore` (if DB/data changed) +3. `npm run devtoprod:images` (if images changed) +4. `npm run prod:docker:publish` (if code changed) +5. Restart **gallery-web** on TrueNAS + +Do **not** edit `infra/docker/.env.prod`, run `db:restore:prod`, `images:sync-to-prod`, or `docker:publish` unless the user clearly targets prod. + +## When the user says "fix", "change", or "modify" + +- Assume **dev** code, **dev** DB, and **dev** data paths +- Test against devgallery or localhost +- Mention prod steps only as optional follow-up for the next release + +## When the user says "prod" or "production" + +- Use `gallery_prod`, `infra/docker/.env.prod`, TrueNAS compose, and prod deploy docs +- Require explicit confirmation before destructive prod operations (`db:restore:prod`, etc.) + +Full operator guide: `Documentation/environments.md` diff --git a/.env.example b/.env.example index 09199cb..776df1c 100644 --- a/.env.example +++ b/.env.example @@ -1,17 +1,29 @@ +# Development only — copy to .env (not committed). All day-to-day work uses gallery_dev. +# Promote to prod on a scheduled release (~weekly); see Documentation/environments.md DB_HOST=192.168.10.122 DB_PORT=5432 DB_USER=gallery -DB_PASSWORD= -DB_NAME=Gallery +DB_PASSWORD=YOUR_POSTGRES_PASSWORD +DB_NAME=gallery_dev -# Production: LAN http://192.168.10.70:3520 and reverse-proxy at gallery.mysuperlab.netcraze.pro -PORT=3520 +# Public dev URL (Keenetic → 192.168.10.70:5173) +# Local-only coding: npm run dev:server + dev:client on 3520 / 5173 instead +PORT=3451 HOST=0.0.0.0 -PUBLIC_URL=http://gallery.mysuperlab.netcraze.pro +PUBLIC_URL=https://devgallery.mysuperlab.netcraze.pro TRUST_PROXY=true IMAGE_DIR=./data/images +# Curator auth (run npm run dev:migrate after setting CURATOR_PASSWORD) +SESSION_SECRET=change-me-to-a-long-random-string +SESSION_COOKIE_SECURE=false +CURATOR_USERNAME=curator +CURATOR_PASSWORD= + +# Production uses infra/docker/.env.prod → gallery_prod at gallery.mysuperlab.netcraze.pro:5173 +# See Documentation/environments.md + # Optional — enable extra museum search in fetch-missing-images / search-missing-paintings # SMITHSONIAN_API_KEY= # HARVARD_ART_API_KEY= diff --git a/.gitignore b/.gitignore index 6774a01..ec7bed2 100644 --- a/.gitignore +++ b/.gitignore @@ -4,6 +4,14 @@ client/node_modules/ .env .env.local .env.*.local +infra/docker/.env.prod +infra/deploy/devtoprod.config.json +infra/deploy/harmonize.config.json +db/SyncReports/ +infra/deploy/last-docker-release.json + +# DB backups (may contain data) +db/DataBackup/ # Recovery / temp files from local restore _extracted/ diff --git a/Documentation/API.md b/Documentation/API.md index 536ad44..3525435 100644 --- a/Documentation/API.md +++ b/Documentation/API.md @@ -1,16 +1,133 @@ # Art Gallery — REST API -Base URL: +Base URL (paths are the same on every host; only the origin changes): -- **Production (public):** `http://gallery.mysuperlab.netcraze.pro` -- **Production (LAN):** `http://192.168.10.70:3520` -- **Development (direct):** `http://localhost:3520` or `http://localhost:3001` depending on `PORT` in `.env` -- **Development (Vite proxy):** `http://localhost:5173` (same paths) +| Context | Base URL | +|---------|----------| +| **Production (public)** | `https://gallery.mysuperlab.netcraze.pro` | +| **Production (LAN)** | `http://192.168.10.122:5173` | +| **Development (public)** | `https://devgallery.mysuperlab.netcraze.pro` | +| **Development (LAN)** | `http://192.168.10.70:5173` | +| **Local Vite proxy** | `http://localhost:5173` (proxies `/api` and `/images` to API on `:3451`) | +| **Local API only** | `http://localhost:3451` (when using `npm run dev:web`) | + +See [environments.md](environments.md) for Keenetic rules, databases, and deploy. All JSON responses use `Content-Type: application/json`. Errors return `{ "error": "message" }` with an appropriate HTTP status. Static images are served at `/images/` from `IMAGE_DIR`. +**Caching:** `/images` responses use `Cache-Control: public, max-age=0, must-revalidate` with `ETag` / `Last-Modified`. Painting and artist JSON payloads include optional **`image_cache_key`** / **`thumbnail_cache_key`** (and **`portrait_cache_key`** / **`portrait_thumb_cache_key`** on artists) — Unix ms from the file’s `mtime` on disk. The client appends `?v=` to image URLs so fix/upload/clear updates show immediately after reload even when the relative path is unchanged. + +**Quick check:** + +```powershell +curl.exe -sk https://gallery.mysuperlab.netcraze.pro/api/bounds +curl.exe -sk https://devgallery.mysuperlab.netcraze.pro/api/bounds +``` + +--- + +## Authentication + +Anonymous visitors have implicit role **`user`** (browse only). **Curator** accounts unlock debug mode, the Checkup page, and all mutating audit routes. + +Sessions use an HTTP-only cookie (`gallery.sid`). The client sends `credentials: 'include'` on API requests. + +### `GET /api/auth/me` + +**Response (anonymous)** + +```json +{ "role": "user" } +``` + +**Response (curator session)** + +```json +{ "role": "curator", "username": "curator" } +``` + +### `POST /api/auth/login` + +**Body:** `{ "username": "curator", "password": "…" }` + +**Response:** `{ "role": "curator", "username": "curator" }` + +**Errors:** `401` invalid credentials, `400` missing fields. + +### `POST /api/auth/logout` + +Destroys the session cookie. + +**Response:** `{ "ok": true }` + +### Curator-only routes + +These return **`401`** with `{ "error": "Curator login required" }` without a valid curator session: + +| Route | Audit action (mutations only) | +|-------|-------------------------------| +| `GET /api/paintings/checkup` | — (read) | +| `GET /api/paintings/:id/debug-image-search` (+ `/more`) | — | +| `GET /api/artists/:id/debug-portrait-search` (+ `/more`) | — | +| `GET /api/debug/image-proxy` | — | +| `PATCH /api/paintings/:id/checkup-flags` | `painting.checkup_flags` | +| `PATCH /api/artists/:id/checkup-flags` | `artist.checkup_flags` | +| `POST /api/paintings/:id/fix-image` | `painting.fix_image` | +| `POST /api/paintings/:id/clear-image` | `painting.clear_image` | +| `POST /api/paintings/:id/upload-image` | `painting.upload_image` | +| `DELETE /api/paintings/:id` | `painting.delete` | +| `POST /api/artists/:id/fix-portrait` | `artist.fix_portrait` | +| `POST /api/artists/:id/clear-portrait` | `artist.clear_portrait` | +| `POST /api/artists/:id/upload-portrait` | `artist.upload_portrait` | + +**Public** (no login): all catalog `GET` routes, `POST /api/artists/:id/preload-images` (local file linking for 3D halls), `/images`, SPA static. + +Curator mutations are recorded in `curator_audit_log` (see [DB_structure.md](DB_structure.md)). + +--- + +## `GET /api/catalog/bootstrap` + +**Preferred for timeline first paint.** Returns bounds, eras, movements, and slim artist rows in a single response (replaces the separate `bounds` + `timeline` + `artists?timeline=1` waterfall). + +**Query** + +| Param | Type | Default | Description | +|-------|------|---------|-------------| +| `start` | int | bounds `min_year` | Window start year | +| `end` | int | bounds `max_year` | Window end year | + +**Response** + +```json +{ + "bounds": { "min_year": -800, "max_year": 2100 }, + "eras": [ … ], + "movements": [ … ], + "artists": [ + { + "id": 1, + "name": "Claude Monet", + "birth_year": 1840, + "death_year": 1926, + "movement_id": 12, + "portrait_path": "portraits/Claude_Monet.jpg", + "portrait_thumb_path": "portraits/thumbs/Claude_Monet_thumb.jpg", + "wikipedia_title": "Claude Monet", + "century": 19, + "movement_name": "Impressionism", + "movement_color": "#6B8E9F" + } + ] +} +``` + +**Caching:** `Cache-Control: public, max-age=300` with `ETag` (304 when catalog row counts unchanged). + +The React home page loads this endpoint **once** on mount. Pan and zoom filter movements and portraits **client-side** — no refetch per view change. + --- ## `GET /api/bounds` @@ -54,6 +171,147 @@ Movements are filtered to those with at least one artist active in the requested --- +## Locale (`?locale=ru`) + +Public catalog endpoints accept optional **`locale`** query param (`en` default, `ru` supported) or `Accept-Language: ru`. + +Affected routes: `/api/catalog/bootstrap`, `/api/timeline`, `/api/search`, `/api/artists`, `/api/artists/:id`, `/api/paintings/:id`, `/api/movements/:id/gallery`, `/api/movements/:id/artists`, `/api/artists/:id/navigation`. + +Responses include `"locale": "ru"` when resolved. Display field names are unchanged; values come from `entity_translations` when `status = published`, else canonical English. + +Full guide: [i18n-russian.md](i18n-russian.md). + +--- + +## Translations (curator) + +Requires curator session. Base path: `/api/translations`. + +| Method | Path | Purpose | +|--------|------|---------| +| `GET` | `/api/translations/coverage?locale=ru` | Coverage stats | +| `GET` | `/api/translations/worklist/:entityType?locale=ru` | Artists/paintings/movements with translation status | +| `GET` | `/api/translations/:entityType/:id` | Canonical + all translation rows | +| `PUT` | `/api/translations/:entityType/:id` | Upsert fields `{ locale, fields, status }` | +| `POST` | `/api/translations/:entityType/:id/publish` | Publish all draft/reviewed rows for locale | + +## Influences (curator) + +Requires curator session. Base path: `/api/influences`. See [influence-import.md](influence-import.md). + +| Method | Path | Purpose | +|--------|------|---------| +| `GET` | `/api/influences` | List edges (`artistId`, `paintingId`, `q`, pagination) | +| `GET` | `/api/influences/graph?artistId=` | Neighborhood nodes/edges for visualization | +| `POST` | `/api/influences` | Create edge `{ paintingId, sourceType, sourceArtistId\|sourcePaintingId\|sourceMovementId, … }` | +| `PATCH` | `/api/influences/:id` | Update notes / remap source | +| `DELETE` | `/api/influences/:id` | Delete edge | +| `POST` | `/api/influences/import/parse` | Parse upload `{ filename, contentBase64, sheet? }` — returns `contentHash` / `payloadHash` / `alreadyImported` | +| `POST` | `/api/influences/import/preview` | Validate `{ rows, mapping, contentHash?, payloadHash? }` | +| `POST` | `/api/influences/import/commit` | Insert `{ proposals, contentHash?, payloadHash?, force? }` — `409` if duplicate unless `force` | + +Public painting detail still exposes read-only `influencedBy` / `influenced` (unchanged). + +--- + +## Tours + +Base path: `/api/tours`. Full guide: [tours.md](tours.md). + +| Method | Path | Auth | Purpose | +|--------|------|------|---------| +| `GET` | `/api/tours` | public | Published tour summaries (`id`, `title`, `description`, cover thumb, `stopCount`) | +| `GET` | `/api/tours/:id` | public* | Full tour + ordered `paintings` + `stopBodies` (*draft tours require curator session) | +| `GET` | `/api/tours/admin` | curator | All tours (any status) | +| `POST` | `/api/tours` | curator | Create `{ title, description?, status? }` | +| `PATCH` | `/api/tours/:id` | curator | Update title / description / status / cover | +| `DELETE` | `/api/tours/:id` | curator | Delete tour + stops | +| `PUT` | `/api/tours/:id/stops` | curator | Replace ordered stops `{ stops: [{ paintingId, body }] }` | + +Detail response shape: + +```json +{ + "tour": { "id": 1, "title": "…", "status": "published", "stopCount": 5 }, + "paintings": [ /* Painting rows in stop order */ ], + "stopBodies": { "42": "Tour text for this stop…" } +} +``` + +--- + +## `GET /api/search` + +Public catalog search over **artists**, **paintings**, and **art movements**. Used by the timeline header search bar (`CatalogSearchBar.tsx`). + +**Query** + +| Param | Type | Default | Description | +|-------|------|---------|-------------| +| `q` | string | — | Search text (min **2** characters after trim; shorter returns `{ q, results: [] }`) | +| `limit` | int | 20 | Max results total (capped at **50**) | +| `types` | string | all | Optional comma list: `artist`, `painting`, `movement` | +| `locale` | string | `en` | `ru` — search and return published Russian aliases when available | + +**Matching (case-insensitive `ILIKE`):** + +| Entity | Fields | +|--------|--------| +| Artist | `name`, `wikipedia_title`, movement name | +| Movement | movement `name`, era name | +| Painting | `title`, `wikipedia_title`, `year` (as text), artist name, movement name | +| All (when `locale=ru`) | Published rows in `entity_translations` for `name` / `title` | + +Prefix matches on primary labels (`name` / `title`) rank before substring matches. + +**Response** + +```json +{ + "q": "monet", + "results": [ + { + "type": "artist", + "id": 19, + "name": "Claude Monet", + "birth_year": 1840, + "death_year": 1926, + "movement_name": "Impressionism", + "portrait_path": "portraits/Claude_Monet.jpg", + "portrait_thumb_path": "portraits/thumbs/Claude_Monet_thumb.jpg" + }, + { + "type": "movement", + "id": 12, + "name": "Impressionism", + "color": "#87CEEB", + "start_year": 1860, + "end_year": 1890, + "era_name": "Modern" + }, + { + "type": "painting", + "id": 241, + "title": "Water Lilies", + "year": 1919, + "artist_id": 19, + "artist_name": "Claude Monet", + "movement_name": "Impressionism", + "thumbnail_path": "paintings/thumbs/Claude_Monet_Water_Lilies_thumb.jpg", + "image_path": "paintings/Claude_Monet_Water_Lilies.jpg" + } + ] +} +``` + +**Indexes:** applied by `npm run dev:migrate` (`db/migrate-search.sql`) or standalone `npm run dev:migrate:search`. + +**Client:** `api.search(q, { limit?, types? })`. + +**Navigation from search:** choosing a **painting** opens detail with `returnTo: timeline`; the client shows **← Back to Timeline** and calls `goToTimelineHome()` (clears gallery session, resets timeline zoom). Choosing an **artist** or **movement** uses the normal gallery entry handlers. + +--- + ## `GET /api/artists` Artists for timeline portraits and the movement flow diagram. @@ -65,9 +323,12 @@ Artists for timeline portraits and the movement flow diagram. | `start` | int | Only artists alive after this year | | `end` | int | Only artists born before this year | | `movement_id` | int | Filter by movement | +| `timeline` | bool | When `1` or `true`, return a lightweight row set for the home-page timeline (omits `bio_short`, `bio_full`; includes `portrait_thumb_path`) | **Response** — array of artist objects with joined `movement_name` and `movement_color`. +The React home page loads the timeline catalog **once** on mount via `GET /api/catalog/bootstrap` (or legacy: `GET /api/bounds` + `GET /api/timeline` + `GET /api/artists?timeline=1`). Pan and zoom filter movements and portraits **client-side** — no refetch per view change. Rapid pan/zoom is batched with `createViewChangeScheduler()` (one React update per animation frame). + --- ## `GET /api/movements/:id/artists` @@ -161,9 +422,9 @@ Each painting includes: | `checkup_checked` | Reviewed in checkup / debug workflow (gold frame in 3D when true) | | `checkup_fixed` | Image replaced via **Fix it** | -Populate biographies with `npm run fetch-artist-bios` (see [data-and-images.md](data-and-images.md)). +Populate biographies with `npm run dev:fetch-artist-bios` (see [data-and-images.md](data-and-images.md)). -Artist objects also include `checkup_checked` and `checkup_fixed` (same semantics as paintings; gold portrait border when reviewed). After `npm run import-painter-palette`, **`palette_metadata`** holds PainterPalette enrichment (nationality, styles, occupations, raw influence fields, etc.). Run `npm run migrate:artist-checkup-flags` and `npm run migrate:artist-palette` on existing databases. +Artist objects also include `checkup_checked` and `checkup_fixed` (same semantics as paintings; gold portrait border when reviewed). After `npm run dev:import-painter-palette`, **`palette_metadata`** holds PainterPalette enrichment (nationality, styles, occupations, raw influence fields, etc.). Run `npm run dev:migrate:artist-checkup-flags` and `npm run dev:migrate:artist-palette` on existing databases. --- @@ -300,7 +561,9 @@ Both lists are grouped by art movement and exclude the current artist. Each arti ## `POST /api/artists/:id/preload-images` -Fast local scan: links paintings to files already on disk. Does **not** download from the internet (safe to call before opening the 3D gallery). +**Public** — no curator login required. + +Fast local scan: links paintings to files already on disk. Does **not** download from the internet. The 3D client calls this automatically when entering an **artist** hall. **Response** @@ -320,7 +583,18 @@ Painting detail with influence graph neighbours. ```json { - "painting": { "id": 10, "title": "...", "artist_name": "...", "image_path": "...", "checkup_checked": false, "checkup_fixed": false, "has_influence_links": true, ... }, + "painting": { + "id": 10, + "title": "...", + "artist_name": "...", + "image_path": "paintings/Artist_Title.jpg", + "thumbnail_path": "paintings/thumbs/Artist_Title_thumb.jpg", + "image_cache_key": 1739123456789, + "thumbnail_cache_key": 1739123456790, + "checkup_checked": false, + "checkup_fixed": false, + "has_influence_links": true + }, "influencedBy": [ { "source_type": "painting", @@ -382,6 +656,8 @@ Returns the image bytes with `Cache-Control: public, max-age=86400`, or `404` if ## Developer image audit +**Curator login required** for every route in this section. See [Authentication](#authentication) above. + Routes for the **Checkup** page and **Debug mode** on painting detail and artist bio. Register `GET /api/paintings/checkup` **before** `GET /api/paintings/:id` so `"checkup"` is not parsed as a painting id. ### `GET /api/paintings/checkup` @@ -478,6 +754,8 @@ Only `imageUrl` is required; optional fields improve fetch success for hotlinked { "imagePath": "paintings/Artist_Title.jpg", "thumbnailPath": "paintings/thumbs/Artist_Title_thumb.jpg", + "image_cache_key": 1739123456789, + "thumbnail_cache_key": 1739123456790, "fixed": true, "checked": true } @@ -538,7 +816,7 @@ Upload a local painting image (base64 JSON body). Validates with `sharp`, writes } ``` -Max decoded size 15 MB (JSON body limit 20 MB on the server). **Response** — same shape as `fix-image` (`imagePath`, `thumbnailPath`, `fixed`, `checked`). +Max decoded size 15 MB (JSON body limit 20 MB on the server). **Response** — same shape as `fix-image` (`imagePath`, `thumbnailPath`, `image_cache_key`, `thumbnail_cache_key`, `fixed`, `checked`). --- @@ -574,22 +852,29 @@ Returns image bytes with appropriate `Content-Type`. ## Frontend helpers -The React client wraps these endpoints in `client/src/api/client.ts`: +The React client wraps these endpoints in `client/src/api/client.ts`. All requests send `credentials: 'include'` for session cookies. | Function | Maps to | |----------|---------| +| `getAuthMe()` | `GET /api/auth/me` | +| `loginCurator(user, pass)` | `POST /api/auth/login` | +| `logoutCurator()` | `POST /api/auth/logout` | | `api.getBounds()` | `GET /api/bounds` | | `api.getTimeline(start, end)` | `GET /api/timeline` | +| `api.search(q, options?)` | `GET /api/search` | | `api.getArtists(...)` | `GET /api/artists` | +| `api.getTimelineArtists()` | `GET /api/artists?timeline=1` | | `api.getArtist(id)` | `GET /api/artists/:id` | | `api.getArtistNavigation(id)` | `GET /api/artists/:id/navigation` | | `api.getPainting(id)` | `GET /api/paintings/:id` | | `preloadArtistImages(id)` | `POST /api/artists/:id/preload-images` | -| `imageUrl(path)` | `/images/` or placeholder | -| `galleryImageUrl(painting)` | Local thumb/full only (3D) | -| `galleryImageUrlWithRevision(painting, revision)` | Local URL with `?v=` cache buster after fix | -| `paintingImageUrl(painting)` | Local file, on-demand API, or `null` when cleared (`checkup_fixed` + no paths) | -| `portraitUrl(path, revision?)` | `/images/` with optional `?v=` cache buster | +| `imageUrl(path, revision?)` | `/images/` or placeholder; optional `?v=` cache buster | +| `paintingImageRevision(painting, sessionRevision?)` | Prefer API `image_cache_key` / `thumbnail_cache_key`, else in-session counter | +| `galleryImageUrl(painting, sessionRevision?)` | Local thumb/full only (3D); auto `?v=` from cache keys | +| `galleryImageUrlWithRevision(painting, sessionRevision?)` | Alias of `galleryImageUrl` | +| `paintingImageUrl(painting, sessionRevision?)` | Local file, on-demand API, or `null` when cleared (`checkup_fixed` + no paths) | +| `portraitUrl(path, revision?, artist?)` | `/images/` with `?v=` from revision or artist cache keys | +| `validateDebugUploadFile(file)` | Client-side size/type check before base64 upload | | `api.getPaintingCheckup()` | `GET /api/paintings/checkup` | | `api.updatePaintingCheckupFlags(id, flags)` | `PATCH /api/paintings/:id/checkup-flags` | | `api.getPaintingDebugImageSearch(id)` | `GET /api/paintings/:id/debug-image-search` | diff --git a/Documentation/DB_structure.md b/Documentation/DB_structure.md index 89cbf7f..b179511 100644 --- a/Documentation/DB_structure.md +++ b/Documentation/DB_structure.md @@ -1,15 +1,21 @@ # Art Gallery — database structure -PostgreSQL schema for the virtual gallery. Canonical DDL lives in **`db/schema.sql`**; **`server/migrate.js`** (`npm run migrate`) applies that file plus idempotent incremental scripts in `db/migrate-*.sql`. This document describes the logical model. +PostgreSQL schema for the virtual gallery. Canonical DDL lives in **`db/schema.sql`**; **`server/migrate.js`** (`npm run dev:migrate`) applies that file plus idempotent incremental scripts in `db/migrate-*.sql`. This document describes the logical model. -Connection settings come from `.env` (see [setup.md](setup.md)). +Connection settings come from `.env` (dev) or `infra/docker/.env.prod` (prod scripts). See [environments.md](environments.md) and [setup.md](setup.md). + +### One-time split (legacy `Gallery` → `gallery_prod` + `gallery_dev`) + +Run [`db/split-dev-prod-pgadmin.sql`](../db/split-dev-prod-pgadmin.sql) in **pgAdmin** on the dev PC (postgres superuser). Alternative: `npm run infra:db:split-dev-prod` with `PGUSER=postgres`. ## Overview | Item | Typical value | |------|----------------| | Engine | PostgreSQL 14+ | -| Database | `Gallery` | +| Database (dev) | `gallery_dev` | +| Database (prod) | `gallery_prod` | +| Legacy name | `Gallery` (one-time split → prod + dev) | | App user | `gallery` | | Time fields | Integer years (negative = BCE) | @@ -67,14 +73,14 @@ Finer-grained styles (Impressionism, Cubism, Suprematism, …). | `birth_year`, `death_year` | INTEGER | Nullable; used for timeline portrait placement | | `movement_id` | FK → `art_movements` | Primary movement | | `portrait_path` | VARCHAR(500) | Relative to `data/images/`; nullable after debug **Clear** | -| `bio_short`, `bio_full` | TEXT | Wikipedia lead section (`npm run fetch-artist-bios`) | +| `bio_short`, `bio_full` | TEXT | Wikipedia lead section (`npm run dev:fetch-artist-bios`) | | `wikipedia_title` | VARCHAR(300) | Source page title | | `century` | INTEGER | Rounded century bucket for seeding limits | | `checkup_checked` | BOOLEAN NOT NULL DEFAULT false | Portrait reviewed in debug workflow (gold border on bio when true) | | `checkup_fixed` | BOOLEAN NOT NULL DEFAULT false | Portrait replaced, cleared, or uploaded via debug | -| `palette_metadata` | JSONB | Enrichment from `Inputs/PainterPalette.csv` (`npm run import-painter-palette`) | +| `palette_metadata` | JSONB | Enrichment from `Inputs/PainterPalette.csv` (`npm run dev:import-painter-palette`) | -Applied by `npm run migrate:artist-checkup-flags` (`db/migrate-artist-checkup-flags.sql`) and `npm run migrate:artist-palette` (`db/migrate-artist-palette.sql`). +Applied by `npm run dev:migrate:artist-checkup-flags` (`db/migrate-artist-checkup-flags.sql`) and `npm run dev:migrate:artist-palette` (`db/migrate-artist-palette.sql`). ### `artist_periods` @@ -108,7 +114,7 @@ Phases within an artist’s career (e.g. “Blue Period”, “Roman Period”). When `checkup_fixed` is true, `checkup_checked` is set automatically and cannot be cleared until **Fixed** is off. A cleared painting (`image_path` and `thumbnail_path` both null, `checkup_fixed` true) is shown as an empty frame in detail view and is not refetched on demand. -Applied by `npm run migrate:checkup-flags` (`db/migrate-checkup-flags.sql`). +Applied by `npm run dev:migrate:checkup-flags` (`db/migrate-checkup-flags.sql`). ### `painting_annotations` @@ -128,7 +134,34 @@ Short art-history notes shown on painting detail (`PaintingAnnotations.tsx`). | `sort_order` | INTEGER | Display order within the painting | | `confidence` | VARCHAR(20) | Default `curated`; Wikipedia pass uses `wikipedia` | -Applied by `npm run migrate:painting-annotations` (`db/migrate-painting-annotations.sql`). Load data with `npm run update-painting-annotations` (curated entries in `scripts/painting-annotations-data.js`; add `--wikipedia` for intro sentences from each work’s `wikipedia_title`). +Applied by `npm run dev:migrate:painting-annotations` (`db/migrate-painting-annotations.sql`). Load data with `npm run dev:update-painting-annotations` (curated entries in `scripts/painting-annotations-data.js`; add `--wikipedia` for intro sentences from each work’s `wikipedia_title`). + +### `tours` / `tour_stops` + +Curated guided tours. See [tours.md](tours.md). + +**`tours`** + +| Column | Type | Notes | +|--------|------|-------| +| `id` | SERIAL PK | | +| `title` | VARCHAR(200) | | +| `description` | TEXT | Default `''` | +| `status` | VARCHAR(20) | `draft` \| `published` | +| `cover_painting_id` | FK → `paintings` | ON DELETE SET NULL | +| `created_at` / `updated_at` | TIMESTAMPTZ | `updated_at` via trigger | + +**`tour_stops`** + +| Column | Type | Notes | +|--------|------|-------| +| `id` | SERIAL PK | | +| `tour_id` | FK → `tours` | ON DELETE CASCADE | +| `painting_id` | FK → `paintings` | ON DELETE CASCADE; UNIQUE with `tour_id` | +| `sort_order` | INTEGER | Visitor / editor order | +| `body` | TEXT | English tour notes for the stop (v1) | + +Applied by `npm run dev:migrate` (`db/migrate-tours.sql`). ### `painting_influences` @@ -148,7 +181,7 @@ Directed edges: *this painting* was influenced by *that painting*. Unique constraint on `(painting_id, influenced_by_painting_id)`. -**Legacy mirror table.** `npm run update-influences` still inserts painting-to-painting rows here when curating data. The same edges are copied into `painting_influence_sources`. The **REST API does not read this table** for painting detail or hall navigation — use `painting_influence_sources` as the source of truth for display. +**Legacy mirror table.** `npm run dev:update-influences` still inserts painting-to-painting rows here when curating data. The same edges are copied into `painting_influence_sources`. The **REST API does not read this table** for painting detail or hall navigation — use `painting_influence_sources` as the source of truth for display. ### `painting_influence_sources` @@ -178,6 +211,51 @@ Unique index on `(painting_id, source_type, source_painting_id, source_artist_id **Canonical influence store.** Used by all API influence queries: painting detail (`influencedBy`, `influenced`), `has_influence_links`, and artist hall navigation (predecessors / successors). Legacy `painting_influences` rows are backfilled here on migration; new curated painting edges are written to both tables by `update-influences`. +### `users` + +Curator accounts (named logins). Anonymous site visitors do not have rows here. + +| Column | Type | Notes | +|--------|------|-------| +| `id` | SERIAL PK | | +| `username` | VARCHAR(64) UNIQUE | Login name | +| `password_hash` | VARCHAR(255) | bcrypt hash | +| `created_at` | TIMESTAMPTZ | | +| `last_login_at` | TIMESTAMPTZ | Updated on successful login | + +First curator is bootstrapped on `npm run dev:migrate` when `users` is empty and `CURATOR_USERNAME` / `CURATOR_PASSWORD` are set in env. + +### `curator_audit_log` + +Append-only log of curator debug mutations (fix/clear/upload/delete, checkup flag changes). + +| Column | Type | Notes | +|--------|------|-------| +| `id` | BIGSERIAL PK | | +| `user_id` | FK → `users` | Who performed the action | +| `action` | VARCHAR(64) | e.g. `painting.fix_image`, `artist.upload_portrait` | +| `resource_type` | VARCHAR(32) | `painting` or `artist` | +| `resource_id` | INTEGER | Target row id | +| `details` | JSONB | Optional metadata (URL, mime type, flag values) | +| `ip_address` | VARCHAR(45) | Client IP (respects `TRUST_PROXY`) | +| `created_at` | TIMESTAMPTZ | | + +**Logged `action` values:** `painting.fix_image`, `painting.clear_image`, `painting.upload_image`, `painting.delete`, `painting.checkup_flags`, `artist.fix_portrait`, `artist.clear_portrait`, `artist.upload_portrait`, `artist.checkup_flags`, `translation.upsert`, `translation.publish`, `influence.create`, `influence.update`, `influence.delete`, `influence.import`. + +Example query in pgAdmin: + +```sql +SELECT l.created_at, u.username, l.action, l.resource_type, l.resource_id, l.details +FROM curator_audit_log l +JOIN users u ON u.id = l.user_id +ORDER BY l.created_at DESC +LIMIT 50; +``` + +### `session` + +PostgreSQL session store for `express-session` (`connect-pg-simple`). Not application data. + ## Indexes - `artists(movement_id)`, `artists(century)` @@ -189,7 +267,7 @@ Unique index on `(painting_id, source_type, source_painting_id, source_artist_id ## First-time setup -The `gallery` database user needs `CREATE` on schema `public` for migrations. If tables cannot be created, run **`db/setup-admin.sql`** as PostgreSQL superuser (or the grants below) before `npm run migrate`: +The `gallery` database user needs `CREATE` on schema `public` for migrations. If tables cannot be created, run **`db/setup-admin.sql`** as PostgreSQL superuser (or the grants below) before `npm run dev:migrate`: ```sql GRANT CREATE ON SCHEMA public TO gallery; @@ -202,10 +280,10 @@ ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON SEQUENCES TO gallery; Then: ```bash -npm run migrate -npm run seed -npm run fetch-artist-bios -npm run expand-catalog +npm run dev:migrate +npm run dev:seed +npm run dev:fetch-artist-bios +npm run dev:expand-catalog ``` ## Data conventions @@ -213,5 +291,5 @@ npm run expand-catalog - **Year zero** is not used; BCE years are negative integers. - **Image paths** are relative to `IMAGE_DIR` (default `./data/images`), e.g. `portraits/Claude_Monet.jpg`, `paintings/thumbs/Raphael_The_School_of_Athens_thumb.jpg`. - **Seeding cap**: curated ingest targets at most ~100 artists per century to keep the catalog manageable. -- **Catalog expansion**: `scripts/famous-paintings-data.js` plus `npm run expand-catalog` raises thin artist catalogs to at least six notable paintings (`MIN_PAINTINGS`, default 6). -- **Biographies**: `bio_short` and `bio_full` are populated by `npm run fetch-artist-bios` from English Wikipedia lead sections; `wikipedia_title` on the artist row is the source article. +- **Catalog expansion**: `scripts/famous-paintings-data.js` plus `npm run dev:expand-catalog` raises thin artist catalogs to at least six notable paintings (`MIN_PAINTINGS`, default 6). +- **Biographies**: `bio_short` and `bio_full` are populated by `npm run dev:fetch-artist-bios` from English Wikipedia lead sections; `wikipedia_title` on the artist row is the source article. diff --git a/Documentation/FAC.md b/Documentation/FAC.md new file mode 100644 index 0000000..daab03b --- /dev/null +++ b/Documentation/FAC.md @@ -0,0 +1,363 @@ +# Gallery — command reference (FAC) + +Quick cheat sheet for daily operations. All `npm` commands run from the **repository root** unless noted. + +**Default target: dev.** Day-to-day commands use `gallery_dev`, repo `data/images/`, and https://devgallery.mysuperlab.netcraze.pro. Prod (`gallery_prod`, Docker, `prod:docker:publish`) is for **scheduled releases** (~weekly) — see [Promote dev → prod](#promote-dev--prod-scheduled-release) below. + +**Script prefixes:** `dev:` → `gallery_dev` / dev PC · `prod:` → production / TrueNAS · `devtoprod:` → promote dev → prod · `prodto:dev:` → refresh dev from prod · `infra:` → one-time setup + +**Environments:** + +| | Dev | Prod | +|---|-----|------| +| URL | https://devgallery.mysuperlab.netcraze.pro | https://gallery.mysuperlab.netcraze.pro | +| Host | Dev PC `192.168.10.70:5173` | TrueNAS `192.168.10.122:5173` | +| Database | `gallery_dev` | `gallery_prod` | + +Details: [environments.md](environments.md) · Deploy: [../infra/docker/DEPLOY-truenas.md](../infra/docker/DEPLOY-truenas.md) + +--- + +## Start and stop servers + +### Start — public dev (Keenetic URL) + +```powershell +cd C:\Users\SNAP\Nextcloud\Personal\Repo\Gallery +npm run dev:web +``` + +Vite on **:5173**, API on **:3451**. Open https://devgallery.mysuperlab.netcraze.pro or http://localhost:5173. + +### Start — local HMR (no Keenetic) + +Two terminals: + +```powershell +npm run dev:server # API — PORT from .env (default 3451) +npm run dev:client # Vite on :5173 +``` + +### Start — production-style (single Node process, built SPA) + +```powershell +npm run prod:start # build client + serve on PORT from .env +# or: +npm run prod:build +npm run dev:start +``` + +### Stop dev servers + +| Method | When | +|--------|------| +| **Ctrl+C** in the terminal running `dev:web` / `dev:server` / `dev:client` | Normal stop | +| Close the terminal tab | Same effect | + +Free ports **5173**, **3451**, **3520** before DB maintenance or if “port in use” errors appear. + +**Find what holds a port (PowerShell):** + +```powershell +netstat -ano | findstr ":5173 :3451" +# Stop by PID: +Stop-Process -Id -Force +``` + +### Stop / restart production (TrueNAS) + +| Action | Where | +|--------|-------| +| **Stop** | TrueNAS Web UI → Apps → **gallery-web** → Stop | +| **Restart** | Same → Restart (after `docker:publish` or config change) | +| **Update image** | Dev PC: `npm run prod:docker:publish` → restart app on TrueNAS | + +--- + +## First-time install + +```powershell +cd C:\Users\SNAP\Nextcloud\Personal\Repo\Gallery +copy .env.example .env # edit DB credentials, PUBLIC_URL +npm install +cd client; npm install; cd .. + +npm run dev:migrate # schema + incremental SQL (+ auth tables, bootstrap curator) +npm run dev:setup # migrate + seed (fresh empty DB only) +``` + +**Curator auth (after migrate):** set in `.env` before first `npm run dev:migrate` if the DB has no curator yet: + +```env +SESSION_SECRET=your-long-random-secret +CURATOR_USERNAME=curator +CURATOR_PASSWORD=your-secure-password +``` + +Then open the gallery → **Curator login** (top-right) → use debug mode / Checkup / Translations / **Influences**. Mutations are logged in `curator_audit_log` (view in pgAdmin). + +**Roles:** + +| Role | Access | +|------|--------| +| Guest (`user`) | Timeline, movement flow, 3D halls, painting detail, bios | +| Curator | Above + debug mode, Checkup, Translations, Influences (import/CRUD/graph), image fix/upload/delete APIs | + +**Audit log (pgAdmin on `gallery_dev` or `gallery_prod`):** + +```sql +SELECT l.created_at, u.username, l.action, l.resource_type, l.resource_id +FROM curator_audit_log l +JOIN users u ON u.id = l.user_id +ORDER BY l.created_at DESC +LIMIT 30; +``` + +**Prod auth env** (TrueNAS app or `infra/docker/.env.prod`): + +```env +SESSION_SECRET=long-random-secret +SESSION_COOKIE_SECURE=true +CURATOR_USERNAME=curator +CURATOR_PASSWORD=your-secure-password +``` + +Run `npm run dev:migrate` against prod DB after first deploy with auth vars set (creates tables + bootstrap curator if `users` is empty). + +--- + +## Database + +| Command | Description | +|---------|-------------| +| `npm run dev:migrate` | Apply `db/schema.sql` + incremental migrations (safe to re-run) | +| `npm run dev:setup` | `migrate` + `seed` — fresh catalog from Wikipedia data | +| `npm run infra:db:split-dev-prod` | One-time: legacy `Gallery` → `gallery_prod` + `gallery_dev` (needs `PGUSER=postgres`) | +| `npm run prodto:dev:db` | Clone `gallery_prod` → `gallery_dev` (TEMPLATE); also runs image sync from prod | +| `npm run dev:db:backup` | Dev data-only backup → `db/DataBackup/*.txt` + `.zip` | +| `npm run prod:db:backup` | Prod backup (reads `infra/docker/.env.prod`) | +| `npm run dev:db:restore -- --file ` | Restore backup into **dev** (truncates tables first; prompts `yes`) | +| `npm run devtoprod:db:restore -- --file ` | Restore into **prod** (requires confirmation) | +| `npm run harmonize` | Bidirectional catalog DB + image merge by `updated_at` / file mtime — [harmonize-dev-prod.md](harmonize-dev-prod.md) | +| `npm run harmonize:schema` | Apply dev migrations to prod schema only (dev → prod) | +| `npm run harmonize:db` / `harmonize:images` | DB or image merge only | +| `npm run dev:backfill-updated-at` | Backfill catalog `updated_at` from image mtimes (dev) | + +**One-time split (recommended):** pgAdmin on dev PC → open [`db/split-dev-prod-pgadmin.sql`](../db/split-dev-prod-pgadmin.sql) → run each STEP on database `postgres`, then verify on `gallery_dev`. + +**Dev `.env` essentials:** + +```env +DB_NAME=gallery_dev +PORT=3451 +PUBLIC_URL=https://devgallery.mysuperlab.netcraze.pro +SESSION_SECRET=your-long-random-secret +SESSION_COOKIE_SECURE=false +CURATOR_USERNAME=curator +CURATOR_PASSWORD=your-secure-password +``` + +--- + +## Import and enrich catalog data + +Run in this order on a **new** or **re-seeded** database: + +| # | Command | What it does | +|---|---------|----------------| +| 1 | `npm run dev:seed` | Eras, movements, artists, one flagship painting per artist | +| 2 | `npm run dev:sync-image-paths` | Import paintings from `data/images/paintings/`; link paths in DB | +| 3 | `npm run dev:fetch-artist-images` | Portraits → `data/images/portraits/`, set `portrait_path` | +| 4 | `npm run dev:fetch-artist-bios` | Wikipedia bios → `bio_short` / `bio_full` | +| 5 | `npm run dev:expand-catalog` | Add famous works per artist (below `MIN_PAINTINGS`) | +| 6 | `npm run dev:update-influences` | Influence graph (detail panels, 3D hall lamps) | +| 7 | `npm run dev:fetch-images -- --limit=50` | Download missing painting files (batch) | + +**One-shot bootstrap:** `npm run dev:setup` = steps 1 + migrate only; still run 2–7 for a full catalog. + +### Useful flags + +```powershell +npm run dev:sync-image-paths -- --dry-run +npm run dev:fetch-artist-images -- --force +npm run dev:fetch-artist-bios -- --force +npm run dev:expand-catalog -- --fetch-images +npm run dev:fetch-images -- --artist="Claude Monet" +npm run dev:fetch-images -- --limit=50 --max-wait=120 +npm run dev:update-influences -- --discover +npm run dev:discover-influences # discovery only, no curated insert +``` + +### Optional migrations / imports + +| Command | Description | +|---------|-------------| +| `npm run dev:migrate:thumbnails` | Add thumbnail columns | +| `npm run dev:migrate:influence-sources` | `painting_influence_sources` table + backfill | +| `npm run dev:migrate:checkup-flags` | Review flags on `paintings` | +| `npm run dev:migrate:artist-checkup-flags` | Review flags on `artists` | +| `npm run dev:migrate:painting-annotations` | Art-history notes table | +| `npm run dev:migrate:artist-palette` | `palette_metadata` JSONB on artists | +| `npm run dev:import-painter-palette` | Enrich from `Inputs/PainterPalette.csv` | +| `npm run dev:update-painting-annotations` | Load curated notes | +| `npm run dev:update-painting-annotations -- --wikipedia` | Add Wikipedia intro sentences | + +### Audit / export + +| Command | Description | +|---------|-------------| +| `npm run dev:audit-painting-images` | Thumb vs full aspect-ratio mismatches | +| `npm run dev:find-duplicates` | Duplicate / near-duplicate painting rows | +| `npm run dev:audit-influence-duplicates` | Duplicate influence edges | +| `npm run dev:analyze-painter-palette` | CSV ↔ artist name match report | +| `npm run dev:export-paintings` | Write `Output/paintings.csv` | + +--- + +## Images (local files) + +| Command | Description | +|---------|-------------| +| `npm run dev:fetch-images` | Search/download missing painting files (alias: `search-missing-paintings`) | +| `npm run dev:fetch-artist-images` | Download or link artist portraits | +| `npm run dev:sync-image-paths` | Align DB paths with files on disk; import new rows | +| `npm run dev:regenerate-thumbnails` | Rebuild painting thumbs from full images | +| `npm run dev:regenerate-portrait-thumbs` | Rebuild timeline portrait thumbs (~256px) | +| `npm run devtoprod:thumbnails` | Both of the above — run on dev before promote backup/sync | +| `npm run devtoprod:release` | Config-driven full promote (see [deploy-dev-to-prod.md](deploy-dev-to-prod.md#one-command-release-automated)) | + +**Local paths:** `data/images/portraits/`, `data/images/paintings/`, `data/images/paintings/thumbs/` + +--- + +## Dev ↔ prod image sync (SMB) + +SMB share **`Gallery`** → `/mnt/BasePool/Applications/Gallery` on TrueNAS. + +```powershell +net use \\192.168.10.122\Gallery /user:YOUR_TRUENAS_USER + +npm run devtoprod:release # full promote from infra/deploy/devtoprod.config.json +npm run devtoprod:thumbnails # rebuild thumb files + DB paths on dev (before backup) +npm run devtoprod:images # dev repo → TrueNAS (promote / first deploy) +npm run prodto:dev:images # TrueNAS → dev repo +npm run harmonize # bidirectional merge (newer wins) — see harmonize-dev-prod.md +npm run harmonize:dry-run # preview DB + image changes only +``` + +Type `yes` when prompted (or set `autoConfirm: true` in release config). Robocopy exit codes **0–7** = success. Deploy scripts print a final **`===== SUCCESS =====`** or **`===== FAILED =====`** banner. + +--- + +## Docker and production deploy + +| Command | Where | Description | +|---------|-------|-------------| +| `npm run prod:docker:publish` | Dev PC, **Admin** PowerShell, Docker Desktop running | Build + push `gallery-web:latest` to Gitea | +| `npm run prod:docker:push-only` | Same | Push only (skip rebuild) | +| `.\infra\docker\save-for-truenas.ps1` | Dev PC | Save image to `gallery-web.tar` (offline deploy) | + +**Before first deploy:** `npm run prod:docker:publish` → TrueNAS Custom App from `infra/docker/compose.truenas.yaml`. + +**After code changes:** `npm run prod:docker:publish` → restart **gallery-web** on TrueNAS. + +```powershell +docker login gitea.mysuperlab.netcraze.pro +``` + +--- + +## Build frontend + +```powershell +npm run prod:build # client → client/dist/ +cd client && npm run build && cd .. +``` + +Prod container serves `client/dist/` from the Docker image (rebuild image after UI changes). + +--- + +## Verify (health checks) + +```powershell +# Dev (servers running) +curl.exe -sk https://devgallery.mysuperlab.netcraze.pro/api/bounds +curl.exe -s http://192.168.10.70:5173/api/bounds + +# Prod (bypass Keenetic) +curl.exe -s http://192.168.10.122:5173/api/bounds + +# Prod (public) +curl.exe -sk https://gallery.mysuperlab.netcraze.pro/api/bounds +``` + +Expect JSON with `min_year` / `max_year`. HTML shell only from `curl` on `/` is normal (Vite dev). + +**PowerShell note:** use `curl.exe`, not `curl` — PowerShell aliases `curl` to `Invoke-WebRequest` (no `-k` flag). + +--- + +## Promote dev → prod (scheduled release) + +~Weekly (or when explicitly releasing to prod). Not part of daily dev. Full runbook with per-change decision matrix and rollback: [deploy-dev-to-prod.md](deploy-dev-to-prod.md). + +**One command (recommended):** copy `infra/deploy/devtoprod.config.example.json` → `infra/deploy/devtoprod.config.json`, edit SMB/git settings, then: + +```powershell +npm run devtoprod:release +# or: deploy-dev-to-prod.cmd +``` + +Dry-run: `npm run devtoprod:release -- -DryRun`. The script pauses for a manual **gallery-web** restart on TrueNAS before verify. + +**Manual steps** (partial releases): + +1. Test on https://devgallery.mysuperlab.netcraze.pro +2. `npm run devtoprod:thumbnails` (if paintings/portraits changed — rebuild thumb files + DB paths on dev) +3. `npm run dev:db:backup` +4. `npm run devtoprod:db:restore -- --file db/DataBackup/gallery_dev_data_....txt` (if DB/catalog changed) +5. `npm run devtoprod:images` (if images changed) +6. `npm run prod:docker:publish` (if code changed) +7. Restart **gallery-web** on TrueNAS +8. Verify https://gallery.mysuperlab.netcraze.pro + +--- + +## Keenetic (router) + +Both rules: **protocol to device = `http`**, **Preserve Host = ON**. + +| Domain | Upstream | +|--------|----------| +| `devgallery.mysuperlab.netcraze.pro` | `192.168.10.70:5173` | +| `gallery.mysuperlab.netcraze.pro` | `192.168.10.122:5173` | + +Wrong IP or `https` to device → **502 / 504** (`Server: Web server`). + +--- + +## Git (Gitea) + +```powershell +git status +git add . +git commit -m "Your message" +git push origin main +``` + +Remote: https://gitea.mysuperlab.netcraze.pro/Danilka/Art-gallery + +--- + +## Related docs + +| Document | Contents | +|----------|----------| +| [deploy-dev-to-prod.md](deploy-dev-to-prod.md) | Step-by-step release runbook (code, DB, data, images) | +| [environments.md](environments.md) | Full dev/prod walkthrough | +| [setup.md](setup.md) | Install, env vars, troubleshooting | +| [data-and-images.md](data-and-images.md) | Catalog and image pipeline | +| [API.md](API.md) | REST endpoints | +| [tours.md](tours.md) | Guided tours | +| [DB_structure.md](DB_structure.md) | PostgreSQL schema | diff --git a/Documentation/Plans.md b/Documentation/Plans.md new file mode 100644 index 0000000..0e447ac --- /dev/null +++ b/Documentation/Plans.md @@ -0,0 +1,10 @@ +this file contains draft for future releases and features + +1. ~~Multi language support, russian version at least~~ — done: UI i18n (EN/RU) + `entity_translations` DB + curator Translations tool — [i18n-russian.md](i18n-russian.md) +2. ~~tool to manage links (influence/influenced by ) import csv's ( define format), edit ,add, delete, visualize, map to pictures/ entities~~ — done: curator Influences page (list CRUD + import wizard CSV/JSON/XLSX + neighborhood graph) — [influence-import.md](influence-import.md) +3. tool to monitor/manage (plan actions) of curator actions, markers to check painting/text ? +4. ~~tool to sync prod /env resources (both ways), db structure, db data, images, users etc~~ — done for catalog DB + images: `npm run harmonize` (schema dev→prod only; users/audit excluded) — [harmonize-dev-prod.md](harmonize-dev-prod.md) +5. ~~curator_audit_log should contain log of actions like fixit, checked, upload etc with details for which entity it was made and details what was the action and outcome~~ — done: table + `logCuratorAction` on fix/clear/upload/delete/checkup flags (and translation upsert/publish); see [DB_structure.md](DB_structure.md#curator_audit_log). (UI to browse logs is still item 3.) +6. ~~create search by entity (painting, artist, movement)~~ — done: timeline header + `GET /api/search` +7. ~~create guided tours (with text/extra infor, set of entities)~~ — done: `tours` / `tour_stops`, public Tours popup + 3D tour hall, curator Tour editor — [tours.md](tours.md) + diff --git a/Documentation/basics.md b/Documentation/basics.md index af5e63a..635502e 100644 --- a/Documentation/basics.md +++ b/Documentation/basics.md @@ -8,10 +8,12 @@ The app is organised as a **drill-down hierarchy**: 1. **Timeline** — historical eras (Ancient → Contemporary) with definite or fuzzy date boundaries. 2. **Movement flow** — art movements as curved SVG streams on the same year axis; documented predecessor→successor branches; portrait thumbnails placed along each stream. -3. **3D gallery** — one personal hall per artist *or* a **movement gallery** (click a movement name on the flow diagram): period-themed interiors, chronological wings of up to ~55 works, side-wall hang only. +3. **3D gallery** — one personal hall per artist, a **movement gallery** (click a movement name), or a **guided tour** hall (timeline → **Tours**): period-themed or tour wings of up to ~55 works, side-wall hang, visit order left→right. 4. **Painting detail** — full work in the centre, *Influenced By* on the left, *Influenced* on the right (paintings, artists, or movements), optional **art-history notes** with image markers, prev/next catalog browsing, optional fullscreen, link to artist biography. 5. **Artist biography** — portrait, lifespan, movement, and Wikipedia-sourced intro text (`bio_short` / `bio_full`). With debug mode on, the same image-audit panel as painting detail (portrait search, **Checked** / **Fix it** / **More** / **Clear** / **Upload**). +**Catalog search** — on the timeline home page, the header search bar (`CatalogSearchBar.tsx`) finds artists, paintings, and movements by name and metadata (year, movement, Wikipedia title). Type at least **2 characters** (300 ms debounce); results group into **Artists**, **Movements**, and **Paintings** with thumbnails. Keyboard: `↑`/`↓` to move, `Enter` to open, `Escape` to close. Choosing a result opens the artist gallery, movement gallery, or painting detail. Paintings opened from search show **← Back to Timeline** and return to the home timeline (full year range), not the previous view. + All artwork images are stored locally under `data/images/` — the UI never hot-links to Wikipedia or Commons at runtime (except optional on-demand fetch when a file is missing). ## Stack @@ -31,7 +33,8 @@ Gallery/ ├── server/ # Express API, DB pool, image service ├── client/ # React/Vite frontend │ ├── src/ # Source (components, pages, 3D scene) -│ │ ├── components/VirtualGallery.tsx # 3D hall (artist + movement modes) +│ │ ├── components/VirtualGallery.tsx # 3D hall (artist + movement modes; WebGL context-loss recovery) +│ │ ├── components/GalleryLoadingMarker.tsx # Loading spinner overlay/banner (catalog, portraits, halls) │ │ ├── components/GalleryWindows.tsx # Side-wall daylight windows (movement) │ │ ├── components/HallPassage.tsx # Open archway between movement wings │ │ ├── components/MovementHallDetails.tsx # Period architectural details @@ -44,8 +47,15 @@ Gallery/ │ │ ├── components/Timeline.tsx # Era bar, year ticks, event markers │ │ ├── components/TimelineEventGuides.tsx # Event vertical guides into movement flow │ │ ├── components/MovementBands.tsx # Movement flow (SVG streams + branches) +│ │ ├── components/CatalogSearchBar.tsx # Timeline header catalog search │ │ ├── components/PaintingAnnotations.tsx # Art-history notes on painting detail │ │ ├── pages/CheckupPage.tsx # Image audit table +│ │ ├── pages/TranslationsPage.tsx # Russian translation review +│ │ ├── pages/InfluencesPage.tsx # Influence links CRUD + import wizard +│ │ ├── pages/ToursPage.tsx # Guided tour editor +│ │ ├── components/ToursPopup.tsx # Public published-tours modal +│ │ ├── i18n/ # react-i18next bootstrap +│ │ └── locales/{en,ru}/ # UI chrome strings │ │ ├── data/historical-events.ts # Timeline event markers (UI) │ │ ├── data/movement-lineage.ts # Curated movement predecessor links (UI) │ │ ├── utils/parquetFloorTexture.ts # Procedural parquet floor @@ -53,6 +63,10 @@ Gallery/ │ │ └── utils/timelineView.ts # Shared zoom/pan math for timeline + movements │ └── dist/ # Production build (served by API when present) ├── scripts/ # Seed, bios, catalog expansion, image fetch, checkup tools +│ ├── seed-wikipedia.js +│ ├── seed-catalog-data.js +│ ├── sync-image-paths.js +│ ├── fetch-artist-images.js │ ├── fetch-artist-bios.js │ ├── expand-paintings.js │ ├── famous-paintings-data.js @@ -66,42 +80,59 @@ Gallery/ ├── Output/ # Generated exports (e.g. paintings.csv) ├── data/images/ # Local portraits and paintings (+ thumbs/) ├── db/ # schema.sql, setup-admin.sql, migrate-*.sql -├── server/migrate.js # npm run migrate — schema + incremental migrations -├── deploy/ # Production nginx + systemd examples +├── server/migrate.js # npm run dev:migrate — schema + incremental migrations +├── deploy/ # Legacy nginx + systemd examples (optional) +├── infra/docker/ # Production Dockerfile, TrueNAS compose, deploy scripts ├── Documentation/ # This folder +│ └── environments.md # Dev/prod URLs, DB split, sync, deploy └── .env # DB and port config (not committed) ``` ## Runtime modes -### Production-style (single process) +**Default:** develop and test on **dev** (`gallery_dev`, devgallery URL). **Production** is updated on a scheduled release (~weekly), not on every edit. See [environments.md — Development-first workflow](environments.md#development-first-workflow-default). + +### Public development (`dev:web`) — primary ```bash -npm run start:prod # build client + serve on PORT (default 3520) -# or: npm run build && npm run start +npm run dev:web # Vite :5173 + API :3451 — https://devgallery.mysuperlab.netcraze.pro ``` -Serves `/api/*`, `/images/*`, and the built SPA from `client/dist` if it exists. +Uses database **`gallery_dev`** on the same PostgreSQL host. -**Deployed URLs:** public http://gallery.mysuperlab.netcraze.pro · LAN http://192.168.10.70:3520 — see [setup.md](setup.md#production-deployment). +### Production (TrueNAS Docker) — scheduled releases -### Development (two processes) +Production runs in **`gallery-web`** on TrueNAS port **5173**, database **`gallery_prod`**, public URL **https://gallery.mysuperlab.netcraze.pro**. Images: `/mnt/BasePool/Applications/Gallery/data/images` (SMB share **`Gallery`**). Deploy via `npm run prod:docker:publish` and the promote checklist in [environments.md](environments.md). + +### Production-style single process (local) ```bash -npm run dev:server # API on PORT from .env (3520 production, 3001 typical dev) -npm run dev:client # Vite on :5173, proxies /api and /images to PORT +npm run prod:start # build client + serve on PORT from .env ``` -Use the Vite URL during frontend work for HMR. +Serves `/api/*`, `/images/*`, and the built SPA from `client/dist`. + +### Local HMR (two processes) + +```bash +npm run dev:server # API on PORT from .env +npm run dev:client # Vite on :5173, proxies /api and /images +``` + +Use for fast frontend iteration without Keenetic. Legacy nginx config in [`deploy/nginx-gallery.conf`](../deploy/nginx-gallery.conf) proxied the public domain to Vite `:5173`. ## User navigation flow ```mermaid flowchart TD A[Home — timeline + movement flow] -->|scroll / drag / zoom| A + A -->|catalog search| S[Search results dropdown] + S -->|artist| C + S -->|movement| G + S -->|painting| D A -->|click movement name| G[Movement gallery — 3D wings] G -->|click painting| D - D -->|Back| G + D -->|Back to Gallery| G G -->|back door / Wings / Exit| H[Wing navigator] H -->|pick wing| G H -->|Exit to Timeline| A @@ -113,16 +144,28 @@ flowchart TD D -->|prev / next| D D -->|click centre image| F[Fullscreen lightbox] F -->|close| D - D -->|Back| C + D -->|Back to Gallery| C C -->|exit doorway / E key| E[Path picker] E -->|predecessors| C E -->|successors| C D -->|influence thumbnail| D D -->|artist link| B - B -->|Back| A - C -->|Back| A + B -->|Back| C + C -->|Back to Timeline| A + D -->|Back to Timeline when opened from search| A ``` +### Back navigation + +| Control | Behaviour | +|---------|-----------| +| **← Back to Timeline** (3D gallery header, movement **Exit to Timeline**) | Always returns to the **home timeline**: unmounts the hall, clears the gallery session, resets timeline zoom to the full catalog year range | +| **← Back to Gallery** (painting detail from a hall) | Returns to the **same hall session** — camera position and wing are preserved | +| **← Back to Timeline** (painting detail opened from catalog search) | Returns to the home timeline (same as gallery **Back to Timeline**) | +| **← Back** (artist bio) | Returns to wherever you opened bio from (usually the artist hall) | + +Implementation: `goToTimelineHome()` in `HomePage.tsx` — do not use the browser **Back** button; it is not wired to app navigation. + ## Timeline and movement flow The home page shows two linked views over the **same year window** (`viewStart` / `viewEnd` in `HomePage.tsx`): @@ -132,12 +175,53 @@ The home page shows two linked views over the **same year window** (`viewStart` | Era bar | `Timeline.tsx` | Historical eras, major event markers, click-to-zoom | | Movement flow | `MovementBands.tsx` | Curved streams per movement, lineage branches, artist portraits | -Both views share zoom/pan behaviour via `client/src/utils/timelineView.ts`. The home page uses a **fixed viewport** (`100vh`): timeline + movement flow sit in a shared `home-timeline-stack` so event guide lines can extend from the era bar down through the movement canvas. The movement flow compresses vertically so all movements in the visible year range fit without page scrolling. +Both views share zoom/pan behaviour via `client/src/utils/timelineView.ts` (`zoomTimelineView`, `panTimelineView`, `chooseTimelineTickInterval`, `createViewChangeScheduler`). The home page uses a **fixed viewport** (`100vh`): timeline + movement flow sit in a shared `home-timeline-stack` so event guide lines can extend from the era bar down through the movement canvas. The movement flow compresses vertically so all movements in the visible year range fit without page scrolling. + +### Catalog search (timeline header) + +`CatalogSearchBar.tsx` calls `GET /api/search?q=…` (public, no login). The dropdown is stacked above the timeline (`z-index` on `.site-header`) so results are not hidden by movement bands. + +| UX | Detail | +|----|--------| +| Minimum query | 2 characters after trim | +| Debounce | 300 ms | +| Result groups | Artists, Movements, Paintings (with thumb or movement colour swatch) | +| Open artist | Preload images → artist 3D hall | +| Open movement | Movement gallery (wing 1) | +| Open painting | Painting detail with `returnTo: timeline` → **Back to Timeline** | +| Keyboard | `↑`/`↓` highlight, `Enter` open, `Escape` close | + +Run `npm run dev:migrate:search` once on existing databases before first use, or rely on `npm run dev:migrate` (includes `migrate-search.sql`). See [API.md — GET /api/search](API.md#get-apsearch). + +### Timeline data loading + +On first visit, `HomePage.tsx` fetches the full catalog once: + +1. `GET /api/bounds` — initialise the year range. +2. `GET /api/timeline?start=…&end=…` — all eras and movements for that range. +3. `GET /api/artists?timeline=1` — lightweight artist rows (portraits, lifespan, movement colour; no full biography text). + +Pan, zoom, and era/event click-to-zoom only update **local** `viewStart` / `viewEnd` state. `MovementBands.tsx` and `Timeline.tsx` filter what is visible for the current window — they do not trigger new API calls. + +**Loading indicators** (`GalleryLoadingMarker.tsx`) keep the user informed while data is still arriving: + +| Marker | When | +|--------|------| +| Overlay **“Loading art history…”** | Until the first catalog fetch (`bounds` + `timeline` + `artists`) completes | +| Bottom banner **“Loading portraits…”** | While artist portrait thumbnails are still downloading on the movement flow (timeline stays interactive) | +| Overlay **“Opening artist/movement gallery…”** | Between clicking a portrait/movement and the 3D hall data being ready | +| Overlay **“Loading gallery…”** | While the 3D canvas initializes after the hall opens (center area; header and controls stay visible) | +| Overlay **“Loading paintings…”** | While wall textures are still downloading in the 3D hall | +| Overlay **“Restoring gallery…”** | Briefly after WebGL context loss while the canvas remounts | + +View updates are **batched to one commit per animation frame** via `createViewChangeScheduler()` in `timelineView.ts` (`HomePage.tsx` → `handleViewChange`), so rapid scroll-wheel events do not flood React with separate renders. ### Timeline year labels Year ticks along the bottom of the era bar use **large, high-contrast** labels (bold cream text with shadow). The active range in the control row (e.g. `1400 CE — 1900 CE`) uses the same stronger styling. +Label density is **dynamic**: `chooseTimelineTickInterval()` in `timelineView.ts` picks a “nice” step (1, 2, 5, 10, … 5000 years) from the visible span and measured bar width so labels stay ~76 px apart. Zoomed-out overviews show fewer dates; zooming in reveals finer steps automatically. + ### Timeline controls | Input | Action | @@ -165,7 +249,7 @@ Each visible movement is drawn as a **portrait-width curved stream** (~54 px str | Vertical depth | Successor movements sit on rows below their deepest parent; sibling movements at the same depth are spread into lanes to limit overlap | | Branch connectors | Smooth curves from the **centre** of a parent stream to the **centre** of each child stream (siblings fan out along the parent’s length) | | Visual blending | Path-aligned SVG gradients with transparent fades at stream ends and branch junctions; streams draw on top of branches so overlap brightness stays uniform | -| Filtering | Same rule as the API: only movements with at least one artist active in the visible year range | +| Filtering | A movement is drawn when its **span overlaps** the visible year range **and** it has at least one catalogued artist — artists whose lifespan falls outside the window still keep their movement visible (their portraits simply do not render). Filtered client-side after initial load | | Viewport layout | Row height and stream width scale from measured canvas size so every visible movement row fits in the remaining screen space | ### Artists on movement streams @@ -184,18 +268,22 @@ Each artist appears as a **portrait circle** on their movement’s stream row: | Input | Action | |-------|--------| -| Scroll wheel on flow canvas | Zoom (same range as timeline) | +| Scroll wheel anywhere on flow canvas | Zoom (same year range as timeline; works over portraits and labels too) | | Drag on flow canvas | Pan | | Click portrait | Open artist biography | | Click **movement name** (label on stream) | Open **movement gallery** for that movement | -Artist portraits stop wheel/drag propagation so zooming over a face does not fight portrait clicks. Hovering a portrait highlights the artist’s lifespan on the era bar and brightens their segment on the movement stream. +**Zoom/pan performance:** The movement canvas uses a **capture-phase** wheel listener so scroll zoom works even when the cursor is over a portrait. While scrolling or dragging, a short **interaction mode** (~120 ms after the last input) draws simplified solid SVG strokes and temporarily hides portrait/label DOM so zoom stays responsive; full gradients and portraits return when you stop. + +Only **mousedown** on portraits and movement labels stops propagation (so drag-to-pan does not start when clicking them). Hovering a portrait highlights the artist’s lifespan on the era bar and brightens their segment on the movement stream. **Note:** Movement lineage is **frontend curation** for layout and labels — it is not stored in PostgreSQL. Painting influence links live in **`painting_influence_sources`** (paintings, artists, or movements as sources). The API reads that table for detail panels, hall navigation, and `has_influence_links`. The legacy **`painting_influences`** table is still written in parallel when curators add painting-to-painting edges but is not queried for display. ## Virtual gallery (3D halls) -The 3D scene supports two modes in `VirtualGallery.tsx`: **artist halls** (personal catalog) and **movement galleries** (full movement collection, chronological). +The 3D scene supports three modes in `VirtualGallery.tsx`: **artist halls** (personal catalog), **movement galleries** (full movement collection, chronological), and **guided tours** (curator-ordered stops — [tours.md](tours.md)). + +**Shared wall hang (all modes):** visit order fills the **left wall first**, then the **right**. The **first** work hangs near the entrance on the left (immediately left of the opening view); the **last** hangs near the entrance on the right. Artist halls use chronological order; movement wings use chronological order within each wing; tours use stop order. ### Artist halls @@ -204,10 +292,9 @@ Each artist has **exactly one hall**. The hall is a rectangular room sized to fi | Rule | Implementation | |------|----------------| | One hall per artist | `VirtualGallery.tsx` builds a single room from that artist’s paintings | -| Catalog depth | Most artists target **≥ 6** notable works via `npm run expand-catalog` and `famous-paintings-data.js`; some masters have larger museum dumps | -| Paintings on walls | Works hang on the **back, left, and right** walls in **one row per wall**; room **depth grows** when the catalog is large | -| Corridor layout | **15+ paintings:** short back wall (up to 8 works), remaining works on extended **left/right** side walls — a long gallery corridor | -| Wall order | On each wall, left → right: **later works on the left**, **earlier works on the right**; undated works sort toward the left | +| Catalog depth | Most artists target **≥ 6** notable works via `npm run dev:expand-catalog` and `famous-paintings-data.js`; some masters have larger museum dumps | +| Paintings on walls | Works hang on the **left and right** walls in **one row per wall**; room **depth grows** when the catalog is large (back wall is for the exit only) | +| Wall order | Chronological visit order: first half on the **left** (entrance → back), second half on the **right** (back → entrance); first work left of the opening view, last on the right | | Floor | Herringbone **parquet** procedural texture (`parquetFloorTexture.ts`) | | Wall tint | Gallery walls blend the artist’s **movement colour** into cream plaster tones | | Frame finish | **Unchecked** works: black moulding; **Reviewed** (`checkup_checked`): bright gold moulding at **double width** | @@ -232,7 +319,7 @@ Each artist has **exactly one hall**. The hall is a rectangular room sized to fi | Click painting | Open detail view | | Exit doorway / `E` / **Exit →** header button | Open path picker | -Predecessors and successors come from **`painting_influence_sources`** (painting and artist sources). Empty lists mean no influence edges are recorded yet for that artist — run `npm run update-influences` or extend `art-influences-data.js`. +Predecessors and successors come from **`painting_influence_sources`** (painting and artist sources). Empty lists mean no influence edges are recorded yet for that artist — run `npm run dev:update-influences` or extend `art-influences-data.js`. ### Movement galleries @@ -243,7 +330,7 @@ Enter from the home page by clicking a **movement name** on the movement flow (` | One gallery per movement | All paintings by artists in that movement, sorted chronologically | | Wings | Catalog split into wings of up to **55 works** (`movementHallLayout.ts`); large movements (e.g. Baroque) use multiple wings | | Paintings on walls | **Left and right walls only** — back wall reserved for exit, front for passage to the next wing | -| Wall order | Along each side wall: **later works on the left**, **earlier on the right** (same convention as artist halls) | +| Wall order | Same shared hang as artist halls: first half left (entrance → back), second half right (back → entrance) | | Frame captions | **Year · artist** label below each frame | | Period interior | Each of the 26 seeded movements maps to a unique style in `movement-interior-styles.ts` (Italian palazzo, Baroque palace, NYC loft, white cube, etc.) | | Wall materials | Hi-res **procedural textures** with normal maps (`galleryProceduralTextures.ts`): real-world stone, marble, wood panelling, brick, velvet, stucco — plus **single-colour painted walls** (`painted-lime`, `painted-oil-matte`, `painted-oil-satin`, `painted-emulsion`, `painted-flat`) tinted per movement for Renaissance salons through modern white cubes | @@ -254,7 +341,7 @@ Enter from the home page by clicking a **movement name** on the movement flow (` | Front wall | Open **“Next wing →”** archway when a later wing exists; walk through or press `E` when near | | Influence lamps | Same golden upside-down lamps as artist halls when `has_influence_links` is true | | Missing images | Draped canvas cover in frame | -| Detail return | Hall stays mounted; camera preserved on **Back to Timeline** / **Back to Gallery** | +| Detail return | **Back to Gallery** from painting detail returns to the same wing with camera preserved; **Back to Timeline** exits the hall entirely | **Controls (movement gallery):** @@ -267,17 +354,34 @@ Enter from the home page by clicking a **movement name** on the movement flow (` Movement galleries do **not** use the predecessor/successor influence picker — that remains artist-hall only. +### Guided tour halls + +Enter from the home page **Tours** popup (`GET /api/tours/:id`). Layout reuses the movement winged hall (`mode: 'tour'` in `VirtualGallery.tsx`). + +| Rule | Implementation | +|------|----------------| +| Visit order | Curator `sort_order` on `tour_stops` (not chronological) | +| Wings | Same ~55-per-wing split as movements; order preserved across wings | +| Wall hang | Same left-then-right rule as other halls | +| Detail | Tour stop text panel; ‹ › walks tour stops | +| Exit | Wing navigator / **Exit to Timeline** (no influence picker) | + +Full guide: [tours.md](tours.md). + ### Shared 3D behaviour -**3D images** use locally cached files only (`galleryImageUrl` in `client/src/api/client.ts`). Remote fetches are too slow for realtime WebGL textures; call `POST /api/artists/:id/preload-images` before entering an **artist** hall to link disk files. Movement galleries load painting lists from the API without a separate preload step. While a texture is loading, the frame shows the canvas cover instead of a white placeholder. +**3D images** use locally cached files only (`galleryImageUrl` in `client/src/api/client.ts`). Remote fetches are too slow for realtime WebGL textures; the client calls `POST /api/artists/:id/preload-images` automatically when entering an **artist** hall (public route — links disk files only). Movement galleries load painting lists from the API without a separate preload step. While a texture is loading, the frame shows the canvas cover instead of a white placeholder. The center of the hall shows **“Loading gallery…”** until the WebGL canvas is ready, then **“Loading paintings…”** until every wall texture has resolved (tracked through `GalleryTextureLoadContext`). The 3D hall stays mounted while painting detail or bio overlays are open; returning remounts the canvas when the hall becomes active again. + +**WebGL context-loss recovery:** on some GPUs/drivers (notably certain Chrome setups) the browser can drop the WebGL context right after entering a hall, which would otherwise leave a permanent dark window. `VirtualGallery.tsx` listens for `webglcontextlost` / `webglcontextrestored`, calls `preventDefault()` so the browser can restore the context, and remounts the `` with a fresh context (a **“Restoring gallery…”** overlay shows briefly). The network-loaded HDR `Environment` map is wrapped in an error boundary so, if it fails to load, the hall still renders without reflections instead of unmounting the whole scene. ## Painting detail view -Opened from the 3D hall (artist or movement wing — click a frame) or from influence thumbnails on another work’s detail page. +Opened from the 3D hall (artist, movement, or tour wing — click a frame) or from influence thumbnails on another work’s detail page. | Layer | What you see | |-------|----------------| | **Detail** | Centre image, *Influenced By* (left) and *Influenced* (right) — painting thumbnails (full work visible, letterboxed), artist portraits, or movement swatches — position in catalog (e.g. `3 of 12`) | +| **Tour notes** | When opened from a guided tour: stop text panel under the image (English body from `tour_stops`) | | **Art history notes** | Numbered markers on the image (when positioned) plus a note list below — short citations from Gombrich, museum catalogs, Wikipedia, etc. (`painting_annotations` table) | | **Fullscreen** | Click the centre image; `Escape` or click anywhere to return to detail only | @@ -285,47 +389,75 @@ Opened from the 3D hall (artist or movement wing — click a frame) or from infl | Input | Action | |-------|--------| -| `‹` / `›` beside image | Previous / next work by the **same artist** (chronological order) | +| `‹` / `›` beside image | Previous / next in the **current catalog** — artist chronology, movement chronology, or **tour stop order** | | `←` / `→` | Same as prev / next (disabled while fullscreen is open) | | Click centre image | Open fullscreen lightbox | | Click influence thumbnail | Open that work’s detail (different artist allowed) | | Click influence artist portrait | Open that artist’s 3D gallery hall | -| **← Back to Gallery** / **← Back to Timeline** | Return to the hall or movement wing you entered from — **3D camera position is preserved** | +| **← Back to Gallery** | Return to the hall or movement wing you entered from — **3D camera position is preserved** | +| **← Back to Timeline** | Return to the home timeline (from search result, or from the 3D gallery header / movement **Exit to Timeline**) — hall unmounts, timeline zoom resets | | **About {artist}** | Open artist biography | **Navigation rules:** -- **Catalog browsing** (‹ › / arrow keys) walks the current artist’s works earliest → latest. It does **not** change the back target: after browsing several works, **Back to Gallery** still returns directly to the hall. +- **Catalog browsing** (‹ › / arrow keys) walks the active catalog (artist / movement chronology, or tour stops). It does **not** change the back target: after browsing several works, **Back to Gallery** still returns directly to the hall. - **Influence links** push a new detail layer; **Back** from an influenced work returns to the painting you came from (and from there back to the gallery if applicable). - Side-panel influence images use **`object-fit: contain`** so tall or wide works are not cropped (dark letterbox background). - The 3D hall stays mounted in the background while detail is open so nothing is lost on return. ### Debug mode (developer) -When **Debug mode** is enabled from the home header, painting detail and artist biography show a bottom-left panel with image search preview and action buttons. See [Developer tools (image audit)](#developer-tools-image-audit). +**Curator login required.** Debug tools are hidden until you sign in from the home header (**Curator login**). After login, enable **Debug mode** from the same header area. + +When debug mode is on, painting detail and artist biography show a bottom-left panel with image search preview and action buttons. See [Developer tools (image audit)](#developer-tools-image-audit). Next to the toggle, **Show more** (checkbox, persisted in `localStorage`) opens the **More** search-results modal automatically whenever you open a painting or artist bio while debug mode is on. ## Key design decisions - **Timeline bounds** derive from the earliest art movement start year, not ancient-era metadata alone, so the default view opens where catalogued content begins. -- **Movement filtering** on zoom only shows movements that have at least one artist active in the visible year range. +- **Timeline catalog** loads once from the API; pan/zoom is client-side only, with per-frame batching via `createViewChangeScheduler()`. +- **Movement flow interaction** uses simplified SVG and hides portrait DOM during active scroll/drag so zoom stays responsive over dense portrait fields. +- **Movement filtering** on zoom shows movements whose span overlaps the visible year range and that have at least one catalogued artist; a movement (e.g. Byzantine or Gothic viewed at 311–1231 CE) still appears even when all its artists lived outside the current window. - **Movement lineage** (`movement-lineage.ts`) documents art-historical predecessor→successor links for the flow diagram; extend that file to add or correct branches. - **One hall per artist** keeps navigation predictable: enter from the timeline or bio, leave via the single exit or back button. - **Movement galleries** complement artist halls: full movement corpus in period-themed wings, entered from the flow diagram. +- **Guided tours** add curator-ordered winged halls with stop text on painting detail — [tours.md](tours.md). +- **Wall hang** is shared: first work on the left at the entrance, last on the right. - **Influence-based hall links** connect artists through documented painting relationships, grouped by movement at the exit. -- **3D gallery images** use locally cached files only; slow remote fetches would break realtime rendering. +- **3D gallery images** use locally cached files only; slow remote fetches would break realtime rendering. The client calls `POST /api/artists/:id/preload-images` automatically when entering an **artist** hall (public route — links disk files only). +- **3D gallery session** stays mounted while painting detail or bio overlays are open; returning to the hall remounts the WebGL canvas when it becomes active again. +- **3D gallery resilience:** a lost WebGL context is recovered by remounting the canvas with a fresh context (rather than showing a dark window), and the HDR environment map is isolated behind an error boundary so its failure never blanks the scene. +- **Loading feedback:** `GalleryLoadingMarker` surfaces catalog load, portrait download, gallery entry, painting-texture load, and context-restore states so the user always knows work is still in progress. - **Influence data** is stored in **`painting_influence_sources`** (directed links from paintings to source paintings, artists, or movements), with optional period fields and citation metadata. Sources include curated scholarship (`art-influences-data.js`) and **PainterPalette** (`discovered_via = painter-palette`). Legacy `painting_influences` mirrors painting-to-painting edges for scripts only. +## User roles and access + +| Role | Who | Can do | +|------|-----|--------| +| **`user`** | Anonymous visitor (default) | Browse timeline, movement flow, 3D artist/movement halls, painting detail, artist bios, images | +| **`curator`** | Named account (`users` table) | Everything above + **Debug mode**, **Checkup**, **Translations**, **Influences**, **Tour editor**, debug API mutations | + +Curators sign in via **Curator login** in the site header. Sessions use an HTTP-only cookie (`gallery.sid`). The UI hides debug controls from guests; the server enforces the same rules on debug/checkup API routes (`401` without a valid session). + +Mutating debug actions (fix/clear/upload/delete, checkup flag changes) are appended to **`curator_audit_log`** with username, action, target id, optional JSON details, and client IP. Query in pgAdmin — see [DB_structure.md](DB_structure.md#curator_audit_log). + ## Developer tools (image audit) -Optional workflow for curating local image files — not part of the public visitor experience. +Curator-only workflow for reviewing and fixing local image files — not part of the public visitor experience. | Feature | Where | Purpose | |---------|--------|---------| -| **Debug mode** | Home header toggle (`client/src/utils/debugMode.ts`) | Persists in `localStorage`; enables debug panel on painting detail and artist bio | -| **Show more** | Home header checkbox (same util) | When debug mode is on, auto-opens the **More** modal on each painting / bio page load | -| **Checkup page** | Home header → **Checkup** (`CheckupPage.tsx`) | Full-catalog table: gallery vs detail thumbnails, search, fix, review flags | +| **Catalog search** | Timeline header (all visitors) | Find artists, paintings, movements; `GET /api/search`; navigate to gallery or detail | +| **Curator login** | Home header (guests) | Username + password modal; unlocks debug tools | +| **Debug mode** | Home header toggle (curators only) | Persists in `localStorage`; enables debug panel on painting detail and artist bio | +| **Show more** | Home header checkbox (curators, when debug on) | Auto-opens the **More** modal on each painting / bio page load | +| **Checkup page** | Home header → **Checkup** (curators only) | Full-catalog table: gallery vs detail thumbnails, search, fix, review flags | +| **Translations** | Home header → **Translations** (curators only) | Review/publish Russian `entity_translations` | +| **Influences** | Home header → **Influences** (curators only) | List/CRUD influence edges, CSV/JSON/XLSX import wizard, neighborhood graph — [influence-import.md](influence-import.md) | +| **Tour editor** | Home header → **Tour editor** (curators only) | Create/publish guided tours and stop text — [tours.md](tours.md) | +| **Tours** | Home header → **Tours** (everyone) | Open published tours in a 3D hall — [tours.md](tours.md) | +| **Logout** | Home header (curators) | Ends session; hides debug tools | | **Debug panel** | Painting detail or artist bio (bottom-left, when debug mode on) | Search preview + action buttons (six on painting detail, five on artist bio) | ### Debug panel (painting detail and artist bio) @@ -335,13 +467,15 @@ When debug mode is on, a panel at the bottom-left shows the image search query, | Button | Painting detail | Artist bio | |--------|-----------------|------------| | **Checked** | Sets `checkup_checked` via `PATCH …/checkup-flags` | Same for artist portrait flags | -| **Fix it** | Replaces local image from top search result | Replaces portrait | -| **More** | Modal with up to **20** results (resolution shown when known) | Same | +| **Fix it** | Replaces full image from search result; **regenerates painting thumb** (~400px JPEG) from that file | Replaces portrait; **regenerates timeline thumb** (256px) | +| **More** | Modal with up to **20** results (resolution shown when known); thumb regenerated from chosen full image | Same | | **Clear** | Deletes files, clears DB paths, empty frame | Clears portrait slot | -| **Upload** | Local file picker → disk + thumbnail | Local file → portrait | +| **Upload** | Local file picker (`DebugUploadButton`) → full image + **auto-generated painting thumb**; full-page **Loading…** overlay; hides current image and pauses search/fix until upload finishes | Local file → portrait + **auto-generated portrait thumb**; same upload overlay behaviour | | **Remove entry** | **Painting detail only** — deletes row from DB, removes image files, refreshes 3D gallery, navigates to next/previous work in catalog (or back to gallery if last work). No confirmation dialog. | — | -After **Fix it**, **More**, **Upload**, or **Clear**, the main view, gallery textures (paintings), and timeline portrait (artists) update without a full page reload. **Remove entry** refetches artist (and movement gallery when relevant) from the API and remounts the 3D hall so the deleted frame disappears immediately. +After **Fix it**, **More**, **Upload**, or **Clear**, the main view, gallery textures (paintings), and timeline portrait (artists) update without a full page reload. Painting and portrait thumbs under `data/images/*/thumbs/` are rebuilt on the server whenever a curator replaces the full image. **Remove entry** refetches artist (and movement gallery when relevant) from the API and remounts the 3D hall so the deleted frame disappears immediately. + +Pressing **Upload** clears the debug search preview and closes **More** before the file picker opens. While uploading, **Fix it**, **More**, and **Checked** are disabled and the main painting/portrait is hidden behind a centered loading overlay. Reviewed portraits show a gold border on the bio page; reviewed paintings use gold frames in the 3D hall. **Back to Gallery** returns to the live hall session, not a stale snapshot. @@ -353,15 +487,22 @@ Influence side-panel thumbnails use **letterboxing** (`object-fit: contain`) so **Search visible** runs image search only for rows currently shown after text/filter — not automatically on page load. Fixing an image sets **Fixed** and **Reviewed**. -Run `npm run migrate:checkup-flags`, `npm run migrate:artist-checkup-flags`, and `npm run migrate:painting-annotations` once on existing databases. Load notes with `npm run update-painting-annotations` (add `--wikipedia` for overview lines from Wikipedia intro text). After server code changes, restart `npm run start` (or `npm run dev:server`) so new routes (e.g. clear, upload, delete painting, portrait debug, annotations) are registered. JSON body limit for uploads is **20 MB** (`express.json` in `server/index.js`); individual files are capped at **15 MB** after decode. +Run `npm run dev:migrate:checkup-flags`, `npm run dev:migrate:artist-checkup-flags`, and `npm run dev:migrate:painting-annotations` once on existing databases. Load notes with `npm run dev:update-painting-annotations` (add `--wikipedia` for overview lines from Wikipedia intro text). After server code changes, restart `npm run dev:start` (or `npm run dev:server`) so new routes are registered. JSON body limit for uploads is **20 MB** (`express.json` in `server/index.js`); individual files are capped at **15 MB** after decode. -See [API.md](API.md#developer-image-audit) and [data-and-images.md](data-and-images.md#duplicate-paintings). +Server-side auth lives in `server/middleware/session.js`, `server/middleware/auth.js`, `server/routes/auth.js`, and `server/audit-log.js`. Client auth context: `client/src/context/AuthContext.tsx`. + +See [API.md](API.md#authentication) and [data-and-images.md](data-and-images.md#duplicate-paintings). ## Related docs | Document | Contents | |----------|----------| | [setup.md](setup.md) | Install, database, npm scripts | +| [deploy-dev-to-prod.md](deploy-dev-to-prod.md) | Release runbook + one-command `devtoprod:release` | +| [harmonize-dev-prod.md](harmonize-dev-prod.md) | Incremental dev ↔ prod merge (catalog DB + images) | | [DB_structure.md](DB_structure.md) | Tables and relationships | | [API.md](API.md) | REST endpoints | +| [influence-import.md](influence-import.md) | Curator Influences tool — import wizard, CRUD, graph | +| [tours.md](tours.md) | Guided tours — editor, public popup, 3D tour hall | +| [i18n-russian.md](i18n-russian.md) | Russian UI + entity_translations | | [data-and-images.md](data-and-images.md) | Image pipeline and seeding | diff --git a/Documentation/data-and-images.md b/Documentation/data-and-images.md index 5d45c47..5b77436 100644 --- a/Documentation/data-and-images.md +++ b/Documentation/data-and-images.md @@ -12,8 +12,10 @@ How catalog content, biographies, and artwork files enter the system. ```text data/images/ -├── portraits/ # Artist headshots -│ └── Claude_Monet.jpg +├── portraits/ # Artist headshots (display ~900px wide) +│ ├── Claude_Monet.jpg +│ └── thumbs/ # Timeline thumbnails (~256px) +│ └── Claude_Monet_thumb.jpg └── paintings/ ├── Claude_Monet_Water_Lilies.jpg └── thumbs/ @@ -22,43 +24,62 @@ data/images/ File names are sanitised `{Artist}_{Title}.{ext}`. The image service can rediscover files on disk even when DB paths are empty (`server/image-service.js` → `syncPaintingFromDisk`). +### Dev vs production image storage + +| Environment | Path on disk | Sync | +|-------------|--------------|------| +| **Development** | `./data/images/` in repo | Working copy on dev PC | +| **Production** | `/mnt/BasePool/Applications/Gallery/data/images` on TrueNAS | SMB `\\192.168.10.122\Gallery\data\images` | + +**One-direction promote:** `npm run devtoprod:images` (dev → prod, skip older). **Refresh dev from prod:** `npm run prodto:dev:images`. **Bidirectional merge** (newer file wins): `npm run harmonize:images` or full `npm run harmonize` — see [harmonize-dev-prod.md](harmonize-dev-prod.md). General sync reference: [environments.md](environments.md). + ## Scripts overview | Script | npm command | Role | |--------|-------------|------| -| `seed-wikipedia.js` | `npm run seed` | Initial eras, movements, artists, paintings, influences | -| `fetch-artist-bios.js` | `npm run fetch-artist-bios` | Wikipedia intros → `bio_short` / `bio_full` | +| `seed-wikipedia.js` | `npm run dev:seed` | Initial eras, movements, artists, one flagship painting per artist | +| `seed-catalog-data.js` | *(data only)* | Eras, movements, artist metadata consumed by seed | +| `sync-image-paths.js` | `npm run dev:sync-image-paths` | Import painting rows from disk; set `image_path` / `thumbnail_path` | +| `fetch-artist-images.js` | `npm run dev:fetch-artist-images` | Download or link artist portraits under `data/images/portraits/` | +| `fetch-artist-bios.js` | `npm run dev:fetch-artist-bios` | Wikipedia intros → `bio_short` / `bio_full` | | `famous-paintings-data.js` | *(data only)* | Curated list of notable works per artist | -| `expand-paintings.js` | `npm run expand-catalog` | Inserts works from data file for thin catalogs | -| `art-influences-data.js` | *(data only)* | Curated painting-to-painting influence edges | -| `update-influences.js` | `npm run update-influences` | Applies influence graph; creates missing artists/works | -| `fetch-missing-images.js` | `npm run fetch-images` | Downloads files for paintings missing on disk | +| `expand-paintings.js` | `npm run dev:expand-catalog` | Inserts works from data file for thin catalogs | +| `art-influences-data.js` | *(data only)* | Curated influence edges (painting / artist / movement) | +| `update-influences.js` | `npm run dev:update-influences` | Applies influence graph; creates missing artists/works | +| `fetch-missing-images.js` | `npm run dev:fetch-images` | Downloads files for paintings missing on disk | | `image-fetcher.js` | *(library)* | Wikimedia / museum resolution used by fetch scripts and API | -| `regenerate-thumbnails.js` | `npm run regenerate-thumbnails` | Rebuild thumbs from full images via `sharp` | -| `audit-painting-images.js` | `npm run audit-painting-images` | Detect thumb/full aspect-ratio mismatches | -| `find-duplicate-paintings.js` | `npm run find-duplicates` | Report exact and near-duplicate catalog rows | -| `migrate-checkup-flags.js` | `npm run migrate:checkup-flags` | Add `checkup_checked` / `checkup_fixed` columns | +| `sync-images-to-prod.ps1` / `sync-images-from-prod.ps1` | `npm run devtoprod:images` / `npm run prodto:dev:images` | Robocopy via SMB `\\192.168.10.122\Gallery` | +| `harmonize-db.js` / `harmonize-images.js` | `npm run harmonize:db` / `harmonize:images` | Bidirectional merge by `updated_at` / file mtime | +| `harmonize.ps1` | `npm run harmonize` | Orchestrator: backups, optional schema, DB + image merge | +| `regenerate-thumbnails.js` | `npm run dev:regenerate-thumbnails` | Rebuild painting thumbs from full images via `sharp` | +| `regenerate-portrait-thumbs.js` | `npm run dev:regenerate-portrait-thumbs` | Rebuild timeline portrait thumbs (~256px) and set `portrait_thumb_path` | +| `audit-painting-images.js` | `npm run dev:audit-painting-images` | Detect thumb/full aspect-ratio mismatches | +| `find-duplicate-paintings.js` | `npm run dev:find-duplicates` | Report exact and near-duplicate catalog rows | +| `migrate-checkup-flags.js` | `npm run dev:migrate:checkup-flags` | Add `checkup_checked` / `checkup_fixed` columns | ## Typical workflow ```text -migrate → seed → fetch-artist-bios → expand-catalog → update-influences → fetch-images (per artist or batch) → build client +migrate → seed → sync-image-paths → fetch-artist-images → fetch-artist-bios → expand-catalog → update-influences → fetch-images (per artist or batch) → build client ``` -1. **Seed** creates the base catalog (often one flagship painting per modern artist). -2. **fetch-artist-bios** fills biography fields for every artist with a `wikipedia_title`. -3. **expand-catalog** brings each artist up to at least **6** notable works (configurable via `MIN_PAINTINGS`). -4. **fetch-images** downloads artwork files; the 3D gallery needs local files for reliable textures. +1. **Seed** creates the base catalog (one flagship painting per artist; ~100 artists). +2. **sync-image-paths** imports additional paintings when `data/images/paintings/` already contains files from a full clone (filename pattern `{Artist}_{Title}.jpg`). +3. **fetch-artist-images** sets `portrait_path` from local files or Wikipedia. +4. **fetch-artist-bios** fills biography fields for every artist with a `wikipedia_title`. +5. **expand-catalog** brings each artist up to at least **6** notable works (configurable via `MIN_PAINTINGS`). +6. **update-influences** loads the influence graph (*Influenced By* / *Influenced* panels, 3D hall lamps, exit navigation). +7. **fetch-images** downloads artwork files still missing on disk; the 3D gallery needs local files for reliable textures. ## Seeding pipeline -`npm run seed` runs `scripts/seed-wikipedia.js`, which: +`npm run dev:seed` runs `scripts/seed-wikipedia.js`, which: -1. Inserts **historical eras** and **art movements** (curated date ranges and colours). +1. Inserts **historical eras** and **art movements** (curated date ranges and colours from `seed-catalog-data.js`). 2. For each curated **artist**: - - Creates **artist periods** and **paintings**. - - May download portraits and painting images (depending on seed script version). -3. Writes **painting_influences** edges from curated scholarship references (mirrored into `painting_influence_sources` when you run `npm run migrate:influence-sources` and `npm run update-influences`). + - Creates **artist periods** and one **flagship painting**. + - May download portraits and painting images when run with `--fetch-images`. +3. Does **not** insert influence edges — run `npm run dev:update-influences` after seed (see [Painting influence graph](#painting-influence-graph)). Those influence edges power **3D hall navigation** and painting detail panels via **`painting_influence_sources`** (see `GET /api/artists/:id/navigation` and `GET /api/paintings/:id` in [API.md](API.md)). @@ -66,7 +87,7 @@ Artists are grouped by movement and century; the seed list targets at most ~100 ## Artist biographies -`npm run fetch-artist-bios` reads each artist’s `wikipedia_title`, fetches the English Wikipedia **lead section**, and stores: +`npm run dev:fetch-artist-bios` reads each artist’s `wikipedia_title`, fetches the English Wikipedia **lead section**, and stores: | Field | Content | |-------|---------| @@ -91,15 +112,15 @@ The bio page (`ArtistBio.tsx`) shows lifespan, movement, summary, full text, and ## Expanding thin catalogs -Many seed artists arrive with only one famous painting. `npm run expand-catalog` runs `scripts/expand-paintings.js`, which: +Many seed artists arrive with only one famous painting. `npm run dev:expand-catalog` runs `scripts/expand-paintings.js`, which: 1. Finds artists with fewer than `MIN_PAINTINGS` (default **6**). 2. Inserts rows from `scripts/famous-paintings-data.js` that are not already present (normalized title matching skips duplicates). 3. Sets `wikipedia_title` on each new painting for image resolution. ```bash -npm run expand-catalog # DB rows only -npm run expand-catalog -- --fetch-images # also download images (very slow) +npm run dev:expand-catalog # DB rows only +npm run dev:expand-catalog -- --fetch-images # also download images (very slow) ``` To add more works, append entries to `famous-paintings-data.js`: @@ -111,13 +132,52 @@ To add more works, append entries to `famous-paintings-data.js`: `wikipedia_title` is optional; it defaults to `title`. Use it when the Wikipedia article name differs from the display title. -Renaissance and medieval masters with large museum catalog dumps (e.g. Raphael, Dürer) are usually above the minimum already; expansion targets Impressionists, modernists, and other artists who had only a single seed painting. +Renaissance and medieval masters with large museum catalog dumps (e.g. Raphael, Dürer) are usually above the minimum already when **`sync-image-paths`** has imported files from disk; expansion targets Impressionists, modernists, and other artists who had only a single seed painting. + +## Importing paintings from disk + +When the repository includes a full `data/images/paintings/` tree but the database was seeded fresh (one row per artist), run: + +```bash +npm run dev:sync-image-paths +``` + +`scripts/sync-image-paths.js`: + +1. Scans `data/images/paintings/` for full-size files (not `thumbs/`). +2. Matches filenames to artists using the same `{Artist}_{Title}` sanitisation as `server/image-service.js`. +3. **Updates** `image_path` / `thumbnail_path` on existing rows when files are found. +4. **Inserts** missing painting rows for files not yet in the catalog. + +Flags: + +- `--dry-run` — report counts only, no DB writes. + +Safe to re-run; already-imported works are skipped by normalized title matching. + +Typical result on a full clone: ~1,000+ paintings linked from ~1,000 on-disk files. + +## Artist portraits + +Timeline movement flow loads **`portrait_thumb_path`** (~256px JPEG under `portraits/thumbs/{Artist}_thumb.jpg`) when available; biography and 3D exit navigation use full `portrait_path`. After adding portraits, run `npm run dev:regenerate-portrait-thumbs` to backfill thumbs on dev. + +`npm run dev:fetch-artist-images` runs `scripts/fetch-artist-images.js`: + +1. For each artist, checks `data/images/portraits/{Artist}.jpg` (or other extensions) and sets `portrait_path` when a local file exists. +2. Otherwise downloads from Wikipedia / search fallbacks via `image-fetcher.js`. + +Flags: + +- `--force` — re-fetch even when `portrait_path` is already set. +- `--limit=N` — process only the first N artists needing portraits. + +Run after seed when portrait files exist on disk but the DB still has null `portrait_path` values. ## Painting influence graph Directed influence links are stored in **`painting_influence_sources`**. Each row connects a painting to a **source** of type `painting`, `artist`, or `movement`, with optional period context (e.g. influence during the work’s creation year). -The legacy **`painting_influences`** table (painting-to-painting only) is still written alongside sources when running `npm run update-influences` — it keeps script compatibility and matches the backfill migration. **The API reads only `painting_influence_sources`**, so each edge appears once in the UI. +The legacy **`painting_influences`** table (painting-to-painting only) is still written alongside sources when running `npm run dev:update-influences` — it keeps script compatibility and matches the backfill migration. **The API reads only `painting_influence_sources`**, so each edge appears once in the UI. Influence data drives: @@ -130,7 +190,7 @@ Influence data drives: Painting-to-painting edges exist in both tables by design. To confirm the database has no stray duplicates and that the API model is clean: ```bash -npm run audit-influence-duplicates +npm run dev:audit-influence-duplicates ``` Reports: edges present in both tables, duplicate rows within either table (should be 0), and legacy-only / sources-only mismatches. If *Influenced* ever shows the same successor twice, restart the server after pulling API fixes — responses must not union legacy and sources tables. @@ -138,19 +198,19 @@ Reports: edges present in both tables, duplicate rows within either table (shoul ### One-time migration ```bash -npm run migrate:influence-sources # create table + backfill legacy painting edges +npm run dev:migrate:influence-sources # create table + backfill legacy painting edges ``` ### Curated updates -`npm run update-influences` runs `scripts/update-influences.js` against `scripts/art-influences-data.js`: +`npm run dev:update-influences` runs `scripts/update-influences.js` against `scripts/art-influences-data.js`: ```bash -npm run update-influences # insert curated edges -npm run update-influences -- --fetch-images # also download images for newly created works -npm run update-influences -- --discover # curated + web discovery pass -npm run discover-influences # discovery only (no curated file pass) -npm run update-influences -- --discover --limit=20 # cap discovery to N works +npm run dev:update-influences # insert curated edges +npm run dev:update-influences -- --fetch-images # also download images for newly created works +npm run dev:update-influences -- --discover # curated + web discovery pass +npm run dev:discover-influences # discovery only (no curated file pass) +npm run dev:update-influences -- --discover --limit=20 # cap discovery to N works ``` Each entry defines a later `work` and one or more `influencedBy` sources. Legacy single-object form is still supported: @@ -205,7 +265,7 @@ To add or fix a branch, edit `MOVEMENT_LINEAGE` in that file and rebuild the cli Separate from movement lineage layout, `client/src/data/movement-interior-styles.ts` defines a **unique 3D interior** for each seeded art movement (26 styles): wall/floor/ceiling textures, trim colours, window style, and architectural details (columns, coffered ceilings, etc.). Textures are generated procedurally in `client/src/utils/galleryProceduralTextures.ts`. -Wing layout (up to 55 works per wing, side-wall-only hang, window gap placement) lives in `client/src/utils/movementHallLayout.ts`. To change a movement’s look, edit its entry in `movement-interior-styles.ts` and rebuild the client. +Wing layout (up to 55 works per wing, side-wall-only hang, window gap placement) lives in `client/src/utils/movementHallLayout.ts`. Visit order fills the **left wall** (first work at the entrance), then the **right** (last work at the entrance). The same hang applies to artist halls and guided tours. To change a movement’s look, edit its entry in `movement-interior-styles.ts` and rebuild the client. ## Historical event markers (frontend timeline) @@ -216,6 +276,8 @@ Wing layout (up to 55 works per wing, side-wall-only hang, window gap placement) | Storage | TypeScript module in the client — **not** a database table | | Format | `{ id, name, startYear, endYear?, shortLabel? }` — omit `endYear` for a single-year pin | | Interaction | Click a marker to zoom the shared timeline/movement view to that period | +| Year axis labels | Dynamic density in `Timeline.tsx` via `chooseTimelineTickInterval()` — fewer labels when zoomed out | +| Pan/zoom batching | `createViewChangeScheduler()` in `timelineView.ts` — one React update per animation frame | | Vertical guides | `TimelineEventGuides.tsx` draws faint gold lines (or shaded spans) from the marker row down through the movement flow, aligned to the same year scale | Edit `HISTORICAL_EVENTS` and rebuild the client to extend the set. @@ -230,14 +292,14 @@ Short curator-style notes on the painting detail page — separate from the infl | UI | `PaintingAnnotations.tsx` — numbered markers on the image (when `pos_x` / `pos_y` set) plus an “Art history notes” list | | API | Included as `annotations[]` on `GET /api/paintings/:id` | | Curated data | `scripts/painting-annotations-data.js` — artist/title keys matched via `influence-resolver.js` | -| Load | `npm run update-painting-annotations` (replaces existing rows per painting by default) | -| Wikipedia pass | `npm run update-painting-annotations -- --wikipedia` — one intro sentence per work from `wikipedia_title`; use `--wiki-delay=3000` if rate-limited; `--no-replace` to append without clearing curated rows | +| Load | `npm run dev:update-painting-annotations` (replaces existing rows per painting by default) | +| Wikipedia pass | `npm run dev:update-painting-annotations -- --wikipedia` — one intro sentence per work from `wikipedia_title`; use `--wiki-delay=3000` if rate-limited; `--no-replace` to append without clearing curated rows | Categories include `subject`, `technique`, `context`, and `symbolism`. Sources cite Gombrich, museum catalogs, and Wikipedia as appropriate. ## Batch image fetch -`npm run fetch-images` (alias: `npm run search-missing-paintings`) runs `scripts/fetch-missing-images.js`. It searches multiple sources for paintings without local files: +`npm run dev:fetch-images` (alias: `npm run dev:search-missing-paintings`) runs `scripts/fetch-missing-images.js`. It searches multiple sources for paintings without local files: | Source | Notes | |--------|--------| @@ -255,13 +317,13 @@ Categories include `subject`, `technique`, `context`, and `symbolism`. Sources c | Harvard Art Museums | Optional (`HARVARD_ART_API_KEY` in `.env`) | ```bash -npm run fetch-images # all missing, catalog order (~hours) -npm run fetch-images -- --limit=50 # random sample of 50; 10s max per painting -npm run fetch-images -- --limit=250 # random sample up to N (caps at current missing count) -npm run fetch-images -- --limit=50 --max-wait=120 # slower, more thorough lookup per painting -npm run fetch-images -- --artist="Albrecht Dürer" # one artist, catalog order -npm run fetch-images -- --discover-only --limit=20 # fix wikipedia_title only -npm run fetch-images -- --web-search-only --limit=50 # DuckDuckGo + Commons + multilingual Wikipedia +npm run dev:fetch-images # all missing, catalog order (~hours) +npm run dev:fetch-images -- --limit=50 # random sample of 50; 10s max per painting +npm run dev:fetch-images -- --limit=250 # random sample up to N (caps at current missing count) +npm run dev:fetch-images -- --limit=50 --max-wait=120 # slower, more thorough lookup per painting +npm run dev:fetch-images -- --artist="Albrecht Dürer" # one artist, catalog order +npm run dev:fetch-images -- --discover-only --limit=20 # fix wikipedia_title only +npm run dev:fetch-images -- --web-search-only --limit=50 # DuckDuckGo + Commons + multilingual Wikipedia ``` Each run prints **`Missing local files: N`** at startup — that is the current count of catalogued paintings with no full-size or thumbnail file under `data/images/`. A painting counts as present if **either** file exists on disk. @@ -297,7 +359,7 @@ Requests are deduplicated (`inflight` map) and timeout after 15 seconds. On-dema ## Preload before 3D gallery -`POST /api/artists/:id/preload-images` runs **local-only** linking — no network. Call this when entering an **artist’s** 3D hall so textures use files already on disk. +`POST /api/artists/:id/preload-images` is a **public** route (no curator login). It runs **local-only** linking — no network. The React client calls it automatically when entering an **artist’s** 3D hall so textures use files already on disk. **Movement galleries** (`GET /api/movements/:id/gallery`) do not use preload — they load the full painting list from the API and resolve local paths the same way as artist halls. Works without files still show the canvas cover in the frame. @@ -317,10 +379,10 @@ These live in `client/public/` (and `client/dist/` after build). 1. Insert rows into `artists`, `artist_periods`, `paintings` (or extend the seed script). 2. Place image files under `data/images/` using the naming convention. -3. Run `npm run fetch-artist-bios` for the new artist’s biography. -4. Add entries to `famous-paintings-data.js` and run `npm run expand-catalog` if needed. -5. Run `npm run fetch-images -- --artist="…"` or rely on preload / on-demand sync. -6. Add influence rows via `npm run update-influences` / `art-influences-data.js` (writes both `painting_influence_sources` and legacy painting edges). +3. Run `npm run dev:fetch-artist-bios` for the new artist’s biography. +4. Add entries to `famous-paintings-data.js` and run `npm run dev:expand-catalog` if needed. +5. Run `npm run dev:fetch-images -- --artist="…"` or rely on preload / on-demand sync. +6. Add influence rows via `npm run dev:update-influences` / `art-influences-data.js` (writes both `painting_influence_sources` and legacy painting edges). ## PainterPalette external dataset @@ -329,8 +391,8 @@ These live in `client/public/` (and `client/dist/` after build). ### One-time setup ```bash -npm run migrate:artist-palette # adds artists.palette_metadata JSONB -npm run import-painter-palette # enrich + influence links +npm run dev:migrate:artist-palette # adds artists.palette_metadata JSONB +npm run dev:import-painter-palette # enrich + influence links ``` ### What gets imported @@ -349,10 +411,10 @@ Name matching uses normalized strings plus aliases in `scripts/painter-palette-l ### Commands ```bash -npm run analyze-painter-palette # match report -npm run import-painter-palette -- --dry-run -npm run import-painter-palette -- --metadata-only -npm run import-painter-palette -- --influences-only +npm run dev:analyze-painter-palette # match report +npm run dev:import-painter-palette -- --dry-run +npm run dev:import-painter-palette -- --metadata-only +npm run dev:import-painter-palette -- --influences-only ``` Re-run `import-painter-palette` after adding gallery artists or updating the CSV; existing palette influence rows are skipped if already present. @@ -362,7 +424,7 @@ Re-run `import-painter-palette` after adding gallery artists or updating the CSV Export the full painting catalog as CSV: ```bash -npm run export-paintings +npm run dev:export-paintings ``` Writes **`Output/paintings.csv`** with columns `artist`, `painting`, `year` (sorted by artist, year, title). The `Output/` folder is git-ignored by convention; regenerate after catalog changes. @@ -394,7 +456,7 @@ The catalog can contain the same work more than once — usually from a **double ### Find duplicates ```bash -npm run find-duplicates +npm run dev:find-duplicates ``` Runs `scripts/find-duplicate-paintings.js`, which reports: @@ -419,9 +481,9 @@ When **Debug mode** is on (home header) or from the **Checkup** page: 1. **Search** — `GET /api/paintings/:id/debug-image-search` (or `…/debug-portrait-search` for artists) tries Google Custom Search (if `GOOGLE_CSE_API_KEY` + `GOOGLE_CSE_CX` are set in `.env`), Google Arts & Culture, Google Images scrape, then DuckDuckGo (`searchGoogleImagesFirst` / `searchArtistPortraitFirst` in `scripts/image-fetcher.js`). 2. **More** — `GET …/debug-image-search/more` or `…/debug-portrait-search/more` returns up to 20 ranked candidates (`searchPaintingImagesMany` / `searchArtistPortraitMany`). The modal shows each thumbnail with **resolution** when the search API provides dimensions; otherwise the client probes via `GET /api/debug/image-proxy`. -3. **Fix** — `POST …/fix-image` or `…/fix-portrait` downloads the chosen URL via `downloadImageForFix` → `replacePaintingImageFromUrl` / `replaceArtistPortraitFromUrl` in `server/image-service.js`, regenerates thumbnails with `sharp`, and sets `checkup_fixed` + `checkup_checked`. +3. **Fix** — `POST …/fix-image` or `…/fix-portrait` downloads the chosen URL via `downloadImageForFix` → `replacePaintingImageFromUrl` / `replaceArtistPortraitFromUrl` in `server/image-service.js`. The server **always regenerates thumbnails from the saved full image** (`writePaintingThumb` / `writePortraitThumb` via `sharp` — not the search-result thumb URL), updates `thumbnail_path` / `portrait_thumb_path`, and sets `checkup_fixed` + `checkup_checked`. 4. **Clear** — `POST …/clear-image` or `…/clear-portrait` deletes local file(s), nulls DB paths, sets both flags. Cleared slots stay empty in the UI (no placeholder; `checkup_fixed` prevents on-demand refetch for paintings). -5. **Upload** — `POST …/upload-image` or `…/upload-portrait` accepts a base64-encoded file in JSON (Express body limit **20 MB**; decoded image max **15 MB**), validates with `sharp`, writes to the standard filename under `data/images/`. +5. **Upload** — `POST …/upload-image` or `…/upload-portrait` accepts a base64-encoded file in JSON (Express body limit **20 MB**; decoded image max **15 MB**), validates with `sharp`, writes to the standard filename under `data/images/`, and regenerates the matching thumbnail the same way as **Fix it**. The file picker uses a native `