Danilka/Art-gallery
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Document gallery loading markers, WebGL recovery, and movement filtering.

Browse Source 
Update basics.md for the new loading indicators, 3D WebGL context-loss recovery, and the corrected movement-visibility rule (span overlap plus any catalogued artist). Refresh remaining docs and config for the current dev-first workflow and npm script names.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Danila KhodjaefandCursor committed 2026-07-07 12:36:34 +03:00
1 parent 1408811948
commit a2263ca186
10 files changed
+288 -275
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

Documentation/data-and-images.md
+54 -54
View File
@@ -31,29 +31,29 @@ File names are sanitised `{Artist}_{Title}.{ext}`. The image service can redisco
| **Development** | `./data/images/` in repo | Working copy on dev PC |
| **Production** | `/mnt/BasePool/Applications/Gallery/data/images` on TrueNAS | SMB `\\192.168.10.122\Gallery\data\images` |
Promote dev → prod files: `npm run images:sync-to-prod` (after `net use \\192.168.10.122\Gallery`). Refresh dev from prod: `npm run images:sync-from-prod`. See [environments.md](environments.md).
Promote dev → prod files: `npm run devtoprod:images` (after `net use \\192.168.10.122\Gallery`). Refresh dev from prod: `npm run prodto:dev:images`. See [environments.md](environments.md).
## Scripts overview
| Script | npm command | Role |
|--------|-------------|------|
| `seed-wikipedia.js` | `npm run seed` | Initial eras, movements, artists, one flagship painting per artist |
| `seed-wikipedia.js` | `npm run dev:seed` | Initial eras, movements, artists, one flagship painting per artist |
| `seed-catalog-data.js` | *(data only)* | Eras, movements, artist metadata consumed by seed |
| `sync-image-paths.js` | `npm run sync-image-paths` | Import painting rows from disk; set `image_path` / `thumbnail_path` |
| `fetch-artist-images.js` | `npm run fetch-artist-images` | Download or link artist portraits under `data/images/portraits/` |
| `fetch-artist-bios.js` | `npm run fetch-artist-bios` | Wikipedia intros → `bio_short` / `bio_full` |
| `sync-image-paths.js` | `npm run dev:sync-image-paths` | Import painting rows from disk; set `image_path` / `thumbnail_path` |
| `fetch-artist-images.js` | `npm run dev:fetch-artist-images` | Download or link artist portraits under `data/images/portraits/` |
| `fetch-artist-bios.js` | `npm run dev:fetch-artist-bios` | Wikipedia intros → `bio_short` / `bio_full` |
| `famous-paintings-data.js` | *(data only)* | Curated list of notable works per artist |
| `expand-paintings.js` | `npm run expand-catalog` | Inserts works from data file for thin catalogs |
| `expand-paintings.js` | `npm run dev:expand-catalog` | Inserts works from data file for thin catalogs |
| `art-influences-data.js` | *(data only)* | Curated influence edges (painting / artist / movement) |
| `update-influences.js` | `npm run update-influences` | Applies influence graph; creates missing artists/works |
| `fetch-missing-images.js` | `npm run fetch-images` | Downloads files for paintings missing on disk |
| `update-influences.js` | `npm run dev:update-influences` | Applies influence graph; creates missing artists/works |
| `fetch-missing-images.js` | `npm run dev:fetch-images` | Downloads files for paintings missing on disk |
| `image-fetcher.js` | *(library)* | Wikimedia / museum resolution used by fetch scripts and API |
| `sync-images-to-prod.ps1` / `sync-images-from-prod.ps1` | `npm run images:sync-*` | Robocopy via SMB `\\192.168.10.122\Gallery` |
| `regenerate-thumbnails.js` | `npm run regenerate-thumbnails` | Rebuild painting thumbs from full images via `sharp` |
| `regenerate-portrait-thumbs.js` | `npm run regenerate-portrait-thumbs` | Rebuild timeline portrait thumbs (~256px) and set `portrait_thumb_path` |
| `audit-painting-images.js` | `npm run audit-painting-images` | Detect thumb/full aspect-ratio mismatches |
| `find-duplicate-paintings.js` | `npm run find-duplicates` | Report exact and near-duplicate catalog rows |
| `migrate-checkup-flags.js` | `npm run migrate:checkup-flags` | Add `checkup_checked` / `checkup_fixed` columns |
| `sync-images-to-prod.ps1` / `sync-images-from-prod.ps1` | `npm run devtoprod:images` / `npm run prodto:dev:images` | Robocopy via SMB `\\192.168.10.122\Gallery` |
| `regenerate-thumbnails.js` | `npm run dev:regenerate-thumbnails` | Rebuild painting thumbs from full images via `sharp` |
| `regenerate-portrait-thumbs.js` | `npm run dev:regenerate-portrait-thumbs` | Rebuild timeline portrait thumbs (~256px) and set `portrait_thumb_path` |
| `audit-painting-images.js` | `npm run dev:audit-painting-images` | Detect thumb/full aspect-ratio mismatches |
| `find-duplicate-paintings.js` | `npm run dev:find-duplicates` | Report exact and near-duplicate catalog rows |
| `migrate-checkup-flags.js` | `npm run dev:migrate:checkup-flags` | Add `checkup_checked` / `checkup_fixed` columns |
## Typical workflow
@@ -71,13 +71,13 @@ migrate → seed → sync-image-paths → fetch-artist-images → fetch-artist-b
## Seeding pipeline
`npm run seed` runs `scripts/seed-wikipedia.js`, which:
`npm run dev:seed` runs `scripts/seed-wikipedia.js`, which:
1. Inserts **historical eras** and **art movements** (curated date ranges and colours from `seed-catalog-data.js`).
2. For each curated **artist**:
- Creates **artist periods** and one **flagship painting**.
- May download portraits and painting images when run with `--fetch-images`.
3. Does **not** insert influence edges — run `npm run update-influences` after seed (see [Painting influence graph](#painting-influence-graph)).
3. Does **not** insert influence edges — run `npm run dev:update-influences` after seed (see [Painting influence graph](#painting-influence-graph)).
Those influence edges power **3D hall navigation** and painting detail panels via **`painting_influence_sources`** (see `GET /api/artists/:id/navigation` and `GET /api/paintings/:id` in [API.md](API.md)).
@@ -85,7 +85,7 @@ Artists are grouped by movement and century; the seed list targets at most ~100
## Artist biographies
`npm run fetch-artist-bios` reads each artist’s `wikipedia_title`, fetches the English Wikipedia **lead section**, and stores:
`npm run dev:fetch-artist-bios` reads each artist’s `wikipedia_title`, fetches the English Wikipedia **lead section**, and stores:
| Field | Content |
|-------|---------|
@@ -110,15 +110,15 @@ The bio page (`ArtistBio.tsx`) shows lifespan, movement, summary, full text, and
## Expanding thin catalogs
Many seed artists arrive with only one famous painting. `npm run expand-catalog` runs `scripts/expand-paintings.js`, which:
Many seed artists arrive with only one famous painting. `npm run dev:expand-catalog` runs `scripts/expand-paintings.js`, which:
1. Finds artists with fewer than `MIN_PAINTINGS` (default **6**).
2. Inserts rows from `scripts/famous-paintings-data.js` that are not already present (normalized title matching skips duplicates).
3. Sets `wikipedia_title` on each new painting for image resolution.
```bash
npm run expand-catalog # DB rows only
npm run expand-catalog -- --fetch-images # also download images (very slow)
npm run dev:expand-catalog # DB rows only
npm run dev:expand-catalog -- --fetch-images # also download images (very slow)
```
To add more works, append entries to `famous-paintings-data.js`:
@@ -137,7 +137,7 @@ Renaissance and medieval masters with large museum catalog dumps (e.g. Raphael,
When the repository includes a full `data/images/paintings/` tree but the database was seeded fresh (one row per artist), run:
```bash
npm run sync-image-paths
npm run dev:sync-image-paths
```
`scripts/sync-image-paths.js`:
@@ -157,9 +157,9 @@ Typical result on a full clone: ~1,000+ paintings linked from ~1,000 on-disk fil
## Artist portraits
Timeline movement flow loads **`portrait_thumb_path`** (~256px JPEG under `portraits/thumbs/{Artist}_thumb.jpg`) when available; biography and 3D exit navigation use full `portrait_path`. After adding portraits, run `npm run regenerate-portrait-thumbs` to backfill thumbs on dev.
Timeline movement flow loads **`portrait_thumb_path`** (~256px JPEG under `portraits/thumbs/{Artist}_thumb.jpg`) when available; biography and 3D exit navigation use full `portrait_path`. After adding portraits, run `npm run dev:regenerate-portrait-thumbs` to backfill thumbs on dev.
`npm run fetch-artist-images` runs `scripts/fetch-artist-images.js`:
`npm run dev:fetch-artist-images` runs `scripts/fetch-artist-images.js`:
1. For each artist, checks `data/images/portraits/{Artist}.jpg` (or other extensions) and sets `portrait_path` when a local file exists.
2. Otherwise downloads from Wikipedia / search fallbacks via `image-fetcher.js`.
@@ -175,7 +175,7 @@ Run after seed when portrait files exist on disk but the DB still has null `port
Directed influence links are stored in **`painting_influence_sources`**. Each row connects a painting to a **source** of type `painting`, `artist`, or `movement`, with optional period context (e.g. influence during the work’s creation year).
The legacy **`painting_influences`** table (painting-to-painting only) is still written alongside sources when running `npm run update-influences` — it keeps script compatibility and matches the backfill migration. **The API reads only `painting_influence_sources`**, so each edge appears once in the UI.
The legacy **`painting_influences`** table (painting-to-painting only) is still written alongside sources when running `npm run dev:update-influences` — it keeps script compatibility and matches the backfill migration. **The API reads only `painting_influence_sources`**, so each edge appears once in the UI.
Influence data drives:
@@ -188,7 +188,7 @@ Influence data drives:
Painting-to-painting edges exist in both tables by design. To confirm the database has no stray duplicates and that the API model is clean:
```bash
npm run audit-influence-duplicates
npm run dev:audit-influence-duplicates
```
Reports: edges present in both tables, duplicate rows within either table (should be 0), and legacy-only / sources-only mismatches. If *Influenced* ever shows the same successor twice, restart the server after pulling API fixes — responses must not union legacy and sources tables.
@@ -196,19 +196,19 @@ Reports: edges present in both tables, duplicate rows within either table (shoul
### One-time migration
```bash
npm run migrate:influence-sources # create table + backfill legacy painting edges
npm run dev:migrate:influence-sources # create table + backfill legacy painting edges
```
### Curated updates
`npm run update-influences` runs `scripts/update-influences.js` against `scripts/art-influences-data.js`:
`npm run dev:update-influences` runs `scripts/update-influences.js` against `scripts/art-influences-data.js`:
```bash
npm run update-influences # insert curated edges
npm run update-influences -- --fetch-images # also download images for newly created works
npm run update-influences -- --discover # curated + web discovery pass
npm run discover-influences # discovery only (no curated file pass)
npm run update-influences -- --discover --limit=20 # cap discovery to N works
npm run dev:update-influences # insert curated edges
npm run dev:update-influences -- --fetch-images # also download images for newly created works
npm run dev:update-influences -- --discover # curated + web discovery pass
npm run dev:discover-influences # discovery only (no curated file pass)
npm run dev:update-influences -- --discover --limit=20 # cap discovery to N works
```
Each entry defines a later `work` and one or more `influencedBy` sources. Legacy single-object form is still supported:
@@ -290,14 +290,14 @@ Short curator-style notes on the painting detail page — separate from the infl
| UI | `PaintingAnnotations.tsx` — numbered markers on the image (when `pos_x` / `pos_y` set) plus an “Art history notes” list |
| API | Included as `annotations[]` on `GET /api/paintings/:id` |
| Curated data | `scripts/painting-annotations-data.js` — artist/title keys matched via `influence-resolver.js` |
| Load | `npm run update-painting-annotations` (replaces existing rows per painting by default) |
| Wikipedia pass | `npm run update-painting-annotations -- --wikipedia` — one intro sentence per work from `wikipedia_title`; use `--wiki-delay=3000` if rate-limited; `--no-replace` to append without clearing curated rows |
| Load | `npm run dev:update-painting-annotations` (replaces existing rows per painting by default) |
| Wikipedia pass | `npm run dev:update-painting-annotations -- --wikipedia` — one intro sentence per work from `wikipedia_title`; use `--wiki-delay=3000` if rate-limited; `--no-replace` to append without clearing curated rows |
Categories include `subject`, `technique`, `context`, and `symbolism`. Sources cite Gombrich, museum catalogs, and Wikipedia as appropriate.
## Batch image fetch
`npm run fetch-images` (alias: `npm run search-missing-paintings`) runs `scripts/fetch-missing-images.js`. It searches multiple sources for paintings without local files:
`npm run dev:fetch-images` (alias: `npm run dev:search-missing-paintings`) runs `scripts/fetch-missing-images.js`. It searches multiple sources for paintings without local files:
| Source | Notes |
|--------|--------|
@@ -315,13 +315,13 @@ Categories include `subject`, `technique`, `context`, and `symbolism`. Sources c
| Harvard Art Museums | Optional (`HARVARD_ART_API_KEY` in `.env`) |
```bash
npm run fetch-images # all missing, catalog order (~hours)
npm run fetch-images -- --limit=50 # random sample of 50; 10s max per painting
npm run fetch-images -- --limit=250 # random sample up to N (caps at current missing count)
npm run fetch-images -- --limit=50 --max-wait=120 # slower, more thorough lookup per painting
npm run fetch-images -- --artist="Albrecht Dürer" # one artist, catalog order
npm run fetch-images -- --discover-only --limit=20 # fix wikipedia_title only
npm run fetch-images -- --web-search-only --limit=50 # DuckDuckGo + Commons + multilingual Wikipedia
npm run dev:fetch-images # all missing, catalog order (~hours)
npm run dev:fetch-images -- --limit=50 # random sample of 50; 10s max per painting
npm run dev:fetch-images -- --limit=250 # random sample up to N (caps at current missing count)
npm run dev:fetch-images -- --limit=50 --max-wait=120 # slower, more thorough lookup per painting
npm run dev:fetch-images -- --artist="Albrecht Dürer" # one artist, catalog order
npm run dev:fetch-images -- --discover-only --limit=20 # fix wikipedia_title only
npm run dev:fetch-images -- --web-search-only --limit=50 # DuckDuckGo + Commons + multilingual Wikipedia
```
Each run prints **`Missing local files: N`** at startup — that is the current count of catalogued paintings with no full-size or thumbnail file under `data/images/`. A painting counts as present if **either** file exists on disk.
@@ -377,10 +377,10 @@ These live in `client/public/` (and `client/dist/` after build).
1. Insert rows into `artists`, `artist_periods`, `paintings` (or extend the seed script).
2. Place image files under `data/images/` using the naming convention.
3. Run `npm run fetch-artist-bios` for the new artist’s biography.
4. Add entries to `famous-paintings-data.js` and run `npm run expand-catalog` if needed.
5. Run `npm run fetch-images -- --artist="…"` or rely on preload / on-demand sync.
6. Add influence rows via `npm run update-influences` / `art-influences-data.js` (writes both `painting_influence_sources` and legacy painting edges).
3. Run `npm run dev:fetch-artist-bios` for the new artist’s biography.
4. Add entries to `famous-paintings-data.js` and run `npm run dev:expand-catalog` if needed.
5. Run `npm run dev:fetch-images -- --artist="…"` or rely on preload / on-demand sync.
6. Add influence rows via `npm run dev:update-influences` / `art-influences-data.js` (writes both `painting_influence_sources` and legacy painting edges).
## PainterPalette external dataset
@@ -389,8 +389,8 @@ These live in `client/public/` (and `client/dist/` after build).
### One-time setup
```bash
npm run migrate:artist-palette # adds artists.palette_metadata JSONB
npm run import-painter-palette # enrich + influence links
npm run dev:migrate:artist-palette # adds artists.palette_metadata JSONB
npm run dev:import-painter-palette # enrich + influence links
```
### What gets imported
@@ -409,10 +409,10 @@ Name matching uses normalized strings plus aliases in `scripts/painter-palette-l
### Commands
```bash
npm run analyze-painter-palette # match report
npm run import-painter-palette -- --dry-run
npm run import-painter-palette -- --metadata-only
npm run import-painter-palette -- --influences-only
npm run dev:analyze-painter-palette # match report
npm run dev:import-painter-palette -- --dry-run
npm run dev:import-painter-palette -- --metadata-only
npm run dev:import-painter-palette -- --influences-only
```
Re-run `import-painter-palette` after adding gallery artists or updating the CSV; existing palette influence rows are skipped if already present.
@@ -422,7 +422,7 @@ Re-run `import-painter-palette` after adding gallery artists or updating the CSV
Export the full painting catalog as CSV:
```bash
npm run export-paintings
npm run dev:export-paintings
```
Writes **`Output/paintings.csv`** with columns `artist`, `painting`, `year` (sorted by artist, year, title). The `Output/` folder is git-ignored by convention; regenerate after catalog changes.
@@ -454,7 +454,7 @@ The catalog can contain the same work more than once — usually from a **double
### Find duplicates
```bash
npm run find-duplicates
npm run dev:find-duplicates
```
Runs `scripts/find-duplicate-paintings.js`, which reports: