Art-gallery/Documentation/setup.md

# Art Gallery — setup and operations

## Prerequisites

- **Node.js** 20+
- **PostgreSQL** reachable from the app host
- Network access to Wikipedia / Wikimedia (for biographies, seeding, and on-demand image fetch)

## Environment

Copy the example file and fill in credentials:

```bash
cp .env.example .env
```

| Variable | Description |
|----------|-------------|
| `DB_HOST` | PostgreSQL host |
| `DB_PORT` | Port (default `5432`) |
| `DB_USER` | Database user |
| `DB_PASSWORD` | Database password |
| `DB_NAME` | Database name (`Gallery`) |
| `PORT` | API listen port (default `3001`; production uses `3520`) |
| `HOST` | Bind address (default `0.0.0.0` — required for LAN access) |
| `PUBLIC_URL` | Optional public URL shown at startup (e.g. `http://gallery.mysuperlab.netcraze.pro`) |
| `TRUST_PROXY` | Set to `true` when behind nginx/reverse proxy (honours `X-Forwarded-*`) |
| `IMAGE_DIR` | Root for cached images (default `./data/images`) |

Optional script tuning:

| Variable | Used by | Description |
|----------|---------|-------------|
| `MIN_PAINTINGS` | `expand-catalog` | Minimum paintings per artist (default `6`) |
| `FETCH_MAX_WAIT_SEC` | `fetch-images` | Max seconds per painting in batch runs (default `10`) |

`.env` is git-ignored; never commit passwords.

## Install

```bash
npm install
cd client && npm install && cd ..
```

## Database bootstrap

One-shot setup (migrate + seed):

```bash
npm run setup
```

Or step by step:

```bash
npm run migrate    # db/schema.sql + db/migrate-*.sql via server/migrate.js
npm run seed       # eras, movements, artists, paintings, influences
```

`npm run migrate` is safe to re-run on existing databases (uses `IF NOT EXISTS` / `ADD COLUMN IF NOT EXISTS`).

If migration fails with permission errors, grant schema rights to the app user first (see [DB_structure.md](DB_structure.md)).

### Recommended post-seed steps

After a fresh seed, run these to match a fully populated local install:

```bash
npm run fetch-artist-bios          # bio_short / bio_full from Wikipedia
npm run expand-catalog             # famous works for artists below MIN_PAINTINGS
npm run update-influences          # painting influence graph for detail view + hall exits
npm run migrate:checkup-flags      # optional: review/fixed flags for Checkup page (paintings)
npm run migrate:artist-checkup-flags  # optional: same flags for artist portraits (bio debug)
npm run migrate:painting-annotations  # optional: art-history notes table
npm run update-painting-annotations   # optional: load curated notes (+ --wikipedia for Wikipedia intros)
npm run fetch-images -- --limit=50                # random sample; 10s max per painting (default)
npm run fetch-images -- --limit=50 --max-wait=120 # same batch size, longer lookup per work
cd client && npm run build && cd ..
```

Image fetch can take hours if you run it for the entire catalog. The first line of each run reports **`Missing local files: N`**. Use **`npm run fetch-images -- --limit=N`** for random batches (10s per painting by default), **`--artist="…"`** for one artist in catalog order, or on-demand resolution when viewing a painting in the detail view.

## Run

| Command | Purpose |
|---------|---------|
| `npm run build` | Build production SPA into `client/dist` |
| `npm run start` | API + static SPA on `HOST`:`PORT` |
| `npm run start:prod` | Build client, then start server |
| `npm run server` | Alias for `start` |
| `npm run dev:server` | API with nodemon reload |
| `npm run dev:client` | Vite dev server on :5173 |
| `npm run dev` | Alias for `start` |

**Production frontend:** build the client, then start the server:

```bash
npm run build
        # or: cd client && npm run build && cd ..
npm run start
```

Open http://localhost:3520 (or your configured `HOST`/`PORT`).

## Production deployment

This install is intended to run at:

| Access | URL |
|--------|-----|
| Public (reverse proxy) | http://gallery.mysuperlab.netcraze.pro |
| LAN direct | http://192.168.10.70:3520 |

### 1. Configure `.env`

Copy `.env.example` → `.env` and set at least:

```env
PORT=3520
HOST=0.0.0.0
PUBLIC_URL=http://gallery.mysuperlab.netcraze.pro
TRUST_PROXY=true
```

`HOST=0.0.0.0` lets the app accept connections on the machine’s LAN IP (`192.168.10.70`). The client uses relative `/api` and `/images` paths, so no frontend URL changes are needed.

### 2. Build and start

```bash
npm install
cd client && npm install && cd ..
npm run start:prod
```

Or use the systemd unit in [`deploy/gallery.service`](../deploy/gallery.service) (adjust `User`, `WorkingDirectory`, and `EnvironmentFile`).

### 3. Reverse proxy (public domain)

Point `gallery.mysuperlab.netcraze.pro` at the host running the app. Example nginx config: [`deploy/nginx-gallery.conf`](../deploy/nginx-gallery.conf).

The proxy forwards to `127.0.0.1:3520`. Keep `TRUST_PROXY=true` in `.env` so Express sees the correct client IP and scheme.

### 4. Firewall

Allow inbound **TCP 3520** on the gallery host if clients reach it directly on the LAN (`192.168.10.70:3520`). The public hostname only needs **80/443** on the reverse-proxy host.

## Maintenance scripts

| Command | Script | Purpose |
|---------|--------|---------|
| `npm run seed` | `scripts/seed-wikipedia.js` | Reload curated Wikipedia data |
| `npm run fetch-artist-bios` | `scripts/fetch-artist-bios.js` | Wikipedia lead sections → `bio_short` / `bio_full` |
| `npm run fetch-artist-bios -- --force` | ↑ | Refresh bios even when already set |
| `npm run expand-catalog` | `scripts/expand-paintings.js` | Insert famous works from `famous-paintings-data.js` |
| `npm run expand-catalog -- --fetch-images` | ↑ | Also download images for new rows (slow) |
| `npm run fetch-images` | `scripts/fetch-missing-images.js` | Search Wikipedia, Commons, museums for missing files |
| `npm run search-missing-paintings` | ↑ (alias) | Same as `fetch-images` |
| `npm run fetch-images -- --artist="Name"` | ↑ | Limit to one artist (catalog order) |
| `npm run fetch-images -- --limit=50` | ↑ | Random sample of N missing paintings |
| `npm run fetch-images -- --limit=250` | ↑ | Larger random batch (processes min(N, missing count)) |
| `npm run fetch-images -- --limit=50 --max-wait=120` | ↑ | Random batch; 120s cap per painting |
| `npm run fetch-images -- --discover-only --limit=20` | ↑ | Fix `wikipedia_title` via search only |
| `npm run regenerate-thumbnails` | `scripts/regenerate-thumbnails.js` | Rebuild all thumbs from full local files |
| `npm run audit-painting-images` | `scripts/audit-painting-images.js` | List thumb/full aspect-ratio mismatches |
| `npm run migrate:thumbnails` | `db/migrate-thumbnails.sql` | Add thumbnail columns |
| `npm run fetch-artist-images` | `scripts/fetch-artist-images.js` | Backfill portrait files *(if present)* |
| `npm run sync-image-paths` | `scripts/sync-image-paths.js` | Align DB paths with disk *(if present)* |
| `npm run migrate:influence-sources` | `scripts/migrate-influence-sources.js` | Create `painting_influence_sources` + backfill legacy edges |
| `npm run migrate:checkup-flags` | `scripts/migrate-checkup-flags.js` | Add `checkup_checked` / `checkup_fixed` on `paintings` |
| `npm run migrate:artist-checkup-flags` | `scripts/migrate-artist-checkup-flags.js` | Add `checkup_checked` / `checkup_fixed` on `artists` (bio debug) |
| `npm run migrate:painting-annotations` | `scripts/migrate-painting-annotations.js` | Create `painting_annotations` table |
| `npm run migrate:artist-palette` | `scripts/migrate-artist-palette.js` | Add `palette_metadata` JSONB on `artists` |
| `npm run import-painter-palette` | `scripts/import-painter-palette.js` | Enrich artists + influence links from `Inputs/PainterPalette.csv` |
| `npm run analyze-painter-palette` | `scripts/analyze-painter-palette.js` | Report CSV ↔ gallery artist name matches |
| `npm run export-paintings` | `scripts/export-paintings-csv.js` | Write `Output/paintings.csv` (artist, painting, year) |
| `npm run update-painting-annotations` | `scripts/update-painting-annotations.js` | Load curated notes from `painting-annotations-data.js` |
| `npm run update-painting-annotations -- --wikipedia` | ↑ | Add intro sentences from each work’s Wikipedia page |
| `npm run update-painting-annotations -- --wikipedia --wiki-delay=3000` | ↑ | Slower Wikipedia pass when rate-limited (429) |
| `npm run find-duplicates` | `scripts/find-duplicate-paintings.js` | Report duplicate and near-duplicate painting rows |
| `npm run audit-influence-duplicates` | `scripts/audit-influence-duplicates.js` | Report mirrored legacy/sources edges and duplicate influence rows |
| `npm run update-influences` | `scripts/update-influences.js` | Insert influence links (painting / artist / movement) from `art-influences-data.js` |
| `npm run update-influences -- --fetch-images` | ↑ | Also download images for newly created works |
| `npm run update-influences -- --discover` | ↑ | Curated pass + web discovery (Wikipedia, Wikidata, Met, art-history sites) |
| `npm run discover-influences` | `update-influences.js --discover-only` | Discovery pass only |

### Scripts in the repository

These are checked in and maintained:

- `image-fetcher.js` — Wikimedia / museum image resolution
- `fetch-artist-bios.js` — artist biographies
- `expand-paintings.js` + `famous-paintings-data.js` — catalog expansion
- `update-influences.js` + `art-influences-data.js` — influence graph (paintings, artists, movements)
- `influence-discovery.js` + `influence-resolver.js` — web discovery and polymorphic source resolution
- `fetch-missing-images.js` — batch image backfill
- `find-duplicate-paintings.js` — duplicate catalog audit
- `audit-influence-duplicates.js` — influence graph duplicate / mirror audit
- `painter-palette-lib.js`, `import-painter-palette.js`, `migrate-artist-palette.js`, `analyze-painter-palette.js` — PainterPalette CSV integration
- `export-paintings-csv.js` — catalog CSV export to `Output/`
- `migrate-checkup-flags.js` — checkup workflow columns (paintings)
- `migrate-artist-checkup-flags.js` — checkup workflow columns (artist portraits)
- `migrate-painting-annotations.js`, `update-painting-annotations.js`, `painting-annotations-data.js` — art-history notes on painting detail
- `regenerate-thumbnails.js`, `audit-painting-images.js`

These are referenced in `package.json` but may need to be restored from git history if missing locally: `seed-wikipedia.js`, `fetch-artist-images.js`, `sync-image-paths.js`.

See [data-and-images.md](data-and-images.md) for pipeline details and override maps.

## Remote repository

Gitea: [Danilka/Art-gallery](https://gitea.mysuperlab.netcraze.pro/Danilka/Art-gallery)

```bash
git clone https://gitea.mysuperlab.netcraze.pro/Danilka/Art-gallery.git
```

After clone: copy `.env.example` → `.env`, install dependencies, run `npm run setup` against your Postgres instance, then the post-seed steps above.

## Troubleshooting

| Symptom | Likely cause | Fix |
|---------|--------------|-----|
| Empty timeline | DB not seeded | `npm run seed` |
| 500 on all `/api/*` | Wrong `.env` or Postgres down | Check connection, logs |
| “Biographical information not yet available” | Bios not fetched | `npm run fetch-artist-bios` |
| Artist hall has only 1–2 paintings | Catalog not expanded | `npm run expand-catalog`; extend `famous-paintings-data.js` |
| Black frames / canvas covers in 3D gallery | No local image for painting | `npm run fetch-images -- --limit=50` or `--artist="…"`; then `POST …/preload-images` |
| Many `⏱ timeout` lines in fetch batch | Default 10s cap too short for hard works | `--max-wait=120` or raise `FETCH_MAX_WAIT_SEC` |
| White/grey flicker on frames | Texture loading or z-fighting with wall | Rebuild client (`cd client && npm run build`); ensure latest `VirtualGallery.tsx` |
| Wrong painting in 3D gallery frame | Stale or mismatched thumbnail file | `npm run regenerate-thumbnails`; gallery prefers full `image_path` |
| Wrong painting image (fetch) | Bad museum / search match on first download | Add entry to `DIRECT_IMAGE_OVERRIDES` in `scripts/image-fetcher.js`, re-fetch file |
| Empty exit navigation lists | No influence edges for artist | `npm run update-influences`; check `painting_influence_sources` |
| Empty *Influenced By* / *Influenced* on painting detail | No edges for that work | `npm run migrate:influence-sources` then `npm run update-influences`; extend `art-influences-data.js` or run `--discover` |
| Same work listed twice under *Influenced* | Stale server merging legacy + sources tables | Restart server; API reads `painting_influence_sources` only — run `npm run audit-influence-duplicates` to verify DB |
| No golden lamps above frames in 3D hall | Stale API process or no influence edges | Restart server after API changes; run `npm run update-influences` |
| Default Vite page instead of gallery | `client/dist` missing or stale | `cd client && npm run build` |
| Permission denied creating tables | `gallery` user lacks CREATE | Run admin grants, then migrate |
| Wikipedia API rate limit during fetch | Too many requests in a row | Wait and re-run; scripts retry with backoff |
| Checkup **Reviewed** toggle returns 404 | Stale server process missing new routes | Restart `npm run dev` after pulling API changes |
| Debug **More** / **Clear** / **Upload** / **Remove entry** returns 404 | Stale server process | Restart `npm run start` or `npm run dev:server`; routes in `server/index.js` + `server/image-service.js` |
| Debug **Remove entry** — button stuck or missing on next painting | Stale client build | `cd client && npm run build`; hard-refresh — detail view remounts per painting id |
| Debug **Upload** returns 413 Payload Too Large | Base64 JSON exceeds body limit | Server allows 20 MB JSON / 15 MB decoded image; compress file or resize before upload |
| No art-history notes on painting detail | Annotations not migrated or loaded | `npm run migrate:painting-annotations` then `npm run update-painting-annotations` |
| **Fix it** fails with `read ECONNRESET` | Remote host dropped connection | Restart server; client sends `searchUrl` / `source`; retry or use Commons URL in overrides |
| Fixed image not shown in 3D gallery | Stale gallery session or cached texture | Rebuild client; fix updates session + `?v=` revision — use **Back to Gallery** (not browser back) |
| Frame still black after **Checked** | Gallery session not synced | Re-enter hall or toggle debug **Checked** from detail with gallery open behind overlay |
| Duplicate works in gallery / timeline | Double import or variant Wikipedia titles | `npm run find-duplicates`; merge or delete spare rows manually |
| **Failed to load movement gallery** / `Cannot GET /api/movements/:id/gallery` | Stale server process missing route | Restart `npm run dev` or `npm run dev:server` after pulling API changes |
| Movement gallery shows generic cream walls | Stale client build | `cd client && npm run build`; hard-refresh browser |
| Windows overlap paintings in movement wing | Stale client | Rebuild client — windows are placed only on side walls in gaps between frames |
| Influence thumbnails cropped on painting detail | Stale client build | `npm run build` — panels use `object-fit: contain` for full image |