Art-gallery/Documentation/setup.md

Art Gallery — setup and operations

Prerequisites

• Node.js 20+
• PostgreSQL reachable from the app host
• Network access to Wikipedia / Wikimedia (for biographies, seeding, and on-demand image fetch)

Environment

Development-first: The root .env is for dev only (DB_NAME=gallery_dev). All routine work — code, migrations, seeds, image scripts, curator edits — targets dev. Production uses infra/docker/.env.prod and TrueNAS; promote on a scheduled release (~weekly), not after every change. See environments.md — Development-first workflow.

Copy the example file and fill in credentials:

cp .env.example .env

Variable	Description
DB_HOST	PostgreSQL host
DB_PORT	Port (default 5432)
DB_USER	Database user
DB_PASSWORD	Database password
DB_NAME	Database name (gallery_dev for dev; prod uses gallery_prod)
PORT	API listen port (3451 for dev:web; prod container uses 5173)
HOST	Bind address (default 0.0.0.0 — required for LAN access)
PUBLIC_URL	Public URL (dev: https://devgallery.mysuperlab.netcraze.pro; prod: https://gallery.mysuperlab.netcraze.pro)
TRUST_PROXY	Set to true when behind nginx/reverse proxy (honours X-Forwarded-*)
IMAGE_DIR	Root for cached images (default ./data/images)
SESSION_SECRET	Random string for signed session cookies (required for curator login)
SESSION_COOKIE_SECURE	Optional — omit for auto (HTTPS via proxy → Secure); set true/false to force
CURATOR_USERNAME	Bootstrap / reset — curator account name (default curator)
CURATOR_PASSWORD	Bootstrap when users is empty; also used by npm run dev:reset-curator

.env is git-ignored; never commit passwords.

Optional script tuning:

Variable	Used by	Description
MIN_PAINTINGS	expand-catalog	Minimum paintings per artist (default 6)
FETCH_MAX_WAIT_SEC	fetch-images	Max seconds per painting in batch runs (default 10)

Install

npm install
cd client && npm install && cd ..

Database bootstrap

One-shot setup (migrate + seed):

npm run dev:setup

Or step by step:

npm run dev:migrate    # db/schema.sql + db/migrate-*.sql via server/migrate.js
npm run dev:seed       # eras, movements, artists, flagship paintings (one per artist)

npm run dev:migrate is safe to re-run on existing databases (uses IF NOT EXISTS / ADD COLUMN IF NOT EXISTS).

If migration fails with permission errors, grant schema rights to the app user first (see DB_structure.md).

Curator accounts (auth migration)

npm run dev:migrate applies db/migrate-auth.sql (users, curator_audit_log, session tables). When the users table is empty and CURATOR_USERNAME / CURATOR_PASSWORD are set in .env, the first curator account is created automatically. To sync the password from .env later, run npm run dev:reset-curator.

After migrate, sign in from the site header (Curator login). Debug mode, Checkup, Translations, Influences, Movement links (top-left), Tour editor, and all mutating debug APIs require an active curator session (Movement links opens a login gate for guests). Anonymous visitors browse the timeline, published Tours, and 3D halls without logging in. dev:migrate also creates movement_lineage_links / movement_lineage_delete_votes and seeds default lineage when the table is empty.

See API.md — Authentication and basics.md — Developer tools.

Recommended post-seed steps

After a fresh seed, run these to match a fully populated local install:

npm run dev:sync-image-paths           # import paintings from data/images/paintings/ (clone with image files)
npm run dev:fetch-artist-images        # link local portraits or download from Wikipedia
npm run dev:fetch-artist-bios          # bio_short / bio_full from Wikipedia
npm run dev:expand-catalog             # famous works for artists below MIN_PAINTINGS
npm run dev:update-influences          # painting influence graph for detail view + hall exits
npm run dev:migrate:checkup-flags      # optional: review/fixed flags for Checkup page (paintings)
npm run dev:migrate:artist-checkup-flags  # optional: same flags for artist portraits (bio debug)
npm run dev:migrate:search               # optional on very old DBs — also applied by dev:migrate / prod Step 4
npm run dev:migrate:i18n                 # optional — entity_translations (also in dev:migrate)
npm run dev:fetch-artist-bios-ru         # draft Russian bios + Cyrillic names from ru.wikipedia
npm run dev:import-translations -- --file path/to/file.json  # bulk translation import
npm run dev:migrate:painting-annotations  # optional: art-history notes table
npm run dev:update-painting-annotations   # optional: load curated notes (+ --wikipedia for Wikipedia intros)
npm run dev:fetch-images -- --limit=50                # random sample; 10s max per painting (default)
npm run dev:fetch-images -- --limit=50 --max-wait=120 # same batch size, longer lookup per work
cd client && npm run build && cd ..

Order matters: dev:sync-image-paths should run when the repo already contains painting files under data/images/paintings/ but the database only has one flagship work per artist (typical after npm run dev:setup on a clone). dev:update-influences is required for Influenced By / Influenced panels and golden lamps in the 3D hall — seed does not insert influence edges.

Image fetch can take hours if you run it for the entire catalog. The first line of each run reports Missing local files: N. Use npm run dev:fetch-images -- --limit=N for random batches (10s per painting by default), --artist="…" for one artist in catalog order, or on-demand resolution when viewing a painting in the detail view.

Run

Command	Purpose
npm run dev:web	Public dev stack — Vite :5173, API :3451 (Keenetic → devgallery…)
npm run prod:docker:publish	Build + push prod image to Gitea
npm run devtoprod:images	Copy data/images/ → TrueNAS via SMB Gallery share
npm run prodto:dev:images	Copy prod images → dev repo
npm run prodto:dev:db	Clone gallery_prod → gallery_dev
npm run dev:db:backup / devtoprod:db:restore	Dev backup / promote catalog DB to prod (prod restore skips staff/session/audit; syncs sequences)
npm run harmonize	Bidirectional catalog + image merge (last-write-wins) — harmonize-dev-prod.md
npm run prod:build	Build production SPA into client/dist
npm run dev:start	API + static SPA on HOST:PORT (uses root .env)
npm run prod:start	Build client, then start server
npm run dev:server	API with nodemon reload (local HMR workflow)
npm run dev:client	Vite dev server on :5173

See environments.md for dev/prod URLs, database split, Docker deploy, and sync commands. For Russian UI + catalog text, see i18n-russian.md. Timeline layouts and Tree of Art: basics.md, movement-tree.md. UI behaviour standards: ui-interaction-and-component-standards.md. Influence link import/CRUD: influence-import.md. Quick reference: FAC.md.

Production frontend: build the client, then start the server:

npm run prod:build
        # or: cd client && npm run build && cd ..
npm run dev:start

Open http://localhost:5173 (Vite) or http://localhost:3451 (API only).

Production deployment

Production runs as gallery-web on TrueNAS at https://gallery.mysuperlab.netcraze.pro (Keenetic → :5173). Full guide: environments.md and infra/docker/DEPLOY-truenas.md.

Access	URL
Production (Keenetic)	https://gallery.mysuperlab.netcraze.pro
Development (Keenetic)	https://devgallery.mysuperlab.netcraze.pro
LAN direct (prod container)	http://192.168.10.122:5173
LAN direct (dev PC)	http://192.168.10.70:5173

Quick deploy checklist:

1. One-time DB split in pgAdmin on dev PC: db/split-dev-prod-pgadmin.sql
2. Dev .env → DB_NAME=gallery_dev, PORT=3451, PUBLIC_URL=https://devgallery.mysuperlab.netcraze.pro, plus auth vars (SESSION_SECRET, CURATOR_*)
3. npm run dev:migrate on dev and prod DBs (includes auth tables + bootstrap curator)
4. net use \\192.168.10.122\Gallery → npm run devtoprod:images
5. npm run prod:docker:publish → TrueNAS Custom App from infra/docker/compose.truenas.yaml (set auth env in compose)
6. Keenetic: both domains → :5173, protocol to device http, correct IP per environment

Legacy deployment (optional)

Node on the dev PC at :3520 with nginx → Vite :5173 is superseded by TrueNAS Docker prod and npm run dev:web for public dev. See deploy/nginx-gallery.conf and deploy/gallery.service only if you need a local nginx/systemd setup.

Maintenance scripts

Command	Script	Purpose
npm run dev:seed	scripts/seed-wikipedia.js	Reload curated Wikipedia data
npm run dev:fetch-artist-bios	scripts/fetch-artist-bios.js	Wikipedia lead sections → bio_short / bio_full
npm run dev:fetch-artist-bios -- --force	↑	Refresh bios even when already set
npm run dev:expand-catalog	scripts/expand-paintings.js	Insert famous works from famous-paintings-data.js
npm run dev:expand-catalog -- --fetch-images	↑	Also download images for new rows (slow)
npm run dev:fetch-images	scripts/fetch-missing-images.js	Search Wikipedia, Commons, museums for missing files
npm run dev:search-missing-paintings	↑ (alias)	Same as fetch-images
npm run dev:fetch-images -- --artist="Name"	↑	Limit to one artist (catalog order)
npm run dev:fetch-images -- --limit=50	↑	Random sample of N missing paintings
npm run dev:fetch-images -- --limit=250	↑	Larger random batch (processes min(N, missing count))
npm run dev:fetch-images -- --limit=50 --max-wait=120	↑	Random batch; 120s cap per painting
npm run dev:fetch-images -- --discover-only --limit=20	↑	Fix wikipedia_title via search only
npm run dev:regenerate-thumbnails	scripts/regenerate-thumbnails.js	Rebuild all thumbs from full local files
npm run dev:audit-painting-images	scripts/audit-painting-images.js	List thumb/full aspect-ratio mismatches
npm run dev:migrate:thumbnails	db/migrate-thumbnails.sql	Add thumbnail columns
npm run dev:fetch-artist-images	scripts/fetch-artist-images.js	Download portraits or link existing files under data/images/portraits/
npm run dev:fetch-artist-images -- --force	↑	Re-fetch even when portrait_path is set
npm run dev:sync-image-paths	scripts/sync-image-paths.js	Link image_path / thumbnail_path and import missing painting rows from disk
npm run dev:sync-image-paths -- --dry-run	↑	Report only, no DB writes
npm run dev:migrate:influence-sources	scripts/migrate-influence-sources.js	Create painting_influence_sources + backfill legacy edges
npm run dev:migrate:checkup-flags	scripts/migrate-checkup-flags.js	Add checkup_checked / checkup_fixed on paintings
npm run dev:migrate:artist-checkup-flags	scripts/migrate-artist-checkup-flags.js	Add checkup_checked / checkup_fixed on artists (bio debug)
npm run dev:migrate:search	scripts/migrate-search.js	Same indexes as migrate-search.sql (also run by dev:migrate)
npm run dev:migrate:painting-annotations	scripts/migrate-painting-annotations.js	Create painting_annotations table
npm run dev:migrate:artist-palette	scripts/migrate-artist-palette.js	Add palette_metadata JSONB on artists
npm run dev:import-painter-palette	scripts/import-painter-palette.js	Enrich artists + influence links from Inputs/PainterPalette.csv
npm run dev:analyze-painter-palette	scripts/analyze-painter-palette.js	Report CSV ↔ gallery artist name matches
npm run dev:export-paintings	scripts/export-paintings-csv.js	Write Output/paintings.csv (artist, painting, year)
npm run dev:update-painting-annotations	scripts/update-painting-annotations.js	Load curated notes from painting-annotations-data.js
npm run dev:update-painting-annotations -- --wikipedia	↑	Add intro sentences from each work’s Wikipedia page
npm run dev:update-painting-annotations -- --wikipedia --wiki-delay=3000	↑	Slower Wikipedia pass when rate-limited (429)
npm run dev:find-duplicates	scripts/find-duplicate-paintings.js	Report duplicate and near-duplicate painting rows
npm run dev:audit-influence-duplicates	scripts/audit-influence-duplicates.js	Report mirrored legacy/sources edges and duplicate influence rows
npm run dev:update-influences	scripts/update-influences.js	Insert influence links (painting / artist / movement) from art-influences-data.js
npm run dev:update-influences -- --fetch-images	↑	Also download images for newly created works
npm run dev:update-influences -- --discover	↑	Curated pass + web discovery (Wikipedia, Wikidata, Met, art-history sites)
npm run dev:discover-influences	update-influences.js --discover-only	Discovery pass only

Scripts in the repository

These are checked in and maintained:

• seed-wikipedia.js, seed-catalog-data.js — initial catalog (npm run dev:seed)
• image-fetcher.js — Wikimedia / museum image resolution
• fetch-artist-images.js — artist portrait download / disk linking
• sync-image-paths.js — import paintings and align paths from data/images/paintings/
• fetch-artist-bios.js — artist biographies
• expand-paintings.js + famous-paintings-data.js — catalog expansion
• update-influences.js + art-influences-data.js — influence graph (paintings, artists, movements)
• influence-discovery.js + influence-resolver.js — web discovery and polymorphic source resolution
• fetch-missing-images.js — batch image backfill
• find-duplicate-paintings.js — duplicate catalog audit
• audit-influence-duplicates.js — influence graph duplicate / mirror audit
• painter-palette-lib.js, import-painter-palette.js, migrate-artist-palette.js, analyze-painter-palette.js — PainterPalette CSV integration
• export-paintings-csv.js — catalog CSV export to Output/
• migrate-checkup-flags.js — checkup workflow columns (paintings)
• migrate-artist-checkup-flags.js — checkup workflow columns (artist portraits)
• migrate-painting-annotations.js, update-painting-annotations.js, painting-annotations-data.js — art-history notes on painting detail
• regenerate-thumbnails.js, audit-painting-images.js

See data-and-images.md for pipeline details and override maps.

Remote repository

Gitea: Danilka/Art-gallery

git clone https://gitea.mysuperlab.netcraze.pro/Danilka/Art-gallery.git

After clone: copy .env.example → .env, install dependencies, run one-time DB split, then post-seed steps. Deploy: environments.md.

Troubleshooting

Symptom	Likely cause	Fix
Empty timeline	DB not seeded	npm run dev:seed
502 / 504 on public URL	Keenetic rule wrong (IP, port, or https to device)	Dev → 192.168.10.70:5173; prod → 192.168.10.122:5173; protocol http — see environments.md
503 on public URL	Dev servers not running	npm run dev:web (or dev:server + dev:client for local HMR)
/api proxy errors on localhost while Vite works	Shell PORT=5173 makes API bind the same port as Vite	Set PORT=3451 in .env or DEV_API_PORT=3451; restart npm run dev:web
Timeline opens classic after bookmark	Wrong or missing ?layout=	Use ?layout=tree or ?layout=vertical; classic omits the param or uses ?layout=classic — see basics.md
manifest unknown on TrueNAS deploy	Image not in Gitea	npm run prod:docker:publish on dev PC first
500 on all /api/*	Wrong .env or Postgres down	Check connection, logs
“Biographical information not yet available”	Bios not fetched	npm run dev:fetch-artist-bios
Placeholder portraits on timeline	portrait_path not set	npm run dev:fetch-artist-images
Artist hall has only 1–2 paintings	DB not expanded / disk not imported	npm run dev:sync-image-paths then npm run dev:expand-catalog; extend famous-paintings-data.js
Movement flow shows “Loading art history…” repeatedly	Stale client refetching on every pan/zoom	Pull latest client; timeline loads catalog once — hard-refresh
Movement flow slow on first load	Large artist payload	Client uses GET /api/artists?timeline=1 (no full bios); rebuild client
Movement flow zoom sluggish / dead over portraits	Stale client build	Rebuild client — wheel uses capture listener + interaction-mode rendering; hard-refresh
Timeline year labels overlap when zoomed out	Stale client	Rebuild client — chooseTimelineTickInterval() adapts step to span and bar width
Black frames / canvas covers in 3D gallery	No local image for painting	npm run dev:fetch-images -- --limit=50 or --artist="…"; then POST …/preload-images
Many ⏱ timeout lines in fetch batch	Default 10s cap too short for hard works	--max-wait=120 or raise FETCH_MAX_WAIT_SEC
White/grey flicker on frames	Texture loading or z-fighting with wall	Rebuild client (cd client && npm run build); ensure latest VirtualGallery.tsx
Wrong painting in 3D gallery frame	Stale or mismatched thumbnail file	npm run dev:regenerate-thumbnails; gallery prefers full image_path
Wrong painting image (fetch)	Bad museum / search match on first download	Add entry to DIRECT_IMAGE_OVERRIDES in scripts/image-fetcher.js, re-fetch file
Empty exit navigation lists	No influence edges for artist	npm run dev:update-influences; check painting_influence_sources
Empty Influenced By / Influenced on painting detail	No edges for that work	npm run dev:migrate:influence-sources then npm run dev:update-influences; extend art-influences-data.js or run --discover
Same work listed twice under Influenced	Stale server merging legacy + sources tables	Restart server; API reads painting_influence_sources only — run npm run dev:audit-influence-duplicates to verify DB
No golden lamps above frames in 3D hall	Stale API process or no influence edges	Restart server after API changes; run npm run dev:update-influences
Default Vite page instead of gallery	client/dist missing or stale	cd client && npm run build
Permission denied creating tables	gallery user lacks CREATE	Run admin grants, then migrate
Wikipedia API rate limit during fetch	Too many requests in a row	Wait and re-run; scripts retry with backoff
Checkup Reviewed toggle returns 404	Stale server process missing new routes	Restart npm run dev:web or npm run dev:server after pulling API changes
Debug More / Clear / Upload / Remove entry returns 404	Stale server process	Restart npm run dev:start or npm run dev:server; routes in server/index.js + server/image-service.js
Debug Remove entry — button stuck or missing on next painting	Stale client build	cd client && npm run build; hard-refresh — detail view remounts per painting id
Debug Upload returns 413 Payload Too Large	Base64 JSON exceeds body limit	Server allows 20 MB JSON / 15 MB decoded image; compress file or resize before upload
Debug Upload — nothing happens after choosing file	Stale client or broken hidden-file click()	Pull latest client (DebugUploadButton uses <label> + <input>); hard-refresh
Uploaded image reverts after reload (old picture)	Browser cached /images/… at same path	Pull latest server + client — API returns image_cache_key and URLs use ?v=; /images no longer uses immutable long-term cache
No art-history notes on painting detail	Annotations not migrated or loaded	npm run dev:migrate:painting-annotations then npm run dev:update-painting-annotations
Fix it fails with read ECONNRESET	Remote host dropped connection	Restart server; client sends searchUrl / source; retry or use Commons URL in overrides
Fixed image not shown in 3D gallery	Stale gallery session or cached texture	Rebuild client; fix/upload updates session + ?v= from image_cache_key — use Back to Gallery (not browser back)
Back to Timeline returns to gallery / previous wing	Stale client build	Pull latest client — goToTimelineHome() unmounts the hall and resets timeline zoom
Catalog search dropdown hidden under timeline	Stale client CSS	Rebuild client — .site-header uses z-index: 110 above the sticky timeline bar
Catalog search returns empty / 500	Search indexes missing	Run npm run dev:migrate (includes migrate-search.sql) or npm run dev:migrate:search; restart API
fetch-artist-bios-ru fails: index row size … exceeds btree maximum	Old entity_translations index on all value text	npm run dev:migrate (partial search index on name/title only), then re-run npm run dev:fetch-artist-bios-ru
Russian UI shows English catalog names	Translations not published	Curator → Translations → Publish; public API serves only status = published
Influence import leaves many unresolved tokens	Names not in catalog DB	Curator → Influences → Import warnings; fix spelling or add artists first; free-text traditions stay unresolved
Influence import created too many edges	Artist-level rows expand to all paintings	Expected (PainterPalette-style); delete unwanted edges in Influences list
Influence import blocked: already imported	Same file bytes or mapped data imported before	Expected; use Import anyway only if intentional, or skip
Frame still black after Checked	Gallery session not synced	Re-enter hall or toggle debug Checked from detail with gallery open behind overlay
Duplicate works in gallery / timeline	Double import or variant Wikipedia titles	npm run dev:find-duplicates; merge or delete spare rows manually
Failed to load movement gallery / Cannot GET /api/movements/:id/gallery	Stale server process missing route	Restart npm run dev:web or npm run dev:server after pulling API changes
Movement gallery shows generic cream walls	Stale client build	cd client && npm run build; hard-refresh browser
Windows overlap paintings in movement wing	Stale client	Rebuild client — windows are placed only on side walls in gaps between frames
Influence thumbnails cropped on painting detail	Stale client build	npm run prod:build — panels use object-fit: contain for full image
Curator login fails / always guest	Auth tables missing or wrong password	Set SESSION_SECRET + CURATOR_PASSWORD in .env, run npm run dev:migrate (or npm run dev:reset-curator), restart server
Debug / Checkup returns 401	Not signed in as curator	Curator login (top-right); session cookie gallery.sid must be sent (credentials: include)
Debug works in UI but API rejects	Stale server without auth middleware	Restart npm run dev:web or npm run dev:server after pulling auth changes
Empty screen entering 3D hall (header missing)	Stale client before gallery-session fix	Hard-refresh; pull latest client — hall renders from view state, not only gallerySession
Dark center entering 3D hall (header visible, no spinner)	Stale client before gallery loading overlay fix	Hard-refresh; latest client shows Loading gallery… / Loading paintings… in the canvas area until ready
3D hall loading overlay never clears	Stuck texture counter or hung HDR	Hard-refresh; current client times out Environment / shader warm-up so the overlay cannot stay forever
3D hall black after returning from painting detail	WebGL context lost while hall was hidden	Hard-refresh; latest client remounts canvas when hall becomes active again