Art-gallery/Documentation/DB_structure.md

# Art Gallery — database structure

PostgreSQL schema for the virtual gallery. Canonical DDL lives in **`db/schema.sql`**; **`server/migrate.js`** (`npm run dev:migrate`) applies that file plus idempotent incremental scripts in `db/migrate-*.sql`. This document describes the logical model.

Connection settings come from `.env` (dev) or `infra/docker/.env.prod` (prod scripts). See [environments.md](environments.md) and [setup.md](setup.md).

### One-time split (legacy `Gallery` → `gallery_prod` + `gallery_dev`)

Run [`db/split-dev-prod-pgadmin.sql`](../db/split-dev-prod-pgadmin.sql) in **pgAdmin** on the dev PC (postgres superuser). Alternative: `npm run infra:db:split-dev-prod` with `PGUSER=postgres`.

## Overview

| Item | Typical value |
|------|----------------|
| Engine | PostgreSQL 14+ |
| Database (dev) | `gallery_dev` |
| Database (prod) | `gallery_prod` |
| Legacy name | `Gallery` (one-time split → prod + dev) |
| App user | `gallery` |
| Time fields | Integer years (negative = BCE) |

## Entity relationship

```mermaid
erDiagram
  historical_eras ||--o{ art_movements : contains
  art_movements ||--o{ artists : groups
  artists ||--o{ artist_periods : has
  artists ||--o{ paintings : created
  artist_periods ||--o{ paintings : groups
  paintings ||--o{ painting_influences : influenced_by
  paintings ||--o{ painting_influences : influences
  paintings ||--o{ painting_influence_sources : influenced_by
  artists ||--o{ painting_influence_sources : artist_source
  art_movements ||--o{ painting_influence_sources : movement_source
```

## Tables

### `historical_eras`

Broad chronological buckets (Ancient, Medieval, Renaissance, …).

| Column | Type | Notes |
|--------|------|-------|
| `id` | SERIAL PK | |
| `name` | VARCHAR(100) | Display label |
| `start_year`, `end_year` | INTEGER | Inclusive range |
| `start_definite`, `end_definite` | BOOLEAN | `false` → render as gradient edge on timeline |
| `description` | TEXT | Tooltip / sidebar copy |
| `sort_order` | INTEGER | Display order |

### `art_movements`

Finer-grained styles (Impressionism, Cubism, Suprematism, …).

| Column | Type | Notes |
|--------|------|-------|
| `id` | SERIAL PK | |
| `name` | VARCHAR(150) | |
| `start_year`, `end_year` | INTEGER | |
| `start_definite`, `end_definite` | BOOLEAN | Same visual semantics as eras |
| `era_id` | FK → `historical_eras` | Optional parent era |
| `description` | TEXT | |
| `color` | VARCHAR(20) | Hex colour for movement stream in the flow diagram |

### `artists`

| Column | Type | Notes |
|--------|------|-------|
| `id` | SERIAL PK | |
| `name` | VARCHAR(200) | |
| `birth_year`, `death_year` | INTEGER | Nullable; used for timeline portrait placement |
| `movement_id` | FK → `art_movements` | Primary movement |
| `portrait_path` | VARCHAR(500) | Relative to `data/images/`; nullable after debug **Clear** |
| `bio_short`, `bio_full` | TEXT | Wikipedia lead section (`npm run dev:fetch-artist-bios`) |
| `wikipedia_title` | VARCHAR(300) | Source page title |
| `century` | INTEGER | Rounded century bucket for seeding limits |
| `checkup_checked` | BOOLEAN NOT NULL DEFAULT false | Portrait reviewed in debug workflow (gold border on bio when true) |
| `checkup_fixed` | BOOLEAN NOT NULL DEFAULT false | Portrait replaced, cleared, or uploaded via debug |
| `palette_metadata` | JSONB | Enrichment from `Inputs/PainterPalette.csv` (`npm run dev:import-painter-palette`) |

Applied by `npm run dev:migrate:artist-checkup-flags` (`db/migrate-artist-checkup-flags.sql`) and `npm run dev:migrate:artist-palette` (`db/migrate-artist-palette.sql`).

### `artist_periods`

Phases within an artist’s career (e.g. “Blue Period”, “Roman Period”).

| Column | Type | Notes |
|--------|------|-------|
| `id` | SERIAL PK | |
| `artist_id` | FK → `artists` | ON DELETE CASCADE |
| `name` | VARCHAR(200) | |
| `start_year`, `end_year` | INTEGER | |
| `description` | TEXT | |
| `sort_order` | INTEGER | Wall order in 3D gallery |

### `paintings`

| Column | Type | Notes |
|--------|------|-------|
| `id` | SERIAL PK | |
| `artist_id` | FK → `artists` | ON DELETE CASCADE |
| `period_id` | FK → `artist_periods` | Optional grouping |
| `title` | VARCHAR(300) | |
| `year`, `year_end` | INTEGER | Creation date(s) |
| `description` | TEXT | |
| `image_path` | VARCHAR(500) | Full-size local file; nullable after debug **Clear** |
| `thumbnail_path` | VARCHAR(500) | Smaller variant for lists / 3D; nullable after **Clear** |
| `wikipedia_title` | VARCHAR(300) | Used by image fetcher |
| `sort_order` | INTEGER | |
| `checkup_checked` | BOOLEAN NOT NULL DEFAULT false | Reviewed in image checkup workflow (UI label: **Reviewed**) |
| `checkup_fixed` | BOOLEAN NOT NULL DEFAULT false | Image corrected, cleared, or uploaded via checkup / debug |

When `checkup_fixed` is true, `checkup_checked` is set automatically and cannot be cleared until **Fixed** is off. A cleared painting (`image_path` and `thumbnail_path` both null, `checkup_fixed` true) is shown as an empty frame in detail view and is not refetched on demand.

Applied by `npm run dev:migrate:checkup-flags` (`db/migrate-checkup-flags.sql`).

### `painting_annotations`

Short art-history notes shown on painting detail (`PaintingAnnotations.tsx`).

| Column | Type | Notes |
|--------|------|-------|
| `id` | SERIAL PK | |
| `painting_id` | FK → `paintings` | ON DELETE CASCADE |
| `label` | VARCHAR(80) | Optional short heading (e.g. figure name) |
| `body` | TEXT | Note text |
| `category` | VARCHAR(30) | Default `subject`; also `technique`, `context`, `symbolism`, etc. |
| `pos_x`, `pos_y` | NUMERIC(5,2) | Optional marker position on image (percent 0–100) |
| `source_author` | VARCHAR(200) | e.g. Gombrich, Met catalog |
| `source` | VARCHAR(500) | Citation label |
| `source_url` | VARCHAR(500) | Reference link |
| `sort_order` | INTEGER | Display order within the painting |
| `confidence` | VARCHAR(20) | Default `curated`; Wikipedia pass uses `wikipedia` |

Applied by `npm run dev:migrate:painting-annotations` (`db/migrate-painting-annotations.sql`). Load data with `npm run dev:update-painting-annotations` (curated entries in `scripts/painting-annotations-data.js`; add `--wikipedia` for intro sentences from each work’s `wikipedia_title`).

### `tours` / `tour_stops`

Curated guided tours. See [tours.md](tours.md).

**`tours`**

| Column | Type | Notes |
|--------|------|-------|
| `id` | SERIAL PK | |
| `title` | VARCHAR(200) | |
| `description` | TEXT | Default `''` |
| `status` | VARCHAR(20) | `draft` \| `published` |
| `cover_painting_id` | FK → `paintings` | ON DELETE SET NULL |
| `created_at` / `updated_at` | TIMESTAMPTZ | `updated_at` via trigger |

**`tour_stops`**

| Column | Type | Notes |
|--------|------|-------|
| `id` | SERIAL PK | |
| `tour_id` | FK → `tours` | ON DELETE CASCADE |
| `painting_id` | FK → `paintings` | ON DELETE CASCADE; UNIQUE with `tour_id` |
| `sort_order` | INTEGER | Visitor / editor order |
| `body` | TEXT | English tour notes for the stop (v1) |

Applied by `npm run dev:migrate` (`db/migrate-tours.sql`).

### `painting_influences`

Directed edges: *this painting* was influenced by *that painting*.

| Column | Type | Notes |
|--------|------|-------|
| `id` | SERIAL PK | |
| `painting_id` | FK → `paintings` | The work being explained |
| `influenced_by_painting_id` | FK → `paintings` | The earlier / source work |
| `notes` | TEXT | Curator summary |
| `source` | VARCHAR(500) | General attribution |
| `aspects` | TEXT | What was borrowed (composition, colour, …) |
| `quote` | TEXT | Short citation |
| `source_author` | VARCHAR(200) | e.g. Gombrich, Janson |
| `source_url` | VARCHAR(500) | Reference link |

Unique constraint on `(painting_id, influenced_by_painting_id)`.

**Legacy mirror table.** `npm run dev:update-influences` still inserts painting-to-painting rows here when curating data. The same edges are copied into `painting_influence_sources`. The **REST API does not read this table** for painting detail or hall navigation — use `painting_influence_sources` as the source of truth for display.

### `painting_influence_sources`

Polymorphic influence links: *this painting* was influenced by a **painting**, **artist**, or **movement**.

| Column | Type | Notes |
|--------|------|-------|
| `id` | SERIAL PK | |
| `painting_id` | FK → `paintings` | The work being explained |
| `source_type` | VARCHAR(20) | `painting`, `artist`, or `movement` |
| `source_painting_id` | FK → `paintings` | When `source_type = painting` |
| `source_artist_id` | FK → `artists` | When `source_type = artist` |
| `source_movement_id` | FK → `art_movements` | When `source_type = movement` |
| `period_note` | VARCHAR(240) | Human-readable period label |
| `period_start_year` | INTEGER | Optional span start |
| `period_end_year` | INTEGER | Optional span end |
| `notes` | TEXT | Curator summary |
| `source` | VARCHAR(500) | General attribution |
| `aspects` | TEXT | What was borrowed |
| `quote` | TEXT | Short citation |
| `source_author` | VARCHAR(200) | e.g. Gombrich, Janson |
| `source_url` | VARCHAR(500) | Reference link |
| `discovered_via` | VARCHAR(120) | e.g. `wikipedia`, `wikidata`, `web:metmuseum.org` |
| `confidence` | VARCHAR(20) | `curated` (default) or `discovered` |

Unique index on `(painting_id, source_type, source_painting_id, source_artist_id, source_movement_id)` with COALESCE for null FKs.

**Canonical influence store.** Used by all API influence queries: painting detail (`influencedBy`, `influenced`), `has_influence_links`, and artist hall navigation (predecessors / successors). Legacy `painting_influences` rows are backfilled here on migration; new curated painting edges are written to both tables by `update-influences`.

### `users`

Curator accounts (named logins). Anonymous site visitors do not have rows here.

| Column | Type | Notes |
|--------|------|-------|
| `id` | SERIAL PK | |
| `username` | VARCHAR(64) UNIQUE | Login name |
| `password_hash` | VARCHAR(255) | bcrypt hash |
| `created_at` | TIMESTAMPTZ | |
| `last_login_at` | TIMESTAMPTZ | Updated on successful login |

First curator is bootstrapped on `npm run dev:migrate` when `users` is empty and `CURATOR_USERNAME` / `CURATOR_PASSWORD` are set in env.

### `curator_audit_log`

Append-only log of curator debug mutations (fix/clear/upload/delete, checkup flag changes).

| Column | Type | Notes |
|--------|------|-------|
| `id` | BIGSERIAL PK | |
| `user_id` | FK → `users` | Who performed the action |
| `action` | VARCHAR(64) | e.g. `painting.fix_image`, `artist.upload_portrait` |
| `resource_type` | VARCHAR(32) | `painting` or `artist` |
| `resource_id` | INTEGER | Target row id |
| `details` | JSONB | Optional metadata (URL, mime type, flag values) |
| `ip_address` | VARCHAR(45) | Client IP (respects `TRUST_PROXY`) |
| `created_at` | TIMESTAMPTZ | |

**Logged `action` values:** `painting.fix_image`, `painting.clear_image`, `painting.upload_image`, `painting.delete`, `painting.checkup_flags`, `artist.fix_portrait`, `artist.clear_portrait`, `artist.upload_portrait`, `artist.checkup_flags`, `translation.upsert`, `translation.publish`, `influence.create`, `influence.update`, `influence.delete`, `influence.import`.

Example query in pgAdmin:

```sql
SELECT l.created_at, u.username, l.action, l.resource_type, l.resource_id, l.details
FROM curator_audit_log l
JOIN users u ON u.id = l.user_id
ORDER BY l.created_at DESC
LIMIT 50;
```

### `session`

PostgreSQL session store for `express-session` (`connect-pg-simple`). Not application data.

## Indexes

- `artists(movement_id)`, `artists(century)`
- `paintings(artist_id)`, `paintings(period_id)`, `paintings(checkup_checked)`, `paintings(checkup_fixed)`
- `artists(checkup_checked)`, `artists(checkup_fixed)`
- `art_movements(era_id)`, `art_movements(start_year, end_year)`
- `painting_influences(painting_id)`, `painting_influences(influenced_by_painting_id)`
- `painting_influence_sources(painting_id)`, `painting_influence_sources(source_artist_id)`, `painting_influence_sources(source_movement_id)`

## First-time setup

The `gallery` database user needs `CREATE` on schema `public` for migrations. If tables cannot be created, run **`db/setup-admin.sql`** as PostgreSQL superuser (or the grants below) before `npm run dev:migrate`:

```sql
GRANT CREATE ON SCHEMA public TO gallery;
GRANT ALL PRIVILEGES ON ALL TABLES IN SCHEMA public TO gallery;
GRANT ALL PRIVILEGES ON ALL SEQUENCES IN SCHEMA public TO gallery;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON TABLES TO gallery;
ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT ALL ON SEQUENCES TO gallery;
```

Then:

```bash
npm run dev:migrate
npm run dev:seed
npm run dev:fetch-artist-bios
npm run dev:expand-catalog
```

## Data conventions

- **Year zero** is not used; BCE years are negative integers.
- **Image paths** are relative to `IMAGE_DIR` (default `./data/images`), e.g. `portraits/Claude_Monet.jpg`, `paintings/thumbs/Raphael_The_School_of_Athens_thumb.jpg`.
- **Seeding cap**: curated ingest targets at most ~100 artists per century to keep the catalog manageable.
- **Catalog expansion**: `scripts/famous-paintings-data.js` plus `npm run dev:expand-catalog` raises thin artist catalogs to at least six notable paintings (`MIN_PAINTINGS`, default 6).
- **Biographies**: `bio_short` and `bio_full` are populated by `npm run dev:fetch-artist-bios` from English Wikipedia lead sections; `wikipedia_title` on the artist row is the source article.