Require painting thumb regen on every picture change and document the invariant.

Fix disk sync, ensure, and preload paths that previously reused the full image as a thumb.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Danila Khodjaef
2026-07-27 14:45:46 +03:00
co-authored by Cursor
parent 67594ea4d3
commit bfa21989c9
4 changed files with 111 additions and 17 deletions
+2
View File
@@ -751,6 +751,8 @@ Google-family image search for debug / checkup (Custom Search → Google Arts &
Download a remote URL and replace the paintings local full image + thumbnail. Sets `checkup_fixed = true` and `checkup_checked = true`.
**Invariant:** every painting picture write (fix, upload, fetch, disk sync) must regenerate a dedicated `paintings/thumbs/*_thumb.jpg` from the full file — never reuse a remote thumb URL or point `thumbnail_path` at the full image. See [data-and-images.md — Painting thumbnail invariant](data-and-images.md#painting-thumbnail-invariant).
Uses `downloadImageForFix` in `scripts/image-fetcher.js` (browser User-Agent, referer fallbacks, Wikimedia URL upgrades) for reliable downloads from Google Arts, Commons, etc.
**Body**
+6
View File
@@ -1,5 +1,11 @@
this file contains draft for future releases and features
## Standing requirements (do not regress)
- **Painting thumbnails:** any create/replace/clear of a paintings full picture must regenerate or remove its dedicated `paintings/thumbs/` file — never use the full image or a remote thumb URL as `thumbnail_path`. See [data-and-images.md — Painting thumbnail invariant](data-and-images.md#painting-thumbnail-invariant).
## Feature backlog
1. ~~Multi language support, russian version at least~~ — done: UI i18n (EN/RU) + `entity_translations` DB + curator Translations tool — [i18n-russian.md](i18n-russian.md)
2. ~~tool to manage links (influence/influenced by ) import csv's ( define format), edit ,add, delete, visualize, map to pictures/ entities~~ — done: curator Influences page (list CRUD + import wizard CSV/JSON/XLSX + neighborhood graph) — [influence-import.md](influence-import.md)
3. tool to monitor/manage (plan actions) of curator actions, markers to check painting/text ?
+17 -2
View File
@@ -7,6 +7,21 @@ How catalog content, biographies, and artwork files enter the system.
1. **No runtime hot-linking** — the UI reads from `/images/…` (local disk). External URLs are used only during ingest.
2. **No AI-generated art or text** — biographies and descriptions come from Wikipedia; influence notes from curated art-history sources.
3. **Local copies** — every displayed image should exist under `data/images/` after seeding or fetch.
4. **Painting thumbnails must stay in sync with the full image** — see [Painting thumbnail invariant](#painting-thumbnail-invariant) below. Treat this as a hard requirement for any new feature or script that writes or replaces painting picture files.
## Painting thumbnail invariant
**Requirement (do not regress):** any action that **creates, replaces, or clears** a paintings full picture must also **regenerate or remove** that paintings dedicated thumbnail under `paintings/thumbs/`. Never point `thumbnail_path` at the full-size file as a stand-in, and never reuse a search-result or Commons thumb URL.
| Change type | Expected thumb behavior | Primary code |
|-------------|-------------------------|--------------|
| Fix / upload / buffer or URL replace | Write full file, then `writePaintingThumb` / `generateThumbnailFromFull` | `server/image-service.js` |
| Fetch / seed / expand / influence ingest | Same: thumb from saved full via `savePaintingImages` | `scripts/image-fetcher.js` |
| Disk sync / preload / ensure when full exists but thumb missing | Generate real `*_thumb.jpg` and update `thumbnail_path` | `ensurePaintingThumbFromFull`, `syncPaintingFromDisk`, `ensurePaintingImages`, `preloadArtistImagesLocal` |
| Clear image / delete painting | Delete thumb file(s) and null paths (or drop row) | `clearPaintingImage`, `deletePainting` |
| Bulk rebuild after sync/harmonize | `regenerate-thumbnails.js` / `harmonize:images` | scripts + npm scripts |
Path-only maintenance (`sync-image-paths.js`) may link existing thumb files without rewriting pixels; if full images changed without a matching thumb step, run `npm run dev:regenerate-thumbnails` (or `harmonize:images`).
## Directory layout
@@ -364,9 +379,9 @@ When a painting has no local file, `GET /api/paintings/:id/image` triggers `ensu
- **Wikipedia search** when catalog labels fail
- Wikidata → Wikimedia Commons → Wikipedia page image
- Fallbacks: Met Museum, Art Institute of Chicago, Cleveland Museum, Rijksmuseum, Smithsonian*, Harvard*
4. Save full image, **generate thumbnail by resizing the full file** (not a separate Commons thumb URL), update DB, serve file.
4. Save full image, **generate thumbnail by resizing the full file** (not a separate Commons thumb URL), update DB, serve file. If a full file already exists but a dedicated thumb under `paintings/thumbs/` is missing, regenerate the thumb (`ensurePaintingThumbFromFull`) rather than serving the full image as a thumb.
Separate Wikipedia/Commons thumbnail URLs often resolve to the **wrong work** (e.g. a different painting with a similar title). Thumbnails are always derived locally from the downloaded full image via `sharp` in `scripts/image-fetcher.js`.
Separate Wikipedia/Commons thumbnail URLs often resolve to the **wrong work** (e.g. a different painting with a similar title). Thumbnails are always derived locally from the downloaded full image via `sharp` in `scripts/image-fetcher.js`. See [Painting thumbnail invariant](#painting-thumbnail-invariant).
Requests are deduplicated (`inflight` map) and timeout after 15 seconds. On-demand resolution uses a ~2.5 s delay between external requests to reduce rate-limit risk; batch `fetch-images` runs skip that delay while the per-painting deadline is active.