Require painting thumb regen on every picture change and document the invariant.

Fix disk sync, ensure, and preload paths that previously reused the full image as a thumb.

Co-authored-by: Cursor <cursoragent@cursor.com>

Documentation/API.md

@@ -751,6 +751,8 @@ Google-family image search for debug / checkup (Custom Search → Google Arts &
 
 Download a remote URL and replace the painting’s local full image + thumbnail. Sets `checkup_fixed = true` and `checkup_checked = true`.
 
+**Invariant:** every painting picture write (fix, upload, fetch, disk sync) must regenerate a dedicated `paintings/thumbs/*_thumb.jpg` from the full file — never reuse a remote thumb URL or point `thumbnail_path` at the full image. See [data-and-images.md — Painting thumbnail invariant](data-and-images.md#painting-thumbnail-invariant).
+
 Uses `downloadImageForFix` in `scripts/image-fetcher.js` (browser User-Agent, referer fallbacks, Wikimedia URL upgrades) for reliable downloads from Google Arts, Commons, etc.
 
 **Body**

Documentation/Plans.md

@@ -1,5 +1,11 @@
 this file contains draft for future releases and features
 
+## Standing requirements (do not regress)
+
+- **Painting thumbnails:** any create/replace/clear of a painting’s full picture must regenerate or remove its dedicated `paintings/thumbs/` file — never use the full image or a remote thumb URL as `thumbnail_path`. See [data-and-images.md — Painting thumbnail invariant](data-and-images.md#painting-thumbnail-invariant).
+
+## Feature backlog
+
 1. ~~Multi language support, russian version at least~~ — done: UI i18n (EN/RU) + `entity_translations` DB + curator Translations tool — [i18n-russian.md](i18n-russian.md)
 2. ~~tool to manage links (influence/influenced by ) import csv's ( define format), edit ,add, delete, visualize, map to pictures/ entities~~ — done: curator Influences page (list CRUD + import wizard CSV/JSON/XLSX + neighborhood graph) — [influence-import.md](influence-import.md)
 3. tool to monitor/manage (plan actions) of curator actions, markers to check painting/text ?

Documentation/data-and-images.md

@@ -7,6 +7,21 @@ How catalog content, biographies, and artwork files enter the system.
 1. **No runtime hot-linking** — the UI reads from `/images/…` (local disk). External URLs are used only during ingest.
 2. **No AI-generated art or text** — biographies and descriptions come from Wikipedia; influence notes from curated art-history sources.
 3. **Local copies** — every displayed image should exist under `data/images/` after seeding or fetch.
+4. **Painting thumbnails must stay in sync with the full image** — see [Painting thumbnail invariant](#painting-thumbnail-invariant) below. Treat this as a hard requirement for any new feature or script that writes or replaces painting picture files.
+
+## Painting thumbnail invariant
+
+**Requirement (do not regress):** any action that **creates, replaces, or clears** a painting’s full picture must also **regenerate or remove** that painting’s dedicated thumbnail under `paintings/thumbs/`. Never point `thumbnail_path` at the full-size file as a stand-in, and never reuse a search-result or Commons thumb URL.
+
+| Change type | Expected thumb behavior | Primary code |
+|-------------|-------------------------|--------------|
+| Fix / upload / buffer or URL replace | Write full file, then `writePaintingThumb` / `generateThumbnailFromFull` | `server/image-service.js` |
+| Fetch / seed / expand / influence ingest | Same: thumb from saved full via `savePaintingImages` | `scripts/image-fetcher.js` |
+| Disk sync / preload / ensure when full exists but thumb missing | Generate real `*_thumb.jpg` and update `thumbnail_path` | `ensurePaintingThumbFromFull`, `syncPaintingFromDisk`, `ensurePaintingImages`, `preloadArtistImagesLocal` |
+| Clear image / delete painting | Delete thumb file(s) and null paths (or drop row) | `clearPaintingImage`, `deletePainting` |
+| Bulk rebuild after sync/harmonize | `regenerate-thumbnails.js` / `harmonize:images` | scripts + npm scripts |
+
+Path-only maintenance (`sync-image-paths.js`) may link existing thumb files without rewriting pixels; if full images changed without a matching thumb step, run `npm run dev:regenerate-thumbnails` (or `harmonize:images`).
 
 ## Directory layout
 
@@ -364,9 +379,9 @@ When a painting has no local file, `GET /api/paintings/:id/image` triggers `ensu
    - **Wikipedia search** when catalog labels fail
    - Wikidata → Wikimedia Commons → Wikipedia page image
    - Fallbacks: Met Museum, Art Institute of Chicago, Cleveland Museum, Rijksmuseum, Smithsonian*, Harvard*
-4. Save full image, **generate thumbnail by resizing the full file** (not a separate Commons thumb URL), update DB, serve file.
+4. Save full image, **generate thumbnail by resizing the full file** (not a separate Commons thumb URL), update DB, serve file. If a full file already exists but a dedicated thumb under `paintings/thumbs/` is missing, regenerate the thumb (`ensurePaintingThumbFromFull`) rather than serving the full image as a thumb.
 
-Separate Wikipedia/Commons thumbnail URLs often resolve to the **wrong work** (e.g. a different painting with a similar title). Thumbnails are always derived locally from the downloaded full image via `sharp` in `scripts/image-fetcher.js`.
+Separate Wikipedia/Commons thumbnail URLs often resolve to the **wrong work** (e.g. a different painting with a similar title). Thumbnails are always derived locally from the downloaded full image via `sharp` in `scripts/image-fetcher.js`. See [Painting thumbnail invariant](#painting-thumbnail-invariant).
 
 Requests are deduplicated (`inflight` map) and timeout after 15 seconds. On-demand resolution uses a ~2.5 s delay between external requests to reduce rate-limit risk; batch `fetch-images` runs skip that delay while the per-painting deadline is active.
 

server/image-service.js

@@ -96,10 +96,15 @@ function localFileExists(relPath) {
   return fs.existsSync(path.join(IMAGE_DIR, relPath));
 }
 
-function syncPaintingFromDisk(row) {
+function isPaintingThumbRel(rel) {
-  const safeBase = `${row.artist_name}_${row.title}`.replace(/[^a-zA-Z0-9_-]/g, '_');
+  if (!rel) return false;
-  let imagePath = row.image_path;
+  return rel.replace(/\\/g, '/').startsWith('paintings/thumbs/');
-  let thumbPath = row.thumbnail_path;
+}
+
+/** Discover full/thumb files on disk for a painting basename. */
+function discoverPaintingFilesOnDisk(safeBase) {
+  let imagePath = null;
+  let thumbPath = null;
 
   for (const ext of ['.jpg', '.jpeg', '.png', '.webp', '.JPG']) {
     const full = path.join(IMAGE_DIR, 'paintings', safeBase + ext);
@@ -111,11 +116,50 @@ function syncPaintingFromDisk(row) {
       thumbPath = `paintings/thumbs/${safeBase}_thumb${ext}`;
     }
   }
-  if (!thumbPath && imagePath) thumbPath = imagePath;
+  const jpgThumb = path.join(IMAGE_DIR, 'paintings', 'thumbs', `${safeBase}_thumb.jpg`);
+  if (!thumbPath && fs.existsSync(jpgThumb)) {
+    thumbPath = `paintings/thumbs/${safeBase}_thumb.jpg`;
+  }
   return { imagePath, thumbPath };
 }
 
-/** Fast preload: link local files only, no external API calls */
+/**
+ * Ensure a dedicated thumbs/ file exists for a full painting image.
+ * Regenerates from the full file when missing.
+ */
+async function ensurePaintingThumbFromFull(fullRel, safeBase) {
+  if (!fullRel || !localFileExists(fullRel)) return null;
+  const expectedThumb = `paintings/thumbs/${safeBase}_thumb.jpg`;
+  if (localFileExists(expectedThumb)) return expectedThumb;
+  try {
+    return await writePaintingThumb(path.join(IMAGE_DIR, fullRel), safeBase);
+  } catch (err) {
+    console.warn(`Painting thumb generation failed for ${safeBase}:`, err.message);
+    return null;
+  }
+}
+
+/** Sync/link paths from disk; generate thumbnail when full exists without a thumbs/ file. */
+async function syncPaintingFromDisk(row) {
+  const safeBase = safePaintingBase(row.artist_name, row.title);
+  let imagePath = localFileExists(row.image_path) ? row.image_path : null;
+  let thumbPath =
+    localFileExists(row.thumbnail_path) && isPaintingThumbRel(row.thumbnail_path)
+      ? row.thumbnail_path
+      : null;
+
+  const discovered = discoverPaintingFilesOnDisk(safeBase);
+  if (!imagePath && discovered.imagePath) imagePath = discovered.imagePath;
+  if (!thumbPath && discovered.thumbPath) thumbPath = discovered.thumbPath;
+
+  if (!thumbPath && imagePath) {
+    thumbPath = (await ensurePaintingThumbFromFull(imagePath, safeBase)) || null;
+  }
+
+  return { imagePath, thumbPath };
+}
+
+/** Fast preload: link local files only, no external API calls; regenerate missing thumbs from full files. */
 async function preloadArtistImagesLocal(artistId) {
   const rows = await pool.query(
     `SELECT p.id, p.title, p.image_path, p.thumbnail_path, a.name AS artist_name
@@ -127,14 +171,14 @@ async function preloadArtistImagesLocal(artistId) {
 
   let linked = 0;
   for (const row of rows.rows) {
-    const hasLocal =
+    const hasFull = localFileExists(row.image_path);
-      localFileExists(row.thumbnail_path) || localFileExists(row.image_path);
+    const hasThumb = localFileExists(row.thumbnail_path) && isPaintingThumbRel(row.thumbnail_path);
-    if (hasLocal) {
+    if (hasFull && hasThumb) {
       linked++;
       continue;
     }
 
-    const synced = syncPaintingFromDisk(row);
+    const synced = await syncPaintingFromDisk(row);
     if (synced.imagePath || synced.thumbPath) {
       await pool.query(
         `UPDATE paintings SET image_path = COALESCE($1, image_path), thumbnail_path = COALESCE($2, thumbnail_path) WHERE id = $3`,
@@ -163,12 +207,27 @@ async function ensurePaintingImages(paintingId, size = 'thumb') {
 
     const row = result.rows[0];
     const wantThumb = size !== 'full';
+    const safeBase = safePaintingBase(row.artist_name, row.title);
 
-    if (wantThumb && localFileExists(row.thumbnail_path)) return row.thumbnail_path;
+    if (wantThumb && localFileExists(row.thumbnail_path) && isPaintingThumbRel(row.thumbnail_path)) {
+      return row.thumbnail_path;
+    }
     if (!wantThumb && localFileExists(row.image_path)) return row.image_path;
-    if (wantThumb && localFileExists(row.image_path)) return row.image_path;
 
-    const synced = syncPaintingFromDisk(row);
+    // Full on disk but no dedicated thumb — regenerate before falling back to full file.
+    if (wantThumb && localFileExists(row.image_path)) {
+      const thumbRel = await ensurePaintingThumbFromFull(row.image_path, safeBase);
+      if (thumbRel) {
+        await pool.query(`UPDATE paintings SET thumbnail_path = $1 WHERE id = $2`, [
+          thumbRel,
+          paintingId,
+        ]);
+        return thumbRel;
+      }
+      return row.image_path;
+    }
+
+    const synced = await syncPaintingFromDisk(row);
     if (synced.imagePath || synced.thumbPath) {
       await pool.query(
         `UPDATE paintings
@@ -177,8 +236,20 @@ async function ensurePaintingImages(paintingId, size = 'thumb') {
          WHERE id = $3`,
         [synced.imagePath, synced.thumbPath, paintingId]
       );
-      if (wantThumb && localFileExists(synced.thumbPath)) return synced.thumbPath;
+      if (wantThumb && localFileExists(synced.thumbPath) && isPaintingThumbRel(synced.thumbPath)) {
-      if (wantThumb && localFileExists(synced.imagePath)) return synced.imagePath;
+        return synced.thumbPath;
+      }
+      if (wantThumb && localFileExists(synced.imagePath)) {
+        const thumbRel = await ensurePaintingThumbFromFull(synced.imagePath, safeBase);
+        if (thumbRel) {
+          await pool.query(`UPDATE paintings SET thumbnail_path = $1 WHERE id = $2`, [
+            thumbRel,
+            paintingId,
+          ]);
+          return thumbRel;
+        }
+        return synced.imagePath;
+      }
       if (!wantThumb && localFileExists(synced.imagePath)) return synced.imagePath;
     }