Add checkup review flags, duplicate detection, and corrected painting images.

Introduce checkup_checked/checkup_fixed on paintings with Checkup page filters and API. Fixed paintings auto-mark as reviewed. Add find-duplicates tooling and document debug/checkup workflow. Include Botticelli and Michelangelo image fixes from checkup.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Danila Khodjaef
2026-06-20 20:45:55 +03:00
co-authored by Cursor
parent 4683089495
commit 4c6acd5a3a
55 changed files with 750 additions and 39 deletions
+37
View File
@@ -36,6 +36,8 @@ File names are sanitised `{Artist}_{Title}.{ext}`. The image service can redisco
| `image-fetcher.js` | *(library)* | Wikimedia / museum resolution used by fetch scripts and API |
| `regenerate-thumbnails.js` | `npm run regenerate-thumbnails` | Rebuild thumbs from full images via `sharp` |
| `audit-painting-images.js` | `npm run audit-painting-images` | Detect thumb/full aspect-ratio mismatches |
| `find-duplicate-paintings.js` | `npm run find-duplicates` | Report exact and near-duplicate catalog rows |
| `migrate-checkup-flags.js` | `npm run migrate:checkup-flags` | Add `checkup_checked` / `checkup_fixed` columns |
## Typical workflow
@@ -305,3 +307,38 @@ Examples already in the repo:
- `Charing Cross Bridge` → Derain (not Monet)
After adding an override, delete any wrong cached file under `data/images/paintings/` and re-run fetch or call the on-demand image endpoint for that painting.
## Duplicate paintings
The catalog can contain the same work more than once — usually from a **double import** (identical artist + title + year + image) or **Wikipedia scrape variants** (different article titles for one icon, e.g. Andrei Rublevs Trinity).
### Find duplicates
```bash
npm run find-duplicates
```
Runs `scripts/find-duplicate-paintings.js`, which reports:
| Report | Rule |
|--------|------|
| **Exact duplicates** | Same `artist_id`, `title`, and `year` |
| **Normalized title duplicates** | Same artist, titles differing only in punctuation/spacing |
| **Rublev / Trinity cluster** | Known multi-entry example for manual merge |
As of a recent audit (~1200 paintings): **52 exact duplicate pairs** (52 removable rows), concentrated in **Hieronymus Bosch** (25), **Albrecht Dürer** (14), and **Domenico Ghirlandaio** (13). Duplicate copies typically share the same image file and have **no influence links**, so the higher id in each pair is safe to delete after review.
**Near-duplicates** (different titles, same work) need curator judgment — e.g. Rublev ids 36, 316, 320, 322, 323 all describe the Trinity icon under different Wikipedia labels; keep id **36** (`Trinity`, wiki `Trinity (Andrei Rublev)`).
`expand-paintings.js` skips inserts when normalized titles match, but duplicates can still appear if seed and expansion use different title strings or if influence discovery creates works independently.
## Debug and checkup image fix
When **Debug mode** is on (home header) or from the **Checkup** page:
1. **Search**`GET /api/paintings/:id/debug-image-search` tries Google Custom Search (if `GOOGLE_CSE_API_KEY` + `GOOGLE_CSE_CX` are set in `.env`), Google Arts & Culture, Google Images scrape, then DuckDuckGo (`searchGoogleImagesFirst` in `scripts/image-fetcher.js`).
2. **Fix**`POST /api/paintings/:id/fix-image` downloads the chosen URL via `replacePaintingImageFromUrl` in `server/image-service.js`, regenerates the thumbnail with `sharp`, and sets checkup flags.
Checkup **Search visible** queues search for filtered rows only (3 concurrent); it does not search the full catalog on load.
See [API.md](API.md#developer-image-audit) and [basics.md](basics.md#developer-tools-image-audit).