Add checkup review flags, duplicate detection, and corrected painting images.
Introduce checkup_checked/checkup_fixed on paintings with Checkup page filters and API. Fixed paintings auto-mark as reviewed. Add find-duplicates tooling and document debug/checkup workflow. Include Botticelli and Michelangelo image fixes from checkup. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
co-authored by
Cursor
parent
4683089495
commit
4c6acd5a3a
@@ -36,6 +36,8 @@ File names are sanitised `{Artist}_{Title}.{ext}`. The image service can redisco
|
||||
| `image-fetcher.js` | *(library)* | Wikimedia / museum resolution used by fetch scripts and API |
|
||||
| `regenerate-thumbnails.js` | `npm run regenerate-thumbnails` | Rebuild thumbs from full images via `sharp` |
|
||||
| `audit-painting-images.js` | `npm run audit-painting-images` | Detect thumb/full aspect-ratio mismatches |
|
||||
| `find-duplicate-paintings.js` | `npm run find-duplicates` | Report exact and near-duplicate catalog rows |
|
||||
| `migrate-checkup-flags.js` | `npm run migrate:checkup-flags` | Add `checkup_checked` / `checkup_fixed` columns |
|
||||
|
||||
## Typical workflow
|
||||
|
||||
@@ -305,3 +307,38 @@ Examples already in the repo:
|
||||
- `Charing Cross Bridge` → Derain (not Monet)
|
||||
|
||||
After adding an override, delete any wrong cached file under `data/images/paintings/` and re-run fetch or call the on-demand image endpoint for that painting.
|
||||
|
||||
## Duplicate paintings
|
||||
|
||||
The catalog can contain the same work more than once — usually from a **double import** (identical artist + title + year + image) or **Wikipedia scrape variants** (different article titles for one icon, e.g. Andrei Rublev’s Trinity).
|
||||
|
||||
### Find duplicates
|
||||
|
||||
```bash
|
||||
npm run find-duplicates
|
||||
```
|
||||
|
||||
Runs `scripts/find-duplicate-paintings.js`, which reports:
|
||||
|
||||
| Report | Rule |
|
||||
|--------|------|
|
||||
| **Exact duplicates** | Same `artist_id`, `title`, and `year` |
|
||||
| **Normalized title duplicates** | Same artist, titles differing only in punctuation/spacing |
|
||||
| **Rublev / Trinity cluster** | Known multi-entry example for manual merge |
|
||||
|
||||
As of a recent audit (~1200 paintings): **52 exact duplicate pairs** (52 removable rows), concentrated in **Hieronymus Bosch** (25), **Albrecht Dürer** (14), and **Domenico Ghirlandaio** (13). Duplicate copies typically share the same image file and have **no influence links**, so the higher id in each pair is safe to delete after review.
|
||||
|
||||
**Near-duplicates** (different titles, same work) need curator judgment — e.g. Rublev ids 36, 316, 320, 322, 323 all describe the Trinity icon under different Wikipedia labels; keep id **36** (`Trinity`, wiki `Trinity (Andrei Rublev)`).
|
||||
|
||||
`expand-paintings.js` skips inserts when normalized titles match, but duplicates can still appear if seed and expansion use different title strings or if influence discovery creates works independently.
|
||||
|
||||
## Debug and checkup image fix
|
||||
|
||||
When **Debug mode** is on (home header) or from the **Checkup** page:
|
||||
|
||||
1. **Search** — `GET /api/paintings/:id/debug-image-search` tries Google Custom Search (if `GOOGLE_CSE_API_KEY` + `GOOGLE_CSE_CX` are set in `.env`), Google Arts & Culture, Google Images scrape, then DuckDuckGo (`searchGoogleImagesFirst` in `scripts/image-fetcher.js`).
|
||||
2. **Fix** — `POST /api/paintings/:id/fix-image` downloads the chosen URL via `replacePaintingImageFromUrl` in `server/image-service.js`, regenerates the thumbnail with `sharp`, and sets checkup flags.
|
||||
|
||||
Checkup **Search visible** queues search for filtered rows only (3 concurrent); it does not search the full catalog on load.
|
||||
|
||||
See [API.md](API.md#developer-image-audit) and [basics.md](basics.md#developer-tools-image-audit).
|
||||
|
||||
Reference in New Issue
Block a user