From 62d7ebbe6acd14adcbf36eeea414240febbbebba Mon Sep 17 00:00:00 2001 From: Danila Khodjaef Date: Wed, 15 Jul 2026 12:21:53 +0300 Subject: [PATCH] Document prod schema migrate before restore and i18n search index fix. Release runbook now describes default migrateProdSchema, auto-migrate safety guard, and troubleshooting for updated_at restore failures; i18n docs cover partial entity_translations index. Co-authored-by: Cursor --- Documentation/deploy-dev-to-prod.md | 10 ++++++---- Documentation/i18n-russian.md | 13 ++++++++----- Documentation/setup.md | 2 ++ infra/deploy/devtoprod.config.example.json | 1 - infra/scripts/deploy-dev-to-prod.ps1 | 5 ++--- 5 files changed, 18 insertions(+), 13 deletions(-) diff --git a/Documentation/deploy-dev-to-prod.md b/Documentation/deploy-dev-to-prod.md index 756c166..956f0ba 100644 --- a/Documentation/deploy-dev-to-prod.md +++ b/Documentation/deploy-dev-to-prod.md @@ -34,10 +34,10 @@ For a typical weekly release, use the orchestrator instead of running each step 2. Edit [`infra/deploy/devtoprod.config.json`](../infra/deploy/devtoprod.config.json) (gitignored — safe for SMB password): - `profile`: `full` | `code` | `data` (base step set; individual `steps` overrides win) + - `steps` — per-step overrides; see table below. On **`full`** profile, `migrateProdSchema` is **on by default** (runs `dev:migrate` on `gallery_prod` before restore). - `autoConfirm`: `true` skips restore/image-sync prompts (`CONFIRM_PROD=1`, `-SkipConfirm`) - `autoStartDevStack`: `true` starts `dev:server` (or `dev:web`) automatically when `validateBuild` finds API down - `git.message`, `git.branch` — used when `gitCommitPush` is enabled - - `schemaChanged`: `true` enables prod schema migration on `full` profile - `smb.user` / `smb.password` — optional; maps `\\host\share` before image sync - `backupFile` — optional fixed path; otherwise uses the newest `gallery_dev_data_*.txt` after backup @@ -67,7 +67,7 @@ npm run devtoprod:release -- -DryRun | `thumbnails` | Step 2 — `devtoprod:thumbnails` | | `backupDev` | Step 3 — `dev:db:backup` | | `backupProd` | Step 3 (rollback) — `prod:db:backup` | -| `migrateProdSchema` | Step 4 — `dev:migrate` on `gallery_prod` | +| `migrateProdSchema` | Step 4 — `dev:migrate` on `gallery_prod` (enabled on `full` profile; auto-runs before `restoreProd` if you disabled it but left restore on) | | `restoreProd` | Step 5 — `devtoprod:db:restore` | | `syncImages` | Step 6 — `devtoprod:images` | | `dockerPublish` | Step 7 — `prod:docker:publish` (auto tag: `YYYYMMDD-HHMMSS-` + push `latest`) | @@ -90,13 +90,13 @@ Use the manual steps below when you need a **partial** release or want to inspec ## What changed → which steps to run -Run only the steps that match what you changed. Steps are independent except that **code** needs a rebuilt image and **schema changes** must be applied before a **data** restore. +Run only the steps that match what you changed. Steps are independent except that **code** needs a rebuilt image and **prod schema must match the dev backup** before a data restore (Step 4 before Step 5). | You changed… | Required steps | |--------------|----------------| | Application code (client/server) | 0 → 1 → 7 → 8 → 9 | | DB schema (new `db/migrate-*.sql`, `schema.sql`) | 0 → 1 → 3 → **4** → (5 if data too) → 8 → 9 | -| Catalog data (movements, artists, paintings, influences, bios) | 0 → 2 → 3 → 5 → 9 | +| Catalog data (movements, artists, paintings, influences, bios) | 0 → 2 → 3 → **4** → 5 → 9 | | Image files (new/replaced paintings, portraits) | 0 → 2 → 3 → 5 → 6 → 9 | | Thumbnails only (rebuild from existing full images) | 0 → 2 → 3 → 5 → 6 → 9 | | Everything (typical weekly release) | 0 → 1 → 2 → 3 → 4 → 5 → 6 → 7 → 8 → 9 | @@ -364,6 +364,7 @@ Keep at least the most recent `gallery_prod_data_*.txt` from Step 3 so a data ro - Prod DB scripts read **only** `infra/docker/.env.prod`; dev scripts refuse `_prod` database names. - Prod restore/backup require typing `yes` (or `CONFIRM_PROD=1` / `autoConfirm` in release config). - `npm run dev:migrate` targets whatever `DB_NAME` is set — always `Remove-Item Env:\DB_NAME` after Step 4 so later commands stay on dev. +- **`restoreProd` safety:** if `migrateProdSchema` is disabled in config but `restoreProd` is enabled, the orchestrator runs Step 4 first so prod has columns/tables present in the dev backup (e.g. `updated_at`, `entity_translations`). - Deploy-related npm scripts and PowerShell helpers print a final **`===== SUCCESS: … =====`** or **`===== FAILED: … =====`** banner as the last line of output. ## Troubleshooting @@ -373,6 +374,7 @@ Keep at least the most recent `gallery_prod_data_*.txt` from Step 3 so a data ro | `/api/bounds` returns `{"min_year":null,"max_year":null}` on prod | `gallery_prod` has schema but no data | Run Step 5 (restore); confirm rows with a count query on `art_movements` / `paintings` | | `robocopy ... ERROR 5 (0x00000005) Access is denied` on Step 6 | SMB share not mapped/authenticated in this Windows session | `net use \\192.168.10.122\Gallery /user:YOUR_TRUENAS_USER`, verify `Test-Path`, then re-run. If still denied after auth, fix TrueNAS dataset ACL (`chown -R 1001:1001`, grant the SMB user write) | | Restore: `permission denied to set parameter "session_replication_role"` | DB role is not superuser | Handled automatically (multi-pass insert). For the fast path, superuser runs `GRANT SET ON PARAMETER session_replication_role TO gallery;` | +| Restore: `column "updated_at" of relation … does not exist` | Prod schema behind dev backup | Run Step 4 before Step 5; `full` profile enables `migrateProdSchema` by default; partial config with `restoreProd` alone also auto-migrates | | Restore: `unterminated quoted string` | Old parser split multi-line values (bios) | Fixed — statements are accumulated until quotes balance; update to latest `scripts/restore-db-data.js` | | `git push` → `Failed to authenticate user` (Gitea) | Git Credential Manager cached an expired token | Clear it: `"protocol=https`nhost=gitea.mysuperlab.netcraze.pro`n" \| git credential reject`, then push again to re-prompt | | Prod tables empty after a failed restore | Restore truncates **before** inserting; a mid-run error leaves tables empty | Dev is untouched — just re-run Step 5 | diff --git a/Documentation/i18n-russian.md b/Documentation/i18n-russian.md index 1eac0f9..e8895ee 100644 --- a/Documentation/i18n-russian.md +++ b/Documentation/i18n-russian.md @@ -79,19 +79,22 @@ API (curator-only): see [API.md](API.md#translations-curator). Public endpoints accept `?locale=ru` or `Accept-Language: ru`. Responses include `"locale": "ru"` on catalog payloads; field names unchanged — values are already resolved. -Search matches canonical text **or** published Russian aliases. +Search matches canonical text **or** published Russian **name** / **title** aliases (partial index `entity_translations_search_alias_idx`; long `bio_full` text is stored but not btree-indexed). --- ## Prod rollout -1. `npm run dev:migrate` on dev; prod schema: `npm run harmonize:schema` (dev → prod only) -2. Seed Russian on dev: `npm run dev:fetch-artist-bios-ru` -3. Curator review + publish -4. `npm run harmonize` to sync `entity_translations` to prod (or full promote if preferred) +1. `npm run dev:migrate` on dev; curator review + publish Russian rows on dev +2. **`npm run devtoprod:release`** (`full` profile) — includes Step 4 schema migrate + Step 5 restore; or `npm run harmonize:schema` then harmonize/sync if you prefer merge over full promote +3. After restore, verify locale toggle on https://gallery.mysuperlab.netcraze.pro `entity_translations` is included in [`harmonize-db.js`](../scripts/harmonize-db.js) catalog sync. +### `dev:fetch-artist-bios-ru` index limit + +Early `migrate-i18n.sql` indexed all `value` text; very long Russian bios (e.g. Michelangelo) could fail with `index row size … exceeds btree maximum`. Current migration uses a **partial** index on `name` and `title` only (`char_length(value) <= 512`). Re-run `npm run dev:migrate` (or prod Step 4) after pulling this fix, then re-run `npm run dev:fetch-artist-bios-ru` for any artists that failed. + --- ## npm scripts diff --git a/Documentation/setup.md b/Documentation/setup.md index 8a6f000..6afe481 100644 --- a/Documentation/setup.md +++ b/Documentation/setup.md @@ -272,6 +272,8 @@ After clone: copy `.env.example` → `.env`, install dependencies, run [one-time | **Back to Timeline** returns to gallery / previous wing | Stale client build | Pull latest client — `goToTimelineHome()` unmounts the hall and resets timeline zoom | | Catalog search dropdown hidden under timeline | Stale client CSS | Rebuild client — `.site-header` uses `z-index: 110` above the sticky timeline bar | | Catalog search returns empty / 500 | Search indexes missing | Run `npm run dev:migrate` (includes `migrate-search.sql`) or `npm run dev:migrate:search`; restart API | +| `fetch-artist-bios-ru` fails: `index row size … exceeds btree maximum` | Old `entity_translations` index on all `value` text | `npm run dev:migrate` (partial search index on name/title only), then re-run `npm run dev:fetch-artist-bios-ru` | +| Russian UI shows English catalog names | Translations not published | Curator → Translations → Publish; public API serves only `status = published` | | Frame still black after **Checked** | Gallery session not synced | Re-enter hall or toggle debug **Checked** from detail with gallery open behind overlay | | Duplicate works in gallery / timeline | Double import or variant Wikipedia titles | `npm run dev:find-duplicates`; merge or delete spare rows manually | | **Failed to load movement gallery** / `Cannot GET /api/movements/:id/gallery` | Stale server process missing route | Restart `npm run dev:web` or `npm run dev:server` after pulling API changes | diff --git a/infra/deploy/devtoprod.config.example.json b/infra/deploy/devtoprod.config.example.json index 91d14b8..b076b6f 100644 --- a/infra/deploy/devtoprod.config.example.json +++ b/infra/deploy/devtoprod.config.example.json @@ -21,7 +21,6 @@ "message": "Release: weekly deploy", "stageAll": true }, - "schemaChanged": false, "backupFile": "", "smb": { "host": "192.168.10.122", diff --git a/infra/scripts/deploy-dev-to-prod.ps1 b/infra/scripts/deploy-dev-to-prod.ps1 index 3928050..af73ec8 100644 --- a/infra/scripts/deploy-dev-to-prod.ps1 +++ b/infra/scripts/deploy-dev-to-prod.ps1 @@ -127,7 +127,7 @@ function Invoke-Step { } function Get-ProfileSteps { - param([string]$Profile, [bool]$SchemaChanged) + param([string]$Profile) $base = @{ validateBuild = $false @@ -367,8 +367,7 @@ $raw = Get-Content -LiteralPath $Config -Raw -Encoding UTF8 $cfg = $raw | ConvertFrom-Json $profile = if ($cfg.profile) { $cfg.profile } else { 'full' } -$schemaChanged = [bool]$cfg.schemaChanged -$steps = Merge-Steps (Get-ProfileSteps -Profile $profile -SchemaChanged $schemaChanged) $cfg.steps +$steps = Merge-Steps (Get-ProfileSteps -Profile $profile) $cfg.steps if ($cfg.autoConfirm) { $env:CONFIRM_PROD = '1'