Document prod schema migrate before restore and i18n search index fix.
Release runbook now describes default migrateProdSchema, auto-migrate safety guard, and troubleshooting for updated_at restore failures; i18n docs cover partial entity_translations index. Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Danila Khodjaef
co-authored by
Cursor
Cursor
parent
a2c27ea526
commit
62d7ebbe6a
@@ -34,10 +34,10 @@ For a typical weekly release, use the orchestrator instead of running each step
|
||||
|
||||
2. Edit [`infra/deploy/devtoprod.config.json`](../infra/deploy/devtoprod.config.json) (gitignored — safe for SMB password):
|
||||
- `profile`: `full` | `code` | `data` (base step set; individual `steps` overrides win)
|
||||
- `steps` — per-step overrides; see table below. On **`full`** profile, `migrateProdSchema` is **on by default** (runs `dev:migrate` on `gallery_prod` before restore).
|
||||
- `autoConfirm`: `true` skips restore/image-sync prompts (`CONFIRM_PROD=1`, `-SkipConfirm`)
|
||||
- `autoStartDevStack`: `true` starts `dev:server` (or `dev:web`) automatically when `validateBuild` finds API down
|
||||
- `git.message`, `git.branch` — used when `gitCommitPush` is enabled
|
||||
- `schemaChanged`: `true` enables prod schema migration on `full` profile
|
||||
- `smb.user` / `smb.password` — optional; maps `\\host\share` before image sync
|
||||
- `backupFile` — optional fixed path; otherwise uses the newest `gallery_dev_data_*.txt` after backup
|
||||
|
||||
@@ -67,7 +67,7 @@ npm run devtoprod:release -- -DryRun
|
||||
| `thumbnails` | Step 2 — `devtoprod:thumbnails` |
|
||||
| `backupDev` | Step 3 — `dev:db:backup` |
|
||||
| `backupProd` | Step 3 (rollback) — `prod:db:backup` |
|
||||
| `migrateProdSchema` | Step 4 — `dev:migrate` on `gallery_prod` |
|
||||
| `migrateProdSchema` | Step 4 — `dev:migrate` on `gallery_prod` (enabled on `full` profile; auto-runs before `restoreProd` if you disabled it but left restore on) |
|
||||
| `restoreProd` | Step 5 — `devtoprod:db:restore` |
|
||||
| `syncImages` | Step 6 — `devtoprod:images` |
|
||||
| `dockerPublish` | Step 7 — `prod:docker:publish` (auto tag: `YYYYMMDD-HHMMSS-<gitsha>` + push `latest`) |
|
||||
@@ -90,13 +90,13 @@ Use the manual steps below when you need a **partial** release or want to inspec
|
||||
|
||||
## What changed → which steps to run
|
||||
|
||||
Run only the steps that match what you changed. Steps are independent except that **code** needs a rebuilt image and **schema changes** must be applied before a **data** restore.
|
||||
Run only the steps that match what you changed. Steps are independent except that **code** needs a rebuilt image and **prod schema must match the dev backup** before a data restore (Step 4 before Step 5).
|
||||
|
||||
| You changed… | Required steps |
|
||||
|--------------|----------------|
|
||||
| Application code (client/server) | 0 → 1 → 7 → 8 → 9 |
|
||||
| DB schema (new `db/migrate-*.sql`, `schema.sql`) | 0 → 1 → 3 → **4** → (5 if data too) → 8 → 9 |
|
||||
| Catalog data (movements, artists, paintings, influences, bios) | 0 → 2 → 3 → 5 → 9 |
|
||||
| Catalog data (movements, artists, paintings, influences, bios) | 0 → 2 → 3 → **4** → 5 → 9 |
|
||||
| Image files (new/replaced paintings, portraits) | 0 → 2 → 3 → 5 → 6 → 9 |
|
||||
| Thumbnails only (rebuild from existing full images) | 0 → 2 → 3 → 5 → 6 → 9 |
|
||||
| Everything (typical weekly release) | 0 → 1 → 2 → 3 → 4 → 5 → 6 → 7 → 8 → 9 |
|
||||
@@ -364,6 +364,7 @@ Keep at least the most recent `gallery_prod_data_*.txt` from Step 3 so a data ro
|
||||
- Prod DB scripts read **only** `infra/docker/.env.prod`; dev scripts refuse `_prod` database names.
|
||||
- Prod restore/backup require typing `yes` (or `CONFIRM_PROD=1` / `autoConfirm` in release config).
|
||||
- `npm run dev:migrate` targets whatever `DB_NAME` is set — always `Remove-Item Env:\DB_NAME` after Step 4 so later commands stay on dev.
|
||||
- **`restoreProd` safety:** if `migrateProdSchema` is disabled in config but `restoreProd` is enabled, the orchestrator runs Step 4 first so prod has columns/tables present in the dev backup (e.g. `updated_at`, `entity_translations`).
|
||||
- Deploy-related npm scripts and PowerShell helpers print a final **`===== SUCCESS: … =====`** or **`===== FAILED: … =====`** banner as the last line of output.
|
||||
|
||||
## Troubleshooting
|
||||
@@ -373,6 +374,7 @@ Keep at least the most recent `gallery_prod_data_*.txt` from Step 3 so a data ro
|
||||
| `/api/bounds` returns `{"min_year":null,"max_year":null}` on prod | `gallery_prod` has schema but no data | Run Step 5 (restore); confirm rows with a count query on `art_movements` / `paintings` |
|
||||
| `robocopy ... ERROR 5 (0x00000005) Access is denied` on Step 6 | SMB share not mapped/authenticated in this Windows session | `net use \\192.168.10.122\Gallery /user:YOUR_TRUENAS_USER`, verify `Test-Path`, then re-run. If still denied after auth, fix TrueNAS dataset ACL (`chown -R 1001:1001`, grant the SMB user write) |
|
||||
| Restore: `permission denied to set parameter "session_replication_role"` | DB role is not superuser | Handled automatically (multi-pass insert). For the fast path, superuser runs `GRANT SET ON PARAMETER session_replication_role TO gallery;` |
|
||||
| Restore: `column "updated_at" of relation … does not exist` | Prod schema behind dev backup | Run Step 4 before Step 5; `full` profile enables `migrateProdSchema` by default; partial config with `restoreProd` alone also auto-migrates |
|
||||
| Restore: `unterminated quoted string` | Old parser split multi-line values (bios) | Fixed — statements are accumulated until quotes balance; update to latest `scripts/restore-db-data.js` |
|
||||
| `git push` → `Failed to authenticate user` (Gitea) | Git Credential Manager cached an expired token | Clear it: `"protocol=https`nhost=gitea.mysuperlab.netcraze.pro`n" \| git credential reject`, then push again to re-prompt |
|
||||
| Prod tables empty after a failed restore | Restore truncates **before** inserting; a mid-run error leaves tables empty | Dev is untouched — just re-run Step 5 |
|
||||
|
||||
@@ -79,19 +79,22 @@ API (curator-only): see [API.md](API.md#translations-curator).
|
||||
|
||||
Public endpoints accept `?locale=ru` or `Accept-Language: ru`. Responses include `"locale": "ru"` on catalog payloads; field names unchanged — values are already resolved.
|
||||
|
||||
Search matches canonical text **or** published Russian aliases.
|
||||
Search matches canonical text **or** published Russian **name** / **title** aliases (partial index `entity_translations_search_alias_idx`; long `bio_full` text is stored but not btree-indexed).
|
||||
|
||||
---
|
||||
|
||||
## Prod rollout
|
||||
|
||||
1. `npm run dev:migrate` on dev; prod schema: `npm run harmonize:schema` (dev → prod only)
|
||||
2. Seed Russian on dev: `npm run dev:fetch-artist-bios-ru`
|
||||
3. Curator review + publish
|
||||
4. `npm run harmonize` to sync `entity_translations` to prod (or full promote if preferred)
|
||||
1. `npm run dev:migrate` on dev; curator review + publish Russian rows on dev
|
||||
2. **`npm run devtoprod:release`** (`full` profile) — includes Step 4 schema migrate + Step 5 restore; or `npm run harmonize:schema` then harmonize/sync if you prefer merge over full promote
|
||||
3. After restore, verify locale toggle on https://gallery.mysuperlab.netcraze.pro
|
||||
|
||||
`entity_translations` is included in [`harmonize-db.js`](../scripts/harmonize-db.js) catalog sync.
|
||||
|
||||
### `dev:fetch-artist-bios-ru` index limit
|
||||
|
||||
Early `migrate-i18n.sql` indexed all `value` text; very long Russian bios (e.g. Michelangelo) could fail with `index row size … exceeds btree maximum`. Current migration uses a **partial** index on `name` and `title` only (`char_length(value) <= 512`). Re-run `npm run dev:migrate` (or prod Step 4) after pulling this fix, then re-run `npm run dev:fetch-artist-bios-ru` for any artists that failed.
|
||||
|
||||
---
|
||||
|
||||
## npm scripts
|
||||
|
||||
@@ -272,6 +272,8 @@ After clone: copy `.env.example` → `.env`, install dependencies, run [one-time
|
||||
| **Back to Timeline** returns to gallery / previous wing | Stale client build | Pull latest client — `goToTimelineHome()` unmounts the hall and resets timeline zoom |
|
||||
| Catalog search dropdown hidden under timeline | Stale client CSS | Rebuild client — `.site-header` uses `z-index: 110` above the sticky timeline bar |
|
||||
| Catalog search returns empty / 500 | Search indexes missing | Run `npm run dev:migrate` (includes `migrate-search.sql`) or `npm run dev:migrate:search`; restart API |
|
||||
| `fetch-artist-bios-ru` fails: `index row size … exceeds btree maximum` | Old `entity_translations` index on all `value` text | `npm run dev:migrate` (partial search index on name/title only), then re-run `npm run dev:fetch-artist-bios-ru` |
|
||||
| Russian UI shows English catalog names | Translations not published | Curator → Translations → Publish; public API serves only `status = published` |
|
||||
| Frame still black after **Checked** | Gallery session not synced | Re-enter hall or toggle debug **Checked** from detail with gallery open behind overlay |
|
||||
| Duplicate works in gallery / timeline | Double import or variant Wikipedia titles | `npm run dev:find-duplicates`; merge or delete spare rows manually |
|
||||
| **Failed to load movement gallery** / `Cannot GET /api/movements/:id/gallery` | Stale server process missing route | Restart `npm run dev:web` or `npm run dev:server` after pulling API changes |
|
||||
|
||||
@@ -21,7 +21,6 @@
|
||||
"message": "Release: weekly deploy",
|
||||
"stageAll": true
|
||||
},
|
||||
"schemaChanged": false,
|
||||
"backupFile": "",
|
||||
"smb": {
|
||||
"host": "192.168.10.122",
|
||||
|
||||
@@ -127,7 +127,7 @@ function Invoke-Step {
|
||||
}
|
||||
|
||||
function Get-ProfileSteps {
|
||||
param([string]$Profile, [bool]$SchemaChanged)
|
||||
param([string]$Profile)
|
||||
|
||||
$base = @{
|
||||
validateBuild = $false
|
||||
@@ -367,8 +367,7 @@ $raw = Get-Content -LiteralPath $Config -Raw -Encoding UTF8
|
||||
$cfg = $raw | ConvertFrom-Json
|
||||
|
||||
$profile = if ($cfg.profile) { $cfg.profile } else { 'full' }
|
||||
$schemaChanged = [bool]$cfg.schemaChanged
|
||||
$steps = Merge-Steps (Get-ProfileSteps -Profile $profile -SchemaChanged $schemaChanged) $cfg.steps
|
||||
$steps = Merge-Steps (Get-ProfileSteps -Profile $profile) $cfg.steps
|
||||
|
||||
if ($cfg.autoConfirm) {
|
||||
$env:CONFIRM_PROD = '1'
|
||||
|
||||
Reference in New Issue
Block a user