Keep prod users local and fix post-restore id sequences.

Prod restore skips users/session/audit, syncs serial sequences after load, and user create re-aligns users_id_seq so new accounts are not misreported as duplicates.

Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
Danila Khodjaef
2026-08-05 19:26:52 +03:00
co-authored by Cursor
parent 8a68e98258
commit cfee69c9a6
7 changed files with 83 additions and 9 deletions
+1 -1
View File
@@ -251,7 +251,7 @@ Append-only log of staff mutations (fix/clear/upload/delete, checkup flags, tran
**Logged `action` values:** `painting.fix_image`, `painting.clear_image`, `painting.upload_image`, `painting.delete`, `painting.checkup_flags`, `painting.update_curator_notes`, `artist.fix_portrait`, `artist.clear_portrait`, `artist.upload_portrait`, `artist.checkup_flags`, `translation.upsert`, `translation.publish`, `influence.create`, `influence.update`, `influence.delete`, `influence.import`, `tour.create`, `tour.update`, `tour.delete`, `tour.stops`, `user.create`, `user.update`, `user.reset_password`.
Admins browse this table in the app (**Activity** / `GET /api/audit*`). Each environments API uses its own DB (`gallery_dev` vs `gallery_prod`); audit history is not synced by harmonize/devtoprod.
Admins browse this table in the app (**Activity** / `GET /api/audit*`). Each environments API uses its own DB (`gallery_dev` vs `gallery_prod`); **`users`**, **`session`**, and audit history are not synced by harmonize/`devtoprod:db:restore`.
Example query in pgAdmin:
+2 -2
View File
@@ -107,7 +107,7 @@ If login fails after changing `CURATOR_PASSWORD` in `.env`, run `npm run dev:res
| Curator | Public browse + assigned permission flags (`images`, `checkup`, `curator_notes`, `translations`, `influences`, `tours`, `users`) |
| Admin | All curator tools + **Users** + **Activity** audit reports |
**Users page:** after admin login, header → **Users** — create/edit staff, reset passwords, disable accounts.
**Users page:** after admin login, header → **Users** — create/edit staff, reset passwords, disable accounts. Prod and dev keep **separate** `users` tables: `devtoprod:db:restore` and harmonize never copy staff accounts. If create fails with a confusing “already exists” after a restore, serial sequences may be lagging — current restore syncs them to `MAX(id)`, and user create re-syncs `users_id_seq` before insert.
**Activity page:** after admin login, header → **Activity** — filterable curator action log (date/time, curator, action, resource, details, IP) plus summary charts. Reads the DB for that environment (`gallery_dev` on devgallery / `npm run dev:web`, `gallery_prod` on prod).
@@ -145,7 +145,7 @@ Run `npm run dev:migrate` against prod DB after first deploy with auth vars set
| `npm run dev:db:backup` | Dev data-only backup → `db/DataBackup/*.txt` + `.zip` |
| `npm run prod:db:backup` | Prod backup (reads `infra/docker/.env.prod`) |
| `npm run dev:db:restore -- --file <path>` | Restore backup into **dev** (truncates tables first; prompts `yes`) |
| `npm run devtoprod:db:restore -- --file <path>` | Restore into **prod** (requires confirmation) |
| `npm run devtoprod:db:restore -- --file <path>` | Restore catalog into **prod** (skips `users` / `session` / `curator_audit_log`; syncs serial sequences; requires confirmation) |
| `npm run harmonize` | Bidirectional catalog DB + image merge by `updated_at` / file mtime — [harmonize-dev-prod.md](harmonize-dev-prod.md) |
| `npm run harmonize:schema` | Apply dev migrations to prod schema only (dev → prod) |
| `npm run harmonize:db` / `harmonize:images` | DB or image merge only (`harmonize:images` also merges artists/paintings checkup flags + image paths, then regenerates thumbs on both sides) |
+4 -3
View File
@@ -244,10 +244,11 @@ The restore loads rows in two ways automatically:
Multi-line values (e.g. artist bios with embedded newlines) are parsed as whole statements, so long text restores correctly.
**Caveats — prod tables are replaced by dev's contents:**
**Caveats — prod catalog tables are replaced by dev's contents:**
- `users` is overwritten. The **dev curator account and password become the prod login**. `curator_audit_log` is **not** synced — prod keeps its existing audit history.
- The `session` table is truncated, so any active prod curator sessions are logged out.
- **`users`**, **`session`**, and **`curator_audit_log`** are **not** truncated or loaded from the backup. Prod staff accounts, passwords, active sessions, and audit history stay as they are on `gallery_prod`.
- Catalog / content tables (`artists`, `paintings`, tours, translations, etc.) are fully replaced by the dev dump.
- After load, serial sequences are reset to `MAX(id)` so new rows (including staff users) do not collide with restored ids.
- The target is guarded: the restore refuses to run unless the database name ends with `_prod` and only reads `infra/docker/.env.prod`.
> **Optional** — to use the faster single-pass load, have the postgres superuser run this once in pgAdmin (role-global, covers dev and prod): `GRANT SET ON PARAMETER session_replication_role TO gallery;`
+1 -1
View File
@@ -117,7 +117,7 @@ npm run infra:db:split-dev-prod
CURATOR_PASSWORD=your-secure-password
```
Omit `SESSION_COOKIE_SECURE` so cookies follow the request scheme (`TRUST_PROXY` + HTTPS → Secure). Set `true`/`false` to force. `npm run dev:migrate` creates auth tables/roles and bootstraps the first **admin** when `users` is empty. Reset that account later with `npm run dev:reset-curator`. Create additional staff via the in-app **Users** page. Admins browse curator actions on **Activity** (`/api/audit`), which always reads the DB named by `DB_NAME` for that environment (`gallery_dev` here; `gallery_prod` on prod). Audit history is not synced by harmonize/devtoprod.
Omit `SESSION_COOKIE_SECURE` so cookies follow the request scheme (`TRUST_PROXY` + HTTPS → Secure). Set `true`/`false` to force. `npm run dev:migrate` creates auth tables/roles and bootstraps the first **admin** when `users` is empty. Reset that account later with `npm run dev:reset-curator`. Create additional staff via the in-app **Users** page. Admins browse curator actions on **Activity** (`/api/audit`), which always reads the DB named by `DB_NAME` for that environment (`gallery_dev` here; `gallery_prod` on prod). **`users`**, **`session`**, and **`curator_audit_log`** are never copied by harmonize or `devtoprod:db:restore` — each env keeps its own staff accounts and audit history.
2. Run:
+1 -1
View File
@@ -112,7 +112,7 @@ Image fetch can take hours if you run it for the entire catalog. The first line
| `npm run devtoprod:images` | Copy `data/images/` → TrueNAS via SMB `Gallery` share |
| `npm run prodto:dev:images` | Copy prod images → dev repo |
| `npm run prodto:dev:db` | Clone `gallery_prod``gallery_dev` |
| `npm run dev:db:backup` / `devtoprod:db:restore` | Dev backup / promote DB to prod |
| `npm run dev:db:backup` / `devtoprod:db:restore` | Dev backup / promote catalog DB to prod (prod restore skips staff/session/audit; syncs sequences) |
| `npm run harmonize` | Bidirectional catalog + image merge (last-write-wins) — [harmonize-dev-prod.md](harmonize-dev-prod.md) |
| `npm run prod:build` | Build production SPA into `client/dist` |
| `npm run dev:start` | API + static SPA on `HOST`:`PORT` (uses root `.env`) |
+47 -1
View File
@@ -19,7 +19,47 @@ const { printCliResult } = require('./lib/cli-result');
const { Client } = pg;
// Prod restore keeps these tables untouched (no TRUNCATE, no INSERT from dev backup).
const PROD_RESTORE_SKIP_TABLES = new Set(['curator_audit_log']);
// Staff accounts, sessions, and audit history stay env-local — never overwrite from gallery_dev.
const PROD_RESTORE_SKIP_TABLES = new Set(['users', 'session', 'curator_audit_log']);
function quoteIdent(name) {
return `"${String(name).replace(/"/g, '""')}"`;
}
/** Align serial/identity sequences with MAX(column) after explicit-id INSERTs. */
async function syncSerialSequences(client) {
const { rows } = await client.query(
`SELECT
c.relname AS table_name,
a.attname AS column_name,
pg_get_serial_sequence(format('%I.%I', n.nspname, c.relname), a.attname) AS seq_name
FROM pg_class c
JOIN pg_namespace n ON n.oid = c.relnamespace
JOIN pg_attribute a ON a.attrelid = c.oid AND a.attnum > 0 AND NOT a.attisdropped
JOIN pg_attrdef d ON d.adrelid = c.oid AND d.adnum = a.attnum
WHERE c.relkind = 'r'
AND n.nspname = 'public'
AND pg_get_expr(d.adbin, d.adrelid) LIKE 'nextval(%'`
);
let synced = 0;
for (const row of rows) {
if (!row.seq_name) continue;
await client.query(
`SELECT setval(
$1::regclass,
GREATEST(
1,
COALESCE((SELECT MAX(${quoteIdent(row.column_name)}) FROM ${quoteIdent(row.table_name)}), 1)
),
true
)`,
[row.seq_name]
);
synced += 1;
}
return synced;
}
function getInsertTableName(statement) {
const match = statement.match(/^INSERT INTO "([^"]+)"/i)
@@ -239,6 +279,12 @@ async function main() {
}
}
// Backups insert explicit primary keys; without this, serial nextval() can
// collide with existing ids (e.g. creating a user fails as "already exists").
console.log('Syncing serial sequences to MAX(id)...');
const synced = await syncSerialSequences(client);
console.log(` synced ${synced} sequence(s)`);
await client.end();
console.log(`Restore complete: ${restored} statements into "${dbName}".`);
if (skippedInserts > 0) {
+27
View File
@@ -38,6 +38,17 @@ async function clearUserSessions(userId) {
await pool.query(`DELETE FROM session WHERE (sess->>'userId')::int = $1`, [userId]);
}
/** Keep users_id_seq ahead of existing rows (restore inserts explicit ids). */
async function syncUsersIdSequence() {
await pool.query(
`SELECT setval(
pg_get_serial_sequence('users', 'id'),
GREATEST(1, COALESCE((SELECT MAX(id) FROM users), 1)),
true
)`
);
}
router.use(requirePermission('users'));
router.get('/', async (_req, res) => {
@@ -76,6 +87,16 @@ router.post('/', async (req, res) => {
const passwordHash = await bcrypt.hash(password, 10);
const storedPermissions = role === 'admin' ? ALL_PERMISSIONS : permissions;
const { rows: taken } = await pool.query(
`SELECT username FROM users WHERE LOWER(username) = LOWER($1) LIMIT 1`,
[username]
);
if (taken[0]) {
return res.status(409).json({ error: 'Username already exists' });
}
await syncUsersIdSequence();
const { rows } = await pool.query(
`INSERT INTO users (username, password_hash, role, permissions, is_active)
VALUES ($1, $2, $3, $4::text[], true)
@@ -95,6 +116,12 @@ router.post('/', async (req, res) => {
res.status(201).json({ user: mapUser(rows[0]) });
} catch (err) {
if (err.code === '23505') {
if (err.constraint === 'users_pkey') {
console.error('Users create PK conflict (sequence lag):', err.detail || err.message);
return res.status(409).json({
error: 'Could not allocate user id — retry create (sequence was out of sync)',
});
}
return res.status(409).json({ error: 'Username already exists' });
}
console.error('Users create error:', err.message);