Add opt-in PostgreSQL support alongside default SQLite

SQLite stays the default (fine for a small single-cafe deployment); this
makes Postgres a supported alternative via DATABASE_URL for clients who
outgrow SQLite's single-writer lock (multiple concurrent admin users,
high write volume).

- requirements.txt: add psycopg2-binary (Postgres driver for SQLAlchemy).
- bober_bbq/migrate.py: the admin_users.created_at/last_login_at ALTER
  TABLE DDL used the type name DATETIME, which SQLite accepts (it only
  has type affinity) but Postgres doesn't recognize as a real type — use
  TIMESTAMP instead, valid on both.
- bober_bbq/admin/inventory.py: the ingredient-name duplicate check used
  db.func.lower() in SQL, which is ASCII-only on SQLite and silently
  fails to match Cyrillic names (same bug already fixed this session in
  admin/suppliers.py's supplier-name dedup) — compare in Python instead.
  Not a Postgres crash risk (Postgres's LOWER() is Unicode-aware), but
  worth fixing for consistency since moving to Postgres would otherwise
  silently change this check's behavior.
- bober_bbq/utils/gdrive_backup.py and admin/system.py were audited for
  SQLite-specific behavior (PRAGMA integrity_check, direct file
  copy/restore) but already guard every such path behind
  `uri.startswith("sqlite:///")` / `_sqlite_path()` checks that
  gracefully no-op with a clear message on Postgres — no changes needed.
- scripts/migrate_sqlite_to_postgres.py (new): standalone, manually-run,
  one-off tool for actually moving a specific client's data off SQLite.
  Reuses the app's own SQLAlchemy models (db.metadata) rather than raw
  table introspection, copies rows in FK-safe order, refuses to run
  against a non-empty target unless --truncate is passed, and resets
  Postgres auto-increment sequences after copying explicit ids. Not
  wired into run_web.py or any automatic startup path.
- docs/DEPLOYMENT.md: new section on when/how to use Postgres instead of
  SQLite, standing up a Postgres instance, the DATABASE_URL format, and
  when/how to run the migration script.

Verified: create_app() + db.create_all() + migrate() + seed() (including
the ALTER-TABLE-ADD-COLUMN code path exercised against a DB missing
those columns) still work cleanly against a fresh temp SQLite DB. The
migration script's row-copy/FK-ordering/truncate/non-empty-guard logic
was verified end-to-end using SQLite as a stand-in target, since no
Postgres server is available in this environment — the Postgres-specific
pieces (actual connection, pg_get_serial_sequence/setval) are
unverified beyond code review.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
byrsapty 2026-08-27 13:31:12 +03:00
parent b636d0584a
commit 1ede4b9692
5 changed files with 341 additions and 3 deletions

View file

@ -81,7 +81,12 @@ def inventory_new():
if not name:
flash("Введіть назву інгредієнта", "error")
return redirect(url_for("admin.inventory_list"))
if Ingredient.query.filter(db.func.lower(Ingredient.name) == name.lower()).first():
# SQLite's LOWER() is ASCII-only, so it silently fails to match
# Ukrainian/Russian names — compare in Python instead (same fix as
# admin/suppliers.py's supplier-name dedup). The ingredient list is
# small, so a full scan here is not a concern.
name_norm = name.lower()
if any(i.name.lower() == name_norm for i in Ingredient.query.all()):
flash(f"Інгредієнт «{name}» вже існує", "error")
return redirect(url_for("admin.inventory_list"))

View file

@ -23,10 +23,16 @@ def _add_column_if_missing(table: str, column: str, ddl: str) -> None:
def migrate() -> None:
# Note: DDL below uses TIMESTAMP rather than DATETIME — SQLite accepts
# any type name (it only has type *affinity*), but Postgres requires a
# real type, and DATETIME isn't one there. TIMESTAMP is a real Postgres
# type and SQLite treats it identically to DATETIME (both get numeric/
# text affinity based on the "date"/"time" substrings it doesn't even
# need), so it's the one spelling that's actually valid on both.
_add_column_if_missing("orders", "cancel_reason", "cancel_reason VARCHAR(256)")
_add_column_if_missing("admin_users", "role", "role VARCHAR(16) DEFAULT 'admin'")
_add_column_if_missing("admin_users", "created_at", "created_at DATETIME")
_add_column_if_missing("admin_users", "last_login_at", "last_login_at DATETIME")
_add_column_if_missing("admin_users", "created_at", "created_at TIMESTAMP")
_add_column_if_missing("admin_users", "last_login_at", "last_login_at TIMESTAMP")
_add_column_if_missing("orders", "promo_code", "promo_code VARCHAR(32)")
_add_column_if_missing("orders", "promo_discount_amount", "promo_discount_amount INTEGER DEFAULT 0")
_add_column_if_missing("orders", "rating", "rating INTEGER")

View file

@ -199,3 +199,99 @@ sudo systemctl restart <SERVICE_WEB> <SERVICE_BOT>
4. Адмінка → Меню: видалити демо-меню (Шашлик/BBQ категорії й товари, що
приходять із `seed.py` при першому запуску порожньої БД) і наповнити
реальним асортиментом клієнта.
## 10. Використання PostgreSQL замість SQLite
За замовчуванням проєкт працює на SQLite (`instance/bober_bbq.db`) — для
одного невеликого кафе цього досить: файл, який не треба адмініструвати
окремо, і бекап якого — просто `cp`. Це лишається дефолтом і для нових
клієнтів; Postgres — свідомий опт-ін для конкретного випадку, не апгрейд
"за замовчуванням".
**Коли варто переходити на Postgres:** не через обсяг даних (навіть кілька
років замовлень одного кафе — це нічого для будь-якої з двох СУБД), а через
**одночасний запис із кількох джерел**. SQLite дозволяє лише одному
писачу за раз (інші запити на запис чекають або отримують `database is
locked`); для одного кафе з одним-двома адмінами й ботом це непомітно, але
стає проблемою, якщо в клієнта:
- кілька адміністраторів одночасно активно редагують меню/склад/замовлення
(не просто переглядають — саме одночасні записи);
- дуже високий потік замовлень (кілька точок продажу на одному бекенді,
інтеграція з зовнішньою системою, що постійно пише в БД).
**Як підняти Postgres.** Найпростіше — окремий контейнер поруч із
systemd-юнітами застосунку (не потребує `apt install postgresql*` і
окремого адміністрування пакета в ОС):
```bash
sudo apt install -y docker.io docker-compose-plugin
```
`docker-compose.postgres.yml` (покласти поруч із `$INSTALL_DIR`, не в git):
```yaml
services:
db:
image: postgres:16-alpine
restart: unless-stopped
environment:
POSTGRES_DB: bober_bbq
POSTGRES_USER: bober_bbq
POSTGRES_PASSWORD: <згенерований пароль>
volumes:
- pgdata:/var/lib/postgresql/data
ports:
- "127.0.0.1:5432:5432" # тільки localhost — бекенд і Postgres на одній машині
volumes:
pgdata:
```
```bash
docker compose -f docker-compose.postgres.yml up -d
```
Рівноцінна альтернатива без Docker — пакет дистрибутива:
```bash
sudo apt install -y postgresql
sudo -u postgres createuser bober_bbq --pwprompt
sudo -u postgres createdb bober_bbq --owner=bober_bbq
```
**`DATABASE_URL`** (у `.env`, замінює дефолтний SQLite-шлях —
`bober_bbq/config.py` читає цю змінну напряму):
```
DATABASE_URL=postgresql+psycopg2://bober_bbq:<пароль>@localhost/bober_bbq
```
`psycopg2-binary` (Postgres-драйвер для SQLAlchemy) уже в
`requirements.txt` — окремо встановлювати не треба, `pip install -r
requirements.txt` (крок 2 / `deploy.sh`) підхопить його завжди, незалежно
від того, яку БД зрештою обрали.
**Перенесення даних наявного клієнта з SQLite.** Тільки для вже працюючого
клієнта, який переїжджає з SQLite на Postgres — новому клієнту досить
одразу вказати Postgres-`DATABASE_URL` перед першим запуском (`seed.py`
наповнить порожню Postgres-БД так само, як наповнив би SQLite). Разова
ручна операція, `scripts/migrate_sqlite_to_postgres.py`:
```bash
sudo systemctl stop <SERVICE_WEB> <SERVICE_BOT> # зупинити запис у SQLite
.venv/bin/python scripts/migrate_sqlite_to_postgres.py \
--sqlite-path instance/bober_bbq.db \
--postgres-url postgresql+psycopg2://bober_bbq:<пароль>@localhost/bober_bbq
# після успішного переносу — прописати DATABASE_URL в .env (вище) і:
sudo systemctl start <SERVICE_WEB> <SERVICE_BOT>
```
Скрипт сам створює схему в порожній Postgres-БД і копіює всі таблиці в
безпечному щодо зовнішніх ключів порядку, використовуючи ORM-моделі
застосунку (а не сирий SQL) — деталі, застереження щодо повторного
запуску й обов'язковий крок скидання Postgres-послідовностей автоінкременту
описані в докстрінгу самого файлу (`--help`). Не запускається автоматично
й ніяк не задіяний у `run_web.py` — лише вручну, один раз, при фактичному
переїзді конкретного клієнта.

View file

@ -4,6 +4,7 @@ Flask-SQLAlchemy>=3.1,<4
Flask-Login>=0.6,<0.7
Flask-WTF>=1.2,<2
SQLAlchemy>=2.0,<3
psycopg2-binary>=2.9,<3 # optional: only imported when DATABASE_URL points at Postgres instead of the default SQLite
python-dotenv>=1.0
requests>=2.32
APScheduler>=3.11,<4

View file

@ -0,0 +1,230 @@
"""One-off MANUAL migration: copy all data from the existing SQLite database
into a fresh PostgreSQL database.
This is a standalone tool for a human to run by hand, once, when actually
moving a specific client's data off SQLite (e.g. because they've outgrown
SQLite's single-writer lock with multiple concurrent admin users). It is
NOT run automatically by the app, NOT wired into run_web.py's startup path,
and has nothing to do with bober_bbq/migrate.py (that module only ever adds
missing *columns* to whatever database is already configured, on every
startup it never moves data between databases).
It reuses the app's own SQLAlchemy models (bober_bbq.models, via
db.metadata) instead of hand-writing table definitions or introspecting
raw SQL that way it automatically stays in sync with the schema, and
SQLAlchemy's per-column type handling (Boolean, Date, DateTime, ...)
converts values correctly between SQLite's and Postgres's native
representations for you.
Usage (run from the repo root):
python scripts/migrate_sqlite_to_postgres.py \\
--sqlite-path instance/bober_bbq.db \\
--postgres-url postgresql+psycopg2://user:pass@localhost/bober_bbq
Both flags can instead come from env vars SQLITE_DB_PATH / POSTGRES_URL.
Requires psycopg2-binary (already in requirements.txt) and a target Postgres
database that already exists (e.g. `createdb bober_bbq`) this script
creates the schema in it itself (a plain `db.create_all()`, safe to run
against an empty database).
IMPORTANT safety / idempotency notes:
- This script REFUSES to run if the target already has rows in any table,
to avoid partial/duplicate data from an accidental second run. Pass
--truncate to instead wipe the target's tables first and start clean —
only ever do that against a scratch/throwaway Postgres database, never
against one already serving real traffic.
- It is therefore safe to *re-run* (each run either refuses, or with
--truncate starts from a clean slate) but it never merges/upserts
against a target that already has *different* data you want to keep.
Treat it as "run once against a fresh Postgres DB", not as a sync tool.
- After copying, it resets each table's Postgres auto-increment sequence
to MAX(id)+1. This step matters: rows are copied with their original
SQLite ids, so without a sequence reset, the very first new row the
app inserts after cutover (e.g. the next order) would collide with a
migrated row's id.
After migration: point DATABASE_URL at the Postgres URL in .env and restart
the app. bober_bbq/migrate.py's startup column-migrations are dialect-
agnostic and will run harmlessly against the already-current schema.
"""
from __future__ import annotations
import argparse
import os
import sys
from pathlib import Path
# Allow running as `python scripts/migrate_sqlite_to_postgres.py` from the
# repo root without installing the package.
sys.path.insert(0, str(Path(__file__).resolve().parent.parent))
from sqlalchemy import Table, create_engine, func, insert, select, text # noqa: E402
from sqlalchemy.engine import Engine # noqa: E402
from bober_bbq import models # noqa: E402,F401 (imported for side effect: registers all tables on db.metadata)
from bober_bbq.extensions import db # noqa: E402
def parse_args() -> argparse.Namespace:
parser = argparse.ArgumentParser(
description=__doc__,
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument(
"--sqlite-path",
default=os.getenv("SQLITE_DB_PATH", ""),
help="Path to the existing SQLite .db file (or env SQLITE_DB_PATH)",
)
parser.add_argument(
"--postgres-url",
default=os.getenv("POSTGRES_URL", ""),
help="Target Postgres SQLAlchemy URL, e.g. "
"postgresql+psycopg2://user:pass@localhost/bober_bbq (or env POSTGRES_URL)",
)
parser.add_argument(
"--truncate",
action="store_true",
help="Delete any existing rows in the target's tables first, instead of refusing to "
"run against a non-empty database. Only use this against a scratch/throwaway "
"Postgres database you're prepared to wipe.",
)
parser.add_argument("--yes", action="store_true", help="Skip the interactive confirmation prompt")
return parser.parse_args()
def _confirm(sqlite_path: Path, postgres_url: str) -> None:
print(f"Source (SQLite): {sqlite_path}")
print(f"Target (Postgres): {postgres_url}")
reply = input("Copy ALL data from source into target? [y/N] ").strip().lower()
if reply != "y":
print("Aborted.")
sys.exit(1)
def _reject_nonempty_target(target: Engine, tables: list[Table], allow_truncate: bool) -> None:
with target.connect() as conn:
nonempty = []
for table in tables:
count = conn.execute(select(func.count()).select_from(table)).scalar_one()
if count:
nonempty.append((table.name, count))
if not nonempty:
return
if not allow_truncate:
print("Target database already has data:", file=sys.stderr)
for name, count in nonempty:
print(f" - {name}: {count} row(s)", file=sys.stderr)
print(
"\nRefusing to copy into a non-empty database — this would violate primary-key/"
"unique constraints and could mix in stale data. Re-run with --truncate to wipe "
"these tables first (only against a scratch database!), or point --postgres-url "
"at a fresh one.",
file=sys.stderr,
)
sys.exit(1)
print("--truncate given: wiping existing rows in the target's tables first...")
with target.begin() as conn:
# Children before parents, so FK constraints don't block the deletes.
for table in reversed(tables):
conn.execute(table.delete())
def _copy_all_rows(source: Engine, target: Engine, tables: list[Table]) -> int:
total = 0
with source.connect() as sconn, target.begin() as tconn:
for table in tables:
# Small-business scale data (single cafe) — loading a whole
# table into memory at once is not a concern here.
rows = [dict(row) for row in sconn.execute(select(table)).mappings().all()]
if not rows:
print(f" {table.name}: 0 rows")
continue
tconn.execute(insert(table), rows)
total += len(rows)
print(f" {table.name}: {len(rows)} row(s) copied")
return total
def _reset_sequences(target: Engine, tables: list[Table]) -> None:
"""Rows were copied with their original SQLite ids, so any Postgres
SERIAL/IDENTITY sequence backing a single-column integer primary key is
still sitting at its start value. Advance each one to MAX(id) so the
next app-generated insert doesn't collide with a migrated row. Tables
with a natural (non-generated) primary key Setting.key,
TelegramUser.telegram_id simply have no such sequence and are
skipped automatically. No-ops entirely if --postgres-url didn't
actually point at Postgres (pg_get_serial_sequence()/setval() are
Postgres-only functions)."""
if target.dialect.name != "postgresql":
return
with target.connect() as conn:
for table in tables:
pk_cols = list(table.primary_key.columns)
if len(pk_cols) != 1 or not isinstance(pk_cols[0].type, (db.Integer, db.BigInteger)):
continue
pk = pk_cols[0]
seq_name = conn.execute(
text("SELECT pg_get_serial_sequence(:tbl, :col)"),
{"tbl": table.name, "col": pk.name},
).scalar_one_or_none()
if not seq_name:
continue # not a generated column (e.g. TelegramUser.telegram_id)
max_id = conn.execute(select(func.max(pk))).scalar_one_or_none()
conn.execute(
text("SELECT setval(:seq, :val, :is_called)"),
{"seq": seq_name, "val": max_id or 1, "is_called": max_id is not None},
)
conn.commit()
def main() -> None:
args = parse_args()
if not args.sqlite_path:
print("Missing --sqlite-path (or env SQLITE_DB_PATH)", file=sys.stderr)
sys.exit(2)
if not args.postgres_url:
print("Missing --postgres-url (or env POSTGRES_URL)", file=sys.stderr)
sys.exit(2)
if not args.postgres_url.startswith("postgresql"):
print(f"--postgres-url doesn't look like a Postgres URL: {args.postgres_url}", file=sys.stderr)
sys.exit(2)
sqlite_path = Path(args.sqlite_path).resolve()
if not sqlite_path.is_file():
print(f"SQLite file not found: {sqlite_path}", file=sys.stderr)
sys.exit(2)
if not args.yes:
_confirm(sqlite_path, args.postgres_url)
source = create_engine(f"sqlite:///{sqlite_path}")
target = create_engine(args.postgres_url)
# sorted_tables is topologically sorted by foreign key so parents are
# always copied (and, reversed, deleted) before/after their children —
# exactly the FK-safe order this migration needs.
tables = db.metadata.sorted_tables
print("Creating schema on target (no-op for tables that already exist)...")
db.metadata.create_all(target)
_reject_nonempty_target(target, tables, args.truncate)
print("Copying rows...")
total_rows = _copy_all_rows(source, target, tables)
print("Resetting Postgres auto-increment sequences...")
_reset_sequences(target, tables)
print(f"\nDone: {total_rows} row(s) copied across {len(tables)} tables.")
print("Next: point DATABASE_URL at the Postgres URL in .env and restart the app.")
if __name__ == "__main__":
main()