docs initial

2026-04-06 01:44:58 +03:00
parent 2a344ad588
commit 694b8bc77c
84 changed files with 6922 additions and 298 deletions
@@ -0,0 +1,9 @@
+# oven/bun Base Image Has Existing Non-Root User
+
+**Applies when:** adding non-root user to any Dockerfile that uses `oven/bun` as base image (Remotion service, or future Bun-based services).
+
+- `oven/bun:1.3.10` ships with a `bun` user (UID 1000) and `bun` group (GID 1000).
+- Home directory is `/home/bun`, shell is `/bin/sh`.
+- Do NOT create a new `app` user with `groupadd`/`useradd` -- GID 1000 collision causes `groupadd: GID '1000' already exists` build failure.
+- Instead: `RUN chown -R bun:bun /app` then `USER bun`.
+- Verified: container runs as `uid=1000(bun) gid=1000(bun)`, `/app/out` is writable.
@@ -0,0 +1,9 @@
+# cap_drop: ALL Breaks redis-alpine Startup
+
+**Applies when:** adding Linux capability restrictions to Docker Compose services, especially Redis or any image that switches users at startup.
+
+- `redis:7-alpine` entrypoint calls `gosu redis` to drop from root to the `redis` user.
+- `gosu` requires `SETUID` and `SETGID` capabilities to switch users.
+- `cap_drop: ALL` without `cap_add: [SETUID, SETGID]` prevents the user switch, causing immediate container exit.
+- The container logs show no error -- it just exits silently with code 1.
+- Decision (2026-03-24): removed all cap_drop/cap_add from both compose files. For a dev-only local stack, the complexity and debugging cost outweigh the security benefit. Revisit for production deployment with proper per-service capability analysis.
@@ -0,0 +1,18 @@
+# Docker Infrastructure Audit Findings
+
+**Applies when:** implementing any Docker fixes, setting up CI/CD, preparing for production deployment, or reviewing PRs that touch Dockerfiles or compose files.
+
+- Backend `.dockerignore` is missing `.env` exclusion -- security risk for future `COPY . .` changes.
+- Backend `.gitignore` is missing `.env` exclusion -- latent secret leak risk.
+- MinIO image is unpinned (`minio/minio` with no tag) -- all others are pinned.
+- No resource limits on any service. Remotion needs 4GB+ for Chromium/FFmpeg renders.
+- Health checks exist only on `db` and `redis`. Missing on `minio`, `api`, `worker`, `remotion`.
+- API health check requires a `GET /api/health/` endpoint (may not exist yet -- needs backend team).
+- No restart policies on any service.
+- Both Dockerfiles run as root -- non-root user should be added to `prod` stages (dev stage has bind-mount permission complications).
+- `build-essential` is in the `base` stage, bloating the prod image by ~200MB. Move to `deps` stage only.
+- Remotion Dockerfile missing BuildKit apt cache mounts (backend has them, remotion does not).
+- Environment variables duplicated between `api` and `worker` (14 identical vars) -- use `x-backend-env` YAML anchor.
+- Worker is missing `JWT_SECRET_KEY` that API has.
+- No CI/CD pipeline exists at all -- zero automation.
+- No frontend Dockerfile -- needs `output: 'standalone'` in next.config.mjs first.
@@ -0,0 +1,16 @@
+# Docker Dev vs Prod Stage Split
+
+**Applies when:** modifying the backend Dockerfile or docker-compose.yml, debugging import issues in containers, or setting up CI/CD image builds.
+
+- Dockerfile has 4 stages: `base` (runtime only: ffmpeg) -> `deps` (build-essential + Python deps) -> `dev` (compose target) -> `prod` (CI/CD target).
+- `base` has only runtime deps (ffmpeg). `deps` adds build-essential for C extension compilation (psycopg2, etc.).
+- `dev` inherits from `deps` (has build-essential -- fine for dev). `prod` inherits from `base` (no build-essential) and copies the pre-compiled `.venv` from `deps` via `COPY --from=deps /app/.venv /app/.venv`.
+- The `dev` stage does NOT run `uv sync` for the project itself. It relies on `PYTHONPATH=/app` + bind-mounted source at `/app/cpv3`. This avoids the stale editable-install-vs-bind-mount conflict.
+- The `prod` stage uses `UV_LINK_MODE=copy` and `uv sync --frozen --no-dev` to create a fully self-contained image with code baked in.
+- `prod` stage runs as non-root user `app` (uid/gid 1000). Dev stage stays as root due to bind-mount permission complications.
+- `docker-compose.yml` targets the `dev` stage via `build.target: dev`.
+- For CI/CD, build the `prod` stage: `docker build --target prod -t cpv3-backend:prod .`
+- The `cpv3` project is declared as `source = { editable = "." }` in `uv.lock`. With `UV_LINK_MODE=copy`, uv creates a `.pth` editable finder that maps imports to `/app/cpv3`. In dev, the bind mount overlays this directory, making the installed copy irrelevant but not harmful. The `dev` stage eliminates this ambiguity entirely.
+- `watchfiles` CLI (from `uvicorn[standard]`) is used for worker auto-restart: `watchfiles --filter python 'dramatiq ...' /app/cpv3`.
+- OrbStack propagates filesystem events natively. Docker Desktop on macOS may need `WATCHFILES_FORCE_POLLING=true`.
+- Worker REMOTION_SERVICE_URL was fixed from `http://localhost:8001` to `http://remotion:3001`.
@@ -0,0 +1,11 @@
+# MinIO Version Pinning and xl Meta Compatibility
+
+**Applies when:** changing MinIO image tag, debugging MinIO startup failures, or resetting MinIO volumes.
+
+- MinIO does NOT support downgrades. Once data is written by a newer version, older versions cannot read it.
+- The xl meta version is a storage format version embedded in MinIO's data files. Version 3 was introduced in 2025 releases.
+- Previous pin `RELEASE.2024-11-07T00-52-20Z` could not read xl meta v3 data written by a `latest` pull.
+- Current pin: `RELEASE.2025-09-07T16-13-09Z` -- the last free release on Docker Hub before MinIO stopped publishing (Oct 2025).
+- `curl` was removed from MinIO Docker images after `RELEASE.2023-10-25T06-33-25Z` (UBI micro base). Healthcheck must use `mc ready local` instead of `curl -f`.
+- If MinIO volume data is truly unrecoverable (corrupted, not just version mismatch), the nuclear option is `docker volume rm cpv3_minio` -- but this destroys all stored media files.
+- MinIO GitHub repo was archived Feb 2026. Future images may need to come from alternative sources (alpine/minio, self-build).
@@ -0,0 +1,11 @@
+# Network Segmentation in Docker Compose
+
+**Applies when:** modifying network topology, adding new services, debugging inter-service connectivity, or reviewing compose files.
+
+- Two custom bridge networks: `db-net` (data stores) and `app-net` (application tier).
+- `db` and `redis`: only on `db-net` -- not reachable from app-net-only services.
+- `minio`: on both `db-net` and `app-net` -- accessible from all services including Remotion.
+- `api` and `worker`: on both `db-net` and `app-net` -- can reach data stores and be reached by Remotion.
+- Remotion service joins `cofee_backend_app-net` (external network) -- can reach `minio` and `api`/`worker`, but NOT `db` or `redis` directly.
+- Remotion compose references `REDIS_URL: redis://redis:6379/0` in its environment -- this will NOT resolve since `redis` is only on `db-net`. If Remotion needs Redis access, Redis must be added to `app-net` as well.
+- The old default network (`cofee_backend_default`) is no longer created. Any external references to it must be updated to `cofee_backend_app-net`.