Add project wiki: overview, architecture, Etsy integration, data import, reports, deployment/CI, troubleshooting

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Architecture.md

@@ -0,0 +1,33 @@
+# Architecture
+
+## Stack
+
+- **Frontend:** React + Redux Toolkit + Tailwind, built with Vite, served by nginx
+- **Backend:** Node.js / Express + Mongoose, port 8080
+- **Database:** MongoDB, per-user isolation (`userId` on every document)
+- **Auth:** JWT (7-day tokens) in localStorage
+- **Hosting:** Docker on the home server, behind Traefik (see [[Deployment & CI]])
+
+## Repo layout
+
+```
+App/
+  client/      React app (pages/, components/, store/ slices, utils/)
+  server/      Express API (routes/, models/, services/, middleware/)
+  Dockerfile   multi-stage: client-build, server-build, production (API), nginx-frontend
+  docker-compose.deploy.yml   the file Dockhand deploys
+  .forgejo/workflows/         CI
+```
+
+## Data model (MongoDB)
+
+- **Order** — `orderNumber` (= Etsy receipt id), items[{title, quantity, price, sku, variant, productId, printingCost, costOfGoods}], `total`, `refundTotal`, `tax`, status, customer, shippingAddress, tracking. Costs are **snapshotted on the order at sync time** so later catalogue edits don't rewrite history.
+- **Product** — title, price, `costOfGoods`, `printingCost`, sku, category, tags, `aliases[]` (past order titles that map here), `etsyListingId`, inventory.
+- **Expense** — category, description, amount, date, vendor, `reference`, taxDeductible. Etsy fees use `reference: etsy-ledger-<entry_id>` for idempotency.
+- **Customer** — name, email, address, etsyUserId, totalOrders, totalSpent.
+- **EtsyConnection** — OAuth tokens, shopId, lastSyncedAt.
+- **EtsySettings** — per-user API keystring, shared secret, redirect URI.
+
+## Matching logic
+
+Order items are matched to catalogue products by, in order: exact title/alias (variant-aware, so `…Size: Large` beats the bare title) → `etsyListingId` (size-disambiguated). Matched titles are saved as aliases and listing IDs are learned, so matching gets more deterministic over time. Unmatched items surface in Settings for resolution.

Data-Import.md

@@ -0,0 +1,22 @@
+# Data Import
+
+Most data now comes from the [[Etsy Integration]]. The Data Import page remains for the one source the Etsy API can't provide and for legacy/manual imports.
+
+## Australia Post (CSV) — the one remaining manual import
+
+Postage bought **directly** from Australia Post (MyPost Business) is only in your AusPost account, not Etsy — so it's imported via CSV on the Data Import page. It creates Shipping & Postage expenses (vendor "Australia Post", keyed by tracking number).
+
+There is **no usable API** for this: MyPost Business has no public transaction-history API, and Etsy's API only knows about labels bought *through Etsy* (those early ones are in the ledger as Shipping & Postage). So the CSV stays.
+
+## Etsy statement CSV (legacy — superseded)
+
+Before the API, fees came from the Etsy statement CSV. That's now replaced by the ledger sync. **Stop importing the statement CSV** to avoid double-counting; use **Settings → Remove Old Fees** once to clear any previously imported ones.
+
+## PDF packing slips (legacy)
+
+The original cost path parsed packing-slip PDFs to match items and capture printing costs. Superseded by API order sync + catalogue matching. Still present but not needed.
+
+## Notes
+
+- Expense imports dedupe on `{reference, vendor, amount, date}`.
+- Orders import via bulk upsert by order number, so re-imports update rather than duplicate.

Deployment-&-CI.md

@@ -0,0 +1,38 @@
+# Deployment & CI
+
+Self-hosted end to end on the home server (Mac Mini, arm64, OrbStack). Code, CI, and the container registry all live on Forgejo at **git.plexultra.com** (migrated off GitHub/GHCR in June 2026).
+
+## Pipeline
+
+1. Commit and push to Forgejo (`origin` is the `forgejo:` SSH alias → 192.168.1.124:2222, LAN only).
+2. **Forgejo Actions** (`.forgejo/workflows/docker-build.yml`) runs on `mac-mini-runner` and builds + pushes two images, natively arm64 (no QEMU):
+   - `git.plexultra.com/dlawler489/etsy-finance-tracker:main` (API, `production` stage)
+   - `git.plexultra.com/dlawler489/etsy-finance-tracker-nginx:main` (nginx, `nginx-frontend` stage)
+   - Both also tagged with the commit SHA (rollback targets).
+3. **Redeploy in Dockhand**, which pulls `docker-compose.deploy.yml` and recreates the stack.
+
+The deploy host must be `docker login git.plexultra.com` (or the packages public) to pull.
+
+## Runner setup (gitea/act_runner in docker-compose)
+
+The runner runs each job in a container and has `/var/run/docker.sock` mounted. Key facts that make the workflow work:
+
+- **No Node in job containers** → can't use JS actions like `actions/checkout`. The workflow is node-free: manual `git init/fetch/checkout` over HTTPS with a token.
+- Job runs in **`container: docker:cli`** (Alpine) → `shell: sh`, and `apk add git` for checkout.
+- act_runner **auto-mounts the Docker socket** into job containers — do **not** add it in the workflow's `volumes:` (causes `Duplicate mount point`).
+- **`runs-on: ubuntu-latest`** — the label only selects the runner; the `docker:cli` container override does the work. (The runner re-registered with default labels once a `config.yaml`/`CONFIG_FILE` was added, dropping the original `docker` label.)
+- Runner `config.yaml` (loaded via `CONFIG_FILE=/data/config.yaml`) sets `container.valid_volumes`.
+
+## Registry auth
+
+Pushing packages needs a **Forgejo token with `write:package`**, stored as the Actions secret **`REGISTRY_TOKEN`** and used for `docker login`. The automatic `GITHUB_TOKEN` cannot push packages (`unauthorized: reqPackageAccess`); it's still used for the git checkout.
+
+## Traefik
+
+Label discovery was unreliable; routing uses the file provider: `/Users/davidlawler/docker/traefik/dynamic/etsy.yml` routes `Host(etsy.plexultra.com)` → `etsy-nginx:80` (websecure, `le` certresolver).
+
+## Persistence & secrets
+
+- MongoDB on a named volume (`etsy_mongodb`) — survives container recreation. Clearing orders/expenses in-app does not touch the volume.
+- `JWT_SECRET` set in Dockhand env (don't leave the default).
+- Etsy keystring/secret and OAuth tokens are stored **unencrypted** in MongoDB — fine for single-user self-host, but a DB backup contains them.

Etsy-Integration.md

@@ -0,0 +1,40 @@
+# Etsy Integration
+
+The Etsy API is the primary data source. Setup and credentials live entirely in the app (Settings → Etsy Integration) and are stored in the database per user — no env vars or redeploy needed.
+
+## One-time setup
+
+1. **Create an Etsy app** at [etsy.com/developers](https://www.etsy.com/developers) (personal-use key is sufficient — it only touches your own shop).
+2. **Register the callback URL** in the Etsy app settings, exactly: `https://etsy.plexultra.com/api/etsy/callback`
+3. In the tracker: **Settings → Etsy Integration → API Configuration**
+   - Paste the **keystring** (OAuth client id)
+   - Paste the **shared secret**
+   - Confirm the **callback URL** (pre-filled)
+   - **Save Configuration**
+4. **Connect Etsy Shop** → approve on Etsy's consent screen → you land back on Settings showing the shop name.
+
+### Credential quirks (learned the hard way)
+
+- The `x-api-key` header for API calls must be **`keystring:sharedsecret`** (both, colon-separated) — not the keystring alone.
+- OAuth token requests are **form-encoded**, not JSON.
+- Scopes requested: `transactions_r listings_r shops_r` (read-only). Listing *creation* would need `listings_w` and a reconnect (not implemented).
+
+## Syncing
+
+**Settings → Sync Orders Now** pulls, idempotently:
+
+- **Orders** — upserted by receipt id (= the packing-slip "Order #"), so they merge with anything already present. Items get costs snapshotted from the catalogue.
+- **Fees** — from the payment-account ledger (chunked in 30-day windows; Etsy caps the query at 31 days). Amounts are integer minor units (÷100). Classified into Marketing & Advertising (`prolist`/offsite ads), Transaction Fees, Payment Processing Fees, Listing Fees, Taxes & GST. Credits, disbursements, and refunds are skipped. Unrecognised debits are reported back, not guessed.
+- **Refunds** — summed from each receipt and stored as `refundTotal`; revenue is reported net.
+- **Customers** — accumulated per buyer.
+
+**Settings → Sync Catalog** pulls active listings into Products — **one product per size variation** (colour collapsed). Updates title/price/tags/sku/quantity/listing-id but **preserves your costs**.
+
+### Resolving unmatched items
+
+After a sync, any item title that didn't match a product appears under a yellow banner with **Resolve Items** — map each to an existing product (saved as an alias) or create it with a cost. You can do a few at a time; the rest stay listed.
+
+## Idempotency / safety
+
+- Re-syncing never duplicates: orders upsert by number, fees by `etsy-ledger-<entry_id>`, and the sync auto-dedupes any historical duplicate fees.
+- The **Remove Old Fees** button clears legacy CSV-imported Etsy fees so they don't double-count with ledger fees.

Home.md

@@ -0,0 +1,33 @@
+# Etsy Finance Tracker
+
+Self-hosted finance and profit tracker for an Etsy shop. Pulls orders, fees, refunds, and listings directly from the Etsy API, matches them against a product catalogue with per-item costs, and reports profit, margins, expenses, and an AU financial-year P&L.
+
+Live at **[etsy.plexultra.com](https://etsy.plexultra.com)**.
+
+## Pages
+
+- [[Architecture]] — stack, structure, data model
+- [[Etsy Integration]] — connecting your shop, syncing orders/fees/listings
+- [[Data Import]] — CSV/PDF imports (Australia Post + legacy)
+- [[Reports & Analytics]] — dashboards, P&L, GST, ad ROI
+- [[Deployment & CI]] — Forgejo Actions, registry, Dockhand, Traefik
+- [[Troubleshooting]] — common issues and fixes
+
+## What it does
+
+- **Order sync** from Etsy — line items, variations/sizes, tracking, customer, with per-item costs snapshotted at sync time
+- **Fee sync** from the Etsy payment ledger — transaction, processing, listing, ads (`prolist`/offsite), and GST, categorised automatically
+- **Refunds** netted from revenue so profit isn't overstated
+- **Catalogue sync** from active listings — one product per size, costs preserved
+- **Customers** built from buyers, with repeat-rate
+- **Reports** — profit/margin dashboards, monthly trends, ad ROAS, AU financial-year P&L, and an indicative GST summary
+
+## What you maintain
+
+Etsy provides the sales/catalogue side. You provide the **cost side** — set printing/material cost once per product (Products → "Missing costs" filter finds the ones needing it). Australia Post postage is imported via CSV (no usable API for MyPost Business history).
+
+## Quick links
+
+- Settings → **Etsy Integration**: connect shop, Sync Orders, Sync Catalog
+- Products → **Missing costs**: fill in per-product printing/material costs
+- Profit Analysis → **P&L / GST**: financial-year reporting

Reports-&-Analytics.md

@@ -0,0 +1,43 @@
+# Reports & Analytics
+
+## Analytics page
+
+Headline metrics for the selected period (date selector includes months, quarters, calendar years, financial years, and custom ranges):
+
+- **Total Revenue** — net of refunds (shows the refunded amount)
+- **Net Profit** and margin — revenue − printing costs − expenses
+- **Orders**, AOV, **Customers**, repeat rate
+- **Printing Costs**
+- **Revenue Trend** — monthly bar chart (scaled to values)
+- **Expense Categories** — breakdown with amounts and %
+- **Advertising Performance** — ad spend, % of revenue, and **ROAS** (revenue per $1 of ad spend). Ads are typically the largest expense, so this is the key lever to watch.
+
+## Profit Analysis page
+
+Tabs:
+
+- **Overview** — profit metrics and cards
+- **Trends** — revenue vs profit by month (bar chart) + a per-month table (orders, revenue, printing, expenses, profit, margin)
+- **Products** — per-product profitability ranking
+- **Orders** — per-order profit with item-level breakdown
+- **P&L / GST** — see below
+
+### P&L / GST tab
+
+A proper profit-and-loss statement for the selected period:
+
+```
+Revenue (net of refunds)
+  − Cost of goods sold (printing + materials)
+  = Gross profit
+  − Operating expenses (by category)
+  = Net profit
+```
+
+Plus an **indicative GST summary** (GST on sales vs GST paid on Etsy fees, and net position). Marked indicative — confirm against your BAS; Etsy generally collects/remits GST on AU sales on your behalf.
+
+**Financial-year reporting:** the date selector offers AU financial years (Jul–Jun), e.g. "FY 2025–26", so the P&L and all reports can be run per financial year.
+
+## Accuracy depends on costs
+
+COGS, margins, and net profit are only right once each product has its **printing/material cost** set. Use **Products → "Missing costs"** to find and fill products imported at $0. (Note: the P&L includes materials in COGS, so its net profit can read slightly lower than the Overview card, which historically counted printing only.)

Troubleshooting.md

@@ -0,0 +1,27 @@
+# Troubleshooting
+
+## Etsy connection / sync
+
+- **"Etsy connection failed"** on connect — check the callback URL registered in your Etsy app exactly matches `https://etsy.plexultra.com/api/etsy/callback`, and that both the keystring and shared secret are saved in Settings. Error detail is shown in the toast.
+- **403 "Shared secret is required" / "incorrect shared secret"** — the `x-api-key` must be `keystring:sharedsecret`; make sure the shared secret is saved (Settings shows a masked value when set).
+- **Ledger "time window … no more than 31 days"** — handled (the sync chunks 30 days at a time); if you see it, the build is stale.
+- **Unmatched items after sync** — use Settings → **Resolve Items** to map each to a product (saved as alias) or create it; then re-sync.
+
+## Numbers look wrong
+
+- **Profit overstated / costs missing** — products imported from the catalogue start at $0 cost. Products → **Missing costs** filter → set printing/material costs.
+- **Two pages disagree** — usually a stale cached bundle; hard-refresh (Cmd+Shift+R). Revenue is net of refunds everywhere; the revenue card shows "net of refunds".
+- **Duplicate fees** — the sync auto-dedupes ledger fees; for legacy CSV fees use **Remove Old Fees** once.
+
+## Deployment / CI
+
+- **Workflow "waiting" forever** — no runner matches `runs-on`, or the runner is offline. Check Settings → Actions → Runners is Idle/online and has the `ubuntu-latest` label; check `docker compose logs forgejo-runner`.
+- **`docker: command not found`** — job ran in a container without docker; ensure the workflow uses `container: docker:cli`.
+- **`Duplicate mount point: /var/run/docker.sock`** — remove any explicit socket mount from the workflow; act_runner mounts it automatically.
+- **`unauthorized: reqPackageAccess` on push** — the `REGISTRY_TOKEN` secret is missing or lacks `write:package`. Recreate the Forgejo token and secret.
+- **Deploy can't pull image** — `docker login git.plexultra.com` on the host, or make the package public in Forgejo.
+- **Stale UI after deploy** — hard-refresh; nginx serves `index.html` no-cache and hashed assets immutable, so a refresh picks up the new bundle.
+
+## Data resets
+
+- **Clear All Orders** (Data Import) bulk-deletes orders only; products, expenses, and customers are untouched. Re-sync from Etsy to rebuild. Product cost mappings (aliases, listing IDs, costs) live on products and survive an order clear.