printhive/README.md

# Printhive

A self-hosted web app for organizing your 3D-printing files (STL, 3MF, OBJ, FBX, STEP, 3DM, GCODE — loose or in ZIP/7Z/RAR/TAR archives) into a searchable library — with in-browser 3D previews, collections, license/commercial-use tracking, and one-paste **import from popular model sites**.

## Features

- **Library** of models, each owning one or more files; tags, collections, descriptions, print settings.
- **Import from URL**:
  - **Thingiverse** — downloads files automatically via the official API (needs a free app token).
  - **Printables** — downloads files automatically via its public GraphQL API (no auth needed for free models).
  - **MakerWorld** — downloads files via FlareSolverr (solves Cloudflare) + your account token. If files can't be downloaded the import says why, and when MakerWorld refuses the token (they expire) Printhive asks for a fresh one and retries.
  - **cubee3d (Hive)** and any other URL — imports title/description/preview image; you add the files manually.
- **In-browser 3D viewer** (Three.js) for STL, OBJ (with its MTL colours/textures), FBX (with textures), STEP/STP (OpenCascade WASM) and Rhino 3DM, and for 3MF via a server-side **trimesh GLB conversion** that streams even huge Bambu/MakerWorld projects.
- **Slicer projects plate by plate** — Bambu Studio / OrcaSlicer 3MFs open on plate 1 with the slicer's plate images as a switcher, parts shown in their filament colours, and a Filaments list; negative/modifier parts are hidden.
- **Archives as folders** — ZIP, 7Z, RAR and TAR/.tar.gz uploads are indexed in the background; browse what's inside and preview or download any file without unpacking. Models inside archives count for duplicate detection.
- **Pictures, PDFs & readmes** — keep them with a model and view them in the app.
- **Open in slicer** — pick your slicer once per browser (Settings → Slicer). Bambu Studio and OrcaSlicer open the file straight from a 10-minute download link (`bambustudioopen://…`, `orcaslicer://open?file=…`), so the server must be reachable at the address the browser uses; Bambu Studio asks before opening files that aren't from MakerWorld. PrusaSlicer (which only accepts Printables/Thingiverse links) and Cura get a normal download, and any other slicer with an open-link can be added as a template. Opening a 3MF project in a different slicer from the one that saved it asks first. Works for files inside archives too.
- **Drag out** — drag a model card (its file, or a zip of all its files) or any file row, including files inside archives, onto your desktop or a folder to save it. Uses Chromium's DownloadURL drag, so it needs Chrome, Edge, Brave or similar.
- **Auto thumbnails** made in the background: an embedded 3MF/plate preview, else a picture that came with the model (or one inside its archive), else a server-side render of the STL/OBJ/3MF; the browser renders FBX/STEP/3DM ones.
- **License & commercial-use tracking** — captured per source, with a "sellable / non-commercial" indicator (a guide; verify before selling).
- **Collections** — group models (a model can be in many); rename/delete without losing models.
- **Editing** — rename/retag, edit print settings, override commercial-use, refresh metadata from source, re-download files, download all files as a zip.
- **Notes** — free-form notes per model, saved as you type.
- **Search, filters & sorting** — search across names, descriptions, notes, tags, file names and designers; filter by format, source and license; include *or exclude* tags (match any/all); sort and group by source or format. Press <kbd>/</kbd> to search.
- **Duplicate finder** — every file gets a SHA-256 content hash plus (for STL/OBJ/3MF) a geometry fingerprint, computed in the background. Models sharing files are grouped as *exact copies* or *same shape* (renamed, re-exported, converted — or mirrored, so check those), with a recommended copy to keep. Keeping one merges the others' tags, collections and notes into it and moves them to the trash; groups can be dismissed as "not duplicates".
- **Watch a MakerWorld collection** — paste a collection link (private ones work, using your MakerWorld token) and Printhive checks it on a schedule you choose, importing models added to it into a collection of the same name here, optionally AI-tagging the new ones. Each check reads the whole collection (paging through MakerWorld's own load-more call, up to ~640 models) and imports up to 15 of the new ones, spaced out, returning shortly after if more are waiting; models you remove on MakerWorld stay in your library. Each watch shows when it last ran, what it imported, and why a check failed — with a one-click token update when MakerWorld refuses your token. A check imports each model's page — name, description, picture, licence — but not its files, so a large collection arrives quickly; models are listed as still without files and you fetch the ones you want with "Get files". MakerWorld refuses downloads fired off in quick succession (a few seconds apart, as a check would), but spacing them out is fine — 67 models in a row downloaded about 20 seconds apart without a single refusal. After a run of downloads MakerWorld may also ask you to prove you're not a robot: the Watched page says so and Printhive stops downloading — checks still import new models, whose files wait — until you solve it on MakerWorld and press "I've solved it".
- **AI tags & descriptions** — bring your own key for DeepSeek (the default), OpenRouter, OpenAI or any OpenAI-compatible server (Ollama, LM Studio). Tag the whole library, only models not yet tagged, or a single model from its page. Each model's name, file names and (optionally) thumbnail are sent; tags are only ever added and descriptions only filled where empty unless you ask. Optionally it also gives models with unclear names (file-name leftovers, typos, site boilerplate) a clearer one; the old name is kept and can be restored with Undo on the model page. It can also suggest collections (reusing yours where they fit): suggestions wait on the AI page, grouped by name, until you create/add or dismiss them. Each run can do any mix of tags & descriptions, names and collections, and a reply that isn't the JSON asked for is retried once. Runs show live progress, tokens and cost, can be stopped and resumed, stop at an optional spending limit, and pause (rather than keep spending) if the server restarts. "Save & test connection" in Settings checks the key, model and whether it accepts images.
- **Search by meaning** — switch the search box to *Meaning* to find models by what they are ("wall hook" finds a "coat hanger bracket"). Uses a small embedding model (bge-small-en-v1.5, ~34 MB) that runs on the server, so it's free and private; the index updates in the background as models change.
- **Automatic tags** — strict whole-word rules over names, file names and archive contents (part types, mechanisms, materials, techniques) plus your own "word → tag" rules. Switch it off (optionally removing its tags) or re-apply it any time. Tags you add yourself are never changed; AI and rule tags are marked on the model page.
- **Filament & what's printable** — keep the spools you own (material, colour, brand, grams left) on the **Filament** page. Bambu Studio / OrcaSlicer projects name the filament in each slot, which Printhive reads once per model in the background, so cards and the model page say whether you can print it now and what you're short of, and the toolbar can show only what's printable. Each slot is matched to a spool by material *and* colour — a four-colour print needs those four colours, and one spool can't fill two slots. Near shades count as the same colour; the model page lists every slot, which spool fills it, and the nearest thing you own when none does. Models that aren't slicer projects don't name their filaments, and say so rather than being called unprintable. The slot list can be edited on the model page when the project's own list is wrong or you plan to print it differently — set a slot to **any colour** when the shade doesn't matter, and it's filled by any spool of that material. Edited lists are marked as yours and left alone by the background reader; "Use the project's list" puts it back. Adding a spool can start from Bambu Lab's own filament list — pick "PLA Basic — Jade White (10100)" and the material, colour and code fill themselves — and slot colours are named where they match it. That list is included by kind permission of [3D Filament Profiles](https://3dfilamentprofiles.com), whose terms otherwise reserve it; ask them before reusing `db/bambu-filaments.json` elsewhere.
- **Printers** — add as many Bambu Lab printers as you have on the Printers page, each with an optional nickname (the model is shown when it's blank), a model picked from the lineup with a drawing of its shape, its AMS type, and its address, serial number and access code. Printhive stays connected to each printer and its card shows what it's doing live: state, the file printing, progress, time left and finish time, layer, nozzle, bed and chamber temperatures, the speed setting, and any error or HMS codes. The drawing colours in the spools loaded in the AMS, with every slot listed underneath. "Test & sync" asks a printer for a full report straight away. "Camera" shows the chamber camera on a P1 or A1 series printer — about a picture a second, connected only while you're watching and never recorded. The connection only listens — Printhive never sends a printer a command, other than asking for a full report — and access codes stay on your server. Spools a printer reads from their RFID tags are added to the Filament page by themselves, within seconds of going in: named and coded from Bambu's filament list, shown with their tag ID, matched to any you'd already added by hand rather than duplicated, and marked as taken out when they leave the printer. Grams left are filled in from an AMS, AMS 2 Pro or AMS HT; an AMS lite only knows full or empty, so those are left to you. Spools without a tag can't be told apart from one another, so they're listed for you to add by hand.
- **Trash** — deleting moves a model to the trash; restore it, delete it forever, or empty the trash (only then are files removed from disk).
- **Settings** — API tokens / FlareSolverr URL stored in the database (with `.env` fallback), secrets masked.

## Tech stack

- Node.js + Express, vanilla-JS frontend (no build step), PostgreSQL.
- Three.js (viewer) via CDN import map.
- Python + [trimesh](https://github.com/mikedh/trimesh) sidecar for 3MF→GLB conversion, volume, and geometry fingerprints.
- Optional [FlareSolverr](https://github.com/FlareSolverr/FlareSolverr) for Cloudflare-protected sources.

## Setup

1. **Postgres** — create a database and point `DATABASE_URL` at it. The schema is applied automatically on startup.
2. **Python sidecar** (for 3MF previews, plates, server-side thumbnails and shape-based duplicate detection — without it, duplicates are still found by content hash):
   ```bash
   python3 -m venv tools/venv
   tools/venv/bin/pip install -r tools/requirements.txt
   ```
3. **Install deps and run**:
   ```bash
   npm install
   cp .env.example .env   # set DATABASE_URL (+ optional tokens)
   npm start
   ```
4. Open http://localhost:3000

Run the unit tests with `npm test`.

### Where your data lives

With Docker, everything sits in folders on the host — `./data` beside `docker-compose.yml` unless `.env` says
otherwise (`DATA_DIR` for the files, `PGDATA_DIR` for the database):

| Folder | Holds |
|---|---|
| `$PGDATA_DIR` | The database: models, tags, collections, watches, spools, printers, settings (tokens and printer access codes live here) |
| `$DATA_DIR/uploads` | Your model files, as uploaded or downloaded |
| `$DATA_DIR/thumbnails` | Generated pictures, including plate images |
| `$DATA_DIR/glb` | Converted 3D previews for the viewer |
| `$DATA_DIR/extracted` | Files unpacked from archives — a cache |

The database is set apart from the rest because Postgres needs a filesystem with real ownership and permissions —
ext4, xfs, APFS and the like. exFAT, FAT and most network shares won't do, so keep `PGDATA_DIR` on local disk if the
files live on one of those.

A backup needs the database (or a `pg_dump`) and `uploads`; the other three rebuild themselves. Everything else in
the container is replaced on each update, so nothing outside these folders is kept.

### Backups

`tools/backup.sh` takes a dated dump of the database and mirrors the model files, leaving Printhive running:

```bash
./tools/backup.sh
```

It writes to `./backups` unless `BACKUP_DIR` says otherwise: `backups/db/printhive-YYYY-MM-DD.dump` (kept for
`KEEP_DAYS`, 14 by default) and `backups/uploads`, an rsync mirror of the files — matching what's there now,
deletions included, rather than a history. Thumbnails, previews and extracted archives are skipped, as they rebuild.
Each dump is read back with `pg_restore -l` before it replaces the day's file, so a failed run leaves the older
backups alone and exits non-zero.

Nightly, at 3:30am — `crontab -e`, with the path to your own copy:

```
30 3 * * * cd /path/to/printhive && ./tools/backup.sh >> backups/backup.log 2>&1
```

Docker has to be running for it to work, and the backups are best kept on a different disk from `DATA_DIR`.

To restore the database into a running stack:

```bash
docker compose exec -T db pg_restore -U printhive -d printhive --clean --if-exists < backups/db/printhive-2026-01-01.dump
```

Upgrading from a version that used Docker named volumes: stop the stack, run `tools/volumes-to-folders.sh` to copy the
data across, then start it again. The script only copies — the old volumes stay until you remove them yourself.

## Configuration

Settings live in the database (editable in the app's **Settings** menu) and fall back to environment variables:

| Setting | Env var | Purpose |
|---|---|---|
| Thingiverse API token | `THINGIVERSE_TOKEN` | Thingiverse auto-download |
| FlareSolverr URL | `FLARESOLVERR_URL` | Read Cloudflare-protected sites (MakerWorld) |
| MakerWorld auth token | `MAKERWORLD_TOKEN` | MakerWorld downloads (bearer token from your session) |
| Printables auth token | `PRINTABLES_TOKEN` | Optional; account-gated Printables files |
| AI provider | `AI_PROVIDER` | `deepseek` (default), `openrouter`, `openai` or `custom` |
| AI API key | `AI_API_KEY` | Your provider key (not needed for a local server) |
| AI model | `AI_MODEL` | Defaults to `deepseek-flash` for DeepSeek |
| AI API address | `AI_BASE_URL` | Defaults per provider; e.g. `http://localhost:11434/v1` for Ollama |
| AI prices | `AI_PRICE_INPUT`, `AI_PRICE_OUTPUT` | $ per million tokens, only for showing costs (DeepSeek defaults to its peak rates) |
| Send thumbnails | `AI_SEND_IMAGES` | `false` to tag from names and files only |

Printers are added on the **Printers** page instead — as many as you like, each with a nickname, model, AMS type,
address, serial number and access code. Access codes stay on the server. A printer set up in Settings by an earlier
version moves there automatically; the old `PRINTER_*` environment variables are no longer read.

## Notes

- Auto-download from MakerWorld/Printables relies on their private/session APIs and can break if they change; failed/partial imports can be recovered with "Download files from source".
- The embedding model for search is downloaded into `./models` on first use locally; the Docker image ships with it. It holds ~150 MB of RAM while in use and is released after 10 idle minutes.
- The commercial-use badge is a best-effort read of the license — always confirm on the source page before selling.