Commit graph

77 commits

Author SHA1 Message Date
dlawler489
5b559326e3 Merge filament inventory and per-model filament reading
All checks were successful
build-and-push / docker (push) Successful in 32s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-17 11:32:52 +10:00
dlawler489
ba3bb415bb Track the filament you have, and what each model asks for
A Bambu Studio or OrcaSlicer 3MF names the filaments its plates use, but
reading that means shelling out to Python, so it can't be done per page load.
A background pass — in the same loop as thumbnails and fingerprints — reads it
once per model into models.filaments, which turns "what can I print now" into
a comparison of two lists.

Adds a Filament page for the spools on hand: material, colour, brand and grams
left, edited in place. Matching is on material only; a project's exact colours
would make almost nothing printable, and a different colour still prints.

Measured on the dev library: 10 models read with no errors, the two Bambu
projects resolving to four PLA colours each, and no plate images written
because the reader skips --images and queues behind anything interactive.

Not done yet: nothing in the library view uses this — no "printable now" badge
or filter — and the Filament page itself has not been seen rendered, because
the local preview browser has no session.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-17 11:18:54 +10:00
dlawler489
c0c5e3300a Add the dev server config the preview tool launches
All checks were successful
build-and-push / docker (push) Successful in 32s
Names the project's dev command and port (npm start, 3100) so the in-app
browser preview can start Printhive without being told how each time.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 20:34:57 +10:00
dlawler489
341f8c5df1 Merge per-watch file downloading
All checks were successful
build-and-push / docker (push) Successful in 24s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 19:18:55 +10:00
dlawler489
32299da05e Let a watch download files as models arrive
Checks have imported metadata only since MakerWorld refused bulk downloads.
Today's measurement changed that picture: it refuses bursts a few seconds
apart, but 67 models downloaded in a row about 20 seconds apart without one
refusal. So downloading during a check is viable — at the right pace.

Per watch, off by default: "Files: on/off" on the row, a checkbox when adding
one. With it on, a check spaces downloads 20 seconds apart instead of 3, and
stops after two refusals in a row, coming back shortly rather than collecting
refusals for the whole batch. A model whose files were refused still lands as
'partial', so "Get files" can finish it later.

Auto-imports now get the same AI treatment a manual run does — tags, a clearer
name where the site's is boilerplate, and collections to review — rather than
tags alone.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 19:18:55 +10:00
dlawler489
8fbcc6a2b8 Merge the corrected download-limit note
All checks were successful
build-and-push / docker (push) Successful in 23s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 19:11:50 +10:00
dlawler489
8adac1018f Correct what the README says about MakerWorld's download limit
It claimed an account may only download "a couple in a row". Measured today:
67 models downloaded in a row, spaced about 20 seconds apart, with no refusal
at all. The refusals happen when downloads are fired a few seconds apart, as a
watch check used to do — so it is a rate limit on bursts, not an allowance.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 18:53:32 +10:00
dlawler489
b2078690bd Merge deadlines for downloads
All checks were successful
build-and-push / docker (push) Successful in 39s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 17:52:12 +10:00
dlawler489
341731a946 Give downloads a deadline instead of hanging for ever
Neither the FlareSolverr call nor the CDN fetch set a timeout, so a stalled
connection hung the request indefinitely: the button sat on "Downloading…"
with nothing to reset it and no error to show. FlareSolverr calls now give up
15s after the deadline FlareSolverr itself was given (and a timeout counts as
retryable, like other transport failures), and a CDN download gives up after
three minutes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 17:50:48 +10:00
dlawler489
27e1caf1a6 Merge one lock order for collection rows
All checks were successful
build-and-push / docker (push) Successful in 21s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 17:44:05 +10:00
dlawler489
9ce95a6521 Take collection rows in one order too
The tag paths are sorted now, but two writers of model_projects still iterated
in caller order: findOrCreateProject over the models it attaches, and
setProjects over a model's collections. Two of either running at once could
take the same rows in opposite orders — the same shape as the tag deadlock, one
table across.

Preventative: unlike the tag case, no failure from these was ever observed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 16:19:32 +10:00
dlawler489
d2dd67c311 Merge the failure list into the page for real
All checks were successful
build-and-push / docker (push) Successful in 26s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 16:10:26 +10:00
dlawler489
07da33a191 Actually put the failure list in the page
The block was built into a `failed` variable and then never interpolated, so
runHtml computed it and threw it away. Two deploys changed nothing visible
because of this: the API carried the failures and the panel fetched them, but
the template never referenced them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 15:09:27 +10:00
dlawler489
099e6d24b4 Merge the tagging deadlock fix
All checks were successful
build-and-push / docker (push) Successful in 22s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 14:56:28 +10:00
dlawler489
8a2ad5785c Stop tagging deadlocking, and retry a transaction that loses one
Two transactions tagging overlapping models could each hold a tag row the other
wanted. tagId used ON CONFLICT DO UPDATE, which rewrites the row and therefore
write-locks it even though the name never changes, and each caller iterated in
whatever order it happened to have. That is what killed "Mushroom Clicker"
during AI run 21, while the collection watcher was importing and rule-tagging
the same tags.

tagId no longer rewrites unchanged rows, and applyAiResult, replaceRuleTags and
setTags all tag in sorted order, so locks are taken in the same sequence every
time. db.transaction retries a deadlock victim up to three times as well, which
covers the paths this doesn't — the victim rolled back and changed nothing, so
running it again is safe.

Measured: the old pattern deadlocks on demand (40P01), and the same collision
run 48 times through the new code fails zero times.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 14:56:28 +10:00
dlawler489
4036139901 Merge failure list rendering for any run with failures
All checks were successful
build-and-push / docker (push) Successful in 23s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 14:52:40 +10:00
dlawler489
96b3fbb1bb Draw the failure list for any run that has one, not just the newest
The failures now reach the API, but the page could still not show them: the AI
panel fetches detail for one run — the running one, else the newest — and
renderAiLive passes detail only to that run, so every other run is drawn with
nothing. Run 21's single failure stayed invisible for exactly this reason,
even after the previous fix.

Details are now fetched for any listed run with failures and keyed by run id.
Usually that is no extra request, and at most a handful.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 14:49:12 +10:00
dlawler489
1f22dd0189 Merge visible AI run failures and retry for finished runs
All checks were successful
build-and-push / docker (push) Successful in 27s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 14:08:08 +10:00
dlawler489
b0054876ff Name every model an AI run failed on, and let a finished run retry them
A run's detail only carried the last dozen items, so a model that failed early
scrolled out of reach: run 21 reported "1 failed" with no way to learn which
model or why. The detail now carries every failed item, listed in full.

"Retry failed" existed but could never appear: it needs a run that is finished
with failures, and such a run is 'done', which neither canResume nor resume()
accepted. Both now do; resume() already no-ops when nothing is pending, so a
clean run is unaffected.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 14:08:08 +10:00
dlawler489
96497d79a8 Merge inline watch errors in place of alert()
All checks were successful
build-and-push / docker (push) Successful in 24s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 14:04:16 +10:00
dlawler489
fc5e81006b Show a watch's failure on its row, not in an alert()
A failed "Get files" popped a browser alert: jarring, and the message is gone
the moment you dismiss it — which matters here, because the message explains
MakerWorld's download limit. It now appears on the watch's row until the next
attempt.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 13:55:41 +10:00
dlawler489
be8776bbce Merge Watched-page refresh, full file-less list, and single collection read
All checks were successful
build-and-push / docker (push) Successful in 30s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 13:36:41 +10:00
dlawler489
e7c65e28a1 Keep the Watched page current, and reach every model waiting for files
Three things the live run showed:

- The page only polled while a check was already running when it rendered, so
  a page opened between checks showed stale numbers for ever — it missed a
  check started by the schedule or another tab. It now refreshes either way.
- Only the five most recent items were listed, so most models still without
  files had no "Get files" button anywhere. They're now in a collapsible list.
- Adding a watch read the whole collection twice: once to create it, once in
  the first check. Measured on a 535-model collection: the add still takes 52s,
  but the first import now lands 6s later instead of another read.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 13:34:59 +10:00
dlawler489
0962216196 Merge metadata-feed watching for MakerWorld collections
All checks were successful
build-and-push / docker (push) Successful in 22s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 13:12:52 +10:00
dlawler489
81b6016d63 Watch collections as a metadata feed, not a downloader
MakerWorld allows an account only a couple of downloads in a row, so a check
that tried to fetch files left most models empty. A check now imports each
model's page — name, description, picture, licence — and marks it 'partial';
"Get files" pulls the files for the ones you actually want to print.

Verified against a public collection: 11 models imported with descriptions and
thumbnails, no files fetched, no MakerWorld token needed for the check.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 13:07:47 +10:00
dlawler489
bf8e95da97 Merge honest handling of MakerWorld's download limit
All checks were successful
build-and-push / docker (push) Successful in 29s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 12:53:36 +10:00
dlawler489
a65c9d047f Don't retry MakerWorld's anti-bot refusal
Measured against the live site: FlareSolverr reports no challenge on that
endpoint and an unauthenticated call gets a normal reply in 3s, so this is
MakerWorld limiting an account to a couple of downloads in a row, not a
Cloudflare page. Two models retried three times each still came back empty —
the retry only made every failure a minute slower.

Keeps the classification, so a refusal is named for what it is and the model
is flagged as still without files.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 12:53:36 +10:00
dlawler489
239654be13 Pausing a watch stops the batch it's in the middle of
Pause only prevented the next check: the batch already running kept importing
its remaining models, which is the opposite of what the button suggests.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 12:50:56 +10:00
dlawler489
cd95890296 Offer "Get files" for any import that lost its files
Models imported before 'partial' existed are stored as 'imported' with an
error, so they'd never show the button. Mark them on startup, and key the
button on the stored error rather than the status.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 12:44:13 +10:00
dlawler489
11fe5c25aa Merge MakerWorld anti-bot retry and partial-import flagging
All checks were successful
build-and-push / docker (push) Successful in 24s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 12:37:20 +10:00
dlawler489
a313e0e224 Retry MakerWorld's anti-bot check, and flag imports that got no files
A run of downloads gets challenged by Cloudflare partway through: the answer
comes back as an ordinary reply, so nothing retried it and the model landed
with only its metadata — looking, on the watch page, like a success.

resolveDownload now recognises a challenge and gives FlareSolverr's browser a
couple more goes; what still fails is recorded as 'partial' and shown with its
reason and a "Get files" button, which clears the flag once the files arrive.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 12:34:04 +10:00
dlawler489
18f15cf3b4 Merge MakerWorld collection watching
All checks were successful
build-and-push / docker (push) Successful in 24s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 12:22:47 +10:00
dlawler489
4c54f76ae3 Read whole collections, and never check one twice at once
- A collection page lists only its first ~20 models. Checks now page through the
  same load-more call the page uses (offset + a shared seed, 64 at a time, up to
  ~640 models), so a watch sees the whole collection.
- A private collection read without a valid token comes back as a placeholder
  (id -1, no models), which was being treated as an empty collection. It now
  reports the token problem instead.
- Adding a watch starts a check, and "Check now" could start a second one
  alongside it, importing models twice. One check per collection at a time now,
  and a check interrupted by a restart no longer looks like it's still running.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 09:58:09 +10:00
dlawler489
0df38c83c7 Watch a MakerWorld collection and import models added to it
A watch reads the collection page through FlareSolverr carrying your token, so
private collections work, and imports the models it hasn't seen into a Printhive
collection of the same name (optionally AI-tagging them). Checks run on your
chosen interval, a few models at a time with pauses between, and each design is
remembered so nothing is imported twice. A refused token stops the check and the
Watched page offers to update it; models removed on MakerWorld stay here.

The import endpoint's work moved into importFromUrl() so watches reuse it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-16 08:53:35 +10:00
dlawler489
66b62cbb1e Merge sturdier AI replies with clearer errors
All checks were successful
build-and-push / docker (push) Successful in 24s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 20:29:28 +10:00
dlawler489
848496a205 Sturdier AI replies: format example, more retries, and errors that show the reply
The Red panda kept failing with "The AI reply wasn't the JSON asked for", and
nothing showed what came back. Now:
- the prompt includes an example object with just the requested fields
  (DeepSeek's JSON mode can otherwise send back an empty reply)
- JSON is found inside code fences, notes, or braces after it
- a bad reply is asked again up to twice: with more room when it was cut off,
  and the last time without JSON mode; max_tokens starts at 1500
- the error says whether the reply was empty or cut off and quotes its start
- tokens spent on failed attempts count towards the run's cost

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 20:26:42 +10:00
dlawler489
6a3e23359f Merge emoji-safe text for AI requests
All checks were successful
build-and-push / docker (push) Successful in 25s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 20:22:06 +10:00
dlawler489
2bbdd5d5c3 Don't split emoji when shortening text for the AI
Descriptions sent to the AI are cut to 800 characters. When the cut landed
inside an emoji (stored as two UTF-16 halves), the request carried half of it,
which DeepSeek rejects: "unexpected end of hex escape". Text is now cut before
the emoji, anything already broken in the library is replaced before sending,
and the tags, names, collection ideas and descriptions read back are cut the
same safe way.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 19:48:07 +10:00
dlawler489
57c514c1af Merge MakerWorld download errors and token prompt
All checks were successful
build-and-push / docker (push) Successful in 28s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 18:33:10 +10:00
dlawler489
4a88d66e90 Ask for a fresh MakerWorld token when MakerWorld refuses it, then retry
When a download fails because MakerWorld refused (or is missing) the auth
token, the error carries reason makerworld_token. The import and "Download
files from source" then open an "Update your MakerWorld token" prompt with
steps to copy the token cookie; saving it retries the download straight away
and says if MakerWorld still refuses it. A missing FlareSolverr URL is reported
separately, since a token alone won't help.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 18:33:10 +10:00
dlawler489
28a95ee330 Say why MakerWorld files didn't download instead of importing silently without them
MakerWorld answers an expired or missing auth token with {"error": "Please log
in to download models."} and no download link, which was skipped without a
word, so imports and "Download files from source" just ended with no files.
Now the reason comes back: missing settings, a refused token (probably expired),
no printable files on the page, or a failed download. The import screen shows
it, and the Settings hint explains where the token comes from.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 18:17:46 +10:00
dlawler489
f80078d7ed Merge AI collection suggestions and reply retry
All checks were successful
build-and-push / docker (push) Successful in 23s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 17:52:41 +10:00
dlawler489
9d7a978f03 AI collection suggestions, pick-what-to-do runs, and a retry for bad replies
- Runs can now do any mix of: tags & descriptions, clearer names, and
  collection suggestions. A run without tags doesn't mark models as tagged.
- Collection ideas reuse your collections and earlier ideas where they fit and
  wait on the AI page, grouped by name, until you create/add or dismiss them.
  Dismissed ideas aren't suggested for that model again; spelling variants
  group together under the capitalised spelling.
- A reply that isn't the JSON asked for is asked again once; both requests
  count towards the run's tokens and cost.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 17:52:41 +10:00
dlawler489
2076717447 Merge AI renaming: clearer names for models with unclear ones, with undo
All checks were successful
build-and-push / docker (push) Successful in 26s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 17:30:35 +10:00
dlawler489
2506f4dce9 Let AI runs give models with unclear names a clearer one, with undo
A run option asks the AI for a clearer name only when the current one is messy
(file-name leftovers, typos, site boilerplate). The previous name is kept, the
run feed shows what was renamed, and the model page offers Undo. Giving a model
a name yourself clears the undo.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 15:26:29 +10:00
dlawler489
b902ea91d6 Merge phase 4 fixes: tag wrapping, model page buttons, search refresh after AI tagging
All checks were successful
build-and-push / docker (push) Successful in 28s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 15:05:21 +10:00
dlawler489
4458963aa4 Fix tags splitting across lines and squeezed model buttons; refresh search after AI tagging
The model page's tag list wasn't a wrapping row, so the last tag could break in
half, and four action buttons crushed onto two lines each. AI runs also never
woke the background worker, so new AI tags and descriptions weren't searchable
until something else changed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 14:39:41 +10:00
dlawler489
c7c39f99fa Merge phase 4: AI tags and descriptions, search by meaning, automatic tags
All checks were successful
build-and-push / docker (push) Successful in 1m9s
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 12:54:03 +10:00
dlawler489
2796fab38c Docker: fail the build if the search model can't be fetched
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 12:54:03 +10:00
dlawler489
0f49fe8c1a AI tags and descriptions, search by meaning, and automatic tags
- AI tagging through any OpenAI-compatible API: DeepSeek by default, or
  OpenRouter, OpenAI, or a local server (Ollama, LM Studio). Settings → AI
  with presets, prices and "Save & test connection"; the key never leaves
  the server except to the provider.
- Runs over untagged models, all models, or one model from its page: live
  progress, tokens and cost, stop/resume (retrying failures), an optional
  spending limit, and paused rather than resumed after a restart. Adapts to
  models without JSON mode or image input; stops on a rejected key or no
  credit. Tags are only added; descriptions filled only where empty unless
  asked.
- Search by meaning: local bge-small-en-v1.5 embeddings (transformers.js),
  indexed in the background whenever a model's text changes; Words/Meaning
  switch on the search box. Stored as REAL[] in Postgres, no pgvector needed.
- Automatic tags: strict whole-word rules (part types, mechanisms, materials,
  techniques) over names, file names and archive contents, plus your own
  word -> tag rules; on/off (optionally removing its tags) and re-apply.
- Tags remember where they came from (user / rule / ai); edits keep that,
  rule and AI passes never touch your own tags, and the model page marks them.
- Docker: ships the embedding model and only Linux onnxruntime binaries.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-15 11:12:58 +10:00